Skip to content

Conversation

@mklimenk
Copy link

@mklimenk mklimenk commented Sep 5, 2025

This is a follow-up PR to #782, that moved most of the options from being set globally (per-process) to locally (per-model).
It is required to properly enable parallel session creation of multiple models with different set of options for each model.

https://jira.devtools.intel.com/browse/CVS-169874

@mklimenk
Copy link
Author

mklimenk commented Sep 5, 2025

@MayureshV1 could you please review it?

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR refactors cache directory and number of streams configuration from global (per-process) settings to session-local (per-model) settings in the OpenVINO provider. This change enables parallel creation of multiple models with different configuration options.

  • Removes global SetCache and SetStreams methods from the OVCore class
  • Updates EnableCaching and EnableStreams methods to accept and populate a device configuration map parameter
  • Shifts from setting properties globally on the core to setting them locally via device configuration

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File Description
ov_interface.h Removes global SetCache and SetStreams method declarations from OVCore
ov_interface.cc Removes global SetCache and SetStreams method implementations from OVCore
basic_backend.h Updates EnableCaching and EnableStreams method signatures to accept device config parameter
basic_backend.cc Refactors EnableCaching and EnableStreams to populate device config instead of setting global properties

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Copy link

@MayureshV1 MayureshV1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This implementation looks good and would make the existing provider options thread safe.

The same two provider options: cache_dir and num_streams are OV properties and can be set using load_config provider option.

@preetha-intel
Copy link

@mklimenk As this is refactoring the properties to set cache and num streams, can it all be broght into the same function , "PopulateConfigValue" instead of seperate calls which does populate the same map.

It can also be loaded from OV config. Can that be validated as well?

Copy link

@preetha-intel preetha-intel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consolidate seperate function calls into single PopulateConfigValue

@preetha-intel
Copy link

Consolidate seperate function calls into single PopulateConfigValue

Can be taken as a follow up as its only cosmetic

@preetha-intel preetha-intel dismissed their stale review September 12, 2025 06:48

Can be added as a follow up

@preetha-intel preetha-intel merged commit 02cf7e3 into intel:ovep-develop Sep 12, 2025
3 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants