-
Notifications
You must be signed in to change notification settings - Fork 3.7k
Add environment variable support for profiling configuration #26798
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,128 @@ | ||
| # Profiling Environment Variables | ||
|
|
||
| ONNX Runtime supports configuring profiling behavior through environment variables, providing a convenient way to enable profiling without modifying application code or session configuration. | ||
|
|
||
| ## Environment Variables | ||
|
|
||
| ### ORT_ENABLE_PROFILING | ||
|
|
||
| Controls whether profiling is enabled for the inference session. | ||
|
|
||
| **Valid values:** | ||
| - `1` - Enable profiling (overrides `SessionOptions.enable_profiling` to `true`) | ||
| - `0` - Use the value from `SessionOptions.enable_profiling` (passthrough mode) | ||
| - Not set - Use the value from `SessionOptions.enable_profiling` (passthrough mode) | ||
|
|
||
| **Default behavior:** If not set, profiling is controlled solely by `SessionOptions.enable_profiling`. | ||
|
|
||
| ### ORT_PROFILE_FILE_PREFIX | ||
|
|
||
| Specifies the prefix for the profiling output file. The profiler will append a timestamp to create the final filename. | ||
|
|
||
| **Valid values:** | ||
| - Any non-empty string - Used as the prefix for the profiling file | ||
| - Empty string `""` - Explicitly sets an empty prefix (overrides `SessionOptions.profile_file_prefix`) | ||
| - Not set - Uses `SessionOptions.profile_file_prefix` if available, otherwise uses the default prefix | ||
|
|
||
| **Default value:** `onnxruntime_profile` | ||
|
|
||
| ## Priority Rules | ||
|
|
||
| When both environment variables and `SessionOptions` are configured, the following priority rules apply: | ||
|
|
||
| ### Enable Profiling Priority | ||
| 1. `ORT_ENABLE_PROFILING="1"` → **Always enables profiling** (highest priority) | ||
| 2. `ORT_ENABLE_PROFILING="0"` or not set → Uses `SessionOptions.enable_profiling` | ||
|
|
||
| ### Profile File Prefix Priority | ||
| 1. `ORT_PROFILE_FILE_PREFIX` (non-empty or empty string) → **Uses environment variable value** (highest priority) | ||
| 2. `SessionOptions.profile_file_prefix` → Uses session option value | ||
| 3. Neither set → Uses default value: `onnxruntime_profile` | ||
|
|
||
| ## Usage Examples | ||
|
|
||
| ### Example 1: Enable profiling via environment variable | ||
|
|
||
| ```bash | ||
| # Linux/macOS | ||
| export ORT_ENABLE_PROFILING=1 | ||
| export ORT_PROFILE_FILE_PREFIX=my_model_profile | ||
|
|
||
| # Windows (PowerShell) | ||
| $env:ORT_ENABLE_PROFILING="1" | ||
| $env:ORT_PROFILE_FILE_PREFIX="my_model_profile" | ||
|
|
||
| # Windows (Command Prompt) | ||
| set ORT_ENABLE_PROFILING=1 | ||
| set ORT_PROFILE_FILE_PREFIX=my_model_profile | ||
| ``` | ||
|
|
||
| Then run your application normally - profiling will be enabled automatically without code changes. | ||
|
|
||
| ### Example 2: Use default settings with environment variable override | ||
|
|
||
| ```cpp | ||
| // C++ code with profiling disabled | ||
| SessionOptions session_options; | ||
| session_options.enable_profiling = false; | ||
|
|
||
| // If ORT_ENABLE_PROFILING=1 is set, profiling will still be enabled | ||
| InferenceSession session(env, model_path, session_options); | ||
| ``` | ||
| ### Example 3: Passthrough mode | ||
| ```bash | ||
| # Set to "0" to use SessionOptions values | ||
| export ORT_ENABLE_PROFILING=0 | ||
| ``` | ||
|
|
||
| In this case, the application's `SessionOptions.enable_profiling` value will be used. | ||
|
|
||
| ## Complete Behavior Table | ||
|
|
||
| The following table shows all possible combinations of environment variables and `SessionOptions`, and the resulting behavior: | ||
|
|
||
| | # | ORT_ENABLE_PROFILING | SessionOptions.enable_profiling | ORT_PROFILE_FILE_PREFIX | SessionOptions.profile_file_prefix | Result: Profiling Enabled | Result: File Prefix | | ||
| |---|---------------------|--------------------------------|------------------------|-----------------------------------|--------------------------|-------------------| | ||
| | 1 | "1" | false | non-empty ("custom") | not set | ✅ true | "custom" | | ||
| | 2 | "1" | false | non-empty ("custom") | set ("session") | ✅ true | "custom" | | ||
| | 3 | "1" | false | empty ("") | not set | ✅ true | "" | | ||
| | 4 | "1" | false | empty ("") | set ("session") | ✅ true | "" | | ||
| | 5 | "1" | false | not set | not set | ✅ true | "onnxruntime_profile" | | ||
| | 6 | "1" | false | not set | set ("session") | ✅ true | "session" | | ||
| | 7 | "1" | true | non-empty ("custom") | not set | ✅ true | "custom" | | ||
| | 8 | "1" | true | non-empty ("custom") | set ("session") | ✅ true | "custom" | | ||
| | 9 | "1" | true | empty ("") | not set | ✅ true | "" | | ||
| | 10 | "1" | true | empty ("") | set ("session") | ✅ true | "" | | ||
| | 11 | "1" | true | not set | not set | ✅ true | "onnxruntime_profile" | | ||
| | 12 | "1" | true | not set | set ("session") | ✅ true | "session" | | ||
| | 13 | "0" | false | non-empty ("custom") | not set | ❌ false | "custom" | | ||
| | 14 | "0" | false | non-empty ("custom") | set ("session") | ❌ false | "custom" | | ||
| | 15 | "0" | false | empty ("") | not set | ❌ false | "" | | ||
| | 16 | "0" | false | empty ("") | set ("session") | ❌ false | "" | | ||
| | 17 | "0" | false | not set | not set | ❌ false | "onnxruntime_profile" | | ||
| | 18 | "0" | false | not set | set ("session") | ❌ false | "session" | | ||
| | 19 | "0" | true | non-empty ("custom") | not set | ✅ true | "custom" | | ||
| | 20 | "0" | true | non-empty ("custom") | set ("session") | ✅ true | "custom" | | ||
| | 21 | "0" | true | empty ("") | not set | ✅ true | "" | | ||
| | 22 | "0" | true | empty ("") | set ("session") | ✅ true | "" | | ||
| | 23 | "0" | true | not set | not set | ✅ true | "onnxruntime_profile" | | ||
| | 24 | "0" | true | not set | set ("session") | ✅ true | "session" | | ||
|
|
||
| **Note:** When `ORT_ENABLE_PROFILING` is not set (empty), it behaves the same as "0" (passthrough mode). | ||
|
|
||
| ### Key Observations from the Table: | ||
|
|
||
| 1. **Rows 1-12**: When `ORT_ENABLE_PROFILING="1"`, profiling is **always enabled** regardless of `SessionOptions.enable_profiling` | ||
| 2. **Rows 13-24**: When `ORT_ENABLE_PROFILING="0"`, `SessionOptions.enable_profiling` determines the final state | ||
| 3. **Prefix Priority**: `ORT_PROFILE_FILE_PREFIX` (when set, even to "") always takes priority over `SessionOptions.profile_file_prefix` | ||
| 4. **Default Prefix**: The default prefix `onnxruntime_profile` is used only when both prefix settings are not set (rows 5, 11, 17, 23) | ||
| 5. **Empty String Behavior**: Setting `ORT_PROFILE_FILE_PREFIX=""` is treated as an explicit override (different from "not set") | ||
|
|
||
| ## Implementation Notes | ||
|
|
||
| - Environment variables are read during `InferenceSession` construction | ||
| - Invalid values for `ORT_ENABLE_PROFILING` (anything other than "0" or "1") are logged as warnings and ignored | ||
| - The prefix from `ORT_PROFILE_FILE_PREFIX` is validated to ensure it doesn't contain path separators or other invalid characters | ||
| - Environment variable values are logged at `INFO` level when profiling is enabled via environment variables | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -512,6 +512,28 @@ void InferenceSession::ConstructorCommon(const SessionOptions& session_options, | |
| } | ||
|
|
||
| session_profiler_.Initialize(session_logger_); | ||
|
|
||
| // Check for environment variable to enable profiling | ||
| const std::string env_enable_profiling = session_env.GetEnvironment().GetEnvironmentVar("ORT_ENABLE_PROFILING"); | ||
| const std::string env_profile_prefix = session_env.GetEnvironment().GetEnvironmentVar("ORT_PROFILE_FILE_PREFIX"); | ||
|
|
||
| if (!env_enable_profiling.empty()) { | ||
| if (env_enable_profiling == "1") { | ||
| // Environment variable enables profiling (does not disable if already enabled via SessionOptions) | ||
| session_options_.enable_profiling = true; | ||
| if (!env_profile_prefix.empty()) { | ||
| session_options_.profile_file_prefix = ToPathString(env_profile_prefix); | ||
| } else if (session_options_.profile_file_prefix.empty()) { | ||
| session_options_.profile_file_prefix = ORT_TSTR("onnxruntime_profile"); | ||
| } | ||
| LOGS(*session_logger_, INFO) << "Profiling enabled via ORT_ENABLE_PROFILING environment variable. " | ||
| << "Profile prefix: " << ToUTF8String(session_options_.profile_file_prefix); | ||
| } else if (env_enable_profiling != "0") { | ||
| LOGS(*session_logger_, WARNING) << "Invalid value for ORT_ENABLE_PROFILING environment variable: '" | ||
| << env_enable_profiling << "'. Expected '0' or '1'. Ignoring."; | ||
| } | ||
| } | ||
|
Comment on lines
+520
to
+535
|
||
|
|
||
| if (session_options_.enable_profiling) { | ||
| StartProfiling(session_options_.profile_file_prefix); | ||
| } | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The documentation claims that "The prefix from ORT_PROFILE_FILE_PREFIX is validated to ensure it doesn't contain path separators or other invalid characters" but this validation is not present in the implementation. The implementation directly converts the environment variable value to a path string without any validation:
Either this validation should be added to the implementation, or this claim should be removed from the documentation.