From b62c2e93ed277b404e8e450c74d0b0da8b9d751b Mon Sep 17 00:00:00 2001 From: Abhishek Jindal Date: Wed, 6 Nov 2024 09:46:55 -0800 Subject: [PATCH 1/3] add set runtime option for C API in GenAI --- docs/genai/api/c.md | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/docs/genai/api/c.md b/docs/genai/api/c.md index 06da3647ea78b..20ded82ff5525 100644 --- a/docs/genai/api/c.md +++ b/docs/genai/api/c.md @@ -414,6 +414,23 @@ A pointer to the token sequence OGA_EXPORT const int32_t* OGA_API_CALL OgaGenerator_GetSequenceData(const OgaGenerator* generator, size_t index); ``` +### Set Runtime Option + +Provides an option to set Runtime options, more parameters will be added to this generic API to support Runtime options. An example to use this API for terminating the current session would be to call the SetRuntimeOption with key as "terminate_session" and value as "1". + +#### Parameters + +* Input: generator The generator on which the Runtime option needs to be set +* Input: key The key for setting the runtime option +* Input: value The value for the key provided + +#### Returns +`void` + +```c +OGA_EXPORT void OGA_API_CALL OgaGenerator_SetRuntimeOption(OgaGenerator* generator, const char* key, const char* value); +``` + ## Enums and structs ```c From 25838d67951a36a7a932e80b61ea72a1d3211709 Mon Sep 17 00:00:00 2001 From: Abhishek Jindal Date: Wed, 6 Nov 2024 11:58:29 -0800 Subject: [PATCH 2/3] add example --- docs/genai/api/c.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/genai/api/c.md b/docs/genai/api/c.md index 20ded82ff5525..ea32e37bcb771 100644 --- a/docs/genai/api/c.md +++ b/docs/genai/api/c.md @@ -416,7 +416,7 @@ OGA_EXPORT const int32_t* OGA_API_CALL OgaGenerator_GetSequenceData(const OgaGen ### Set Runtime Option -Provides an option to set Runtime options, more parameters will be added to this generic API to support Runtime options. An example to use this API for terminating the current session would be to call the SetRuntimeOption with key as "terminate_session" and value as "1". +An API to set Runtime options, more parameters will be added to this generic API to support Runtime options. An example to use this API for terminating the current session would be to call the SetRuntimeOption with key as "terminate_session" and value as "1": OgaGenerator_SetRuntimeOption(generator, "terminate_session", "1") #### Parameters From 11716783ca62d6463df51a4f03f19716e5a48189 Mon Sep 17 00:00:00 2001 From: Abhishek Jindal Date: Wed, 6 Nov 2024 17:28:27 -0800 Subject: [PATCH 3/3] add details on available runtime options --- docs/genai/api/c.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/genai/api/c.md b/docs/genai/api/c.md index ea32e37bcb771..c08c1936f125b 100644 --- a/docs/genai/api/c.md +++ b/docs/genai/api/c.md @@ -418,6 +418,8 @@ OGA_EXPORT const int32_t* OGA_API_CALL OgaGenerator_GetSequenceData(const OgaGen An API to set Runtime options, more parameters will be added to this generic API to support Runtime options. An example to use this API for terminating the current session would be to call the SetRuntimeOption with key as "terminate_session" and value as "1": OgaGenerator_SetRuntimeOption(generator, "terminate_session", "1") +More details on the current runtime options can be found [here](https://github.com/microsoft/onnxruntime-genai/blob/main/documents/Runtime_option.md). + #### Parameters * Input: generator The generator on which the Runtime option needs to be set