Skip to content

Commit

Permalink
add set runtime option for C API in GenAI (#22748)
Browse files Browse the repository at this point in the history
### Description
Add Set runtime option for C API in GenIA



### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
  • Loading branch information
ajindal1 authored Nov 7, 2024
1 parent b57e2e1 commit 09b0d8f
Showing 1 changed file with 19 additions and 0 deletions.
19 changes: 19 additions & 0 deletions docs/genai/api/c.md
Original file line number Diff line number Diff line change
Expand Up @@ -414,6 +414,25 @@ A pointer to the token sequence
OGA_EXPORT const int32_t* OGA_API_CALL OgaGenerator_GetSequenceData(const OgaGenerator* generator, size_t index);
```

### Set Runtime Option

An API to set Runtime options, more parameters will be added to this generic API to support Runtime options. An example to use this API for terminating the current session would be to call the SetRuntimeOption with key as "terminate_session" and value as "1": OgaGenerator_SetRuntimeOption(generator, "terminate_session", "1")

More details on the current runtime options can be found [here](https://github.com/microsoft/onnxruntime-genai/blob/main/documents/Runtime_option.md).

#### Parameters

* Input: generator The generator on which the Runtime option needs to be set
* Input: key The key for setting the runtime option
* Input: value The value for the key provided

#### Returns
`void`

```c
OGA_EXPORT void OGA_API_CALL OgaGenerator_SetRuntimeOption(OgaGenerator* generator, const char* key, const char* value);
```
## Enums and structs
```c
Expand Down

0 comments on commit 09b0d8f

Please sign in to comment.