Skip to content

[PY] feat: o1 model support #2123

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Oct 23, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion python/packages/ai/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ dataclasses-json = "^0.6.4"
azure-ai-contentsafety = "^1.0.0"
msal = "^1.28.0"
botbuilder-dialogs = "^4.14.8"
openai = "^1.27.0"
openai = "^v1.52.0"

[tool.poetry.group.dev.dependencies]
pytest = "^8.1.1"
Expand Down
42 changes: 34 additions & 8 deletions python/packages/ai/teams/ai/models/openai_model.py
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,24 @@ class AzureOpenAIModelOptions:
class OpenAIModel(PromptCompletionModel):
"""
A `PromptCompletionModel` for calling OpenAI and Azure OpenAI hosted models.

The model has been updated to support calling OpenAI's new o1 family of models. That currently
comes with a few constraints. These constraints are mostly handled for you but are worth noting:

- The o1 models introduce a new `max_completion_tokens` parameter and they've deprecated the
`max_tokens` parameter. The model will automatically convert the incoming `max_tokens` parameter
to `max_completion_tokens` for you. But you should be aware that o1 has hidden token usage and costs
that aren't constrained by the `max_completion_tokens` parameter. This means that you may see an
increase in token usage and costs when using the o1 models.

- The o1 models do not currently support the sending of system message so the model will map them to user message
in this case.

- The o1 models do not currently support setting the `temperature`, `top_p`, and `presence_penalty`
parameters so they will be ignored.

- The o1 models do not currently support the use of tools so you will need to use the "monologue"
augmentation to call actions.
"""

_options: Union[OpenAIModelOptions, AzureOpenAIModelOptions]
Expand Down Expand Up @@ -162,6 +180,7 @@ async def complete_prompt(
if template.config.completion.model is not None
else self._options.default_model
)
is_o1_model = model.startswith("o1-")

res = await template.prompt.render_as_messages(
context=context,
Expand Down Expand Up @@ -232,10 +251,17 @@ async def complete_prompt(
content=msg.content if msg.content else "",
)
elif msg.role == "system":
param = chat.ChatCompletionSystemMessageParam(
role="system",
content=msg.content if msg.content is not None else "",
)
# o1 models do not support system messages
if is_o1_model:
param = chat.ChatCompletionUserMessageParam(
role="user",
content=msg.content if msg.content is not None else "",
)
else:
param = chat.ChatCompletionSystemMessageParam(
role="system",
content=msg.content if msg.content is not None else "",
)

if msg.name:
param["name"] = msg.name
Expand All @@ -250,11 +276,11 @@ async def complete_prompt(
completion = await self._client.chat.completions.create(
messages=messages,
model=model,
presence_penalty=template.config.completion.presence_penalty,
presence_penalty=template.config.completion.presence_penalty if not is_o1_model else 0,
frequency_penalty=template.config.completion.frequency_penalty,
top_p=template.config.completion.top_p,
temperature=template.config.completion.temperature,
max_tokens=template.config.completion.max_tokens,
top_p=template.config.completion.top_p if not is_o1_model else 1,
temperature=template.config.completion.temperature if not is_o1_model else 1,
max_completion_tokens=template.config.completion.max_tokens,
tools=tools if len(tools) > 0 else NOT_GIVEN,
tool_choice=tool_choice if len(tools) > 0 else NOT_GIVEN,
parallel_tool_calls=parallel_tool_calls if len(tools) > 0 else NOT_GIVEN,
Expand Down
Loading