Text-based termination is too brittle. Likely we need some other verifiers to ensure behavior.

Currently, we can advice that if a task is not completed, just run the team again with a new prompt to continue.

Text-based termination is too brittle. Likely we need some other verifiers to ensure behavior.

Currently, we can advice that if a task is not completed, just run the team again with a new prompt to continue.

Agreed. Maybe we just change the example slightly somehow.

+1 with o3-mini I have a similar issue.

@spirosoik have you tried to update the system message for the planner agent?

Alternatively, I think you don't need to use a planner agent if your model is o3-mini. Remove the planner agent, and make sure you have customized your selector prompt following the latest documentation.

@ekzhu I am following this https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/selector-group-chat.html which is confusing even with the selector prompt.

Honestly what I did is to keep it as is without terminate in text and implement a a Judge agent who approves accordingly or not.

Hmm okay. Looks like for o3 mini and reasoning models we need a different type of pattern.

Why is it confusing can you elaborate? How can the doc be improved? Besides changing the pattern.

Second this but with o1-mini - I get an error

BadRequestError                           Traceback (most recent call last)
Cell In[4], [line 25](vscode-notebook-cell:?execution_count=4&line=25)
     [19](vscode-notebook-cell:?execution_count=4&line=19) termination = TextMentionTermination("TERMINATE")
     [20](vscode-notebook-cell:?execution_count=4&line=20) team = SelectorGroupChat(
     [21](vscode-notebook-cell:?execution_count=4&line=21)         [categorizer_agent, reviewer_agent],
     [22](vscode-notebook-cell:?execution_count=4&line=22)         model_client=model_client,
     [23](vscode-notebook-cell:?execution_count=4&line=23)         termination_condition=termination,
     [24](vscode-notebook-cell:?execution_count=4&line=24)     )
---> [25](vscode-notebook-cell:?execution_count=4&line=25) await Console(team.run_stream(task=selector_prompt))
     [28](vscode-notebook-cell:?execution_count=4&line=28) print("Team defined successfully with categorizer_agent and reviewer_agent.")  

File c:\Projects\AutoGenStudio\.vcagent\Lib\site-packages\autogen_agentchat\ui\_console.py:49, in Console(stream, no_inline_images)
     [45](file:///C:/Projects/AutoGenStudio/.vcagent/Lib/site-packages/autogen_agentchat/ui/_console.py:45) total_usage = RequestUsage(prompt_tokens=0, completion_tokens=0)
     [47](file:///C:/Projects/AutoGenStudio/.vcagent/Lib/site-packages/autogen_agentchat/ui/_console.py:47) last_processed: Optional[T] = None
---> [49](file:///C:/Projects/AutoGenStudio/.vcagent/Lib/site-packages/autogen_agentchat/ui/_console.py:49) async for message in stream:
     [50](file:///C:/Projects/AutoGenStudio/.vcagent/Lib/site-packages/autogen_agentchat/ui/_console.py:50)     if isinstance(message, TaskResult):
     [51](file:///C:/Projects/AutoGenStudio/.vcagent/Lib/site-packages/autogen_agentchat/ui/_console.py:51)         duration = time.time() - start_time

File c:\Projects\AutoGenStudio\.vcagent\Lib\site-packages\autogen_agentchat\teams\_group_chat\_base_group_chat.py:404, in BaseGroupChat.run_stream(self, task, cancellation_token)
    [398](file:///C:/Projects/AutoGenStudio/.vcagent/Lib/site-packages/autogen_agentchat/teams/_group_chat/_base_group_chat.py:398) shutdown_task = asyncio.create_task(stop_runtime())
    [400](file:///C:/Projects/AutoGenStudio/.vcagent/Lib/site-packages/autogen_agentchat/teams/_group_chat/_base_group_chat.py:400) try:
    [401](file:///C:/Projects/AutoGenStudio/.vcagent/Lib/site-packages/autogen_agentchat/teams/_group_chat/_base_group_chat.py:401)     # Run the team by sending the start message to the group chat manager.
    [402](file:///C:/Projects/AutoGenStudio/.vcagent/Lib/site-packages/autogen_agentchat/teams/_group_chat/_base_group_chat.py:402)     # The group chat manager will start the group chat by relaying the message to the participants
    [403](file:///C:/Projects/AutoGenStudio/.vcagent/Lib/site-packages/autogen_agentchat/teams/_group_chat/_base_group_chat.py:403)     # and the closure agent.
...
   (...)
   [1652](file:///C:/Projects/AutoGenStudio/.vcagent/Lib/site-packages/openai/_base_client.py:1652)     retries_taken=retries_taken,
   [1653](file:///C:/Projects/AutoGenStudio/.vcagent/Lib/site-packages/openai/_base_client.py:1653) )

BadRequestError: Error code: 400 - {'error': {'message': "Unsupported value: 'messages[0].role' does not support 'system' with this model.", 'type': 'invalid_request_error', 'param': 'messages[0].role', 'code': 'unsupported_value'}

The o family does not support system messages but developer messages.

Second this but with o1-mini - I get an error

BadRequestError                           Traceback (most recent call last)
Cell In[4], [line 25](vscode-notebook-cell:?execution_count=4&line=25)
     [19](vscode-notebook-cell:?execution_count=4&line=19) termination = TextMentionTermination("TERMINATE")
     [20](vscode-notebook-cell:?execution_count=4&line=20) team = SelectorGroupChat(
     [21](vscode-notebook-cell:?execution_count=4&line=21)         [categorizer_agent, reviewer_agent],
     [22](vscode-notebook-cell:?execution_count=4&line=22)         model_client=model_client,
     [23](vscode-notebook-cell:?execution_count=4&line=23)         termination_condition=termination,
     [24](vscode-notebook-cell:?execution_count=4&line=24)     )
---> [25](vscode-notebook-cell:?execution_count=4&line=25) await Console(team.run_stream(task=selector_prompt))
     [28](vscode-notebook-cell:?execution_count=4&line=28) print("Team defined successfully with categorizer_agent and reviewer_agent.")  
...

BadRequestError: Error code: 400 - {'error': {'message': "Unsupported value: 'messages[0].role' does not support 'system' with this model.", 'type': 'invalid_request_error', 'param': 'messages[0].role', 'code': 'unsupported_value'}

The o family does not support system messages but developer messages.

Are you using OpenAI or Azure? On OpenAI o1 and o3-mini support system messages by simply treating them as developer messages. The OpenAI documentation lists them as aliases.

I just checked, and o1-mini does not support developer OR system messages. It acts more like the o1-preview model, which had neither.

@mwarqee yes, we do have a note on AssistantAgent that for o1-preview and o1-mini we don't support system message.

https://microsoft.github.io/autogen/stable/reference/python/autogen_agentchat.agents.html#module-autogen_agentchat.agents

So in this case just don't use system message.

@ekzhu - sorry if this question might have an obvious answer (i did not found it in the docs for class AssistantAgent), but what are options are available to pass instructions to the different AssistantAgent(s) ? Should we pass them as tasks?

Yes, for o1-mini and o1-preview -- they just take the task. I believe this prompting guide is useful when it comes to prompting these types of models. https://techcommunity.microsoft.com/blog/azure-ai-services-blog/prompt-engineering-for-openai%E2%80%99s-o1-and-o3-mini-reasoning-models/4374010

For this issue. Let's focus on adding some explanation in the SelectorGroupChat guide about how to use it with reasoning model like o3-mini -- it comes down to the prompting strategy for agents, and avoid using the planning agent pattern, just rely on the team's selector itself.