Skip to content

fix(skill): set suggested model from original agent#2371

Merged
penge merged 1 commit intomainfrom
fix/skill-llm-model-capability-requirements
Mar 11, 2026
Merged

fix(skill): set suggested model from original agent#2371
penge merged 1 commit intomainfrom
fix/skill-llm-model-capability-requirements

Conversation

@penge
Copy link
Contributor

@penge penge commented Mar 11, 2026

Problem

When wrapping an agent, the skill used a generic model as the suggested hint in LLMServiceExtensionSpec.single_demand(). The platform may allocate a different model if the suggested one is unavailable, which can cause runtime failures.

Changes

  • references/llm-services.md: Add Anti-Pattern bullet — do not use a generic model as suggested; use the same model identifier as the original agent.
  • SKILL.md: Add Step 5a checklist item — set suggested model to match the model used in the original agent.

@dosubot dosubot bot added the size:M This PR changes 30-99 lines, ignoring generated files. label Mar 11, 2026
@penge penge self-assigned this Mar 11, 2026
@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses a critical issue where agents relying on tool-calling functionality can fail at runtime if the platform falls back to an LLM that does not support function calling. The changes introduce comprehensive documentation and updated skill guides to ensure developers explicitly specify and verify that suggested models possess the necessary tool-calling capabilities, thereby preventing runtime failures caused by incompatible fallback models.

Highlights

  • Model Capability Documentation: Introduced a new "Model Capability Requirements" section in references/llm-services.md to address issues with LLM fallback models not supporting tool-calling, including concrete rules and code examples.
  • Skill Guide Updates: Updated SKILL.md with new checklist items (5a, 5b, 5c) and an action block to guide users on verifying model capabilities, especially for agents utilizing tool/function calling.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog
  • skills/agentstack-wrapper/SKILL.md
    • Added new checklist items (5a, 5b, 5c) under Step 5 to explicitly guide users on identifying tool-calling usage, verifying suggested model support, and matching the suggested model to the original agent's model.
    • Inserted an action block with detailed instructions and a "Why this matters" explanation, clarifying the risk of fallback models lacking tool-calling support.
  • skills/agentstack-wrapper/references/llm-services.md
    • Introduced a new "Model Capability Requirements" section detailing the necessity of tool-calling support for suggested models when the original agent uses tools.
    • Provided a clear explanation of the fallback risk and the resulting runtime error.
    • Outlined three concrete rules for selecting and documenting tool-calling capable models.
    • Included "BAD" and "GOOD" Python code examples illustrating correct single_demand usage.
    • Added a new anti-pattern bullet advising against generic model hints for tool-calling agents.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@penge penge force-pushed the fix/skill-llm-model-capability-requirements branch from 43b8d36 to 39c005b Compare March 11, 2026 09:02
@dosubot dosubot bot added size:XS This PR changes 0-9 lines, ignoring generated files. and removed size:M This PR changes 30-99 lines, ignoring generated files. labels Mar 11, 2026
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request effectively addresses the critical issue of agents failing at runtime when a fallback LLM does not support tool calling. The updates to SKILL.md introduce clear checklist items for developers to verify model capabilities, while references/llm-services.md provides detailed explanations, concrete rules, and helpful BAD/GOOD code examples. These changes significantly improve the clarity and robustness of the skill development process by guiding users to explicitly consider and configure model capabilities, thereby preventing common runtime failures.

Note: Security Review has been skipped due to the limited scope of the PR.

@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Mar 11, 2026
@penge penge force-pushed the fix/skill-llm-model-capability-requirements branch from 39c005b to cf1f02a Compare March 11, 2026 09:04
@penge penge changed the title fix(skill): require tool-calling support for model fix(skill): set suggested model from original agent Mar 11, 2026
@penge penge force-pushed the fix/skill-llm-model-capability-requirements branch from cf1f02a to 200eccc Compare March 11, 2026 09:08
- Add Anti-Pattern bullet to llm-services.md: use the same model identifier
  as the original agent, not a generic one
- Add Step 5a checklist item to SKILL.md: set suggested model to match the
  model used in the original agent

Signed-off-by: penge <ppenge@gmail.com>
@penge penge force-pushed the fix/skill-llm-model-capability-requirements branch from 200eccc to 26d4b1c Compare March 11, 2026 09:39
@penge penge merged commit f0bf9f7 into main Mar 11, 2026
15 checks passed
@penge penge deleted the fix/skill-llm-model-capability-requirements branch March 11, 2026 09:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lgtm This PR has been approved by a maintainer size:XS This PR changes 0-9 lines, ignoring generated files.

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants