-
Notifications
You must be signed in to change notification settings - Fork 6.7k
Inject SKILL.md when it's explicitly mentioned. #7763
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
💡 Codex Review
codex/codex-rs/core/src/codex.rs
Lines 2159 to 2163 in a98bfdf
| // Note that pending_input would be something like a message the user | |
| // submitted through the UI while the model was running. Though the UI | |
| // may support this, the model might not. | |
| let pending_input = sess | |
| .get_pending_input() |
Skill contents are injected only once from the initial input (line 2139), but follow-up messages queued in pending_input inside the turn loop are never scanned for $<skill> mentions. If a user sends an additional message while the model is running that names a skill, that SKILL.md is never inlined, so the model responds without the requested instructions. Consider running inject_skills on pending_input before adding it to history so mid-turn skill mentions get the same treatment as the initial message.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| fn parse_user_message(message: &[ContentItem]) -> Option<UserMessageItem> { | ||
| if UserInstructions::is_user_instructions(message) { | ||
| if UserInstructions::is_user_instructions(message) | ||
| || SkillInstructions::is_skill_instructions(message) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
needs a test. see below in this file
|
|
||
| pub const USER_INSTRUCTIONS_OPEN_TAG_LEGACY: &str = "<user_instructions>"; | ||
| pub const USER_INSTRUCTIONS_PREFIX: &str = "# AGENTS.md instructions for "; | ||
| pub const SKILL_INSTRUCTIONS_PREFIX: &str = "# SKILL.md instructions for "; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should just do the xml tag without the prefix.
| } | ||
|
|
||
| impl From<SkillInstructions> for ResponseItem { | ||
| fn from(si: SkillInstructions) -> Self { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Needs a test. See examples below.
codex-rs/core/src/codex.rs
Outdated
| } | ||
| } | ||
|
|
||
| async fn inject_skills( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure if re-parsing user message in core is the right approach.
It might be better to pass selected skills as one of UserInputs or as an explicit parameter on UserTurn similar to how we handle images.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The other problem with the current approach is that new skills added during the session won't be mentionable. And the contents of the skills won't be reloaded.
| /// Combines `Config::instructions` and `AGENTS.md` (if present) into a single | ||
| /// string of instructions. | ||
| pub(crate) async fn get_user_instructions(config: &Config) -> Option<String> { | ||
| pub(crate) async fn get_user_instructions( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
now what we have a separate skills instruction element should we render skills into it instead of injecting skills into user instructions?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The skills list in project docs is for discovery/implicit triggers only. SKILL instructions are explicit “use this skill” signals that loop in the full SKILL.md context.
pakrym-oai
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR needs integration tests, that send user messages with skills and assert that correct information is sent to responses api.
630a6f1 to
55777bf
Compare
|
@codex review again except for the issue with |
|
Codex Review: Didn't find any major issues. Breezy! ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
codex-rs/core/src/state/session.rs
Outdated
| pub(crate) session_configuration: SessionConfiguration, | ||
| pub(crate) history: ContextManager, | ||
| pub(crate) latest_rate_limits: Option<RateLimitSnapshot>, | ||
| pub(crate) skills: Option<SkillLoadOutcome>, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's put this onto SessionServices.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Session state contains frequently mutated state.
codex-rs/core/src/codex.rs
Outdated
| selected | ||
| } | ||
|
|
||
| fn paths_match(a: &Path, b: &Path) -> bool { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we need this check?
codex-rs/core/src/codex.rs
Outdated
| } | ||
| } | ||
|
|
||
| async fn inject_skills( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can we please mode this and related skill helpers into skill.rs?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can also take session as a reference instead of having an instance method on it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is to keep the size of codex.rs sane
|
|
||
| fn skill_load_outcome_for_client( | ||
| outcome: Option<&SkillLoadOutcome>, | ||
| ) -> Option<SkillLoadOutcomeInfo> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you think we can move SkillLoadOutcome to the protocol directly and avoid double copies?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Using the protocol type inside core would couple the loader/cache to the external serde/TS model. The current split keeps boundaries clean, and the mapping is trivial. wdyt?
|
|
||
| pub const USER_INSTRUCTIONS_OPEN_TAG_LEGACY: &str = "<user_instructions>"; | ||
| pub const USER_INSTRUCTIONS_PREFIX: &str = "# AGENTS.md instructions for "; | ||
| pub const SKILL_INSTRUCTIONS_PREFIX: &str = "<SKILL"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can we please do lower case, mostly to be consistent with other elements
| role: "user".to_string(), | ||
| content: vec![ContentItem::InputText { | ||
| text: format!( | ||
| "<SKILL name=\"{name}\" path=\"{path}\">\n{contents}\n</SKILL>", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: for env context we used sub tags instead of attributes. I don't have a strong opinion but might be good to stay consistent.
| } | ||
|
|
||
| #[tokio::test(flavor = "multi_thread", worker_threads = 2)] | ||
| async fn user_turn_includes_skill_instructions() -> Result<()> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
very nice!!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
consider a test for when skills fail to load.
codex-rs/protocol/src/models.rs
Outdated
| } | ||
| } | ||
| }, | ||
| UserInput::Skill { .. } => ContentItem::InputText { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this correct ? Should we be creating an extra empty content item?
| Ok(()) | ||
| } | ||
|
|
||
| pub async fn submit_items( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
most of our tests just call self.codex.submit(Op::UserTurn directly with whatever parameters they need.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Otherwise we'll end up with too many of these overloads.
codex-rs/core/src/codex.rs
Outdated
|
|
||
| let user_instructions = get_user_instructions( | ||
| &config, | ||
| skills_outcome.as_ref().and_then(|outcome| { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why aren't we passin outcome.skills directly?
|
@codex review again |
|
Codex Review: Didn't find any major issues. Nice work! ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Uh oh!
There was an error while loading. Please reload this page.