Repository navigation
Conversation
Chromium keeps what its AI reviewer knows as one markdown file per rule plus COMMON_METADATA files that say when each rule applies, by path and by what the change adds. review.upstreamRules in a profile points /review-prs at a synced copy, and prepare-review.py picks the rules a diff meets from that metadata without a model. brave-core turns it on, restricted to rules a diff triggers by content; no other profile sets the key.
Member
|
Claudio is handling this differently. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
User impact: once the rules are synced, a brave-core pull request is also checked against Chromium's own review rules, and a finding links to the rule's file on chromium.googlesource.com. No other project changes.
The problem
Chromium publishes the 838 rules its AI reviewer applies, under
agents/knowledge_base/rules/. netzenbot reads none of them, so a brave-core change that breaks one (a newUnretainedin a posted task, a WebUI component with no Mocha test) is caught only if our own best-practices docs say the same.Reproduce
project.profileset tobrave-core, runpython3 scripts/sync-upstream-rules.py./review-prs #<number> openon a brave-core pull request whose diff addsbase::BindOnce(&Foo::Bar, base::Unretained(this))to a posted task.manifest.jsonin the work directory and read the pull request'ssubagent_prompts.Before: every
rulesentry is one of our docs (coding-standards.md_0and so on). Expected: one more entry,chromium-rules-1.md_0, carrying the Chromium rules that diff meets.I have not run these steps against a live pull request. I ran the same selection and prompt build on the real synced rules with a synthetic diff adding that line: one pool, two rules,
prefer-weakptr-over-unretained-for-posted-tasksamong them, a 10,313 character prompt.tests/test_upstream_rules.pycovers it and fails on the parent commit, wherescripts/lib/upstream_rules.pydoes not exist.The fix
Chromium's tree is its own index: each directory's
COMMON_METADATAlists groups of rules with apath_expressionover the changed file and optionalcontent_patternsover the lines the change adds.scripts/sync-upstream-rules.pydownloads the tree, andprepare-review.pyreads that metadata, with no model, to decide which rules a diff meets. Rules meeting the same files share a pool, chunked like one of our docs, so each chunk is an ordinary detect subagent. A finding must carry the link of a synced rule; a link that names none is dropped.Only brave-core turns it on, with
triggeredOnly: a rule is checked only when the diff adds what it is about.blockingrules are reported asmediumand the rest aslow, so none stops an approval until someone raises it inseverityMap. A profile without the key, or one not yet synced, reviews as before.A rule reaches the model like this, trimmed from the real run:
and the profile key it needs:
Cost, and what is not covered
On the 60 most recent brave-core commits (October 2026), checking every group a file path alone meets would have been a median of 86 rules and about five more detect subagents per change.
triggeredOnlyis about half of one. That is why it ships on; turning it off is a one-line change once the findings show they are worth the tokens.main. Brave's Chromium may lack APIs a rule mentions, and the localsrc/agentshas noknowledge_baseyet.chromium_src/is translated to the path a rule is written against.browser/is notchrome/browser/, so the Chrome-specific rules underchrome/do not fire on our code. Thegeneral/rules are not tied to a directory.*.mdandCOMMON_METADATAare kept. A download that does not parse into rules leaves the ones already synced.proposeBestPracticesagainst upstream headings.Test plan
make lint— passed.venv/bin/python -m pytest tests/test_upstream_rules.py— 52 passedmake teston the rebased branch — 1727 passed, 5 failed: the same four that fail on this machine without these changes (test_the_reviewer_is_asked_again_after_the_pushand three inTestReviewRequestQueue, which depend on the local reviewer account), andTestStartDelay, which passes alone and failed while the scan ran alongsidemake check-reviewdog— no findings; both runners also printed "could not run cleanly" because thelandrunsandbox is not installed here, so they ran unsandboxedpython3 scripts/sync-upstream-rules.py --archiveagainst the real 838-rule tarball, run twice — 838 rules in 145 groups, none in no group, the second run replaced the firstpython3 scripts/sync-upstream-rules.pyrun on the deployment (until then brave-core reviews log a warning and run as before)/review-prsrun against a real brave-core pull request shows achromium-rules-*entry