Skip to content

Update model catalog #85

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
DerekWWang merged 2 commits into from
Jun 2, 2026
Merged

Update model catalog #85

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f76b0ac
Update model catalog
DerekWWang Jun 2, 2026
3a4b74e
Updating pyproject.toml to ignore duplicate-inclusion issue
DerekWWang Jun 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions services/bench/common/bench_common/model_catalog.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,10 @@
"gemini/gemini-3.5-flash",
"gemini/gemini-flash-latest",
"gemini/gemini-flash-lite-latest",
# Frontier models benched ad hoc via scripts/run_benchmark_matrix.py. Kept
# out of FULL_BENCH_MODELS so the canonical full-bench set stays at five.
"anthropic/claude-opus-4-8",
"openai/gpt-5.5",
)

ALLOWED_MODELS: tuple[str, ...] = FULL_BENCH_MODELS + EXTRA_MODEL_ALIASES
Expand All @@ -37,4 +41,6 @@
"gemini/gemini-3.5-flash": "Gemini 3.5 Flash",
"gemini/gemini-flash-latest": "Gemini Flash (latest)",
"gemini/gemini-flash-lite-latest": "Gemini Flash Lite (latest)",
"anthropic/claude-opus-4-8": "Claude Opus 4.8",
"openai/gpt-5.5": "GPT-5.5",
}
8 changes: 5 additions & 3 deletions services/bench/common/pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,11 @@ build-backend = "hatchling.build"

[tool.hatch.build.targets.wheel]
packages = ["bench_common"]

[tool.hatch.build.targets.wheel.force-include]
"bench_common/cli/templates" = "bench_common/cli/templates"
# NOTE: do not add a force-include for bench_common/cli/templates — `packages`
# already ships every file under bench_common/ (including non-.py template
# files), so force-including the same paths makes hatchling add each file
# twice and fail the build ("A second file is being added ... at the same
# path"). See git history for the duplicate-inclusion regression.

[tool.ruff]
target-version = "py311"
Expand Down
Loading