Skip to content

fix: add Gemini -latest aliases so a new API key can actually run the pipeline - #379

Open
ayo0la wants to merge 1 commit into
interviewstreet:mainfrom
ayo0la:fix/gemini-model-availability
Open

fix: add Gemini -latest aliases so a new API key can actually run the pipeline#379
ayo0la wants to merge 1 commit into
interviewstreet:mainfrom
ayo0la:fix/gemini-model-availability

Conversation

@ayo0la

@ayo0la ayo0la commented Jul 27, 2026

Copy link
Copy Markdown

Summary

No Gemini model in providers.json completes a request on a newly created API key. Two of the five fail with a hard 404 model-availability error, and they are exactly the two .env.example recommends.

Probed each shipped model against the same endpoint OpenAICompatibleProvider uses (POST https://generativelanguage.googleapis.com/v1beta/openai/chat/completions), with a key created today:

model status message
gemini-2.0-flash 429 quota
gemini-2.0-flash-lite 429 quota
gemini-2.5-flash 404 ... is no longer available to new users
gemini-2.5-flash-lite 404 ... is no longer available to new users
gemini-2.5-pro 429 quota

The 404s are the hard failure — those versions are retired at the model level for new keys, so no quota or billing change recovers them. I am deliberately not claiming the 429s as proven config bugs: I had spent quota on that key earlier the same day and cannot fully rule out ordinary project quota. They are listed as observed.

Why this bites new contributors specifically

.env.example recommended "gemini-2.5-pro" and "gemini-2.5-flash", and README.md used gemini-2.5-pro and gemini-2.0-flash as its worked examples. Following the README verbatim with a fresh key failed on the first run, with an error that reads like a bad API key rather than stale config.

Fix

Adds the two -latest aliases, which track the current version rather than going stale, and points .env.example and README.md at them:

  • gemini-flash-latest
  • gemini-flash-lite-latest

The existing dated pins are left in place on purpose. They still resolve for keys that already had access, and removing them would break those users for no benefit. The docs simply stop recommending them, and the README now notes why aliases are preferable.

This is the part that stops the bug recurring — the same retirement will hit any gemini-3.x pin in a few months.

Testing

  • Both new aliases return 200 on a freshly created key against the repo's own endpoint; the five existing pins are unchanged.

  • providers.json validates as JSON, and config.provider_for() resolves both new entries to the Gemini provider with the expected temperature / top_p.

  • Full end-to-end run with DEFAULT_MODEL=gemini-flash-lite-latest:

    python score.py ./resume/sample.pdf --role software_engineering_intern
    

    Section extraction, evaluation, and scoring all complete and return well-formed output — 0.0/100 with coherent per-category evidence, which is the correct result for the bundled non-engineer sample resume.

Reported in #378. Fixes #378

… pins

No Gemini model in providers.json completes a request on a newly created
API key. Probing each shipped model against the same endpoint
OpenAICompatibleProvider uses:

  gemini-2.0-flash        429  quota
  gemini-2.0-flash-lite   429  quota
  gemini-2.5-flash        404  "no longer available to new users"
  gemini-2.5-flash-lite   404  "no longer available to new users"
  gemini-2.5-pro          429  quota

The 404s are the hard failure: those two versions are retired at the
model level for new keys, so no quota or billing change recovers them.
The 429s are reported as observed rather than as proven config bugs,
since project quota can't be fully ruled out.

This bites new contributors specifically, because .env.example and the
README both steered them to exactly those retired pins — a first run
with a fresh key failed with an error that reads like a bad API key
rather than stale config.

Adds the gemini-flash-latest and gemini-flash-lite-latest aliases, which
track the current version instead of going stale, and points the docs at
them. The existing dated pins are left in place: they still resolve for
keys that already had access, and removing them would break those users
for no benefit.

Verified end to end with DEFAULT_MODEL=gemini-flash-lite-latest on
resume/sample.pdf --role software_engineering_intern: extraction,
evaluation, and scoring all complete and return well-formed output. Both
new aliases return 200 on a fresh key; the retired pins are unchanged.

Fixes interviewstreet#378
@ayo0la

ayo0la commented Aug 8, 2026

Copy link
Copy Markdown
Author

Still reproduces on main today (2026-08-08). Re-probed every shipped Gemini model against the same endpoint OpenAICompatibleProvider uses:

model status
gemini-2.0-flash 429 (quota, availability masked)
gemini-2.0-flash-lite 429 (quota, availability masked)
gemini-2.5-flash 404
gemini-2.5-flash-lite 404
gemini-2.5-pro 429 (quota, availability masked)
gemini-flash-latest 200
gemini-flash-lite-latest 200

Worth noting the 404 set has moved since I first filed this in July: back then gemini-2.5-flash and gemini-2.5-pro were the two hard 404s, and gemini-2.5-flash-lite was reachable. Today gemini-2.5-flash-lite 404s instead. That drift is the actual argument for the rolling aliases: whichever dated pin is listed will eventually retire, so the config needs re-editing every time rather than just working.

One note for triage, since there is overlap: #367 also touches this, but is narrower and now partly stale.

Happy to close this in favour of #367 if you would rather take that one, though that would still leave the docs pointing at models that do not resolve.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Bug: no Gemini model in providers.json is usable on a new API key (2.5-flash / 2.5-flash-lite return 404 "no longer available to new users")

1 participant