Skip to content

serving: release_id keys audits, quarantine and the READY pool; gateway routes by model - #1728

Merged
anderdc merged 1 commit into
testfrom
serving/release-id
Aug 28, 2026
Merged

serving: release_id keys audits, quarantine and the READY pool; gateway routes by model#1728
anderdc merged 1 commit into
testfrom
serving/release-id

Conversation

@anderdc

@anderdc anderdc commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator

One model can have several blessed runtimes, and phase 1 assigns gpu -> release. Keying the audit window, quarantine, READY pool and served requests by model_id collides the moment two releases serve the same model — so everything the validator tracks moves to release_id. It defaults to model_id, so today's single-release loadout behaves identically.

The bar this is aimed at: adding a second model (a 27B dense on the same sparkinfer runtime) should be a loadout entry plus a reference, not a refactor.

  • loadoutrelease_id on ServingRelease, unique across the loadout; get() resolves a release_id then a model_id. An optional audit block per release overrides the three logprob bands (a non-deterministic runtime ships its own; the constants stay the default).
  • gateway/v1/models lists every release; model in the request body picks one (404 when it is not served); the request is audited against that release's reference.
  • minerInferenceSynapse carries release_id, and a miner serving another release refuses the call rather than answering for a release it was not blessed for.
  • window — strikes counted per (hotkey, release) and persisted, so escalating quarantine becomes a policy constant rather than a schema change.
  • storeaudit_values / quarantine rename model_id -> release_id in place and gain strikes; existing rows carry over under their old key.

Pairs with entrius/gittensor-db#NN (release_id columns) and entrius/das-gittensor#NN — the DB columns should land first.

CI: ruff, format, pyright, vulture, 742 tests green.

…ay routes by model

One model can have several blessed runtimes, and phase 1 assigns gpu -> release. Keying
the audit window, quarantine, READY pool and served requests by model_id collides the
moment two releases serve the same model, so everything the validator tracks moves to
release_id (defaults to model_id, so a one-release loadout is unchanged).

- loadout: release_id on ServingRelease, unique across the loadout; get() resolves a
  release_id then a model_id; an `audit` block per release overrides the logprob bands
  (a non-deterministic runtime ships its own; the constants stay the default).
- gateway: /v1/models lists every release; `model` in the request body picks one
  (404 when it is not served); the request is audited against that release's reference.
- miner: InferenceSynapse carries release_id and a miner serving another release refuses
  the call rather than answering for a release it was not blessed for.
- window: strikes are counted per (hotkey, release) and persisted, so escalating
  quarantine is a policy change rather than a schema one.
- store: audit_values/quarantine rename model_id -> release_id in place; existing rows
  carry over under their old key.
@anderdc
anderdc merged commit 7f03e0b into test Aug 28, 2026
4 checks passed
@anderdc
anderdc deleted the serving/release-id branch August 28, 2026 20:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant