Skip to content

refactor(gemma4): one forward path carries the numeric gates - #1015

Merged
FeathBow merged 1 commit into
pegainfer-project:mainfrom
FeathBow:refactor/gemma4-one-forward
Aug 31, 2026
Merged

refactor(gemma4): one forward path carries the numeric gates#1015
FeathBow merged 1 commit into
pegainfer-project:mainfrom
FeathBow:refactor/gemma4-one-forward

Conversation

@FeathBow

Copy link
Copy Markdown
Collaborator

Description

Closes #1014

The serving slices already retired the probe forward: the HF fixture gates drive the serving path teacher forced, with the same fixtures, tolerances and distributional floors. What remained of the old arrangement goes here. An incremental-versus-recompute gate answers for the one path's self-consistency where the probe comparison used to: every prompt position's logits through the decode arena match a whole-prompt recompute, and four decode steps fed the recompute's own picks match it too. The release-freezing test switch retires with the eviction gate it served — window_left masks out-of-window keys in-kernel, the waypoint gates cross the window against HF with the resident-page arithmetic asserted, and both arms of the new gate release their fronts as production does.

Test Env

Single GPU (sm_89, 48 GiB, x86_64)

Verification

  • cargo fmt --all -- --check and git diff --check against main pass; the Gemma 4 release server builds; all-target Gemma Clippy passes with -D warnings; release lib tests pass.
  • incremental_serving_matches_recompute passes 1/1 through the runner (whose membership check owns the manifest swap), on the golden fixture's short prompt the 2.0 ceiling was calibrated on: per-position max |dlogit| runs 0 to 0.79 against the 2.0 ceiling, with one near-tie top-1 flip admitted under the rule that both rows must rank the two candidates within the ceiling of each other; the four decode steps stay under it too. The 12B waypoint and greedy HF gates pass unchanged with the same fixtures and tolerances.
  • The 26B checkpoint answers four greedy public /v1/completions requests (31-, 35-, 33- and 1421-token prompts, 64 forced tokens) with bytes identical to main's binary, md5 219de1e3c09c (everything removed was #[cfg(test)]).

@FeathBow
FeathBow force-pushed the refactor/gemma4-one-forward branch 2 times, most recently from 6bea7ad to 7d3ee9a Compare August 31, 2026 10:47
Signed-off-by: Feathbow <feathbow@gmail.com>
@FeathBow
FeathBow force-pushed the refactor/gemma4-one-forward branch from 7d3ee9a to 4ead109 Compare August 31, 2026 11:02
@FeathBow

Copy link
Copy Markdown
Collaborator Author

The strict gate has its exact-head runner output. At the rebased head the reworked gate already ran with the strict assert and passed under the default bf16 profile: thirteen compared rows, zero argmax flips, per-position max |dlogit| 0 to 1.5 against the 2.0 ceiling. Under PEGAINFER_GATE_STORAGE=fp8 the same gate is red by design — the e4m3 pool's first divergent decision lands at position 2 and the strict assert stops it — which is the color that profile exists to show, beside its red agreement and waypoint gates. The earlier "one admitted near-tie flip" line came from a calibration on the pre-rebase base and does not reproduce on this one, which is exactly what let the contract tighten to strict with no separate margin constant.

The body and Issue text move with it: the short trajectory itself never crosses the window and says so, the crossing stays pinned by the waypoint, mixed-window, overlap and ragged gates, the no-coverage claim contracts to what the gate uniquely adds (full-vocabulary drift at every position of the nine-token prompt plus four non-cascading continuation steps), and the near-tie description is gone.

@FeathBow
FeathBow merged commit 994e88b into pegainfer-project:main Aug 31, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

gemma4: the probe forward and the serving path both carry the numeric gates

1 participant