refactor(gemma4): one forward path carries the numeric gates - #1015
Conversation
6bea7ad to
7d3ee9a
Compare
Signed-off-by: Feathbow <feathbow@gmail.com>
7d3ee9a to
4ead109
Compare
|
The strict gate has its exact-head runner output. At the rebased head the reworked gate already ran with the strict assert and passed under the default bf16 profile: thirteen compared rows, zero argmax flips, per-position max |dlogit| 0 to 1.5 against the 2.0 ceiling. Under The body and Issue text move with it: the short trajectory itself never crosses the window and says so, the crossing stays pinned by the waypoint, mixed-window, overlap and ragged gates, the no-coverage claim contracts to what the gate uniquely adds (full-vocabulary drift at every position of the nine-token prompt plus four non-cascading continuation steps), and the near-tie description is gone. |
Description
Closes #1014
The serving slices already retired the probe forward: the HF fixture gates drive the serving path teacher forced, with the same fixtures, tolerances and distributional floors. What remained of the old arrangement goes here. An incremental-versus-recompute gate answers for the one path's self-consistency where the probe comparison used to: every prompt position's logits through the decode arena match a whole-prompt recompute, and four decode steps fed the recompute's own picks match it too. The release-freezing test switch retires with the eviction gate it served —
window_leftmasks out-of-window keys in-kernel, the waypoint gates cross the window against HF with the resident-page arithmetic asserted, and both arms of the new gate release their fronts as production does.Test Env
Single GPU (
sm_89, 48 GiB, x86_64)Verification
cargo fmt --all -- --checkandgit diff --checkagainstmainpass; the Gemma 4 release server builds; all-target Gemma Clippy passes with-D warnings; release lib tests pass.incremental_serving_matches_recomputepasses 1/1 through the runner (whose membership check owns the manifest swap), on the golden fixture's short prompt the 2.0 ceiling was calibrated on: per-position max |dlogit| runs 0 to 0.79 against the 2.0 ceiling, with one near-tie top-1 flip admitted under the rule that both rows must rank the two candidates within the ceiling of each other; the four decode steps stay under it too. The 12B waypoint and greedy HF gates pass unchanged with the same fixtures and tolerances./v1/completionsrequests (31-, 35-, 33- and 1421-token prompts, 64 forced tokens) with bytes identical tomain's binary, md5219de1e3c09c(everything removed was#[cfg(test)]).