Why does LoopGain need ~6 iterations before it trusts a trend?

[fitz2882]

[fitz2882]

Question: The README recommends a minimum of 6 iterations for reliable trend significance. My loops usually finish in 3. Is LoopGain useless for me?

Answer: Partly, and here's the honest mechanics of why.

The trajectory classifier decides using an OLS slope on log10(E) plus a t-test on that slope. At n≤4 the t-test is severely underpowered — with df=2 you need |t| > 4.3 to clear p<0.05. So on very short loops the significance test almost never fires, and the classifier conservatively falls back to STALLING rather than claiming a trend it can't support.

That conservatism is deliberate. Real LLM error series are noisy, and a classifier that calls DIVERGING on three noisy points would abort good loops. Requiring both statistical significance and meaningful cumulative motion is what keeps false-positive aborts down.

What still works on short loops:

• The target_error short-circuit fires the moment observed error hits your target, independent of any trend analysis. On a loop that reaches zero failing tests at iteration 3, that's the mechanism doing the work — and it's exact, not statistical.
• Best-so-far rollback still applies. You get argmin(error) back regardless of how the loop terminated.

Where it genuinely doesn't help: a 3-iteration loop that plateaus above target. There isn't enough evidence to distinguish "plateaued" from "still descending slowly," so LoopGain will stall out rather than guess. If that's your workload, the win is smaller and you should measure it rather than take the headline number.

If you're running short loops and seeing something surprising, post the error_history — I'd like to see real short traces.