forked from openai/parameter-golf
-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathfull_4pass_stdout.log
More file actions
35 lines (34 loc) · 2.39 KB
/
full_4pass_stdout.log
File metadata and controls
35 lines (34 loc) · 2.39 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
START full run: 4-pass noRMS jac=0.1, 80min, 80GB cap (Thu Mar 26 13:35:54 UTC 2026)
run_full_4pass.sh: line 65: 277899 Killed $PYTHON train_gpt_recurrent.py --feedback-mode diagonal --feedback-rank 2 --residual-scale-init 0.5 --jacobian-proxy-weight 0.1 --no-interpass-rmsnorm > "$LOG" 2>&1
FAILED (exit=137)
step:2700/20000 train_loss:2.0195 train_time:3686191ms step_avg:1365.26ms
step:2750/20000 train_loss:2.0010 train_time:3754675ms step_avg:1365.34ms
step:2800/20000 train_loss:2.0359 train_time:3823161ms step_avg:1365.41ms
swa:start step:2850
step:2850/20000 train_loss:1.9860 train_time:3891626ms step_avg:1365.48ms
step:2900/20000 train_loss:2.0033 train_time:3960176ms step_avg:1365.58ms
step:2950/20000 train_loss:2.0417 train_time:4028712ms step_avg:1365.67ms
late_qat:enabled step:2990 scale:0.1498
step:3000/20000 train_loss:1.9297 train_time:4097686ms step_avg:1365.90ms
step:3000/20000 val_loss:1.9846 val_bpb:1.1754 train_time:4097782ms step_avg:1365.93ms
step:3050/20000 train_loss:1.9368 train_time:4166118ms step_avg:1365.94ms
step:3100/20000 train_loss:2.0003 train_time:4234401ms step_avg:1365.94ms
step:3150/20000 train_loss:2.0099 train_time:4302671ms step_avg:1365.93ms
step:3200/20000 train_loss:1.9846 train_time:4370945ms step_avg:1365.92ms
step:3250/20000 train_loss:1.9515 train_time:4439218ms step_avg:1365.91ms
step:3300/20000 train_loss:1.9330 train_time:4507468ms step_avg:1365.90ms
step:3450/20000 train_loss:1.9670 train_time:4712315ms step_avg:1365.89ms
step:3500/20000 train_loss:1.9497 train_time:4781271ms step_avg:1366.08ms
step:3500/20000 val_loss:1.9558 val_bpb:1.1583 train_time:4781368ms step_avg:1366.11ms
step:3514/20000 val_loss:1.9557 val_bpb:1.1583 train_time:4800532ms step_avg:1366.12ms
stopping_early: wallclock_cap train_time:4800532ms step:3514/20000
peak memory allocated: 50545 MiB reserved: 50594 MiB
ema:applying EMA weights
DIAGNOSTIC post_ema val_loss:1.9532 val_bpb:1.1568 eval_time:32961ms
Serialized model: 106023671 bytes
Code size: 99082 bytes
Serialized model int6+lzma: 14754232 bytes
Total submission size int6+lzma: 14853314 bytes
final_int6_roundtrip val_loss:1.9685 val_bpb:1.1659 eval_time:63637ms
final_int6_roundtrip_exact val_loss:1.96850574 val_bpb:1.16585998
FINISHED (Thu Mar 26 15:18:57 UTC 2026)