item 22 pre-registered: E2-L rung B internalization ladder (front-first deletion, 10 pauses/step, warm-started d=1..3); trainer gains --drop-steps/--warm-start/--steps/--lr; smoke-tested 2 steps on Spark

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 10:13:01 +02:00
parent 272a7b1d1f
commit 20f7014f5b
3 changed files with 85 additions and 4 deletions
@@ -525,3 +525,33 @@ much reduced vs answer-only carry (83->72-83 vs 83->45). Ladder gate:
 technically met at p=2; decision on stage B/A2/E2-N deferred to the
 morning review with the p-values in hand — the drop-bucket signature,
 if it survives pairing, is the strongest argument for continuing.
+
+22. **E2-L rung B: internalization ladder, front-first step deletion
+    (pre-registered 2026-07-16 ~10:40, before running; Nils chose stage B
+    over A2/E2-N at the morning review. Gate state: pre-registered +3
+    overall met on points (+4.3 at p=2); paired McNemar overall ns, but
+    the drop-bucket signature significant, 2:6 p=0.0094).** Design:
+    delete the first d scratchpad lines of each verified terse CoT
+    (d=1,2,3 — front-first: the deleted computation must ride the
+    pause-chain before the visible remainder), each deleted step replaced
+    by 10 pauses (median step = 10 tokens, compute-matched); unparseable
+    cots (14/427 without exactly one Answer line) pass through intact.
+    Step-count distribution 1/2/3/4/5+: 11/164/196/27/15 — so d=3 is
+    effectively rung C (pauses only) for ~87% of items. Each rung
+    warm-starts from the previous (d=1 from rung-A e400), brief retrain:
+    200 steps, LR 3e-4 cosine, seed 0. Eval: GSM test n=256, cells 0:0
+    (k=0 sanity, expect ~base 10.9), 2:(2+10d), 2:(6+10d); e200
+    checkpoints; per-item logs kept so rung-vs-rung McNemar is offline.
+    Known approximation, stated in advance: items with fewer than d
+    steps train at smaller effective p than the eval cell
+    (ndel=min(d,n_steps)). Predictions: (a) d=1 best cell within 5
+    points of rung A's 57.4 — one step fits the recurrence budget (the
+    drop-bucket reach evidence says the whiteboard already carries
+    step-sized computation); (b) monotone decline across d; (c) at d=3
+    accuracy stays above BOTH base (10.9) and cold answer-only carry
+    (9.4) — curriculum beats cold training even where the ladder breaks.
+    Deliverable: the break rung = first d whose best cell falls >=5
+    points below the previous rung's best — the measured capacity of
+    this recurrence budget to absorb computation. Job:
+    scripts/jobs/zzz_m_gsm_rungb.sh (single submit, ~3x(40min train +
+    eval) on the Spark).