item 22 pre-registered: E2-L rung B internalization ladder (front-first deletion, 10 pauses/step, warm-started d=1..3); trainer gains --drop-steps/--warm-start/--steps/--lr; smoke-tested 2 steps on Spark

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

results-loop/PROTOCOL_UNIFIED.md

@@ -525,3 +525,33 @@ much reduced vs answer-only carry (83->72-83 vs 83->45). Ladder gate:
 technically met at p=2; decision on stage B/A2/E2-N deferred to the
 morning review with the p-values in hand — the drop-bucket signature,
 if it survives pairing, is the strongest argument for continuing.
+
+22. **E2-L rung B: internalization ladder, front-first step deletion
+    (pre-registered 2026-07-16 ~10:40, before running; Nils chose stage B
+    over A2/E2-N at the morning review. Gate state: pre-registered +3
+    overall met on points (+4.3 at p=2); paired McNemar overall ns, but
+    the drop-bucket signature significant, 2:6 p=0.0094).** Design:
+    delete the first d scratchpad lines of each verified terse CoT
+    (d=1,2,3 — front-first: the deleted computation must ride the
+    pause-chain before the visible remainder), each deleted step replaced
+    by 10 pauses (median step = 10 tokens, compute-matched); unparseable
+    cots (14/427 without exactly one Answer line) pass through intact.
+    Step-count distribution 1/2/3/4/5+: 11/164/196/27/15 — so d=3 is
+    effectively rung C (pauses only) for ~87% of items. Each rung
+    warm-starts from the previous (d=1 from rung-A e400), brief retrain:
+    200 steps, LR 3e-4 cosine, seed 0. Eval: GSM test n=256, cells 0:0
+    (k=0 sanity, expect ~base 10.9), 2:(2+10d), 2:(6+10d); e200
+    checkpoints; per-item logs kept so rung-vs-rung McNemar is offline.
+    Known approximation, stated in advance: items with fewer than d
+    steps train at smaller effective p than the eval cell
+    (ndel=min(d,n_steps)). Predictions: (a) d=1 best cell within 5
+    points of rung A's 57.4 — one step fits the recurrence budget (the
+    drop-bucket reach evidence says the whiteboard already carries
+    step-sized computation); (b) monotone decline across d; (c) at d=3
+    accuracy stays above BOTH base (10.9) and cold answer-only carry
+    (9.4) — curriculum beats cold training even where the ladder breaks.
+    Deliverable: the break rung = first d whose best cell falls >=5
+    points below the previous rung's best — the measured capacity of
+    this recurrence budget to absorb computation. Job:
+    scripts/jobs/zzz_m_gsm_rungb.sh (single submit, ~3x(40min train +
+    eval) on the Spark).

scripts/jobs/zzz_m_gsm_rungb.sh

@@ -0,0 +1,13 @@
+# gpuq-in:  results-loop/star_data.json results-loop/gsm_cot_data.json results-loop/adapter_carrycot_e400.pt
+# gpuq-out: results-loop/eval_gsm_carrycot_b*.json results-loop/train_carrycot_b*_log.json results-loop/adapter_carrycot_b*.pt
+git pull origin main -q 2>/dev/null
+P=/home/nils/jspace/.venv/bin/python
+export JLENS_MODEL=google/gemma-4-E2B-it LOOP_OUT=/home/nils/jspace/results-loop
+cd /home/nils/jspace/scripts
+WARM=$LOOP_OUT/adapter_carrycot_e400.pt
+for D in 1 2 3; do
+  $P train_carry_cot.py --drop-steps $D --warm-start $WARM --steps 200 --lr 3e-4
+  WARM=$LOOP_OUT/adapter_carrycot_b${D}_e200.pt
+  $P eval_carry_cot.py --adapter $WARM --tag gsm_carrycot_b$D \
+     --grid 0:0,2:$((2+10*D)),2:$((6+10*D)) --n 256
+done

scripts/train_carry_cot.py

@@ -43,12 +43,35 @@ ap.add_argument("--lensnoise", default=None, metavar="RANK,SCALE",
                      "training, shaped by the top-RANK sensitivity "
                      "directions of jbar at the band entrance; noise norm "
                      "= SCALE * per-position state norm (e.g. 32,0.05)")
+ap.add_argument("--drop-steps", type=int, default=0, metavar="D",
+                help="E2-L rung B: delete the first D scratchpad steps, "
+                     "each replaced by --pause-per-step extra pauses")
+ap.add_argument("--pause-per-step", type=int, default=10,
+                help="pauses per deleted step (median step = 10 tokens)")
+ap.add_argument("--warm-start", default=None, metavar="ADAPTER_PT")
+ap.add_argument("--steps", type=int, default=STEPS)
+ap.add_argument("--lr", type=float, default=LR)
 ARGS = ap.parse_args()
+STEPS, LR = ARGS.steps, ARGS.lr
 TAG = ("carrycot_ff" if ARGS.feedforward else "carrycot") + (
+    f"_b{ARGS.drop_steps}" if ARGS.drop_steps else "") + (
     f"_ln{ARGS.lensnoise.replace(',', '_')}" if ARGS.lensnoise else "") + (
     f"_s{ARGS.seed}" if ARGS.seed else "")
 
 
+def drop_cot_steps(cot, d):
+    """Delete the first d scratchpad lines (front-first: the deleted
+    computation must ride the pause-chain before the visible remainder).
+    Returns (new_cot, n_deleted); unparseable cots pass through intact."""
+    lines = [l for l in cot.split("\n") if l.strip()]
+    ans = [l for l in lines if l.startswith("Answer")]
+    steps = [l for l in lines if not l.startswith("Answer")]
+    if len(ans) != 1 or not steps:
+        return cot, 0
+    n = min(d, len(steps))
+    return "\n".join(steps[n:] + ans), n
+
+
 class LensNoiseWrapper(torch.nn.Module):
     """Perturb the carried state s (not the anchor e) before the merge,
     within the span of the lens's top-r readout-sensitive directions.
@@ -88,8 +111,9 @@ def build_batch(tok, items, p, device="cuda"):
                  add_special_tokens=False)["input_ids"]
         a = tok(it["cot"] + "<end_of_turn>",
                 add_special_tokens=False)["input_ids"]
-        seqs.append(pr + [PAUSE_ID] * p + a)
+        pi = p + it.get("extra_pauses", 0)
-        labs.append([-100] * (len(pr) + p) + a)
+        seqs.append(pr + [PAUSE_ID] * pi + a)
+        labs.append([-100] * (len(pr) + pi) + a)
         plens.append(len(pr))
     T = max(len(s) for s in seqs)
     pad = tok.pad_token_id or 0
@@ -125,12 +149,21 @@ def main():
             if it["split"] == "train"}
     cots = json.load(open(OUT / "gsm_cot_data.json"))
     data = []
+    n_dropped = 0
     for r in cots:
         it = star.get(r["idx"])
         if it is None:
             continue
+        cot, ndel = (drop_cot_steps(r["cot"], ARGS.drop_steps)
+                     if ARGS.drop_steps else (r["cot"], 0))
+        n_dropped += ndel
         data.append({"question": it["question"], "label": r["label"],
-                     "cot": r["cot"]})
+                     "cot": cot,
+                     "extra_pauses": ndel * ARGS.pause_per_step})
+    if ARGS.drop_steps:
+        print(f"rung B d={ARGS.drop_steps}: {n_dropped} steps deleted "
+              f"across {len(data)} items "
+              f"({ARGS.pause_per_step} pauses each)", flush=True)
 
     model, tok = load_model(dtype=torch.bfloat16)
     for pp in model.parameters():
@@ -147,11 +180,16 @@ def main():
                                    float(sc)).cuda()
         print(f"lens-noise: rank={r} scale={sc} on jbar L{BAND[0]}",
               flush=True)
+    if ARGS.warm_start:
+        (adapter.base if ARGS.lensnoise else adapter).load_state_dict(
+            torch.load(ARGS.warm_start, map_location="cuda"))
+        print(f"warm-started from {ARGS.warm_start}", flush=True)
     opt = torch.optim.AdamW(adapter.parameters(), lr=LR, weight_decay=0.01)
 
     keep = [it for it in data
             if len(tok(it["question"])["input_ids"])
-            + len(tok(it["cot"])["input_ids"]) + 16 <= 460]
+            + len(tok(it["cot"])["input_ids"])
+            + it.get("extra_pauses", 0) + 16 <= 460]
     rng.shuffle(keep)
     pool = {l: [it for it in keep if it["label"] == l]
             for l in ("easy", "hard")}