item 24 pre-registered: d=1 retry with loop-only band-LoRA r=16 (4.8M params, whole band, k=0 bit-exact); trainer/eval gain --bandlora; smoke-tested train+eval

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 21:44:01 +02:00
parent 7a141948fa
commit fe89820e64
4 changed files with 83 additions and 3 deletions
@@ -630,3 +630,28 @@ chapter closes as scored in item 22; the loop program's live paths
 are state-side (rung-2 band-LoRA / wider merge, E2-N hardening,
 coarse-target auxiliary supervision matched to the medium) or the
 hybrid (A2), not longer/denser pause-chains.
+
+24. **E2-L d=1 with a trainable band: loop-only band-LoRA
+    (pre-registered 2026-07-17 ~00:30, before running; Nils: "can we
+    unfreeze the entire band and try this again?").** Item 23 located
+    the constraint in the carried state's per-position transformation;
+    this is the state-side attack. Full unfreeze rejected in design
+    (642M params vs 427 examples, and it would break the frozen-model
+    guarantee); instead LoopLoRA (lora_band.py, built for the rung-2
+    design): rank-16 deltas on q/v/down of EVERY band layer (L14-30,
+    uniform scale 1.0, 4.8M params), active ONLY during band re-runs —
+    initial forward and k=0 stay bit-exact by construction. Otherwise
+    identical to item-22 d=1: front-first deletion, 10 pauses/step,
+    warm-start rung-A e400 (adapter lr 3e-4; fresh LoRA lr 1e-3, B
+    zero-init so step-0 matches item 22 exactly), 200 steps, seed 0.
+    Eval n=256: 0:0 (sanity, must equal base), 2:12, 2:16; e200.
+    Decision rule (same bands as item 23, vs d=1's 31.6): >=+5
+    (>36.6) = band expressivity was binding -> escalate (rank 64, all
+    projections, or full-band rung ladder); within +-5 = the medium
+    verdict stands even with a trainable band -> internalization line
+    CLOSED, program pivots (A2 / E2-N / coarse-aux / divergence
+    batch). Prediction, honest: mild gain but under threshold (~33-36)
+    — the microscopy says the board's failure mode is WHAT it carries,
+    not how flexibly it transforms it; but this is the experiment the
+    'wider state' hypothesis deserves before the line closes. Job:
+    scripts/jobs/zzz_p_rungb_blr.sh.