nils/jspace
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
jspace/results-loop/PROTOCOL_UNIFIED.md
T

69 KiB
Raw Permalink Blame History

Pre-registered protocol: unified adapter eval (written before any test numbers)

Date: 2026-07-13, after val@499, before step-799 completion. No test-set number for the unified adapter exists at time of writing.

  1. Primary loop depth: k=2, for both tasks. Chosen on val CE with the tie-breaking rule: prefer the SMALLEST k whose hard-cell val CE is within 0.01 nats of the best k. (Current val: MBPP hard k2−k4 = 0.004, GSM hard k2−k4 = 0.006 → both ties → k=2.) The full k-curve is secondary/descriptive.

  2. Checkpoint selection criterion (scalar, fixed now): mean of the two hard-cell val CEs at k=2, tasks weighted equally: crit = (gsm_hard_k2 + mbpp_hard_k2)/2. Lowest crit among saved checkpoints wins. (At writing: step 499, crit = (0.475+0.205)/2 = 0.340.)

  3. Primary endpoints: (a) MBPP test pass@1 hard-bucket at k=2 vs k=0; (b) GSM8K test accuracy hard-bucket at k=2 vs k=0. McNemar, paired by item. Overall accuracy is secondary (known to be underpowered at n=250/256).

  4. Same-harness rule: all k, INCLUDING k=0 baselines, measured by the prompt-only fast-path scripts (generate_frozen_prompt; k=0 = plain cached generate inside the same function). No numbers carried over from the full-position-loop harness.

  5. Known missing control (not covered by this run): a same-size, no-recurrence adapter (h -> h + MLP(h) at the L13->L14 boundary, no loop, no band re-run) trained on identical data/objective. Until it exists, "the loop does the work (vs. 1.6M new weights anywhere doing it)" is NOT established. Queued as the next training run. Note the k=0 column is gated off by construction and is a sanity check only — it is not this control.

  6. Symmetric interference check (missing): dedicated GSM8K prompt-only adapter as the reference for "unified costs GSM nothing". Queued. Until then the no-interference claim is one-directional (MBPP side only).

  7. Band-location ablation (pre-registered 2026-07-13, before any arm ran). Arms, all else identical (adapter size/init, data, curriculum, k, scripts; MBPP): early L2-12, mid-narrow L17-27, late L24-34 (width-matched, 11 layers); shifted L6-22 (width-matched to the original 17). Reference: workspace L14-30 (already run, 3 seeds). Prediction: workspace-centered arms (L14-30, L17-27) exceed early/late on hard-bucket pass@1 at k=2-4 by a wide margin; shifted intermediate. Falsification: near-parity across arms demotes the lens claim from "locates where to loop" to "convenient discovery tool"; to be reported either way. Primary readout: hard-bucket pass@1 at k=4, e400 checkpoints throughout.

  8. Language commitments for the writeup: the k0->k1 CE collapse (e.g. 4.36->0.18) is format/template learning expected from any trained adapter and must not be quoted as evidence of routing/planning; informative comparisons are within k>=1 cells only. Depth ordering k2 vs k4 deltas (0.001-0.006 nats) are inside checkpoint jitter and must be described as "k>=2 fits hard items equally well; k=1 slightly worse."

  9. Anchor/entrance sweep (pre-registered 2026-07-14 ~03:00, before any arm ran). Arms: bands (13,30), (12,30), (11,30) — injection point shifted up from L14 at fixed tap L30; plus tap-23 = (14,23). All E2B/MBPP, same recipe, e400, primary readout hard-bucket pass@1 at k=4. Competing predictions: (a) "L14 special" (last full-attention KV-computing layer, lens boundary) → anchor-13 drops; (b) "KV-channel count" (anchors 11-13 add 1-3 extra KV-recomputing attention channels) → holds or improves. Body-length confound noted: earlier anchors lengthen the loop body; if results shift, run matched-length control (12,28) before interpreting.

  10. L9 discriminator arm (pre-registered 2026-07-14 ~10:00, before running). Band (9,30): anchor at L9 — the only other full-attention, KV-computing layer below the boundary — deep in the lens's sensor regime, tap fixed at L30. Separates the two cliff explanations: (a) "lens boundary" predicts catastrophic (like anchors 11-13: 25-34% overall); (b) "full-attention KV-layer entry" predicts partial recovery (clearly above the L11-13 trend, i.e. >40% overall or hard >25%). Registered prediction: (a) — the sensor-region content dominates; layer type does not rescue it. Same recipe/checkpoint/eval as the anchor sweep (250 items, ks 0,2,4).

Outcomes vs pre-registrations (scored 2026-07-14, after all arms completed)

  1. k=2 primary depth — held. All primary comparisons reported at k=2; k-curves descriptive. k≥2 plateau confirmed (k=8 gen-eval flat).

  2. Checkpoint criterion — applied as written for the unified adapter. Separately reported: val-CE is a poor proxy for generation accuracy; later arms therefore pre-committed to fixed steps (e400) instead.

  3. Primary endpoints (unified adapter, k=2 vs k=0) — (a) MBPP hard 3.6% → 28.6% (direction as predicted); (b) GSM hard 0% → 6.3%, overall 10.5% → 9.0% (no overall win — the math boundary result). Both reported.

  4. Same-harness rule — held throughout (all final tables fast-path, k=0 included).

  5. Missing weights control — run: trained FF adapter = 17.9% hard, exactly the untrained-loop level. Loop-vs-weights gap established.

  6. Symmetric interference check — run (dedicated GSM adapter); mixed-task training regressed both tasks; reported as negative result.

  7. Band-location ablation — prediction CONFIRMED with a caveat: L14-30 hard 43.6% ≫ early L2-12 (23.6%, overall destroyed 22.8%) and shifted L6-22 (21.8%, overall 29.0%). Caveat discovered: L17-27 and L24-34 are structurally null (KV sharing; k>0 ≡ k=0 bit-identical), so the "mid-narrow beats late" half of the prediction was untestable at E2B; the 12B replication (no shared KV) carries that weight instead.

  8. Language commitments — honored in PAPER.md (k0→k1 CE collapse not cited as planning evidence; k2-vs-k4 nats described as jitter).

  9. Anchor/entrance sweep — prediction (a) "L14 special" CONFIRMED: anchor-13 hard 17.9%/overall 34.4%; 12: 28.6%/30.8%; 11: 25.0%/25.2%; monotone collapse below the boundary. Tap-23 arm died in training (never rerun); exits 27/30/32/34 within seed noise, so exit choice is free. Matched-length control not needed (results did not shift with body length in the informative direction).

  10. L9 discriminator — registered prediction (a) CONFIRMED: band (9,30) overall 14.0-21.4%, hard ≤21.4% — catastrophic, like anchors 11-13, despite L9 being a full-attention KV-computing layer. The lens boundary, not layer type, gates the retrofit.

  11. Recurrent-regime arm (pre-registered 2026-07-15, before training). Huginn-style retrofit on the frozen E2B band: RecurrentAdapter (learned A,B init α·I/(1−α)·I + zero-init MLP), h0 = norm-scaled noise, log-uniform random depth k∈[1,16], bptt=4, same data/steps/ checkpoint rule (e400 primary) as all merge arms. Eval ks 0,2,4,8,16,32 on the 250-item MBPP set. Competing predictions: (a) "amortization is intrinsic to frozen-band retrofits" → performance plateaus by k≈4 at or below the merge arm's level, no depth-monotone gain; (b) "fixed- point behavior was an artifact of our fixed-shallow-k training" (Huginn regime transfers) → monotone hard-bucket improvement past k=8 and reduced noise-seed sensitivity after training. Secondary readout: path independence (two noise seeds → output agreement rate) at e400. Known risk, stated in advance: 600 steps may be far too little for this regime (McLeish et al. use ~50B tokens); a null here bounds the cheap-retrofit budget only, not the regime.

  12. Parcae-constrained recurrent arm (pre-registered 2026-07-15, before training; Prairie et al. 2026 parameterization). Same as item 11 but A = exp(−Δt·exp(a)) diagonal → ρ(A) < 1 by construction; init exactly the α=0.3 merge (verified bit-equal at init). ρ(A) logged every 10 steps in BOTH arms. Theory-derived predictions, stated in advance: (a) contraction ⇒ fixed point is a function of e ⇒ the Parcae arm SATURATES in k (no depth-monotone gain) and its converged performance is amortizable — if so, our deflationary result is a corollary of ρ<1, and our observed k≈3–4 convergence is the geometric rate 0.3^k; (b) the UNCONSTRAINED item-11 arm either drifts toward ρ≥1 (watch the ρ log: divergent runs should show ρ≥1 before loss spikes) or, if it gains monotone depth-performance, does so with ρ near 1 — the edge of stability is where genuine iteration must live. Either outcome formalizes "the anchor coefficient is the stability dial" as "the anchor coefficient is the spectral radius".

  13. Per-depth adapter arm + free-ACT probe (pre-registered 2026-07-15, before training). (a) PerDepthAdapter: one merge adapter per iteration (n=4, Bae-style depth-wise relaxation at the entrance; breaks time-invariance — LTV, no fixed-point guarantee), standard curriculum, e400, eval ks 0,2,4,8. Prediction: lands at or below the distill/rung-2 amortization ceiling (~46% hard) because depth-indexed weights add content, not state-evolution; exceeding it would show per-iteration expressivity was binding and amend the deflationary claim. Depths >4 reuse adapter 4 (stated: k=8 cell is then fixed-point-like by construction). (b) Free-ACT probe on the standard merge arm: record per-item convergence depth (cos>0.9995) at k=8 cap. Predictions: accuracy unchanged vs fixed k (post-convergence no-ops); mean k_conv ≈ 3; hard-labeled items converge SLOWER than easy ones (adaptive compute allocates like ACT without any learned halting parameter).

--- Outcome, item 11 (scored 2026-07-15, k=16/32 cells cancelled by decision after k<=8): PREDICTION (a) SUBSTANTIALLY CONFIRMED, with one twist. The unconstrained arm left contraction immediately (rho(A): 0.3 -> 3.4 by step 100, plateau ~4.5) yet trained smoothly — per-iteration norm-matching converts magnitude explosion into directional churn, so "rho>=1 => divergence" becomes "rho>=1 => divergence OR stationary churn" under a norm projection. Consequences as predicted: substrate damage (easy 98.4 -> ~69% at all k>0, far exceeding any contractive arm's tax), val CE flat k=1..16 (stationary, not progressive), hard bucket at merge level (35.7/39.3/42.9% at k=2/4/8 — a one-item-per-depth-doubling crawl that at k=8 reaches what the contractive merge reaches at k=4, never approaching the amortization ceiling from above). 4x parameters bought nothing. Depth-monotone computation did not emerge at this budget.

  1. Tied-alpha arm (pre-registered 2026-07-15, before training). TiedAlphaAdapter: x = (1−a)⊙e + a⊙ŝ + MLP([e;ŝ]), a = σ(â) per-dim learned, init a=0.3 everywhere (bit-equal to MergeAdapter at step 0, verified). B tied to (1−a): convex combination keeps the LTI fixed point on the e–ŝ segment (substrate-anchored by construction), ρ = max(a) < 1 guaranteed, +d≈1.5K params. Standard curriculum, s0 = band(e), e400, eval ks 0,2,4,8 on 250 items. This is the one untested cell combining parcae's learnable decay with the merge's anchoring. Predictions: (a) substrate fidelity preserved (easy ≈ merge's 88%, unlike both rec arms' ~70%) because anchoring, not ρ, controls fidelity; (b) hard-bucket at merge level (no significant gain — per-dim constant α is not where capability lives, per the adaptive-α E2B result); (c) learned a drifts slightly DOWN from 0.3 (as in parcae). If (a) holds while rec arms failed it, the fixed-point-location dial is causally isolated: same learnable-decay freedom, only the tie to (1−a) differs from parcae.

--- Outcome, item 12 (scored 2026-07-15): prediction (a) CONFIRMED in its dynamics half, REFUTED in its fidelity half — and the refutation is the finding. Dynamics: rho stayed in (0,1) throughout (0.300 -> 0.292, the optimizer drifting MORE contractive when confined to the stable region); loss trajectory as good as or better than the unconstrained arm at every checkpoint (the rec arm's flight to rho~4.5 was epiphenomenal — all fit lives in the MLP); eval saturates completely (hard 42.9/42.9/39.3/39.3/ 39.3 at k=2/4/8/16/32, easy flat ~71%). Fidelity: easy items were NOT preserved (71% vs the merge's 88.5%) despite guaranteed contraction — substrate fidelity is controlled by fixed-point LOCATION (anchored B + curriculum), not by rho. Conclusion: stability and fidelity are independent dials (fig_phase.png); the Parcae constraint delivers exactly what it promises (robust training, convergence, certified tail gradients) and exactly nothing more. Item 14 (tied-alpha) is the causal isolation of the fidelity dial.

  1. Fidelity factorial + capacity control + seed (pre-registered 2026-07-15 ~03:15, before any of these arms ran; overnight batch). The fidelity loss of both rec arms (easy 88.5 -> ~71%) confounds three deltas from the winning merge: (i) learned B, (ii) random-depth training instead of the difficulty->depth curriculum, (iii) noise s0. Item 14 (tied-alpha) tests (i) with anchoring. New single-variable cells, everything else = standard merge recipe (fixed B, band(e) s0, curriculum, e400, eval ks 0,2,4,8 on 250 items): a. merge+randk — only (ii) changed (log-uniform k in [1,16], bptt 4). b. merge+noises0 — only (iii) changed. c. merge h=2048 — capacity control for the per-depth arm (6.4M shared vs 6.4M depth-indexed): if per-depth beats the ceiling but h2048 does not, time-variation (not capacity) is credited; if both do, it was capacity all along. d. parcae seed 1 — robustness of the fidelity refutation. Predictions: (a) and (b) each cost a few points of easy at most (anchored fixed point dominates); neither reproduces the ~17-point drop — the culprit is the learned/free B (with item 14 as the positive control). h2048 stays at the ceiling (hard <=46%), fidelity intact. parcae s1 reproduces easy ~71% within seed noise.

--- Outcome, item 13a (scored 2026-07-15): prediction CONFIRMED — per-depth lands below/at the ceiling, never above. Detail is instructive: fidelity preserved throughout (easy 88.5/89.3/86.9 at k=2/4/8 — anchored B), but hard-bucket content is DEPTH-STRANDED: 17.9% at k=2 (adapters 3-4, which hold the hard-trained content, never execute), 35.7% at k=4, 42.9% at k=8 — where depths 5-8 reuse adapter 4, i.e. the architecture reverts to shared-map iteration and the fixed-point mechanism collects the remaining gain. Time-variation adds a fragility (content unavailable except at its training depth) and no capability; map-sharing is load-bearing for the anytime-usable gain. Depth-4 adapter overfit visible in val (hard k4 CE 0.188@99 -> 0.371@599) — LTV concentrates small-pool overfitting into single depths.

--- Outcome, item 13b (scored 2026-07-15): accuracy prediction CONFIRMED (k=8 halt run 52.0/90.2/42.9 = plateau level); convergence predictions REFUTED. Per-item state-cosine (thresh 0.9995, k=8 cap): k_conv distribution 4:3, 5:57, 6:47, 7:17, never-within-8:126 — mean 7, and NO difficulty gradient (easy 7.01 vs hard 7.00). The earlier "bit-exact by k3-4" was the single dynamics-probe example, not the population: outputs plateau by k~2-4 while the state keeps drifting at 1e-3..1e-4 cosine scale; the fixed point is an OUTPUT-stable orbit (suffix layers + decode wash out residual state motion), not a literal state fixed point for most prompts. Free-ACT via state-cosine therefore yields no early exit at this threshold, and no ACT-like difficulty allocation falls out for free — output-level halting signals would be needed. Paper's dynamics claims softened accordingly.

--- Outcome, item 14 (scored 2026-07-15): ALL THREE PREDICTIONS CONFIRMED. (a) Fidelity fully preserved: easy 93.4/91.0/90.2 at k=2/4/8 (merge: 92.6/88.5; parcae with identical decay freedom but untied B: ~71%) — the free B is causally isolated as the fidelity culprit, the anchoring tie as the protection. (b) Hard at merge level exactly (35.7/42.9/39.3 = merge's k-curve within noise); no gain from the freedom. (c) Learned a essentially unmoved: mean 0.298, range [0.285, 0.310], 0/1536 dims moved

0.05 from init — the anchor coefficient is not a useful learnable DOF; hand-tuned 0.3 was already optimal. Recipe consequence: fixed-alpha anchored merge is the recommended design; learnable-alpha safe but pointless, learnable-B harmful, per-depth strands the gain.

  1. Code→GSM8K cross-task transfer (pre-registered 2026-07-15 ~14:10, before running). The MBPP-trained loop adapter (adapter_code, s0) and the noise-s0 variant evaluated on GSM8K test (n=256, prompt-only loop, same harness as eval_gsmonly). Extends the transfer-distance ladder (HumanEval tie -> LCB trained-hurts) across tasks. Predictions: (a) hard-bucket gain ~0 (plan content is task-local; GSM8K needs evolving state, not static plans); (b) easy items damaged at k>0 (~93 -> 50-70%), comparable to or worse than the GSM-trained merge — substrate damage on GSM8K is perturbation-driven and content-agnostic; (c) overall at k>0 below k=0 (no rescue). If instead hard gains appear (>5 points), plan-shaped content is partially task-general — would weaken the task-local claim from LCB.

--- Amendment to item 15 (2026-07-15 ~13:15): noise-s0 arm EXCEEDED prediction (b) upward: hard 50.0/53.6/50.0 at k=2/4/8 with easy 88-90% — nominally the best hard cells of the project (merge best 46.4; seed mean 37.5±5.5). Paired vs tied-alpha (only same-day per-item baseline): discordants 5-1/3-0/3-0 in noise-s0's favor, each k p≈0.22-0.25 at n=28 — consistent direction, not individually significant. Denoising interpretation: training the loop to reach the fixed point from noise regularizes the content. SEED ARMS QUEUED (s1, s2, same recipe/eval, pre-registered here): if seed-mean hard(k=4) > 46.4 (the merge's best single cell), the recommended recipe gains noise-s0; if seed mean falls back into 37-46, it was a lucky seed.

--- Outcome, item 15c (h2048 capacity control, scored 2026-07-15): the per-depth exoneration is CLEAN — shared 6.4M params reach hard 42.9/53.6/ 50.0 at k=2/4/8 vs per-depth's 17.9/35.7/42.9 at the same capacity; time-variation is strictly worse than weight-sharing at matched params. Fidelity prediction confirmed and exceeded (easy 95.1% at k=2 — best looped fidelity of the project; 90.2% at k=4/8). Ceiling prediction (hard <= 46%) REFUTED UPWARD like noise-s0: k=4/8 at 53.6/50.0. Two independent variations (noise s0, 4x MLP) now sit at 50-54% where the original merge reached 46.4 — suggests 46.4 was an UNDER-estimate of the recipe family's level, not a ceiling it defined. The distill-parity deflation claim is unaffected statistically (53.6 vs 45.7 at hard n=28 is within noise) but the language "every regime tops out at the same ceiling" should become "at the same level within noise" — pending the noise-s0 seed arms.

--- Outcome, item 15d (parcae seed 1, scored 2026-07-15): CONFIRMED — the fidelity refutation replicates. easy 70.5/73.0/72.1 at k=2/4/8 (seed 0: 72.1/71.3/70.5); hard 32.1/39.3/35.7 (seed 0: 42.9/42.9/39.3, ordinary seed spread at n=28). Two-seed conclusion: contraction-with- free-B loses ~17 points of easy items regardless of seed; the phase diagram's Parcae point is solid.

--- Outcome, item 16 (code->GSM8K transfer, scored 2026-07-15): ALL THREE PREDICTIONS CONFIRMED, emphatically. MBPP-trained loop on GSM8K: hard 0.8-1.6% at every k (prediction a: ~0 gain — plan content is task-local); easy 93.1 -> 27.6-44.8% (prediction b: damaged, in fact WORSE than the GSM-trained merge's 48%); overall strictly below k=0 at every k>0 (prediction c). noise-s0 variant identical (easy 34.5, hard 1.6). The transfer-distance ladder ends cleanly: near (HumanEval) tie, far-code (LCB) trained-hurts, cross-task (GSM8K) trained-content actively toxic while gaining nothing. Task-locality of the learned content is now a three-point monotone result.

--- Closure of the item-15b/15c "ceiling nudged upward" question (2026-07-15, after ns seeds): LUCKY SEED, per the pre-registered rule. noise-s0 hard(k=4) across seeds: 53.6 / 39.3 / 35.7 -> seed mean 42.9, inside the 37-46 band. Fidelity across seeds intact (easy 90.2-94.3 — the factorial conclusion is seed-robust); the 50-54% cells (ns seed 0, h2048 single seed) were upper-tail draws of the same distribution the merge's 46.4 came from. No recipe amendment; the abstract's original "same level within noise" framing stands; single-cell records are not levels — only seed means are.

  1. GSM-only, current recipe (pre-registered 2026-07-15 ~20:45, before running). train_merge_unified.py --tasks gsm: MergeAdapter, prompt- only loop, curriculum, GSM8K data ONLY — removes the mixed-task interference confound from the adapter_uni run, completing the "winning recipe trained on GSM" question. Eval: prompt-only, n=256, ks 0,1,2,4, e400. Predictions: (a) hard <= 10% at every k (supervision density is structural: ~3 answer tokens; the recipe's dense-output ingredient cannot exist here); (b) easy damaged at k>0 (to 40-70%); (c) overall never beats k=0. If hard exceeds 15% or overall beats k=0, task interference in the mixed run was masking a real GSM capability — would reopen the GSM chapter.

    Scope note (item 17): the design-space arms of items 11-15 are NOT crossed with GSM8K, deliberately. Exclusion by dominance: fidelity- failing regimes (rec, parcae) cannot improve on a task MORE fidelity- fragile than MBPP; architecture-failing (per-depth) and equivalent (tied-alpha -> merge) and k-placement-only (randk) and same-family (noise-s0, h2048) variants have no mechanism by which task change could invert their MBPP verdict. Only the recipe family's best member (this item) is informative on GSM8K.

--- Outcome, item 17 (GSM-only, current recipe, scored 2026-07-15): predictions (a) and (b) CONFIRMED, (c) nominally exceeded but not meaningfully. hard 8.7/5.5/4.7% at k=1/2/4 (below the 10% bar; nowhere near the 15% reopen threshold); easy 93.1 -> 48-52% at k>0; overall 11.7/10.9/10.2 vs k0's 10.5 — the k=1 cell is +1.2 points nominal (~3 items at n=256, not significant), the rest below. Removing the mixed-task interference bought ~2 points over adapter_uni (9.4 -> 11.7 at k=1) — interference was real but marginal, not masking a capability. The GSM8K chapter is closed: the recipe family's best member, trained on GSM alone in the correct regime, delivers no usable gain and the standard fidelity damage; combined with the scope note, the boundary claim (structural: supervision density + state-evolution bottleneck) is fully supported.

  1. E1: learned per-prompt halting gate (pre-registered 2026-07-16 ~00:20, before any arm runs; PLAN_SELFPACED.md). HaltingMergeAdapter: frozen-recipe merge + ACT-style halting head on the last prompt position's workspace state; soft state-mixture training, CE + lambda * E[iters], penalty warmup at step 100; NO difficulty curriculum (mixed batches — the gate must discover the allocation). k_max=4, e400/e600 checkpoints, deploy = sequential halting at 0.5 cumulative mass, generation via frozen-prompt at per-item k*. Arms: lambda in {0, 1e-3, 1e-2}, seed 0. Eval: 250 items, vs anchors k=0 (0.488), uniform merge k=4 (0.512/0.885/0.464), probe-gate E0 (0.520/0.975/0.286). Predictions: (a) some lambda gives overall >= 0.512 at mean E[k] <= 2.4 (60% of uniform-4); (b) easy >= 0.95 at that lambda; (c) k*-vs-hard point-biserial r > 0.3; (d) hard >= 0.286 (beats E0's frozen probe). Collapse (E[k] pinned at 1 or 4 for all lambda) falsifies E1 and triggers the plan's kill criterion. lambda=0 control isolates whether the CE gradient alone moves the gate (expected: barely — penalty provides the pressure).

    Item 18 amendment (2026-07-16 ~23:45, before results): arms run on a rented 4xH100 node in parallel instead of the Spark queue; a fourth arm (lambda=1e-3, seed 1) is added for immediate seed replication of the expected-winner penalty. Spark's queued gate jobs will be dropped to avoid duplication. Everything else per registration.

--- Outcome, item 18 (scored 2026-07-16 ~00:40): predictions (b), (c) REFUTED, (a) marginal miss, (d) trivial pass. All arms converge to UNIFORM depth (lambda 0/1e-3/1e-2 -> E[k] 4/2-or-4/1; the two 1e-3 seeds picked different plateaus — degenerate penalty landscape), r = 0.000 everywhere. Mechanism identified and consistent with prior findings: teacher-forced CE is depth-flat (stationarity), so CE provides no per-item depth gradient; the penalty alone cannot teach selectivity. The state DOES carry the signal (E0 probe: train acc 1.0) — the failure is the training signal, not the representation. E1-as-designed is dead; kill criterion NOT fully triggered (E2 untested, and the mechanism points at a repair).

  1. E1b: label-supervised halting head (pre-registered 2026-07-16 ~00:45, before running). Freeze the curriculum merge (adapter_code s0); train ONLY the halting head (BCE): target halt=0 at iterations below the label's depth (easy->1, hard->4, per STaR label), halt=1 at or above it. 300 steps, mixed batches, head-only params. Eval: gated eval as item 18, n=250. Predictions: (a) r(k*, hard) > 0.5 (the head is a trained difficulty classifier now); (b) easy >= 95% at k*=1 (near-E0's 97.5); (c) hard >= 35.7% (>= best uniform arm, via better recall than E0's frozen probe: more than 18/28 hard items routed deep); (d) overall >= 52.0 at E[k] <= 2.2. If (c) fails while (a,b) hold, halting-head recall saturates at probe level and gate quality, not gate training, is the binding constraint.

--- Outcome, item 19 / E1b (scored 2026-07-16 ~01:15): prediction (c) CONFIRMED (hard 39.3 >= 35.7 at mean k* 2.18), (a) FAILED at r=0.217 (selectivity real — hard routed 2x deeper than easy (2.18 vs 1.08), the program's first nonzero gate correlation — but weak at deploy), (b,d) FAILED for a traced design reason: halted_k_per_item lacked k*=0, so easy items were forced through >=1 iteration and landed on the merge's WORST easy depth (k=1: 85.2%); E0's 97.5% came precisely from k=0 routing. E1c amendment (pre-registered before running, same session): pre-loop halt consult on s_0 enabling k*=0; targets easy->0, hard->4; threshold 0.5 unchanged (calibration deferred unless E1c misses). Predictions: easy >= 95%, hard >= 35.7%, r >= 0.4, overall >= 51.2 at E[k] <= 1.5.

--- Outcome, item 19 / E1c (scored 2026-07-16 ~01:50, Spark re-run): prediction (b) CONFIRMED — easy 95.9% with mean k*=0.11 (the k=0 routing fix worked perfectly for easy items); (a) FAILED (r=0.220, unchanged); (c) FAILED HARD — hard 21.4% at mean k*=1.5: with the pre-loop halt consult, weakly-discriminated hard items now exit at k*=0/1 where before they got >=1 iteration; the recall problem became more expensive, not better. (d) FAILED (overall 50.0 at E[k]=0.74). Net Pareto: E1c = (50.0 overall, 95.9 easy, 21.4 hard, 0.74 mean iters — 82% compute saved); E0 probe-gate = (52.0, 97.5, 28.6, ~2.2) still dominates on accuracy. The learned-head line has ONE identified un-tried knob: deploy-threshold calibration on val for hard-recall (the 0.5 threshold is arbitrary; raising it routes more items deep, trading easy tax for hard recall — a tunable curve E0 cannot offer). E1 arc summary for PLAN_SELFPACED: gating machinery works end-to-end, easy-item protection and compute savings are demonstrated and cheap; difficulty-selective DEPTH allocation remains unsolved at 3K-param-head scale — binding constraint is classifier quality on the k=0/s0 state, exactly where E0 started. Next per plan: threshold sweep (cheap) before any E2.

  1. E1 threshold curve + oracle bound (pre-registered 2026-07-16 ~02:15, before running). Phase 1: record E1c head's halt probabilities per test item (one GPU pass). Phase 2: per-item outcomes for the frozen curriculum merge at k=0/1/2/4 (four generation sweeps, tag merge_lut — doubles as the reusable gate-evaluation lookup table and supplies the long-missing per-item logs for the canonical merge). Phase 3 (offline): gated accuracy at thresholds .3-.99 by composing k*(theta) with the lookup; plus the ORACLE gate (best k per item) = the ceiling any gate can reach with this merge. Predictions: (a) some theta gives hard >= 32% with easy >= 93% and E[k] <= 2.2 (dominating E0 on compute at comparable accuracy); (b) the curve is monotone in theta; (c) oracle overall >= 55% — if so, gate-quality headroom is large and further gate work is justified; if oracle < 53%, gating this merge is nearly saturated and the program pivots to E2 or closes.

--- Outcome, item 20 (scored 2026-07-16 ~02:35): (b) CONFIRMED — clean monotone threshold curve (hard 7->50%, easy 96.7->87.7%, E[k] 0.43->2.63 across theta .3->.99). (a) FAILED — no theta reaches easy>=93 AND hard>=32; at matched easy the E0 frozen probe dominates the entire learned-head curve: the BCE-trained 3K head is strictly worse than the class-balanced logistic probe it was meant to replace. (c) CONFIRMED, emphatically: ORACLE gate = 59.6 overall / easy 100% / hard 64.3% at E[k]=0.24. Key insight: hard items are DEPTH-DIVERSE — 18/28 solvable at some k in {0,1,2,4} but no single k solves more than 13; a third of the hard bucket lives in per-item depth selection. Program continues per rule; binding constraint quantified: gate quality is worth ~9.6 overall points (50.0 deployed vs 59.6 oracle). Also noted: the LUT re-run of the canonical merge shows small systematic drift vs the Jul-13 eval (k4 hard 46.4 identical, k1/k2 hard 3 items lower) — the LUT (per-item, single harness run) is now the canonical reference. Next candidates, in cost order: (i) deploy E0's probe AS the gate against the LUT (free, offline); (ii) stronger classifier (multi-position features, more data, calibrated threshold); (iii) oracle-gap error analysis on the hard items no fixed k solves but some k does.

  1. E2 stage A: dense short-CoT supervision through the carry whiteboard, GSM8K (pre-registered 2026-07-16 ~02:55, before running; PLAN_SELFPACED E2 / the hybrid from the internalization discussion). Prep: harvest TERSE verified CoTs ("at most 3 short steps", answer- verified, STaR filter) for GSM train. Arms: (A) carry regime (k=2 prefill, pauses easy p=2 / hard p=6) trained with CE on scratchpad+answer (~30-60 dense tokens — the ingredient every latent GSM arm lacked); (B) CONTROL: identical supervision, feedforward adapter, no recurrence. Eval: GSM test 256, grid 0:0 (base), 2:2, 2:6; e400 checkpoints. Predictions: (a) arm A beats every previous GSM arm's overall (>12.1%) — dense supervision is the binding fix; (b) the A-vs-B delta isolates the whiteboard: if A > B by >=3 points overall, recurrence adds value beyond visible-scratchpad training; if A ~= B, the scratchpad text alone carries it (deflation, GSM edition); (c) easy-bucket damage smaller than answer-only carry's (83->45%) because training and deployment output formats now match. Honest note: arm outputs are VISIBLE tokens (~40) — this is the budget-CoT-with-loop hybrid, a scope change from latent planning, run at Nils's explicit direction ("do gsm8k and such").

--- Outcome, item 21 (scored 2026-07-16 ~06:30). Harvest: 427 verified terse CoTs (292 hard, 135 easy; 73.5% yield, ~3 min). Grid (n=256, base 10.9/93.1/0.8/0.0 overall/easy/hard/drop): arm A carry 2:2 53.9 (82.8 / 60.6 / 37.0) arm A carry 2:6 57.4 (72.4 / 63.8 / 45.0) control FF 2:2 49.6 (79.3 / 59.8 / 28.0) control FF 2:6 54.7 (82.8 / 66.9 / 31.0) (a) CONFIRMED, dramatically: best cell 57.4% vs the previous best GSM arm's 12.1% — a 5x jump to full-CoT territory (~53%) at ~1/4 the visible tokens. Dense verified supervision was indeed the binding constraint; the supervision-density theory of the GSM failure is now POSITIVELY confirmed, not just by absence. (b) MIXED: A-vs-B delta +4.3 at p=2 (clears the >=3 bar), +2.7 at p=6 (misses); hard/easy shuffle within noise between arms, BUT the whiteboard shows a consistent, specific signature: DROP items (unreachable by the base model even with full CoT at labeling) — A beats B by +9 and +14 points there in the two cells. Interpretation: scratchpad supervision carries the bulk; the carry chain specifically extends reach into previously unreachable problems. McNemar (scored 2026-07-16 morning): overall A-vs-B not significant in either cell (2:2 discordants 34-23 p=0.185; 2:6 33-26 p=0.435), but the pre-identified drop-bucket signature IS: 2:6 drop discordants 20-6, exact McNemar p=0.0094 (survives Bonferroni x4 = 0.038); 2:2 drop 16-7, p=0.093 (same direction, marginal). Verdict: the whiteboard's edge is real and specific to extending reach into drop items, not a general lift over matched supervision. (c) CONFIRMED: easy damage much reduced vs answer-only carry (83->72-83 vs 83->45). Ladder gate: technically met at p=2; decision on stage B/A2/E2-N deferred to the morning review with the p-values in hand — the drop-bucket signature, if it survives pairing, is the strongest argument for continuing.

  1. E2-L rung B: internalization ladder, front-first step deletion (pre-registered 2026-07-16 ~10:40, before running; Nils chose stage B over A2/E2-N at the morning review. Gate state: pre-registered +3 overall met on points (+4.3 at p=2); paired McNemar overall ns, but the drop-bucket signature significant, 2:6 p=0.0094). Design: delete the first d scratchpad lines of each verified terse CoT (d=1,2,3 — front-first: the deleted computation must ride the pause-chain before the visible remainder), each deleted step replaced by 10 pauses (median step = 10 tokens, compute-matched); unparseable cots (14/427 without exactly one Answer line) pass through intact. Step-count distribution 1/2/3/4/5+: 11/164/196/27/15 — so d=3 is effectively rung C (pauses only) for ~87% of items. Each rung warm-starts from the previous (d=1 from rung-A e400), brief retrain: 200 steps, LR 3e-4 cosine, seed 0. Eval: GSM test n=256, cells 0:0 (k=0 sanity, expect ~base 10.9), 2:(2+10d), 2:(6+10d); e200 checkpoints; per-item logs kept so rung-vs-rung McNemar is offline. Known approximation, stated in advance: items with fewer than d steps train at smaller effective p than the eval cell (ndel=min(d,n_steps)). Predictions: (a) d=1 best cell within 5 points of rung A's 57.4 — one step fits the recurrence budget (the drop-bucket reach evidence says the whiteboard already carries step-sized computation); (b) monotone decline across d; (c) at d=3 accuracy stays above BOTH base (10.9) and cold answer-only carry (9.4) — curriculum beats cold training even where the ladder breaks. Deliverable: the break rung = first d whose best cell falls >=5 points below the previous rung's best — the measured capacity of this recurrence budget to absorb computation. Job: scripts/jobs/zzz_m_gsm_rungb.sh (single submit, ~3x(40min train + eval) on the Spark).

--- Outcome, item 22 (scored 2026-07-16 ~15:30). k=0 sanity row reproduced base exactly in all three rung evals (10.9/93.1/0.8/0.0). Ladder (best cell overall, n=256): rung A 57.4 -> d=1 31.6 (2:12) -> d=2 18.4 (2:26) -> d=3 19.1 (2:32). (a) FAILED, decisively: the break rung is d=1 — deleting ONE compute-matched step costs 25.8 points (easy 83->62, hard 64->36, drop 45->17); the recurrence cannot absorb even one step's computation at this budget/recipe. (b) monotone through d=2, then a plateau (d=2 18.4 -> d=3 19.1, within n=256 noise): the decay bottoms out ABOVE the floor rather than collapsing to it. (c) CONFIRMED: d=3 (pause-only for ~87% of items) = 19.1 vs base 10.9 — paired McNemar 33-12 discordants, p=0.0025 — and vs cold answer-only carry 9.4: the curriculum-reached latent loop DOUBLES the cold-trained equivalent. Composition of the d=3 edge: easy is DAMAGED (58.6 vs 93.1 base) while hard (18.9 vs 0.8) and drop (8.0 vs 0.0) are lifted — the latent loop trades easy-bucket reliability for reach, echoing item 21's drop-bucket signature in latent form. Deliverable: measured capacity of this recurrence budget = a plateau at ~19% overall / ~2x the cold floor; the visible scratchpad carried the other ~38 points. Val-loss note: hard val rose with d (0.36 -> 0.56 -> 0.59) while easy fell (d=3 0.13, mostly answer-line targets) — CE fit does not track eval accuracy at deep rungs. Interpretation for the plan: rung C' (no pauses) is moot as a capability claim — C already plateaued at 19; the E2-L line's value is now the 2x-over-cold curriculum effect + the reach trade, not scratchpad-free parity. Next knobs if the line continues: longer per-rung retraining, finer deletion schedule (fractional rungs), or E2-N noise-hardening stacked on the rungs.

  1. E2-L d=1 capacity-ceiling controls (pre-registered 2026-07-16 ~15:55, before running; Nils: "run that one control"). Item 22's break-at-d=1 has two untested confounds; one arm each, single-knob changes from item 22's d=1 (both warm-start rung-A e400, seed 0): arm A "x600" = 600 steps instead of 200 (3x training; tests the brief-retrain objection — note d=1 val had plateaued at 0.36, so prediction is NO recovery); arm B "pp30" = 30 pauses per deleted step instead of 10 (3x latent bandwidth, same 200 steps; tests whether inert-pause capacity, not training, binds). Eval n=256: arm A cells 0:0, 2:12, 2:16 (e600); arm B cells 0:0, 2:32, 2:36 (e200). Decision rule, stated in advance: item 22 d=1 best = 31.6; a control within +-5 points confirms the ceiling on that axis;

    =+5 (>36.6) reopens the ladder on that knob (longer schedules or wider pause-chains); if BOTH land within +-5, the d=1 ceiling is confirmed structural and the ladder chapter closes as scored. Job: scripts/jobs/zzz_n_rungb_ctrl.sh. AMENDMENT (2026-07-16 ~19:05, mid-run, Nils's call): arm A's eval skipped to save ~50min GPU — the verdict was already decided by training evidence (train loss memorized to 0.02-0.10 by step 200; val easy 0.386->0.397->0.426, hard 0.357->0.376->0.421 across e200/e400/e600 — monotone UP, textbook overfit, no recovery; k=0 sanity row reproduced base before the kill). Arm A scored from the val trajectory: training time is NOT the binding constraint. Arm B (pp30) runs in full as registered.

--- Outcome, item 23 (scored 2026-07-16 ~23:55; rc=0). k=0 sanity rows reproduced base in both arms' evals (arm A's before its eval was skipped). Arm A (x600): scored from val trajectory per amendment — no recovery, mild overfit; training-time axis CONFIRMS the ceiling. Arm B (pp30): best cell 29.3 (2:36; 28.5 at 2:32) vs item-22 d=1's 31.6 — WITHIN the +-5 band, slightly below: 3x latent positions bought nothing (hard 33.9/36.2 vs 36.2; drop 14 vs 17; easy 51.7-55.2 vs 62.1). Decision rule: BOTH axes confirm -> the d=1 break is STRUCTURAL. Neither longer training nor a longer pause-chain lets this carry absorb one scratchpad step; the binding constraint is the carried state itself (its per-position expressivity/fidelity, not its compute budget). Converging evidence from the same evening's microscopy (probe_discount*/probe_gsm*): the board natively carries plans ("Multiply", "subtract"), coarse magnitudes ("sixty/eighty"), and completion-state ("plus" kept alive at the 430 divergence — carry defers where FF commits a wrong digit at 99.3%), while exact digits appear only just-in-time, 1-2 positions pre-emission. The ladder chapter closes as scored in item 22; the loop program's live paths are state-side (rung-2 band-LoRA / wider merge, E2-N hardening, coarse-target auxiliary supervision matched to the medium) or the hybrid (A2), not longer/denser pause-chains.

  1. E2-L d=1 with a trainable band: loop-only band-LoRA (pre-registered 2026-07-17 ~00:30, before running; Nils: "can we unfreeze the entire band and try this again?"). Item 23 located the constraint in the carried state's per-position transformation; this is the state-side attack. Full unfreeze rejected in design (642M params vs 427 examples, and it would break the frozen-model guarantee); instead LoopLoRA (lora_band.py, built for the rung-2 design): rank-16 deltas on q/v/down of EVERY band layer (L14-30, uniform scale 1.0, 4.8M params), active ONLY during band re-runs — initial forward and k=0 stay bit-exact by construction. Otherwise identical to item-22 d=1: front-first deletion, 10 pauses/step, warm-start rung-A e400 (adapter lr 3e-4; fresh LoRA lr 1e-3, B zero-init so step-0 matches item 22 exactly), 200 steps, seed 0. Eval n=256: 0:0 (sanity, must equal base), 2:12, 2:16; e200. Decision rule (same bands as item 23, vs d=1's 31.6): >=+5 (>36.6) = band expressivity was binding -> escalate (rank 64, all projections, or full-band rung ladder); within +-5 = the medium verdict stands even with a trainable band -> internalization line CLOSED, program pivots (A2 / E2-N / coarse-aux / divergence batch). Prediction, honest: mild gain but under threshold (~33-36) — the microscopy says the board's failure mode is WHAT it carries, not how flexibly it transforms it; but this is the experiment the 'wider state' hypothesis deserves before the line closes. Job: scripts/jobs/zzz_p_rungb_blr.sh.

--- Outcome, item 24 (closed 2026-07-17 ~02:10, STOPPED by Nils mid-eval: "stop that idea. does not seem to work either"). Training completed; evidence at stop: val easy 0.346 / hard 0.405 vs adapter-only 0.356/0.361 — the trainable band did not improve fit (hard slightly worse), consistent with the registered prediction that capacity/expressivity is not what binds. k=0 sanity row with trained LoRA loaded reproduced base EXACTLY (10.9/93.1/0.8) — the loop-only LoopLoRA guarantee holds in practice; the mechanism is validated even though the hypothesis died (useful for any future band-side work). Accuracy cells not measured (eval killed at Nils's call to save ~50min GPU); adapter+lora e200 checkpoints in the bucket if anyone ever wants the number. Verdict: with items 22+23 this closes the E2-L internalization line entirely — adapter capacity, training time, pause bandwidth, and band expressivity have all now failed to move the d=1 break. The carried state's native cargo (plans, magnitudes, completion-state — see the probe series) is the program's remaining asset; next candidates: divergence batch replay, coarse-target auxiliary supervision, A2, E2-N.

  1. E2-L d=1 with latent process supervision through the lens (pre-registered 2026-07-17 ~02:45, before running; Nils's idea: "for training, i wonder if we could calculate, using jspace lens, how each iteration should think"). Items 22-24 all trained the latent chain blind — output CE only — and all failed; this changes the INFORMATION reaching the chain, not its capacity. New loss: the lens readout softmax(W_U·finalnorm(J̄_L30·h)) is differentiable in h, so at the 10 replacement pauses we apply lens-CE against the DELETED step's tokens, aligned 1:1 (pause j <-> step token j, truncated at 10) — the board is trained to write the deleted step in lens-readable code at the time it would have been written. Mixed loss CE_out + λ·CE_lens. Two arms, single submit: λ=0.3 and λ=1.0. Otherwise identical to item-22 d=1 (front-first deletion, warm-start rung-A e400, adapter-only 3e-4, 200 steps, seed 0; no band-LoRA — one knob). Smoke: step-0 lce=10.3 (~uniform: pauses currently encode nothing about the step; large fresh gradient). Eval n=256: 0:0, 2:12, 2:16 per arm. Decision vs d=1's 31.6, same bands: >36.6 = latent supervision was the missing ingredient -> ladder REOPENS with lens-taught rungs (and the 2D per-iteration variant becomes item 26); within +-5 = even telling the board exactly what to write doesn't make the carry compute it -> the strongest closure evidence yet. Caveats pre-stated: J̄ is prompt-averaged (global directions); the loss forces a verbalizable code (microscopy suggests that IS the board's working code, but a native non-verbal code would be fought); the 1:1 temporal alignment is one choice among several (bag-of-tokens, result-digits-only are untested alternatives if this null's). Job: scripts/jobs/zzz_q_rungb_lt.sh. IN-FLIGHT NOTE (~03:20): λ=0.3 arm's lens-CE collapsed 10.3 -> 2.3 by step 40 — world 1 (cannot encode) is eliminated before the eval; the open question is world 2 (writes but doesn't compute) vs world 3 (reopens the ladder).

--- Outcome, item 25 (closed 2026-07-17 ~01:25, TRUNCATED by Nils: "cancel that 2:16 and then try the new approach"). λ=0.3 arm: lens-CE converged to ~1.9 (the pause-chain reliably writes the deleted step in lens code, top-few over 262k vocab) while eval 2:12 = 31.2 (drop .18 / easy .586 / hard .354) vs the 31.6 reference — FLAT, not even a redistribution across buckets. k=0 sanity clean. World 2 confirmed on the measured cell: WRITING IS NOT COMPUTING — the carried state can be taught to verbalizably display the missing step at the right timesteps and the displayed content is not consumed downstream. 2:16 and the λ=1.0 arm cancelled mid-run (λ=0.3's flat cell made λ=1.0 uninformative; ~2.5h GPU saved). Program value: the cleanest representation/computation dissociation in the series; the read-side (what the suffix computation consumes from the board) is now the prime suspect, and the pause-tape architecture allows display-without-routing. Item 27 (zero-pause internal looping, no tape to display on) runs next by Nils's direction; item 26 requeued after it (job renamed zzz_t).

  1. Result-staging supervision during generation (pre-registered 2026-07-17 ~03:25, before running; Nils: "can we try to train the loop using jspace during generation?"). The microscopy's signature of the healthy board: each line's RESULT is on the carried state before it is emitted ("1/twelve" at the '=', P=0.997). This trains that signature directly. For every visible scratchpad line, at the pre-'=' positions (the result is NOT yet in the causal context there — nothing can be copied; low loss requires computation) lens-CE the L30 carried state against the line's result tokens; Answer-line analog via the ':' separator. Spans computed in token space (gen_staging_targets; 1102 spans on the d=1-transformed 427). Two arms, one submit, both otherwise item-22 d=1 (warm-start rung-A e400, 200 steps, adapter-only): (a) lg=0.3 alone — staging supervision without pause supervision; (b) lt=0.3 + lg=0.3 — the full lens curriculum (deleted step taught at the pauses, results staged during emission). Eval n=256: 0:0, 2:12, 2:16 per arm; decision vs 31.6, same bands as items 23-25. Distinct from item 25's question: 25 teaches WHAT the silent chain should hold; 26 teaches the compute-before-emit discipline on the visible chain (the behavior that separated carry from FF at the 430 divergence). Caveat: on-policy ("during generation" in the free-running sense) is NOT this — that's a rollout-based follow-up (latent DAgger) if either lens arm moves. Job: scripts/jobs/zzz_r_rungb_lg.sh.

--- Outcome, item 26 (scored 2026-07-17 ~10:50; ran last, requeued as zzz_t after items 27/28 per Nils's mid-night reprioritization). k=0 sanity clean both arms. Arm a (staging-only, lg03): 24.2 / 23.4 — the series' first ACTIVELY NEGATIVE result, ~7 points below the 31.6 reference, outside the band on the downside (easy 44.8-48.3, hard ~26-28, drop 12-15). Arm b (combined lt03+lg03): 15.6 / 14.1 — compounding destruction (easy 27.6, hard 18-21, drop 5-6), worse than either constraint alone. Reading: the microscopy showed the healthy board computes results 1-2 positions pre-emission (just-in-time); the staging loss demanded the result across the ENTIRE pre-'=' span, including positions where operands haven't been read — forcing thought earlier than the computation can exist corrupts the schedule rather than accelerating it. Lesson for any future lens supervision: match the loss's temporal demand to the board's measured schedule (supervise at the '=' +- 1 position only), don't blanket-force. Overnight program complete: items 25-28 all scored; write-side exhausted; the standing decision (clamp test vs pivot to hybrid/A2) is unchanged by this result.

  1. Internal band looping, zero pause tokens (pre-registered 2026-07-17 ~01:10, before running; Nils's clarified structure, option A confirmed via question: one silent burst before output). Architecture: NO pause tokens anywhere — sequence is prompt + visible output only. After the k=2 prompt settle, the last prompt position runs M=10 IN-PLACE band iterations (s^i seeds from the position's own previous band output — carry_common inplace updates), then generation proceeds with the normal single-pass carry under the visible tokens. Vertical thought burst instead of a horizontal pause tape: iterations leave no KV entries — only the final state survives, a true internal loop (the C'-flavored architecture the ladder never tested; every prior rung bought compute with positions). Supervision: lens-CE iteration i <-> deleted-step token i (λ=0.3, the stable value), same d=1 deletion, warm-start rung-A e400, 200 steps, seed 0. Eval n=256: 0:0 (sanity), 2:0 with --inner-iters 10 (matched), 2:0 with --inner-iters 0 (ablation: does the burst matter at inference?). References: positional d=1 31.6 (same ±5 / >36.6 bands) and the ii0 ablation cell (isolates the burst's causal contribution). Honest priors: item 25's 2:12 just landed at 31.2 (flat), so the write-vs-compute dissociation likely carries over; the architectural deltas that could matter here: no attention-tape (forces state-borne computation rather than KV re-reading) and anchor-at-prompt (iterations see the full settled question). Job: scripts/jobs/zzz_s_rungb_ii.sh. IN-FLIGHT NOTE (~02:20): lce 9.4 -> 2.9 by step 60 — the tape-free burst also encodes the step trajectory; encoding is never the obstacle. Accuracy pending.

--- Outcome, item 27 (scored 2026-07-17 ~04:35). k=0 sanity clean. Matched cell 2:0 (burst on) = 34.0 (drop .19 / easy .621 / hard .394) — nominally the BEST d=1 number of the series (vs 31.6 positional, 31.2 lens-pauses) with ZERO pause tokens; ablation (same adapter, burst off) = 32.4. But both pre-registered comparisons are noise-compatible when paired: burst-on vs burst-off 10-6 discordants, p=0.45; vs positional d=1 44-38, p=0.58. Verdict: within the +-5 band — the ceiling holds on the internal-iteration axis too; the burst's causal contribution is directionally positive (all of it in the hard bucket, +3.2) but unproven at n=256. Notable anyway: the zero-pause FORMAT loses nothing (ablation 32.4 >= positional 31.6 with 12 fewer latent positions) — the pause tape was contributing ~nothing, consistent with item 25's display-without-routing reading. Item 28 (teacher-state targets, same architecture) runs next and completes the 2x2 (verbal/full-state x tape/no-tape).

  1. Teacher-state distillation into the burst (pre-registered 2026-07-17 ~02:30, before running; Nils's variant: "meaningful CoT chunks yield internal state that we then teacher-force into the loop-only model"). Items 25/27 supervise the VERBAL SHADOW of the deleted step (token targets through the lens); this forces the FULL state. Teacher = the frozen warm-start (rung-A e400) adapter running the complete cot (deleted step visible), zero pauses, same carry architecture; capture its band-exit state at the deleted step's LAST token — "the state of having finished thinking the step" — one no-grad pass over the 413 parseable items at startup (targets fixed, no moving teacher). Student: identical zero-pause M=10 burst as item 27, but the loss is cosine distance between the burst's FINAL iterate s^10 and the teacher state, plus output CE. Weight AMENDED pre-run λ=1.0 -> 5.0 after the smoke measured the starting cosine distance at only 0.113 (nearby band-exit states share most structure); at 1.0 the term would be ~10x weaker than the CE and a null would be underpowered — 5x0.113 puts the two terms at comparable initial scale. No lens loss (one knob vs item 27: full-state targets instead of verbal-shadow targets; trajectory supervision dropped — only the endpoint is forced). Known approximation, stated in advance: teacher state is captured ~10 positions later in the sequence than the student anchor (RoPE position coloring); cosine + the carry machinery's routine state transplantation across nearby positions make this tolerable, but a null could partly hide here. Eval n=256: 0:0, 2:0 ii10 (matched), 2:0 ii0 (ablation). Decision vs 31.6, same bands; the informative three-way is 25 (verbal, tape) vs 27 (verbal, no tape) vs 28 (full state, no tape) — if 28 moves where 25/27 don't, the computation-carrying content lives OUTSIDE the verbalizable subspace; if none move, the read-side clamp test (item 29 candidate) decides. Job: scripts/jobs/zzz_sa_rungb_ts.sh.

--- Outcome, item 28 (scored 2026-07-17 ~06:20). k=0 sanity clean. Distillation SUCCEEDED geometrically: cos-dist 0.113 -> 0.044 (burst endpoint lands ~60% closer to the teacher state), with the lowest output-CE tax of any supervised arm (vals 0.384/0.376). Function did NOT follow: matched 2:0 = 30.1, ablation (burst off) = 29.7 — burst contribution nil; hard 34.6 indistinguishable from the pack; and the state-forcing DAMAGED easy (48.3 vs 62.1 in every other d=1 arm — the forced endpoint displaces something easy handling needed). Verdict: null, below reference. The 2x2 (items 25/27/28 + 22 as the unsupervised corner) is complete and uniform: no state-side supervision — verbal shadow or full teacher state, tape or burst — makes the loop compute the deleted step. Combined with items 22-24 (capacity/time/positions/expressivity all null), the write-side is exhausted end to end. The one untested link is READ-side: whether the suffix computation consumes the workspace at all when the right state is present — the clamp test (write the teacher state into the anchor at inference via the lens/embedding-basis machinery, no training, and measure) is item 29's natural content and would assign blame causally. Nils's morning decision: clamp test vs pivot to the hybrid/A2 line (where the 57.4 and the drop-bucket reach already live).

  1. Trajectory teacher-forcing of the burst (pre-registered 2026-07-17 ~11:50, before running; Nils's design: "measure the CoT run's L30 at 10 evenly spaced intervals and teacher force them into the loop — we know both the previous that goes into the merge layer and the next L30 state"). The key upgrade over item 28: the teacher trajectory decomposes the burst's learning into TEN INDEPENDENT SUPERVISED TRANSITIONS — iteration i receives teacher waypoint T[i-1] as its merge input and its band output is pulled onto T[i] (cosine, λ=5.0) — dense gradient at every iteration, no long BPTT chain, no compounding; and T[0] == the student's own settled anchor state exactly (shared prompt+settle), so the chain starts aligned by construction. Exposure gap handled by a second arm adding the free-running loss (burst's own s_i onto T[i], λ=5.0). The free-running burst ALWAYS runs last and seeds the answer scan (training matches inference); TF transitions are scaffolding erased from context afterward. Teacher = frozen warm-start adapter on the full cot, 10 evenly spaced L30 states. Two jobs, per Nils (both chosen): JOB 1 (zzz_u) d=1, waypoints across the deleted step, arms tf-only and tf+fr; reference 31.6, same bands; eval 0:0 / 2:0 ii10 / 2:0 ii0 per arm. JOB 2 (zzz_v) THE HEADLINE: d=all (answer-only output, rung C), waypoints across the FULL CoT — the burst as a 10-waypoint compressed rehearsal of the entire reasoning; references: cold answer-only 9.4, curriculum plateau 19.1, base 10.9; same arms and cells. Smokes: ltf/lfr baseline 0.116 both modes; d=99 deletes 1118 steps across 427. Priors, stated: the transition-decomposition is the first mechanism that trains the loop's DYNAMICS rather than its endpoints — if the loop can compute at all, this is its best chance; if job 2 beats 19.1 meaningfully, compressed-trajectory rehearsal becomes the internalization method. All prior caveats (position coloring, teacher=warm-start quality) carry over. Jobs: scripts/jobs/zzz_u_traj_d1.sh, scripts/jobs/zzz_v_traj_full.sh. --- Outcome, item 29 COMPLETE (scored 2026-07-17 ~17:30; job-1 tf+fr ablation and job-2 remainder cancelled by Nils after the informative cells). JOB 1 (d=1): tf-only 39.1 matched / 39.1 ablated — the program's first significant positive (p=0.045 paired vs 31.6), gain attributed to the TRAINING SIGNAL (transition regressions improve the merge's state-folding under visible tokens), burst inert at inference (8-8, p=1.0). tf+fr arm 30.5 — the free-running term HURTS: clean teacher-forced transition gradients are the active ingredient; constraining the burst's own states distorts them. JOB 2 (answer-only, full-CoT waypoints): tf-only 15.6 (easy .586 / hard .094 / drop .11) — BELOW the 19.1 curriculum plateau, above the 9.4 cold floor. Compressed-rehearsal internalization FAILS, and the mechanism reading predicts exactly this: the trajectory-TF gain acts through visible- token carry steps, which answer-only mode lacks. Net: trajectory teacher-forcing enters the recipe book as an ADAPTER TRAINING SIGNAL for hybrid (visible-scratchpad) regimes; it is not an internalization mechanism. Follow-ups on file: seeds for the 39.1; trajectory TF on rung A (nothing deleted, move the 57.4).

    IN-FLIGHT, arm 1 (tf-only, d=1) scored ~15:40: THE FIRST SIGNIFICANT POSITIVE OF THE PROGRAM, WITH A MECHANISM TWIST. Vals best-in-series (easy 0.218, hard 0.347). Matched 2:0 = 39.1 (drop .25 / easy .69 / hard .433 — every bucket a d=1 record; easy IMPROVED) — clears the pre-registered >36.6 threshold; paired vs positional d=1: 50-31, McNemar p=0.045. BUT the ablation also scores 39.1 (burst-on vs burst-off 8-8, p=1.0): the test-time burst is causally INERT. Attribution: trajectory teacher-forcing is a superior TRAINING SIGNAL for the merge adapter — the ten transition regressions teach state-folding that pays off at every visible-token carry step — not a working inference-time loop. The program-long pattern holds: the loop at inference has never been causal; the loop as training scaffold just produced the largest adapter gain ever measured. ltf plateaued ~0.085 (from 0.116) — the autonomous operator absorbs ~25% of the driven transitions. Next-experiment implication (Nils to confirm): apply trajectory TF to rung A itself (nothing deleted) — if the signal generalizes, the 57.4 headline moves.

  2. The carried state as a metacognitive signal (pre-registered 2026-07-17 ~13:20, before running; Nils's design: "treat the carry state as a metacognitive signal and build on it explicitly... exploit what the loop provably does instead of fighting the consumption wall. This never requires the frozen band to read anything new"). Synthesis of three established results: the carry's inference-time value is deferral/completion-state (430 divergence; drop McNemar p=0.0094); gate quality is worth ~9.6 points with CLASSIFIER quality binding on the pre-loop state (item 20); the consumption wall blocks residual-injection reads (items 25-28) but a NEW head can read anything, and its decisions reach the frozen model as tokens. Design: answer-readiness head g(s) — LayerNorm+MLP(64), class-balanced — on the carried L30 state at scratchpad line boundaries of rung-A rollouts. Labels mechanical: fork a forced "Answer: " completion at each boundary, check against gold. Harvests: train (427) and test (256) x {carry, feedforward-control} — the FF arm trains the identical head on the FF states. Readouts, all offline from one instrumented pass (item-20 LUT methodology): (a) PRIMARY: carry-head test AUC vs FF-head test AUC — if carry > FF, the recurrence carries metacognitive signal the feedforward path lacks: the first measured POSITIVE FUNCTIONAL ROLE for the carried state; (b) theta sweep of "answer at first boundary with g>=theta else natural end": accuracy vs mean scratchpad length vs the 57.4 fixed-format baseline; (c) ORACLE stop bound (best boundary per item) = the ceiling adaptive stopping can reach. Predictions: carry AUC > 0.65 and > FF AUC (the deferral evidence says the signal exists); theta curve dominates fixed-format on tokens at matched accuracy; oracle meaningfully above 57.4 (early-stop rescues drift cases). Arm 2 (later): line-correctness / deferral head gating re-expansion. Job: scripts/jobs/zzz_w_metacog.sh.

--- Outcome, item 30 (scored 2026-07-17 ~19:40; rc=0). Harvests: 4319/1058 carry, 3408/879 FF boundary rows; readiness rates 33-41%. (a) PRIMARY REFUTED on specificity, CONFIRMED on existence: answer-readiness is STRONGLY decodable from the whiteboard — test AUC 0.798 — but the FF control decodes it equally (0.791): the metacognitive signal is whiteboard-general, NOT recurrence-specific. The carry's drop-bucket edge does not manifest as a linearly-more- readable readiness code at line boundaries. (b) Theta sweep: early stopping LOSES at every theta <= 0.95 (0.441-0.520 vs natural-end 0.527); theta=0.99 reaches parity with 37/256 early answers. (c) ORACLE stop = 0.566 (+3.9 over natural end at mean boundary 3.58) — a weak ceiling vs item 20's +9.6 depth oracle: the model's natural halting already sits near its readiness frontier, and early-stop errors are cost-asymmetric. Verdict: the readiness HEAD works as instrumentation (0.8 AUC from a frozen state, zero human labels); early-stop DEPLOYMENT is not the lever. Live follow-ups: arm 2 — deferral/re-expansion gating (EXTENDING computation where the head is uncertain; its oracle is unmeasured and the 430 mechanism points there, not at stopping); and using g as a per-item difficulty signal for the item-20 depth gate (where the oracle headroom actually is).

  1. Synthetic memory tokens: per-layer KV prefix (pre-registered 2026-07-17 ~15:50, before running; Nils's design: "the one read mechanism a frozen transformer natively possesses is attention over the KV cache... adapter maps L30 state -> KV entries at band layers"; injection variant chosen by Nils: per-layer KV prefix). The read-side attack, composed with item 29: warm-start the arm-1 adapter (its burst COMPUTES — trajectory-trained, ltf 0.085 — but is provably unread: ablation 8-8 p=1.0) and FREEZE it; train only a KVMemoryAdapter (6.3M params: shared code-512 trunk + per-band- layer k/v heads with per-layer geometry — gemma-4 mixes 256-d sliding and 512-d global heads) mapping the 10 burst iterates to post-RoPE (k,v) columns appended to every band layer's attention during the answer scan and at generation. Engineering, validated in smokes: the registered attention implementation is wrapped IN PLACE (config name untouched -> mask construction identical; disarmed = bit-exact, diff 0.0); zero-init value heads + learnable per-layer gate bias (init -10) make the memory silent at init (step-0 loss 0.108 = the arm-1 endpoint exactly); memory stays armed through backward (checkpoint recompute). Output CE only, 200 steps, lr 1e-3. Eval: 0:0 sanity + 2:0 matched (burst + memory); the ablation is item 29's matched cell BY CONSTRUCTION (same frozen adapter and burst, memory absent) = 39.1. Decision:

    44.1 (>=+5 over 39.1) = the consumption wall was a READ-PATH problem and native attention over state-derived KV breaches it; within +-5 = even natively readable computed states go unused -> the wall is not about the read mechanism either, and the hybrid/ metacog lines carry the program. In-flight note on item 29 arm 2 (tf+fr): matched 30.5 — the free-running term HURT (vs 39.1 tf-only), reinforcing the training-signal attribution: clean TF transition gradients are the active ingredient. Job: scripts/jobs/zzz_x_kvmem.sh.

--- Outcome, item 31 (scored 2026-07-17 ~21:20; rc pending job end). k=0 sanity exact. Matched 2:0 = 39.5 vs the by-construction ablation 39.1 — FLAT (paired 6-5, p=1.0). Forensics: ALL 17 gates unmoved at -10.00 (+-0.01); value heads learned content behind the closed gate (weight norms 1.2-3.6) but the channel never opened. CONFOUND, recorded honestly: the silent-start init (-10) puts the gate gradient in a vanishing regime (both the memory's influence and the gate's gradient scale with e^-10 ~ 5e-5), so "optimizer declined the read path" and "optimizer couldn't reach it" are NOT separated by this run; a gate-init -3 rerun (~1.5h) would separate them. Also noted: output CE was near-saturated (~0.05-0.11) — little gradient pressure to explore any new channel; a headroom-bearing task would be a fairer test. Verdict as registered: within +-5 -> no evidence that a native attention read path breaches the consumption wall, with the init confound flagged as the one loose thread.

  1. The discrete latent chain — "latent paper" (pre-registered 2026-07-18 ~00:50, before running; Nils's synthesis: "the loop needs paper, but we don't want that to be full tokens but still latent space"). Diagnosis from the full matrix: every latent medium lacked DISCRETENESS — tokens' magic is the snap (error-correction per step), not visibility. The lens is a native codebook: argmax over its readout quantizes any band state onto the model's own symbol space. Design (each burst tick): read s_{i-1} through the frozen lens; snap to a token (straight-through over top-32, hard forward / soft gradient; tick 0 = newline "a step begins"); feed E(token) back through a ZERO-INIT projector alongside the analog carry: x_i = merge(e, s_{i-1}) + proj(E(sym)). Two rails, matching the microscopy's own two-channel algorithm: analog (plans/magnitudes)
    • discrete (exact symbols). Writing and computing coincide by construction: what the lens reads IS what gets transported — item 25's lce loss (reused, λ=0.3) is now load-bearing, and its proven concentration effect (10.3->1.9) doubles as the quantization pressure that collapses the diffuse thinking-state superposition (measured: pause states spread over ~hundreds of tokens, committed states 1-2). Merge FROZEN at item-29 arm-1 (39.1); only the 2.4M projector trains — the by-construction ablation is 39.1. Arms: (a) sctf — teacher-forced symbols (ground-truth deleted-step tokens; item-29's winning recipe); (b) scst — free-running straight-through snaps. Eval always hard-argmax free-running: 0:0 sanity + 2:0, n=256. Decision vs 39.1: >44.1 = discreteness was the missing paper property, the latent-chain program reopens (widenings pre-sketched: top-k parallel snaps, L22/L26 depth rails, tape via slots/kvmem); within +-5 = the discrete channel adds nothing over the analog carry and the "loop = plan machine, tokens = executor" division stands as final. Smokes: TF/ST train (loss arithmetic exact, warm-start vals intact), eval generates with hard snaps. Job: scripts/jobs/zzz_y_symchain.sh.

--- Outcome, item 32 (scored 2026-07-18 ~06:10). k=0 sanity clean both arms. BOTH ARMS BELOW REFERENCE — latent paper is net-harmful at this scale, with both failure modes mapped: (a) sctf (teacher-forced symbols): lens-CE concentrated to 1.6-2.0 (below item 25's endpoint, through the zero-init projector alone against a frozen merge) yet free-running eval = 20.3 (easy .345!) — EXPOSURE CATASTROPHE: a wrong snap is a hard corruption; a projector trained to trust symbols propagates it and poisons the analog rail (-18.8 vs ablation). (b) scst (straight-through self-exposure): lce plateaus 2.8-4.4 (never fully concentrates under its own snap noise), eval = 27.0 — recovers 7 of TF's 19 lost points exactly as scheduled-sampling theory predicts, but still -12.1 vs the pure analog 39.1. Conclusion: the snap's error-correction is real but its error-INJECTION dominates at 2B/10-tick scale — discreteness in the loop costs more than it corrects; visible tokens win not just by discreteness but by discreteness + a VERIFIED sampling distribution (the LM head was trained for token commitment; the lens readout was not). Per the pre-registered decision rule: the division stands as FINAL — the loop is a plan machine; tokens are the executor. The latent-paper widenings (top-k, depth rails, tape) are closed unless a soft-snap variant (mixture feedback, no hard argmax) is ever pre-registered. Item 32 closes the architecture tree: pauses, bursts, KV memory, analog TF chains, and discrete chains all have controlled answers.

Reference in New Issue View Git Blame Copy Permalink