diff --git a/related_work/relevant_to_us.md b/related_work/relevant_to_us.md new file mode 100644 index 0000000..dd251d4 --- /dev/null +++ b/related_work/relevant_to_us.md @@ -0,0 +1,132 @@ +# Related work — relevance to the jspace looping project + +Running notes on papers in this folder: what they show, where they overlap +with our claims, and what remains ours. Update when a new PDF lands here. + +Our shorthand below: "our loop" = frozen gemma-4 base + 1.6M anchor-dominant +merge adapter at L14, band L14–30, prompt-only latent planning, STaR-labeled +difficulty→depth curriculum, frozen-prompt KV trick, attribution ladder +(untrained / FF / pause / loop / explicit plan), gate probe. + +--- + +## 2511.07384 — McLeish et al., "Teaching Pretrained Language Models to +## Think Deeper with Retrofitted Recurrence" (UMD/LLNL/Tübingen, Nov 2025) + +**What they do.** Convert pretrained 1B models (TinyLlama, OLMo-2, Llama-3.2) +into depth-recurrent models: remove middle layers, split the rest into +prelude → recurrent block → coda; recurrent input is a **linear adapter over +[e; s]** (prelude output ⊕ previous iterate; s₀ = random noise). Then full +continued pretraining: ~50B tokens math-heavy data, all parameters trained, +Muon optimizer, Poisson-Lognormal recurrence sampling with a curriculum +ramping mean recurrence to 32, truncated BPTT (last 8 iterations), plus a +"healing" phase (26B tokens FineWeb-Edu) to recover from layer surgery. +Result: at matched training FLOPs the retrofit beats continued-pretraining +the non-recurrent parent on GSM8K/MATH; accuracy scales with test-time +recurrence; pretrained init beats random init by ≥950B tokens of training. + +**Overlap — must cite, cannot claim as novel:** +- Retrofitting recurrence into a pretrained fixed-depth model works and + beats the non-recurrent baseline (their headline, at much larger scale). +- Merge adapter at loop entry combining embedding-side and state-side + inputs ([e; s] → linear, vs our (1−α)e + α·ŝ + MLP([e;ŝ])). +- Recurrence-depth curriculum during training. +- "Pretrain traditional, then convert" (our rung-2 vision): they did the + continued-pretraining version at 1B/50B tokens. The generic claim is theirs. + +**What remains ours (Paper A positioning):** +1. **Where to loop is interpretability-derived.** They pick splits by + benchmark search and name layer choice as an open problem ("future work + could identify a more optimal method for layer choice"). We derive the + band from the J-lens workspace regime and back it causally: anchor cliff + at L14, tap invariance, band-10–20 location ablation ≈ 0, KV-sharing + hazard. We answer their stated open problem. +2. **Frozen base, 1.6M-param adapter, ~600 steps on a desk machine** vs all + parameters, 50B tokens, MI300A cluster — ~5 orders of magnitude apart on + the cost curve. No layer removal → no healing phase needed, and k=0 + exactly recovers the base model (they cannot say that). +3. **Prompt-only latent planning + frozen-prompt KV trick: zero decode-time + cost.** Their recurrence runs on every generated token (decode cost ×r). +4. **Attribution controls** (untrained / FF / pause / explicit plan). They + have no compute-matched token-space control. +5. **Difficulty-adaptive depth** (STaR buckets, gate probe): named in their + Discussion as unsolved future work. Our gate is a first result on it. + +**Worth stealing:** +- **Muon > AdamW for recurrent training** (AdamW loss-spikes to NaN) — + directly relevant when we unfreeze the band / fade-off unfreezing. +- s₀ init: theirs is random noise, ours the actual residual state — likely + why a frozen base works for us at all; expect a reviewer question here. +- FLOPs accounting convention for recurrent models: FLOPs = (6N₁ + 2N₂)D + with N₁ = params with gradients, N₂ = forward-only (truncated BPTT). + +**Where to cite:** Paper A related work (primary contrast), Paper B +(layer-choice open problem → our mechanistic answer), rung-2 planning notes. + +--- + +## 2602.14759 — Lys et al., "Inner Loop Inference for Pretrained +## Transformers: Unlocking Latent Capabilities Without Training" +## (IMT Atlantique / Sony, Mar 2026) + +**What they do.** Training-free "middle looping" of frozen off-the-shelf +models (Gemma-2 2B/9B, Llama-3-8B): re-apply a block range [s, e) R times at +inference. Key findings: (1) **naive looping systematically degrades** +(distribution shift — looped activations leave the manifold the model was +trained on, some configs collapse to chance); (2) **regularized looping +rescues it**: interpolate the looped state with cached states / the baseline +state (uniform average, moving average ĥ = η·h⁽⁰⁾ + (1−η)·h⁽ᵗ⁾, +softmax auto-alignment) → modest but consistent gains across WinoGrande, +ARC, GSM8K, HellaSwag, MMLU (likelihood-scored, mostly multiple-choice). +Full start×end layer-pair sweep heatmaps. Frames looping as "logits +refinement" — depth as iterative refinement of a shared latent state. + +**Overlap — must cite:** +- Their moving-average regularization η·h⁽⁰⁾ + (1−η)·h⁽ᵗ⁾ is exactly the + anchor term of our merge, minus the trained MLP. Independent confirmation + that **anchoring to the un-looped state is the thing that makes frozen-band + looping viable** — cite as convergent evidence for the anchor-dominant + design (α=0.3). +- "Training-free looping gives modest gains on a frozen model" ≈ our + untrained-loop arm (17.9% hard vs 5.5 baseline on MBPP-hard, but = FF + control). Their whole paper lives inside one cell of our attribution table. +- Layer-pair sweeps parallel our location ablation (theirs benchmark-driven, + ours interpretability-predicted then confirmed). + +**What remains ours:** +1. **A trained merge** — their gains are modest by their own description; + our trained adapter roughly doubles the untrained/FF level on hard items + (17.9 → 37.4) and the pause/plan ladder bounds what the recurrence adds. +2. **Generative evals with per-item verification** (MBPP tests, GSM answer + match, Rust compile-run) vs their likelihood-scored multiple choice — + ours measures the regime where latent planning should matter. +3. **Prompt-only looping + frozen-prompt KV trick** (they loop everything, + every position; no decode-cost story). +4. Curriculum, difficulty gating, cross-task transfer arms, KV-sharing + hazard (their Gemma-2 has no KV sharing; our E2B finding that + interventions entered ≥L15 are structurally null is a hazard their + sweep methodology would silently hit on models that do share). + +**Worth stealing:** +- Their distribution-shift framing of *why* naive looping fails is a clean + citable explanation for why zero-init MLP + anchor is the right + parameterization (we motivate; they demonstrate the failure mode). +- Softmax auto-alignment interpolation: a training-free adaptive α — cheap + ablation candidate against our fixed α=0.3 (relevant to the 12B α + miscalibration result). + +**Where to cite:** Paper A (untrained/anchor cell; convergent evidence for +anchoring), Paper B (distribution-shift account of loop instability; +KV-sharing hazard contrast). + +--- + +## Combined positioning takeaway + +The two papers bracket us: McLeish et al. = full-retraining recurrence at +scale (expensive end), Lys et al. = zero-training looping (free end). Our +niche is the middle, and it is still open: **interpretability-chosen band + +tiny trained merge on a frozen base + prompt-only latent planning with zero +decode cost + a full attribution ladder + difficulty gating.** Neither paper +touches any of the last three items; both strengthen the premise that band +looping is a real mechanism rather than a curiosity.