diff --git a/PAPER.md b/PAPER.md index 4f0626d..d33bb14 100644 --- a/PAPER.md +++ b/PAPER.md @@ -214,6 +214,20 @@ convergent evidence for our anchor-dominant merge; their whole setting corresponds to the untrained cell of our attribution table (17.9% hard = our FF/untrained level), evaluated by likelihood rather than execution. +**One mechanism, three regimes.** All three works are variants of a single +design: mix the fed-back state with an anchor derived from the un-looped +computation. Lys et al.'s inference-time moving average η·h⁽⁰⁾+(1−η)·h⁽ᵗ⁾ is +an untrained anchor-mixing coefficient; our (1−α)e + α·ŝ + MLP([e;ŝ]) is its +trained analogue (fixed mix + learned correction); McLeish et al.'s +concatenated input injection is the fully learned limit, trained end-to-end. +The anchor coefficient is the stability dial of frozen-band looping: Lys +et al.'s naive-looping collapse is the zero-anchor (α→1) limit, their +regularization gains are the untrained anchored regime, and our 12B failure +at the 2B-tuned α=0.3 — with the untrained-substrate arm showing the damage +is pre-training-of-the-adapter — is the same dial mis-set at a new scale. +Stability of retrofitted recurrence appears to be governed by how strongly +the loop is anchored, across all three training budgets. + Earlier lineage: Universal Transformers (adaptive depth); DEQ (fixed-point inference); Huginn (arXiv:2502.05171) — prelude/core/coda from scratch; Mixture-of-Recursions (arXiv:2507.10524) — learned per-token depth; Relaxed