unification paragraph: anchor coefficient as the stability dial across Lys/ours/McLeish

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 10:25:04 +02:00
parent 8e7bd2019e
commit ba3bacc252
1 changed files with 14 additions and 0 deletions
@@ -214,6 +214,20 @@ convergent evidence for our anchor-dominant merge; their whole setting
 corresponds to the untrained cell of our attribution table (17.9% hard =
 our FF/untrained level), evaluated by likelihood rather than execution.
 **One mechanism, three regimes.** All three works are variants of a single
 design: mix the fed-back state with an anchor derived from the un-looped
 computation. Lys et al.'s inference-time moving average η·h⁽⁰⁾+(1−η)·h⁽ᵗ⁾ is
 an untrained anchor-mixing coefficient; our (1−α)e + α·ŝ + MLP([e;ŝ]) is its
 trained analogue (fixed mix + learned correction); McLeish et al.'s
 concatenated input injection is the fully learned limit, trained end-to-end.
 The anchor coefficient is the stability dial of frozen-band looping: Lys
 et al.'s naive-looping collapse is the zero-anchor (α→1) limit, their
 regularization gains are the untrained anchored regime, and our 12B failure
 at the 2B-tuned α=0.3 — with the untrained-substrate arm showing the damage
 is pre-training-of-the-adapter — is the same dial mis-set at a new scale.
 Stability of retrofitted recurrence appears to be governed by how strongly
 the loop is anchored, across all three training budgets.
 Earlier lineage: Universal Transformers (adaptive depth); DEQ (fixed-point
 inference); Huginn (arXiv:2502.05171) — prelude/core/coda from scratch;
 Mixture-of-Recursions (arXiv:2507.10524) — learned per-token depth; Relaxed