From 13643fa50449e5972594aed557b44c3fc1a4184c Mon Sep 17 00:00:00 2001 From: Nils Date: Wed, 15 Jul 2026 11:42:15 +0200 Subject: [PATCH] abstract rewritten: fold in regime sweep (two independent dials, B-tie isolation, ceiling survival, output-orbit) Co-Authored-By: Claude Fable 5 --- PAPER.md | 71 +++++++++++++++++++++++++++++--------------------------- 1 file changed, 37 insertions(+), 34 deletions(-) diff --git a/PAPER.md b/PAPER.md index 3af8b84..a5a5e7e 100644 --- a/PAPER.md +++ b/PAPER.md @@ -11,41 +11,44 @@ https://git.draic.info/nils/jspace (public). Statistics: Interpretability work with an averaged-Jacobian lens ("J-lens") partitions a pretrained language model's depth into regimes, including a mid-depth *workspace* band that holds verbalizable, unspoken intermediate content. We -retrofit recurrence onto this band in a **frozen** model: a 1.6M-parameter -anchor-dominant merge adapter (0.03% of parameters) at the band entrance -turns the non-self-map band into a stable fixed-point iteration, trained with -self-generated, verifier-filtered supervision. Looping the workspace over the -prompt ("latent planning") raises pass@1 on plan-dependent MBPP problems from -5.5% to 37.5±5.5 over five seeds (best seed 43.6%), with zero visible tokens -and zero additional decode cost. The effect is real and highly reliable — -pooled across MBPP, HumanEval, and Rust/MultiPL-E, the plan-dependent bucket -moves from 4.2% to 35.6% (McNemar p≈1.5e-10) — and **placement is decisive, -not a convenience**: the gain appears only when the loop enters at the -lens-identified boundary (L14), collapsing at L13 and below and structurally -nulling above. A substantial part of the gross effect is generic -perturbation — the untrained merge alone reaches 20.0% on the same bucket -(it conditions on k=0 failure, so regression-to-mean contributes to any -intervention) — but the loop-specific net effect over that floor is real: -paired on items, loop-vs-untrained-merge is 17–4 discordant (p=0.007). +retrofit recurrence onto this band in a **frozen** gemma-4-E2B: a +1.6M-parameter anchor-dominant merge adapter (0.03% of parameters) at the +band entrance turns the non-self-map band into a stable recurrence, trained +with self-generated, verifier-filtered supervision. Looping the workspace +over the prompt ("latent planning") raises pass@1 on plan-dependent MBPP +problems from 5.5% to 37.5±5.5% over five seeds — pooled across MBPP, +HumanEval, and Rust/MultiPL-E, 4.2%→35.6% (McNemar p≈1.5e-10) — with zero +visible tokens and zero decode cost. Placement is decisive, not convenient: +the gain exists only at the lens-identified boundary (L14), collapsing below +it, and KV-sharing structurally nulls entrances above it. Net of the +untrained-merge perturbation floor (20.0%), the loop-specific effect +survives paired testing (p=0.007). + +A two-part attribution program then bounds the mechanism. First, the +content is *amortized, not computed*: recurrence-free plan-distillation +into the same adapter matches the loop, gains do not stack, and inference +depth beyond k≈4 is flat — the state trajectory is an output-stable orbit, +not a converging computation (half of prompts' states never converge at +cos 0.9995 by k=8, with no difficulty gradient, so no convergence-based +early exit falls out). Width rivals depth on code (trained pause registers: +36.4%); recurrence is needed where state must evolve (GSM8K carry, +Blocksworld planning). Second, a pre-registered regime sweep spanning +unconstrained learned recurrence (Huginn-style), spectrally constrained +state maps (Parcae-style), per-iteration weights (Bae-style), and learned +anchor coefficients shows that **dynamical stability and substrate fidelity +are independent dials**: spectral radius governs convergence only (an +unconstrained map drifts to ρ≈4.5 with no fit benefit; a constrained one +stays at ρ≈0.3 with no fit cost — both lose 17 points of easy-item +accuracy), while fidelity is governed by fixed-point *location*, causally +isolated to one design choice — tying the input map to the anchor's convex +complement, B=(1−α)I. Per-iteration weights strand the gain at trained +depths; every regime buys the same hard-bucket gain (36–46%); no regime +exceeds the amortization ceiling at this budget. The hand-tuned recipe is +thus the measured optimum of its design space, not a lucky point in it. +At 12B the anchor coefficient must become state-dependent (3.8K parameters) +to preserve the substrate — the one dial that is task- and scale-dependent. +Details and exact numbers: §1 and §3. -A complete attribution program then deflates the mechanism's mystique: the -loop's content is *amortizable* (plan-distillation into the same adapter, -recurrence-free, matches it; nothing stacks; looping distilled weights -degrades them), width rivals depth (trained pause registers reach 36.4%), -and verifier-assisted (oracle) sampling wins overall accuracy at matched -compute — though the *deployable* selector loses that edge entirely. What -survives is precise: the implant owns exactly the plan-dependent slice at -zero visible tokens and zero decode cost, and its placement is dictated by -the lens. Transfer dissociates by distance: near-distribution the trained -and untrained implants tie (HumanEval); far from it (LiveCodeBench) the -*untrained* merge significantly helps while the trained content -significantly hurts — the learned content is task-local, the recurrence -substrate is general. At 12B a -constant merge coefficient destroys the substrate; a state-dependent -coefficient (3.8K parameters) restores MBPP but not Blocksworld or GSM8K — -the anchor coefficient is the stability dial that unifies this work with -McLeish et al. (2511.07384) and Lys et al. (2602.14759), and it is task- -and scale-dependent. Details and exact numbers: §1 and §3. ## 1. What this paper claims