item 27 pre-registered: zero-pause internal band looping — single M=10 in-place burst at prompt end, iteration-aligned lens targets, ii0 inference ablation; carry_steps gains inplace updates, trainer gains --inner-iters/--base-pauses
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -732,3 +732,29 @@ auxiliary supervision, A2, E2-N.
|
||||
generation" in the free-running sense) is NOT this — that's a
|
||||
rollout-based follow-up (latent DAgger) if either lens arm moves.
|
||||
Job: scripts/jobs/zzz_r_rungb_lg.sh.
|
||||
|
||||
27. **Internal band looping, zero pause tokens (pre-registered
|
||||
2026-07-17 ~01:10, before running; Nils's clarified structure,
|
||||
option A confirmed via question: one silent burst before output).**
|
||||
Architecture: NO pause tokens anywhere — sequence is prompt +
|
||||
visible output only. After the k=2 prompt settle, the last prompt
|
||||
position runs M=10 IN-PLACE band iterations (s^i seeds from the
|
||||
position's own previous band output — carry_common inplace
|
||||
updates), then generation proceeds with the normal single-pass
|
||||
carry under the visible tokens. Vertical thought burst instead of
|
||||
a horizontal pause tape: iterations leave no KV entries — only
|
||||
the final state survives, a true internal loop (the C'-flavored
|
||||
architecture the ladder never tested; every prior rung bought
|
||||
compute with positions). Supervision: lens-CE iteration i <->
|
||||
deleted-step token i (λ=0.3, the stable value), same d=1 deletion,
|
||||
warm-start rung-A e400, 200 steps, seed 0. Eval n=256: 0:0
|
||||
(sanity), 2:0 with --inner-iters 10 (matched), 2:0 with
|
||||
--inner-iters 0 (ablation: does the burst matter at inference?).
|
||||
References: positional d=1 31.6 (same ±5 / >36.6 bands) and the
|
||||
ii0 ablation cell (isolates the burst's causal contribution).
|
||||
Honest priors: item 25's 2:12 just landed at 31.2 (flat), so the
|
||||
write-vs-compute dissociation likely carries over; the
|
||||
architectural deltas that could matter here: no attention-tape
|
||||
(forces state-borne computation rather than KV re-reading) and
|
||||
anchor-at-prompt (iterations see the full settled question).
|
||||
Job: scripts/jobs/zzz_s_rungb_ii.sh.
|
||||
|
||||
Reference in New Issue
Block a user