item 28 pre-registered (Nils's variant): teacher-state distillation — frozen full-cot teacher's band-exit state at step end, cosine into burst s^10; λ amended 1.0→5.0 pre-run (smoke: baseline cos-dist 0.113)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Nils
2026-07-17 00:46:19 +02:00
co-authored by Claude Fable 5
parent 62ce30e557
commit bbf8a02c6b
3 changed files with 96 additions and 2 deletions
+35
View File
@@ -776,3 +776,38 @@ after it (job renamed zzz_t).
(forces state-borne computation rather than KV re-reading) and
anchor-at-prompt (iterations see the full settled question).
Job: scripts/jobs/zzz_s_rungb_ii.sh.
IN-FLIGHT NOTE (~02:20): lce 9.4 -> 2.9 by step 60 — the tape-free
burst also encodes the step trajectory; encoding is never the
obstacle. Accuracy pending.
28. **Teacher-state distillation into the burst (pre-registered
2026-07-17 ~02:30, before running; Nils's variant: "meaningful CoT
chunks yield internal state that we then teacher-force into the
loop-only model").** Items 25/27 supervise the VERBAL SHADOW of
the deleted step (token targets through the lens); this forces the
FULL state. Teacher = the frozen warm-start (rung-A e400) adapter
running the complete cot (deleted step visible), zero pauses, same
carry architecture; capture its band-exit state at the deleted
step's LAST token — "the state of having finished thinking the
step" — one no-grad pass over the 413 parseable items at startup
(targets fixed, no moving teacher). Student: identical zero-pause
M=10 burst as item 27, but the loss is cosine distance between the
burst's FINAL iterate s^10 and the teacher state, plus output CE.
Weight AMENDED pre-run λ=1.0 -> 5.0 after the smoke measured the
starting cosine distance at only 0.113 (nearby band-exit states
share most structure); at 1.0 the term would be ~10x weaker than
the CE and a null would be underpowered — 5x0.113 puts the two
terms at comparable initial scale. No lens loss (one knob vs item 27: full-state
targets instead of verbal-shadow targets; trajectory supervision
dropped — only the endpoint is forced). Known approximation,
stated in advance: teacher state is captured ~10 positions later
in the sequence than the student anchor (RoPE position coloring);
cosine + the carry machinery's routine state transplantation
across nearby positions make this tolerable, but a null could
partly hide here. Eval n=256: 0:0, 2:0 ii10 (matched), 2:0 ii0
(ablation). Decision vs 31.6, same bands; the informative
three-way is 25 (verbal, tape) vs 27 (verbal, no tape) vs 28
(full state, no tape) — if 28 moves where 25/27 don't, the
computation-carrying content lives OUTSIDE the verbalizable
subspace; if none move, the read-side clamp test (item 29
candidate) decides. Job: scripts/jobs/zzz_sa_rungb_ts.sh.