item 25 pre-registered: latent process supervision via differentiable lens readout — pause j trained to lens-encode deleted-step token j (λ=0.3/1.0 arms); carry_logits gains return_states
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -674,3 +674,32 @@ the d=1 break. The carried state's native cargo (plans, magnitudes,
|
||||
completion-state — see the probe series) is the program's remaining
|
||||
asset; next candidates: divergence batch replay, coarse-target
|
||||
auxiliary supervision, A2, E2-N.
|
||||
|
||||
25. **E2-L d=1 with latent process supervision through the lens
|
||||
(pre-registered 2026-07-17 ~02:45, before running; Nils's idea:
|
||||
"for training, i wonder if we could calculate, using jspace lens,
|
||||
how each iteration should think").** Items 22-24 all trained the
|
||||
latent chain blind — output CE only — and all failed; this changes
|
||||
the INFORMATION reaching the chain, not its capacity. New loss:
|
||||
the lens readout softmax(W_U·finalnorm(J̄_L30·h)) is differentiable
|
||||
in h, so at the 10 replacement pauses we apply lens-CE against the
|
||||
DELETED step's tokens, aligned 1:1 (pause j <-> step token j,
|
||||
truncated at 10) — the board is trained to write the deleted step
|
||||
in lens-readable code at the time it would have been written.
|
||||
Mixed loss CE_out + λ·CE_lens. Two arms, single submit: λ=0.3 and
|
||||
λ=1.0. Otherwise identical to item-22 d=1 (front-first deletion,
|
||||
warm-start rung-A e400, adapter-only 3e-4, 200 steps, seed 0; no
|
||||
band-LoRA — one knob). Smoke: step-0 lce=10.3 (~uniform: pauses
|
||||
currently encode nothing about the step; large fresh gradient).
|
||||
Eval n=256: 0:0, 2:12, 2:16 per arm. Decision vs d=1's 31.6, same
|
||||
bands: >36.6 = latent supervision was the missing ingredient ->
|
||||
ladder REOPENS with lens-taught rungs (and the 2D per-iteration
|
||||
variant becomes item 26); within +-5 = even telling the board
|
||||
exactly what to write doesn't make the carry compute it -> the
|
||||
strongest closure evidence yet. Caveats pre-stated: J̄ is
|
||||
prompt-averaged (global directions); the loss forces a
|
||||
verbalizable code (microscopy suggests that IS the board's working
|
||||
code, but a native non-verbal code would be fought); the 1:1
|
||||
temporal alignment is one choice among several (bag-of-tokens,
|
||||
result-digits-only are untested alternatives if this null's).
|
||||
Job: scripts/jobs/zzz_q_rungb_lt.sh.
|
||||
|
||||
Reference in New Issue
Block a user