item 32 pre-registered (Nils's synthesis): discrete latent chain — lens-snapped token embeddings fed back via zero-init projector (ST top-32, TF/free-running arms, frozen arm-1 merge); sym_iterate in carry_common, trainer/eval wiring
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -1038,3 +1038,38 @@ to explore any new channel; a headroom-bearing task would be a fairer
|
||||
test. Verdict as registered: within +-5 -> no evidence that a native
|
||||
attention read path breaches the consumption wall, with the init
|
||||
confound flagged as the one loose thread.
|
||||
|
||||
32. **The discrete latent chain — "latent paper" (pre-registered
|
||||
2026-07-18 ~00:50, before running; Nils's synthesis: "the loop
|
||||
needs paper, but we don't want that to be full tokens but still
|
||||
latent space").** Diagnosis from the full matrix: every latent
|
||||
medium lacked DISCRETENESS — tokens' magic is the snap
|
||||
(error-correction per step), not visibility. The lens is a native
|
||||
codebook: argmax over its readout quantizes any band state onto
|
||||
the model's own symbol space. Design (each burst tick):
|
||||
read s_{i-1} through the frozen lens; snap to a token
|
||||
(straight-through over top-32, hard forward / soft gradient; tick
|
||||
0 = newline "a step begins"); feed E(token) back through a
|
||||
ZERO-INIT projector alongside the analog carry:
|
||||
x_i = merge(e, s_{i-1}) + proj(E(sym)). Two rails, matching the
|
||||
microscopy's own two-channel algorithm: analog (plans/magnitudes)
|
||||
+ discrete (exact symbols). Writing and computing coincide by
|
||||
construction: what the lens reads IS what gets transported —
|
||||
item 25's lce loss (reused, λ=0.3) is now load-bearing, and its
|
||||
proven concentration effect (10.3->1.9) doubles as the
|
||||
quantization pressure that collapses the diffuse thinking-state
|
||||
superposition (measured: pause states spread over ~hundreds of
|
||||
tokens, committed states 1-2). Merge FROZEN at item-29 arm-1
|
||||
(39.1); only the 2.4M projector trains — the by-construction
|
||||
ablation is 39.1. Arms: (a) sctf — teacher-forced symbols
|
||||
(ground-truth deleted-step tokens; item-29's winning recipe);
|
||||
(b) scst — free-running straight-through snaps. Eval always
|
||||
hard-argmax free-running: 0:0 sanity + 2:0, n=256. Decision vs
|
||||
39.1: >44.1 = discreteness was the missing paper property, the
|
||||
latent-chain program reopens (widenings pre-sketched: top-k
|
||||
parallel snaps, L22/L26 depth rails, tape via slots/kvmem);
|
||||
within +-5 = the discrete channel adds nothing over the analog
|
||||
carry and the "loop = plan machine, tokens = executor" division
|
||||
stands as final. Smokes: TF/ST train (loss arithmetic exact,
|
||||
warm-start vals intact), eval generates with hard snaps.
|
||||
Job: scripts/jobs/zzz_y_symchain.sh.
|
||||
|
||||
Reference in New Issue
Block a user