item 29 pre-registered (Nils's design): trajectory teacher-forcing — 10 waypoint transitions supervised independently (T[i-1]→T[i]), TF and TF+FR arms; job 1 d=1 step-span, job 2 answer-only full-CoT span
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -867,3 +867,36 @@ at inference via the lens/embedding-basis machinery, no training, and
|
||||
measure) is item 29's natural content and would assign blame causally.
|
||||
Nils's morning decision: clamp test vs pivot to the hybrid/A2 line
|
||||
(where the 57.4 and the drop-bucket reach already live).
|
||||
|
||||
29. **Trajectory teacher-forcing of the burst (pre-registered
|
||||
2026-07-17 ~11:50, before running; Nils's design: "measure the CoT
|
||||
run's L30 at 10 evenly spaced intervals and teacher force them
|
||||
into the loop — we know both the previous that goes into the merge
|
||||
layer and the next L30 state").** The key upgrade over item 28:
|
||||
the teacher trajectory decomposes the burst's learning into TEN
|
||||
INDEPENDENT SUPERVISED TRANSITIONS — iteration i receives teacher
|
||||
waypoint T[i-1] as its merge input and its band output is pulled
|
||||
onto T[i] (cosine, λ=5.0) — dense gradient at every iteration, no
|
||||
long BPTT chain, no compounding; and T[0] == the student's own
|
||||
settled anchor state exactly (shared prompt+settle), so the chain
|
||||
starts aligned by construction. Exposure gap handled by a second
|
||||
arm adding the free-running loss (burst's own s_i onto T[i],
|
||||
λ=5.0). The free-running burst ALWAYS runs last and seeds the
|
||||
answer scan (training matches inference); TF transitions are
|
||||
scaffolding erased from context afterward. Teacher = frozen
|
||||
warm-start adapter on the full cot, 10 evenly spaced L30 states.
|
||||
Two jobs, per Nils (both chosen): JOB 1 (zzz_u) d=1, waypoints
|
||||
across the deleted step, arms tf-only and tf+fr; reference 31.6,
|
||||
same bands; eval 0:0 / 2:0 ii10 / 2:0 ii0 per arm. JOB 2 (zzz_v)
|
||||
THE HEADLINE: d=all (answer-only output, rung C), waypoints across
|
||||
the FULL CoT — the burst as a 10-waypoint compressed rehearsal of
|
||||
the entire reasoning; references: cold answer-only 9.4, curriculum
|
||||
plateau 19.1, base 10.9; same arms and cells. Smokes: ltf/lfr
|
||||
baseline 0.116 both modes; d=99 deletes 1118 steps across 427.
|
||||
Priors, stated: the transition-decomposition is the first
|
||||
mechanism that trains the loop's DYNAMICS rather than its
|
||||
endpoints — if the loop can compute at all, this is its best
|
||||
chance; if job 2 beats 19.1 meaningfully, compressed-trajectory
|
||||
rehearsal becomes the internalization method. All prior caveats
|
||||
(position coloring, teacher=warm-start quality) carry over.
|
||||
Jobs: scripts/jobs/zzz_u_traj_d1.sh, scripts/jobs/zzz_v_traj_full.sh.
|
||||
|
||||
Reference in New Issue
Block a user