jspace/HANDOFF.md

Session handoff — 2026-07-16 ~09:45

Previous session ended because the auto-mode permission classifier entered a persistent session-level lockout (blocked nearly all Bash/Write on "earlier conversation content" grounds, ~7h intermittent). Everything below is current as of ~09:45. Repo knowledge map: INDEX.md. Scored experiment record: results-loop/PROTOCOL_UNIFIED.md (items 1–21, all scored except pending stats below). Research plan: PLAN_SELFPACED.md.

RUNNING RIGHT NOW — do not disturb, monitor these

Rented H100 node ("node 1")

• ssh -p 52525 root@185.151.171.35 (vast.ai; greets with a banner — filter it). Costs money while it runs.
• /root/pipeline5.log: lens-mapping pipeline (setsid, survives ssh drops). Sequence: 31B jbar DONE (saved+synced) → 31B exp4 scan DONE as a printed TABLE (its regimes.pt save crashed — known bug, see fix-list) → 31B weights cleaned → 26B-A4B jacobian scan RUNNING (started ~07:40, ~40-90s/prompt × 256; expect done late morning). After it: 26B exp4 scan (same save bug will strike — harmless, the table survives in the log), CLEANED, PIPELINE-DONE.
• /root/node_sync.sh sidecar (also setsid): rclone-pushes /workspace/jspace/results/ (jbar_.pt, exp4_.log) and pipeline5.log to jspace:jspace/results-lens/ every 120s. The node HAS rclone credentials (Nils placed them himself — never copy credentials to nodes from automation; hard-blocked + it's his call).
• When PIPELINE-DONE: verify results-lens/ has jbar_31b.pt, jbar_26b_a4b.pt, exp4_31b.log, exp4_26b_a4b.log — then the node can be destroyed (tell Nils; it's his dashboard).
• Storage lore: /workspace = disk (survives stop, dies on destroy); /dev/shm = RAM (dies on stop). See memory/vast-node-storage.md.

Spark (this machine)

• gpuq worker in tmux session gpuq_spark:gpuq0, queue EMPTY, healthy. Worker code now has: data contract (# gpuq-in:/# gpuq-out: job comments → auto-sync inputs/outputs with the bucket, outputs every 120s during the job) and graceful drain (touch ~/gpuq/spark-gpu0/STOP — NEVER kill the tmux session mid-job; that cost 3.5h once).
• Thermal: box hard-froze 4× on 2026-07-15 under sustained load — cabinet now open, stable since. If it freezes: reboot, restart worker (tmux new-session -d -s gpuq_spark -n gpuq0 'bash ~/jspace/scripts/gpuq_worker.sh spark-gpu0 0'). NO @reboot cron (Nils vetoed — crash-loop risk).

IMMEDIATE PENDING (blocked by the classifier, run first)

1. .venv/bin/python scripts/mcnemar_carrycot.py — the paired p-values for last night's headline result, especially the drop-bucket test. THE morning number; item 21's outcome note says "pending".
2. Commit everything uncommitted: git add -A scripts/ PLAN_SELFPACED.md HANDOFF.md && git add -f results-loop/PROTOCOL_UNIFIED.md results-loop/eval_gsm_carrycot*.json results-loop/gsm_cot_data.json && git commit && git push origin main (uncommitted: mcnemar script, lens-noise trainer changes [train_carry_cot.py --lensnoise, UNVERIFIED — parse-check first], plan E2-L/E2-A2/E2-N sections, protocol item-21 scoring, this file).
3. Parse the 31B (and later 26B) regime tables from exp4_.log into regimes_.pt — copy the pattern in scripts/parse_e4b_regimes.py (adjust layer count: 31B=60, 26B=48? read from log). Then fill REGIMES.json rows (results/REGIMES.json) and re-render scripts/fig_regimes.py (5-scale figure, panels auto-fill).
4. Patch scripts/exp4_regimes.py to take an explicit output path (torch.save to CWD-relative "results/" has now crashed 3 scans).

LAST NIGHT'S RESULTS (already scored in the protocol)

• Item 21, the headline: GSM8K carry-cot (dense self-distilled terse scratchpads through the carry whiteboard): 57.4% best cell vs 12.1% all-time prior best; base 10.9. Control (same supervision, no recurrence): 54.7 best. Whiteboard's specific edge: DROP items (+9/+14 across cells) — reach into problems unreachable at labeling. If McNemar confirms → gates open for: stage B internalization ladder (PLAN E2-L), A2 on-policy refresh (E2-A2), lens-shaped noise (E2-N, Nils's idea, N1 already implemented as --lensnoise, unverified).
• Item 20: gate threshold curve — E0's frozen logistic probe dominates every learned gate; ORACLE gate = 59.6% overall at 0.24 mean iters (hard items depth-diverse: 18/28 solvable at some k, ≤13 at any single k). Gate program continues; binding constraint = classifier quality on the pre-loop state.
• E4B lens surprise (REGIMES.json, provisional): no E2B-style workspace signature — sensor L11-22, motor from L23, persistence bump ABSENT; candidate band sits inside the KV-shared zone. The elastic pair (E2B⊂E4B) reorganized. Needs ignition cross-check + jbar top-up before strong claims.

MORNING DECISION QUEUE (Nils decides, one submit each)

With McNemar in hand: which of stage B (internalization) / A2 (re-harvest) / E2-N1 (lens-noise, verify parse first) gets the Spark. All pre-registered or planned in PLAN_SELFPACED.md; job template pattern: scripts/jobs/zzz_l_gsm_e2a.sh (uses the data contract).

OPERATIONAL CAUTIONS (paid for in blood, see LESSONS.md 1-12 + memory)

• pkill -f self-match; k=0 sanity row in every eval; e400 pre-commit; seeds before believing single cells (noise-s0 taught this twice); monitors: grep patterns must match eval_loop.py's "acc=" (GSM) vs "pass@1=" (MBPP); artifacts leave nodes within one sync cycle.