item 21 McNemar scored: drop-bucket edge significant (2:6 p=0.0094, survives Bonferroni x4); overall A-vs-B ns; + session handoff, fig cleanup

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

HANDOFF.md

@@ -0,0 +1,98 @@
+# Session handoff — 2026-07-16 ~09:45
+
+Previous session ended because the auto-mode permission classifier entered
+a persistent session-level lockout (blocked nearly all Bash/Write on
+"earlier conversation content" grounds, ~7h intermittent). Everything
+below is current as of ~09:45. Repo knowledge map: `INDEX.md`. Scored
+experiment record: `results-loop/PROTOCOL_UNIFIED.md` (items 1–21, all
+scored except pending stats below). Research plan: `PLAN_SELFPACED.md`.
+
+## RUNNING RIGHT NOW — do not disturb, monitor these
+
+### Rented H100 node ("node 1")
+- **ssh -p 52525 root@185.151.171.35** (vast.ai; greets with a banner —
+  filter it). Costs money while it runs.
+- **`/root/pipeline5.log`**: lens-mapping pipeline (setsid, survives ssh
+  drops). Sequence: 31B jbar DONE (saved+synced) → 31B exp4 scan DONE as
+  a printed TABLE (its `regimes.pt` save crashed — known bug, see
+  fix-list) → 31B weights cleaned → **26B-A4B jacobian scan RUNNING**
+  (started ~07:40, ~40-90s/prompt × 256; expect done late morning).
+  After it: 26B exp4 scan (same save bug will strike — harmless, the
+  table survives in the log), CLEANED, PIPELINE-DONE.
+- **`/root/node_sync.sh` sidecar** (also setsid): rclone-pushes
+  `/workspace/jspace/results/` (jbar_*.pt, exp4_*.log) and pipeline5.log
+  to `jspace:jspace/results-lens/` every 120s. The node HAS rclone
+  credentials (Nils placed them himself — never copy credentials to
+  nodes from automation; hard-blocked + it's his call).
+- When PIPELINE-DONE: verify `results-lens/` has jbar_31b.pt,
+  jbar_26b_a4b.pt, exp4_31b.log, exp4_26b_a4b.log — then the node can be
+  destroyed (tell Nils; it's his dashboard).
+- Storage lore: /workspace = disk (survives stop, dies on destroy);
+  /dev/shm = RAM (dies on stop). See memory/vast-node-storage.md.
+
+### Spark (this machine)
+- gpuq worker in tmux session `gpuq_spark:gpuq0`, queue EMPTY, healthy.
+  Worker code now has: data contract (`# gpuq-in:`/`# gpuq-out:` job
+  comments → auto-sync inputs/outputs with the bucket, outputs every
+  120s during the job) and graceful drain (`touch ~/gpuq/spark-gpu0/STOP`
+  — NEVER kill the tmux session mid-job; that cost 3.5h once).
+- Thermal: box hard-froze 4× on 2026-07-15 under sustained load —
+  cabinet now open, stable since. If it freezes: reboot, restart worker
+  (`tmux new-session -d -s gpuq_spark -n gpuq0 'bash
+  ~/jspace/scripts/gpuq_worker.sh spark-gpu0 0'`). NO @reboot cron
+  (Nils vetoed — crash-loop risk).
+
+## IMMEDIATE PENDING (blocked by the classifier, run first)
+
+1. `.venv/bin/python scripts/mcnemar_carrycot.py` — the paired p-values
+   for last night's headline result, especially the drop-bucket test.
+   THE morning number; item 21's outcome note says "pending".
+2. Commit everything uncommitted:
+   `git add -A scripts/ PLAN_SELFPACED.md HANDOFF.md && git add -f
+   results-loop/PROTOCOL_UNIFIED.md results-loop/eval_gsm_carrycot*.json
+   results-loop/gsm_cot_data.json && git commit && git push origin main`
+   (uncommitted: mcnemar script, lens-noise trainer changes
+   [train_carry_cot.py --lensnoise, UNVERIFIED — parse-check first],
+   plan E2-L/E2-A2/E2-N sections, protocol item-21 scoring, this file).
+3. Parse the 31B (and later 26B) regime tables from exp4_*.log into
+   regimes_*.pt — copy the pattern in `scripts/parse_e4b_regimes.py`
+   (adjust layer count: 31B=60, 26B=48? read from log). Then fill
+   REGIMES.json rows (results/REGIMES.json) and re-render
+   `scripts/fig_regimes.py` (5-scale figure, panels auto-fill).
+4. Patch `scripts/exp4_regimes.py` to take an explicit output path
+   (torch.save to CWD-relative "results/" has now crashed 3 scans).
+
+## LAST NIGHT'S RESULTS (already scored in the protocol)
+
+- **Item 21, the headline**: GSM8K carry-cot (dense self-distilled terse
+  scratchpads through the carry whiteboard): **57.4%** best cell vs
+  12.1% all-time prior best; base 10.9. Control (same supervision, no
+  recurrence): 54.7 best. Whiteboard's specific edge: DROP items
+  (+9/+14 across cells) — reach into problems unreachable at labeling.
+  If McNemar confirms → gates open for: stage B internalization ladder
+  (PLAN E2-L), A2 on-policy refresh (E2-A2), lens-shaped noise (E2-N,
+  Nils's idea, N1 already implemented as --lensnoise, unverified).
+- **Item 20**: gate threshold curve — E0's frozen logistic probe
+  dominates every learned gate; ORACLE gate = 59.6% overall at 0.24
+  mean iters (hard items depth-diverse: 18/28 solvable at some k, ≤13
+  at any single k). Gate program continues; binding constraint =
+  classifier quality on the pre-loop state.
+- **E4B lens surprise** (REGIMES.json, provisional): no E2B-style
+  workspace signature — sensor L11-22, motor from L23, persistence bump
+  ABSENT; candidate band sits inside the KV-shared zone. The elastic
+  pair (E2B⊂E4B) reorganized. Needs ignition cross-check + jbar top-up
+  before strong claims.
+
+## MORNING DECISION QUEUE (Nils decides, one submit each)
+
+With McNemar in hand: which of stage B (internalization) / A2
+(re-harvest) / E2-N1 (lens-noise, verify parse first) gets the Spark.
+All pre-registered or planned in PLAN_SELFPACED.md; job template
+pattern: scripts/jobs/zzz_l_gsm_e2a.sh (uses the data contract).
+
+## OPERATIONAL CAUTIONS (paid for in blood, see LESSONS.md 1-12 + memory)
+
+- pkill -f self-match; k=0 sanity row in every eval; e400 pre-commit;
+  seeds before believing single cells (noise-s0 taught this twice);
+  monitors: grep patterns must match eval_loop.py's "acc=" (GSM) vs
+  "pass@1=" (MBPP); artifacts leave nodes within one sync cycle.

results-loop/PROTOCOL_UNIFIED.md

@@ -513,8 +513,14 @@ consistent, specific signature: DROP items (unreachable by the base
 model even with full CoT at labeling) — A beats B by +9 and +14 points
 there in the two cells. Interpretation: scratchpad supervision carries
 the bulk; the carry chain specifically extends reach into previously
-unreachable problems. McNemar p-values pending (mcnemar_carrycot.py
+unreachable problems. McNemar (scored 2026-07-16 morning): overall
-ready; classifier-blocked at scoring time). (c) CONFIRMED: easy damage
+A-vs-B not significant in either cell (2:2 discordants 34-23 p=0.185;
+2:6 33-26 p=0.435), but the pre-identified drop-bucket signature IS:
+2:6 drop discordants 20-6, exact McNemar p=0.0094 (survives
+Bonferroni x4 = 0.038); 2:2 drop 16-7, p=0.093 (same direction,
+marginal). Verdict: the whiteboard's edge is real and specific to
+extending reach into drop items, not a general lift over matched
+supervision. (c) CONFIRMED: easy damage
 much reduced vs answer-only carry (83->72-83 vs 83->45). Ladder gate:
 technically met at p=2; decision on stage B/A2/E2-N deferred to the
 morning review with the p-values in hand — the drop-bucket signature,

scripts/mcnemar_carrycot.py

@@ -22,5 +22,8 @@ for cell in ("2:2", "2:6"):
              if star.get(i) == "drop" and a[i] and not b[i])
     dy = sum(1 for i in ids
              if star.get(i) == "drop" and b[i] and not a[i])
+    dn = dx + dy
+    dp = (min(1, sum(comb(dn, k) for k in range(max(dx, dy), dn + 1))
+              / 2 ** dn * 2) if dn else 1)
     print(f"{cell}: carry-only={x} ff-only={y}  McNemar p={p:.4f}  "
-          f"drop-bucket discordants {dx}-{dy}")
+          f"drop-bucket discordants {dx}-{dy} McNemar p={dp:.4f}")