E1c scored: easy routing solved (95.9% at 0.11 iters), hard recall regressed; threshold calibration = identified next knob

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Nils
2026-07-16 01:48:14 +02:00
co-authored by Claude Fable 5
parent a643b4a146
commit 21908a0e33
2 changed files with 1538 additions and 0 deletions
+19
View File
@@ -420,3 +420,22 @@ E1c amendment (pre-registered before running, same session): pre-loop
halt consult on s_0 enabling k*=0; targets easy->0, hard->4; threshold halt consult on s_0 enabling k*=0; targets easy->0, hard->4; threshold
0.5 unchanged (calibration deferred unless E1c misses). Predictions: 0.5 unchanged (calibration deferred unless E1c misses). Predictions:
easy >= 95%, hard >= 35.7%, r >= 0.4, overall >= 51.2 at E[k] <= 1.5. easy >= 95%, hard >= 35.7%, r >= 0.4, overall >= 51.2 at E[k] <= 1.5.
--- Outcome, item 19 / E1c (scored 2026-07-16 ~01:50, Spark re-run):
prediction (b) CONFIRMED — easy 95.9% with mean k*=0.11 (the k=0 routing
fix worked perfectly for easy items); (a) FAILED (r=0.220, unchanged);
(c) FAILED HARD — hard 21.4% at mean k*=1.5: with the pre-loop halt
consult, weakly-discriminated hard items now exit at k*=0/1 where before
they got >=1 iteration; the recall problem became more expensive, not
better. (d) FAILED (overall 50.0 at E[k]=0.74). Net Pareto: E1c =
(50.0 overall, 95.9 easy, 21.4 hard, 0.74 mean iters — 82% compute
saved); E0 probe-gate = (52.0, 97.5, 28.6, ~2.2) still dominates on
accuracy. The learned-head line has ONE identified un-tried knob:
deploy-threshold calibration on val for hard-recall (the 0.5 threshold
is arbitrary; raising it routes more items deep, trading easy tax for
hard recall — a tunable curve E0 cannot offer). E1 arc summary for
PLAN_SELFPACED: gating machinery works end-to-end, easy-item protection
and compute savings are demonstrated and cheap; difficulty-selective
DEPTH allocation remains unsolved at 3K-param-head scale — binding
constraint is classifier quality on the k=0/s0 state, exactly where E0
started. Next per plan: threshold sweep (cheap) before any E2.
File diff suppressed because it is too large Load Diff