E1c scored: easy routing solved (95.9% at 0.11 iters), hard recall regressed; threshold calibration = identified next knob
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -420,3 +420,22 @@ E1c amendment (pre-registered before running, same session): pre-loop
|
|||||||
halt consult on s_0 enabling k*=0; targets easy->0, hard->4; threshold
|
halt consult on s_0 enabling k*=0; targets easy->0, hard->4; threshold
|
||||||
0.5 unchanged (calibration deferred unless E1c misses). Predictions:
|
0.5 unchanged (calibration deferred unless E1c misses). Predictions:
|
||||||
easy >= 95%, hard >= 35.7%, r >= 0.4, overall >= 51.2 at E[k] <= 1.5.
|
easy >= 95%, hard >= 35.7%, r >= 0.4, overall >= 51.2 at E[k] <= 1.5.
|
||||||
|
|
||||||
|
--- Outcome, item 19 / E1c (scored 2026-07-16 ~01:50, Spark re-run):
|
||||||
|
prediction (b) CONFIRMED — easy 95.9% with mean k*=0.11 (the k=0 routing
|
||||||
|
fix worked perfectly for easy items); (a) FAILED (r=0.220, unchanged);
|
||||||
|
(c) FAILED HARD — hard 21.4% at mean k*=1.5: with the pre-loop halt
|
||||||
|
consult, weakly-discriminated hard items now exit at k*=0/1 where before
|
||||||
|
they got >=1 iteration; the recall problem became more expensive, not
|
||||||
|
better. (d) FAILED (overall 50.0 at E[k]=0.74). Net Pareto: E1c =
|
||||||
|
(50.0 overall, 95.9 easy, 21.4 hard, 0.74 mean iters — 82% compute
|
||||||
|
saved); E0 probe-gate = (52.0, 97.5, 28.6, ~2.2) still dominates on
|
||||||
|
accuracy. The learned-head line has ONE identified un-tried knob:
|
||||||
|
deploy-threshold calibration on val for hard-recall (the 0.5 threshold
|
||||||
|
is arbitrary; raising it routes more items deep, trading easy tax for
|
||||||
|
hard recall — a tunable curve E0 cannot offer). E1 arc summary for
|
||||||
|
PLAN_SELFPACED: gating machinery works end-to-end, easy-item protection
|
||||||
|
and compute savings are demonstrated and cheap; difficulty-selective
|
||||||
|
DEPTH allocation remains unsolved at 3K-param-head scale — binding
|
||||||
|
constraint is classifier quality on the k=0/s0 state, exactly where E0
|
||||||
|
started. Next per plan: threshold sweep (cheap) before any E2.
|
||||||
|
|||||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user