item 30 scored: readiness strongly decodable (AUC 0.80) but NOT recurrence-specific (FF 0.79); early-stop loses everywhere, oracle only +3.9 — head is instrumentation, not a stopping lever; extension/deferral gating stays live
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -970,6 +970,27 @@ rung A (nothing deleted, move the 57.4).
|
||||
rescues drift cases). Arm 2 (later): line-correctness /
|
||||
deferral head gating re-expansion. Job: scripts/jobs/zzz_w_metacog.sh.
|
||||
|
||||
--- Outcome, item 30 (scored 2026-07-17 ~19:40; rc=0). Harvests:
|
||||
4319/1058 carry, 3408/879 FF boundary rows; readiness rates 33-41%.
|
||||
(a) PRIMARY REFUTED on specificity, CONFIRMED on existence:
|
||||
answer-readiness is STRONGLY decodable from the whiteboard — test AUC
|
||||
0.798 — but the FF control decodes it equally (0.791): the
|
||||
metacognitive signal is whiteboard-general, NOT recurrence-specific.
|
||||
The carry's drop-bucket edge does not manifest as a linearly-more-
|
||||
readable readiness code at line boundaries. (b) Theta sweep: early
|
||||
stopping LOSES at every theta <= 0.95 (0.441-0.520 vs natural-end
|
||||
0.527); theta=0.99 reaches parity with 37/256 early answers. (c)
|
||||
ORACLE stop = 0.566 (+3.9 over natural end at mean boundary 3.58) —
|
||||
a weak ceiling vs item 20's +9.6 depth oracle: the model's natural
|
||||
halting already sits near its readiness frontier, and early-stop
|
||||
errors are cost-asymmetric. Verdict: the readiness HEAD works as
|
||||
instrumentation (0.8 AUC from a frozen state, zero human labels);
|
||||
early-stop DEPLOYMENT is not the lever. Live follow-ups: arm 2 —
|
||||
deferral/re-expansion gating (EXTENDING computation where the head is
|
||||
uncertain; its oracle is unmeasured and the 430 mechanism points
|
||||
there, not at stopping); and using g as a per-item difficulty signal
|
||||
for the item-20 depth gate (where the oracle headroom actually is).
|
||||
|
||||
31. **Synthetic memory tokens: per-layer KV prefix (pre-registered
|
||||
2026-07-17 ~15:50, before running; Nils's design: "the one read
|
||||
mechanism a frozen transformer natively possesses is attention over
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
{"carry": 0.7981836131859141, "ff": 0.791359567919433}
|
||||
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user