item 17 scored: GSM-only current recipe fails as predicted; GSM chapter closed

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Nils
2026-07-15 21:14:58 +02:00
co-authored by Claude Fable 5
parent 2cd1f37f5f
commit cb0f1d950c
2 changed files with 4157 additions and 0 deletions
+14
View File
@@ -345,3 +345,17 @@ levels — only seed means are.
(noise-s0, h2048) variants have no mechanism by which task change (noise-s0, h2048) variants have no mechanism by which task change
could invert their MBPP verdict. Only the recipe family's best member could invert their MBPP verdict. Only the recipe family's best member
(this item) is informative on GSM8K. (this item) is informative on GSM8K.
--- Outcome, item 17 (GSM-only, current recipe, scored 2026-07-15):
predictions (a) and (b) CONFIRMED, (c) nominally exceeded but not
meaningfully. hard 8.7/5.5/4.7% at k=1/2/4 (below the 10% bar; nowhere
near the 15% reopen threshold); easy 93.1 -> 48-52% at k>0; overall
11.7/10.9/10.2 vs k0's 10.5 — the k=1 cell is +1.2 points nominal
(~3 items at n=256, not significant), the rest below. Removing the
mixed-task interference bought ~2 points over adapter_uni (9.4 -> 11.7
at k=1) — interference was real but marginal, not masking a capability.
The GSM8K chapter is closed: the recipe family's best member, trained
on GSM alone in the correct regime, delivers no usable gain and the
standard fidelity damage; combined with the scope note, the boundary
claim (structural: supervision density + state-evolution bottleneck)
is fully supported.
File diff suppressed because it is too large Load Diff