n=500 controls: paired net-effect significant (loop vs untrained merge 17-4, p=0.007); ladder and net accounting finalized

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

results-loop/STATS.md

@@ -2,6 +2,8 @@
 
 ## Headline numbers (Wilson 95% CIs)
 
+- **MBPP untrained merge k=4 (n=500 rerun)**: overall 0.502 [0.458, 0.546] (n=500); hard 0.200 [0.116, 0.324] (n=55)
+- **MBPP trained FF k=1 (n=500 rerun)**: overall 0.536 [0.492, 0.579] (n=500); hard 0.273 [0.173, 0.402] (n=55)
 - **MBPP loop s0 k=0 (base)**: overall 0.518 [0.474, 0.561] (n=500); hard 0.055 [0.019, 0.149] (n=55)
 - **MBPP loop s0 k=2**: overall 0.530 [0.486, 0.573] (n=500); hard 0.309 [0.203, 0.440] (n=55)
 - **MBPP loop s0 k=4**: overall 0.536 [0.492, 0.579] (n=500); hard 0.436 [0.314, 0.567] (n=55)
@@ -22,6 +24,10 @@
 
 - loop k=4 vs k=0, overall: A-only 30, B-only 39, n=500, p=0.3356 (n.s.)
 - loop k=4 vs k=0, hard: A-only 1, B-only 22, n=55, p=5.722e-06 (**significant**)
+- loop k=4 vs UNTRAINED merge k=4, hard (net effect): A-only 4, B-only 17, n=55, p=0.007197 (**significant**)
+- loop k=4 vs UNTRAINED merge k=4, overall: A-only 29, B-only 46, n=500, p=0.06395 (n.s.)
+- trained FF vs UNTRAINED merge, hard: A-only 7, B-only 11, n=55, p=0.4807 (n.s.)
+- distill k=1 vs UNTRAINED merge k=4, hard: A-only 3, B-only 15, n=55, p=0.007538 (**significant**)
 - distill k=1 vs loop k=4, overall: A-only 33, B-only 37, n=500, p=0.7202 (n.s.)
 - distill k=1 vs loop k=4, hard: A-only 10, B-only 9, n=55, p=1 (n.s.)
 - stack-train k=4 vs distill k=1, hard: A-only 10, B-only 6, n=55, p=0.4545 (n.s.)

results-loop/stats_final.json

@@ -1,4 +1,40 @@
 {
+ "MBPP untrained merge k=4 (n=500 rerun)": {
+  "overall": {
+   "acc": 0.502,
+   "n": 500,
+   "ci": [
+    0.458,
+    0.546
+   ]
+  },
+  "hard": {
+   "acc": 0.2,
+   "n": 55,
+   "ci": [
+    0.116,
+    0.324
+   ]
+  }
+ },
+ "MBPP trained FF k=1 (n=500 rerun)": {
+  "overall": {
+   "acc": 0.536,
+   "n": 500,
+   "ci": [
+    0.492,
+    0.579
+   ]
+  },
+  "hard": {
+   "acc": 0.2727272727272727,
+   "n": 55,
+   "ci": [
+    0.173,
+    0.402
+   ]
+  }
+ },
  "MBPP loop s0 k=0 (base)": {
   "overall": {
    "acc": 0.518,
@@ -275,6 +311,30 @@
    "b_only": 22,
    "p": 5.7220458984375e-06
   },
+  "loop k=4 vs UNTRAINED merge k=4, hard (net effect)": {
+   "n": 55,
+   "a_only": 4,
+   "b_only": 17,
+   "p": 0.007197380065917969
+  },
+  "loop k=4 vs UNTRAINED merge k=4, overall": {
+   "n": 500,
+   "a_only": 29,
+   "b_only": 46,
+   "p": 0.06394991646706696
+  },
+  "trained FF vs UNTRAINED merge, hard": {
+   "n": 55,
+   "a_only": 7,
+   "b_only": 11,
+   "p": 0.480682373046875
+  },
+  "distill k=1 vs UNTRAINED merge k=4, hard": {
+   "n": 55,
+   "a_only": 3,
+   "b_only": 15,
+   "p": 0.007537841796875
+  },
   "distill k=1 vs loop k=4, overall": {
    "n": 500,
    "a_only": 33,