HumanEval transfer: trained-vs-untrained-merge paired test (p=0.80, n.s.) -- transfer gain is the merge itself, not trained content

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 16:30:36 +02:00
parent d1bafc2d4d
commit dcd0b0b2dc
4 changed files with 17 additions and 4 deletions
@@ -305,6 +305,12 @@
   "b_only": 17,
   "p": 0.06391465663909912
  },
+  "HumanEval trained vs UNTRAINED merge (k=2), overall": {
+   "n": 164,
+   "a_only": 7,
+   "b_only": 9,
+   "p": 0.803619384765625
+  },
  "Rust loop k=4 vs k=0, overall": {
   "n": 154,
   "a_only": 11,