nils/jspace
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

pre-register item 16: code->GSM8K cross-task transfer

Browse Source 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Nils
2026-07-15 12:54:58 +02:00
co-authored by Claude Fable 5
parent b013ca27f1
commit ebf0eb8f49
1 changed files with 13 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
results-loop/PROTOCOL_UNIFIED.md
+13
View File
@@ -256,3 +256,16 @@ essentially unmoved: mean 0.298, range [0.285, 0.310], 0/1536 dims moved
hand-tuned 0.3 was already optimal. Recipe consequence: fixed-alpha hand-tuned 0.3 was already optimal. Recipe consequence: fixed-alpha
anchored merge is the recommended design; learnable-alpha safe but anchored merge is the recommended design; learnable-alpha safe but
pointless, learnable-B harmful, per-depth strands the gain. pointless, learnable-B harmful, per-depth strands the gain.
16. **Code→GSM8K cross-task transfer (pre-registered 2026-07-15 ~14:10,
before running).** The MBPP-trained loop adapter (adapter_code, s0) and
the noise-s0 variant evaluated on GSM8K test (n=256, prompt-only loop,
same harness as eval_gsmonly). Extends the transfer-distance ladder
(HumanEval tie -> LCB trained-hurts) across tasks. Predictions:
(a) hard-bucket gain ~0 (plan content is task-local; GSM8K needs
evolving state, not static plans); (b) easy items damaged at k>0
(~93 -> 50-70%), comparable to or worse than the GSM-trained merge —
substrate damage on GSM8K is perturbation-driven and content-agnostic;
(c) overall at k>0 below k=0 (no rescue). If instead hard gains
appear (>5 points), plan-shaped content is partially task-general —
would weaken the task-local claim from LCB.