TiedAlphaAdapter: learned per-dim alpha with tied B (anchored by construction); pre-registration item 14
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -163,3 +163,20 @@ merge level (35.7/39.3/42.9% at k=2/4/8 — a one-item-per-depth-doubling
|
||||
crawl that at k=8 reaches what the contractive merge reaches at k=4,
|
||||
never approaching the amortization ceiling from above). 4x parameters
|
||||
bought nothing. Depth-monotone computation did not emerge at this budget.
|
||||
|
||||
14. **Tied-alpha arm (pre-registered 2026-07-15, before training).**
|
||||
TiedAlphaAdapter: x = (1−a)⊙e + a⊙ŝ + MLP([e;ŝ]), a = σ(â) per-dim
|
||||
learned, init a=0.3 everywhere (bit-equal to MergeAdapter at step 0,
|
||||
verified). B tied to (1−a): convex combination keeps the LTI fixed
|
||||
point on the e–ŝ segment (substrate-anchored by construction),
|
||||
ρ = max(a) < 1 guaranteed, +d≈1.5K params. Standard curriculum,
|
||||
s0 = band(e), e400, eval ks 0,2,4,8 on 250 items. This is the one
|
||||
untested cell combining parcae's learnable decay with the merge's
|
||||
anchoring. Predictions: (a) substrate fidelity preserved (easy ≈
|
||||
merge's 88%, unlike both rec arms' ~70%) because anchoring, not
|
||||
ρ, controls fidelity; (b) hard-bucket at merge level (no significant
|
||||
gain — per-dim constant α is not where capability lives, per the
|
||||
adaptive-α E2B result); (c) learned a drifts slightly DOWN from 0.3
|
||||
(as in parcae). If (a) holds while rec arms failed it, the
|
||||
fixed-point-location dial is causally isolated: same learnable-decay
|
||||
freedom, only the tie to (1−a) differs from parcae.
|
||||
|
||||
Reference in New Issue
Block a user