# Prototype plan: the self-paced workspace (v2)

*Drafted 2026-07-16, pre-registration-style. Goal: test whether the model
can learn to allocate workspace-loop compute ON ITS OWN — per prompt and
per generation step — rather than at a swept hyperparameter k.*

## The concept

At every step the system chooses: emit, or spend a band iteration updating
the workspace first. Make that choice a learned gate g(workspace state).
Compute becomes a decision, not a constant. Gate-bought iterations emit no
tokens, so they are exposure-safe by construction (deterministic given
state — the pause-position property).

## What already exists (de-risked ingredients)

| ingredient | evidence | where |
|---|---|---|
| static per-prompt gate (E0) | gated 52.0 overall, easy 97.5 (vs 88.5 uniform), hard 28.6; bottleneck = probe recall (18/28 tp, 95 predicted hard) | `gate_probe.py`, `eval_gated.json` |
| state-dependent control heads train | adaptive-α rescued 12B (3.8K params) | AdaptiveMergeAdapter |
| evolving state during generation | carry beats registers on GSM (hard 0→9.4) | `carry_common.py`, `eval_carry.json` |
| loop-capacity knob | rung-2 band-LoRA = best hard numbers (42.9/46.4) | `lora_band.py` |
| gates over DEPTH (complementary axis) | learnable compute envelope g[t,l] | `path_gates.py` (Nils, in progress) |
| free gate supervision | STaR difficulty labels; per-position labels derivable | prep_star/prep_mbpp |

## Experiments

### E1 — learned per-prompt halting (prompt side, MBPP)
Replace fixed k with a trained soft halting gate. Architecture: after each
iteration i, gate head h(e, ŝ_i) → p_halt,i (zero-init to fixed-k
behavior); training uses the soft mixture of iteration outputs weighted by
halting distribution (ACT-style), CE + λ·E[iterations] compute penalty;
deploy = argmax halt. Trains end-to-end, NO RL. Arms: λ ∈ {1e-3, 1e-2},
vs E0 probe-gate and uniform-k anchors.
**Pre-registered predictions:** (a) accuracy ≥ uniform k=4 overall at ≤60%
of its mean iterations; (b) easy ≥ 95% (gate protects the substrate);
(c) allocation correlates with STaR label (point-biserial r > 0.3);
(d) hard ≥ E0's 28.6% (learned gate beats frozen probe recall).
**Failure mode to watch:** gate collapse (all-0/all-1) — mitigate with
penalty warmup + entropy bonus; collapse at all λ falsifies E1.

### E2 — generation-side gating (GSM, gated carry)
Substrate: design-C carry + short verified-CoT supervision (dense targets;
harvest with "solve in ≤3 short steps", answer-verified). Gate per token
step decides whether the carry state updates through the band or passes
through: x_t = g·merge(e_t, s_{t−1}) + (1−g)·e_t, penalty λ·E[g].
Anchors: carry-always, carry-never (same supervision).
**Predictions:** (a) gate fires non-uniformly, concentrated near numeric/
operator tokens (measurable); (b) accuracy ≥ carry-always (gating as
protection); (c) easy-bucket damage < carry-always's (83→45 was the
unprotected number). Hard-bucket *gain* over carry-always is hoped for,
not predicted.

### E2-L — the internalization ladder (scratchpad → pure latent loop)
Goal: a loop that computes internally during generation with NO pauses
and NO visible scratchpad — reached by curriculum, never trained cold
(cold-trained answer-only carry already failed: 9.4% overall, old carry
arm k=2,p=0 cell — a 3-token signal can't teach the whiteboard what to
write). Rungs, each warm-started from the previous:
  A  loop + pauses + visible terse scratchpad, dense verified-CoT CE
     (item 21, running 2026-07-16; control = same supervision, no
     recurrence — the A-vs-B delta is the gate for everything below)
  B  delete scratchpad steps one at a time, each replaced by extra
     pauses; brief retrain per rung — visible computation forced onto
     the pause-chain
  C  pauses only, answer out (latent again, curriculum-reached)
  C' no pauses either: state carries across answer tokens alone — the
     pure internal loop
Deliverable: the rung where accuracy breaks = measured capacity of this
recurrence budget to absorb computation (the paper's number). Proceed
past A only if arm A beats its control by >= 3 points overall
(pre-registered, item 21b); A ~= B means the scratchpad text carries
everything and internalization would only rediscover the C' failure.

### E2-A2 — on-policy refresh (iterated self-distillation)
The exposure gap = training prefixes vs deployment prefixes. Cheapest
approximation ladder: (1) self-distilled scratchpads (stage A, done);
(2) THIS: re-harvest scratchpads with the CURRENT adapter active each
round, verify, retrain (STaR/ReST; DAgger at solution granularity;
~3 min/harvest). Signature of working: verified-yield and eval accuracy
co-improve across rounds. Gated on the A-vs-B verdict.

### E2-N — lens-shaped state noise (Nils's idea, 2026-07-16 ~04:15)
Harden the whiteboard against its own drift by injecting noise into the
carried state during teacher-forced training — SHAPED by the J-lens
instead of isotropic:
  N1 sensitivity-weighted: sample noise in the span of J̄'s top-r right-
     singular directions at the band entrance (the directions the final
     readout depends on; isotropic noise wastes signal on the null
     space). Cheap: jbar.pt exists; --lensnoise rank,scale flag.
  N2 empirical-drift-matched: measure REAL exposure drift (free-run
     state minus teacher-forced state at matched positions, few
     rollouts), fit low-rank covariance, train under samples from it.
     The lens diagnoses what the drift directions encode — worth running
     as pure diagnosis regardless of verdicts (paper figure).
  N3 concept-jitter: lens-read the carried concept (e.g. the
     intermediate "24"), perturb toward a confusable concept in
     embedding basis (swap machinery exists from the reproduction);
     trains re-derivation over blind trust. Most ambitious.
Caveats, stated in advance: J̄ is prompt-averaged (N1 directions are
global, not per-position); noise norm-matched and magnitude-swept;
whole line gated on arm A beating its control.

### E3 — power knob (only if E1 or E2 shows clean gating)
Warm-start rung-2 band-LoRA under the gate; joint fine-tune. Question: do
gate-bought iterations do MORE per iteration with a trainable band?
Metric: the internalization count (how many scratchpad steps can be
removed post-hoc, E2 curriculum) as a function of LoRA rank.

### Lens verification (throughout — our home advantage)
J-lens reads of gated vs ungated positions: do bought iterations sharpen
task-relevant concepts at the positions where the gate fired? This is the
mechanistic check that the gate allocates *meaningfully*, not just
correlationally.

## Explicitly out of scope for the prototype
Outcome-RL training of the gate (GRPO with compute price) — stage 2, only
if E1–E3 show selective gating. 12B/scale transfer. Cross-task gates.

## Budget & order
E1: 3 arms × ~75 min (Spark). E2: harvest ~30 min + 3 arms × ~90 min.
E3: +2 arms. Total ≈ 1.5 Spark-days. Runs after the lens campaign; queue
via gpuq as usual, every arm pre-registered in PROTOCOL_UNIFIED.md before
launch (items 18+).

## Kill criteria (decided in advance)
- E1 gate collapse at all λ AND E2 uniform firing → the state does not
  carry usable "needs compute" signal at this scale; program stops, E0's
  static-gate deployment note stands as the practical answer.
- E1 works but hard < E0 → learned gate worse than probe; ship probe-gate,
  keep E2 only if its (a)/(b) hold.