item 31 pre-registered (Nils's design): synthetic memory tokens — burst states → per-band-layer KV prefix via KVMemoryAdapter (in-place attention wrap, bit-exact disarmed, gated silent init); composed with frozen item-29 arm-1; +29 tf+fr in-flight note (30.5, FR term hurts)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Nils
2026-07-17 13:25:25 +02:00
co-authored by Claude Fable 5
parent 11604d0949
commit b18ffbd739
6 changed files with 231 additions and 11 deletions
+7 -2
View File
@@ -128,7 +128,7 @@ def carry_logits(looper, adapter, input_ids, attention_mask, prompt_lens,
@torch.no_grad()
def generate_carry_c(looper, adapter, tok, input_ids, attention_mask,
k, p, max_new_tokens=10, feedforward=False,
inner_iters=0):
inner_iters=0, kvmem=None):
"""Greedy design-C generation (left-padded batch, uniform positions).
Appends p pause tokens, prefill-loops the prompt, carries through the
@@ -167,7 +167,12 @@ def generate_carry_c(looper, adapter, tok, input_ids, attention_mask,
# p=0: iterate at the last prompt position, before any
# visible token — no pause tokens involved
updates = (inner + updates) if p == 0 else (updates + inner)
S, X = carry_steps(looper, adapter, e, calls, S, X, updates)
itst = [] if (kvmem is not None and inner_iters) else None
S, X = carry_steps(looper, adapter, e, calls, S, X, updates,
iter_states=itst)
if kvmem is not None and itst:
from kv_memory import arm_memory
arm_memory(kvmem(torch.stack(itst, 1)))
else:
X = torch.cat([X_store, e[:, X_store.shape[1]:]], 1)
rows = torch.arange(B, device=dev)