nils/jspace
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

26B-A4B lens mapped: 30 layers (not 48), same early-persist/terminal-motor family as E4B+31B; patched save delivered full regimes.pt incl. per-layer ignition; 5-scale figure complete

Browse Source 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Nils
2026-07-16 11:03:38 +02:00
co-authored by Claude Fable 5
parent 20f7014f5b
commit 4e744e7d94
4 changed files with 62 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
results/REGIMES.json
+5 -5
View File
@@ -48,12 +48,12 @@
},
"google/gemma-4-26B-A4B-it": {
"revision": "01e5b3ee840d3a9e0b0b493c593e85398a30ef75",
"num_layers": null,
"hidden": null,
"num_layers": 30,
"hidden": 2816,
"workspace_band": null,
"kv_share_start": null,
"jbar": "pending (H100 pipeline v4)",
"scan": "pending",
"status": "queued after 31B; MoE \u2014 lens validity check required before trusting band"
"jbar": "bucket:results-lens/jbar_26b_a4b.pt (256/256 prompts)",
"scan": "results/exp4_26b_a4b.log + results/regimes_26b_a4b.pt (saved directly \u2014 first scan through the patched exp4_regimes.py; includes full per-layer ignition tensors)",
"status": "mapped 2026-07-16, PROVISIONAL: only 30 layers (MoE, A4B active; num_kv_shared_layers=0 in config \u2014 all entrances structurally live). Same family pattern as E4B/31B, NOT E2B: persistence bump EARLY and thin (L5-6, 0.33/0.25), then flat ~0.03-0.08; sensor lobe L10-18 (peak 0.21@L13); motor weak and only terminal (0.11@L29); content high throughout mid (0.6-0.93). Ignition heavily saturated (many +/-1.00 cells) but write-responsive late: L22-26 commitment flips to the written concept at w>=0.5 on all three pairs (dog/piano L26 -1.00 -> +1.00). MoE lens-validity check still required before trusting any band; no band entered."
}
}
results/exp4_26b_a4b.log
+57
View File
@@ -0,0 +1,57 @@
Loading weights: 0%| | 0/1013 [00:00<?, ?it/s]
Loading weights: 100%|██████████| 1013/1013 [00:00<00:00, 11419.56it/s]
layer | sensor(top==cur) motor(top==next) persist(top10 Jaccard) content-word
L 0 | 0.00 | 0.00 | 0.07 | 0.82
L 1 | 0.00 | 0.00 | 0.08 | 0.48
L 2 | 0.00 | 0.00 | 0.18 | 0.64
L 3 | 0.00 | 0.00 | 0.08 | 0.61
L 4 | 0.00 | 0.00 | 0.08 | 0.87
L 5 | 0.00 | 0.00 | 0.33 | 0.93
L 6 | 0.00 | 0.00 | 0.25 | 0.93
L 7 | 0.00 | 0.00 | 0.15 | 0.57
L 8 | 0.07 # | 0.00 | 0.05 | 0.40
L 9 | 0.07 # | 0.00 | 0.06 | 0.79
L10 | 0.18 ### | 0.01 | 0.04 | 0.66
L11 | 0.17 ### | 0.01 | 0.04 | 0.64
L12 | 0.14 ## | 0.01 | 0.04 | 0.72
L13 | 0.21 #### | 0.02 | 0.05 | 0.64
L14 | 0.16 ### | 0.02 | 0.03 | 0.65
L15 | 0.12 ## | 0.03 | 0.04 | 0.59
L16 | 0.12 ## | 0.02 | 0.03 | 0.72
L17 | 0.14 ## | 0.04 | 0.02 | 0.62
L18 | 0.12 ## | 0.02 | 0.03 | 0.60
L19 | 0.07 # | 0.04 | 0.04 | 0.71
L20 | 0.09 # | 0.01 | 0.05 | 0.79
L21 | 0.09 # | 0.02 | 0.04 | 0.87
L22 | 0.09 # | 0.04 | 0.06 | 0.91
L23 | 0.10 # | 0.05 | 0.08 | 0.87
L24 | 0.08 # | 0.06 # | 0.08 | 0.88
L25 | 0.07 # | 0.06 # | 0.07 | 0.80
L26 | 0.07 # | 0.07 # | 0.08 | 0.80
L27 | 0.07 # | 0.07 # | 0.07 | 0.85
L28 | 0.06 # | 0.09 # | 0.05 | 0.80
L29 | 0.00 | 0.11 ## | 0.05 | 0.38
ignition: commitment C=(P1-P2)/(P1+P2) at read pos, by layer
pair | w: 0.00 0.25 0.50 0.75 1.00
dog/piano L 2 | +1.00 +1.00 +1.00 +1.00 +1.00
dog/piano L 6 | -0.74 -0.75 -0.83 -0.80 -0.81
dog/piano L10 | +0.58 +0.63 +0.91 +0.92 +0.92
dog/piano L14 | +0.85 +0.17 +0.69 +0.81 +0.15
dog/piano L18 | -0.00 -0.01 +0.66 +0.94 +0.67
dog/piano L22 | +0.94 +0.93 +0.98 +1.00 +1.00
dog/piano L26 | -1.00 -1.00 +0.39 +1.00 +1.00
ocean/violin L 2 | +1.00 +1.00 +1.00 +1.00 +1.00
ocean/violin L 6 | +1.00 +1.00 +1.00 +1.00 +1.00
ocean/violin L10 | +0.00 +0.00 +0.00 +0.00 +0.00
ocean/violin L14 | +0.00 +0.00 +0.00 +0.00 +0.00
ocean/violin L18 | +0.00 +0.00 +0.29 +0.27 +0.00
ocean/violin L22 | +0.98 +0.28 +1.00 +1.00 +1.00
ocean/violin L26 | -0.56 -0.26 +0.98 +1.00 +1.00
dragon/bicycle L 2 | +0.00 +0.00 +0.00 +0.00 +0.00
dragon/bicycle L 6 | -1.00 -1.00 -1.00 -1.00 -1.00
dragon/bicycle L10 | +0.00 +0.00 +0.00 +0.00 +0.00
dragon/bicycle L14 | +0.00 +0.00 +0.00 +0.00 +0.00
dragon/bicycle L18 | +0.00 +0.00 +0.00 +0.00 +0.00
dragon/bicycle L22 | +0.98 +0.40 +1.00 +0.86 +0.91
results/regimes_26b_a4b.pt
BIN
View File
Binary file not shown.
results/regimes_all_models.png
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 188 KiB

After

Width:  |  Height:  |  Size: 203 KiB