26B-A4B lens mapped: 30 layers (not 48), same early-persist/terminal-motor family as E4B+31B; patched save delivered full regimes.pt incl. per-layer ignition; 5-scale figure complete
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -48,12 +48,12 @@
|
|||||||
},
|
},
|
||||||
"google/gemma-4-26B-A4B-it": {
|
"google/gemma-4-26B-A4B-it": {
|
||||||
"revision": "01e5b3ee840d3a9e0b0b493c593e85398a30ef75",
|
"revision": "01e5b3ee840d3a9e0b0b493c593e85398a30ef75",
|
||||||
"num_layers": null,
|
"num_layers": 30,
|
||||||
"hidden": null,
|
"hidden": 2816,
|
||||||
"workspace_band": null,
|
"workspace_band": null,
|
||||||
"kv_share_start": null,
|
"kv_share_start": null,
|
||||||
"jbar": "pending (H100 pipeline v4)",
|
"jbar": "bucket:results-lens/jbar_26b_a4b.pt (256/256 prompts)",
|
||||||
"scan": "pending",
|
"scan": "results/exp4_26b_a4b.log + results/regimes_26b_a4b.pt (saved directly \u2014 first scan through the patched exp4_regimes.py; includes full per-layer ignition tensors)",
|
||||||
"status": "queued after 31B; MoE \u2014 lens validity check required before trusting band"
|
"status": "mapped 2026-07-16, PROVISIONAL: only 30 layers (MoE, A4B active; num_kv_shared_layers=0 in config \u2014 all entrances structurally live). Same family pattern as E4B/31B, NOT E2B: persistence bump EARLY and thin (L5-6, 0.33/0.25), then flat ~0.03-0.08; sensor lobe L10-18 (peak 0.21@L13); motor weak and only terminal (0.11@L29); content high throughout mid (0.6-0.93). Ignition heavily saturated (many +/-1.00 cells) but write-responsive late: L22-26 commitment flips to the written concept at w>=0.5 on all three pairs (dog/piano L26 -1.00 -> +1.00). MoE lens-validity check still required before trusting any band; no band entered."
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -0,0 +1,57 @@
|
|||||||
|
|
||||||
|
Loading weights: 0%| | 0/1013 [00:00<?, ?it/s]
|
||||||
|
Loading weights: 100%|██████████| 1013/1013 [00:00<00:00, 11419.56it/s]
|
||||||
|
layer | sensor(top==cur) motor(top==next) persist(top10 Jaccard) content-word
|
||||||
|
L 0 | 0.00 | 0.00 | 0.07 | 0.82
|
||||||
|
L 1 | 0.00 | 0.00 | 0.08 | 0.48
|
||||||
|
L 2 | 0.00 | 0.00 | 0.18 | 0.64
|
||||||
|
L 3 | 0.00 | 0.00 | 0.08 | 0.61
|
||||||
|
L 4 | 0.00 | 0.00 | 0.08 | 0.87
|
||||||
|
L 5 | 0.00 | 0.00 | 0.33 | 0.93
|
||||||
|
L 6 | 0.00 | 0.00 | 0.25 | 0.93
|
||||||
|
L 7 | 0.00 | 0.00 | 0.15 | 0.57
|
||||||
|
L 8 | 0.07 # | 0.00 | 0.05 | 0.40
|
||||||
|
L 9 | 0.07 # | 0.00 | 0.06 | 0.79
|
||||||
|
L10 | 0.18 ### | 0.01 | 0.04 | 0.66
|
||||||
|
L11 | 0.17 ### | 0.01 | 0.04 | 0.64
|
||||||
|
L12 | 0.14 ## | 0.01 | 0.04 | 0.72
|
||||||
|
L13 | 0.21 #### | 0.02 | 0.05 | 0.64
|
||||||
|
L14 | 0.16 ### | 0.02 | 0.03 | 0.65
|
||||||
|
L15 | 0.12 ## | 0.03 | 0.04 | 0.59
|
||||||
|
L16 | 0.12 ## | 0.02 | 0.03 | 0.72
|
||||||
|
L17 | 0.14 ## | 0.04 | 0.02 | 0.62
|
||||||
|
L18 | 0.12 ## | 0.02 | 0.03 | 0.60
|
||||||
|
L19 | 0.07 # | 0.04 | 0.04 | 0.71
|
||||||
|
L20 | 0.09 # | 0.01 | 0.05 | 0.79
|
||||||
|
L21 | 0.09 # | 0.02 | 0.04 | 0.87
|
||||||
|
L22 | 0.09 # | 0.04 | 0.06 | 0.91
|
||||||
|
L23 | 0.10 # | 0.05 | 0.08 | 0.87
|
||||||
|
L24 | 0.08 # | 0.06 # | 0.08 | 0.88
|
||||||
|
L25 | 0.07 # | 0.06 # | 0.07 | 0.80
|
||||||
|
L26 | 0.07 # | 0.07 # | 0.08 | 0.80
|
||||||
|
L27 | 0.07 # | 0.07 # | 0.07 | 0.85
|
||||||
|
L28 | 0.06 # | 0.09 # | 0.05 | 0.80
|
||||||
|
L29 | 0.00 | 0.11 ## | 0.05 | 0.38
|
||||||
|
|
||||||
|
ignition: commitment C=(P1-P2)/(P1+P2) at read pos, by layer
|
||||||
|
pair | w: 0.00 0.25 0.50 0.75 1.00
|
||||||
|
dog/piano L 2 | +1.00 +1.00 +1.00 +1.00 +1.00
|
||||||
|
dog/piano L 6 | -0.74 -0.75 -0.83 -0.80 -0.81
|
||||||
|
dog/piano L10 | +0.58 +0.63 +0.91 +0.92 +0.92
|
||||||
|
dog/piano L14 | +0.85 +0.17 +0.69 +0.81 +0.15
|
||||||
|
dog/piano L18 | -0.00 -0.01 +0.66 +0.94 +0.67
|
||||||
|
dog/piano L22 | +0.94 +0.93 +0.98 +1.00 +1.00
|
||||||
|
dog/piano L26 | -1.00 -1.00 +0.39 +1.00 +1.00
|
||||||
|
ocean/violin L 2 | +1.00 +1.00 +1.00 +1.00 +1.00
|
||||||
|
ocean/violin L 6 | +1.00 +1.00 +1.00 +1.00 +1.00
|
||||||
|
ocean/violin L10 | +0.00 +0.00 +0.00 +0.00 +0.00
|
||||||
|
ocean/violin L14 | +0.00 +0.00 +0.00 +0.00 +0.00
|
||||||
|
ocean/violin L18 | +0.00 +0.00 +0.29 +0.27 +0.00
|
||||||
|
ocean/violin L22 | +0.98 +0.28 +1.00 +1.00 +1.00
|
||||||
|
ocean/violin L26 | -0.56 -0.26 +0.98 +1.00 +1.00
|
||||||
|
dragon/bicycle L 2 | +0.00 +0.00 +0.00 +0.00 +0.00
|
||||||
|
dragon/bicycle L 6 | -1.00 -1.00 -1.00 -1.00 -1.00
|
||||||
|
dragon/bicycle L10 | +0.00 +0.00 +0.00 +0.00 +0.00
|
||||||
|
dragon/bicycle L14 | +0.00 +0.00 +0.00 +0.00 +0.00
|
||||||
|
dragon/bicycle L18 | +0.00 +0.00 +0.00 +0.00 +0.00
|
||||||
|
dragon/bicycle L22 | +0.98 +0.40 +1.00 +0.86 +0.91
|
||||||
Binary file not shown.
Binary file not shown.
|
Before Width: | Height: | Size: 188 KiB After Width: | Height: | Size: 203 KiB |
Reference in New Issue
Block a user