nils/jspace
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

item 28 pre-registered (Nils's variant): teacher-state distillation — frozen full-cot teacher's band-exit state at step end, cosine into burst s^10; λ amended 1.0→5.0 pre-run (smoke: baseline cos-dist 0.113)

Browse Source 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Nils
2026-07-17 00:46:19 +02:00
co-authored by Claude Fable 5
parent 62ce30e557
commit bbf8a02c6b
3 changed files with 96 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/jobs/zzz_sa_rungb_ts.sh
+14
View File
@@ -0,0 +1,14 @@
# gpuq-in: results-loop/star_data.json results-loop/gsm_cot_data.json results-loop/adapter_carrycot_e400.pt
# gpuq-out: results-loop/eval_gsm_carrycot_b1_ts*.json results-loop/train_carrycot_b1_ii10_ts50_p0_log.json results-loop/adapter_carrycot_b1_ii10_ts50_p0_e200.pt
git pull origin main -q 2>/dev/null
P=/home/nils/jspace/.venv/bin/python
export JLENS_MODEL=google/gemma-4-E2B-it LOOP_OUT=/home/nils/jspace/results-loop
cd /home/nils/jspace/scripts
$P train_carry_cot.py --drop-steps 1 --pause-per-step 0 --base-pauses 0 \
--inner-iters 10 --teachstate 5.0 \
--warm-start $LOOP_OUT/adapter_carrycot_e400.pt --steps 200 --lr 3e-4
A=$LOOP_OUT/adapter_carrycot_b1_ii10_ts50_p0_e200.pt
$P eval_carry_cot.py --adapter $A --tag gsm_carrycot_b1_ts10 \
--grid 0:0,2:0 --n 256 --inner-iters 10
$P eval_carry_cot.py --adapter $A --tag gsm_carrycot_b1_ts10_ablate \
--grid 2:0 --n 256 --inner-iters 0