nils/jspace
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

item 24 pre-registered: d=1 retry with loop-only band-LoRA r=16 (4.8M params, whole band, k=0 bit-exact); trainer/eval gain --bandlora; smoke-tested train+eval

Browse Source 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Nils
2026-07-16 21:44:01 +02:00
co-authored by Claude Fable 5
parent 7a141948fa
commit fe89820e64
4 changed files with 83 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/jobs/zzz_p_rungb_blr.sh
+11
View File
@@ -0,0 +1,11 @@
# gpuq-in: results-loop/star_data.json results-loop/gsm_cot_data.json results-loop/adapter_carrycot_e400.pt
# gpuq-out: results-loop/eval_gsm_carrycot_b1_blr16.json results-loop/train_carrycot_b1_blr16_log.json results-loop/adapter_carrycot_b1_blr16_*.pt results-loop/lora_carrycot_b1_blr16_*.pt
git pull origin main -q 2>/dev/null
P=/home/nils/jspace/.venv/bin/python
export JLENS_MODEL=google/gemma-4-E2B-it LOOP_OUT=/home/nils/jspace/results-loop
cd /home/nils/jspace/scripts
$P train_carry_cot.py --drop-steps 1 --warm-start $LOOP_OUT/adapter_carrycot_e400.pt \
--steps 200 --lr 3e-4 --bandlora 16 --lora-lr 1e-3
$P eval_carry_cot.py --adapter $LOOP_OUT/adapter_carrycot_b1_blr16_e200.pt \
--bandlora $LOOP_OUT/lora_carrycot_b1_blr16_e200.pt \
--tag gsm_carrycot_b1_blr16 --grid 0:0,2:12,2:16 --n 256