LoRA Speedrun β a public wall-clock leaderboard for fine-tuning techniques
LoRA Speedrun π How fast can you LoRA-fine-tune Qwen2.5-1.5B to β₯ 57% on GSM8K β on a single L40S? This is modded-nanogpt for fine-tuning: a frozen task, frozen hardware, and a public leaderboard of wall-clock records. Every record is independently re-run 3Γ with fresh seeds on identical hardware before it counts.
- βͺLoRA Speedrun π How fast can you LoRA-fine-tune Qwen2.5-1.5B to β₯ 57% on GSM8K β on a single L40S?
- βͺThis is modded-nanogpt for fine-tuning: a frozen task, frozen hardware, and a public leaderboard of wall-clock records.
- βͺEvery record is independently re-run 3Γ with fresh seeds on identical hardware before it counts.
Opening excerpt (first ~120 words) tap to expand
LoRA Speedrun π How fast can you LoRA-fine-tune Qwen2.5-1.5B to β₯ 57% on GSM8K β on a single L40S? This is modded-nanogpt for fine-tuning: a frozen task, frozen hardware, and a public leaderboard of wall-clock records. Every record is independently re-run 3Γ with fresh seeds on identical hardware before it counts. Attempting and verifying are free: official timing runs on a Modal L40S sandbox, and Modal's free monthly compute credits cover full runs β so anyone can compete, and anyone can re-verify any record with one command. Leaderboard Current record: 6m 05s by @Saivineeth147 β Sequence packing + completion-only loss masking, 2 epochs. Same LoRA config as #0; ~2x faster at higher accuracy.
β¦
Excerpt limited to ~120 words for fair-use compliance. The full article is at GitHub.