nicorobinreal/llmspeed

Feel how fast a local LLM runs before you buy the hardware.

Pick a GPU or Mac and a model, and watch the answer stream at the real speed, including the wait before the first token.

RTX 5090 vs Mac Studio M5 Ultra 512GB on GLM-5.3 Flash (Q8). The 5090 can't load it.

• Real-time simulation: text streams at the predicted tokens/sec, after a realistic time to first token for your prompt length
• Race mode: run two setups side by side, e.g. a 5090 against a Mac Studio
• Memory fit: see whether a model fits at each quantization, needs partial offload to system RAM, or won't run at all
• 30 GPUs, Macs and CPUs × 15 popular models, across CUDA, Metal, ROCm, Vulkan and CPU
• Shareable links to any setup or race

Every estimate is labeled with how much to trust it:

Sources. Benchmarks are collected and verified from r/LocalLLaMA, localscore.ai, llama.cpp discussions and more.

The model. When no benchmark exists, llmspeed falls back to first principles:

• Generation speed is bound by memory bandwidth: every token reads the active weights once. For mixture-of-experts models only the active parameters count.
• Prompt processing is bound by compute (FP16 TFLOPS).
• Both use per-backend efficiency coefficients calibrated against the measured benchmarks, plus a slowdown as context grows.
• Partial offload uses a hybrid model: the GPU-resident share streams at GPU bandwidth and the spilled share at an effective system-RAM bandwidth, calibrated from measured offload runs.
• Memory fit counts weights, KV cache and runtime overhead against usable memory: about 75% of unified memory on Macs and 93% of VRAM on discrete GPUs.

The most valuable contribution is a real measurement. To add one, open a PR that appends a row to benchmarks/submissions.csv.

1. Run llama.cpp's benchmark tool on your machine:

   llama-bench -m your-model.gguf -p 512 -n 128

2. Add a row with your results:

   Column	Example
   hardware	NVIDIA RTX 4090 24GB
   model	Llama 3.1 8B Instruct
   quant	Q4_K_M
   backend	CUDA, Metal, ROCm, Vulkan or CPU
   engine_version	llama.cpp b6500
   gen_tps	128.4 (the tg128 result)
   prefill_tps	9850 (the pp512 result)
   prompt_tokens	512
   source_url	a link to your post, issue or gist with the full output
   notes	anything unusual: power limits, flash attention, offload, …

3. In the PR description, paste the full llama-bench output.

Results from other engines (MLX, vLLM, ExLlama, …) are welcome too; just name the engine in engine_version. Want a model or GPU added? Open an issue.

npm install
npm run dev      # local dev server
npm run build    # production build + prerendered landing pages + sitemap

Built with Vite, Preact and TypeScript. The site is fully static and deployed to GitHub Pages on every push to main. All predictions live in data/app-data.json, which is generated by the benchmark pipeline.

• Single device only. Multi-GPU setups (tensor or layer split) aren't simulated yet.
• Estimates, not guarantees. Drivers, engine versions, flash attention, power limits and thermals can all move real numbers by 10–30%.
• Projected numbers for unreleased hardware will be replaced with measurements as they appear.

MIT. Benchmark data credits go to the r/LocalLLaMA and localscore.ai communities and everyone who has shared their numbers.