Running DeepSeek locally, minus the illusion
The DeepSeek you can run at home is a distill: a Qwen or Llama model trained on the big model's outputs. The full V3/R1 is a 671B mixture-of-experts that needs hundreds of gigabytes of memory even brutally quantized. That distinction sets every expectation that follows, so this page starts there, then gets to what each distill needs and how fast it actually runs.
Updated July 12, 2026
What fits where
| Variant | VRAM | Hardware | Speed |
|---|---|---|---|
| R1 Distill 1.5B | ~1–3 GB (Q4) | Any GPU, most laptops | Fast everywhere |
| R1 Distill 7B / 8B | ~4–8 GB (Q4) | 8 GB cards (RTX 4060 up) | 50–100+ t/s on RTX 40-series |
| R1 Distill 14B | ~8–12 GB (Q4) | 12–16 GB cards | 50–65 t/s on a 4090 |
| R1 Distill 32B | ~18–22 GB (Q4) | 24 GB (RTX 4090) or 48 GB Macs | 25–40 t/s on a 4090; 10–20 t/s on M3/M4 Max |
| R1 Distill 70B | ~36–45 GB (Q4) | 48 GB+: A6000, multi-GPU, high-end Macs | 5–10 t/s on M3/M4 Max |
| Full V3 / R1 (671B MoE) | ~335–400 GB (Q4) | Not consumer hardware | ~2 t/s on dual 4090s with heavy offload |
Figures assume Q4_K_M weights and modest context; add 10–30% for KV cache as context grows. The community consensus sweet spot is the 32B distill on a 24 GB card.
The distill confusion, addressed head-on
This is the most common complaint in every local-DeepSeek thread from 2025 onward. Ollama's tags call the distills deepseek-r1:7b and so on, and plenty of people concluded they were running the frontier reasoning model on a laptop. One much-quoted Reddit thread put it bluntly: Ollama is confusing people by pretending the little models are R1. The distills are genuinely good, and the larger ones carry a real share of R1's reasoning. They also hallucinate more, lose more from quantization on multi-step logic, and are not what the R1 benchmark numbers were measured on. Knowing which one you are running is the difference between a fair evaluation and a disappointed one.
Can you run the real one? The numbers say no
The arithmetic is unforgiving. Full R1 at Q4 wants roughly 335–400 GB for weights alone; the celebrated 1.58-bit dynamic quants compress it to about 131 GB, at which point the KV cache becomes the problem, and pushing past a few thousand tokens of context blows through whatever RAM was left. One documented attempt with 512 GB of system RAM still hit timeouts at 32K context. Dual-4090 rigs with aggressive offloading report around 2 tokens per second, which is a tech demo, not a tool. If you want the most frontier-like behavior your own hardware can deliver in 2026, that means the 32B or 70B distill.
The runners
One command per distill size: deepseek-r1:7b through :70b. The tags made these models accessible and also caused most of the naming confusion below.
Full GGUF support including the dynamic low-bit quants people use for full-model attempts. Pass --jinja if the chat template misbehaves; several early quants needed it.
Apple's array framework, and the fast path on Macs, where unified memory is what makes the 32B and 70B distills reachable at all.
The 7B and 8B distills are comfortable citizens of the 8 GB VRAM class, and CPU-only setups can run the small ones at usable speeds; see running an LLM without a GPU.
Distill or just run Qwen?
The distills are literally Qwen models underneath, so this is a fair question. Pick the distill when raw reasoning and math depth is the job: the R1 training shows, and the MIT license is the cleanest in the open-weight world for building derivatives. Pick a native Qwen when you want breadth, multilingual strength, or reliable tool calling, where the distills' reasoning-first behavior gets in the way. The Qwen local guide covers that side, including the thinking-mode quirks.
Questions people actually ask
Am I actually running R1?
If it loaded on one GPU, no. The full R1 is a 671B-parameter mixture-of-experts model that needs hundreds of gigabytes even quantized. What Ollama serves under deepseek-r1:7b and similar tags are distills: Qwen and Llama models fine-tuned on R1's outputs. They inherit a real share of the reasoning ability and they are what local DeepSeek means in practice, but benchmarks from the full model don't transfer to them.
Which quantization should I pick?
Q4_K_M is the sensible default and what the speed figures on this page assume. Going up to Q5 or Q6 costs roughly proportional memory for a small quality gain; dropping to Q2 or Q3 shows severe degradation, and reasoning chains suffer from aggressive quantization before general chat does, which matters for a model whose whole point is reasoning.
What about the license?
One of DeepSeek's genuine advantages: the R1 line is MIT-licensed and the Qwen-based distills are Apache-2.0, so local and commercial use are both clean. The details, and how that compares to Llama-style licenses, are on the commercially usable AI page.
How much context can I use?
The distills support 128K context on paper, but KV cache grows with every token of it and comes out of the same memory as the weights. A 32B distill that fits a 4090 at short context will not fit with tens of thousands of tokens loaded. Raise context gradually and watch memory rather than setting the maximum because the model card allows it.
Sources
Every runner above is one row in the Awesome Open Source AI registry, which resyncs twice a day from the curated GitHub list. The Inference Engines & Serving category holds the rest of the local-inference toolbox.
