Which open model should you run in 2026?
A practical shortlist for agents, coding, multimodal work, and smaller private deployments.
The answer
Choose for the workload, not the leaderboard. Small dense models are the practical default; large mixture-of-experts models make sense when quality or agent depth justifies multi-GPU inference.
The short list
- Always-on agents: NVIDIA Nemotron 3.5 Lightning is designed around long-running, tool-using workflows.
- Local coding and multimodal work: Muse Glimmer 30B is a compact multimodal option aimed at coding agents.
- General multilingual workloads: Qwen is the ecosystem base with the widest derivative activity on Hugging Face in 2026.
- Lightweight deployment: Gemma 4 spans small dense and mixture-of-experts sizes with text, audio, image, and long-context support.
- Frontier experimentation: Kimi K3 and Qwen3.8-class models require serious multi-GPU capacity and operational tuning.
What matters more than the model name
Check the licence, supported precision, context length, tokenizer and tool-call template before sizing hardware. A model that fits but cannot produce reliable structured tool calls is not the right agent model.
Benchmark with your own prompts. Public scores do not include your retrieval quality, latency target, language mix, or the cost of keeping enough replicas warm.
A sensible first deployment
Start with a model in the 8B to 35B range, a tested 4-bit or 8-bit quantization, and one GPU. Measure time to first token, output tokens per second, peak VRAM and task success before moving upward.