The best local LLMs in 2026, by the hardware you have

There is no single "best" local LLM. The best model is the largest good one that fits comfortably in your device's memory and runs at a speed you can live with. So this guide organises the options by hardware tier, uses published sizes rather than guesses, and labels every benchmark with who produced it.

We make Loci, an app that runs these models on iPhone, iPad, Android, Mac and Windows, so some speed figures come from our own users' devices. We say where each number comes from.

Model sizes and tags come from the Ollama library.

How to choose a local model

Four things decide what works:

1. Memory. The model must fit in RAM (or unified memory on a Mac), with room for the system and the conversation. A model's download size is a good proxy. Quantization (compressing the weights) is what makes this possible.
2. Speed. Measured in tokens per second. Larger models are slower on the same device.
3. Task. Chat, coding, reasoning, images, long documents, or many languages.
4. License. Check it if you build a product on a model.

The families worth knowing

Gemma 4 (Google DeepMind)

Gemma 4 is a family of multimodal models that take text and image input and produce text. Per the Ollama library, sizes are:

Variant	Ollama tag	Download	Context	Notes
E2B ("effective" 2B)	gemma4:e2b	4.6-7.5 GB	128K	Text, image, audio input; edge devices
E4B	gemma4:e4b	6.6-9.5 GB	128K	Text, image, audio
12B	gemma4:12b	7.7-8.0 GB	256K	Text, image
26B A4B (MoE)	gemma4:26b	16-19 GB	256K	25.2B total parameters, 3.8B active
31B dense	gemma4:31b	19-20 GB	256K	Text, image

All models have configurable thinking modes and native support for a system prompt.

Qwen3.5 (Alibaba)

Qwen3.5 is an open-source multimodal family with sizes from 0.8B to 122B parameters, 256K context, and support for 201 languages and dialects (as stated by Qwen).

Variant	Download (Ollama)
0.8B	1.2-1.3 GB
2B	2.7-3.1 GB
4B	3.3-4.0 GB
9B	6.6-7.6 GB
27B	17-20 GB
35B	22 GB
122B	81 GB

gpt-oss (OpenAI)

OpenAI's open-weight models come in 20B (14 GB on Ollama, 128K context) and 120B (65 GB). They are released under Apache 2.0, support configurable reasoning effort (low, medium, high) and are post-trained with MXFP4 quantization. Ollama says the 20B model can run on systems with as little as 16 GB of memory, and the 120B model fits on a single 80 GB GPU. They are text-only on Ollama.

Best by hardware tier

Our recommendations below are judgement calls based on the sizes above; they are not benchmark rankings.

Phones and 8 GB computers: Gemma 4 E2B, Qwen3.5 2B or 4B

These are the smallest models that still hold a useful conversation. Real speeds from Loci devices, September 30 to October 30, 2026:

Device	Model	Median speed
iPhone 17 Pro	Gemma 4 E2B	65 tok/s (26 phones, 142 answers)
iPhone 15	Gemma 4 E2B	34 tok/s (26 phones)
iPhone 12	Qwen3.5 2B	13 tok/s (14 phones)
Galaxy S25 Ultra	Gemma 4 E2B	19 tok/s (58 phones)
Pixel 8	Gemma 4 E2B	9.9 tok/s (13 phones)
M2 Mac, 8 GB	Qwen3.5 2B	44 tok/s (2 Macs)

Android phones in this table are slower than recent iPhones. Mac samples are two machines each, so treat those as indications only. The speed pages carry the current medians and sample sizes.

16 GB computers: Gemma 4 E4B, Qwen3.5 9B, Gemma 4 12B

Sizes of 6.6-9.5 GB leave comfortable room on a 16 GB machine. In our data an M5 Mac with 16 GB ran Gemma 4 E4B at a median 33 tok/s (2 Macs).

24-32 GB computers: Gemma 4 12B, gpt-oss 20B, Gemma 4 26B

In our data an M5 Pro Mac with 24 GB ran Gemma 4 12B at 31 tok/s (4 Macs), an M2 Pro with 32 GB at 18 tok/s (2 Macs), and an M5 with 24 GB at 15 tok/s (2 Macs). Differences between chips are visible, but samples are small. gpt-oss 20B (14 GB) suits 16 GB+ machines per Ollama; the 26B MoE (16-19 GB) activates only 3.8B parameters per token.

64 GB and up: Gemma 4 31B, Qwen3.5 27B/35B, gpt-oss 120B

At 19-22 GB these are workstation-class models; gpt-oss 120B needs about 65 GB, and Qwen3.5 122B 81 GB. In our data an M5 Max Mac with 128 GB ran Ternary Bonsai 2 27B at 40 tok/s (2 Macs; the Ollama-listed models above were not what was measured there), so this is anecdotal. We have no data here worth ranking.

What do the benchmarks say?

Benchmarks are the vendor's own, run on instruction-tuned models, and reported on the Gemma 4 model card (as copied on ollama.com). We have not re-run them. [Vendor-reported, October 2026]

Benchmark	Gemma 4 31B	26B A4B	E4B	E2B	Gemma 3 27B
MMLU Pro	85.2%	82.6%	69.4%	60.0%	67.6%
LiveCodeBench v6	80.0%	77.1%	52.0%	44.0%	29.1%
GPQA Diamond	84.3%	82.3%	58.6%	43.4%	42.4%

What this shows: size matters a lot (31B vs E2B), and Google claims big gains over its previous generation. What it doesn't show: how any model feels for your tasks, or how these compare with other vendors' models, because each vendor chooses its own tests. For other families, check each vendor's own model card.

Best for coding?

We don't have our own coding benchmark. Google reports LiveCodeBench v6 of 80.0% for Gemma 4 31B and 77.1% for the 26B MoE (above), and Ollama lists coding agents (Claude Code, OpenCode and others) as applications for these models. For serious coding, pick the biggest model your machine can run at a speed you can tolerate.

How to run any of these

• Easiest: an app that picks the model for you, like Loci (free, no account; iPhone, iPad, Android, Windows 11, and Apple silicon Macs on macOS 26.2 or later) or Jan on desktop.
• More control: LM Studio, or GPT4All.
• Command line: ollama run gemma4:e4b (swap the tag).

See our guides on running an LLM locally and running ChatGPT-style AI locally, and the comparisons if you are choosing an app.

FAQ

What is the best local LLM overall?

There isn't one. By vendor-reported benchmarks Gemma 4 31B is the strongest of the models we list, but it needs 19-20 GB for the download alone. The best model is the biggest good one your device runs at a comfortable speed.

What is the best local LLM for 8 GB of RAM?

A 2B-4B class model: Gemma 4 E2B (4.6-7.5 GB on Ollama) or Qwen3.5 2B/4B (2.7-4.0 GB). Close other apps to leave headroom, and use the app's fit warning.

What is the best local LLM for 16 GB of RAM?

Gemma 4 E4B, Qwen3.5 9B or Gemma 4 12B fit comfortably; gpt-oss 20B (14 GB) is possible according to Ollama but tight.

Can I run gpt-oss locally?

Yes. ollama run gpt-oss:20b needs about 16 GB of memory, per Ollama; the 120B model needs about 65 GB.

Is there a private AI app for iPhone that works offline?

Yes. Loci runs a downloaded model on the iPhone itself. After that download, typed chat works with no connection and no account. The model has to be small enough for the phone. The speed table above is the measured range.

Are local LLMs free?

The models above are open-weight, so you can download them without paying. Check each license for commercial use. gpt-oss is Apache 2.0.

How fast are local models on a phone?

On the published speed table for September 30 to October 30, 2026, Gemma 4 E2B runs at 65 tok/s on the iPhone 17 Pro (26 phones) and 34 tok/s on the iPhone 15 (26 phones). The Galaxy S25 Ultra in the same table is at 19 tok/s (58 phones), and the Pixel 8 at 9.9 tok/s (13 phones).