How to run a ChatGPT-style AI on your own computer

The short version. You can't run ChatGPT itself on your own device: OpenAI hosts it on its servers. What you can run are open-weight language models that behave like it: you type, they answer, and everything stays on your computer. OpenAI itself released two open-weight models, gpt-oss-20b and gpt-oss-120b, under the Apache 2.0 license. Google (Gemma) and Alibaba (Qwen) publish others. This guide shows three ways to run them, what hardware you need, and where local models fall short.

What "running ChatGPT locally" really means

ChatGPT is a product: a chat interface on top of large models that run in data centres. A local alternative has two parts you choose:

1. A model: a file of "weights" that you download once. Examples: Gemma 4, Qwen3.5, gpt-oss.
2. An app: the software that loads the model and gives you a chat window.

After the download, the model runs on your device's processor and memory. Your messages don't need to travel anywhere, and chat works without an internet connection.

What you give up (and keep)

Be realistic before you start:

• Local models are smaller than the models behind cloud chatbots. The biggest open models need serious hardware (Ollama lists the largest gpt-oss at 65 GB and Qwen3.5 122B at 81 GB). What fits on a laptop or a phone is far smaller, so expect weaker results on hard reasoning, long documents and obscure facts.
• No live information by default. A local model knows only what it was trained on. Some apps can add web search when you switch it on; that part does use the internet.
• You keep privacy, offline use and zero per-message cost. For everyday writing, summarising, brainstorming and Q&A, a modern small model is often enough.

What hardware do you need?

The practical limit is memory. Ollama's model library publishes the download size of each model, which is a good guide to how much memory you need (the model must fit, with room left for your system and the conversation).

Model	Ollama tag	Download size	Context window
Qwen3.5 2B	qwen3.5:2b	2.7-3.1 GB	256K
Gemma 4 E2B	gemma4:e2b	4.6-7.5 GB	128K
Qwen3.5 9B	qwen3.5:9b	6.6-7.6 GB	256K
Gemma 4 E4B	gemma4:e4b	6.6-9.5 GB	128K
Gemma 4 12B	gemma4:12b	7.7-8.0 GB	256K
gpt-oss 20B	gpt-oss:20b	14 GB	128K
Gemma 4 26B (MoE, 4B active)	gemma4:26b	16-19 GB	256K
Gemma 4 31B	gemma4:31b	19-20 GB	256K
gpt-oss 120B	gpt-oss:120b	65 GB	128K

Two notes. First, the ranges are Ollama's: sizes differ by quantization format. Second, OpenAI says gpt-oss-20b runs on systems with as little as 16 GB of memory.

A rule of thumb (our estimate, not a benchmark): match the model's download size to roughly half of your free memory or less. 8 GB laptop or recent phone: a 2B-4B-class model. 16 GB: up to roughly a 9B-12B-class model. 32 GB or more: the 20B-31B class.

How fast will it be?

Speed is measured in tokens per second (a token is roughly three-quarters of a word). Anything above about 10 is comfortable to read live; below 5 feels slow.

We publish speeds that Loci users' devices have reported in the last 30 days, on our speed pages. These are our own data, with small samples for some devices (the page shows the sample size). From September 30 to October 30, 2026:

Device	Model	Median speed	Sample
iPhone 17 Pro	Gemma 4 E2B	65 tok/s	26 phones, 142 answers
Galaxy S25 Ultra	Gemma 4 E2B	19 tok/s	58 phones
Pixel 8	Gemma 4 E2B	9.9 tok/s	13 phones
M5 Mac, 16 GB	Gemma 4 E4B	33 tok/s	2 Macs
M2 Mac, 8 GB	Qwen3.5 2B	44 tok/s	2 Macs

The Mac figures come from only two machines each, so treat them as indications. See the full speed tables.

Three ways to do it

Option 1: An app that sets everything up (easiest)

Install an app that picks a model for your device. Loci (our app) does this on first launch on Mac (Apple silicon; the direct download needs macOS 26.2 or later), Windows 11, iPhone, iPad and Android. It is free with no account. Jan does something similar on desktop. Steps: install; open it; wait for the first model to download; start chatting. Best for: anyone who doesn't want to choose or configure anything.

Option 2: A desktop model app (more control)

Install LM Studio, Jan or GPT4All, search the built-in catalogue for a model that suits your memory, download it and chat. LM Studio is free and has detailed controls; Jan and GPT4All are open source. Steps: install; search "Gemma 4" or "Qwen3.5"; pick a size from the table above; download; load; chat. Best for: people who like to compare models.

Option 3: Ollama (command line)

Ollama is a developer-oriented tool with a free local runtime. After installing it from ollama.com/download:

`` ollama run gemma4:e4b ``

downloads the model on first run and opens a chat in the terminal. Replace the tag with any in the table. Best for: developers, scripts, servers and Linux.

Make it feel more like ChatGPT

• Web search: switch it on in apps that offer it, and expect that query to leave your device. In Loci it is one switch and queries go to search providers directly. Elsewhere it may need plugins or an account.
• Documents and images: many current open models accept images (Gemma 4 and Qwen3.5 are both listed as accepting text and image input on Ollama). Apps differ on how they let you attach files.
• Voice: Loci includes on-device dictation; other apps vary.
• Thinking mode: Gemma 4 has configurable thinking modes, and gpt-oss has adjustable reasoning effort (low, medium, high). More thinking is slower but can help on hard problems.

Is a local model as good as ChatGPT?

Not on the hardest problems. A local model is smaller than the models behind ChatGPT, and ChatGPT is stronger there, and at making images. A local model is still enough for a lot of everyday writing, summaries and questions, and it stays on your device. Vendor-reported results show how fast small models are improving: Google reports that Gemma 4 31B scores 85.2% on MMLU Pro and 80.0% on LiveCodeBench v6, and the small E4B scores 69.4% and 52.0%. [Vendor-reported, from the Gemma 4 model card as shown on ollama.com; these are not our tests, and we make no comparison to ChatGPT.] The gap between sizes is large, which is why matching the model to your hardware matters.

If you want the detailed trade-offs against ChatGPT itself, read Loci vs ChatGPT.

FAQ

Can I run ChatGPT locally?

No. ChatGPT runs on OpenAI's servers. You can run open-weight models, including OpenAI's own gpt-oss models, on your own device.

Do I need a GPU?

Not necessarily. Modern phones and laptops with Apple silicon, recent Intel/AMD chips or Snapdragon can run small models on the processor or built-in graphics; a faster GPU helps with larger ones.

How much RAM do I need?

As a rough guide (our estimate), the model's download size should fit comfortably in memory with room to spare: about 8 GB for 2B-4B models, 16 GB for 9B-12B models and 32 GB or more for 20B-31B models.

Is running AI locally free?

Yes for the apps named here (LM Studio, Jan, GPT4All, Ollama's local runtime, Loci), apart from your electricity and storage. Ollama's cloud models are paid.

Can I run it on my phone?

Yes. Loci runs models on iPhone, iPad and Android, and PocketPal is another phone app. Newer phones with more memory run bigger models faster.

Is there a private AI app for iPhone that works offline?

Yes. Loci runs the model on the iPhone. After the download, typed chat works with no connection and no account. It is not as capable as ChatGPT on the hardest questions.

Is it private?

Local chat doesn't need to send your text anywhere. Turning on web search or cloud features changes that, so check each app's settings.