# What is local AI, and how do you run it?

> Local AI runs a model on your own phone or computer instead of a company's servers. What it does, what you need, and the simplest way to start, for free.

Canonical: https://askloci.ai/guides/what-is-local-ai/
Last updated: 2026-11-15

**Most AI you have used runs in a data center.** You type, your words travel to a company's servers, and the answer travels back. Local AI skips the trip: the model is a file on your phone or computer, and your own device does the work. You will also see it called on-device AI, offline AI, a local LLM, local artificial intelligence or, as one word, localai.

This page explains what local AI is, what it can do, what you need, and how to start. We make Loci, one of the apps mentioned, and we say so where it comes up.

## Local AI at a glance

| Question | Short answer |
| --- | --- |
| What is it? | An AI model that is stored on your own device and runs there. |
| What does it do? | Answers questions, drafts and rewrites text, summarises, explains, helps with code and, with the right model, reads documents and images. |
| Where does it run? | Recent iPhones, iPads and Android phones, Macs with Apple silicon, Windows 11 PCs and Linux computers. |
| What do you need? | An app, one model download of a few gigabytes, and enough memory to hold it. |
| Does it need the internet? | Only to download the model and for extras such as web search. Chat works offline after that. |
| What does it cost? | The apps in this guide are free for local use, and open models are free to download. |
| What is the catch? | The model has to fit on one device, so it is smaller than the biggest cloud models and weaker on the hardest problems. |

## How local AI works

Three parts work together:

- **The model** is the AI itself: a large file of numbers, called weights, learned during training. Open models you can download include Gemma from Google, Qwen from Alibaba and gpt-oss from OpenAI. A model's size is given in parameters, such as 2B or 12B for two or twelve billion. Bigger usually means smarter and slower.
- **The engine** is the software that runs the model on your chip. The two you will meet most are llama.cpp, which reads GGUF files and runs on almost any hardware, and MLX, which Apple makes for its own chips. [GGUF vs MLX](https://askloci.ai/guides/gguf-vs-mlx/) covers the difference.
- **The app** is the part you see. It downloads the model, loads it into memory and gives you a place to type. Loci, LM Studio, Ollama and Jan are apps.

When you send a message, four things happen, all on your device:

1. The app has already downloaded the model file, once.
2. It loads the model into memory.
3. Your message is split into tokens, which are small pieces of words, and fed to the model.
4. The model writes the reply one token at a time.

No step needs a server. That is the difference between local AI and cloud AI.

## Local AI vs cloud AI

|  | Local AI | Cloud AI |
| --- | --- | --- |
| Where the model runs | On your phone, tablet or computer | On the provider's servers |
| Internet | Chat works without it after the download | Every answer needs a connection |
| Your prompts | Processed on your device | Sent to the provider, under its retention and training terms |
| Account | Often none | Usually required |
| Cost | Free apps and free open models; you supply the device | Free tiers with limits, or a subscription |
| Strength | Limited by your device's memory | The largest models, on server hardware |
| Live information | Only through a separate online search | Often built in |

Neither is better at everything. Local AI wins on privacy, offline use and having no usage caps. Cloud AI wins on the hardest reasoning and on anything that needs today's information. [How a local model compares with ChatGPT](https://askloci.ai/guides/run-chatgpt-locally/) goes deeper.

## What you need to run local AI

**A device with enough memory.** Memory, more than processor speed, decides which model you can run. This table is our estimate from the download sizes Ollama lists, not a benchmark:

| Device | Model size that fits | Examples |
| --- | --- | --- |
| Recent phone, or 8 GB computer | 2B-4B | Gemma 4 E2B, Qwen3.5 2B or 4B |
| 16 GB computer | 9B-12B | Gemma 4 E4B, Qwen3.5 9B, Gemma 4 12B |
| 24-32 GB computer | 12B-26B | Gemma 4 12B, gpt-oss 20B, Gemma 4 26B |
| 64 GB and up | 27B and larger | Gemma 4 31B, Qwen3.5 27B/35B |

Three things people often miss:

- **The download size is not the whole memory bill.** The conversation takes memory too, and long chats and long documents take more. Other open apps share the same memory.
- **You do not need a graphics card.** A recent phone, an Apple silicon Mac or a Windows 11 laptop runs small models without one. On a PC with a separate graphics card, a model runs fastest when all of it fits in the card's own memory.
- **Phones and thin laptops get warm.** A long answer from a large model can heat a device and slow it down. A smaller model helps.

[How much RAM you need](https://askloci.ai/guides/how-much-ram-for-local-ai/) shows what each memory size runs in practice, [best local LLMs](https://askloci.ai/guides/best-local-llms/) names a model for each memory tier, and [which model fits your device](https://askloci.ai/can-my-device-run/) checks a specific phone or Mac.

## How to run local AI: the simplest way

The shortest route is an app that chooses the model for you. With Loci there is one step: [download Loci](https://askloci.ai/download/) and open it. It picks a model that fits your device and downloads it once. Then you ask a question, and the answer is written on your device.

Loci is free on iPhone, iPad, Android, Mac and Windows. There is no account to create, no subscription and nothing to configure. Use Wi-Fi for the first download, because the model is a large file. To check that it is local, switch on airplane mode and ask another question.

On a Mac, Loci needs Apple silicon and macOS 26.2 or later. On a PC it needs Windows 11. Both desktop apps are labelled beta.

## Other ways to run local AI

Different local AI apps suit different people:

- **LM Studio** is a desktop app for browsing models and adjusting their settings, on Mac, Windows and Linux.
- **Ollama** runs models from a terminal and serves them to other software, on Mac, Windows and Linux. It is the usual choice for developers.
- **Jan** and **GPT4All** are free desktop apps for chatting with a local model.
- **PocketPal** is an open-source phone app for small models.

[How to run an LLM locally](https://askloci.ai/guides/run-llm-locally/) has step-by-step instructions for each device, and the [comparison pages](https://askloci.ai/compare/) set the apps side by side.

## What local AI is good at, and where it is weaker

**Good at:** drafting and rewriting, summarising, brainstorming, explaining things, simple coding help and working with files you give it.

**Weaker at:** hard multi-step reasoning, very long documents, obscure facts and anything that changed recently. A local model only knows what it learned in training. For current facts it needs a web search, and that request does go online.

Different jobs also need different models. A text model cannot make images. Making images or speech takes a different model, with its own download and memory needs.

Local models make mistakes, as every AI model does. Check anything that matters.

## Is local AI private?

It is more private than cloud AI in one specific way: a local chat does not have to be sent to anyone to get an answer. You can check this yourself, because the model keeps answering with the internet off.

It has limits worth knowing:

- **Online features are separate.** Downloading a model, web search, updates and any cloud model an app offers all use the internet. A good app tells you what each one sends.
- **The app matters.** Running the model locally does not stop an app from collecting other data. Read its privacy policy.
- **Your device still needs protecting.** Local chats are stored on your device, so anyone who can unlock it, malware on it, or a backup that leaves it can expose them.

[Private AI](https://askloci.ai/private-ai/) shows where a local chat goes, and [what a local AI assistant is](https://askloci.ai/local-ai-assistant/) shows how to check an app in five minutes.

## Does local AI work offline?

Yes. Once the model is downloaded, chat works with no connection: on a flight, in a dead zone, or with Wi-Fi switched off. The first download, web search, live information and app updates still need the internet. An offline model cannot know today's news. [Offline AI](https://askloci.ai/offline-ai/) lists what works and what waits.

## How fast is local AI?

Two numbers describe speed, and they measure different things:

- **First-word wait**, also called time to first token, is how long you wait before the answer starts. It grows with the length of what you sent, because the model reads all of it first.
- **Tokens per second** is how fast the answer is written once it starts. A token is about four characters, so 20 tokens per second is roughly 15 words a second.

Over the last 30 days, Gemma 4 E2B runs at 66 tokens per second on the iPhone 17 Pro (26 iPhones), and Gemma 4 E2B runs at 28 tokens per second on the Galaxy S25+ (19 phones). Older and mid-range phones are slower; each device has its own page. The [local AI speed statistics](https://askloci.ai/speed/) list every phone and Mac measured in Loci.

## Words you will run into

- **LLM (large language model):** the kind of AI model that reads and writes text.
- **Inference:** a model producing an answer. Local inference means that happens on your device.
- **Weights:** the numbers inside a model file. Open weights means the file is published for anyone to download. The model's license still says what you may do with it.
- **Parameters:** how many weights a model has. 4B means four billion.
- **Token:** a piece of a word. Models read and write in tokens.
- **Context window:** how much text a model can keep in view at once, counted in tokens.
- **Quantization:** storing a model's numbers with fewer bits, so the file is smaller and fits in less memory, at some cost in quality. A 4-bit model is quantized.
- **GGUF and MLX:** two file formats for local models. GGUF runs almost anywhere. MLX is for Apple silicon.
- **On-device AI and edge AI:** other names for local AI.
- **LocalAI:** usually just local AI written as one word. It is also the name of a separate open-source project that serves local models to other software.
- **Self-hosted AI:** a model on a server you control, such as a home server or a company's own cloud. It is yours, but it is not necessarily on the device in your hand.

## FAQ

### What is local AI in simple terms?

An AI model that lives on your own phone or computer and answers there, instead of on a company's servers. You download it once and it keeps working without the internet.

### What is the easiest way to run AI locally?

Install an app that picks the model for you. Loci does this on iPhone, iPad, Android, Mac and Windows: open it, let the model download, and ask a question.

### Is local AI free?

The apps in this guide are free for local use, and open models are free to download. Some apps sell optional cloud features. Loci is free, with no account, subscription or ads.

### Can I run local AI on my phone?

Yes. Recent iPhones and Android phones run small models on the phone itself. The [speed pages](https://askloci.ai/speed/) show how fast each phone is.

### Do I need a powerful computer or a graphics card?

No. Small models run on a recent phone, an Apple silicon Mac or a Windows 11 laptop. A graphics card helps with larger models when the model fits in the card's memory.

### How much RAM do I need for local AI?

About 8 GB runs small 2B-4B models, 16 GB runs 9B-12B, and 24 GB or more runs 20B-31B. These are our estimates from the sizes Ollama lists.

### Is local AI as good as ChatGPT?

A local model handles everyday writing and questions. The models behind ChatGPT remain stronger for the hardest problems and for making images.

### Is local AI safe?

Running a model locally keeps your prompts on your device. As with any software, use an app from a developer you trust, get models through the app or from the model's maker, and keep the app updated.

### Does local AI collect my data?

It depends on the app. A local chat does not need to be sent anywhere, but an app can still collect diagnostics or send web searches. In Loci, local chats are processed on your device and are not uploaded to a Loci server in ordinary use.

### Is local AI the same as offline AI?

Nearly. Local AI says where the model runs, and offline AI says it works without a connection. A local model works offline once it is downloaded, which also makes it usable off-grid.

### Can local AI search the web?

Yes, if the app offers it and you are online. The search leaves your device, so it is a separate step from local chat.

### What is the difference between local AI and a local LLM?

A local LLM is a text model running on your device. Local AI is the wider term: it also covers speech recognition, image understanding and other models that run on-device.

### Where can I download a local AI app?

Loci is on the App Store and Google Play, and as a direct download for Mac and Windows: [download Loci](https://askloci.ai/download/). LM Studio, Ollama and Jan are downloaded from their own websites.

### Which local AI model is best?

The largest good model that fits your memory. [Best local LLMs](https://askloci.ai/guides/best-local-llms/) lists them by device.

