How to run AI on your own computer: the hardware you need and how to set it up.

Why run AI on your own computer?

Three reasons: your data never leaves the machine, there is no per-use cost, and it works offline.

When you use ChatGPT, Claude or Copilot, your prompts go to the provider's servers. A local model runs entirely on your hardware, so drafts, client notes and internal documents stay with you. That makes the Privacy Act questions about sending personal information to an overseas provider much simpler (see our guide to AI and the Privacy Act).

The trade-offs are real. A model small enough for a laptop is less capable than the leading cloud assistants, it does not search the web unless you add that, and its knowledge stops at its training date. Local AI suits specific jobs, such as summarising, drafting and answering questions over your own documents, more than it replaces a cloud assistant. Our guide to open source AI models covers who makes them and how they compare.

How much memory do you need to run an AI model?

As a rule of thumb, about 0.6GB for every billion parameters in the usual compressed form, plus room for the conversation and your operating system.

A model's size is counted in parameters, its learned settings. Each is stored as a number, and compressing those numbers, called quantisation, shrinks the model. At full 16-bit precision a model needs about 2GB per billion parameters. The common 4-bit versions (often labelled Q4) need about a quarter of that, with only a small loss in quality. Google's published figures for Gemma 4 work out at about 0.56GB per billion parameters at 4-bit, including loading overhead.

Then add room for the conversation. The longer the document or chat, the more memory the model's working space, its context window, uses. And leave memory for your operating system and other apps: on a 16GB laptop, plan on roughly 10GB to 12GB being available for AI.

Speed is a separate question. How fast a model writes depends mostly on memory bandwidth, how quickly the chip can read the model. That is why a Mac with a Pro or Max chip, or a PC graphics card, writes faster than a basic laptop running the same model.

What hardware runs which AI models?

Match your memory to a model size. On a PC that means the graphics card's memory; on a Mac, the unified memory.

Model sizes from each lab's model pages; Mac mini memory options from Apple; checked 8 November 2026. Fit assumes 4-bit models and moderate conversation length.

Mac, Windows PC or laptop: which is best for local AI?

A Mac uses all its memory for AI; a PC is limited by its graphics card; a laptop works for small models.

• Mac (Apple Silicon). The processor and graphics share one pool of unified memory, so nearly all of it is available to the model. That makes a Mac mini or Mac Studio with plenty of memory one of the simplest ways to run medium-sized models. Bandwidth sets the speed: the M5 Pro's 307GB/s is about twice the M6's 153GB/s. Older Intel Macs are not supported by most local AI apps.
• Windows PC. What counts is the graphics card's own memory (VRAM). A model that fits in VRAM runs fast; one that spills into ordinary system memory slows sharply. An NVIDIA card with 16GB or more is the usual choice.
• Laptop. Stick to models in the 16GB band or smaller. Expect the fans to run and the battery to drain faster while the model is working.
• Dedicated AI boxes. Small desktop machines with 128GB of shared memory, such as NVIDIA's DGX Spark and PCs built on AMD's Ryzen AI Max+ chip, now sit between a Mac Studio and a server. They suit a team sharing one model.

Before you buy anything, check what you already have. On a Mac, open the Apple menu, then About This Mac, and look at Memory. On Windows, open Task Manager, then Performance, then GPU, and look at Dedicated GPU memory.

How do you set up LM Studio?

LM Studio is a free desktop app for Mac, Windows and Linux that lets you download and chat with models without touching the command line.

1. Check your machine. LM Studio needs a Mac with Apple Silicon and macOS 14 or newer, or a Windows PC with a processor that supports AVX2. It recommends 16GB of memory, and on a PC at least 4GB of graphics memory.
2. Download and install LM Studio from lmstudio.ai.
3. Find a model. Open the search (Discover) tab and search for a model that suits your memory, such as "gpt-oss-20b" or "gemma 4". Choose a version from the original lab or a well-known publisher, and a 4-bit (Q4) download.
4. Chat. Once it has downloaded, open a chat, load the model at the top of the window and start typing. The first answer takes a few seconds while the model loads.
5. Prove it is local. Turn off Wi-Fi and keep chatting. LM Studio's documentation confirms it can run entirely offline once a model is downloaded.

How do you set up Ollama?

Ollama is a free tool that runs models in the background, so other apps and scripts on your computer can use them.

1. Download and install Ollama from ollama.com/download for Mac, Windows or Linux, then open it.
2. Run your first model. Open Terminal on a Mac or PowerShell on Windows and type ollama run gpt-oss:20b. The first time, it downloads the model, which takes a while on a slow connection. Then type your question.
3. Finish. Type /bye to leave the chat. The model stays downloaded for next time.
4. Connect other tools. Ollama serves the model on your own computer at localhost:11434, so compatible apps can use it without an internet connection or an API key.
5. Stay local. Ollama also offers cloud models, which send your prompts to Ollama's servers. If the point is to keep data on your machine, use only models you have downloaded.

Both tools are built on the same open source engine, llama.cpp, so the same model gives similar results in either. Pick LM Studio if you want an app, and Ollama if you want to connect a model to other software.

Which model should you start with?

Start with the largest model your memory comfortably holds, from a lab with a permissive licence.

• 8GB: Google's Gemma 4 E4B.
• 16GB: OpenAI's gpt-oss-20b.
• 32GB: Gemma 4 31B or Alibaba's Qwen3.6-27B.
• 64GB to 128GB: OpenAI's gpt-oss-120b, which needs about 80GB, or NVIDIA's Nemotron 3.5 Super.

All of these are released under the Apache 2.0 licence, apart from Nemotron, which has its own open licence, so they can be used commercially. Try two on the same real task and keep the one that does the job.

Our view.

What are the risks of running AI locally?

Five come up most often, and each has a simple guardrail.

• Lower quality than cloud assistants. Small models make more mistakes. Keep a person checking outputs, and see why AI models hallucinate.
• Out-of-date knowledge. A local model knows nothing after its training date and cannot look things up. Give it the documents it needs.
• Unvetted downloads. Anyone can upload a copy of a model. Download from the original lab or a well-known publisher only.
• Cloud features switched on by default. Some local apps offer cloud models too. Check the settings if data must stay on the machine.
• Shadow AI. Staff can install these tools in minutes without IT knowing. Add local AI to your shadow AI checks and your approved tools list.

Where should you start?

Know where you stand, check the hardware you already own, then test one model on one job.

1. Find out where you stand. The free NZ AI Benchmark is 40 questions, takes about 20 minutes, and shows where your organisation stands and the next move.
2. Check your memory using the steps above, and find your row in the hardware table.
3. Install LM Studio and download one model that fits.
4. Test it on one real task for a week, using non-sensitive material at first, and compare it with the tool you use today.
5. Add it to your AI policy and approved tools list. Our AI policy guide covers who may approve new tools.

Frequently asked questions.

Sources.

1. gpt-oss-20b model card, OpenAI (checked 8 November 2026)
2. gpt-oss-120b model card, OpenAI (checked 8 November 2026)
3. Gemma 4 model overview and memory requirements, Google (checked 8 November 2026)
4. Qwen models, Alibaba (Hugging Face; checked 8 November 2026)
5. Nemotron 3.5 Super model card, NVIDIA (checked 8 November 2026)
6. Mac mini technical specifications, Apple (checked 8 November 2026)
7. System requirements, LM Studio (checked 8 November 2026)
8. Quickstart, Ollama (checked 8 November 2026)
9. State of Open Models: Summer 2026 Observations, Hugging Face (14 August 2026; accessed 8 November 2026)