How to Run Llama 3 Locally: A Beginner's Hardware Guide

Ollama is the simplest entry point for beginners, and it is the tool most people should start with. You download one installer for macOS, Windows, or Linux, run it, and the rest happens in a terminal. There is nothing to configure and no model files to hunt down by hand.

Once it is installed, three commands cover almost everything you will do at first. Running ollama run llama3 pulls the default 8B model and starts a chat session. Running ollama list shows what you have downloaded, and ollama rm llama3 removes a model when you want the disk space back. The first run takes a few minutes because it downloads several gigabytes, but every run after that is instant.

Behind the scenes, Ollama picks a sensible quantized version of the model and uses your GPU automatically if you have one. For a beginner, that means you can ignore most of the technical knobs and still get a good result. The sections below explain what those knobs do, but you do not need to touch them to get started today.

Llama 3 comes in different sizes, and the size you pick is the single biggest factor in what hardware you need. The number in a model's name, like 8B or 70B, is its parameter count, and roughly speaking, more parameters means smarter answers but much heavier memory demands. For local use, the memory that matters is GPU VRAM, or on a Mac, the unified memory shared between CPU and GPU.

The numbers below are for the popular Q4_K_M quantization, which is the default most beginners should use. Treat them as practical estimates, not hard floors, because longer conversations use extra memory for context and your operating system needs headroom too.

There are three tools worth knowing about, and they share the same underlying engine, so the model quality is the same across them. What differs is the experience and how much control you get. Most beginners only ever need one of them.

The practical advice is to start with Ollama or LM Studio, and reach for llama.cpp only when you have a specific reason. Many people install both Ollama and LM Studio: LM Studio to browse and test models visually, and Ollama to actually serve a model to other apps.

Quantization is the trick that makes local Llama 3 practical. The full-precision model is large, so tools compress the weights to fewer bits. The Q4_K_M format compresses to roughly 4 bits and cuts memory use by about three quarters compared to full precision, while keeping quality high enough that most people cannot tell the difference in everyday use. For beginners, Q4_K_M is the right default, and you only need to consider heavier quantization if a model does not fit your memory.

Speed is measured in tokens per second, where a token is roughly three quarters of a word. On a modern GPU or recent Apple Silicon Mac, an 8B model typically runs comfortably fast for chat, often in the range of tens of tokens per second, which feels close to a typing assistant. A 70B model is far heavier and tends to land in the single digits to low double digits of tokens per second even on capable hardware, so it reads more like a slow but steady writer.

If you only have a CPU and system RAM, Llama 3 still runs, but expect it to be slow, sometimes just a few tokens per second on 8B. This RAM-offload path is a useful fallback for trying a model out or running occasional tasks where speed does not matter, but it is not pleasant for back-and-forth chat. The honest takeaway is that a GPU or a Mac with generous unified memory is what turns local Llama 3 from a curiosity into something you actually use.

The simplest way to decide is to start small and move up only if you need to. Pull Llama 3 8B first, because it runs almost anywhere and is good enough for summarizing, drafting, coding help, and general questions. If it feels slow, the problem is usually that the model is falling back to CPU, which means your GPU or memory cannot hold it, and dropping to a smaller model or a lighter quantization fixes that.

Only reach for 70B if you have specifically hit a quality ceiling with 8B and you own, or are willing to buy, the memory it demands. Jumping straight to the biggest model on hardware that cannot hold it is the most common beginner mistake, and the result is a model that crawls or refuses to load. It is far better to run a smaller model quickly than a larger one that barely works.

If you are not sure where your machine lands, the find my setup wizard on this site asks a few questions about your hardware and points you to a model size and tool that will actually run well. The starter builds, including the local dev starter and a budget Llama 8B configuration, show concrete machines at different price points, and the start here guide for newcomers walks through the whole decision from scratch.