Run an AI model on your own computer — no GPU, no cloud, no account

BASIC

Tested for real: I ran every command in this guide top-to-bottom on a fresh Ubuntu 24.04 virtual machine (4 CPUs, 8 GB RAM) on 13 August 2026. Versions at test time: Ollama 0.32.9, model llama3.2:3b (2.0 GB), on Ubuntu 24.04.4 LTS. The machine had no GPU at all — that is the point.

Found a problem? Tell me via the contact page.

What you'll end up with

An AI model running on your own machine, answering questions in your terminal. No account, no API key. Your prompts and replies stay on this machine while you run the model locally. I asked mine what it was, and here is what it said:

A large language model (LLM) is a type of artificial intelligence that uses
complex algorithms and vast amounts of training data to understand and
generate human-like language.

That answer was generated on 4 ordinary CPU cores with 8 GB of RAM. No graphics card anywhere in sight.

Who this is for

BASIC. One path, copy-paste-safe, no branching. If you can open a terminal on your Ubuntu machine, you can do this. You do not need a GPU — a GPU (graphics processing unit, the chip that makes games and AI fast) makes this quicker, but this guide is for the rest of us.

A note on how I tested this

Where you will type into an interactive chat, my test rig drove the same model with the same questions from the command line. A script cannot type into a live chat convincingly. Every answer printed in this guide is what the model really said on the test machine. The speed numbers are measured, not guessed.

Time and cost

The commands took under 3 minutes of machine time on my test line. But the downloads total about 4 GB, so your broadband sets the pace. Allow 30 minutes end to end; on a slow line, allow an hour. Cost: free. About 4.5 GB of disk space.

Words you'll meet

• Model — the AI itself: one big file of learned numbers. You download it once and run it as often as you like. See /basics.
• LLM — large language model. The kind of AI that reads and writes text, like ChatGPT. Ours is small enough to live on your machine.
• Ollama — the free program that downloads models and runs them for you. Think of it as an app store plus player for LLMs.
• Token — the chunks a model reads and writes in. Roughly three-quarters of a word each.
• Tokens per second — the model's writing speed. Ten per second is about as fast as most people read.
• Service — a program that runs in the background from boot. Ollama installs itself as one.
• 127.0.0.1 — the address that means "this machine only". Nothing on your network can reach a program listening there.

Placeholders

Anywhere you see CAPS-WITH-DASHES like YOUR-USERNAME-HERE, swap in your own value. This guide has none in the commands — everything is copy-paste as written.

Before you start

• An Ubuntu 24.04 machine you can log into, with about 5 GB of free disk.
• A normal user account with sudo, not root. My first hour guide sets this up.
• 8 GB of RAM. Less can work with smaller models, but 8 GB fits this guide comfortably.
• No GPU needed. Really.

The steps

Step 1 — Download the install script and look at it

Ollama's official install method is a shell script from their website. Many guides pipe it straight into your shell without showing you what it does. We download it to a file first, so you can read it before anything runs.

curl -fsSL https://ollama.com/install.sh -o install-ollama.sh

About install scripts from the internet. Running a downloaded script with sudo hands that script full control of your machine, so know what you are running. This one is 455 lines and you can read it at ollama.com/install.sh. What it does: detects your CPU type, downloads the Ollama program to /usr/local, creates a system user called ollama, sets up a background service, and starts it. It only touches GPU drivers if it finds a GPU — ours won't.

Prefer not to run a script at all? Ollama documents a manual install: download the release archive from github.com/ollama/ollama/releases and unpack it yourself, following docs.ollama.com/linux. Same result, more steps.

Check it worked.

wc -l install-ollama.sh

Your output will vary, but look for:

455 install-ollama.sh

Your line count may differ slightly — the script gets small updates. If the file is missing or empty, check your internet connection and run the download again.

Step 2 — Run the installer

The script needs sudo for the steps above, and it will ask for your password. On my test line the whole install took 27 seconds; the download is over a gigabyte, so on home broadband expect a few minutes.

sh install-ollama.sh

You should see something like:

>>> Installing ollama to /usr/local
>>> Downloading ollama-linux-amd64.tar.zst
######################################################################## 100.0%
>>> Creating ollama user...
>>> Creating ollama systemd service...
>>> Enabling and starting ollama service...
WARNING: No NVIDIA/AMD GPU detected. Ollama will run in CPU-only mode.
>>> The Ollama API is now available at 127.0.0.1:11434.
>>> Install complete. Run "ollama" from the command line.

The important line is The Ollama API is now available at 127.0.0.1:11434.

That WARNING is not a problem. It is the installer confirming what we already knew: no GPU here, the processor will do the work. That is this guide's whole premise.

Check it worked.

ollama --version

ollama version is 0.32.9

Your version may be newer — that's fine. And confirm the background service is running:

systemctl is-active ollama

active

Step 3 — Download a model

Now we need an actual model. llama3.2:3b is Meta's small Llama model — "3b" means 3 billion parameters (the learned numbers inside it). It is a 2.0 GB download and it runs well on an 8 GB machine with no GPU. That is exactly why I chose it.

ollama pull llama3.2:3b

On my test line this took 32 seconds; on home broadband expect 5 to 15 minutes. A passing run includes:

pulling dde5aa3fc5ff: 100% ▕██████████████████▏ 2.0 GB
verifying sha256 digest
writing manifest
success

The important line is success.

Check it worked.

ollama list

NAME           ID              SIZE      MODIFIED
llama3.2:3b    a80c4f17acd5    2.0 GB    Less than a second ago

The important line is the llama3.2:3b row — the model is on your disk.

Step 4 — Talk to it

This opens a live chat with the model in your terminal. Type a question, press Enter, and watch it answer word by word. Type /bye to leave the chat when you are done.

ollama run llama3.2:3b

The first response takes a moment to start — the model is loading into memory (about 6 seconds on my test machine). Then the words begin to appear. Here is my real first exchange:

>>> Explain what a large language model is, in two short sentences.
A large language model (LLM) is a type of artificial intelligence that uses
complex algorithms and vast amounts of training data to understand and
generate human-like language. LLMs can be trained on vast datasets to learn
patterns and relationships in language, enabling them to perform tasks such
as text classification, sentiment analysis, and language translation.

>>> /bye

Check it worked. The check here is the answer itself. If the model replied to you, everything underneath is working: the service, the model, the lot. Your prompts and replies stay on this machine while you run the model locally.

Step 5 — Measure how fast it really is

Let's get an honest number instead of a feeling. The --verbose flag prints timing statistics after the answer. Giving the question on the command line like this runs one answer and returns to your prompt.

ollama run llama3.2:3b --verbose "Why does an AI model need so much memory? Answer in one sentence."

You should see the answer, then something like:

An AI model needs a significant amount of memory because it requires the
storage and processing of complex mathematical operations, large datasets,
and intermediate results to learn and make predictions.

total duration:       15.352397049s
load duration:        6.462283921s
prompt eval rate:     25.90 tokens/s
eval rate:            9.08 tokens/s

The important line is eval rate — the writing speed. My no-GPU machine managed 9 tokens per second, roughly the pace you read at. A GPU can be much faster.

Here is the honest framing: this will not replace a paid chatbot for speed. But it is private, free, offline-capable, and yours. For learning how these things work — and for anything you would rather not paste into someone else's cloud — 9 tokens a second is plenty.

Step 6 — See what it costs in memory

Ollama keeps the model in RAM for about 5 minutes after you use it, then unloads it. This shows what is loaded right now.

ollama ps

The important line is:

NAME           ID              SIZE      PROCESSOR    CONTEXT    UNTIL
llama3.2:3b    a80c4f17acd5    2.6 GB    100% CPU     4096       4 minutes from now

100% CPU confirms the processor is doing all the work. The 2.6 GB is the model plus its working space. On my 8 GB machine, total memory in use was 2.9 GB with the model loaded — comfortable room to spare.

Check it worked. If ollama ps shows an empty table, the model has already unloaded. That is normal — run step 5's command again and re-check within 5 minutes.

Step 7 — Confirm it's private

This is the selling point, so let's prove it rather than claim it. Ollama answers on a network address, and this command shows which one. ss lists the "sockets" — the doors your machine has open.

ss -tln | grep 11434

You should see something like:

LISTEN 0      4096       127.0.0.1:11434      0.0.0.0:*

The important part is 127.0.0.1. That address means "this machine only". No phone, laptop or intruder on your network can reach your AI, and nothing is sent to any cloud. Your questions stay in your RAM.

Check it worked. Ask it locally, the way other programs on your machine would:

curl -s http://127.0.0.1:11434/api/version

{"version":"0.32.9"}

Something went wrong?

• You see WARNING: No NVIDIA/AMD GPU detected during install → it means Ollama will use your processor instead of a graphics card → nothing to fix; this guide assumes exactly that. Carry on.
• You type a question and the cursor sits there doing nothing → it means the model is loading into RAM, which took 6 seconds on my machine and can take longer on slower disks → wait; once words start they keep flowing.
• You see userdel: group ollama not removed because it has other members during the undo → it means the installer added your own user to the ollama group, so the group is not empty → harmless; the next command (groupdel) removes the group anyway.

Undo all of this

Tested for real on the same machine, straight after everything above. This is Ollama's own documented removal sequence. It deletes the program, the background service, and every model you downloaded.

Stop and disable the background service:

sudo systemctl stop ollama

sudo systemctl disable ollama

Remove the service file, the program, and its libraries:

sudo rm /etc/systemd/system/ollama.service

sudo rm $(which ollama)

sudo rm -rf /usr/local/lib/ollama

Delete the models and the system user the installer created:

sudo rm -r /usr/share/ollama

sudo userdel ollama

sudo groupdel ollama

Finally, remove the install script we downloaded in step 1:

rm install-ollama.sh

Check it worked.

ollama --version

bash: ollama: command not found

That error is the pass mark — the program is gone.

And the door it was answering on is closed:

ss -tln | grep 11434

Empty output — nothing listening. Your machine is back where it started.

Where to go next

• A proper chat interface for your local AI (Open WebUI + Ollama) — the GOING FURTHER follow-on: the same model, in a proper web chat with history, from any device in your house.
• Official reference: docs.ollama.com/linux — install, manual install and uninstall; this guide follows it with real outputs added.

---

Last tested: 13 August 2026 on Ubuntu 24.04.4 (4 vCPU, 8 GB RAM, no GPU). Versions: Ollama 0.32.9, llama3.2:3b. Measured: 9.08 tokens/s on CPU.