Ollama — The Easiest Way to Start

Ollama is a single binary bundling a model runtime (built on llama.cpp), a model library, and a local HTTP server.

curl -fsSL https://ollama.com/install.sh | sh

Confirm the install from any terminal:

ollama --version

Then confirm the local server is reachable. Ollama listens on port 11434 by default:

curl http://localhost:11434/api/version

The fastest path is ollama run. If the model is not on disk, Ollama downloads it, then drops you into a chat prompt. Start with a small, capable model:

ollama run llama3.2

A progress bar shows the download, then a prompt appears. Type a message and press Enter:

>>> Explain what a context window is in one sentence.

To download without chatting (useful in scripts or CI), use pull:

ollama pull qwen3:8b

Names follow a name:tag convention; the tag usually encodes size and/or quantization — storing weights at lower precision (e.g. 4-bit vs 16-bit) to shrink memory use, at a small accuracy cost. Choosing tags is covered in Lesson 4: Model Selection.

When you run a model, three things happen:

1. Resolve and download. Ollama checks its store for the model+tag. If missing, it downloads the GGUF weights (the format llama.cpp uses) plus a manifest with the template and defaults.
2. Load into memory. The weights map into RAM. With a supported GPU — NVIDIA CUDA, AMD ROCm, or Apple's Metal on Apple Silicon — Ollama offloads as many layers as fit into VRAM/unified memory.
3. Serve. A background process (ollama serve) holds the loaded model and answers HTTP requests on port 11434.

See what is loaded and where (CPU vs GPU):

ollama ps

NAME           ID            SIZE      PROCESSOR    UNTIL
llama3.2:latest a1b2c3d4e5f6  6.7 GB    100% GPU     4 minutes from now

A loaded model is unloaded automatically after a few minutes of inactivity to free memory. The UNTIL column shows when.

Inside an ollama run session you get a REPL with built-in slash commands for managing the session without quitting:

Keyboard shortcuts mirror common shells:

For multi-line prompts (pasting a code block, say), wrap them in triple quotes ("""). You can also tune behaviour live — lowering temperature makes output more deterministic:

>>> /set parameter temperature 0.1
>>> /show info

Pull a second model so you have at least two locally, then list what you have:

ollama pull gemma3:4b
ollama list

NAME            ID            SIZE      MODIFIED
llama3.2:latest a1b2c3d4e5f6  2.0 GB    10 minutes ago
qwen3:8b        f6e5d4c3b2a1  5.2 GB    8 minutes ago
gemma3:4b       001122334455  3.3 GB    1 minute ago

Models, manifests, and blobs live in one store. On macOS and a Linux user-install that is ~/.ollama/models (a Linux service install uses /usr/share/ollama/.ollama/models); on Windows, C:\Users\<you>\.ollama\models. Inspect its size, then free space by removing unused models:

du -sh ~/.ollama/models
ollama rm gemma3:4b

Relocate the store by setting the OLLAMA_MODELS environment variable before starting the server — handy when your home volume is small but you have a large secondary drive.

A Modelfile is a small text file that layers your defaults onto a base model — a system prompt and sampling parameters. It is the local equivalent of a "custom GPT," but the result is a real, named model on your machine. Create a file named Modelfile (no extension):

FROM llama3.2

# Lower temperature for consistent, factual answers
PARAMETER temperature 0.2
PARAMETER num_ctx 8192

SYSTEM """
You are a terse senior engineer. Answer in at most three sentences.
Prefer concrete commands and code over prose. If unsure, say so.
"""

The instructions: FROM (required base model), PARAMETER (runtime settings like temperature, num_ctx for context size, top_p, repeat_penalty), SYSTEM (persistent system prompt), plus TEMPLATE, ADAPTER (for LoRA fine-tunes — see Lesson 7), and MESSAGE for few-shot examples.

Build and run your custom model:

ollama create terse-eng -f ./Modelfile
ollama run terse-eng "How do I list the largest files in a directory?"

To see how an existing model is configured, dump its Modelfile:

ollama show --modelfile llama3.2

Finally, the same model is callable over the HTTP API — the bridge that lets local models plug into editors and tools later. The default base URL is http://localhost:11434:

curl http://localhost:11434/api/chat -d '{
  "model": "terse-eng",
  "messages": [{ "role": "user", "content": "One-liner to count lines in all .py files?" }],
  "stream": false
}'