Install Ollama — Run LLMs on Your VPS for Free

August 2026 · 10 min read

Ollama is a runtime that makes running Large Language Models (LLMs) locally on Linux dead simple. With a single command you can download and run models like Llama 3.1, Mistral, Gemma 2, Phi-3, and Qwen 2.5 — over 100 models from the Ollama library — without paying for expensive cloud GPU time. Everything stays on your VPS: no data leaves your server, making it ideal for developers who need a private AI API or a self-hosted chatbot.

A VPS for Ollama should have at least 8 GB RAM for 7B models and 16 GB+ for 13B models. A modern CPU is sufficient — a GPU is not required (but dramatically speeds things up). All examples in this guide use Ubuntu 22.04 LTS.

System Requirements

Install Ollama on Ubuntu/Debian

Use the official Ollama installer script — it handles everything:

# Update packages first
sudo apt update && sudo apt upgrade -y

# Install Ollama via the official script
curl -fsSL https://ollama.com/install.sh | sh

# Verify the installed version
ollama --version

The script installs the Ollama binary to /usr/bin/ollama and automatically creates a systemd service.

Download and Run Models

Use ollama run to pull a model and start an interactive chat session:

# Llama 3.2 3B — lightweight, good for low-RAM VPS (~2 GB)
ollama run llama3.2

# Llama 3.1 8B — balanced quality and speed (~5 GB RAM)
ollama run llama3.1:8b

# Mistral 7B — excellent English quality (~4 GB RAM)
ollama run mistral

# Gemma 2 9B — Google's model (~6 GB RAM)
ollama run gemma2

# Qwen 2.5 7B — strong multilingual support (~5 GB RAM)
ollama run qwen2.5

# List all downloaded models
ollama list

Using the Ollama REST API

Ollama exposes an HTTP API on port 11434 by default. You can query it directly with curl:

# Generate text — single response (no streaming)
curl http://localhost:11434/api/generate \
  -d '{
    "model": "llama3.1:8b",
    "prompt": "Explain Docker in 3 sentences.",
    "stream": false
  }'

# Chat API — conversational context
curl http://localhost:11434/api/chat \
  -d '{
    "model": "llama3.1:8b",
    "messages": [
      {"role": "user", "content": "What is a VPS and why do I need one?"}
    ]
  }'

# List available models via API
curl http://localhost:11434/api/tags

Bind Ollama to All Interfaces

By default Ollama only listens on 127.0.0.1. To expose it on the network (for a reverse proxy), set the environment variable in the systemd override:

# Edit the systemd service override
sudo systemctl edit ollama

# Add the following inside the editor:
[Service]
Environment="OLLAMA_HOST=0.0.0.0"

# Reload and restart
sudo systemctl daemon-reload
sudo systemctl restart ollama

# Confirm it's listening on all interfaces
sudo ss -tlnp | grep 11434

Nginx Reverse Proxy for Ollama

Put Nginx in front of Ollama to serve it on a domain with HTTPS and basic authentication:

# Install Nginx if not already installed
sudo apt install nginx -y

# Create the site config
sudo nano /etc/nginx/sites-available/ollama

# Paste this configuration:
server {
    listen 80;
    server_name ai.yourdomain.com;

    location / {
        proxy_pass http://127.0.0.1:11434;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_buffering off;
        proxy_read_timeout 300s;
        # Protect with HTTP basic auth
        auth_basic "Ollama API";
        auth_basic_user_file /etc/nginx/.htpasswd;
    }
}

# Create credentials file
sudo apt install apache2-utils -y
sudo htpasswd -c /etc/nginx/.htpasswd myuser

# Enable the site
sudo ln -s /etc/nginx/sites-available/ollama /etc/nginx/sites-enabled/
sudo nginx -t
sudo systemctl reload nginx

Install Open WebUI (ChatGPT-style Interface)

Open WebUI provides a polished chat interface that connects to Ollama. Install it with Docker:

# Install Docker if not already installed
curl -fsSL https://get.docker.com | sh

# Run Open WebUI
docker run -d \
  -p 3000:8080 \
  --add-host=host.docker.internal:host-gateway \
  -v open-webui:/app/backend/data \
  -e OLLAMA_BASE_URL=http://host.docker.internal:11434 \
  --name open-webui \
  --restart always \
  ghcr.io/open-webui/open-webui:main

# Verify the container is running
docker ps | grep open-webui

Open http://your-vps-ip:3000 in a browser, create the first admin account, and start chatting with your locally-running AI.

Enable Ollama on Boot

# Check current service status
sudo systemctl status ollama

# Enable auto-start at boot
sudo systemctl enable ollama

# Stream Ollama logs
sudo journalctl -u ollama -f

Create a Custom Model with Modelfile

Ollama supports Modelfile — a Dockerfile-like format for creating models with a custom system prompt and parameters:

# Create the Modelfile
cat > Modelfile << 'EOF'
FROM llama3.1:8b

PARAMETER temperature 0.7
PARAMETER top_p 0.9

SYSTEM """
You are a Linux and VPS expert. Answer concisely with practical examples.
Always include actual shell commands in your responses.
"""
EOF

# Build the custom model
ollama create linux-expert -f Modelfile

# Test it
ollama run linux-expert "Nginx is slow. How do I diagnose it?"

Verify Ollama Is Working Correctly

# Check service status
sudo systemctl status ollama

# Confirm port 11434 is listening
sudo ss -tlnp | grep 11434

# Hit the API health endpoint
curl http://localhost:11434/

# List downloaded models
ollama list

# Run a short test prompt
curl http://localhost:11434/api/generate \
  -d '{"model":"llama3.2","prompt":"ping","stream":false}' \
  | python3 -m json.tool

Troubleshooting Common Issues

Error: "model not found" or "pull model manifest: file does not exist"

The model name is incorrect or the model hasn't been downloaded yet. Check the exact name:

# List downloaded models
ollama list

# Pull the model explicitly
ollama pull llama3.1:8b

# Verify you have enough disk space
df -h /usr/share/ollama

Error: "out of memory" — Ollama crashes when loading a model

Not enough RAM for the selected model. Switch to a smaller model or add swap:

# Check available RAM
free -h

# Add 4 GB swap
sudo fallocate -l 4G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab

# Use a lighter model
ollama run llama3.2:1b  # only ~1.3 GB RAM

Ollama API is very slow or times out

CPU inference is slower than GPU, but abnormal slowness usually points to resource contention or a model that's too large:

# Check CPU during inference
top -bn1 | head -20

# Inspect Ollama logs for errors
sudo journalctl -u ollama -n 50

# Try a Q4-quantized model — half the size, similar quality
ollama pull llama3.1:8b-instruct-q4_0

Error: "bind: address already in use" on port 11434

Another process or a stale Ollama instance is already using that port:

# Find what's using port 11434
sudo lsof -i :11434

# Kill the blocking process
sudo kill -9 $(sudo lsof -ti :11434)

# Restart Ollama
sudo systemctl restart ollama

Open WebUI can't connect to Ollama inside Docker

Docker containers can't reach the host via localhost — use host.docker.internal or the --network host flag:

# Confirm OLLAMA_HOST is set to 0.0.0.0
sudo systemctl show ollama | grep OLLAMA_HOST

# Alternative: run Open WebUI with host networking
docker run -d \
  --network host \
  -v open-webui:/app/backend/data \
  -e OLLAMA_BASE_URL=http://localhost:11434 \
  --name open-webui2 \
  ghcr.io/open-webui/open-webui:main

Frequently Asked Questions

Can Ollama run on CPU only, or is a GPU required?

Ollama works perfectly on CPU-only servers via llama.cpp — no GPU needed. Speed will be 5–20× slower than a GPU, but for personal use or non-time-critical inference, a modern 8-core VPS will generate roughly 5–15 tokens per second for a 7–8B model. That's fast enough for interactive chat once you get used to it.

Does Ollama send my data anywhere?

No. All inference happens entirely on your VPS. Ollama does not phone home or send prompts to any external server. This makes it ideal for sensitive workloads, internal tools, or any scenario where data privacy is non-negotiable.

How does Ollama compare to LM Studio or Jan?

Ollama is designed for headless servers and exposes a clean REST API — the right tool for a VPS. LM Studio and Jan are desktop applications with polished GUIs, better suited to MacBooks and PCs. If you need an API endpoint for your application or want to integrate AI into a backend service, Ollama is the clear choice. The Open WebUI container gives you a desktop-grade chat UI on top of Ollama when you need one.