Self-Hosted AI Stack — Part 3: Core Services

The project layout

Everything lives under a single project directory — this series uses ~/ai-stack inside your WSL2 distro:

~/ai-stack/
+-- docker-compose.yml
+-- ollama/          < Ollama models persist here
+-- comfyui/
|   +-- run/         < ComfyUI runtime/user data
|   +-- basedir/     < ComfyUI models, checkpoints, custom nodes
+-- open-webui/      < Chat history, settings
+-- searxng/         < SearXNG config

Every service gets its own subfolder mounted as a volume, so container data survives even if you tear down and recreate the containers — the only thing you'd lose by running docker compose down is anything you never wrote to a mounted path.

The full docker-compose.yml

services:
  ollama:
    image: ollama/ollama
    container_name: ollama
    restart: unless-stopped
    gpus: all
    ports:
      - "11434:11434"
    volumes:
      - ./ollama:/root/.ollama

  comfyui:
    image: mmartial/comfyui-nvidia-docker:ubuntu24_cuda12.8-latest
    container_name: comfyui
    restart: unless-stopped
    gpus: all
    environment:
      - WANTED_UID=1000
      - WANTED_GID=1000
      - BASE_DIRECTORY=/basedir
      - USE_UV=true
    ports:
      - "8188:8188"
    volumes:
      - ./comfyui/run:/comfy/mnt
      - ./comfyui/basedir:/basedir

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    restart: unless-stopped
    depends_on:
      - ollama
    ports:
      - "3000:8080"
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434
    volumes:
      - ./open-webui:/app/backend/data
    extra_hosts:
      - "host.docker.internal:host-gateway"

  searxng:
    container_name: searxng
    image: searxng/searxng:latest
    restart: unless-stopped
    ports:
      - "8080:8080"
    volumes:
      - ./searxng:/etc/searxng:rw
    cap_drop:
      - ALL
    cap_add:
      - CHOWN
      - SETGID
      - SETUID
      - DAC_OVERRIDE

Why each piece is set up this way

ollama — the model server. gpus: all passes your entire GPU through to the container (this is what Part 2's NVIDIA Container Toolkit setup enables). Port 11434 is Ollama's standard API port — every other service in this stack talks to Ollama through it. The volume mount is critical: without it, every model you pull gets re-downloaded from scratch any time you recreate the container.

comfyui — image generation. The image tag ubuntu24_cuda12.8-latest isn't arbitrary: CUDA 12.8 is required for Blackwell-generation GPUs — an older CUDA tag will either fail to detect your GPU or run without acceleration. The WANTED_UID/WANTED_GID environment variables tell the container to run as UID/GID 1000 internally, which should match your WSL2 user's UID — mismatched IDs are the single most common cause of "permission denied" errors when ComfyUI tries to write to a mounted folder. USE_UV=true switches the container's internal Python package management to uv, which is substantially faster than pip for the custom-node installs ComfyUI relies on.

open-webui — the unified chat interface. depends_on: [ollama] controls start order only — Docker Compose starts Ollama first, but doesn't wait for it to be fully ready to accept connections before starting Open WebUI. In practice Ollama comes up fast enough that this rarely matters, but if you ever see Open WebUI fail to find any models immediately after a fresh docker compose up, that's why — give it a few seconds and refresh. OLLAMA_BASE_URL=http://ollama:11434 works because Docker Compose creates an internal network where each service can reach the others by container name. The port mapping 3000:8080 means you'll reach the web UI at localhost:3000, even though the container listens on 8080 internally.

searxng — private web search, wired into Open WebUI so your models can pull live information instead of relying only on training data. The cap_drop: [ALL] plus a short cap_add list is a security hardening pattern: instead of trusting the container's default Linux capabilities, you strip everything and add back only the four capabilities SearXNG actually needs to run.

Bringing it up

cd ~/ai-stack
docker compose up -d

Then verify each service independently before moving on:

# Ollama — should return a JSON list (empty until you pull a model)
curl http://localhost:11434/api/tags

# Open WebUI — open in browser
# http://localhost:3000

# ComfyUI — open in browser
# http://localhost:8188

# SearXNG — open in browser
# http://localhost:8080

Pulling your first model

docker exec -it ollama ollama pull qwen3-coder:30b

Swap in whichever model tag matches your VRAM tier from Part 1. Once it's pulled, refresh Open WebUI — it should appear in the model dropdown automatically, since Open WebUI queries Ollama's /api/tags endpoint on its own.

Wiring SearXNG into Open WebUI

In Open WebUI: Admin Settings → Web Search, enable web search, set the engine to SearXNG, and point it at http://searxng:8080/search (container-to-container, so use the service name, not localhost). Save, then test it by asking your model something that requires current information it couldn't know from training alone.

Troubleshooting

ComfyUI container starts but GPU isn't detected

Almost always a CUDA/driver mismatch. Confirm nvidia-smi works both on the Windows host and inside a bare test container before assuming ComfyUI itself is broken:

docker run --rm --gpus all nvidia/cuda:12.8.0-base-ubuntu24.04 nvidia-smi

If the base test fails, the problem is in Part 2's NVIDIA Container Toolkit setup, not this compose file.

Permission denied errors on ComfyUI's mounted folders

Check that WANTED_UID/WANTED_GID in the compose file actually match your WSL2 user's UID/GID:

id -u && id -g

Mismatches here are the most common cause of permission errors on mounted volumes.

Open WebUI shows no models

Give it a few seconds after startup — depends_on doesn't wait for Ollama to be ready. If it's still empty after a minute, confirm Ollama has at least one model pulled:

docker exec -it ollama ollama list

And confirm OLLAMA_BASE_URL in the compose file matches the Ollama service name exactly (http://ollama:11434).

SearXNG returns no results in Open WebUI

Test SearXNG directly at http://localhost:8080 in a browser before assuming the Open WebUI wiring is at fault. If SearXNG itself returns no results, the issue is in its own config (./searxng folder), not the integration.

Series Navigation

• ← Part 2 — Environment Setup
• → Part 4 — Image Generation