Self-Hosted AI Stack — Part 5: IDE-Integrated Coding Agent

NOTE

Goal for this part: wire your locally-hosted coding model into VS Code as a real agentic assistant — not just chat-box Q&A, but something that can read your open file, edit code in place, and reason about actual bugs.

NOTE

Prerequisite: Ollama is running (Part 3) with a coding-capable model pulled. This series uses qwen3-coder:30b. Confirm it's available with docker exec -it ollama ollama list before starting.

Why a separate extension instead of just using Ollama's API directly

Ollama exposes a plain API — you could talk to it with curl if you wanted to. What you actually want in an IDE is something that understands editor context: which file is open, what's selected, and how to turn a model's text response into an actual applied code change. That's what the Continue extension does — it's an open-source VS Code extension purpose-built to bridge a local (or remote) model into real editor workflows: chat, inline editing, and autonomous multi-step agent tasks.

Installing and configuring Continue

In VS Code, open the Extensions panel (Ctrl+Shift+X) and search for "Continue" — install the one published by Continue Dev.
Once installed, click the Continue icon in the sidebar to open its panel.
Open Continue's configuration (gear icon, or the config.yaml it generates) and point it at your local Ollama instance:

models:
  - name: Qwen3 Coder 30B
    provider: ollama
    model: qwen3-coder:30b
    apiBase: http://localhost:11434
roles:
  - chat
  - edit
  - apply

Since Ollama is running in Docker with its port mapped to the host (11434:11434 from Part 3), localhost:11434 reaches it fine from VS Code running on the host — no container networking gymnastics needed here.

Restart VS Code (or reload the window) and confirm the model shows up in Continue's model dropdown.

TIP

VS Code may show a "Python extension" recommendation prompt when you open .py files — this is unrelated to Continue or Ollama and purely about linting and debugging features. Safe to dismiss if you don't need it.

Verifying the pipeline: three checks

Don't just trust that it's wired up correctly — verify each capability separately, since a broken config often only fails at one of these three levels.

1. Basic chat

Open Continue's chat panel and ask something simple and self-contained:

"Write a memoized Fibonacci function in Python, and explain the complexity improvement over the naive recursive version."

A working setup returns actual code (not an error, not a generic non-answer) and a correct explanation of why memoization turns exponential time complexity into linear.

2. Fresh code generation

Ask it to create a new file from scratch — this confirms it can go beyond chat replies into actual file operations.

3. Bug-fixing on real code (the test that actually matters)

Chat responses are the easy case. What you actually want is for the model to reason about your existing code and patch it correctly. Create a small test file:

# test_agent.py
def fibonacci(n):
    """Return the nth Fibonacci number."""
    # TODO: implement this
    return

def divide_list(numbers, divisor):
    """Divide every number in the list by divisor."""
    result = []
    for n in numbers:
        result.append(n / divisor)
    return result

# Deliberate bug: this will crash on divisor=0
print(divide_list([10, 20, 30], 0))

Select the divide_list function and the crashing call at the bottom, then use Continue's inline edit (Ctrl+I) rather than the sidebar chat, and give it a direct instruction:

"This crashes with a divide by zero, fix it and add proper validation."

The inline edit path is the important detail here — it feeds the selected code directly into the model as context and shows the fix as an inline diff you can accept or reject, rather than a chat reply describing what you could do.

WARN

Don't rely on Agent mode alone for single-file edits. Continue's Agent mode can autonomously search your filesystem and read files on its own — useful for multi-file tasks, but it's overkill for a straightforward "fix this function" request on code you already have open. If you ask in Agent mode without selecting any code, the model may try to autonomously search for the relevant file rather than using what's right in front of it, and that search can fail silently, leading to a generic answer instead of an actual fix. For single-file bug fixes, select the code and use inline edit (Ctrl+I) or attach the selection to chat (Ctrl+L) instead.

A correct result: the function gains a check for divisor == 0 (raising an error or returning a sensible default) and the crashing call either gets fixed or clearly flagged. That's proof the model is reasoning about your actual code, not just producing generic boilerplate.

Troubleshooting

Continue can't reach the model / "connection refused"

Confirm the Ollama container is running:

docker ps

And that port 11434 is mapped to the host in your compose file. Test directly:

curl http://localhost:11434/api/tags

If that fails, the problem is Ollama/Docker, not Continue.

Model responds but ignores your selected code

Check which mode you're in. Plain chat in Agent mode without an attached selection may try to search for context autonomously instead of using your selection. Switch to inline edit (Ctrl+I) or explicitly attach the selection to chat (Ctrl+L) so the code is guaranteed to be in the model's context.

Responses are slow or the editor freezes briefly

A 30B-class model doing inline generation is a meaningfully heavier load than short chat replies. Check nvidia-smi while a request is running — if VRAM is maxed out because ComfyUI or another model is also loaded, free that up first, or consider a smaller/more quantized model for the coding role specifically.

Series Navigation

← Part 4 — Image Generation
→ Part 6 — Remote Access