Self-Hosted AI Stack — Part 4: Image Generation

Why ComfyUI instead of a simpler UI

ComfyUI is node-based rather than form-based — every step of image generation (loading a checkpoint, encoding your prompt, sampling, decoding back to pixels) is its own node that you wire together explicitly. That's more setup friction than a single-page UI, but it means you can see and control exactly what's happening at each stage, which matters a lot once you're troubleshooting a generation that isn't coming out right — you can inspect and modify any single step in isolation.

Getting your first checkpoint

Download an SDXL checkpoint — AlbedoBase XL is a solid general-purpose starting point — and place it in ~/ai-stack/comfyui/basedir/models/checkpoints/. ComfyUI scans that folder on startup, so restart the container (or use its refresh option) if a newly-added checkpoint doesn't show up in the dropdown.

The basic text-to-image graph

Six nodes, wired together, is the minimum viable graph:

1. Load Checkpoint — select your .safetensors file
2. CLIP Text Encode (Prompt) × 2 — one for positive prompt, one for negative
3. Empty Latent Image — sets your canvas dimensions
4. KSampler — the actual diffusion sampling step
5. VAE Decode — turns the sampled latent back into a viewable image
6. Save Image

Wiring: Load Checkpoint's MODEL → KSampler's model; your two CLIP Text Encode nodes' CONDITIONING outputs → KSampler's positive and negative; Empty Latent's LATENT → KSampler's latent_image; KSampler's LATENT output → VAE Decode's samples; Load Checkpoint's VAE → VAE Decode's vae; VAE Decode's IMAGE → Save Image.

Resolution: stay near your model's native training resolution

This is the single most common cause of distorted, duplicated-looking output, and it's easy to get wrong. SDXL-family models are trained around roughly 1 megapixel (1024×1024, or equivalent-area buckets like 896×1152). Pushing your Empty Latent Image well past that — say, to 1280×2048 (2.6 megapixels) — doesn't just make a bigger picture. It pushes the model outside the composition scale it actually learned, and the classic symptom is duplicated limbs, elongated bodies, or warped anatomy, especially with more than one subject in frame.

Samplers and settings that work well as defaults

dpmpp_2m with the karras scheduler is a solid, fast, reliable default — 20–25 steps, cfg around 7.0. dpmpp_2m_sde is a reasonable step up in fine detail at a small speed cost. Avoid plain dpm_2 (note the missing "pp") unless you have a specific reason — it's an older, generally lower-quality sampler that's easy to select by accident since the names look similar in the dropdown.

Negative prompts: a solid general-purpose starting point

worst quality, low quality, blurry, jpeg artifacts, watermark, signature,
text, extra limbs, extra fingers, missing fingers, deformed hands, bad
anatomy, disfigured, mutated, cropped, out of frame

Treat this as a template, not gospel — once you see actual output, you'll notice which specific artifacts keep showing up and can tune the list toward those.

The hard part: getting multiple subjects right

This is where most of the real troubleshooting time goes, and it's worth understanding why it's hard rather than just memorizing fixes.

Why models struggle with counting and multiple subjects

Diffusion models are noticeably weaker at exact subject counts than at single-subject fidelity — ask for three people and you'll often get two, four, or subjects that partially merge into each other. This gets worse the more visually similar the subjects are.

Fix 1 — tag-based reinforcement

If your checkpoint is trained on Danbooru-style tagging (many anime/illustration-focused SDXL finetunes are), use explicit count tags near the front of your prompt — 2boys, 1girl, 2boys, 3girls — rather than relying on natural-language phrasing like "three people." These are literal trained tags and work far more reliably than prose.

Give each subject its own tight cluster of concrete visual descriptors, not just a name or role — a bare word like "hero" gives the model little to render distinctly, while muscular hero, lion pelt cloak, club weapon gives it something concrete to latch onto.

Fix 2 — distinct poses beat identical poses

Subjects in visibly different poses relative to each other separate far more reliably than subjects in similar poses. Two figures both "standing side by side" are prone to merging; a scene where one figure is kneeling and another has their head resting in someone's lap gives the model strong distinct spatial anchors and tends to compose correctly without any extra work.

Fix 3 — regional conditioning

ComfyUI's Conditioning (Set Area) + Conditioning (Combine) nodes let you assign different prompt text to different regions of the canvas — encode each subject separately, wrap each in a Set Area node with pixel coordinates, and combine the results before feeding KSampler's positive input.

Two things to get right:

• Area height should match your full canvas height, not just an upper portion — if you set a region's height smaller than the canvas, only that vertical slice gets subject-specific reinforcement
• Overlap your regions slightly (10–25%) rather than splitting the canvas with a hard boundary — a hard seam tends to produce an awkward gap or visible dividing line between subjects

Fix 4 — two-pass inpainting compositing (the reliable fallback)

When regional conditioning isn't cutting it, generating each subject separately and compositing via inpainting is the most reliable option using only stock ComfyUI nodes:

1. Generate your first subject alone on the full canvas, positioned toward one side of the frame
2. Load that output image back in, and use ComfyUI's built-in Mask Editor (right-click the image thumbnail → Open in MaskEditor) to paint a mask over the empty space where your second subject should appear
3. Feed the image and mask into a VAE Encode (for Inpainting) node — search for it specifically, it's different from plain VAE Encode — along with your VAE
4. Connect its LATENT output to KSampler's latent_image, switch positive to your second subject's prompt, and set denoise around 0.9–0.95
5. Run — the masked region regenerates with your second subject while the rest of the image stays untouched

This guarantees separation because you're explicitly directing where the new content goes, rather than hoping conditioning math keeps two similar subjects apart.

Troubleshooting

"2 errors found" banner in ComfyUI with no obvious cause

Click "View details" — but also manually check for two extremely common culprits it doesn't always surface clearly: an unconnected negative input on KSampler (every positive needs a matching negative, even an empty one), and a Load Image node with no actual file selected (its placeholder text can look like a valid filename at a glance).

Duplicated limbs, warped anatomy, or "extra person" artifacts

Check your Empty Latent Image dimensions first — this is disproportionately caused by generating well above your model's native training resolution, even more than prompt wording. See the Resolution section above.

Two subjects merge into one, or inherit each other's features

Work through the fixes in order: count tags → concrete per-subject descriptors → distinct poses → regional conditioning → inpainting compositing. Stop as soon as one actually solves it — don't jump straight to inpainting if a simple pose change would have fixed it.

Inpainted region doesn't change at all, even after fixing wiring

Almost always an empty mask. Wire your Load Image node's MASK output into a temporary Preview Image node — a mask renders as black-and-white, and solid black means nothing is actually marked for inpainting. If it's blank, the mask editor save didn't take — repaint and confirm the shape is visible before wiring it back in.

Series Navigation

• ← Part 3 — Core Services
• → Part 5 — IDE Coding Agent