Uploaded March 2026 | Updated September 2026, 3 weeks ago
I was spending $52/month on ChatGPT, ElevenLabs, and image generation APIs. So I moved everything local — LLMs, text-to-speech, image generation — all running
on hardware I own, for $0/month.
Here's the full stack:
- Ollama + Open WebUI (private ChatGPT replacement)
- Chatterbox TTS (beats ElevenLabs in blind tests — MIT licensed)
- ComfyUI (Stable Diffusion + Flux image generation)
- Tailscale (access everything from anywhere, no port forwarding)
Everything runs in Docker on a single GPU machine.
⏱️ Timestamps
00:00 The cost problem
00:36 The full stack overview
01:12 Hardware: it's all about VRAM
02:21 Ollama + Open WebUI setup
03:41 Chatterbox TTS setup
04:51 ComfyUI image generation
05:47 Tailscale remote access
06:44 The big picture
07:06 Final thoughts + real numbers
🔗 Resources
Docker Compose files → (link in pinned comment)
Ollama: ollama.com
Open WebUI: github.com/open-webui/open-webui
Chatterbox TTS: github.com/travisvn/chatterbox-tts
ComfyUI: github.com/comfyanonymous/ComfyUI
Tailscale: tailscale.com
💰 Hardware Recommendations
- 8GB VRAM → Small LLMs + SDXL (entry point)
- 12GB VRAM → Chatterbox + 14B models
- 16GB+ VRAM → Flux image generation
- 24GB VRAM → Everything runs simultaneously
This voiceover was generated by Chatterbox running on local hardware.
#ai #homelab #selfhosted #ollama #docker #privacy
I was spending $52/month on ChatGPT, ElevenLabs, and image generation APIs. So I moved everything local — LLMs, text-to-speech, image generation — all running
on hardware I own, for $0/month.
Here's the full stack:
- Ollama + Open WebUI (private ChatGPT replacement)
- Chatterbox TTS (beats ElevenLabs in blind tests — MIT licensed)
- ComfyUI (Stable Diffusion + Flux image generation)
- Tailscale (access everything from anywhere, no port forwarding)
Everything runs in Docker on a single GPU machine.
⏱️ Timestamps
00:00 The cost problem
00:36 The full stack overview
01:12 Hardware: it's all about VRAM
02:21 Ollama + Open WebUI setup
03:41 Chatterbox TTS setup
04:51 ComfyUI image generation
05:47 Tailscale remote access
06:44 The big picture
07:06 Final thoughts + real numbers
🔗 Resources
Docker Compose files → (link in pinned comment)
Ollama: ollama.com
Open WebUI: github.com/open-webui/open-webui
Chatterbox TTS: github.com/travisvn/chatterbox-tts
ComfyUI: github.com/comfyanonymous/ComfyUI
Tailscale: tailscale.com
💰 Hardware Recommendations
- 8GB VRAM → Small LLMs + SDXL (entry point)
- 12GB VRAM → Chatterbox + 14B models
- 16GB+ VRAM → Flux image generation
- 24GB VRAM → Everything runs simultaneously
This voiceover was generated by Chatterbox running on local hardware.
#ai #homelab #selfhosted #ollama #docker #privacy



![Can a 3.5GB model replace my 35B daily driver? (Bonsai 27B)
PrismML compressed Qwens 27B dense model down to 3.5 GB with true 1-bit weights. Not quantization: the weights were trained as −1/+1 from day one. So I put both Bonsai builds up against my actual daily driver, a 21 GB Qwen 3.6 35B MoE, plus Gemma 4 12B as the fair-size control, and ran about thirty tests on one RTX 3060.
Same landing-page brief. Same broken server over SSH. Same sampling settings, same KV cache quant, same effort level, every model.
What came out of it surprised me more than once. The 21 GB model is more than twice as fast as a 7 GB one, and the reason has nothing to do with file size. The only model that could not reliably finish a long task was not the smallest one. And whether Bonsai makes sense on your machine comes down to a number almost nobody quotes on a spec sheet.
If you run this on a Mac or a small card, tell me what you get in the comments.
CHAPTERS
0:00 Two models, one broken server
0:39 What Bonsai actually is (1-bit vs quantization)
3:24 Test 1 — can it design a webpage?
6:34 Test 2 — SSH debugging, everyone passed
9:26 Test 3 — the trap that survives a restart
13:29 Where it breaks (Gemmas loop)
15:35 Why the bigger model is faster (A3B vs dense)
17:54 The number nobody quotes (total footprint)
20:06 The verdict — who should run what
THE MODELS
Bonsai 27B (binary Q1_0, 3.5 GB / ternary Q2_0, 7 GB): [https://huggingface.co/prism-ml]
Qwen 3.6 35B-A3B: [https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF]
llama.cpp Metal Q2_0 support (merged): github.com/ggml-org/llama.cpp/pull/25419
llama.cpp CUDA Q2_0 support (open PR I built from): github.com/ggml-org/llama.cpp/pull/25707
THE RIG:
RTX 3060 12 GB
llama.cpp built from PR #25707
KV cache q8_0 (all models)
reasoning effort medium (all models)
MoE runs with 26 expert layers offloaded to system RAM
MY RESULTS AT A GLANCE
Generation speed at depth (tg, tokens/sec):
MoE 47.9 · Bonsai Q1 34.7 · Bonsai Q2 21.6
Total memory footprint:
MoE ~21 GB (10.5 VRAM + 10.6 RAM)
ternary 8.8 GB
binary 5.3 GB
Server-repair task, wall clock:
MoE ~56s
ternary ~114s
binary ~134s
Design test:
ternary and the MoE land in the same class.
Binary and Gemma sit a tier below.
#localai #llm #bonsai #quantization #1bit #qwen #llamacpp #selfhosted #ai #rtx3060 Can a 3.5GB model replace my 35B daily driver? (Bonsai 27B)](https://i.ytimg.com/vi/rBLWDJrXCp0/mqdefault.jpg)






