Uploaded June 2026 | Updated September 2026, 3 weeks ago
Your laptop can already run ChatGPT-class AI โ the model was never the hard part. The hard part is everything around it: a chat UI, a private knowledge base that reads your own documents, a coding agent, automations that run while you sleep โ the whole stack, on hardware you own, with nothing leaving your machine.
This is the full build. One local engine (llama.cpp), and every tool you'd actually reach for hanging off a single OpenAI-compatible endpoint โ no cloud, no subscriptions, no price change or policy decision that can take it away from you.
What we wire up:
๐ฑ The engine โ llama.cpp, with llama-server / llama-swap as the router
๐ฌ A chat UI โ AnythingLLM (or Open WebUI)
๐ RAG โ chat with your own PDFs, fully local (โ goodbye NotebookLM)
โจ๏ธ A coding agent โ Pi, running on your own models (โ Cursor / Copilot)
๐ค Automation โ n8n agents working 24/7 (โ Zapier)
๐ Bonus โ the homelab tips that turn this from a weekend project into something you actually rely on
โฑ Chapters
0:00 Why local AI isn't optional
1:08 The engine โ llama.cpp + the router
2:55 Chat UI โ AnythingLLM
4:49 RAG โ chat with your own documents
7:08 Local coding agent โ Pi
9:05 Automation โ n8n agents that run 24/7
12:38 Bonus โ homelab tips for an always-on rig
13:54 The full stack โ taking back control
๐ Tools & resources
โข llama.cpp โ github.com/ggml-org/llama.cpp
โข llama-swap โ github.com/mostlygeek/llama-swap
โข AnythingLLM โ anythingllm.com
โข Open WebUI โ openwebui.com
โข Pi coding agent โ https://pi.dev
โข OpenCode โ opencode.ai
โข n8n โ n8n.io
โข Portainer โ portainer.io
โข Tailscale โ tailscale.com
โข ๐ My full llama-server setup guide โ youtu.be/0AqpaFm11oI
โข AnythingLLM is built by Tim Carambat โ go check out his channel
This is the stack I run myself. One engine, every tool branching off it, all on hardware you own โ that's what taking back control actually looks like.
What would you wire up first? Drop a comment. And if there's a piece you want me to go deeper on, tell me โ I'm building the whole self-hosted stack, one piece at a time.
๐ Subscribe for the rest of the build.
#localai #selfhostedai #llamacpp #homelab #privateai #n8n #codingagent #retrievalaugmentedgeneration #chatgpt #anthropic #googleai
Your laptop can already run ChatGPT-class AI โ the model was never the hard part. The hard part is everything around it: a chat UI, a private knowledge base that reads your own documents, a coding agent, automations that run while you sleep โ the whole stack, on hardware you own, with nothing leaving your machine.
This is the full build. One local engine (llama.cpp), and every tool you'd actually reach for hanging off a single OpenAI-compatible endpoint โ no cloud, no subscriptions, no price change or policy decision that can take it away from you.
What we wire up:
๐ฑ The engine โ llama.cpp, with llama-server / llama-swap as the router
๐ฌ A chat UI โ AnythingLLM (or Open WebUI)
๐ RAG โ chat with your own PDFs, fully local (โ goodbye NotebookLM)
โจ๏ธ A coding agent โ Pi, running on your own models (โ Cursor / Copilot)
๐ค Automation โ n8n agents working 24/7 (โ Zapier)
๐ Bonus โ the homelab tips that turn this from a weekend project into something you actually rely on
โฑ Chapters
0:00 Why local AI isn't optional
1:08 The engine โ llama.cpp + the router
2:55 Chat UI โ AnythingLLM
4:49 RAG โ chat with your own documents
7:08 Local coding agent โ Pi
9:05 Automation โ n8n agents that run 24/7
12:38 Bonus โ homelab tips for an always-on rig
13:54 The full stack โ taking back control
๐ Tools & resources
โข llama.cpp โ github.com/ggml-org/llama.cpp
โข llama-swap โ github.com/mostlygeek/llama-swap
โข AnythingLLM โ anythingllm.com
โข Open WebUI โ openwebui.com
โข Pi coding agent โ https://pi.dev
โข OpenCode โ opencode.ai
โข n8n โ n8n.io
โข Portainer โ portainer.io
โข Tailscale โ tailscale.com
โข ๐ My full llama-server setup guide โ youtu.be/0AqpaFm11oI
โข AnythingLLM is built by Tim Carambat โ go check out his channel
This is the stack I run myself. One engine, every tool branching off it, all on hardware you own โ that's what taking back control actually looks like.
What would you wire up first? Drop a comment. And if there's a piece you want me to go deeper on, tell me โ I'm building the whole self-hosted stack, one piece at a time.
๐ Subscribe for the rest of the build.
#localai #selfhostedai #llamacpp #homelab #privateai #n8n #codingagent #retrievalaugmentedgeneration #chatgpt #anthropic #googleai


![Can a 3.5GB model replace my 35B daily driver? (Bonsai 27B)
PrismML compressed Qwens 27B dense model down to 3.5 GB with true 1-bit weights. Not quantization: the weights were trained as โ1/+1 from day one. So I put both Bonsai builds up against my actual daily driver, a 21 GB Qwen 3.6 35B MoE, plus Gemma 4 12B as the fair-size control, and ran about thirty tests on one RTX 3060.
Same landing-page brief. Same broken server over SSH. Same sampling settings, same KV cache quant, same effort level, every model.
What came out of it surprised me more than once. The 21 GB model is more than twice as fast as a 7 GB one, and the reason has nothing to do with file size. The only model that could not reliably finish a long task was not the smallest one. And whether Bonsai makes sense on your machine comes down to a number almost nobody quotes on a spec sheet.
If you run this on a Mac or a small card, tell me what you get in the comments.
CHAPTERS
0:00 Two models, one broken server
0:39 What Bonsai actually is (1-bit vs quantization)
3:24 Test 1 โ can it design a webpage?
6:34 Test 2 โ SSH debugging, everyone passed
9:26 Test 3 โ the trap that survives a restart
13:29 Where it breaks (Gemmas loop)
15:35 Why the bigger model is faster (A3B vs dense)
17:54 The number nobody quotes (total footprint)
20:06 The verdict โ who should run what
THE MODELS
Bonsai 27B (binary Q1_0, 3.5 GB / ternary Q2_0, 7 GB): [https://huggingface.co/prism-ml]
Qwen 3.6 35B-A3B: [https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF]
llama.cpp Metal Q2_0 support (merged): github.com/ggml-org/llama.cpp/pull/25419
llama.cpp CUDA Q2_0 support (open PR I built from): github.com/ggml-org/llama.cpp/pull/25707
THE RIG:
RTX 3060 12 GB
llama.cpp built from PR #25707
KV cache q8_0 (all models)
reasoning effort medium (all models)
MoE runs with 26 expert layers offloaded to system RAM
MY RESULTS AT A GLANCE
Generation speed at depth (tg, tokens/sec):
MoE 47.9 ยท Bonsai Q1 34.7 ยท Bonsai Q2 21.6
Total memory footprint:
MoE ~21 GB (10.5 VRAM + 10.6 RAM)
ternary 8.8 GB
binary 5.3 GB
Server-repair task, wall clock:
MoE ~56s
ternary ~114s
binary ~134s
Design test:
ternary and the MoE land in the same class.
Binary and Gemma sit a tier below.
#localai #llm #bonsai #quantization #1bit #qwen #llamacpp #selfhosted #ai #rtx3060 Can a 3.5GB model replace my 35B daily driver? (Bonsai 27B)](https://i.ytimg.com/vi/rBLWDJrXCp0/mqdefault.jpg)






