Uploaded August 2026 | Updated September 2026, 3 weeks ago
Building autonomous agents that actually work requires rethinking your architecture for long-running AI tasks. Learn why minimal design is the key to reliability.
Most developers struggle with agent fragility because they overcomplicate the underlying logic. This video breaks down the essential design principles—minimalism, transparency, and pluggability—required to build systems that handle extended processes without crashing. Whether you are scaling internal tools or building custom automation, these architectural shifts will help you create more stable software.
We also evaluate the trade-offs between cloud infrastructure and local AI agents. Often, local execution provides superior control for complex workflows, making your agents easier to debug and audit. If you are tired of black-box AI systems that fail silently, these concepts will provide a roadmap for better engineering practices.
Which autonomous agents have you abandoned recently and why did they fail? Subscribe for weekly software architecture breakdowns and practical AI engineering guides.
On April 4th, Anthropic stopped covering third-party harnesses under Claude subscription limits. The setup I'd been living in stopped working, so I tried to rebuild it on hardware I own — one RTX 3060 running a 35B model at about 70 tokens a second.
I tried OpenClaw. I tried Hermes. Both beat me. Not because they're bad, but because they were built for a model you rent: they assume tokens are cheap and they don't show you what they're spending them on. On my box I could afford neither the context nor the blindness. So I gave up on autonomous local AI entirely and settled for a coding agent plus some n8n automations.
Then I got annoyed that anything agentic meant leaving my laptop open, and I built a small web UI for a pi session on my server. Then I got lazy and added Telegram. Then schedules. Then memory.
Somewhere in there I looked at what I'd built and realised it was the thing I quit.
This is that story — including the bugs, which are the actual content: four .jsonl files that were supposed to be one conversation, a Telegram deadlock that took a day to see, and a local model that read its own identity file correctly and still told me it was made by Baidu. Each one taught me something about what a harness actually is.
And the part I didn't expect: every design decision that made this good came from the model being local. Context is the budget, so tools have to be earned instead of granted. Latency you'd never notice on a hosted API is painful when you're already waiting on prefill. Small models take instructions literally in ways Claude never does. None of those pressures exist when you're renting frontier tokens — which is exactly why nobody builds for them.
Nothing leaves the network. llama-server on localhost, portal on the LAN or over Tailscale.
⏱️ CHAPTERS
0:00 — The Laptop Is Closed and It's Still Working
0:58 — The Rule That Changed Under Me
2:03 — I Tried the Existing Harnesses. I Lost.
3:15 — One Small Annoyance (and the portal)
4:58 — Then I Got Lazy (Telegram, and the deadlock)
6:32 — Wait — I've Built This Before
7:44 — Schedules, and the Memory Problem
8:41 — "I'm Pi, Created by Baidu"
10:29 — Why It Only Worked Because It's Local
12:12 — What It Actually Is
13:51 — Which One Did You Give Up On?
🔗 LINKS
▶ Pithagoras : github.com/thecodacus/pithagoras
▶ pi : github.com/earendil-works/pi
▶ understory: github.com/thecodacus/understory
🛠️ THE RIG
RTX 3060 12GB
· Ryzen 5 5600X
· 32GB DDR4
Model: qwen36-35b-a3b-mtp (~70 tok/s) (mtp + custom llama.cpp improvement)
· 128k context budget
#localai #aiagents #selfhosted #openclaw #hermes #llamacpp
Building autonomous agents that actually work requires rethinking your architecture for long-running AI tasks. Learn why minimal design is the key to reliability.
Most developers struggle with agent fragility because they overcomplicate the underlying logic. This video breaks down the essential design principles—minimalism, transparency, and pluggability—required to build systems that handle extended processes without crashing. Whether you are scaling internal tools or building custom automation, these architectural shifts will help you create more stable software.
We also evaluate the trade-offs between cloud infrastructure and local AI agents. Often, local execution provides superior control for complex workflows, making your agents easier to debug and audit. If you are tired of black-box AI systems that fail silently, these concepts will provide a roadmap for better engineering practices.
Which autonomous agents have you abandoned recently and why did they fail? Subscribe for weekly software architecture breakdowns and practical AI engineering guides.
On April 4th, Anthropic stopped covering third-party harnesses under Claude subscription limits. The setup I'd been living in stopped working, so I tried to rebuild it on hardware I own — one RTX 3060 running a 35B model at about 70 tokens a second.
I tried OpenClaw. I tried Hermes. Both beat me. Not because they're bad, but because they were built for a model you rent: they assume tokens are cheap and they don't show you what they're spending them on. On my box I could afford neither the context nor the blindness. So I gave up on autonomous local AI entirely and settled for a coding agent plus some n8n automations.
Then I got annoyed that anything agentic meant leaving my laptop open, and I built a small web UI for a pi session on my server. Then I got lazy and added Telegram. Then schedules. Then memory.
Somewhere in there I looked at what I'd built and realised it was the thing I quit.
This is that story — including the bugs, which are the actual content: four .jsonl files that were supposed to be one conversation, a Telegram deadlock that took a day to see, and a local model that read its own identity file correctly and still told me it was made by Baidu. Each one taught me something about what a harness actually is.
And the part I didn't expect: every design decision that made this good came from the model being local. Context is the budget, so tools have to be earned instead of granted. Latency you'd never notice on a hosted API is painful when you're already waiting on prefill. Small models take instructions literally in ways Claude never does. None of those pressures exist when you're renting frontier tokens — which is exactly why nobody builds for them.
Nothing leaves the network. llama-server on localhost, portal on the LAN or over Tailscale.
⏱️ CHAPTERS
0:00 — The Laptop Is Closed and It's Still Working
0:58 — The Rule That Changed Under Me
2:03 — I Tried the Existing Harnesses. I Lost.
3:15 — One Small Annoyance (and the portal)
4:58 — Then I Got Lazy (Telegram, and the deadlock)
6:32 — Wait — I've Built This Before
7:44 — Schedules, and the Memory Problem
8:41 — "I'm Pi, Created by Baidu"
10:29 — Why It Only Worked Because It's Local
12:12 — What It Actually Is
13:51 — Which One Did You Give Up On?
🔗 LINKS
▶ Pithagoras : github.com/thecodacus/pithagoras
▶ pi : github.com/earendil-works/pi
▶ understory: github.com/thecodacus/understory
🛠️ THE RIG
RTX 3060 12GB
· Ryzen 5 5600X
· 32GB DDR4
Model: qwen36-35b-a3b-mtp (~70 tok/s) (mtp + custom llama.cpp improvement)
· 128k context budget
#localai #aiagents #selfhosted #openclaw #hermes #llamacpp




![How Fast Can One RTX 3060 Actually Run 35B (llama.cpp enhancement)?
I built an expert cache for MoE models, keep the hottest experts parked in VRAM, stream the rest from system RAM.
The baseline was 42. This is what it took to find out why, and where it ended up: 80 tokens a second at peak, and still over 70 with real context in the window.
One RTX 3060. A 35B model. The part I did not expect: the cache is worth almost nothing on its own. What makes it worth something is what it does to speculative decoding underneath it.
⚙️ THE TWO FLAGS
Everything else here is the command you already run. These two are new:
moe-cache-profile [path.csv]
moe-cache-slots [n]
Full commands, both steps, in the pinned comment.
⏱️ CHAPTERS
0:00 Intro
0:31 The graveyard
2:35 The data says dont
5:47 The build, and the gut punch
7:42 The scheduler betrayal
10:54 Turning it up
14:08 Why it works
17:58 Your GPU
📊 THE NUMBERS
baseline, no cache ............ 42 tok/s
cache working ............... 6 tok/s ← the two days
cache actually working ........ 44 tok/s
cache, 124 slots .............. 55 tok/s
speculative decoding alone .... 55 tok/s
both together ................. 70 tok/s ← more than either
parallel GPU/CPU chains ....... 75 tok/s
peak .......................... 80 tok/s
with real context ............. 70+ tok/s
⚠️ WILL IT HELP YOU?
Depends how much of the model fits. On a 3060 with the 35B, about half the experts fit and the gain is large. On a 118B model where only 36 of 256 experts fit, the gain drops to 5%. Below roughly 15–20% resident, a fixed set stops being the right approach at all. Chapter 7 covers where the line is for your card.
🔧 SETUP
Qwen3.6-35B-A3B UD-Q4_K_M
RTX 3060 12GB · Ryzen 5600X · 30GB RAM · Ubuntu 24.04 llama.cpp with MoE expert caching
—
If you run models locally and want to get more out of the hardware you already own, thats what this channel is for. Subscribe theres a lot more coming.
#localllm #llamacpp #rtx3060 #qwen3 #selfhostedai #localai How Fast Can One RTX 3060 Actually Run 35B (llama.cpp enhancement)?](https://i.ytimg.com/vi/k_LostFpatg/mqdefault.jpg)





![Can a 3.5GB model replace my 35B daily driver? (Bonsai 27B)
PrismML compressed Qwens 27B dense model down to 3.5 GB with true 1-bit weights. Not quantization: the weights were trained as −1/+1 from day one. So I put both Bonsai builds up against my actual daily driver, a 21 GB Qwen 3.6 35B MoE, plus Gemma 4 12B as the fair-size control, and ran about thirty tests on one RTX 3060.
Same landing-page brief. Same broken server over SSH. Same sampling settings, same KV cache quant, same effort level, every model.
What came out of it surprised me more than once. The 21 GB model is more than twice as fast as a 7 GB one, and the reason has nothing to do with file size. The only model that could not reliably finish a long task was not the smallest one. And whether Bonsai makes sense on your machine comes down to a number almost nobody quotes on a spec sheet.
If you run this on a Mac or a small card, tell me what you get in the comments.
CHAPTERS
0:00 Two models, one broken server
0:39 What Bonsai actually is (1-bit vs quantization)
3:24 Test 1 — can it design a webpage?
6:34 Test 2 — SSH debugging, everyone passed
9:26 Test 3 — the trap that survives a restart
13:29 Where it breaks (Gemmas loop)
15:35 Why the bigger model is faster (A3B vs dense)
17:54 The number nobody quotes (total footprint)
20:06 The verdict — who should run what
THE MODELS
Bonsai 27B (binary Q1_0, 3.5 GB / ternary Q2_0, 7 GB): [https://huggingface.co/prism-ml]
Qwen 3.6 35B-A3B: [https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF]
llama.cpp Metal Q2_0 support (merged): github.com/ggml-org/llama.cpp/pull/25419
llama.cpp CUDA Q2_0 support (open PR I built from): github.com/ggml-org/llama.cpp/pull/25707
THE RIG:
RTX 3060 12 GB
llama.cpp built from PR #25707
KV cache q8_0 (all models)
reasoning effort medium (all models)
MoE runs with 26 expert layers offloaded to system RAM
MY RESULTS AT A GLANCE
Generation speed at depth (tg, tokens/sec):
MoE 47.9 · Bonsai Q1 34.7 · Bonsai Q2 21.6
Total memory footprint:
MoE ~21 GB (10.5 VRAM + 10.6 RAM)
ternary 8.8 GB
binary 5.3 GB
Server-repair task, wall clock:
MoE ~56s
ternary ~114s
binary ~134s
Design test:
ternary and the MoE land in the same class.
Binary and Gemma sit a tier below.
#localai #llm #bonsai #quantization #1bit #qwen #llamacpp #selfhosted #ai #rtx3060 Can a 3.5GB model replace my 35B daily driver? (Bonsai 27B)](https://i.ytimg.com/vi/rBLWDJrXCp0/mqdefault.jpg)