Uploaded August 2026 | Updated September 2026, 3 weeks ago
Run 360GB models on 8GB RAM using Colibri technology. This approach challenges NVIDIA dominance in local LLM inference.
The demand for massive hardware to support large language models is changing. While tools like Lama.cpp started the movement, Colibri technology introduces a new method for handling model components. We look at how data processing is optimized to bypass the typical reliance on high-end GPUs, making 360GB models accessible on standard consumer CPUs.
By visualizing the internal model processing, you can see exactly where the efficiency gains originate. This breakdown is intended for anyone building local AI infrastructure on a budget who needs to run large language models without shelling out for enterprise-grade hardware. It is a clear look at a potential path forward for local LLM inference that prioritizes accessibility over expensive component requirements.
Subscribe for weekly local AI infrastructure breakdowns, and tell us in the comments if you want a direct performance test against Lama.cpp.
CHAPTERS
00:00 The claim
00:46 What it claims
04:04 Why I was skeptical
06:42 The setup
08:32 Run one — colibrì
11:11 Run two — llama.cpp
13:26 The dig
15:24 Run two, properly
19:05 Pushing harder
22:12 The verdict
THE SETUP
Model:
DeepSeek-V4-Flash-0731, 284B total / 13B active, 43 layers, 256 routed experts , plus 1 shared, top-6 routing
Box:
Ryzen 5 5600X (12 threads, no AVX-512),
61 GB RAM, RTX 3060 12 GB, NVMe
Weights
colibrì: official HF checkpoint, 166.9 GB, no conversion
llama.cpp: unsloth UD-Q8_K_XL, 161.9 GB
Size-matched within 3.1%, both verified byte-exact against the HF API.
THE FLAG
llama.cpp will not load this model at all without -nr (--no-repack). Without it, it tries to build a 147 GB repacked buffer in RAM and dies.
That flag is not in the docs folder, it's in the argument parser source and a couple of tool READMEs. llama-bench doesn't accept it at all, which is why every measurement here was done by hand.
LINKS
colibrì — github.com/JustVugg/colibri
llama.cpp — github.com/ggml-org/llama.cpp
The open work on this, in llama.cpp:
github.com/ggml-org/llama.cpp/pull/24524
github.com/ggml-org/llama.cpp/pull/25294
github.com/ggml-org/llama.cpp/pull/25932
github.com/ggml-org/llama.cpp/pull/26003
github.com/ikawrakow/ik_llama.cpp/pull/2101
Other people building in this space:
github.com/lyogavin/airllm
github.com/Helldez/BigMoeOnEdge
github.com/FareedKhan-dev/kimi-k3-in-c
Every number in this video came off my own box. If you get different results on yours I genuinely want to hear about it — the whole point is that this stuff is testable.
What's the biggest model you've got running at home, and what did it take to get there?
Comment below.
#colibri #llamacpp #deepseek #homelab #localai
Run 360GB models on 8GB RAM using Colibri technology. This approach challenges NVIDIA dominance in local LLM inference.
The demand for massive hardware to support large language models is changing. While tools like Lama.cpp started the movement, Colibri technology introduces a new method for handling model components. We look at how data processing is optimized to bypass the typical reliance on high-end GPUs, making 360GB models accessible on standard consumer CPUs.
By visualizing the internal model processing, you can see exactly where the efficiency gains originate. This breakdown is intended for anyone building local AI infrastructure on a budget who needs to run large language models without shelling out for enterprise-grade hardware. It is a clear look at a potential path forward for local LLM inference that prioritizes accessibility over expensive component requirements.
Subscribe for weekly local AI infrastructure breakdowns, and tell us in the comments if you want a direct performance test against Lama.cpp.
CHAPTERS
00:00 The claim
00:46 What it claims
04:04 Why I was skeptical
06:42 The setup
08:32 Run one — colibrì
11:11 Run two — llama.cpp
13:26 The dig
15:24 Run two, properly
19:05 Pushing harder
22:12 The verdict
THE SETUP
Model:
DeepSeek-V4-Flash-0731, 284B total / 13B active, 43 layers, 256 routed experts , plus 1 shared, top-6 routing
Box:
Ryzen 5 5600X (12 threads, no AVX-512),
61 GB RAM, RTX 3060 12 GB, NVMe
Weights
colibrì: official HF checkpoint, 166.9 GB, no conversion
llama.cpp: unsloth UD-Q8_K_XL, 161.9 GB
Size-matched within 3.1%, both verified byte-exact against the HF API.
THE FLAG
llama.cpp will not load this model at all without -nr (--no-repack). Without it, it tries to build a 147 GB repacked buffer in RAM and dies.
That flag is not in the docs folder, it's in the argument parser source and a couple of tool READMEs. llama-bench doesn't accept it at all, which is why every measurement here was done by hand.
LINKS
colibrì — github.com/JustVugg/colibri
llama.cpp — github.com/ggml-org/llama.cpp
The open work on this, in llama.cpp:
github.com/ggml-org/llama.cpp/pull/24524
github.com/ggml-org/llama.cpp/pull/25294
github.com/ggml-org/llama.cpp/pull/25932
github.com/ggml-org/llama.cpp/pull/26003
github.com/ikawrakow/ik_llama.cpp/pull/2101
Other people building in this space:
github.com/lyogavin/airllm
github.com/Helldez/BigMoeOnEdge
github.com/FareedKhan-dev/kimi-k3-in-c
Every number in this video came off my own box. If you get different results on yours I genuinely want to hear about it — the whole point is that this stuff is testable.
What's the biggest model you've got running at home, and what did it take to get there?
Comment below.
#colibri #llamacpp #deepseek #homelab #localai
