How I Built a Truly Private AI: The Full Software Setup @2GuysTek
How I Built a Truly Private AI: The Full Software Setup  @2GuysTek
Uploaded September 2026 | Updated November 2026, 2 days ago
This is the second and final video on my 3x RTX 3090 beastly local AI server. I promised you a complete walkthrough of the AI software stack, start to finish, and friends, I'm here to deliver! In Part 2 of my local AI server build, I break down the entire self-hosted AI stack running on this box: Ollama, Open WebUI, Qdrant, Apache Tika, SearXNG, Infinity, and Open Terminal. I cover what each one does, why I picked it, and provide example scenarios of how they all work together in one Docker Compose file.

I also cover the most fundamental lesson I learned while building this system: AI isn't magic; it's just software. The "it just knows" experience you get from ChatGPT or Claude is really a model calling a bunch of tools behind the scenes. So I walk through three real scenarios step by step: searching the web for current info, chatting with your own documents using RAG with hybrid search and reranking, and having the model write and run its own script in a sandboxed terminal.

Then we go through the full compose file block by block, so you can deploy the same stack on your own hardware.

The result is seven containers and one compose file on a single box. It can search the web, read your documents, and write and run its own code, and none of your prompts go to a cloud model.

๐Ÿ”— Get the Docker Compose file: 2guystek.tv/homelab/ai/infrastructure/2026/07/31/building-my-ai-homelab-server-three-rtx-3090s
๐Ÿ”— Part 1, the hardware build: youtube.com/watch?v=TRpr_qKydSo

Root-level Patreon and YouTube members get access to this system. Come build with us:
Patreon: patreon.com/c/2GuysTek
YouTube Membership: youtube.com/@2GuysTek/membership

**GET SOCIAL AND MORE WITH US HERE!**
Get help with your Homelab, ask questions, and chat with us!
๐ŸŽฎ discord.gg/5TcfBWBB7S

**Check out HomeLab Gear for all your homelab needs!**
๐Ÿ’ฟ https://homelabgear.shop/

Subscribe and follow us on all the socials, would ya?
๐Ÿ“ธ instagram.com/2guystek
๐Ÿ’ป facebook.com/2guystek

Find all things 2GT on our website!
๐ŸŒ 2guystek.tv

More of a podcast kinda person? Check out our Podcast here:
๐ŸŽ™๏ธ buzzsprout.com/1852562

** Timestamps **
00:00 | Intro
01:52 | AI isn't magic, it's tools
03:47 | Foundation: Linux and Docker
05:13 | Model runner: Ollama
06:13 | User interface: Open WebUI
07:44 | Vector database: Qdrant
08:50 | Document extraction: Apache Tika
09:34 | Web search: SearXNG
10:31 | Reranking: Infinity
11:27 | Sandboxed CLI: Open Terminal
12:43 | How it all connects
13:15 | Three scenarios to show how it works
18:06 | Why sandboxing matters
18:48 | Docker Compose walkthrough
19:23 | Open WebUI config
21:05 | Qdrant config
21:31 | Tika config
21:45 | SearXNG config
22:20 | Ollama config
23:21 | Open Terminal config
23:50 | Infinity config
24:30 | Placeholders and spinning it up
25:04 | Wrap up: Digital sovereignty
How I Built a Truly Private AI: The Full Software SetupHow I learned to love backups again! - Synology Active Backup for Business3D-Printed Mega Homelab Server Build!End of year wrap-up & #Homelab Q&A!2GT #Homelab Hour!  - EP 63 - Eaton up my rack space!The end is here - VMware by BroadcomBackup and Disaster Recovery - Tech ChatsComplete Homelab Tour! - Hardware, Networking, and Apps!The Founder Who Built a Cloud to Fit Your Business, Not the Other Way Around2GT #Homelab Hour - EP 34 - Yes, I am a traitor to pfSense!The Homelab Hour w/ Special Guest Raid Owl!The #Homelab Hour! - EP 10
2GuysTek |

How I Built a Truly Private AI: The Full Software Setup

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER