Uploaded May 2025 | Updated September 2026, 2 weeks ago
Building your own Private AI server is one of the most fun and exciting projects you can try out in your home lab environment. It is also a great way to learn more about AI, LLM models, and GPU passthrough configuration. Self-hosting and configuring your own AI environment is not as difficult as it might sound. Let's take a look at Proxmox, Ollama, and OpenWebUI, as well as Docker Desktop, GPU passthrough a my best practices and tips and tricks. I also show you what projects I am using Private AI servers for. Check it out!
GPU passthrough to LXC containers in Proxmox: virtualizationhowto.com/2025/05/how-to-enable-gpu-passthrough-to-lxc-containers-in-proxmox
Run Ollama with NVIDIA GPU in Proxmox VMs and LXC containers: virtualizationhowto.com/2025/05/run-ollama-with-nvidia-gpu-in-proxmox-vms-and-lxc-containers
My 2025 Proxmox Build (affiliate links)
Minisforum BD795M – geni.us/bd795m
RackChoice 2U Micro ATX Compact – geni.us/rackchoice2u
Cooler Master MWE Gold 850 V2 – geni.us/coolermaster850
Crucial 128 GB 5600MT/sec RAM – geni.us/crucial128gbramkit
Noctua NH-L9i-17xx, Premium Low-Profile CPU Cooler – amzn.to/4hAKIwG
Crucial 96GB kit of DDR5 SODIMM memory kit – geni.us/noctuanhl9i17xx
Intel X520-DA2 10 GbE network adapter – geni.us/intelx520
Kingston 240 GB drive for boot – geni.us/kingston240
Samsung EVO 990 Pro 2TB – geni.us/samsung990pro2tb
MX-4 Thermal paste – geni.us/mx4thermalpaste
Case fans: geni.us/arcticp8
Check out the VHT forums to get your questions answered: virtualizationhowto.com/community
Join the coolest home lab community here!
skool.com/homelabexplorers/about?ref=25f64c297b724689ae81c7dd30ba2c21
★ Subscribe to the channel: youtube.com/channel/UCrxcWtpd1IGHG9RbD_9380A?sub_confirmation=1
★ My blog: virtualizationhowto.com
★ Twitter: twitter.com/vspinmaster
★ LinkedIn: linkedin.com/in/brandon-lee-vht
★ Github: github.com/brandonleegit
★ Facebook: facebook.com/people/VirtualizationHowto/100092747277326
★ Discord: discord.gg/Zb46NV6mB3
★ Pinterest: pinterest.com/brandonleevht
Introduction to Private AI in the home lab - 0:00
Why self-host GPT for private AI - 0:54
Prerequisites to running private AI at home - 1:27
Can you run this on something old? 1:55
Docker to play around with AI - 2:12
Ollama overview and what it does - 2:33
OpenWebUI chatGPT style interface to interact with Ollama - 2:44
Using Docker and overview of the setup - 3:06
Docker run or Compose can be used - 3:49
Looking at Docker Desktop - 4:11
Command to spin up ollama in Docker Desktop - 4:36
Spinning the OpenWebUI container from the docker command line - 5:55
Overview of what happens after we spin up both containers - 7:06
Showing how to browse out to the OpenWebUI - 7:40
First steps and adding models to your Ollama instance - 7:47
Talking about better performance with LLMs with GPUs - 8:17
Enabling the NVIDIA container toolkit and when needed - 8:43
Overview of GPU passthrough with Proxmox - 9:00
Options for running GPU passthrough in Proxmox (LXC containers, and Virtual Machines) - 10:00
Steps for GPU passthrough overview - 10:50
IOMMU requirement - 11:05
Checking IOMMU - 11:20
Grepping for the device - 11:56
Claiming the IDs for GPU passthrough in Proxmox - 12:53
Blacklisting the drivers for the host - 13:02
Seeing the RTX video card recognized in the VM running in Proxmox - 13:29
Adding a PCI Device in your virtual machine - 13:51
Passthrough challenges in Proxmox - 15:00
Performance of GPU - 15:41
Quick tips and best practices of using this in the home lab - 16:20
Wrapping up self-hosting private AI in your home - 17:00
Building your own Private AI server is one of the most fun and exciting projects you can try out in your home lab environment. It is also a great way to learn more about AI, LLM models, and GPU passthrough configuration. Self-hosting and configuring your own AI environment is not as difficult as it might sound. Let's take a look at Proxmox, Ollama, and OpenWebUI, as well as Docker Desktop, GPU passthrough a my best practices and tips and tricks. I also show you what projects I am using Private AI servers for. Check it out!
GPU passthrough to LXC containers in Proxmox: virtualizationhowto.com/2025/05/how-to-enable-gpu-passthrough-to-lxc-containers-in-proxmox
Run Ollama with NVIDIA GPU in Proxmox VMs and LXC containers: virtualizationhowto.com/2025/05/run-ollama-with-nvidia-gpu-in-proxmox-vms-and-lxc-containers
My 2025 Proxmox Build (affiliate links)
Minisforum BD795M – geni.us/bd795m
RackChoice 2U Micro ATX Compact – geni.us/rackchoice2u
Cooler Master MWE Gold 850 V2 – geni.us/coolermaster850
Crucial 128 GB 5600MT/sec RAM – geni.us/crucial128gbramkit
Noctua NH-L9i-17xx, Premium Low-Profile CPU Cooler – amzn.to/4hAKIwG
Crucial 96GB kit of DDR5 SODIMM memory kit – geni.us/noctuanhl9i17xx
Intel X520-DA2 10 GbE network adapter – geni.us/intelx520
Kingston 240 GB drive for boot – geni.us/kingston240
Samsung EVO 990 Pro 2TB – geni.us/samsung990pro2tb
MX-4 Thermal paste – geni.us/mx4thermalpaste
Case fans: geni.us/arcticp8
Check out the VHT forums to get your questions answered: virtualizationhowto.com/community
Join the coolest home lab community here!
skool.com/homelabexplorers/about?ref=25f64c297b724689ae81c7dd30ba2c21
★ Subscribe to the channel: youtube.com/channel/UCrxcWtpd1IGHG9RbD_9380A?sub_confirmation=1
★ My blog: virtualizationhowto.com
★ Twitter: twitter.com/vspinmaster
★ LinkedIn: linkedin.com/in/brandon-lee-vht
★ Github: github.com/brandonleegit
★ Facebook: facebook.com/people/VirtualizationHowto/100092747277326
★ Discord: discord.gg/Zb46NV6mB3
★ Pinterest: pinterest.com/brandonleevht
Introduction to Private AI in the home lab - 0:00
Why self-host GPT for private AI - 0:54
Prerequisites to running private AI at home - 1:27
Can you run this on something old? 1:55
Docker to play around with AI - 2:12
Ollama overview and what it does - 2:33
OpenWebUI chatGPT style interface to interact with Ollama - 2:44
Using Docker and overview of the setup - 3:06
Docker run or Compose can be used - 3:49
Looking at Docker Desktop - 4:11
Command to spin up ollama in Docker Desktop - 4:36
Spinning the OpenWebUI container from the docker command line - 5:55
Overview of what happens after we spin up both containers - 7:06
Showing how to browse out to the OpenWebUI - 7:40
First steps and adding models to your Ollama instance - 7:47
Talking about better performance with LLMs with GPUs - 8:17
Enabling the NVIDIA container toolkit and when needed - 8:43
Overview of GPU passthrough with Proxmox - 9:00
Options for running GPU passthrough in Proxmox (LXC containers, and Virtual Machines) - 10:00
Steps for GPU passthrough overview - 10:50
IOMMU requirement - 11:05
Checking IOMMU - 11:20
Grepping for the device - 11:56
Claiming the IDs for GPU passthrough in Proxmox - 12:53
Blacklisting the drivers for the host - 13:02
Seeing the RTX video card recognized in the VM running in Proxmox - 13:29
Adding a PCI Device in your virtual machine - 13:51
Passthrough challenges in Proxmox - 15:00
Performance of GPU - 15:41
Quick tips and best practices of using this in the home lab - 16:20
Wrapping up self-hosting private AI in your home - 17:00
