Digital Spaceport | Homelab Al Server Multi GPU Benchmarks - Multiple 3090s and 3060ti mixed PCIe VRAM Performance @DigitalSpaceport | Uploaded September 2024 | Updated October 2024, 3 hours ago.
Looking for information on what performance you can expect on your Homelab OpenwebUI and Ollama based Ai Home server? This is the video! We will be using Llama3.1 70b and assessing Tokens per Second on many model variations. Have questions about mixed VRAM setups? Need to run your GPU on a smaller PCIe slot than a full 16x? I dive into testing to answer these questions that YOU have asked recently around mixing GPUs of different VRAM size, adding extra GPUs to your own Ai Home Based Rig.

Ai GPU Benchmarking 3090s and 3060s + VRAM Testing - youtu.be/-heFPHKy3jY
Ollama Ai Home Server Build - youtu.be/JN4EhaM7vyw
Bare Metal OpenwebUI Ai Server Setup Guide - youtu.be/q_cDvCq1pww
Proxmox LXC Docker Ai Setup + GPU Passthrough - youtu.be/TmNSDkjDTOs
Results Table - digitalspaceport.com/llama3-1-70b-benchmarks-on-gpus

GPUs reviewed
3090 geni.us/3090GPU_y3jY
3060ti geni.us/3060tiGPU_y3jY

Ai Server Build
GPU Rack Frame geni.us/GPU_Rack_Frame
Gigabyte MZ32-AR0 Motherboard geni.us/mz32-ar0_motherboard
RTX 3090 24GB GPU (x4) geni.us/GPU3090
256GB DDR4 2400 RAM geni.us/256GB_DDR4_RAM
PCIe4 Risers (x4) geni.us/PCIe4_Riser_Cable
EPYC 7702 geni.us/EPYC_7702p
Corsair H170i ELITE CAPELLIX geni.us/iCUE_H170i_Capellix
SP3/TR4/5 Retention Kit geni.us/ELITE_RetentionKit
ARCTIC MX4 Thermal Paste geni.us/Arctic_ThermalPaste
CORSAIR HX1500i PSU geni.us/Corsair_HX1500iPSU
4i SFF-8654 to 4i SFF-8654 (x4) geni.us/SFF8654_to_SFF8654
HDD Rack Screws for Fans geni.us/HDD_RackScrews

Chapters
0:00 Home Ai Server GPU Benchmarking
2:06 70b q8 4x 3090 + 1x 3060ti Mixed Test
6:33 70b q4 4x 3090 + 1x 3060ti Mixed Test
7:31 MZ32-AR0 Bios PCIe Configuration
8:48 Llama 3.1 70b q8 4x 3090 Test
10:00 Llama 3.1 70b q8 4x 3090 Test
11:15 70b q8 3x 3090 VRAM Test
12:39 3x 3090 LLM Benchmark
16:33 2x 3090 Llama 3.1 70b q4 Benchmark
17:00 Conclusions

Be sure to 👍✅Subscribe✅👍 for more content like this!

Join this channel youtube.com/@DigitalSpaceport/join

Please share this video to help spread the word and drop a comment below with your thoughts or questions. Thanks for watching!

Digital Spaceport Website
🌐 digitalspaceport.com

🛒Shop (Channel members get a 3% or 5% discount)
Check out Shop.DigitalSpaceport.com for great deals on hardware and merch.

*****
As an Amazon Associate I earn from qualifying purchases.

When you click on links to various merchants on this site and make a purchase, this can result in this site earning a commission. Affiliate programs and affiliations include, but are not limited to, the eBay Partner Network.
*****