Digital Spaceport | Homelab AI Server Multi GPU Benchmarks - Dual 4090s + 1070ti added in (CRAZY Results!) @DigitalSpaceport | Uploaded September 2024 | Updated October 2024, 1 hour ago.
We are benchmarking the killer Proxmox based, LXC and Docker Homelab running OpenwebUI and Ollama. We are continuing our Ai GPU Llama 3.1 testing at several parameter levels and mixed GPU Generation and today we are looking at dual 4090s and tossing in a real wildcard extra GPU, an older 1070ti. The results are pretty crazy!
Ai GPU Benchmarking 3090s and 3060s + VRAM Testing - youtu.be/-heFPHKy3jY
Ollama Ai Home Server Build - youtu.be/JN4EhaM7vyw
Bare Metal OpenwebUI Ai Server Setup Guide - youtu.be/q_cDvCq1pww
Proxmox LXC Docker Ai Setup + GPU Passthrough - youtu.be/TmNSDkjDTOs
Results Table - digitalspaceport.com/llama3-1-70b-benchmarks-on-gpus
GPUs
4090 geni.us/4090GPU_5N_s
1070ti geni.us/1070tiGPU_5N_s
Ai Server Build
GPU Rack Frame geni.us/GPU_Rack_Frame
Gigabyte MZ32-AR0 Motherboard geni.us/mz32-ar0_motherboard
RTX 3090 24GB GPU (x4) geni.us/GPU3090
256GB DDR4 2400 RAM geni.us/256GB_DDR4_RAM
PCIe4 Risers (x4) geni.us/PCIe4_Riser_Cable
EPYC 7702 geni.us/EPYC_7702p
Corsair H170i ELITE CAPELLIX geni.us/iCUE_H170i_Capellix
SP3/TR4/5 Retention Kit geni.us/ELITE_RetentionKit
ARCTIC MX4 Thermal Paste geni.us/Arctic_ThermalPaste
CORSAIR HX1500i PSU geni.us/Corsair_HX1500iPSU
4i SFF-8654 to 4i SFF-8654 (x4) geni.us/SFF8654_to_SFF8654
HDD Rack Screws for Fans geni.us/HDD_RackScrews
Chapters
0:00 Ai GPU Benchmarking 4090s and 1070ti
0:42 Adding 4090s to Rack
2:41 Llama 3.1 70b Q4 2x 4090s
4:33 Llama 3.1 8b Inst fp16 2x 4090s
5:10 Llama 3.1 8b Q4 2x 4090s
6:15 Mixing Pascal and Ada GPUs
7:53 Codestral 22b 8Q 1070ti and 4090
9:05 Llama 3.1 8b 4K 1070ti
10:40 Conclusions
Be sure to 👍✅Subscribe✅👍 for more content like this!
Join this channel youtube.com/@DigitalSpaceport/join
Please share this video if you know someone who needs this video 🙂 Thanks for watching!
Digital Spaceport Website
🌐 digitalspaceport.com
🛒Shop (Channel members get a 3% or 5% discount)
Check out Shop.DigitalSpaceport.com for great deals on hardware and merch.
*****
As an Amazon Associate I earn from qualifying purchases.
When you click on links to various merchants on this site and make a purchase, this can result in this site earning a commission. Affiliate programs and affiliations include, but are not limited to, the eBay Partner Network.
*****
#digitalspaceport
We are benchmarking the killer Proxmox based, LXC and Docker Homelab running OpenwebUI and Ollama. We are continuing our Ai GPU Llama 3.1 testing at several parameter levels and mixed GPU Generation and today we are looking at dual 4090s and tossing in a real wildcard extra GPU, an older 1070ti. The results are pretty crazy!
Ai GPU Benchmarking 3090s and 3060s + VRAM Testing - youtu.be/-heFPHKy3jY
Ollama Ai Home Server Build - youtu.be/JN4EhaM7vyw
Bare Metal OpenwebUI Ai Server Setup Guide - youtu.be/q_cDvCq1pww
Proxmox LXC Docker Ai Setup + GPU Passthrough - youtu.be/TmNSDkjDTOs
Results Table - digitalspaceport.com/llama3-1-70b-benchmarks-on-gpus
GPUs
4090 geni.us/4090GPU_5N_s
1070ti geni.us/1070tiGPU_5N_s
Ai Server Build
GPU Rack Frame geni.us/GPU_Rack_Frame
Gigabyte MZ32-AR0 Motherboard geni.us/mz32-ar0_motherboard
RTX 3090 24GB GPU (x4) geni.us/GPU3090
256GB DDR4 2400 RAM geni.us/256GB_DDR4_RAM
PCIe4 Risers (x4) geni.us/PCIe4_Riser_Cable
EPYC 7702 geni.us/EPYC_7702p
Corsair H170i ELITE CAPELLIX geni.us/iCUE_H170i_Capellix
SP3/TR4/5 Retention Kit geni.us/ELITE_RetentionKit
ARCTIC MX4 Thermal Paste geni.us/Arctic_ThermalPaste
CORSAIR HX1500i PSU geni.us/Corsair_HX1500iPSU
4i SFF-8654 to 4i SFF-8654 (x4) geni.us/SFF8654_to_SFF8654
HDD Rack Screws for Fans geni.us/HDD_RackScrews
Chapters
0:00 Ai GPU Benchmarking 4090s and 1070ti
0:42 Adding 4090s to Rack
2:41 Llama 3.1 70b Q4 2x 4090s
4:33 Llama 3.1 8b Inst fp16 2x 4090s
5:10 Llama 3.1 8b Q4 2x 4090s
6:15 Mixing Pascal and Ada GPUs
7:53 Codestral 22b 8Q 1070ti and 4090
9:05 Llama 3.1 8b 4K 1070ti
10:40 Conclusions
Be sure to 👍✅Subscribe✅👍 for more content like this!
Join this channel youtube.com/@DigitalSpaceport/join
Please share this video if you know someone who needs this video 🙂 Thanks for watching!
Digital Spaceport Website
🌐 digitalspaceport.com
🛒Shop (Channel members get a 3% or 5% discount)
Check out Shop.DigitalSpaceport.com for great deals on hardware and merch.
*****
As an Amazon Associate I earn from qualifying purchases.
When you click on links to various merchants on this site and make a purchase, this can result in this site earning a commission. Affiliate programs and affiliations include, but are not limited to, the eBay Partner Network.
*****
#digitalspaceport
