Digital Spaceport | REFLECTION Llama3.1 70b Tested on Ollama Home Ai Server - Best Ai LLM? @DigitalSpaceport | Uploaded September 2024 | Updated October 2024, 1 hour ago.
The buzz about the new Reflection Llama 3.1 Fine Tune being the worlds best LLM is all over the place, and today I am testing it out with you on the Home Ai Server that we put together recently. Running in Docker on a LXC in proxmox on my Epyc quad 3090 GPU Ai Rig. Ollama got out support for this fast, and thankfully they did! Some surprising results and unexpected results in the mix. You can run the 8q version with 4 24GB GPUs and the 4q version with 2 24GB GPUs (or some combo of gpus to hit ~40GB VRAM). UPDATE on Reflection 70b (Is it broken?) youtu.be/nIl4Q8DCtuo
Ai Server Build Guide - youtu.be/JN4EhaM7vyw
Proxmox Docker LXC GPU Ai Server Setup - youtu.be/TmNSDkjDTOs
Bare Metal Ai Server Setup Guide - youtu.be/q_cDvCq1pww
GROK vs LLAMA 3.1 70 Stock - youtu.be/r2tlah94v84
Reflection - huggingface.co/mattshumer/Reflection-Llama-3.1-70B
Ollama Reflection Model - ollama.com/library/reflection
The Tweet - https://x.com/mattshumer_/status/1831767014341538166
Ai Server Build
GPU Rack Frame geni.us/GPU_Rack_Frame
Gigabyte MZ32-AR0 Motherboard geni.us/mz32-ar0_motherboard
RTX 3090 24GB GPU (x4) geni.us/GPU3090
256GB DDR4 2400 RAM geni.us/256GB_DDR4_RAM
PCIe4 Risers (x4) geni.us/PCIe4_Riser_Cable
EPYC 7702 geni.us/EPYC_7702p
Corsair H170i ELITE CAPELLIX geni.us/iCUE_H170i_Capellix
SP3/TR4/5 Retention Kit geni.us/ELITE_RetentionKit
ARCTIC MX4 Thermal Paste geni.us/Arctic_ThermalPaste
CORSAIR HX1500i PSU geni.us/Corsair_HX1500iPSU
4i SFF-8654 to 4i SFF-8654 (x4) geni.us/SFF8654_to_SFF8654
HDD Rack Screws for Fans geni.us/HDD_RackScrews
Be sure to 👍✅Subscribe✅👍 for more content like this!
Join this channel youtube.com/@DigitalSpaceport/join
Please share this video to help spread the word and drop a comment below with your thoughts or questions. Thanks for watching!
Digital Spaceport Website
🌐 digitalspaceport.com
🛒Shop (Channel members get a 3% or 5% discount)
Check out Shop.DigitalSpaceport.com for great deals on hardware and merch.
*****
As an Amazon Associate I earn from qualifying purchases.
When you click on links to various merchants on this site and make a purchase, this can result in this site earning a commission. Affiliate programs and affiliations include, but are not limited to, the eBay Partner Network.
Other Merchant Affiliate Partners for this site include, but are not limited to, Newegg and Best Buy. I earn a commission if you click on links and make a purchase from the merchant.
*****
The buzz about the new Reflection Llama 3.1 Fine Tune being the worlds best LLM is all over the place, and today I am testing it out with you on the Home Ai Server that we put together recently. Running in Docker on a LXC in proxmox on my Epyc quad 3090 GPU Ai Rig. Ollama got out support for this fast, and thankfully they did! Some surprising results and unexpected results in the mix. You can run the 8q version with 4 24GB GPUs and the 4q version with 2 24GB GPUs (or some combo of gpus to hit ~40GB VRAM). UPDATE on Reflection 70b (Is it broken?) youtu.be/nIl4Q8DCtuo
Ai Server Build Guide - youtu.be/JN4EhaM7vyw
Proxmox Docker LXC GPU Ai Server Setup - youtu.be/TmNSDkjDTOs
Bare Metal Ai Server Setup Guide - youtu.be/q_cDvCq1pww
GROK vs LLAMA 3.1 70 Stock - youtu.be/r2tlah94v84
Reflection - huggingface.co/mattshumer/Reflection-Llama-3.1-70B
Ollama Reflection Model - ollama.com/library/reflection
The Tweet - https://x.com/mattshumer_/status/1831767014341538166
Ai Server Build
GPU Rack Frame geni.us/GPU_Rack_Frame
Gigabyte MZ32-AR0 Motherboard geni.us/mz32-ar0_motherboard
RTX 3090 24GB GPU (x4) geni.us/GPU3090
256GB DDR4 2400 RAM geni.us/256GB_DDR4_RAM
PCIe4 Risers (x4) geni.us/PCIe4_Riser_Cable
EPYC 7702 geni.us/EPYC_7702p
Corsair H170i ELITE CAPELLIX geni.us/iCUE_H170i_Capellix
SP3/TR4/5 Retention Kit geni.us/ELITE_RetentionKit
ARCTIC MX4 Thermal Paste geni.us/Arctic_ThermalPaste
CORSAIR HX1500i PSU geni.us/Corsair_HX1500iPSU
4i SFF-8654 to 4i SFF-8654 (x4) geni.us/SFF8654_to_SFF8654
HDD Rack Screws for Fans geni.us/HDD_RackScrews
Be sure to 👍✅Subscribe✅👍 for more content like this!
Join this channel youtube.com/@DigitalSpaceport/join
Please share this video to help spread the word and drop a comment below with your thoughts or questions. Thanks for watching!
Digital Spaceport Website
🌐 digitalspaceport.com
🛒Shop (Channel members get a 3% or 5% discount)
Check out Shop.DigitalSpaceport.com for great deals on hardware and merch.
*****
As an Amazon Associate I earn from qualifying purchases.
When you click on links to various merchants on this site and make a purchase, this can result in this site earning a commission. Affiliate programs and affiliations include, but are not limited to, the eBay Partner Network.
Other Merchant Affiliate Partners for this site include, but are not limited to, Newegg and Best Buy. I earn a commission if you click on links and make a purchase from the merchant.
*****
