TechteamGB | How to Turn Your AMD GPU into a Local LLM Beast: A Beginner's Guide with ROCm @TechteamGB | Uploaded 5 months ago | Updated 14 hours ago
RX 7600 XT on Amazon (affiliate): https://locally.link/kEJG
LM Studio: https://lmstudio.ai/rocm
Products provided by Gigabyte
Those of us with NVIDIA GPUs, particularly ones with enough VRAM, have been able to run large language models locally for quite a while. I did a guide last year showing you how to run Vicuna locally, but that really only worked on NVIDIA GPUs. Support has improved a little, including running it all on your CPU instead, but with the launch of AMDs ROCm software, its now not only possible to run large language models locally, but insanely easy. Now if you are just here for the guide, skip here. If you want to know how this works so damn well, stick around!
BUY AN OSRTT (Open Source Response Time Tool): https://osrtt.com
Become a Youtube Member and get sponsor free videos and access to our private Discord chat: https://www.youtube.com/techteamgb/join
Locally Links - Global Short Linking for Creators: https://locally.link/techteamgb
Use referral code "techteamgb20" when signing up!
Patreon: https://www.patreon.com/techteamgb
Donations: https://streamlabs.com/techteamgb
OverclockersUK Affiliate link: http://techteamgb.co.uk/ocuk
Discord! https://discord.gg/NxFzaWy
As an Amazon Associate I earn from qualifying purchases, using the links below or other Amazon affiliate links here.
Want a cool T-Shirt or hoodie? https://teespring.com/en-GB/stores/techteamgb
Private Internet Access (VPN): http://techteamgb.co.uk/PIA
NordVPN: https://nordvpn.org/techteamgb
HUMBLE BUNDLE: https://www.humblebundle.com/monthly?partner=techteamgb
Zen Internet - My (UK) ISP: https://locally.link/b3DC
WEB HOSTING: http://techteamgb.co.uk/domaincom
Check out our awesome website! http://techteamgb.co.uk
Bitcoin donations: 1PqsJeJsDbNEECjCKbQ2DsQxJWYqmTvt4E
My Monitor - Philips EVNIA 8600 QD-OLED: https://locally.link/H33q
- My PC - AMAZON AFFILIATE LINKS -
5900X on Amazon: https://techteamgb.co.uk/5900x
32GB Corsair 3000MHz on Amazon: https://techteamgb.co.uk/cors32gbrgb
GBT X570 Master on Amazon: https://techteamgb.co.uk/gbtx570master
RTX 2080 on Amazon: https://techteamgb.co.uk/RTX2080
H100i Pro on Amazon: https://techteamgb.co.uk/h100ipro
WD 1TB NVME on Amazon: https://techteamgb.co.uk/wdbl1tb
Patriot VP4100 on Amazon: https://techteamgb.co.uk/vp4100
Sabrent Rocket Q 4TB on Amazon: https://techteamgb.co.uk/rocketq4
If you are interested in contacting us, then please email: inbox@techteamgb.com and we will respond as soon as possible.
RX 7600 XT on Amazon (affiliate): https://locally.link/kEJG
LM Studio: https://lmstudio.ai/rocm
Products provided by Gigabyte
Those of us with NVIDIA GPUs, particularly ones with enough VRAM, have been able to run large language models locally for quite a while. I did a guide last year showing you how to run Vicuna locally, but that really only worked on NVIDIA GPUs. Support has improved a little, including running it all on your CPU instead, but with the launch of AMDs ROCm software, its now not only possible to run large language models locally, but insanely easy. Now if you are just here for the guide, skip here. If you want to know how this works so damn well, stick around!
BUY AN OSRTT (Open Source Response Time Tool): https://osrtt.com
Become a Youtube Member and get sponsor free videos and access to our private Discord chat: https://www.youtube.com/techteamgb/join
Locally Links - Global Short Linking for Creators: https://locally.link/techteamgb
Use referral code "techteamgb20" when signing up!
Patreon: https://www.patreon.com/techteamgb
Donations: https://streamlabs.com/techteamgb
OverclockersUK Affiliate link: http://techteamgb.co.uk/ocuk
Discord! https://discord.gg/NxFzaWy
As an Amazon Associate I earn from qualifying purchases, using the links below or other Amazon affiliate links here.
Want a cool T-Shirt or hoodie? https://teespring.com/en-GB/stores/techteamgb
Private Internet Access (VPN): http://techteamgb.co.uk/PIA
NordVPN: https://nordvpn.org/techteamgb
HUMBLE BUNDLE: https://www.humblebundle.com/monthly?partner=techteamgb
Zen Internet - My (UK) ISP: https://locally.link/b3DC
WEB HOSTING: http://techteamgb.co.uk/domaincom
Check out our awesome website! http://techteamgb.co.uk
Bitcoin donations: 1PqsJeJsDbNEECjCKbQ2DsQxJWYqmTvt4E
My Monitor - Philips EVNIA 8600 QD-OLED: https://locally.link/H33q
- My PC - AMAZON AFFILIATE LINKS -
5900X on Amazon: https://techteamgb.co.uk/5900x
32GB Corsair 3000MHz on Amazon: https://techteamgb.co.uk/cors32gbrgb
GBT X570 Master on Amazon: https://techteamgb.co.uk/gbtx570master
RTX 2080 on Amazon: https://techteamgb.co.uk/RTX2080
H100i Pro on Amazon: https://techteamgb.co.uk/h100ipro
WD 1TB NVME on Amazon: https://techteamgb.co.uk/wdbl1tb
Patriot VP4100 on Amazon: https://techteamgb.co.uk/vp4100
Sabrent Rocket Q 4TB on Amazon: https://techteamgb.co.uk/rocketq4
If you are interested in contacting us, then please email: inbox@techteamgb.com and we will respond as soon as possible.
