Uploaded June 2024 | Updated November 2026, 5 days ago
In this video, we'll run a state of the art LLM on your laptop and create a webpage you can use to interact with it. All in about 5 minutes. Seriously!
We'll be using Llama.cpp's python bindings to run the LLM on our machine and Gradio to build the webpage.
Resources mentioned in the video:
* Llama.cpp python: github.com/abetlen/llama-cpp-python
* Gradio: github.com/gradio-app/gradio
* Qwen-2 0.5B Instruct Model (GGUF): huggingface.co/Qwen/Qwen2-0.5B-Instruct-GGUF
* Llama.cpp's chat completion API: github.com/abetlen/llama-cpp-python?tab=readme-ov-file#chat-completion
* Gradio Chatbot Guide: https://www.gradio.app/guides/creating-a-chatbot-fast
In this video, we'll run a state of the art LLM on your laptop and create a webpage you can use to interact with it. All in about 5 minutes. Seriously!
We'll be using Llama.cpp's python bindings to run the LLM on our machine and Gradio to build the webpage.
Resources mentioned in the video:
* Llama.cpp python: github.com/abetlen/llama-cpp-python
* Gradio: github.com/gradio-app/gradio
* Qwen-2 0.5B Instruct Model (GGUF): huggingface.co/Qwen/Qwen2-0.5B-Instruct-GGUF
* Llama.cpp's chat completion API: github.com/abetlen/llama-cpp-python?tab=readme-ov-file#chat-completion
* Gradio Chatbot Guide: https://www.gradio.app/guides/creating-a-chatbot-fast










