Uploaded August 2024 | Updated September 2026, 2 weeks ago
In this video, you will learn about Arcee-Spark, a new state-of-the-art 8-billion parameter model created by Arcee.ai from Llama-3.1 8B.
⭐️⭐️⭐️ Don't forget to subscribe to be notified of future videos. Follow me on Medium at julsimon.medium.com or Substack at https://julsimon.substack.com. ⭐️⭐️⭐️
At the time of recording, Spark outperforms its base model on most language and math benchmarks.
The model is a merge of Llama-3.1 8B with additional training. It was built on Arcee Cloud and you can learn more at https://hubs.li/Q02KhKzK0
Here, I first run a 5-bit quantized version on my M3 MacBook and ollama. Then, I deploy the model on AWS with Amazon SageMaker. I run both synchronous and streaming inference. I also show you how to use the OpenAI Messages API, allowing you to invoke the model with the OpenAI prompting format. Finally, I compare ChatGPT and Arcee-Spark, and explain the benefits of high-quality small language models like Arcee-Spark compared to closed models.
* Blog post: https://hubs.li/Q02KhY700
* Model page (full precision model): youtube.com/redirect?event=video_description&redir_token=QUFFLUhqbjVqLW9meGdqUHJIWUcwbHBMd1lnUktTaVlSUXxBQ3Jtc0ttT2NpNFpyNTB3QWx3Z0FoNWZjWDZoT1VEZk5oNXFOM1ZRZWFmZDlvRFVVMkxNMGlJRXROc3N3STViNi02Y3FvT0d2MkoxcERiM3MtbXpQM3BDd2l4aWdxRDVvQWhBRjFwNlpKUHI1N2hTN1pDbmxyZw&q=https%3A%2F%2Fhuggingface.co%2Farcee-ai%2FLlama-Spark&v=Mni0lx6Bf5s
* Model page (quantized models): youtube.com/redirect?event=video_description&redir_token=QUFFLUhqa2pleUUtUGJ0eTIxU0hoWUhEelYxNk1WTGdYd3xBQ3Jtc0ttbmZhX2xjQTVad0ZGbTRJR2lNcFRjalM3T3lGRVBZdGp4ZWFMbkhELVVXVVhyUTZCejlOdDFId0hMMm9MM1VDMVhMc0lxUXhrWTlQdTdNalhrTkw2VGxuY3hMQ1RUeE9EU21qYUl1ZG5CLWFXeU1EOA&q=https%3A%2F%2Fhuggingface.co%2Farcee-ai%2FLlama-Spark-GGUF&v=Mni0lx6Bf5s
* Dataset page: youtube.com/redirect?event=video_description&redir_token=QUFFLUhqbm9uRThPaG4zd3BjdWNULXEtOXlncGZNaTV3QXxBQ3Jtc0tscXc3NEg4aWJSRHloMXQtQkwtazZ5UWUwcmdCRnNXN3FZM2JNazd6NFlPNS1zN3dpaDdJMUhjSFdwTnRIZi1DbEdjS1Y4SlRZLUZpLWRscVo2eUZzY2ExN0Fza05TOFpmOHZyd0hTbWN5R2xBVWpfMA&q=https%3A%2F%2Fhuggingface.co%2Fdatasets%2Farcee-ai%2FThe-Tome&v=Mni0lx6Bf5s
* Notebook: youtube.com/redirect?event=video_description&redir_token=QUFFLUhqa1AwM0RWYW1ocmdkeTRPV1VtdUs5VjNZZWRoQXxBQ3Jtc0tuMjVRZGtXaHBpRlo5aHFkSm0xLTRfdzBMWmlTdGM1SnZKbkoweDhDTnA0NF81TXNiaTUwSDlCWk1XVDJCekZQNk9qT0xrRDVldFpMOXJBcEx3VTVOUFROUC10d3JYc1lud0syQmtjczEyclBLTUE4cw&q=https%3A%2F%2Fgitlab.com%2Fjuliensimon%2Farcee-demos%2F-%2Fblob%2Fmain%2Fsagemaker%2Fdeploy_spark_gpu.ipynb&v=Mni0lx6Bf5s
00:00 Introduction
00:38 Introducing Arcee-Spark
02:30 Running Arcee-Spark locally with ollama
04:50 Deploying Arcee-Spark on AWS with Amazon SageMaker
07:35 Running inference with Arcee-Spark on Amazon SageMaker
10:25 Arcee-Spark vs. ChatGPT
13:25 Why small language models make more sense than closed models!
Configuration file for ollama:
FROM ./llama-spark-dpo-v0.3-Q5_K_S.gguf
#ai #aws #slm #llm #openai #chatgpt #opensource #huggingface
Sign up for Arcee Cloud at https://hubs.li/Q02Kh_YQ0 and please follow Arcee.ai on LinkedIn to stay on top of the latest Small Language Model action!
In this video, you will learn about Arcee-Spark, a new state-of-the-art 8-billion parameter model created by Arcee.ai from Llama-3.1 8B.
⭐️⭐️⭐️ Don't forget to subscribe to be notified of future videos. Follow me on Medium at julsimon.medium.com or Substack at https://julsimon.substack.com. ⭐️⭐️⭐️
At the time of recording, Spark outperforms its base model on most language and math benchmarks.
The model is a merge of Llama-3.1 8B with additional training. It was built on Arcee Cloud and you can learn more at https://hubs.li/Q02KhKzK0
Here, I first run a 5-bit quantized version on my M3 MacBook and ollama. Then, I deploy the model on AWS with Amazon SageMaker. I run both synchronous and streaming inference. I also show you how to use the OpenAI Messages API, allowing you to invoke the model with the OpenAI prompting format. Finally, I compare ChatGPT and Arcee-Spark, and explain the benefits of high-quality small language models like Arcee-Spark compared to closed models.
* Blog post: https://hubs.li/Q02KhY700
* Model page (full precision model): youtube.com/redirect?event=video_description&redir_token=QUFFLUhqbjVqLW9meGdqUHJIWUcwbHBMd1lnUktTaVlSUXxBQ3Jtc0ttT2NpNFpyNTB3QWx3Z0FoNWZjWDZoT1VEZk5oNXFOM1ZRZWFmZDlvRFVVMkxNMGlJRXROc3N3STViNi02Y3FvT0d2MkoxcERiM3MtbXpQM3BDd2l4aWdxRDVvQWhBRjFwNlpKUHI1N2hTN1pDbmxyZw&q=https%3A%2F%2Fhuggingface.co%2Farcee-ai%2FLlama-Spark&v=Mni0lx6Bf5s
* Model page (quantized models): youtube.com/redirect?event=video_description&redir_token=QUFFLUhqa2pleUUtUGJ0eTIxU0hoWUhEelYxNk1WTGdYd3xBQ3Jtc0ttbmZhX2xjQTVad0ZGbTRJR2lNcFRjalM3T3lGRVBZdGp4ZWFMbkhELVVXVVhyUTZCejlOdDFId0hMMm9MM1VDMVhMc0lxUXhrWTlQdTdNalhrTkw2VGxuY3hMQ1RUeE9EU21qYUl1ZG5CLWFXeU1EOA&q=https%3A%2F%2Fhuggingface.co%2Farcee-ai%2FLlama-Spark-GGUF&v=Mni0lx6Bf5s
* Dataset page: youtube.com/redirect?event=video_description&redir_token=QUFFLUhqbm9uRThPaG4zd3BjdWNULXEtOXlncGZNaTV3QXxBQ3Jtc0tscXc3NEg4aWJSRHloMXQtQkwtazZ5UWUwcmdCRnNXN3FZM2JNazd6NFlPNS1zN3dpaDdJMUhjSFdwTnRIZi1DbEdjS1Y4SlRZLUZpLWRscVo2eUZzY2ExN0Fza05TOFpmOHZyd0hTbWN5R2xBVWpfMA&q=https%3A%2F%2Fhuggingface.co%2Fdatasets%2Farcee-ai%2FThe-Tome&v=Mni0lx6Bf5s
* Notebook: youtube.com/redirect?event=video_description&redir_token=QUFFLUhqa1AwM0RWYW1ocmdkeTRPV1VtdUs5VjNZZWRoQXxBQ3Jtc0tuMjVRZGtXaHBpRlo5aHFkSm0xLTRfdzBMWmlTdGM1SnZKbkoweDhDTnA0NF81TXNiaTUwSDlCWk1XVDJCekZQNk9qT0xrRDVldFpMOXJBcEx3VTVOUFROUC10d3JYc1lud0syQmtjczEyclBLTUE4cw&q=https%3A%2F%2Fgitlab.com%2Fjuliensimon%2Farcee-demos%2F-%2Fblob%2Fmain%2Fsagemaker%2Fdeploy_spark_gpu.ipynb&v=Mni0lx6Bf5s
00:00 Introduction
00:38 Introducing Arcee-Spark
02:30 Running Arcee-Spark locally with ollama
04:50 Deploying Arcee-Spark on AWS with Amazon SageMaker
07:35 Running inference with Arcee-Spark on Amazon SageMaker
10:25 Arcee-Spark vs. ChatGPT
13:25 Why small language models make more sense than closed models!
Configuration file for ollama:
FROM ./llama-spark-dpo-v0.3-Q5_K_S.gguf
#ai #aws #slm #llm #openai #chatgpt #opensource #huggingface
Sign up for Arcee Cloud at https://hubs.li/Q02Kh_YQ0 and please follow Arcee.ai on LinkedIn to stay on top of the latest Small Language Model action!
