Uploaded October 2024 | Updated September 2026, 2 weeks ago
Learn about power efficient AI at the edge by building local assistants with Intel® Core™ Ultra. Create personalized AI assistants with Intel® Core™ Ultra. Discover how to leverage LLMs, speech transcription, and optimization techniques for efficient deployment.
In this session, we explore how to build and deploy local LLM-based assistants on the AI PC or at the edge. Our pipeline leverages a real-time speech transcription model (Distil-Whisper), and Large Language Model (LLama3) powered chatbots leveraging Retrieval Augmented Generation (RAG), to personalize user interactions via text generation and summarization over prior interaction history. We discuss how the Intel® Core™ Ultra enables efficient deployment of LLMs on the CPU, iGPU, and NPU, through optimization techniques such as quantization and OpenVINO™ compression libraries. Live demos will be presented throughout the session to ensure developers can see the work in action.
Intel Innovation: intel.ly/46ixwbA
Intel Events: intel.ly/4dCbLWb
Intel Architecture Set Extensions Programming Reference Sheet Download: intel.ly/4eArOF8
Optimizing Software for Intel® Hybrid Processors: intel.ly/482cr6g
Processor Power Management Options | Microsoft Learn: intel.ly/3U4cCbo
Process Threads API: Win32 apps | Microsoft Learn: bit.ly/3Y5te3M
System CPU Set Information | Win32 Apps | Microsoft Learn: intel.ly/3zHAcDU
Microsoft Media Foundation - Win32 apps | Microsoft Learn: intel.ly/47Waxnz
Custom AI Assistants: intel.ly/3zXm75m
About Intel Software:
Intel® Developer Zone is committed to empowering and assisting software developers in creating applications for Intel hardware and software products. The Intel Software YouTube channel is an excellent resource for those seeking to enhance their knowledge. Our channel provides the latest news, helpful tips, and engaging product demos from Intel and our numerous industry partners. Our videos cover various topics; you can explore them further by following the links.
Connect with Intel Software:
INTEL SOFTWARE WEBSITE: intel.ly/2KeP1hD
INTEL SOFTWARE on FACEBOOK: bit.ly/2z8MPFF
INTEL SOFTWARE on TWITTER: bit.ly/2zahGSn
INTEL SOFTWARE GITHUB: bit.ly/2zaih6z
INTEL DEVELOPER ZONE LINKEDIN: bit.ly/2z979qs
INTEL DEVELOPER ZONE INSTAGRAM: bit.ly/2z9Xsby
INTEL GAME DEV TWITCH: bit.ly/2BkNshu
#intelsoftware #innovation
Deploying AI Assistants with OpenVINO™ on the AI PC | Technical Talk | Innovation Selects
Learn about power efficient AI at the edge by building local assistants with Intel® Core™ Ultra. Create personalized AI assistants with Intel® Core™ Ultra. Discover how to leverage LLMs, speech transcription, and optimization techniques for efficient deployment.
In this session, we explore how to build and deploy local LLM-based assistants on the AI PC or at the edge. Our pipeline leverages a real-time speech transcription model (Distil-Whisper), and Large Language Model (LLama3) powered chatbots leveraging Retrieval Augmented Generation (RAG), to personalize user interactions via text generation and summarization over prior interaction history. We discuss how the Intel® Core™ Ultra enables efficient deployment of LLMs on the CPU, iGPU, and NPU, through optimization techniques such as quantization and OpenVINO™ compression libraries. Live demos will be presented throughout the session to ensure developers can see the work in action.
Intel Innovation: intel.ly/46ixwbA
Intel Events: intel.ly/4dCbLWb
Intel Architecture Set Extensions Programming Reference Sheet Download: intel.ly/4eArOF8
Optimizing Software for Intel® Hybrid Processors: intel.ly/482cr6g
Processor Power Management Options | Microsoft Learn: intel.ly/3U4cCbo
Process Threads API: Win32 apps | Microsoft Learn: bit.ly/3Y5te3M
System CPU Set Information | Win32 Apps | Microsoft Learn: intel.ly/3zHAcDU
Microsoft Media Foundation - Win32 apps | Microsoft Learn: intel.ly/47Waxnz
Custom AI Assistants: intel.ly/3zXm75m
About Intel Software:
Intel® Developer Zone is committed to empowering and assisting software developers in creating applications for Intel hardware and software products. The Intel Software YouTube channel is an excellent resource for those seeking to enhance their knowledge. Our channel provides the latest news, helpful tips, and engaging product demos from Intel and our numerous industry partners. Our videos cover various topics; you can explore them further by following the links.
Connect with Intel Software:
INTEL SOFTWARE WEBSITE: intel.ly/2KeP1hD
INTEL SOFTWARE on FACEBOOK: bit.ly/2z8MPFF
INTEL SOFTWARE on TWITTER: bit.ly/2zahGSn
INTEL SOFTWARE GITHUB: bit.ly/2zaih6z
INTEL DEVELOPER ZONE LINKEDIN: bit.ly/2z979qs
INTEL DEVELOPER ZONE INSTAGRAM: bit.ly/2z9Xsby
INTEL GAME DEV TWITCH: bit.ly/2BkNshu
#intelsoftware #innovation
Deploying AI Assistants with OpenVINO™ on the AI PC | Technical Talk | Innovation Selects
