Qualcomm Developer Network | Qualcomm AI Hub Tutorial 5: Specify a compute unit @QualcommDeveloper | Uploaded 2 days ago | Updated 13 hours ago
Welcome back to the Qualcomm AI Hub video series! In this tutorial, we'll guide you through specifying which compute units you'd like your model to run on when submitting a job to Qualcomm AI Hub. Let's dive right in!
Specifying Compute Units:
Understanding Compute Units:
When submitting a job to Qualcomm AI Hub, you might want to specify certain compute units for your ML model to run on.
You can target specific compute units in any type of job: compile, profile, or inference.
Specifying Compute Units:
Specify any combination of compute units to restrict where the model should run.
If you don't have any restrictions and want your model to run the fastest, you don't need to specify any options or you can pass compute_unit=all.
Example Scenarios:
For a gaming application that uses the GPU for graphics, pass compute_unit=NPU,CPU to avoid using the overloaded GPU.
If only the GPU is available, pass compute_unit=GPU.
Practical Implementation:
Specify compute units as an option by writing compute_unit followed by the processors you'd like to target, separated by commas.
For this example, we'll use pre-trained MobileNet and submit a compile and profile job targeting the NPU.
Submitting the Job:
Name the job mobile_net_NPU and pass the option to run it specifically on the NPU for the compile job.
Pass the same option for the profile job.
Viewing the Results:
Go to Qualcomm AI Hub to see your jobs.
For the compile job, notice the option is listed.
For the profile job, check the performance. You'll see that all 70 layers have run on the NPU.
Select "visualize" on the job page for a visual representation of the layers in the model, including tensor shapes and attributes.
That's it! You're now all set to submit a job with Qualcomm AI Hub, specifying any combination of compute units to meet your needs. If you encounter any issues or have questions, join our Qualcomm AI Hub Community on Slack: join.slack.com/t/qualcomm-ai-hub/shared_invite/zt-2qv4i2g2c-2gOlnYJVJk_5Z~lGAcVFqg
Stay tuned for more videos in this series!
Subscribe: More QDN videos: http://tinyurl.com/2p8xmcw6
Join our developer community at Discord: discord.gg/THUPBtskgs
ABOUT Qualcomm Developer Network (QDN) is a comprehensive program designed to equip the next generation of mobile pioneers to develop what’s next. Our collection of software and hardware tools and resources is designed so you can build upon our foundational technologies in new and innovative ways, creating the power to build products, enrich lives and even transform entire industries. At Qualcomm Developer Network, we aim to help you kickstart your development by being the catalyst for your vision, today, tomorrow, and in the future.
#QualcommAIHub #OnDeviceAI #DeveloperTips #AIModelDeployment
Welcome back to the Qualcomm AI Hub video series! In this tutorial, we'll guide you through specifying which compute units you'd like your model to run on when submitting a job to Qualcomm AI Hub. Let's dive right in!
Specifying Compute Units:
Understanding Compute Units:
When submitting a job to Qualcomm AI Hub, you might want to specify certain compute units for your ML model to run on.
You can target specific compute units in any type of job: compile, profile, or inference.
Specifying Compute Units:
Specify any combination of compute units to restrict where the model should run.
If you don't have any restrictions and want your model to run the fastest, you don't need to specify any options or you can pass compute_unit=all.
Example Scenarios:
For a gaming application that uses the GPU for graphics, pass compute_unit=NPU,CPU to avoid using the overloaded GPU.
If only the GPU is available, pass compute_unit=GPU.
Practical Implementation:
Specify compute units as an option by writing compute_unit followed by the processors you'd like to target, separated by commas.
For this example, we'll use pre-trained MobileNet and submit a compile and profile job targeting the NPU.
Submitting the Job:
Name the job mobile_net_NPU and pass the option to run it specifically on the NPU for the compile job.
Pass the same option for the profile job.
Viewing the Results:
Go to Qualcomm AI Hub to see your jobs.
For the compile job, notice the option is listed.
For the profile job, check the performance. You'll see that all 70 layers have run on the NPU.
Select "visualize" on the job page for a visual representation of the layers in the model, including tensor shapes and attributes.
That's it! You're now all set to submit a job with Qualcomm AI Hub, specifying any combination of compute units to meet your needs. If you encounter any issues or have questions, join our Qualcomm AI Hub Community on Slack: join.slack.com/t/qualcomm-ai-hub/shared_invite/zt-2qv4i2g2c-2gOlnYJVJk_5Z~lGAcVFqg
Stay tuned for more videos in this series!
Subscribe: More QDN videos: http://tinyurl.com/2p8xmcw6
Join our developer community at Discord: discord.gg/THUPBtskgs
ABOUT Qualcomm Developer Network (QDN) is a comprehensive program designed to equip the next generation of mobile pioneers to develop what’s next. Our collection of software and hardware tools and resources is designed so you can build upon our foundational technologies in new and innovative ways, creating the power to build products, enrich lives and even transform entire industries. At Qualcomm Developer Network, we aim to help you kickstart your development by being the catalyst for your vision, today, tomorrow, and in the future.
#QualcommAIHub #OnDeviceAI #DeveloperTips #AIModelDeployment
