Qualcomm Developer Network | Qualcomm AI Hub Tutorial 6: Check your model’s accuracy @QualcommDeveloper | Uploaded 2 days ago | Updated 10 hours ago
Welcome to Qualcomm AI Hub tutorial series! In this tutorial, we'll show you how to check the accuracy of your model after optimizing its performance, ensuring you're ready for deployment. Let's dive right in!
Checking Model Accuracy:
Ensure Numerical Accuracy:
After optimizing your model, ensure it's numerically accurate by comparing on-device results to the reference implementation.
Submit an inference job to Qualcomm AI Hub using a previously compiled job.
Prepare Dictionary Input:
The inference job requires a dictionary input to check if the predictions are accurate.
Use a previously uploaded dataset or upload a new image and transform it into a dictionary for inference.
Example Walkthrough:
We'll use MobileNet and validate the numerics of a Qualcomm AI Engine Direct model library on-device by comparing it to the PyTorch reference implementation.
Target the Samsung Galaxy S24 powered by the Snapdragon 8 Gen 3 chipset.
Submit a Compile Job:
Submit a compile job to get the target model .so file for running inference with the input data on your device.
Provide Test Image:
Provide a test image for testing the model. In this case, we'll use a URL to the image and resize it to match the expected input specs.
Submit an Inference Job:
Submit an inference job and download the output data once it's successfully completed.
Compare Model Outputs:
Compare the model outputs for the on-device model versus the PyTorch MobileNet model.
Use the on-device raw output to generate the probabilities of class predictions and identify the top five predictions for both models.
Verify Accuracy:
Since the predictions are nearly equivalent, you can be confident that the model optimized by Qualcomm AI Hub maintains accuracy and should behave as expected once deployed.
Further Accuracy Calculations:
Use the output data from your inference job to perform additional accuracy calculations and test with more images to increase confidence.
Deploy the Model:
Once satisfied with the model accuracy, download the compiled asset to bundle into your application and deploy on-device.
Now that you know how to check the accuracy of your model with confidence, we encourage you to try compiling and running inference on your own model. If you encounter any issues or have questions, join our Qualcomm AI Hub Community on Slack: join.slack.com/t/qualcomm-ai-hub/shared_invite/zt-2qv4i2g2c-2gOlnYJVJk_5Z~lGAcVFqg
Stay tuned for more videos in this series!
Subscribe: More QDN videos: http://tinyurl.com/2p8xmcw6
Join our developer community at Discord: discord.gg/THUPBtskgs
ABOUT Qualcomm Developer Network (QDN) is a comprehensive program designed to equip the next generation of mobile pioneers to develop what’s next. Our collection of software and hardware tools and resources is designed so you can build upon our foundational technologies in new and innovative ways, creating the power to build products, enrich lives and even transform entire industries. At Qualcomm Developer Network, we aim to help you kickstart your development by being the catalyst for your vision, today, tomorrow, and in the future.
Welcome to Qualcomm AI Hub tutorial series! In this tutorial, we'll show you how to check the accuracy of your model after optimizing its performance, ensuring you're ready for deployment. Let's dive right in!
Checking Model Accuracy:
Ensure Numerical Accuracy:
After optimizing your model, ensure it's numerically accurate by comparing on-device results to the reference implementation.
Submit an inference job to Qualcomm AI Hub using a previously compiled job.
Prepare Dictionary Input:
The inference job requires a dictionary input to check if the predictions are accurate.
Use a previously uploaded dataset or upload a new image and transform it into a dictionary for inference.
Example Walkthrough:
We'll use MobileNet and validate the numerics of a Qualcomm AI Engine Direct model library on-device by comparing it to the PyTorch reference implementation.
Target the Samsung Galaxy S24 powered by the Snapdragon 8 Gen 3 chipset.
Submit a Compile Job:
Submit a compile job to get the target model .so file for running inference with the input data on your device.
Provide Test Image:
Provide a test image for testing the model. In this case, we'll use a URL to the image and resize it to match the expected input specs.
Submit an Inference Job:
Submit an inference job and download the output data once it's successfully completed.
Compare Model Outputs:
Compare the model outputs for the on-device model versus the PyTorch MobileNet model.
Use the on-device raw output to generate the probabilities of class predictions and identify the top five predictions for both models.
Verify Accuracy:
Since the predictions are nearly equivalent, you can be confident that the model optimized by Qualcomm AI Hub maintains accuracy and should behave as expected once deployed.
Further Accuracy Calculations:
Use the output data from your inference job to perform additional accuracy calculations and test with more images to increase confidence.
Deploy the Model:
Once satisfied with the model accuracy, download the compiled asset to bundle into your application and deploy on-device.
Now that you know how to check the accuracy of your model with confidence, we encourage you to try compiling and running inference on your own model. If you encounter any issues or have questions, join our Qualcomm AI Hub Community on Slack: join.slack.com/t/qualcomm-ai-hub/shared_invite/zt-2qv4i2g2c-2gOlnYJVJk_5Z~lGAcVFqg
Stay tuned for more videos in this series!
Subscribe: More QDN videos: http://tinyurl.com/2p8xmcw6
Join our developer community at Discord: discord.gg/THUPBtskgs
ABOUT Qualcomm Developer Network (QDN) is a comprehensive program designed to equip the next generation of mobile pioneers to develop what’s next. Our collection of software and hardware tools and resources is designed so you can build upon our foundational technologies in new and innovative ways, creating the power to build products, enrich lives and even transform entire industries. At Qualcomm Developer Network, we aim to help you kickstart your development by being the catalyst for your vision, today, tomorrow, and in the future.
