Tips and tricks for distributed large model training @TensorFlow
Tips and tricks for distributed large model training  @TensorFlow
Uploaded May 2022 | Updated November 2026, 1 day ago
Discover several different distribution strategies and related concepts for data and model parallel training. Walk through an example of training a 39 billion parameter language model on TPUs, and conclude with the challenges and best practices of orchestrating large scale language model training.

Resource:
TensorFlow website → https://goo.gle/3KejoUZ

Speakers: Nikita Namjoshi, Vaibhav Singh

Watch more:
All Google I/O 2022 Sessions → https://goo.gle/IO22_AllSessions
ML/AI at I/O 2022 playlist → https://goo.gle/IO22_ML-AI
All Google I/O 2022 technical sessions → https://goo.gle/IO22_Sessions

Subscribe to TensorFlow → https://goo.gle/TensorFlow

#GoogleIO
Tips and tricks for distributed large model trainingGradient ascent for your career | Breakout sessionIntroduction to On-device Machine Learning Solutions with MediaPipePowered by TensorFlow: How machine learning helps advance access to human rights informationTop takeaways from Women in Machine Learning Symposium 2021Adding Machine Learning to your developer toolboxWhats new in Machine Learning for Google Developers - American Sign LanguageTensorFlow Serving performance optimizationRecommendation with TF Agents Bandits Library (Reinforcement learning with TensorFlow Agents)Creating successful study groups to empower everyoneChip floorplanning with deep reinforcement learningTake action and start building with Google AI
TensorFlow |

Tips and tricks for distributed large model training

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER