Uploaded July 2026 | Updated September 2026, 2 weeks ago
LongCat 2.0: How Meituan Trained a 1.6T Model Without NVIDIA GPUs or TPUs
In this video, I break down how Meituan open sourced LongCat 2.0 (Owl Alpha on OpenRouter), a 1.6T-parameter MoE model trained without NVIDIA GPUs or Google TPUs, and why that matters for reducing reliance on NVIDIA’s hardware and CUDA software stack. I explain the core tradeoffs between parameters and compute, why simply adding more experts can hit diminishing returns, and how LongCat uses n-gram embeddings to increase corpus information more cheaply than adding experts. I cover the long-context cost problem and their modified sparse attention approach (inspired by DeepSeek) that makes the helper lighter via predictable memory access, caching across layers, and coarse-to-fine selection. I also discuss speculative decoding with a draft “picker” model, custom ASICs tuned for prefill vs decode, training on 50,000+ chips over 35T tokens, and I demo it generating a 3D ISS tracker on longcat.chat.
https://longcat.chat/blog/longcat-2.0/
DeepSpec Video: youtu.be/eFgknPFK-g0
My voice to text App: whryte.com
Website: engineerprompt.ai
RAG Beyond Basics Course:
prompt-s-site.thinkific.com/courses/rag
Signup for Newsletter, localgpt:
https://tally.so/r/3y9bb0
Let's Connect:
🦾 Discord: discord.com/invite/t4eYQRUcXB
☕ Buy me a Coffee: ko-fi.com/promptengineering
|🔴 Patreon: patreon.com/PromptEngineering
💼Consulting: calendly.com/engineerprompt/consulting-call
📧 Business Contact: engineerprompt@gmail.com
Become Member: tinyurl.com/y5h28s6h
💻 Pre-configured localGPT VM: bit.ly/localGPT (use Code: PromptEngineering for 50% off).
Signup for Newsletter, localgpt:
https://tally.so/r/3y9bb0
00:00 LongCat Shockwave
00:39 Why Hardware Matters
01:20 MoE Limits and N-Grams
03:40 Cheaper Long Context
06:23 Custom Chips and Stack
08:23 Demo Results and Takeaways
LongCat 2.0: How Meituan Trained a 1.6T Model Without NVIDIA GPUs or TPUs
In this video, I break down how Meituan open sourced LongCat 2.0 (Owl Alpha on OpenRouter), a 1.6T-parameter MoE model trained without NVIDIA GPUs or Google TPUs, and why that matters for reducing reliance on NVIDIA’s hardware and CUDA software stack. I explain the core tradeoffs between parameters and compute, why simply adding more experts can hit diminishing returns, and how LongCat uses n-gram embeddings to increase corpus information more cheaply than adding experts. I cover the long-context cost problem and their modified sparse attention approach (inspired by DeepSeek) that makes the helper lighter via predictable memory access, caching across layers, and coarse-to-fine selection. I also discuss speculative decoding with a draft “picker” model, custom ASICs tuned for prefill vs decode, training on 50,000+ chips over 35T tokens, and I demo it generating a 3D ISS tracker on longcat.chat.
https://longcat.chat/blog/longcat-2.0/
DeepSpec Video: youtu.be/eFgknPFK-g0
My voice to text App: whryte.com
Website: engineerprompt.ai
RAG Beyond Basics Course:
prompt-s-site.thinkific.com/courses/rag
Signup for Newsletter, localgpt:
https://tally.so/r/3y9bb0
Let's Connect:
🦾 Discord: discord.com/invite/t4eYQRUcXB
☕ Buy me a Coffee: ko-fi.com/promptengineering
|🔴 Patreon: patreon.com/PromptEngineering
💼Consulting: calendly.com/engineerprompt/consulting-call
📧 Business Contact: engineerprompt@gmail.com
Become Member: tinyurl.com/y5h28s6h
💻 Pre-configured localGPT VM: bit.ly/localGPT (use Code: PromptEngineering for 50% off).
Signup for Newsletter, localgpt:
https://tally.so/r/3y9bb0
00:00 LongCat Shockwave
00:39 Why Hardware Matters
01:20 MoE Limits and N-Grams
03:40 Cheaper Long Context
06:23 Custom Chips and Stack
08:23 Demo Results and Takeaways
