Today, I got a deep dive into open-sora from top AI infra expert Yang You. His insights were spot on, especially on low-cost training strategies—definitely worth checking out.
Professor You Yang is from the Computer Science Department at National University of Singapore(NUS) and is also the founder of Luchentech. He earned his PhD from UC Berkeley under James Demmel. Luchentech has a well-known AI infra framework: Colossal-AI, which introduced the 3D TP concept.
We also discussed the business models for AI infra and came away with a few key takeaways:
- Once LLM structures converge, the room for inference optimization is nearly maxed out. The top foundational model companies are likely to dominate cloud-based inference, making it tough for pure inference AI infra to profit solely through APIs
- Video generation haven’t yet found the best scaling law path and still have room for improvement