Andy Feng, Distinguished Architect, Yahoo

on YouTube ↗27m

" summary

Covers Yahoo’s work to scale supervised, unsupervised and deep learning libraries to billions of examples and millions of features using commodity clusters. Describes evaluation of system architectures (Hadoop, Spark, Storm), a fault-tolerant MPI solution for hundreds of machines to jointly build models, and ongoing collaboration with open-source projects to revise Yahoo ML libraries for lower latency and higher scalability.

AI-written summary · about

" chapters (13)

1. 0:00Start
2. 0:19My Background
3. 3:11Personalized Homepage
4. 9:56Machine Learning @ Yahoo
5. 10:46Our Challenges
6. 13:13Our Approach Big-Data Machine Learning
7. 13:54Apache Hadoop
8. 14:16Apache Storm
9. 15:5930x Speedup for GBOT
10. 21:36Design Patters Enabled
11. 22:22a. ML in Hadoop Reducers
12. 24:121d. Approximate Computing
13. 25:14Realtime Training in Storm Bolts

" related talks

• 04mYuhao Yang InterviewAlexy Khrabrov, Yuhao Yang · Scale By the Bay 2017
• 42mXavier Amatriain, VP of Engineering, QuoraXavier Amatriain · MLconf SF 2015
• 36mKeynote: AI, ML, Cloud for MLAndrei Lopatenko · Global Artificial Intelligence Virtual Conference Sep 2020
• 22mYi Wang, Tech Lead of AI Platform, Baidu, NYC 2017Yi Wang · MLconf NYC 2017
• 18mKeynote: Machine Learning & Apache Spark: A Dynamic DuoJohn Thomas · ApacheCon North America 2017
• 39mBig Data Information Architecture for AIToby Woolfe · Big Data Conference Europe 2019
• 48mKey trends in Big Data and new reference architecture from HPGilles Noisette · HighLoad++ 2015
• 03mInterviewHolden Karau · Scale By The Bay 2018