Yahoo!, Big Data, and Microsoft BI: Bigger and Better Together

on YouTube ↗01h 06m

" summary

Yahoo builds a Tau analytics pipeline that combines Hadoop (2 PB in a 14 PB cluster) with Oracle staging and SSAS cubing to serve ad optimization queries in seconds. The system ingests about 3.5 billion ad impressions per day, loads roughly 464 billion rows per quarter, and maintains a 26 terabyte cube partitioned hourly and by market. Average ad-hoc query times run ~2s in Tableau and ~5s for web reports; processing uses 48 cores/256 GB nodes. A one percent sample for uniques and frequency supplies usable distincts until full-scale distinct-count solutions mature.

AI-written summary · about

" chapters (11)

1. 0:00Yahoo and Microsoft BI intro
2. 1:27Business case for BI with Hadoop
3. 3:56Yahoo ad network optimization
4. 15:13Hadoop’s role in big data
5. 27:55BI cube challenges
6. 33:14Big data analytics challenges
7. 43:05Hadoop to Analysis Services
8. 45:50Data sorting in BI
9. 53:11Geeky references and giveaway
10. 56:00Demo of Yahoo BI tools
11. 1:05:00Q&A

" related talks

• 05mCan Big Data Reach One Billion People?Quentin Clark · Strata Conference New York + Hadoop World 2013
• 11mMicrosoft & Power BIQuentin Clarks · Strata Conference 2014
• 29mRealtime Analytics Unleashed: Unlocking the Potential of Data at ScaleBarkha Herman · Shift Miami 2023
• 10mBringing Big Data to One Billion PeopleQuentin Clark · Strata Conference 2014
• 41mBig data as we know it @bol.comDebarshi Basak, Mark Nijenhuis · J-Fall 2015
• 50mPower BI + Azure Synapse Analytics: Beginning your journeyPatrick LeBlanc, Adam Saxton · SQLBits 2023
• 17mA Real Time Analytics FrameworkHiral Patel · Data By the Bay 2016
• 24mGive meaning to 100 billion analytics events a day, analytics at TeadsPar Alban Perillat-Merceroz · DataXDay 2019