Online Advertising: Hybrid Approach to Large-Scale Data Analysis
summary
Agency and Microsoft build a hybrid pipeline that keeps raw logs on HDInsight on Azure, runs MapReduce/Hive for predictive scoring, then pulls selected subsets into a PDW appliance for fast interactive SQL and Excel analysis. Prototype processes ~6.8 billion impressions (10‑day sample), runs a daily group‑by in ~19 seconds, loads a compressed terabyte in ~30–60 minutes, surfaces click‑fraud patterns and campaign predictions, and reduces storage cost by keeping cold raw data in the cloud.
AI-written summary · about
chapters (20)
- 0:00Introduction
- 1:23Project goals and challenges
- 2:05Advertising ecosystem complexity
- 4:57Big data principles
- 7:09Ad budgets and data growth
- 8:13Media consumption trends
- 10:21Personalized engagement demand
- 12:07Customer attitudes beyond behavior
- 13:35Engagement dialogue example
- 15:44Attribution and conversion challenges
- 16:47Cloud advantages for data processing
- 18:33Hybrid cloud approach
- 21:45Data flow and hybrid architecture
- 26:18Hadoop demo in Azure HDInsight
- 38:23Predictive analytics for campaigns
- 43:36Proof of concept overview
- 48:35On-premise data warehouse
- 57:45Excel and PowerPivot analysis
- 1:00:11PolyBase for Hadoop queries
- 1:03:22Conclusion and Q&A
related talks
24mImprove your marketing reach using ...
09mHDInsight and Hadoop
11mMicrosoft & Power BI
01h 35mDeep Dive into the Big Data Landscape
10mBringing Big Data to One Billion People
02h 05mUsing Hadoop with Excel and Power BI, Plus Data Discovery 101
49mBuilding a 100‑Petabyte Analytics Warehouse
25mTransforming Your Data Into A Strategic Asset