David Abercrombie - HP Big Data 2015 - theCUBE
summary
Tapjoy runs over 500 million active monthly users and uses HP Vertica for SQL on Hadoop to move curated data from Hadoop ML jobs into operational star schemas. Real-time serving shifts from HBase queues and batch ETL toward in‑memory SQL (Mem) for low‑latency rule evaluation. Toolchain includes Kafka, Spark and traditional ETL; biggest friction is metadata and semantic joins (user IDs, country). Core claim: SQL elasticity and real‑time availability give the necessary accuracy and transparency for both ML and BI.
AI-written summary · about
chapters (10)
- 0:00Introduction to HP Big Data 2015
- 0:44Tapjoy's mobile monetization growth
- 2:30Mobile revolution and data community
- 3:35Big data challenges and data quality
- 5:47Data flexibility vs. precision trade-off
- 7:13Metadata and semantic challenges
- 8:17Building systems of intelligence
- 9:19Real-time data processing shift
- 10:25ETL offload and Vertica integration
- 11:52Future trends in big data
