Chuck Yarbrough, Pentaho A Hitachi Group Company - Big Data NYC 2016
summary
Announces a Pentaho big data enhancement release that adds Spark on SQL, Kafka support, extended Kerberos and Apache Sentry integration to secure Hadoop clusters, and a dynamic metadata injection feature that generates PDI transformations at runtime. Claims one customer faced ~6,000 disparate CSVs where each ingest took a day and ~$1,000 to productionize; metadata injection converts thousands of files with one template, outputs Avro/Parquet for conforming, and focuses on operationalizing pipelines rather than hand-crafting ETL.
AI-written summary · about
chapters (11)
- 0:00Welcome to Big Data NYC
- 0:21Pentaho’s role in big data
- 1:05Big data enhancements release
- 2:48Prioritizing customer needs
- 4:36Hybrid cloud approach
- 5:59Securing the data pipeline
- 7:45Filling the data lake
- 9:28Metadata injection benefits
- 11:12Data lake vs data warehouse
- 13:21Operationalizing data lakes
- 14:45Pentaho’s solutions approach
