The Yellow Elephant and His Friends

on YouTube ↗46m

" summary

A presentation about building an automated VAT-declaration validation system for the Russian tax authority using a Hadoop-based stack. It describes challenges encountered and compares approaches including YARN MapReduce, Impala, recent Spark versions, and LLAP long-lived executors (developer-preview).

AI-written summary · about

" chapters (8)

1. 0:00Introduction
2. 3:36Cluster life story
3. 7:28Cluster config problems
4. 13:39Security and culture limits
5. 22:51Metrics and monitoring
6. 27:57Performance issues
7. 38:18Docker and testing
8. 42:35Lessons learned

" related talks

• 01h 29mHadoop 201 -- Deeper into the ElephantRoman Shaposhnik · SpringOne2GX 2014
• 50mBabar-ians at the Gate: Data Protection at Massive ScaleDavi Ottenheimer · Black Hat USA 2014
• 47mSolving Real & Big (Data) Problems Using HadoopEva Andreasson · GOTO Chicago 2014
• 01h 09mHadoop Security: Seven Ways to Kill an ElephantDavi Ottenheimer · Black Hat Europe 2014
• 22mA Cat On A Horse & Actively Embracing Distributed Data AnalyticsUmair Waheed · Big Data LDN 2022
• 02h 05mUsing Hadoop with Excel and Power BI, Plus Data Discovery 101Andrew J. Brust · PASS Business Analytics Conference 2015
• 53mBig Data: from mammoth to elephantRoman Nikitchenko · JavaDay Lviv 2015
• 25mJeff Veis & Anand Raman, Impetus Technologies | Databricks Data + AI Summit 2024Anand Raman, Jeff Veis · Databricks Data + AI Summit 2024