Debugging Hadoop

on YouTube ↗09m

" summary

Claims that debugging shifts from single-machine deep dives to first locating outlier machines across clusters and tracing cross-node interactions. Recommends distributed tracing stacks (X‑Trace/Dapper/Zipkin) and mentions ongoing Zipkin integration with HBase/HDFS. Describes Cloudera Manager features: repeatable config, monitoring, queryable metrics and visualizations to spot anomalies, and concrete alerts such as secondary namenode not checkpointed >1h. Describes resource-management “soundproofing” to isolate noisy neighbors at the cost of capacity.

AI-written summary · about

" chapters (4)

1. 0:00Intro
2. 1:28Why is debugging so reactive
3. 2:47Pain points in debugging
4. 4:53Clanderas approach

" related talks

• 49mDistributed Tracing: How the Pros Debug Concurrent and Distributed SystemsAaron Stannard · QCon São Paulo 2018
• 24mDistributed Tracing : New DevOps Foundation.Jayesh Bapu Ahire · Devoxx Ukraine 2020
• 41mFinding the Needle in the Haystack or Troubleshooting Distributed SystemsAnthony Molinaro · Erlang Factory SF Bay Area 2014
• 46mHow Cheap Learning (not deep learning) Finds Faults in Distributed SystemsTed Dunning · Jfokus 2019
• 30mDistributed Tracing: From Theory to PracticeStella Cotton · LISA17
• 38mDistributed Tracing: Practices, Patterns & Technologies for Debugging MicroservicesElemar Júnior · DevDay 2019
• 11mKeynote: Distributed Tracing - The Struggle is RealIan Smith · Open Observability Day North America 2022
• 01h 00mSystems @Scale Summer Q&A Week 4Yun Jin · Systems @Scale Summer 2021