End-2-End Resource Accounting Leveraging Distributed Tracing
summary
A system called Transitive Resource Accounting builds on Canopy distributed traces to attach per-request CPU cost back to the originating front-end, starting traces from Facebook, Instagram and Messenger and targeting WhatsApp via an Erlang SDK. Traces reveal that transitive demand can exceed direct-call counts by up to 16x; current coverage captures about 60% of power use. Sampling and overhead-elimination models reduce tracing cost; AB-test analyses predict 5-10% CPU savings (up to ~25% with heavier feature rollbacks) on impacted services.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 1:53Capacity planning at scale
- 3:16Distributed capacity planning challenges
- 7:08Introducing TRN
- 8:55Full trace coverage
- 10:39Two perspectives on accounting data
- 11:23TRN for feature capacity
- 12:31Handling shortages with toggles
- 14:17TRN implementation learnings
- 16:22TRN coverage and improvements
related talks
16mPragmatic execution tracing
46mVisualising and tracing requests through a cluster: Intergrating OpenTracing into OpenStack Swift
01h 30mIntroducing CallTracing(tm), based on RabbitMQ, Spring and Zipkin
26mPerformance Analysis of Cloud Applications
26mDistributed Tracing FAQ
19mOops my app performance sucks... what if observability could be a game changer?
35mTracing is More Than Traces: The Insights in Trace Aggregates
42mDistributed Tracing Opening the Door to Visualisation and Correlation