Ordering the chaos - cleaning logs and ordering events in microservices
summary
Timestamps are unreliable across microservices because clocks drift, DST and tzdb changes and even leap seconds (2012 caused MySQL CPU spikes). The practical fix is sending a Lamport timestamps integer with each message plus a stable correlation id header (HTTP, queue metadata) and updating time with max+1 via interlocked ops; examples use .NET, tz/CLDR, NTP. For stronger ordering use vector clocks or a Google-style TrueTime 7ms bound when available.
AI-written summary · about
chapters (21)
- 0:00Intro
- 0:42About me
- 6:40What is timezone
- 9:08Timezones change
- 10:40Daylight Saving Time
- 12:52Leap second
- 14:03How is time distributed
- 26:42Network Time Protocol (NTP)
- 28:08Other approaches
- 30:35Lamport timestamp
- 36:47Implementation
- 40:47Logger
- 42:52Step 1
- 43:36Step 2
- 44:25Step 3
- 45:17Step 4
- 49:48Vector clock
- 52:53Byzantine generals
- 55:02Consensus problem
- 56:16Raft
- 59:22References
related talks
56mOrdering the chaos - cleaning logs and ordering events in microservices
39mKeeping Time in Real Systems
14mLightning Talk: Beyond NTP: High-accuracy clocks for distributed observability
43mDistributed Systems
45mHello from the other side
26mReal-Time Decision Making in Financial Networks
55mThe hardest part of microservices: Your Data
42mHow to Have your Causality and Wall Clocks, Too