Quicksilver How Cloudflare Controls it's Network Using Go
summary
Cloudflare describes Quicksilver, a distributed datastore implemented in Go that serves as the control plane for its global infrastructure across 165 datacenters. The talk explains how Quicksilver was built to deliver low-latency, resilient operation with low operational overhead while handling about 10^12 requests per day.
AI-written summary · about
chapters (23)
- 0:00Intro
- 2:18Cloudflare network
- 3:05What a Cloudflare server does
- 3:47Distributing config across our servers
- 4:57The problem of managing config at scale
- 8:52Technology behind Quicksilver
- 10:47Storage engine: LMDB
- 13:50Quicksilver daemon running on each server
- 16:44Quicksilver replica nodes
- 17:31Quicksilver primary node
- 18:02Avoiding misconfiguration in replication
- 18:28Logical replication of changes
- 19:06Chain of hashed long entries
- 20:07Applying a log-entry to the DB
- 21:00Overloading a single node with replication
- 21:22Global replication topology
- 22:13Redundant tiered replication topology
- 22:58Replication inside a data center
- 23:14How do we bootstrap an empty server
- 24:04Replication topology heartbeat
- 25:33Data integrity
- 26:16Observability
- 26:56Reasons for the success of this project
related talks
46mSimplifying Scalable Cloud Application Development with Go
06mInterview – John Graham Cumming
28mMicroservices – Boyan Dimitrov
15mI came for the easy concurrency I stayed for the easy composition
16mScale with NSQ: a realtime distributed messaging platform
29mGolang and JavaScript: The Future of Cloud
33mCloudstate Towards Stateful Serverless
25mCloudstate—Towards Stateful Serverless