Centralize Commercial Consent Database for 80M Citizens in Over 100K Companies
summary
Centralized consent DB serves 80 million citizens and 100k+ companies, storing 5 billion consent records with 110 million unique phone numbers and 50 million emails, handling ~500 million transactions/day and peaks of 1 million writes/min and 5 million reads/min. Architecture uses YugabyteDB as the strongly consistent, auto-sharded primary store, Elasticsearch for queries, Apache Pulsar for events, HSM-backed HSM encryption, Keycloak auth, Prometheus/Grafana monitoring, and asynchronous multi-DC replication to partner data centers to meet low-latency SLAs.
AI-written summary · about
chapters (12)
- 0:00Intro
- 0:49softtech is a leading technology company
- 2:41We build data-driven agile architectures
- 3:34Today and tomorrow of the use case
- 4:5980M 100K Consent database
- 6:17Business Requirements
- 7:22Modern problems require modern solutions
- 8:26Selecting Database
- 10:51Our architecture
- 12:57Typical AP request flow
- 15:04Multi DC replication
- 17:11Technology Stack (current)
related talks
18mData Modernization Using Confluent and Yugabyte
18mBuilding a Scalable, Resilient System for Real-Time Fraud Detection
19mBuild Global-Scale Apps Using Globally Distributed Autonomous Databases
24mScaling to Billions of Internet Observations per Day
24mGlobally Distributed Databases on Tencent Cloud
23mAdmiral Scales Globally with YugabyteDB on GCP + Single-Digit Latency
19mDataEngBytes 2023 - SYD-T1-04 - Yash Sharma
17mBuilding a Scalable and Performant SaaS Platform with YugabyteDB