Journey of Building a Scalable, Domain-Agnostic Order Management System
summary
CRED builds a domain‑agnostic OMS that uses MySQL → DMS → Kafka → custom connector → OpenSearch with a CQRS split: MySQL for writes, OpenSearch for reads. System indexes >4 billion documents at ~20K writes/min and ~26K searches/min; optimizations include merging tiny shards and using aliases, per‑user routing plus time‑bounded queries to cut P99 from ~400ms to ~35ms, live reindexing via update_by_query, moving to memory‑optimized nodes and disabling swap, hitting 1500 RPS at ~30ms.
AI-written summary · about
chapters (13)
- 0:00Introduction
- 0:22About CRED
- 1:51Order management challenges
- 3:57Domain-agnostic order system
- 6:47Event-driven architecture
- 7:10OpenSearch in OMS queries
- 8:14Scaling OpenSearch
- 9:18Initial OpenSearch setup
- 10:00Shard and index management
- 10:45Reducing latency
- 11:28Reindexing without downtime
- 12:31Fixing latency spikes
- 13:34Performance achievements
related talks
09mIntroducing Opensearch in existing multisource enterprise App
25mHow Sharekhan Scaled Trading Platform with Yugabyte's Distributed SQL
14mKey Considerations For Success When Deploying A Solution For Best-of-breed Observability At Scale
49m10ms Response with Transactions, Big Data, Flexible Logic and OpenSource
35mInnovating at Scale: Lessons from Pegasystems' OpenSearch Journey
29mApplying (D)DDD and CQ(R)S to Cloud Architectures
22mImproving the efficiency of more than 800 databases with o11y
41mDynamoDB, Neptune and OpenSearch: A Unified Approach to Collaborative Editing and Search #JSConfAM25