Lessons Learned on Building Data Aggregators: an engineering perspective
summary
Revolut onboards businesses by merging data from 7 now growing to ~11–12 external sources into a single internal model, achieving a record under six minutes for simple cases. Integrations use JSON/CSV/SOAP, decorators for caching, serialized execution, rate limits and raw-response storage with per-provider TTLs. Pricing forces architecture choices: per-request billing nearly causes a half a million dollar bill, so aggressive caching and per-identifier strategies are mandatory. Name matching uses Levenshtein distance and meaning mismatches (eg. “insolvent” vs court ruling) drive false positives.
AI-written summary · about
chapters (13)
- 0:00Introduction and speaker background
- 2:12Business onboarding process at Revolut
- 6:25Types of third-party integrations
- 8:10Data formats and authentication challenges
- 10:14Uptime and graceful degradation
- 12:21Building and mapping data models
- 14:27Parsing numeric values and currencies
- 15:50Handling dates and names
- 17:18Meaning and data interpretation risks
- 19:03Evaluating data quality from providers
- 22:12Impact of pricing models on architecture
- 26:49Code architecture and caching strategies
- 31:10Key takeaways and Q&A
related talks
19mA History of Automatic Aggregations
18mLessons from Analysing all of Europe's Data
29mHow I built an entire data platform by myself that thousands of data engineers use in one year
43mArchitecting for Speed & Scale
31mFrom Raw Data to Real Impact: Lessons from Real-World Transformations
36mFrom Machine Learning Startup to Big Data Company
20mSession: From Chaos to Clarity: The Journey to Building a Money-Saving BI Tool
28mBeyond the Balance Sheet: Transactional Data as a Window to Company Health