Data integration at scale: Strategies to unlock SAP and mainframe

on YouTube ↗01h 01m

" summary

Qlik Replicate powers a production data-replication-as-a-service: 18 cloud servers handle ~400M mainframe transactions/day with 2 seconds target latency and zero-loss failover using AWS FSx mirroring; log-streaming cuts mainframe MIPS from hundreds to single digits. Targets include S3, Kinesis (with Lambda denormalization), Aurora and DynamoDB; full loads: ~2M rows ≈1.5 minutes, ~30M rows ≈15 minutes. Deployments use CloudFormation + Lambda + Secrets Manager for infrastructure as code and provide a golden-path self-service experience.

AI-written summary · about

" chapters (8)

1. 0:00Introduction and speakers
2. 1:06Integration at scale overview
3. 8:27Vanguard’s Mainframe modernization
4. 19:55Data replication as a service
5. 30:29Log streaming for scalability
6. 44:45SAP data unlocking strategies
7. 55:51ABB SAP integration case study
8. 1:00:29Final thoughts and Q&A

" related talks

• 45mModern Data ArchitectureAlexey Grishchenko · JavaDay Kyiv 2015
• 49mBuilding a Data-Driven BankFelipe Melo · Build Stuff 2022
• 39mStreamlining Manufacturing ReportsSebastian Collins, Michele Lamarca, Dennis Kemmerer, Daniel Lütkehermöller · AWS Summit Berlin 2023
• 42mTransform your data approach: Develop a modern data strategyBlair Layton, Saeed Barghi · AWS Summit Singapore 2022
• 27mModernize Your Data WarehouseGal Barnea · AWS AMER Summit May 2021
• 01h 03mReinvente su estrategia de datos con Lagos de Datos TransaccionalesJavier Quiroz, Santiago Abarca · AWS Summit Mexico City 2022
• 01h 03mDiscovering the value of a cloud data platformMichael Oppenheim, Arunashish Majumdar, Chad Duncan · AWS re:Invent 2019
• 39mLarge scale streaming analyticsJay Palaniappan · Data Con LA 2019