SnapLogic and Amazon Redshift transform insights at Lumeris
summary
Project Fuse stitches SnapLogic pipelines into Amazon Redshift so metadata drives extract, transform, load across SQL Server, Postgres, DynamoDB, S3 and SaaS sources. Built with audit, row-level validation and incident-ticket plugins, the core was prototyped over a weekend and productionized in under a month with ~1.5 part-time engineers; it’s been running 9–10 months. It loads ~500k rows in ~50 minutes, cuts dataset delivery from weeks to 2–3 days, and lets analysts spawn a dataset in ~2 minutes.
AI-written summary · about
chapters (13)
- 0:00Introduction
- 1:12What is Lumeris
- 2:15How Lumeris makes money
- 4:16Challenges at Lumeris
- 5:45Project Fuse
- 8:08Incident Management
- 9:39Data Validation
- 11:57SnapLogic Pipeline
- 12:42SnapLogic Framework
- 13:23Metadata
- 14:22Logs
- 14:53Dashboard
- 15:38Best practices
related talks
21mSimplify Your Data Stack: Automate Orchestrate and Integrate Data
09mCuring modern data architecture complexity
29mSimplifying data flows
28mTurn Data Into Insights
43mData warehousing reinvented for today's needs
59mModern data lakes with Lake Formation, Amazon Redshift & AWS Glue
01h 02mData warehousing reinvented for today’s needs
42mTransform your data approach: Develop a modern data strategy