A Tale of Two Clouds: Analytics Platforms at ING
summary
ING runs analytics on two clouds: an ING private cloud (44 servers, 35 compute nodes, 20 GPUs, 8 masternodes) for steady on‑prem workloads and Google Cloud (BigQuery, Vertex AI, Dataflow, Dataproc, Looker) for scale. Platform is Kubernetes-based with JupyterLab/Vertex workbenches, Spark, Flink, Alluxio caching and Iceberg files over Dell ECS S3. About 2,500 users and a 70‑person team face GDPR, Schrems II/US‑nexus and DORA constraints, so the engineering choice is hybrid, cloud‑native, open‑source first and product‑focused.
AI-written summary · about
chapters (9)
- 0:00Intro
- 3:36The ocean got even bigger
- 9:23Context for a bank
- 15:40ING's on-premise servers
- 20:28Tools
- 30:53The journey is not a straight line
- 36:55Obstacles & how to tackle them
- 41:21It's about the user experience
- 43:32Outro
related talks
27mData Analytics Democratization: How ING Data Analytics Platform Bootstraps New...
43mDevOps at ING Analytics
34mThe Data Analytics Platform or How to Make Data Science in a Box Possible
46mFast Data at ING
50mData Science in the Enterprise
38mTaking away customer friction through streaming analytics
05mHadoop in the cloud: A Nielsen use case
37mHybrid OnPrem + Cloud Architectures