Data on Kubernetes Day North America 2023
2023-11-06
Chicago, United States
as the organisers put it
Running data workloads on Kubernetes has a transformative impact on organizations who benefit from increased productivity, revenue growth, market share, and margin. Data on Kubernetes Day (DoK Day) is…
who it's for
platform engineers, data engineers, site reliability engineers, database administrators, machine learning engineers, infrastructure architects, devops engineers
topics
Kubernetes operators for databases; statefulset persistent volumes; CSI drivers and FUSE object mounts; NVMe over TCP for PostgreSQL; distributed vector databases and HNSW; PGVector and embedding stores; Spark and Flink on Kubernetes; multi-region automated failover; Kafka production practices with Strimzi; data integrity tests in CI/CD; security hardening for operators; QoS-aware colocated scheduling
| title | author | length |
|---|---|---|
| Opening Remarks - More Than Kubernetes: The Data on Kubernetes Ecosystem, Program Committee Member | Melissa Logan | 10m |
Community-run by Costantia.io launches a searchable resource library after KubeCon cataloging hundreds of videos and tutorials, runs monthly virtual meetups… | ||
| Transforming Data Processing with Kubernetes: Journey Towards a Self-Serve Data Mesh | Rakesh Subramanian Suresh, Jainik Vora | 34m |
Intuit reworks data pipelines onto Kubernetes to serve a self-serve Data Mesh that handles 100,000 data pipelines and 1000+ engineers; runtime uses Apache… | ||
| Zero-Touch Fault-Tolerance for Cloud-Native Geo-Distributed Databa... | Selvi Kadirvel, Aman Nijhawan | 32m |
Demo shows a multi-cluster orchestrator (Nova) deploying YugabyteDB across three regions (three clusters per region, replication factor 3) on five GKE clusters… | ||
| Distributed Vector Databases - What, Why, and How | Steve Pousty | 32m |
Explains how unstructured data turns into 512-dimensional vectors via pretrained embedding models (OpenAI, Hugging Face), how those vectors live in… | ||
| Stateful Workloads in Kubernetes: A Deep Dive | Kaslin Fields, Michelle Au | 29m |
Kubernetes treats statefulness as “something else cares about this workload,” and the concrete toolkit includes StatefulSet sticky identity for stable network… | ||
| Run PostgreSQL in Warp Speed Using NVMe/TCP in the Cloud | Sagi Volkov | 28m |
Demonstrates running PostgreSQL on OpenShift in AWS using NVMe over TCP across three instance-store nodes and reports results from six 30-minute sysbench runs… | ||
| Lightning Talk: Should You Run Your Database on Kubernetes | George Peter Hantzaras | 08m |
Argues that running databases on Kubernetes is viable today by combining statefulsets for stable volume attachment with custom resources and operators to… | ||
| ⚡ Lightning Talk: Colocating Data Workloads and Web Services on Kubernetes to Improve Resource Utilization | He Cao | 06m |
Katalyst colocates CPU‑heavy online services and memory/throughput‑heavy data jobs on Kubernetes and claims deployment on 500,000+ nodes with tens of millions… | ||
| CSI Driver X FUSE Drivers: a Kubernetes Object Storage Solution for AI/ML Data Portability | Jiaxun Song | 22m |
A sidecar-based CSI solution mounts object buckets as FUSE filesystems by having the CSI node open /dev/fuse, send the file descriptor over a Unix domain… | ||
| Is It Safe? Security Hardening for Databases Using Kubernetes Operators | Robert Hodges | 33m |
Operator-based deployment secures databases by shrinking configuration to a small YAML surface and automating credential and certificate provisioning… | ||
| Stop Worrying and Keep Querying, Using Automated Multi-Region Disaster Recovery | Shivani Gupta, Sergey Pronin | 34m |
Automated orchestration uses Percona Operator for PostgreSQL and a multi-cluster control plane (Elotl Nova) to duplicate manifests across regions, wire the… | ||
| Make Your Kafka Cluster Production-Ready | Jakub Scholz | 24m |
Kubernetes-hosted Kafka needs explicit choices: use block or local persistent storage not NFS, enable TLS/mTLS and ACLs, and integrate Strimzi’s user operator… | ||
| Data Integrity Tests in CI/CD Pipelines | Christopher Crow, Stephen Atwell | 22m |
Commit triggers a pipeline that snapshots a production copy to staging, runs a DB migration, then fires a GitHub Actions integration test that validates row… | ||
| Dynamic Large Scale Spark on Kubernetes: Empowering the Community with Argo Workflows and Argo Events | Ovidiu Valeanu, Vara Bonthu | 31m |
A live demo builds an EKS cluster with Argo Workflows and Argo Events, an SQS event source, and the Spark operator to launch a DAG of Spark jobs; Carpenter… | ||
| Lightning Talk: Scaling Time Series Databases on Kubernetes | Ken Ahrens | 07m |
Speedscale runs TimescaleDB to track billions of API interactions and claims single-digit-hundreds-millisecond queries at scale. Deploying the Helm chart works… | ||
| ⚡ Lightning Talk: Using Decentralized File System to Optimize ML/AI Workloads on Kubernetes | Shawn Sun | 05m |
Describes a decentralized virtual file system (Alluxio) that removes master nodes and uses consistent hashing on the client side to place data and metadata on… | ||
| Closing Remarks | Melissa Logan | 01m |
Announces three named community contacts: ambassadors Robert Alex Gabriel available for questions during the conference, shows their photos on-screen, and… | ||
| nothing matches | ||