Advancing security analytics with Snowflake and AWS at Snyk (PRT329)
summary
Snyk builds a customer-facing reporting product on Snowflake + dbt + TopCoat, ingesting events via Debezium → Kafka and using streams and tasks to create history and current-state tables; many queries run near real time (seconds–minutes) with under-an-hour latency for modeled marts. Producers conform to a shared issue schema as a data contract, automated dbt tests enforce quality, clustering and pre-aggregation reduce scanned micro-partitions, and the stack deploys globally via Infrastructure as Code—product launched in closed beta and meets the performance and extensibility goals.
AI-written summary · about
chapters (11)
- 0:00Introduction to Snyk
- 1:25Data challenges at scale
- 4:30Building on data warehouses
- 7:29Defining project requirements
- 10:16Implementing the data stack
- 13:20Low-latency data pipelines
- 15:26Ensuring data quality
- 17:40Optimizing for performance
- 21:32Deployment and infrastructure
- 23:29User-focused analytics design
- 27:23Future and extensibility
related talks
52mHow Snowflake uses Snyk to give data-driven analytics for security
23mDelivering Retail Analytics & Data Projects With Data Created For Purpose
21mSnowflake and dbt -- Our Journey to the Cloud
01h 17mData Engineering in the AI Era: Building a Robust Foundation for AI Applications
30mBuilding Data AI Application with the Snowflake AI Data Cloud
28mUsing Dataops To Build And Manage Data Products On Snowflake
39mBREAKING YOUR DATA TEAM OUT OF THE SERVICE TRAP
01h 20mSupercharge Your Real-Time Analytics: Snowpipe Streaming & Dynamic Tables in Snowflake