Caching Snowflake data with Aerospike for advanced analytics
summary
Fidelity builds a Snowflake -> Aerospike pipeline using a custom Java 11 loader that pulls SQL via the Snowflake JDBC, maps types (storing VARIANT/ARRAY as strings), and multi‑threads to push into Aerospike clusters on AWS with 3× replication and XDR cross‑region. Performance tests show 200,000 writes/sec and 25–40k reads/sec for mixed workloads; production nodes run on i3en.3xlarge 7.5TB NVMe with indexes in memory and automated 60‑day node rehydration.
AI-written summary · about
chapters (7)
- 0:00Start
- 0:28Caching Snowflake Data with Aerospike
- 1:16Top Fidelity Use Cases for Aerospike
- 1:37Fidelity Use Cases
- 5:27Market Risk Analysis
- 12:41Caveats with Snowflake
- 15:17Why Fidelity Chose Aerospike over Other Nosql Platforms
related talks
40mDeveloping highly scalable applications with Aerospike
24mCloud Workloads At Petabyte Scale
15mAWS On Air Voice of the Customer ft. Aerospike, Inc.
33mInnovation with Aerospike: How AppLovin Handles Billions of Requests per Day
22mThe Art of the Possible
29mDream11: Aerospike Beyond Cache
13mLightning Talk: Highlights of the Snowflake Elastic Data Warehouse
30mWelcome and the Aerospike Journey