Building a Global CyberGrid on Cassandra
summary
Builds a global cyber grid that ingests hundreds of organizations and tens of terabytes per day using Kafka for ingestion, an NLP stage and graph processing, then persists JSON blobs in Cassandra with a single keyspace, orgID+partition TimeUUID primary key, replication factor 3, compression and TTL per cluster for automatic garbage collection. Queries use partition summary scans and full-partition fetches to avoid no secondary indexes on Cassandra. Cross-cluster sharing uses REST JSON transport and configurable TTLs; deployments range from Raspberry Pi to multi-cloud datacenters and prepare vectorized outputs for LLMs.
AI-written summary · about
chapters (9)
- 0:00Global CyberGrid intro
- 1:48Project scope and needs
- 4:38Distributing config
- 7:07Data philosophy
- 8:11Keyspace and schema
- 13:14Querying and indexing
- 15:18NLP and graph analysis
- 17:24Federated data sharing
- 19:57Q&A on TTL and compaction
related talks
24mScaling to Billions of Internet Observations per Day
25mThe Importance Of Real time Data In Fighting Cyber Attacks
18mBuilding a Scalable, Resilient System for Real-Time Fraud Detection
28mCloud Computing for the Power Grid: From Service Composition to Assured Clouds
05mScaling Kubescape to Thousands of Nodes
24mon the hunt for cannabis farms and electrical grid stability
29mScaling to Millions of ML Models to Solve the Problems of SRE and Security
49mA Real-World Example of Using Network Behaviour to Detect Cyber-Compromise