ApertureDB: Designing a Purpose-built System for Visual Data and Data Science
summary
Purpose-built ApertureDB stores metadata as a property graph (pmgd persistent memory graph), serves images/videos from S3 or filesystem, and exposes a visual-first JSON API with Python/C++ clients, RBAC and dataset loaders. Benchmarks use the Yahoo Flickr ~100M image corpus: DIY (Postgres + Apache + OpenCV) is up to 35x slower and 15x slower on average; pmgd outruns Neo4j by ~14x on some queries and >3x throughput in reads. It supports pre-processing near data and feature-vector similarity search, and the system is production-ready at 300M+ images.
AI-written summary · about
chapters (12)
- 0:00Carnegie Mellon University
- 12:43ApertureDB: Purpose-built for visual analytics
- 14:37How ApertureDB Fits In User Ecosystem
- 19:15Example Metadata Schema (COCO)
- 19:38Another Example Schema
- 20:03In-Memory Graph Database
- 21:53Salient Design Features
- 26:24PMGD vs. Neo4j using LDBC - Circa 2019
- 28:07Visual Compute Module
- 32:00Query Engine (Orchestrator)
- 33:21Visual-first Native JSON API
- 40:37Upcoming Features
related talks
22mSuperDuperDB: Bring AI to your favourite database!
34mAnalytics and AI with PostgreSQL - Mission Possible!
40mData Intelligence Platform on AWS
33mStreamlining Metadata Management with PostgreSQL and Apache Gravitino
34mAnalytics and AI with PostgreSQL - Mission Possible!
29mPurpose-Built Databases for Modern Applications
38mThe SmugMug Tale
31mOpenDataCube - Petabyte Scale Satellite Imagery Processing in Python