Guillotina - An Async REST Resource DB to manage millions of objects
summary
Guillotina is an async, transactional REST resource DB written for Python 3.6 that maps JSON Schema to objects exposed via traversal URLs and scales statelessly across frontends. It runs on PostgreSQL and CockroachDB, indexes with Elasticsearch, supports resumable uploads and WebSockets, and integrates with TensorFlow Serving for inference and distributed training. Deployments handle 10 million–100 million objects in production, use workers coordinated via etcd, and report low memory footprints (single-digit megabytes per API).
AI-written summary · about
chapters (22)
- 0:00Introduction
- 1:55Data scientist vs engineer challenges
- 3:43Web frameworks to REST APIs
- 6:15Python and ZODB history
- 8:04From Pyramid to Guillotina
- 9:52Guillotina architecture
- 13:07Performance and async
- 16:01Database support
- 17:26Indexing search caching
- 18:30Kubernetes Python focus
- 20:01Guillotina use cases
- 22:32Quick start API
- 25:24Data modeling
- 27:08Events endpoints permissions
- 28:35Future for data scientists
- 30:22Distributed task execution
- 31:24TensorFlow model inference
- 32:35Distributed model training
- 34:02Restricted Python
- 35:31Community events
- 36:21Q&A production readiness
- 37:46Q&A indexing performance
related talks
25mProcessing Biggish Data with DuckDB and Python
52mCockroachDB
26mGizmoEdge: A distributed DuckDB engine for IoT
47mMy entire stack is in my Postgres DB and a few power tools for my backend
29mBuilding a biological database with Python
23mWhy Squarespace adopted CockroachDB for better global performance at scale
33mStreamlining Metadata Management with PostgreSQL and Apache Gravitino
24mHow Iterable enables personalized experiences for billions of users