Digitising 100 years of parliamentary data - Django as a framework for government data management
summary
Parliamentary Handbook data spanning 100 years is migrated from messy Excel into a Django system built on the Popolo civic model, supporting interactive biographies, ministries, parties and custom dynamic tables that update as records change. Developers rewrite models ~20 times, load roughly 1,600 individuals, and run ad hoc queries (the new ministry is only about two years younger on average). Tools include Django, Haystack + Whoosh, a Django data interrogator UI and a doc generator called spaghetti and meatballs; the system works and is being open sourced.
AI-written summary · about
chapters (19)
- 0:00Introduction
- 0:24Who is Samuel Spencer
- 3:17Long names for MPs
- 3:29Writing models
- 5:18Edmund Barton
- 6:48Proof of concept
- 8:32Graphs
- 10:44Senate history
- 12:10Dates in history
- 12:43Mandatory architecture
- 14:05Why Django
- 15:48How did we do it
- 18:30Why did we do it
- 19:07Jacobsons Law
- 20:14Assumptions
- 21:37Timelines
- 21:55Lessons
- 22:05Summary
- 22:17Questions
related talks
27mA Data Harmonization Engine on top of Django - The Good, the Bad, and the Ugly
31mDjango for life (sciences)
16mBrazil: How we are gaining space with Django
26mA Fireside Chat
42mThe Texas Tribune: Making Data, and State Politics, Public
40mNo More Spreadsheets! Building PyLadiesCon Infrastructure with Python and Django
26mBuilding Dynamic Dashboards With Django and D3
29mHow Django helped us make a better conference website