Liberation and modernization of government legacy data using Django

on YouTube ↗34m

" summary

Puerto Rico builds the LIBRE API engine on Django and Django REST Framework to liberate legacy files from 142 agencies, converting Excel, shapefiles and proprietary exports into JSON, WGS84 geodata and renderable maps. A custom liquid query language makes URL-driven filtering, subqueries and map/chart renderers possible; imports do heavy lifting (initially base64‑pickled blobs) so reads are fast. Results include iframe-embeddable crime maps, dashboards that reveal lost industrial customers and an estimated net migration of ~1,000 residents (5.2M out vs 5.2M back) used to change operational decisions.

AI-written summary · about

" chapters (46)

1. 0:00Introduction
2. 0:32About me
3. 0:42About the company
4. 0:57About the founder
5. 1:29How I got into it
6. 1:57The problem
7. 2:42Checklist
8. 3:10Start Development
9. 3:22Libre
10. 3:39Ego Eye View
11. 5:38Data Sharing
12. 6:10Spatial Capabilities
13. 7:06Modernization for Developers
14. 7:51People have the weight of the infrastructure
15. 8:15Django ministration
16. 8:48Exporting data
17. 9:00Web services
18. 9:53Standardization
19. 10:06Interoperability
20. 10:30REST
21. 10:41JSON Self Documenting
22. 10:59Django Rest
23. 11:24Unified Query Language
24. 11:43SQL injection
25. 12:33SQL is not sustained
26. 12:44A Stack Overflow
27. 12:58Liquid Query Language
28. 13:18REST Query Language
29. 14:17REST Query Example
30. 14:35Rewriting URL
31. 14:52Customizing map
32. 15:25Cosmetic markup
33. 16:58Query Builder
34. 17:35Query String
35. 18:10Feature Analysis
36. 21:24Read
37. 22:03Why the project died
38. 23:06Creating dashboards
39. 23:27Plotting data
40. 26:20Crime Map
41. 27:11Solid Waste Data
42. 27:32GeoTires
43. 28:31Instant Republic
44. 28:48Institute of Statistics
45. 29:10Microsoft Office
46. 31:07Puerto Rico Exodus

" related talks

• 22mGovernment dashboards and open data.Paul Haesler · PyCon Australia 2015
• 22mGovernment dashboards and open data.Paul Haesler · PyCon Australia 2015 Science/Data Miniconf
• 43mBehind the Scenes at Gapminder: A Story about Data and VisualizationsJasper Heeffer · Øredev 2018
• 47mBrasil.IO: Liberating Public Data with PythonÁlvaro Fernandes de Abreu Justen · QCon São Paulo 2019
• 20mOpen Source Software in Action: Expanding the Spatial Equity Data ToolGabriel Morrison · posit::conf(2024)
• 28mAutomate Your City Data With PythonPhilip James · PyBay 2024
• 33mData Science & Data Visualization in PythonRadovan Kavicky · PyData Berlin 2017
• 48mSaving Gaia with jQuery Mobile and GeoDjangoCalvin Cheng · EuroPython 2011