So who wants to be a Munger?
summary
A concrete workflow emerges: separate reader, munge, writer; build a Munger class that yields to a block; parse fixed‑width, repeated headers and hierarchical categories with unpack; drop human lines and subtotals, normalize "K" suffixes into integers and export via faster_csv into a database. The claim is practical and proven on a messy 58,000-line report, reducing manual Excel typing and automating reliable, repeatable imports.
AI-written summary · about
chapters (23)
- 0:00Data munging in Ruby
- 1:45Rule of three in processing
- 3:10Separate reading from munging
- 6:04Flexible Munger class
- 7:06Structured vs unstructured data
- 8:10Messy data example
- 9:57Filter irrelevant report lines
- 11:21Unpack fixed-width data
- 13:29Parse repeating headers
- 14:57Track header sections
- 16:00Assoc for category lookups
- 17:44Reader output structure
- 18:47Write data with FasterCSV
- 19:33Munge transformed data
- 20:59Output to different formats
- 22:03Drop data into SQLite
- 23:08CSV into ActiveRecord
- 24:18Handle thousand abbreviations
- 25:43Regex anchors for matching
- 26:50Corporate data restrictions
- 28:14From typing to scripts
- 29:38Scripts to productivity
- 30:21Q&A
related talks
25mScope of data journalism for developers
25mThe Hell, According to a Data Scientist
48mMind of a Hacker
58m10 Techniques to Understand Existing Code
01h 15m10x Development: LLMs For the working Programmer
22mA Neatly Typed Message: Improving Code Readability
34mNarrative Reflections: Transmuting Ruby Code into...
44mIts all About the Journey