Confessions of a Data Junkie: Fetching, Parsing, and Visualization
summary
Uses Ruby libraries (grit) to pull GitHub repos, exports to line-by-line JSON, imports that into Mongo, runs map-reduce to sum per-author totals, and filters commits >300 lines to reduce noise. Builds a small Sinatra route for repo commits and generates visualizations with Protovis client-side and server-side SVG via ruby-vis. Produces concrete artifacts: a 22,000-commit Ruby repo timeline, a code-size-over-time chart, a last-20-commits bar chart, and a committer bubble plot, all rendered as static SVG files that work without requerying.
AI-written summary · about
chapters (18)
- 0:00Intro
- 0:42Data Visualization
- 1:41Grid
- 5:23Importing Data
- 7:07No Relation
- 9:24Originalization Libraries
- 10:18Proteas
- 12:04Time
- 13:54Snappy
- 15:57Visualization
- 18:08Javascript
- 19:13Ruby
- 24:17HTML Output
- 24:35Commuters
- 26:00MapReduce
- 27:20Recreated Array
- 28:16Labels
- 28:33SVG
related talks
32mCode More, Draw Less, and Debug Just a Little!
29mDemystifying Data Science: A Live Tutorial
07mLightning Talk: Smaug's Cave: Studying Thousands of Gems at Shopify
31mA Picture is Worth a Thousand Words: Bring Data Visualization in Backstage
18mData Visualisation with Ruby
31mBeyond the Bar Graph
25mPowerful (but Easy) Data Visualization with the Graph Gem
44mAnthropology of Ruby: Excavating its Gems