Analytics at Scale V02

on YouTube ↗18m

" summary

Dune grows from a Telegram group into a live analytics platform, credits community-curated dashboards for the DeFi Summer inflection, and scales engineering to 45 people while battling node strain from Ethereum forks and massive throughput on Solana support. The product adds APIs that turn queries into endpoints and data shares so any dashboard becomes a usable data feed, experiments with natural language SQL generation but says AI assistance still needs human curation, and shifts toward an open platform for third-party data ingestion.

AI-written summary · about

" chapters (35)

1. 0:00Introduction to Dune Analytics and Fredrik Haga
2. 0:51The Early Days of Blockchain Data Challenges
3. 1:26Founding Dune During Crypto Winter 2018
4. 2:02Recognizing the Need for On-Chain Analytics
5. 2:35The Epiphany: Public Blockchain Data as Opportunity
6. 3:04Unique Aspects of Open Blockchain Data
7. 3:36Building for Open Data vs Traditional Silos
8. 4:07Has Querying On-Chain Data Gotten Easier?
9. 4:36The Current State of Blockchain Data Tooling
10. 5:05Progress and Remaining Challenges in Data Analytics
11. 5:39The Dune Wizards Community Origin Story
12. 6:09Serving the Early Crypto Data Enthusiasts
13. 6:37Community Growth During DeFi Summer 2020
14. 7:05Open Collaboration vs Data Silos
15. 7:35The Inflection Point of DeFi Summer
16. 8:08Engineering Challenges of Scaling Dune
17. 8:40Handling Arbitrary User Queries at Scale
18. 9:15Challenges with Ethereum Forks and High-Throughput Chains
19. 9:42Supporting Solana and Other High-Volume Chains
20. 10:20Scaling the Dune Team to 45 People
21. 10:51Managing Through Crypto Market Cycles
22. 11:27The Coming Wave of Thousands of L2s and App Chains
23. 11:58Dune's Platform Approach to Chain Fragmentation
24. 12:29Transitioning to an Open-Ended Data Platform
25. 12:57Enabling Community Data Contributions
26. 13:25Expanding Beyond Strictly On-Chain Data
27. 13:57APIs and Data Sharing Features
28. 14:27AI and Natural Language Query Innovations
29. 14:56The Role of Human Curation in Analytics
30. 15:29Combining Community Insights with AI
31. 16:04Underutilized Potential of On-Chain Data
32. 16:34Comparing Web2 vs Web3 Data Accessibility
33. 17:03Accelerating Startup Feedback Loops
34. 17:33The Power of Open Data for Builders
35. 18:02Closing Thoughts on Open Data Ecosystems

" related talks

• 24mBlockchain Analytics with DuneMersiha Karamustafic · DSC Europe 24
• 29mHow Developer Tooling Shapes LandscapeChi Zhang, Paul Kim, Timmy Han · Korea Blockchain Week 2023
• 19mInsights for Everyone - Democratising AnalyticsKamalakannan Thiruvadi · MachineCon 2022
• 13mConnecting data sources with AILuka Fontanilla · Google Cloud Next 2024
• 27mDashboards Are Dead: How To Democratise Data With Live Analytics (with fault)Daniel Nazeriha, Luke Treglown · Big Data LDN 2022
• 32mUsing Dashboards to Help Manage All Community ChannelsSrini Kadamati · ApacheCon@Home 2021
• 53mDemocratising Attention Data at theguardian.comGraham Tackley · GOTO Aarhus 2013
• 33mData: The Land DevOps ForgotMichael Nygard · GOTO Copenhagen 2024