Stop querying tables: give AI governed metrics instead
summary
Claim: letting LLMs query raw tables produces hallucinated KPIs (example: AI reports average revenue per order $258 when dashboard shows much higher), misreads time semantics (AI gives 689k YTD vs dashboard 574k because financial year starts April 1), and triggers expensive scans. Built a metrics layer (Rill) that stores SQL-defined metrics plus YAML context (Squamel), exposes metrics via MCP and Metric SQL, and auto-generates measures like net sales = sum(sales)-sum(discount); with the metrics layer answers match dashboards and stay performant.
AI-written summary · about
chapters (14)
- 0:00AI and metrics layers intro
- 1:03AI shouldn’t query raw tables
- 6:16Need for business semantics layer
- 7:18Rill’s AI-native BI stack
- 8:00Building metrics layer with AI
- 12:35Conversational analytics with metrics
- 14:39Rill’s AI analytics architecture
- 15:41Metrics as AI building blocks
- 17:25Security and governance in metrics
- 18:48Best practices from Rill
- 21:14Transparent AI responses
- 22:40Key takeaways and future
- 24:46Recommended papers and resources
- 25:33Q&A
related talks
18mIntroducing a SQL-based metrics layer powered by
26mMetrics-driven Data Architectures
27mMetric Store
40mMetrics and Semantics: An evolution in data consumption
45mForget SQL! Create your own AI Data Analyst
29mAI Driven Observability based on Open Source
29mKeynote: LLMs, what are they good for?
23mAI/ML + Grafana: How to Create Intelligent Grafana Apps Leveraging LLMs