(Re)building a Framework: Lessons, Community, and Impostor Syndrome
summary
Rebuilding a framework becomes a platform for eval-driven development: open-source code plus plugin manifests, types and schemas feed LLMs and produce traces that are scored, not just final outputs. A prototype agent-generated web app is produced but quality is ambiguous, so the pipeline measures NPM install success, dev run, AST checks, token use and tool calls, and recommends 8–15 stack tests to distinguish “bells and whistles” results. Integration with AppKit/Databricks and Vite-era bundling proves useful; skills and manifests reduce hallucination and speed iteration.
AI-written summary · about
chapters (15)
- 0:00Introduction and welcome
- 1:27Why I build frameworks
- 3:33AI and slop in development
- 5:03UX vs AI experience
- 7:57What matters now in frameworks
- 10:47Security and custom solutions
- 13:38AI for citizen developers
- 15:03Integration over novelty
- 16:26Solid’s declarative goals
- 17:50Databricks framework goals
- 19:14Eval-driven development
- 22:23Lessons from evals
- 24:14Closing the loop with evals
- 25:15Code discoverability tools
- 26:19Final takeaways
related talks
55mKeynote: Exoskeletons, not Autopilots
32mStop Chat-and-Hope and Start Building Better Apps with Agents
12mDay 2: From Foundations to Production
52mHow to win with automation and influence people
44mAvoiding the pitfalls of rebuilding software
30mOptimizing for Developer Delight
19mWrap: What To Build Next
01h 01mArchitecture Decisions: A Case Study -