Skill issue: Lessons from skilling up coding agents to use Langfuse
summary
Claude Code first adds Langfuse from stale pretraining context, ships broken instrumentation, then fetches current docs to fix it, leaving a trace of two LLM calls with no visibility into actions. The team builds a Langfuse skill: a CLI + references + a surfacing a search endpoint that answers natural‑language doc queries, an auto-research loop that proposes improvements (six ideas, three shipped), and guidance to prefer referencing rather than duplicating content after hitting 478 docs and a target‑function that nearly removes necessary doc fetches.
AI-written summary · about
chapters (12)
- 0:00Introduction to Marc Klingen and Langfuse
- 1:22Conceptual mental model for agent skills
- 4:04The problem: scaling documentation and onboarding for coding agents
- 9:03Six key learnings from building a Langfuse skill
- 10:47Learning 2: Helping agents navigate information (sitemaps/formats)
- 11:40Learning 3: Surfacing a search endpoint for docs
- 12:36Learning 4: Implementing basic evaluation (eval) setups
- 13:53Learning 5: Referencing rather than duplicating content
- 14:24Learning 6: Using auto-research loops with target functions
- 17:24Discussion on challenges: distribution, versioning, and target functions
- 19:21Roadmap and future outlook for agent automation
- 20:09Q&A session
related talks
29mLangfuse, OpenLIT, and Phoenix: Observability for the GenAI Era
29mStop Prompting, Start Composing: Agent Skills in Practice
27mSkills are the New SDKs
27mAgentic Coding with Discipline & Skill: Practical Techniques and Tips
26mThe State of GraphQL Agent Skills
29mSupercharging Axum Development with Claude Code
21mBuilding Great Agent Skills: The Missing Manual
38mHUMAN.md: Are we losing our skills in the SKILL.md era?