API Drift: A Hidden Crisis in MCP Servers
summary
Claims that MCP servers suffer spec/drift get concrete validation: a team scans 200 MCP servers and builds an open source fuzzer called Specmatic MCP auto test that interrogates the MCP JSON‑RPC tool list. Running it against Postman’s MCP exercises 33 tools: 27 passed, 6 failed and enabling resiliency tests on one endpoint yields create workspace: 77 tests, 40 passed, 37 failed. Failures stem from missing JSON‑Schema constraints, absent oneOf semantics, and implementations accepting wrong or undocumented parameters.
AI-written summary · about
chapters (7)
- 0:00API drift in MCP servers
- 2:32MCP server communication
- 8:10Live demo of MCP testing
- 12:25Automated MCP testing
- 21:13Resiliency and schema issues
- 27:20JSON schema problems in MCP
- 29:48Summary and tools
related talks
21mMCP at 1. The state of the protocol you love (or hate)
23mThe Design of MCP Servers
44mSupercharging DevOps with MCP (Without Opening a Security Hole)
51mThe MCP Megalith: Surviving the 1,000-Tool Explosion and Auth Fragmentation
27mYour MCP Server is Bad (and You Should Feel Bad)
30mLet's make an MCP Server!
17mBuilding Production-Ready MCP Servers: From AI Agents to Secure Enterprise APIs.
39mHow to connect your systems to AI Tools with MCP Servers