Tilburg Researchers Map 73 Ways MCP Servers Silently Break AI Tools
Researchers analyzed 837 GitHub fault threads across 473 MCP servers to map the 73 distinct ways your AI tool integrations can silently break at runtime
Takeaways
- First empirical taxonomy: Researchers analyzed 837 real GitHub fault threads from 473 MCP server repos to produce the first structured map of MCP runtime failures.
- 73 fault types across 11 categories: The taxonomy covers tool invocation, schema enforcement, session lifecycle, security, transport, and more -- validated by 55 active MCP developers.
- Tool faults dominate: Tool execution, data/schema mismatches, and state/config issues account for 57.8% of all confirmed faults -- not transport or protocol handshake errors.
- Silent failures are the hardest problem: Many MCP servers return well-formed JSON-RPC success responses while silently violating the protocol contract, giving the LLM no error signal.
- Security is underreported: Security faults appear rarely in public GitHub data but are consistently rated high-severity by developers, likely because they surface in private audits.
- Replication package available: Datasets, coding records, and scripts are publicly available on Figshare for follow-on research and tooling.
MCP servers are quietly becoming load-bearing infrastructure for AI applications. When your LLM calls a tool -- querying a database, running code, hitting an API -- it goes through an MCP server. And yet, until now, nobody had systematically catalogued how those servers actually fail in production. A new paper from researchers at Tilburg University, the University of Luxembourg, and the University of Naples Federico II changes that, delivering the first empirical taxonomy of runtime faults in MCP servers.
The M×N Problem MCP Was Built to Solve
Before MCP, connecting AI models to external tools meant writing custom glue code for every model-tool pair. Every AI app needed custom code for every tool, meaning M apps times N tools equals M×N unique integrations to build and maintain. MCP solves this with a unified client-server protocol: MCP enables LLMs to interact with external tools and data sources via a standardized protocol. Each server exposes three primitives -- Tools (callable operations), Resources (contextual data), and Prompts (reusable instruction templates) -- and communicates over JSON-RPC 2.0.

The problem is that standardizing the interface does not standardize correctness. Rapid adoption in tool-augmented AI workflows has introduced new reliability challenges, such as configuration parameters that are accepted but not enforced at runtime, leading to unintended default behavior. A server can compile cleanly, start up without errors, and still violate the protocol contract mid-session -- and the LLM on the other end has no idea.
What the Researchers Actually Did
The team did not rely on synthetic benchmarks or theoretical models. They went straight to the source: GitHub issue threads from real MCP server repositories. The pipeline involved several filtering stages to separate genuine MCP runtime faults from generic programming bugs and deployment issues.
- Repository collection: They compiled a dataset from two public sources -- MCP Market and Awesome MCP Servers -- yielding 2,112 entries. After deduplication and validation, the dataset contained 1,774 active, publicly accessible repositories.
- Quality filtering: Repositories were required to have at least 5 contributors, open issues, 100+ commits, and 10+ stars and forks, reducing the set to 473 actively maintained repositories.
- Issue mining: The mining stage produced 78,267 issues and 49,155 pull requests. After merging each issue with its linked pull requests and associated artifacts, they obtained 78,267 consolidated issue-centered threads.