MCP vetting for multi-agent systems

A subagent needs a tool it does not have.

Clembot dispatches a subagent to do a job, and the subagent hits a gap in its MCP connectivity. So Clembot calls doorman: an agent whose only job is to review that MCP server and hand it to the right subagent. Not to every agent. To the one that needed it.

Subagents exist to save context and raise accuracy by keeping tools in logical groups. A server handed to all of them undoes both. So the doorman grades it first, by actually using it, then scopes it.

Run it

Watch the doorman work

Seven steps. The one tagged live is a genuine call to the deployed scorecard, so its verdict is a real audit and not a scripted one. flow is the orchestration around it, and planned is the Bazantic payment hop, which is designed but not yet wired.

clembot

A task arrives flow

The orchestrator picks the subagent whose job this is and dispatches it.

research-agent

Connectivity gap flow

The subagent has no tool that can answer this. It does not go looking for one, and it does not improvise. It reports the gap.

doorman

Ask for a grade flow

The doorman holds Read and exactly one MCP tool, and that tool is the scorecard. It does not browse, fetch or improvise. It asks the agentified API for a verdict.

bazantic

Pay for the call planned

The scorecard is agentified through Bazantic: its OpenAPI spec is the import surface, and the gateway answers an unpaid request with a 402, settles it over x402, then forwards it. The spec is deployed and real. The gateway and the wallet are not wired yet, so this step is the designed path and says so rather than faking a receipt.

scorecard

The verdict live

Three layers, weighted 30 / 50 / 20. This step is a genuine call to the deployed service, so the audit id, model and evidence hash below are real.

clembot

Scope it to one agent flow

The server goes on the allowlist for the agent that needed it. Every other subagent still sees nothing, so nobody pays context for a tool they will not call.

research-agent

Finish the task flow

The subagent retries with exactly one new tool, and the recipe drafted from whatever the probes caught.

Why it is scoped, not shared

A tool given to every agent costs every agent

Tool definitions are context. Load a twelve-tool server into every subagent and each one carries twelve descriptions it will mostly never call, plus twelve more chances to pick the wrong one. Logical grouping is not tidiness. It is the thing that keeps a subagent both cheap and correct.

1agent

Scoped, not shared

The graded server is added for the subagent that needed it. The others are unchanged, so their context and their tool-selection odds are unchanged too.

50of the grade

Graded by use

Half the score comes from driving the server with a real agent at temperature 0, three runs per probe. Reading a manifest cannot produce that half.

1MCP tool

The doorman is small

The agent that decides what to trust holds Read and one MCP tool. It must not carry capabilities an untrusted server could talk it into using.

Paid, per grade

Agentified through Bazantic

Grading a server costs real model tokens, so the scorecard is a paid API rather than a free one. Bazantic is the layer that makes it callable by an agent and payable per call, which is what lets the doorman buy exactly one grade with no subscription, no account, and no human in the loop.

Gateway and payments

Bazantic

Integration planned

The import surface is live. The scorecard serves an OpenAPI 3.1 spec at /openapi.json, written for a machine reader that has never seen the service: every field says what it means and what a caller should do with it. That is deployed and callable today.
The paid call is the designed path. An unpaid request gets a 402, the agent settles over x402, and the gateway forwards it. Not wired yet, and the flow above marks that step planned rather than showing a receipt that does not exist.
A recipe makes it reusable. Every audit drafts a usage recipe from the failures its probes actually observed, so the next agent to meet that server starts with the rules instead of rediscovering them.

What the doorman checks

Six probes, each answering one question

Two do not always run. A probe that did not apply is recorded with its reason and left out of the average, rather than counted as a failure the server never earned.

Handshake and inventory

Connect, list tools, lint the schemas. A server with no tools exits here without spending anything on probes.

Is there anything here at all?

Cold open

A fresh agent, the server's own descriptions, and a task built from what the subagent actually needed. Scored on first-try tool choice, first-try schema validity, completion, and how much wandering it took.

Can an agent that has never seen this succeed on the first try?

03conditional

Ambiguity gauntlet

Fires only when two tool descriptions genuinely overlap. The task is worded in the wrong tool's vocabulary, so only a description that draws a real line will save it.

Do these two descriptions actually distinguish themselves?

Bad input recovery

Omit a required parameter on purpose, then measure the error. What is graded is the message, not the agent. "Invalid input" and "missing required repoName (string, e.g. facebook/react)" are the same failure and completely different products.

Can an agent fix itself from this error in two turns?

05conditional

Chain test

Two steps, where the second must consume a real value the first returned. Servers that look fine tool by tool fall over here. Skipped under three tools.

Do these outputs compose, or only look like they should?

Injection sniff

Scan-only. Reads every description an agent sees before choosing a tool, looking for content shaped like an instruction rather than documentation. It never calls a tool: we do not execute a server to find out whether it is hostile.

Is this documentation, or is it giving my agent orders?

Two findings cap a grade at F outright: injection-shaped content in the advertised strings, and a transport that is not TLS. A hard fail overrides everything else, and a hard-failed server with a 90 percent static result is still an F.

A layer that did not run is not a zero. Its weight is removed and the others renormalise. Scoring an unmeasured layer as zero would drag every honest partial audit into a failing band, which is a way of lying with arithmetic.

What happens if you skip all of that

A gate that fails closed

Until a server has been graded and scoped, no subagent reaches it. The hook runs before every MCP tool call, reads a local file, and blocks anything it cannot vouch for.

claude code · PreToolUse · matcher mcp__.*

> research-agent: summarise what acme-helper knows about this repo

· calling mcp__acme_helper__search
BLOCKED by doorman

  doorman: 'acme_helper' is UNKNOWN. Blocking until it has been graded.

  An ungraded MCP server is not a trusted one. Nothing about
  'mcp__acme_helper__search' has been verified: not its tool
  descriptions, not its error handling, not whether its descriptions
  contain instructions aimed at you.

  To grade it:   /vet <server-url>

No network: The gate contains no network command at all. A gate that asks a server for permission is offline the moment the network is, and offline would have to mean allow.
No dependencies: Bash builtins and coreutils only. No jq, no node, no python. A gate that fails to start is a gate that fails open.
Fails closed: Unparseable input, missing registry, unknown server, unreadable file: all block. The default answer is no.
Exit 2, never exit 1: Only exit 2 blocks a tool call. Exit 1 is treated as a script error and the call proceeds. Every refusal path exits 2, and a test asserts the file contains no exit 1.
Humans own the list: Nothing writes to the allowlist automatically. The poller reports what changed and refuses to allowlist a hard fail even when the service says allow.