Skip to content

mcp: bound tool_name metric label to registered tools - #4912

Open
a-palamarchuk wants to merge 1 commit into
redpanda-data:mainfrom
a-palamarchuk:mcp-metrics-bounded-tool-names
Open

a-palamarchuk wants to merge 1 commit into
redpanda-data:mainfrom
a-palamarchuk:mcp-metrics-bounded-tool-names

Conversation

@a-palamarchuk

Copy link
Copy Markdown

What

The MCP server's metrics middleware labels mcp_tool_invocations_total,
mcp_tool_execution_duration_ns and mcp_tool_concurrent_executions with
tool_name, taken verbatim from the tools/call request. Tool names are chosen
by the client, so every call naming a tool that doesn't exist creates new series
in all three metrics. That can happen through a typo, a tool name hallucinated by
an LLM, or a deliberate scan, and the series count grows without bound
(CONTRIBUTING §1.2.1: metrics should "avoid excessive cardinality").

This change has the resources wrapper record the tool names it registers, and the
metrics middleware reports calls to any other name under tool_name="unknown" (the
value it already used as a fallback).

Reproduce

redpanda-connect mcp-server --address … --observability-address … with one
processor tool (upper) and prometheus metrics. Then make 50 tools/call
requests with random nonexistent names and one call to upper.

  • Before: /metrics has 51 distinct tool_name series per metric (one per random
    name, plus upper).
  • After: exactly two, tool_name="unknown" (counter = 50) and tool_name="upper".

Clients still receive the same -32602 unknown tool JSON-RPC error; only the
metric labelling changes.

Design notes

  • The go-sdk server has no public "is this tool registered" lookup, and inferring
    "unknown tool" from the handler's error would be fragile and would happen too
    late for the concurrency gauge, which is incremented before the call. So
    ResourcesWrapper, the only place tools are registered, now routes every
    registration through a small addTool helper and exposes HasTool.
  • HasTool is guarded by an RWMutex. Today all tools are registered before the
    server starts serving, but the lock keeps this race-free if tools are ever added
    while serving.
  • A real tool literally named unknown would share its series with unregistered
    calls. That label value was already this code's fallback, so I kept it rather
    than changing what existing dashboards see.

Tests

  • New TestToolMetricsCollapseUnknownToolNames (internal/mcp/metrics, which had
    no tests before) builds resources with the prometheus exporter on an
    in-memory mux, sends tools/call requests through the middleware for one
    registered and two unregistered names, then scrapes /metrics. It asserts the
    registered tool has its own series, the two unregistered calls are counted
    under tool_name="unknown", and no requested made-up name appears anywhere.
    With the new check disabled it fails and prints the scrape showing the
    per-name series.
  • New TestResourcesWrapperHasTool (internal/mcp/tools) covers the derived cache
    tool names (get-/set-), an MCP-enabled processor, a processor with MCP
    disabled, and an unregistered name.
  • go test -race -shuffle=on ./internal/mcp/..., the in-process MCP integration
    tests (-run '^TestIntegrationMCP') and ./internal/cli/ pass;
    golangci-lint run and golangci-lint fmt --diff are clean.

Tool names in tools/call requests are chosen by the client, and the MCP
metrics middleware used them verbatim as the tool_name label. Every call
naming a tool that doesn't exist (a typo, a name hallucinated by an LLM, or
a deliberate scan) created new series for the invocation counter, duration
timer and concurrency gauge, so the series count grew without bound.

Track the tool names registered by the resources wrapper and report calls to
any other name under tool_name="unknown".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant