mcp: bound tool_name metric label to registered tools - #4912
Open
a-palamarchuk wants to merge 1 commit into
Open
a-palamarchuk wants to merge 1 commit into
a-palamarchuk wants to merge 1 commit into
Conversation
Tool names in tools/call requests are chosen by the client, and the MCP metrics middleware used them verbatim as the tool_name label. Every call naming a tool that doesn't exist (a typo, a name hallucinated by an LLM, or a deliberate scan) created new series for the invocation counter, duration timer and concurrency gauge, so the series count grew without bound. Track the tool names registered by the resources wrapper and report calls to any other name under tool_name="unknown". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The MCP server's metrics middleware labels
mcp_tool_invocations_total,mcp_tool_execution_duration_nsandmcp_tool_concurrent_executionswithtool_name, taken verbatim from thetools/callrequest. Tool names are chosenby the client, so every call naming a tool that doesn't exist creates new series
in all three metrics. That can happen through a typo, a tool name hallucinated by
an LLM, or a deliberate scan, and the series count grows without bound
(CONTRIBUTING §1.2.1: metrics should "avoid excessive cardinality").
This change has the resources wrapper record the tool names it registers, and the
metrics middleware reports calls to any other name under
tool_name="unknown"(thevalue it already used as a fallback).
Reproduce
redpanda-connect mcp-server --address … --observability-address …with oneprocessor tool (
upper) andprometheusmetrics. Then make 50tools/callrequests with random nonexistent names and one call to
upper./metricshas 51 distincttool_nameseries per metric (one per randomname, plus
upper).tool_name="unknown"(counter = 50) andtool_name="upper".Clients still receive the same
-32602 unknown toolJSON-RPC error; only themetric labelling changes.
Design notes
"unknown tool" from the handler's error would be fragile and would happen too
late for the concurrency gauge, which is incremented before the call. So
ResourcesWrapper, the only place tools are registered, now routes everyregistration through a small
addToolhelper and exposesHasTool.HasToolis guarded by anRWMutex. Today all tools are registered before theserver starts serving, but the lock keeps this race-free if tools are ever added
while serving.
unknownwould share its series with unregisteredcalls. That label value was already this code's fallback, so I kept it rather
than changing what existing dashboards see.
Tests
TestToolMetricsCollapseUnknownToolNames(internal/mcp/metrics, which hadno tests before) builds resources with the
prometheusexporter on anin-memory mux, sends
tools/callrequests through the middleware for oneregistered and two unregistered names, then scrapes
/metrics. It asserts theregistered tool has its own series, the two unregistered calls are counted
under
tool_name="unknown", and no requested made-up name appears anywhere.With the new check disabled it fails and prints the scrape showing the
per-name series.
TestResourcesWrapperHasTool(internal/mcp/tools) covers the derived cachetool names (
get-/set-), an MCP-enabled processor, a processor with MCPdisabled, and an unregistered name.
go test -race -shuffle=on ./internal/mcp/..., the in-process MCP integrationtests (
-run '^TestIntegrationMCP') and./internal/cli/pass;golangci-lint runandgolangci-lint fmt --diffare clean.