[karolko9]

Summary

When the RPC gave no answer, /verify answered 500 internal_error. A seller could not tell an outage it may retry from a bug of ours. The MCP e2e met it on the slow public testnet RPC.

 POST /verify
   verify(payload, requirements)
-    throws anything                     -> 500 {"error":"internal_error"}
+    throws UpstreamTimeoutError         -> 503 {"error":"rpc_unavailable"}, Retry-After: 5
+    throws RpcUnavailableError          -> 503, the same
+    throws anything else                -> 500 {"error":"internal_error"}

Situation	Before	After
Upstream verify gives no answer within 10 s on every RPC_URLS entry	500 internal_error, http_error log	503 rpc_unavailable, Retry-After: 5, verify_rpc_unavailable warn log
Our ledger read fails on every URL (RpcUnavailableError)	500 internal_error	503 rpc_unavailable
Horizon hangs during upstream's call	500 internal_error	503 rpc_unavailable (it blames the RPC)
Two URLs: one hangs, the other answers upstream's unexpected_verify_error	500 or 503 by which URL came first	500 whatever the order: a 503 means every URL timed out
Any 503 above	no alarm (http-500 counts 500 only)	verify_rpc_unavailable_total +1, verify-rpc-unavailable event alarm

Unchanged: every verdict, valid or not, stays a 200. Upstream's unexpected_verify_error stays a 500, because it doesn't say whether the RPC failed. Other thrown errors stay a 500. /settle doesn't change. A seller on @x402/core still answers the buyer 402 for both 500 and 503: its client throws on any non-2xx answer without isValid.

Evidence

• Before: the MCP e2e's run 2 on 2026-11-04, from our facilitator's log, then the buyer saw payment_rejected:

  { "event": "rpc_exhausted", "component": "verify", "tried": [{ "reason": "timeout" }] }
  { "event": "http_error", "error": { "name": "UpstreamTimeoutError", "message": "upstream verify gave no answer within 10000 ms" } }

• New test, before the change: verify-route.test.ts "answers 503 rpc_unavailable when the RPC gave no answer", one case per error class: 3 failed.

  expected [503, "5", { error: "rpc_unavailable" }]
  received [500, null, { error: "internal_error" }]
  

• Order of URLs, new test before the change: upstream-failover.test.ts "decides the same whatever endpoint comes first" failed: promise rejected "UpstreamTimeoutError: a: upstream verify …" instead of resolving.

• After (c7fdd716): verify-route.test.ts 25 of 25, counting the counter per 503; upstream-failover.test.ts 18 of 18; the whole facilitator suite 1006 passed, 3 skipped; metrics-catalog 31 of 31; infra 337 of 337, with the new alarm checked against 0027's alarm list. Typecheck, pnpm lint and format:check pass.

• Whole facilitator suite: 975 passed, 1 failed, 3 skipped. The failure is settle.test.ts "never lets a waiting repeat end a claim reopened since it read the earlier one". That file doesn't touch /verify, and it passed 5 of 5 runs alone, so it looks like the timing flakiness task 0074 tracks.

Not run: the MCP e2e or a real RPC outage. The test fakes the thrown error.

Considered and rejected

• 200 with isValid: false and an RPC reason — it says the payment is invalid, which nothing checked.
• Keep the 500 — an outage outside our code reads as our bug, and a seller gets no hint to retry.
• 503 for unexpected_verify_error too — upstream also gives it for its own errors on a strange payload, so a retry could repeat forever.
• Counting 503 in http-500 — a lease or drain 503 on /settle is routine on every deploy.
• 503 when any URL hung — a payload that can never pass would get "retry".
• The same change for /settle's checks — its own PR, fix(lore-0046): answer rpc_unavailable when /settle's checks get no answer #159: /settle answers in x402 bodies (AD-6).

Merge Danger

Door: two-way

Revert the commit. A seller that already reads 503 then sees 500 again.

Blast Radius: API

A client or alert that counts 5xx sees the same total. One that reads the status code or error sees a new value. The http-500 alarm no longer fires on these hangs; the new verify-rpc-unavailable alarm does once infra is deployed. Retry-After: 5 is the owner's choice, not a measurement: one ledger.

🤖 Generated with Claude Code