Skip to content

Migrate SD performance test from nats to nats-tls - #377

Merged
kart2bc merged 2 commits into
developfrom
fix/sd-perf-nats-tls
Nov 4, 2026
Merged

kart2bc merged 2 commits into
developfrom
fix/sd-perf-nats-tls

Conversation

@geofffranks

@geofffranks geofffranks commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Summary

nats-release removed the plain nats job after v57.23.0 (cloudfoundry/nats-release#102), so the run-service-discovery-performance-tests CI job fails to deploy with "Job 'nats' not found in release 'nats'" against nats-release 57.26.0. TNZ-189085

  • The performance-test-sd job consumes the nats-tls link. It renders the NATS CA, client cert chain and private key from the new nats.cert_chain and nats.private_key properties.
  • nats_url uses the link's nats.hostname (the SAN-covered name, the same way service-discovery-controller does it) and falls back to the instance address.
  • The Go performance test dials tls:// with RootCAs and ClientCert for mutual TLS. Monitoring stays on plain HTTP via the link's monitor_port.
  • The test fixture manifest deploys nats-tls on port 4222 with external and client TLS. It issues client certs for the errand and SDC, plus a server cert that carries the NATS IP SAN.

Companion CI change (merge first): cloudfoundry/wg-app-platform-runtime-ci fix/sd-perf-nats-tls @ d8b213ce updates cf-networking-release/manifests/performance-sd.yml to deploy nats-tls.

Backward Compatibility

Breaking Change? Yes, for the performance-test-sd errand job only.

  • The optional nats link was removed, and the nats-tls link is now required.
  • The new properties nats.cert_chain and nats.private_key have no defaults and must be set to a client cert signed by the nats-tls client.tls CA.
  • A deployment that still provides plain nats will fail at deploy time. Migrate it to the nats-tls job as in d8b213ce.
  • No production job changes. service-discovery-controller already supported nats-tls.

🤖 Generated with Claude Code

nats-release removed the plain `nats` job after v57.23.0, so the
service-discovery performance test deployment failed with "Job 'nats'
not found in release 'nats'". TNZ-189085

- performance-test-sd job consumes the `nats-tls` link and renders the
  NATS CA, client cert chain and private key (new nats.cert_chain /
  nats.private_key properties).
- The Go performance test dials tls:// with RootCAs and ClientCert for
  mutual TLS; the monitoring port still comes from the link.
- The test fixture manifest deploys nats-tls on port 4222 with external
  and client TLS, and issues client certs for the errand and SDC plus a
  server cert carrying the NATS IP SAN.

ai-assisted=yes
AI-Tool-Used: Claude, Codex
AI-Tool-Use-Level: High
AI-Code-Category: Production
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kart2bc

kart2bc commented Nov 4, 2026

Copy link
Copy Markdown
Contributor
Medium

1. nats_url dials the link instance address, which the server cert may not cover — jobs/performance-test-sd/templates/config.json.erb:5 — [correctness]
   Impact: The Go test now builds tls://<address>:<port>, and nats.go verifies the server cert against that host. The server cert SANs are ((nats_ip)), nats.service.cf.internal and *.nats.service.cf.internal. If the director uses DNS addresses for links, instances[0].address is a name like q-s0.nats.default.<net>.bosh. The handshake then fails with an x509 hostname mismatch at runtime, not at deploy time. I could not check the director setting offline, so I can't say whether this fires in CI.
   Evidence: The previous plain-NATS connection never verified a host name, so nothing exercised this before. service-discovery-controller/templates/config.json.erb dials the link's nats.hostname instead. That is the property meant to hold the SAN-covered name (nats-tls/spec:59-61). Both manifests set it (hostname: ((nats_ip)) here, ((NATS_IP)) in CI), and the PR adds that IP to the server cert SANs.
   Fix: Render nats_url from link('nats-tls').p('nats.hostname', link('nats-tls').instances[0].address), the same way the SDC does. Alternatively, set ServerName explicitly in the Go TLS options.

Low

2. PR body leaves Summary and Backward Compatibility unfilled — jobs/performance-test-sd/spec:15 — [requirements]
   Impact: The job's contract changed. The nats link was removed, nats-tls is now required, and two new properties have no defaults. Anyone deploying the job with a plain nats provider gets a deploy failure with no migration note. The text above the template is a good description, but "Breaking Change? Yes/No" is unanswered.
   Evidence: gh pr view 377 body. The spec diff drops - name: nats / type: nats / optional: true.
   Fix: Fill in both template sections. State that the link and properties are now required and reference wg-app-platform-runtime-ci d8b213ce.

The perf test now verifies the NATS server certificate against the host
it dials. instances[0].address can be a BOSH DNS name that the server
cert SANs do not cover, so prefer the link's nats.hostname (the
SAN-covered name, as service-discovery-controller does) and fall back
to the instance address. TNZ-189085

ai-assisted=yes
AI-Tool-Used: Claude
AI-Tool-Use-Level: High
AI-Code-Category: Production
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@geofffranks

Copy link
Copy Markdown
Contributor Author

Feedback addressed 🤞

@github-project-automation github-project-automation Bot moved this from Inbox to Pending Merge | Prioritized in Application Runtime Platform Working Group Nov 4, 2026
@kart2bc
kart2bc merged commit 0375246 into develop Nov 4, 2026
9 checks passed
@kart2bc
kart2bc deleted the fix/sd-perf-nats-tls branch November 4, 2026 18:08
@github-project-automation github-project-automation Bot moved this from Pending Merge | Prioritized to Done in Application Runtime Platform Working Group Nov 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Development

Successfully merging this pull request may close these issues.

2 participants