[richiemcilroy]

Benchmarks for Cap's GPUI patch stack, run per platform with base and head interleaved on the same machine, plus a gate that fails if a patch changes what gets drawn.

flowchart LR
  P[PR touches apps/desktop-gpui/patches] --> W1[gpui-bench.yml<br/>macOS · Windows · Linux]
  P --> W2[gpui-app-bench.yml<br/>macOS]
  W1 --> B[build base + head<br/>from one checkout path]
  B --> M[interleaved Criterion rounds<br/>+ Window::draw per frame]
  B --> D[scene digests<br/>frame by frame]
  B --> T[gpui / platform crate tests]
  W2 --> A[real cap-gpui, base + head<br/>editor timeline bench on the<br/>reference recording]

Loading

Frame benchmarks (apps/desktop-gpui/gpui-benches)

• 14 workloads modelled on Cap windows, each drawing real frames through Window::draw with the platform text system; on macOS each frame is also submitted to Metal through the headless renderer:
  • cards_{64,256,1024}: bordered, shadowed cards with two labels each.
  • text_rows_static_{50,200} and text_rows_changing_{50,200}: label rows, the second with a timestamp that changes every frame.
  • paragraphs_{12,48}: wrapped text in columns.
  • recordings_scroll: a 10 000-row uniform_list scrolling.
  • hover_cards: pointer moves across 256 hoverable cards.
  • cached_panels: 40 .cached() views, one of which changes per frame.
  • primitives_{1024,8192}: raw quads, shadows and paths.
• The crate lives in Cap. scripts/gpui-bench-build.sh copies it into the prepared zed checkout, so it builds against the exact patched gpui and Zed's lockfile.
• Each benchmark registers through a small runner (run in benches/frame.rs) rather than #[gpui::bench], so the headless renderer can be wrapped.

Measuring (scripts/gpui-bench-compare.py)

• Base and head are built from the same checkout path, swapped in turn. With identical sources, builds from two different paths differ by a few percent (paths end up in symbol hashes).
• Runs alternate base, head, head, base, … and pair each round's Criterion median; a change is called faster or slower only when every round agrees.
• A second table reports Window::draw alone, from GPUI's own frame report. On macOS, light frames are bound by the headless GPU handing back instance buffers, so wall time hides CPU work.

Scene equivalence (scripts/gpui-scene-digest.py)

• With GPUI_SCENE_DIGEST=<file>, the runner hashes every scene a benchmark submits before passing it to the real renderer. The hash covers all primitives in the order the renderer receives them, plus the pixels uploaded into every atlas tile a sprite shows.
• CI compares base and head at every (benchmark, frame) position both builds reached, and fails on any difference. It also fails when a benchmark is missing, draws fewer than 16 frames on either side, or (on macOS) digests nothing.
• Changing a single bit of every rasterized glyph makes 704 of 800 positions differ, and the gate fails.
• Only macOS has a headless renderer, so the step reports a skip elsewhere.

Real app (.github/workflows/gpui-app-bench.yml, scripts/gpui-app-bench.py)

• Builds cap-gpui against base and head, then runs CAP_GPUI_AUTO_TIMELINE_BENCH=1 (src/editor_bench.rs) on CapSoftware/cap-performance-fixtures' reference recording. Every run gets a fresh profile and a fresh copy of the project.
• Compares each scenario's median draw over 5 interleaved rounds. The editor decodes video while it plays, so means and tails swing with the decoder.

Profiling (.github/workflows/gpui-profile.yml)

• On pushes to gpui-profile/**, runs each benchmark under callgrind on Linux and uploads the annotated self and inclusive costs.

Who can trigger it

• The benchmark and test jobs run on pushes, workflow_dispatch, and pull requests from this repository. Fork pull requests don't start them.

Tests

• gpui-bench.yml also runs cargo test -p gpui --lib --tests and the platform crate's tests (gpui_macos, gpui_windows, or gpui_wgpu on Vulkan) against the head stack on all three platforms.

🤖 Generated with Claude Code

[sandchest-com]

Sandchest reviewed e9fad53: nothing exploitable found in this change.

_{Sandchest reviews every change for vulnerabilities an attacker could use · 2 credits · Review details}

[superagent-security]

Superagent found 1 security concern(s).

[superagent-security]

P2: Untrusted fork PRs can repeatedly queue multi-hour GPUI benchmark jobs by editing benchmark-only paths

New PR workflows trigger 120-150 min macOS/Windows/Linux benchmark jobs when only the workflow, its scripts, or gpui-benches crate change.

Gate self-file/benchmark-triggered jobs to internal PRs or a maintainer-approved label/environment; keep fork runs bounded.

AI prompt

Check if this security scanner issue is valid. If so, understand the root cause and fix it. If appropriate, update or add tests. Keep the change focused and preserve intended behavior.

<file name=".github/workflows/gpui-app-bench.yml">
<violation number="1" location=".github/workflows/gpui-app-bench.yml:15">
<priority>P2</priority>
<title>Untrusted fork PRs can repeatedly queue multi-hour GPUI benchmark jobs by editing benchmark-only paths</title>
<evidence>The new `pull_request` trigger lists the workflow itself and `scripts/gpui-app-bench.py` as paths (lines 11-15), so a fork PR that changes nothing but this workflow or its runner script starts the `editor` job on `blacksmith-6vcpu-macos-15` with `timeout-minutes: 150` (lines 36-39). That job runs `node scripts/setup.js`, prepares head and base GPUI checkouts, builds `cap-gpui` twice (lines 107-112), and then runs the real editor timeline benchmark for `--rounds 5` interleaved A/B rounds over the reference recording (lines 126-139). The new `.github/workflows/gpui-bench.yml` has the same shape: paths at lines 12-17 include `apps/desktop-gpui/gpui-benches/**`, `scripts/gpui-bench-*`, and the workflow file itself, and each matching PR push queues three `bench` jobs (150-minute timeout) plus three `test` jobs (120-minute timeout) across macOS, Windows, and Linux. Neither workflow carries a `github.event.pull_request.head.repo.full_name == github.repository` condition, a required label, or an `environment:` approval gate; `concurrency` with `cancel-in-progress: true` only cancels overlapping runs and does not stop sequential pushes from each consuming a full run. The 150-minute and 120-minute timeouts are the worst-case per-run cost, and each run performs two full Cargo builds plus the interleaved measurement workload, so a fork contributor can repeatedly trigger expensive shared-runner work with a trivial edit to benchmark-only files.</evidence>
<recommendation>Restrict the self-file/benchmark-only triggers to internal pull requests, e.g. add `if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository` to the `bench`/`test`/`editor` jobs (or drop the workflow/script/crate paths from `pull_request.paths` and run them only from `workflow_dispatch` / `push` on trusted branches). For fork PRs, keep the benchmark suite reachable only through a maintainer-applied label or an `environment:` approval, and keep a short bounded validation run for workflow-only changes.</recommendation>
</violation>
</file>

[richiemcilroy]

Fixed in e3a8c7f: the bench and test jobs in gpui-bench.yml and the editor job in gpui-app-bench.yml now run only when github.event.pull_request.head.repo.full_name == github.repository (or on push/workflow_dispatch). Fork PRs no longer start them; maintainers can still measure a fork's patch with workflow_dispatch.

[superagent-security]

Correctly fixed: gating the long-running bench, test, and editor jobs to internal PRs via github.event.pull_request.head.repo.full_name == github.repository (and push/workflow_dispatch) prevents untrusted forks from automatically queuing expensive benchmark runs, while preserving maintainer flexibility. Dismissing.

[capy-ai]

Capy found 2 potential issues (2 medium).

View 1 other finding in Capy.

[capy-ai]

Scene equivalence succeeds when scene coverage is missing

Medium Severity

On macOS, the patched current_headless_renderer() returns None when Metal initialization fails, so a benchmark can exit successfully without producing any digests. If only head encounters that failure, this branch skips the gate despite base having rendered scenes. Partial loss is also ignored because only intersecting frame keys are compared: head can stop submitting frames after an unchanged prefix and still report identical scenes. Restrict skipping to explicitly unsupported platforms and require expected benchmark/frame coverage before comparing hashes.

[richiemcilroy]

Fixed in e3a8c7f. An empty digest on macOS now fails instead of skipping (skipping only happens when neither side digests anything on a non-macOS runner). Every benchmark must draw at least 16 frames on both sides, and every (benchmark, frame) position both builds reached must be present and equal, so a head that stops submitting a workload fails. Positions beyond what both reached aren't compared, because a faster build legitimately draws more frames in Criterion's fixed time.

[greptile-apps]

Missing frames pass the gate

If the head stops submitting a workload or submits fewer frames, the comparison drops those positions because it checks only shared keys. If either side submits nothing, it skips entirely. A patch that stops drawing can therefore pass the scene check. Require matching benchmark/frame keys, and allow empty results only on platforms explicitly known to lack a renderer.

Prompt To Fix With AI

This is a comment left during a code review.
Path: scripts/gpui-scene-digest.py
Line: 70-74

Comment:
**Missing frames pass the gate**

If the head stops submitting a workload or submits fewer frames, the comparison drops those positions because it checks only shared keys. If either side submits nothing, it skips entirely. A patch that stops drawing can therefore pass the scene check. Require matching benchmark/frame keys, and allow empty results only on platforms explicitly known to lack a renderer.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

[richiemcilroy]

Same fix as above (e3a8c7f): a missing benchmark or one with fewer than 16 frames on either side fails, and every frame position both builds reached must be present and identical.

[greptile-apps]

Relative output paths break

The helper creates out_dir from the caller's directory, then changes to zed_dir before opening cargo-messages.json and copying the executable. Calling scripts/gpui-bench-build.sh ../zed-cap ./bench-out therefore writes to a different location or fails before Cargo runs. Resolve out_dir to an absolute path before changing directories.

Prompt To Fix With AI

This is a comment left during a code review.
Path: scripts/gpui-bench-build.sh
Line: 36-38

Comment:
**Relative output paths break**

The helper creates `out_dir` from the caller's directory, then changes to `zed_dir` before opening `cargo-messages.json` and copying the executable. Calling `scripts/gpui-bench-build.sh ../zed-cap ./bench-out` therefore writes to a different location or fails before Cargo runs. Resolve `out_dir` to an absolute path before changing directories.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

[richiemcilroy]

Fixed in e9fad53: out_dir is resolved to an absolute path right after it's created, before the script changes directory.

[greptile-apps]

Comments repeat the code

This comment restates what SceneDigestRenderer does. The comments above cards and primitives do the same, and the pattern appears elsewhere. AGENTS.md requires comments to explain non-obvious context rather than narrate the code. Remove these descriptions while keeping comments that explain choices such as the measured build-path effects. This repository requirement must be satisfied before merging.

Context Used: AGENTS.md (source)

Prompt To Fix With AI

This is a comment left during a code review.
Path: apps/desktop-gpui/gpui-benches/benches/frame.rs
Line: 585

Comment:
**Comments repeat the code**

This comment restates what `SceneDigestRenderer` does. The comments above `cards` and `primitives` do the same, and the pattern appears elsewhere. `AGENTS.md` requires comments to explain non-obvious context rather than narrate the code. Remove these descriptions while keeping comments that explain choices such as the measured build-path effects. This repository requirement must be satisfied before merging.

**Context Used:** AGENTS.md ([source](https://github.com/capsoftware/cap/blob/main/AGENTS.md))

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[richiemcilroy]

Fixed in e9fad53 (and in zed-perf.patch on #3887): removed the comments that describe the code, keeping only ones that record non-obvious context (e.g. why base and head build from one path, why tile pixels must be hashed, why a tile id can be reused).

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated
cap-status	Ignored	Preview	Nov 6, 2026 2:07pm UTC

[capy-ai]

Capy found no issues.

View 1 other finding in Capy.

[capy-ai]

Capy found no issues.

View 1 other finding in Capy.