Skip to content

improvement(file-search): only start the dispatcher when there is dispatch work - #8714

Closed
waleedlatif1 wants to merge 1 commit into
stagingfrom
improvement/file-search-dispatch-gate
Closed

waleedlatif1 wants to merge 1 commit into
stagingfrom
improvement/file-search-dispatch-gate

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

  • The per-minute workspace-file-search-dispatch cron started a Trigger.dev run every tick even when nothing needed dispatching. enqueueWorkspaceFileSearchDispatch now checks first (same pattern as /api/cron/knowledge-projection) and starts a run only if one of the dispatcher's own phases would act:
    • the backfill cursor is missing or a page is due — isBackfillPageDue, now shared with seedBackfillPage
    • a build has expired — fileSearchBuildExpired, now shared with cleanupFileSearchBuilds
    • a claim is stale — staleClaim, now shared with reapStaleClaims
    • a workspace is queued for claiming
  • Fails open: a check error logs a warning and dispatches anyway. Covers both the Trigger.dev and inline backends; the cron route returns 200 triggered: false on an idle tick (202 when it dispatches)
  • enqueue-dispatch.ts now uses import type for the task instead of a runtime import() used only for its type
  • Reconcile interval is unchanged (hourly): environments built with db:push don't install the mark-pending trigger and rely on that walk

Type of Change

  • Improvement

Testing

  • Integration on real Postgres (dispatcher.integration.ts): each work leg reports work (queued workspace, expired build, claim past stale window, claim past handoff deadline, backfill never completed / due, missing cursor); the idle case reports no work AND prepareWorkspaceFileSearchDispatch does nothing on the same data; EXPLAIN asserts no Seq Scan
  • Unit (enqueue-dispatch.test.ts, both backends): no work → no run; check error → run
  • Each guard shown red by reverting it; bun run lint, bun run type-check, bun run check:audits — green; 15 files / 191 tests in lib/workspace-files/search + cron routes

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing (new tests pass the test-audit authoring gate)
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel

vercel Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
docs Skipped Skipped Oct 7, 2026 1:18am UTC

Request Review

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 6 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Turn on auto-fix | Re-trigger cubic

@greptile-apps

greptile-apps Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 4/5

[Medium risk] File search dispatcher now skips runs when there is no work.

Fix the explicit testing-rule violations before merging; the timeout and plan-test improvements are otherwise non-blocking.

Findings

  1. P2 The check can stall dispatch ▶
  2. P2 The test checks another index ▶
  3. P2 The mock loads real imports ▶
  4. P2 The test checks a spy ▶

Summary

The PR skips idle file-search dispatcher runs and returns HTTP 200 when no run starts. It shares the backfill, expired-build, and stale-claim checks with the dispatcher.

  • Probe errors still allow dispatch.
  • Add short database timeouts to the new check.
  • Make the plan test match production's index.
  • Fix two explicit testing-rule violations before merging.
Diagram
%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Authenticated cron request] --> B[Check dispatch work]
  B -->|No work| C[Return 200 without a run]
  B -->|Work found| D[Choose backend]
  B -->|Check throws| D
  D -->|Trigger.dev enabled| E[Enqueue dispatcher]
  D -->|Inline backend| F[Start detached dispatcher]
  E --> G[Return 202]
  F --> G
Loading

Reviews (1) · Last reviewed commit: "improvement(file-search): only start the..." · Reviewed by Greptile

Comment on lines +475 to +480
export async function hasWorkspaceFileSearchDispatchWork(now: Date): Promise<boolean> {
const [cursor] = await db
.select({ completedAt: workspaceFileSearchBackfill.completedAt })
.from(workspaceFileSearchBackfill)
.where(eq(workspaceFileSearchBackfill.id, BACKFILL_CURSOR_ID))
.limit(1)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 The check can stall dispatch

The new database reads run before enqueueing without setting query or lock timeouts. If a table lock or slow query holds either read past the cron request's 60-second limit, hasDispatchWork never reaches its catch and no dispatcher starts. Run the check in a short, timeout-bound transaction so a stalled check can fall back to dispatching.

Comment on lines +643 to +645
expect(plan).toContain('workspace_file_search_build_cleanup_idx')
expect(plan).toMatch(/using workspace_file_search_revision_workspace_id_dispatched_at_idx/)
expect(plan).not.toMatch(/Seq Scan/)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 The test checks another index

The new plan test expects a fixture index on (workspace_id, dispatched_at), but production's workspace_file_search_revision_active_idx uses (dispatched_at, workspace_id). It therefore does not check the production index promised by the test. Create that index in the fixture and assert its name. Also distinguish forced index use from the plan PostgreSQL would normally choose.

Comment on lines +21 to 23
vi.mock('@/lib/workspace-files/search/index-state', async (importOriginal) => ({
...(await importOriginal<typeof import('@/lib/workspace-files/search/index-state')>()),
cleanupFileSearchBuilds: vi.fn().mockResolvedValue(0),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 The mock loads real imports

The new partial mock uses importOriginal. The testing guide explicitly says never to use vi.importActual() or importOriginal to build a partial mock; use the central mock instead. This loads the real module and its imports just to retain fileSearchBuildExpired. Keep the real module unmocked for this integration test, or use a faithful shared mock. This repository requirement must be satisfied before merging.

Context Used: CLAUDE.md (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

triggered: true,
backend: 'trigger-dev',
})
expect(mockTrigger).toHaveBeenCalledTimes(1)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 The test checks a spy

The new failure-path test proves dispatch by asserting that mockTrigger was called. CLAUDE.md explicitly forbids tests that assert mock calls. Check the actual enqueue boundary and its accepted result instead, so the test catches a failed handoff rather than just a call to a spy. This repository requirement must be satisfied before merging.

Context Used: CLAUDE.md (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

Folded into #8713 so the outbox and file-search gates can share one scheduled-pass helper (no change to the file-search gate itself).

This branch was previously deployed

1 inactive deployment
Preview — faa32e48 Deployed Oct 7, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant