improvement(cron): only start outbox and file-search passes when work is due, via a shared scheduled-pass helper by waleedlatif1 · Pull Request #8713 · simstudioai/sim

waleedlatif1 · 2026-10-07T01:13:09Z

Summary

The per-minute outbox and workspace-file-search crons started a Trigger.dev run every tick even with nothing to do. Both now check first, and all three gated scheduled passes share one helper.

Shared helper — lib/core/async-jobs/scheduled-pass.ts

triggerScheduledPass: the windowed tasks.trigger (idempotency key ${keyPrefix ?? taskId}:${window}, idempotencyKeyTTL: '5m', region) that outbox, file search, and knowledge projection each hand-rolled. Keys and options are byte-identical to before, so in-flight dedupe is unaffected across the deploy
startScheduledPass: due → inline or Trigger.dev → shared ScheduledPassResult ({ triggered, backend, jobId }), replacing KnowledgeProjectionSweepResult / the file-search result type
Each call site keeps its policy explicit by computing due itself: outbox maintenance window + fail-open, file search fail-open, knowledge projection fail-closed (unchanged). Outbox keeps its synchronous self-hosted branch local because the route reports its output. cleanup-stale-executions uses the job-queue path and is left as is

Outbox gate — starts a run only when:

a pending event is due — the claim phase's own readyEventTypesQuery, so gate and claimer share one predicate
a processing lease is stale — the reaper's predicate, now shared via isStuckProcessing / STUCK_PROCESSING_THRESHOLD_MS
or it's a maintenance tick (every 5th minute), so document recovery, background-work reaping, and pruning keep running on an idle queue

File-search dispatch gate — starts a run only when one of the dispatcher's own phases would act, each sharing its predicate with that phase: backfill page due (isBackfillPageDue), expired build (fileSearchBuildExpired), stale claim (staleClaim), or a queued workspace. Reconcile interval unchanged (hourly) since db:push environments rely on that walk. enqueue-dispatch.ts now uses import type for the task instead of a runtime import

Both gates fail open (a check error starts the run as before — billing, seat sync, invitations, and KB/file indexing ride these crons) and cover both the Trigger.dev and inline backends. Cron routes return 200 triggered: false on an idle tick; response bodies otherwise unchanged. Latency for due work stays ≤ 1 tick.

Supersedes #8714 (folded in so both gates can share the helper).

Type of Change

Improvement

Testing

Integration on real Postgres:
- outbox: future pending / fresh lease / completed / dead-letter → no work; event due exactly at now → work; lease exactly at the stale threshold → work; plan has no Seq Scan and stays under 100 buffers on a 100k-row backlog
- file search: each work leg reports work; the idle case reports no work and prepareWorkspaceFileSearchDispatch does nothing on the same data; no Seq Scan
Unit: each backend's own result for idle / maintenance / check-failure ticks; exact tasks.trigger call pinned for all three passes (keys, TTLs, maxDuration, region)
Every guard and the helper's branches shown red by reverting them (maintenance window, fail-open, window instant, key prefix, inline path, due)
bun run lint, bun run type-check, bun run check:audits, check:test-patterns — green; 26 files / 309 unit tests; 2 integration suites / 40 tests

Checklist

Code follows project style guidelines
Self-reviewed my changes
Tests added/updated and passing (new tests pass the test-audit authoring gate)
No new warnings introduced
I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

…ntenance is scheduled

vercel · 2026-10-07T01:13:12Z

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated
docs	Skipped		Oct 7, 2026 2:17am UTC

cubic-dev-ai

All reported issues were addressed across 9 files

_{Reply with feedback, questions, or to request a fix.

Fix all with cubic | Turn on auto-fix | Re-trigger cubic}

greptile-apps · 2026-10-07T01:17:48Z

Confidence Score: 5/5

[High risk] Refactors cron job scheduling with new shared helper and database queries.

The PR appears safe to merge; no new actionable issue was found.

What we checked:

Different tasks share a fake run: JSON.stringify([taskId, key]) keeps both strings separate. Repeated pairs still find the same saved run.

Summary

This PR skips idle outbox and file-search runs and shares scheduled-pass dispatch with knowledge projection.

Outbox keeps its maintenance ticks and runs when the work check fails.
File search checks the same conditions used by its dispatch phases.
Since the last review, only the test fake’s key encoding changed. No new issue was found.
The earlier findings are addressed: tests check returned results, cleanup runs without the partial mock, and the query-plan test uses production indexes without forcing the plan.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  Tick[Cron tick] --> Due{Pass due?}
  Due -->|No| Idle[Return triggered false]
  Due -->|Yes| Backend{Trigger.dev enabled?}
  Backend -->|Yes| Trigger[Start window-keyed run]
  Backend -->|No| Inline[Run inline]
  Trigger --> Accepted[Return accepted job ID]

_{Reviews (6) · Last reviewed commit: "test(testing): key the idempotent Trigge..." · Reviewed by Greptile}

…ty as scheduled

waleedlatif1 · 2026-10-07T01:27:36Z

@greptile

waleedlatif1 · 2026-10-07T01:27:37Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-10-07T01:27:41Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

No issues found across 9 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{Turn on auto-fix | Re-trigger cubic}

…patch work

…search, and knowledge projection

waleedlatif1 · 2026-10-07T01:44:40Z

@greptile

waleedlatif1 · 2026-10-07T01:44:41Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-10-07T01:44:45Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

All reported issues were addressed across 17 files

_{Reply with feedback, questions, or to request a fix.

Fix all with cubic | Turn on auto-fix | Re-trigger cubic}

…e checked

waleedlatif1 · 2026-10-07T01:51:15Z

@greptile

waleedlatif1 · 2026-10-07T01:51:17Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-10-07T01:51:23Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

All reported issues were addressed across 17 files

_{Requires human review: Auto-approval blocked because this review re-detected 1 unresolved issue already reported by Cubic.

Turn on auto-fix | Re-trigger cubic}

…igger fake; plan test on production indexes

waleedlatif1 · 2026-10-07T02:01:54Z

@greptile

waleedlatif1 · 2026-10-07T02:01:55Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-10-07T02:02:01Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

All reported issues were addressed across 18 files

_{Reply with feedback, questions, or to request a fix.

Fix all with cubic | Turn on auto-fix | Re-trigger cubic}

…/key pair

waleedlatif1 · 2026-10-07T02:17:33Z

@greptile

waleedlatif1 · 2026-10-07T02:17:35Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-10-07T02:17:40Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

No issues found across 18 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{Turn on auto-fix | Re-trigger cubic}

improvement(outbox): only start the processor when work is due or mai…

a9a825d

…ntenance is scheduled

vercel Bot temporarily deployed to Preview October 7, 2026 01:13 Inactive

cubic-dev-ai Bot reviewed Oct 7, 2026

View reviewed changes

Comment thread apps/sim/lib/core/outbox/retention.ts Outdated

greptile-apps Bot reviewed Oct 7, 2026

View reviewed changes

Comment thread apps/sim/lib/core/outbox/enqueue.test.ts Outdated

test(outbox): assert each backend's own result; describe prune capaci…

349979c

…ty as scheduled

vercel Bot temporarily deployed to Preview October 7, 2026 01:27 Inactive

cubic-dev-ai Bot reviewed Oct 7, 2026

View reviewed changes

waleedlatif1 added 2 commits October 6, 2026 18:34

improvement(file-search): only start the dispatcher when there is dis…

9c97dc0

…patch work

refactor(cron): share the scheduled-pass trigger across outbox, file …

154f1c8

…search, and knowledge projection

vercel Bot temporarily deployed to Preview October 7, 2026 01:44 Inactive

waleedlatif1 changed the title ~~improvement(outbox): only start the processor when work is due or maintenance is scheduled~~ improvement(cron): only start outbox and file-search passes when work is due, via a shared scheduled-pass helper Oct 7, 2026

waleedlatif1 mentioned this pull request Oct 7, 2026

improvement(file-search): only start the dispatcher when there is dispatch work #8714

Closed

6 tasks done

cubic-dev-ai Bot reviewed Oct 7, 2026

View reviewed changes

Comment thread apps/sim/lib/workspace-files/search/dispatcher.integration.ts Outdated

greptile-apps Bot reviewed Oct 7, 2026

View reviewed changes

Comment thread apps/sim/lib/workspace-files/search/enqueue-dispatch.test.ts Outdated

Comment thread apps/sim/lib/workspace-files/search/dispatcher.integration.ts Outdated

Comment thread apps/sim/lib/workspace-files/search/dispatcher.integration.ts Outdated

refactor(file-search): key the dispatch window by the instant the gat…

e7892cd

…e checked

vercel Bot temporarily deployed to Preview October 7, 2026 01:51 Inactive

cubic-dev-ai Bot reviewed Oct 7, 2026

View reviewed changes

test(cron): assert scheduled-pass outcomes through a deduplicating Tr…

0509b49

…igger fake; plan test on production indexes

vercel Bot temporarily deployed to Preview October 7, 2026 02:01 Inactive

cubic-dev-ai Bot reviewed Oct 7, 2026

View reviewed changes

Comment thread packages/testing/src/mocks/trigger-sdk.mock.ts Outdated

test(testing): key the idempotent Trigger fake by an unambiguous task…

e448279

…/key pair

vercel Bot temporarily deployed to Preview October 7, 2026 02:17 Inactive

cubic-dev-ai Bot reviewed Oct 7, 2026

View reviewed changes

waleedlatif1 merged commit 18ad8b8 into staging Oct 7, 2026
33 checks passed

waleedlatif1 deleted the improvement/outbox-work-gate branch October 7, 2026 05:48

Conversation

waleedlatif1 commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Type of Change

Testing

Checklist

Uh oh!

vercel Bot commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

greptile-apps Bot commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 5/5

Summary

Uh oh!

Uh oh!

waleedlatif1 commented Oct 7, 2026

Uh oh!

waleedlatif1 commented Oct 7, 2026

Uh oh!

cubic-dev-ai Bot commented Oct 7, 2026

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

waleedlatif1 commented Oct 7, 2026

Uh oh!

waleedlatif1 commented Oct 7, 2026

Uh oh!

cubic-dev-ai Bot commented Oct 7, 2026

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

waleedlatif1 commented Oct 7, 2026

Uh oh!

waleedlatif1 commented Oct 7, 2026

Uh oh!

cubic-dev-ai Bot commented Oct 7, 2026

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

waleedlatif1 commented Oct 7, 2026

Uh oh!

waleedlatif1 commented Oct 7, 2026

Uh oh!

cubic-dev-ai Bot commented Oct 7, 2026

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

waleedlatif1 commented Oct 7, 2026

Uh oh!

waleedlatif1 commented Oct 7, 2026

Uh oh!

cubic-dev-ai Bot commented Oct 7, 2026

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

This branch was previously deployed

Reviewers

Assignees

Labels

waleedlatif1 commented Oct 7, 2026 •

edited

Loading

vercel Bot commented Oct 7, 2026 •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

greptile-apps Bot commented Oct 7, 2026 •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading