Repository navigation
Conversation
This was referenced Oct 18, 2026
axieax
marked this pull request as ready for review
October 18, 2026 15:15
kraftp
added a commit
to dbos-inc/dbos-transact-py
that referenced
this pull request
Oct 20, 2026
## Summary Related issue: dbos-inc/dbos-transact-ts#1411. Coordinate the shared schema changes required by dbos-inc/dbos-transact-ts#1410. Its latest-version lookup rewrite uses bounded owned/unclaimed index probes instead of scanning growing release history. Proposed shared slots: - 124: `idx_application_versions_timestamp(version_timestamp DESC)` - 125: `idx_application_versions_owner_timestamp(application_name, version_timestamp DESC)` Add the identical keys using this SDK's existing online-index migration protocol. PostgreSQL uses `CONCURRENTLY IF NOT EXISTS`, Cockroach rendering omits `CONCURRENTLY`, and SQLite variants are included where this SDK supports SQLite. Existing history, columns and version timestamps are preserved. This companion changes migrations only; the Python latest-version query itself is unchanged. ## Coordination 124/125 are proposed, not reserved. Coordinate TypeScript, Python, Go, Java and Rust migration allocation before merging/releasing any companion. Coordinated proposals: - TypeScript: dbos-inc/dbos-transact-ts#1410 - Python: #900 - Go: dbos-inc/dbos-transact-golang#504 - Java: dbos-inc/dbos-transact-java#629 - Rust: dbos-inc/dbos-transact-rust#106 Existing application code remains compatible with these additive indexes. ## Verification Native PostgreSQL migration tests: 5 passed; SQLite migration/index tests: 3 passed. Black, isort, strict mypy and compile checks passed. Exact 124/125 SQL from all five SDKs was executed twice against isolated PostgreSQL 16 schemas: identical valid index keys, idempotent execution, existing rows and promoted version preserved. Python/Go SQLite definitions also produced identical key order/directions. Full SDK CI and Cockroach execution remain outstanding. No live consumer schema or pinned SDK was changed. --------- Co-authored-by: Peter Kraft <petereliaskraft@gmail.com>
kraftp
added a commit
to dbos-inc/dbos-transact-ts
that referenced
this pull request
Oct 20, 2026
## Summary Fixes #1411. Queue dequeue selects the latest eligible application version on every poll. With no timestamp index, the lookup scans a growing deployment history. A timestamp-only index still leaves named applications vulnerable to scanning many newer peer versions. Add online shared migrations 124/125 for timestamp and application-name/timestamp indexes. Named lookups compare the newest owned row and newest unclaimed row using two indexed LIMIT 1 probes in one statement; unnamed/empty scopes retain global lookup. The dequeue and public latest-version APIs share this query. Registration, timestamp promotion, history retention and routing semantics are unchanged. The constant application name remains in each probe's ORDER BY: PostgreSQL 16 otherwise sorts the full IS NULL cohort despite the composite index. ### Shared migration coordination Slots 124/125 are proposed, not reserved. Matching Python, Go, Java and Rust changes must be coordinated before merge/release. Rust also needs its missing shared 122/123 predecessors. Companion migration proposals: - TypeScript: #1410 - Python: dbos-inc/dbos-transact-py#900 - Go: dbos-inc/dbos-transact-golang#504 - Java: dbos-inc/dbos-transact-java#629 - Rust: dbos-inc/dbos-transact-rust#106 A v4.27 backport would let existing v4 consumers adopt the fix without an unrelated major upgrade. ## Verification - PostgreSQL 16, Node 24.21: migration, application ownership, versioning and new scaling suites: 38 passed. - Existing ordinary versionless dequeue/promotion and batched partition version-eligibility tests: 2 passed. - New regression test alone: 5 failures (270 buffer blocks each) on the unchanged base, then 7 passed with the fix; independently repeated in a disposable baseline clone. - Core TypeScript build/typecheck, scoped ESLint and Prettier checks pass. - Independent adversarial actual-API probe with 300,000 versions, including 299,980 newer peer versions: eligible owned/unclaimed/absent scopes require at most 8 buffer blocks; global/empty scopes require 4. Synthetic mixed-owner benchmark, warmed PostgreSQL 16 EXPLAIN samples (median of seven; not production timings): | Versions | Before execution | After execution | Before buffer blocks | After buffer blocks | | ---: | ---: | ---: | ---: | ---: | | 1,054 | 0.221 ms | 0.028 ms | 9 | 6 | | 30,000 | 2.383 ms | 0.012 ms | 270 | 6 | | 300,000 | 13.054 ms | 0.011 ms | 2,738 | 8 | Companion checks: Python native PostgreSQL/SQLite tests pass; Go native migration test and formatting pass; Rust 17 migration unit tests, formatting and integration-test compilation pass; Java formatting and production/test compilation pass. Exact proposed index SQL from all five SDKs was executed twice against PostgreSQL 16 and produced identical valid indexes without changing existing rows. Full workspace/SDK suites, other Node versions, Cockroach execution and Java/Rust database integration execution remain for CI. ## Risk and rollback The migrations add indexes through the existing online runner and its interrupted-index cleanup/replay protocol. Existing executors remain compatible, and indexes can remain across code rollback. No rows are removed or retimed. Preserve shared migration numbering across SDKs before releasing. Co-authored-by: Peter Kraft <peter.kraft@dbos.dev>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Related issue: dbos-inc/dbos-transact-ts#1411.
Coordinate the shared schema changes required by dbos-inc/dbos-transact-ts#1410. Its latest-version lookup rewrite uses bounded owned/unclaimed index probes instead of scanning growing release history.
Proposed shared slots:
idx_application_versions_timestamp(version_timestamp DESC)idx_application_versions_owner_timestamp(application_name, version_timestamp DESC)Add the identical keys using this SDK's existing online-index migration protocol. PostgreSQL uses
CONCURRENTLY IF NOT EXISTS, Cockroach rendering omitsCONCURRENTLY, and SQLite variants are included where this SDK supports SQLite. Existing history, columns and version timestamps are preserved. This companion changes migrations only; the Java latest-version query itself is unchanged.Coordination
124/125 are proposed, not reserved. Coordinate TypeScript, Python, Go, Java and Rust migration allocation before merging/releasing any companion. Coordinated proposals:
Existing application code remains compatible with these additive indexes.
Verification
JDK 21 / Gradle 9.7.1: Spotless and production/test compilation passed. Exact PostgreSQL index DDL executed twice successfully; native Testcontainers integration execution remains for upstream CI.
Exact 124/125 SQL from all five SDKs was executed twice against isolated PostgreSQL 16 schemas: identical valid index keys, idempotent execution, existing rows and promoted version preserved. Python/Go SQLite definitions also produced identical key order/directions.
Full SDK CI and Cockroach execution remain outstanding. No live consumer schema or pinned SDK was changed.