Repository navigation
feat: persist Anthropic Messages threads - #508
Merged
Merged
Conversation
Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com>
Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com>
Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com>
franciscojavierarceo
force-pushed
the
codex/anthropic-message-threads
branch
from
November 4, 2026 04:39
caa27e6 to
75c74c2
Compare
franciscojavierarceo
marked this pull request as ready for review
November 4, 2026 11:33
franciscojavierarceo
requested review from
bbrowning,
haoshan98,
jiahuei,
leseb,
maralbahari,
noobHappylife,
qandrew and
tjtanaa
as code owners
November 4, 2026 11:33
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved bugs can publish invalid continuation checkpoints, lose streamed citations, misreport persistence failures as successes, and let the end-to-end test modify the user’s real configuration.
5 open issues
What changed in this PR
Adds database-backed Anthropic Messages threads, letting clients send only new messages while the gateway restores history for a stateless upstream. Thread state remains separate from Responses storage.
Changes:
- Adds thread contracts, validation, durable checkpoints, and exclusive continuation leases.
- Integrates persistence with JSON/SSE completion, shutdown, and Claude Code compatibility.
- Adds storage and HTTP tests, recorded vLLM replay scenarios, CI coverage, and usage documentation.
| File | Description |
|---|---|
scripts/tests/messages-claude-cli-e2e-test.py |
Adds real Claude Code smoke testing. |
docs/guides/harness-cli-testing.md |
Documents CLI thread verification. |
docs/api/index.md |
Documents thread usage and lifecycle rules. |
crates/agentic-server/tests/messages_threads_test.rs |
Tests HTTP/SSE continuation and failure handling. |
crates/agentic-server/tests/messages_threads_cassette_test.rs |
Replays recorded threads across restarts. |
crates/agentic-server/tests/common/messages_database.rs |
Provides isolated test databases. |
crates/agentic-server/src/server.rs |
Drives lease maintenance and cleanup. |
crates/agentic-server/src/handler/http/messages.rs |
Routes threads and establishes ownership. |
crates/agentic-server-core/tests/messages_thread_storage_test.rs |
Tests persistence, fencing, and rollback. |
crates/agentic-server-core/tests/cassettes/record_messages_threads.py |
Records live thread scenarios. |
crates/agentic-server-core/tests/cassettes/README.md |
Documents recording and replay procedures. |
crates/agentic-server-core/tests/cassettes/messages/threads/messages-thread-text-Qwen-Qwen3.8-27B-FP8-streaming.yaml |
Captures upstream text SSE turns. |
crates/agentic-server-core/tests/cassettes/messages/threads/messages-thread-text-Qwen-Qwen3.8-27B-FP8-streaming.json |
Captures public text SSE turns. |
crates/agentic-server-core/tests/cassettes/messages/threads/messages-thread-text-Qwen-Qwen3.8-27B-FP8-nonstreaming.yaml |
Captures upstream text JSON turns. |
crates/agentic-server-core/tests/cassettes/messages/threads/messages-thread-text-Qwen-Qwen3.8-27B-FP8-nonstreaming.json |
Captures public text JSON turns. |
crates/agentic-server-core/tests/cassettes/messages/threads/messages-thread-client-Qwen-Qwen3.8-27B-FP8-streaming.yaml |
Captures upstream client-tool SSE turns. |
crates/agentic-server-core/tests/cassettes/messages/threads/messages-thread-client-Qwen-Qwen3.8-27B-FP8-streaming.json |
Captures public client-tool SSE turns. |
crates/agentic-server-core/tests/cassettes/messages/threads/messages-thread-client-Qwen-Qwen3.8-27B-FP8-nonstreaming.yaml |
Captures upstream client-tool JSON turns. |
crates/agentic-server-core/tests/cassettes/messages/threads/messages-thread-client-Qwen-Qwen3.8-27B-FP8-nonstreaming.json |
Captures public client-tool JSON turns. |
crates/agentic-server-core/src/types/messages/thread.rs |
Defines thread, checkpoint, and error types. |
crates/agentic-server-core/src/types/messages/request.rs |
Adds the optional thread parameter. |
crates/agentic-server-core/src/types/messages/mod.rs |
Exposes thread types and updates module documentation. |
crates/agentic-server-core/src/storage/models/mod.rs |
Registers the thread row model. |
crates/agentic-server-core/src/storage/models/message_thread.rs |
Defines the checkpoint database row. |
crates/agentic-server-core/src/storage/mod.rs |
Exposes Messages storage. |
crates/agentic-server-core/src/storage/messages.rs |
Implements checkpoints, leases, and expiry. |
crates/agentic-server-core/src/executor/request.rs |
Initializes storage and readiness checks. |
crates/agentic-server-core/src/executor/mod.rs |
Exposes thread orchestration. |
crates/agentic-server-core/src/executor/messages_usage.rs |
Detects fallback response usage. |
crates/agentic-server-core/src/executor/messages_threads/validation.rs |
Validates thread constraints and fixed fields. |
crates/agentic-server-core/src/executor/messages_threads/tests/claude_code_compatibility.rs |
Tests repeated fields and attribution handling. |
crates/agentic-server-core/src/executor/messages_threads/tests.rs |
Tests admission, retention, and rehydration. |
crates/agentic-server-core/src/executor/messages_threads/claude_code.rs |
Implements Claude Code field compatibility. |
crates/agentic-server-core/src/executor/messages_threads.rs |
Coordinates admission and checkpoint publication. |
crates/agentic-server-core/src/executor/messages_stream/ingestion.rs |
Adds thread stream limits and terminal checks. |
crates/agentic-server-core/src/executor/messages_stream.rs |
Publishes checkpoints before SSE completion. |
crates/agentic-server-core/src/executor/messages_loop.rs |
Publishes checkpoints before JSON delivery. |
crates/agentic-server-core/src/executor/messages_context.rs |
Carries thread state through execution. |
crates/agentic-server-core/src/error.rs |
Adds thread initialization errors. |
crates/agentic-server-core/migrations/0009_message_threads.sql |
Creates the independent checkpoint table. |
crates/agentic-server-core/Cargo.toml |
Adds ownership-hashing dependency. |
Cargo.toml |
Declares shared SHA-256 dependency. |
Cargo.lock |
Records the core dependency update. |
ARCHITECTURE.md |
Documents thread ownership and persistence flow. |
.github/workflows/rust.yml |
Adds PostgreSQL thread test coverage. |
Review effort: Balanced
Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com>
maralbahari
reviewed
Nov 5, 2026
Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com>
Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com>
Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com>
Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com>
Collaborator
|
Thanks, all six points are addressed. Two optional follow-ups from the typed refactor:
|
Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com>
maralbahari
pushed a commit
that referenced
this pull request
Nov 6, 2026
## Summary Follow up on [Mara’s two optional comments on #508](#508 (comment)). Gateway-tool Messages requests with an unknown thinking or cache-control type previously returned 400. Preserve those provider tags and extension fields through the shared typed request, while continuing to reject malformed known thinking payloads. Update OpenAPI schemas to accept future variants. Restore the direct `verify_persistence_ready` timeout in `ExecutionContext::storage_ready`, removing the redundant async block and boolean conversion. ## Test Plan - Full workspace suite: **3,064 passed, 26 ignored** with `agentic-server/vendored-openssl`. - New unit tests cover known/future parameter round-trips, invalid known payloads, and cache-control extension fields; the final focused run passes all three tests. - New mock-upstream regression verifies JSON and SSE gateway-tool rounds preserve future thinking and request/system/content-block cache-control fields. All 14 tool-choice integration tests pass. - OpenAPI fixtures accept both known and future variants; existing Messages thread cassettes and integration tests pass in the workspace suite. - Workspace Clippy across all targets with `-D warnings`, formatting, and pre-commit checks pass. Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add local, stateful Anthropic Messages threads behind
message-threads-2026-08-12. Clients create a thread and continue from a completed message ID with only new messages; the gateway rehydrates the full conversation for a stateless upstream.message_threadstable in migration0012_message_threads.sql. Reuse the shared typedMessagesRequest,MessageParam,SystemPrompt,ToolParam, and losslessContentBlockfor thread admission and history, with a typed fingerprint and checkpoint instead of parallel JSON envelopes.storage/models/, state/checkpoint-column conversions instorage/types/, and production store access inexecutor/modes/messages_threads; verify the thread table through common schema/readiness paths.ParsedMessagesRequestand branch on the typedthread. Malformed upstream terminal fields or known content blocks produce server/protocol errors without consuming the parent.AGENTIC_API_HOME.Known follow-up: remaining architectural review findings include accumulated historical cache breakpoints in the upstream projection; malformed streamed client-tool arguments and out-of-order indices that can make retained history differ from delivered content. These remain separate from the main integration, consistent with RFC #500.
Mara’s two optional typed-parameter/readiness follow-ups are implemented separately in #510.
Test Plan
Eryx runtime setup retries at most three times.
bash scripts/tests/setup-eryx-runtime-test.shcovers immediate success, recovery after transient failures, and permanent failure with the original exit status; Bash syntax checks and all pre-commit hooks pass.Full workspace tests: 3,060 passed, 26 ignored (default ignored cases; relevant PostgreSQL checks ran separately). Workspace Clippy with
agentic-server/vendored-openssl; formatting and all pre-commit hooks.OpenAPI feature compilation and workspace schema/fixture validation. Regressions cover nullable citation metadata, omitted tool input/output fields, MCP token exclusion, default fallback routing, ordinary context-edit forwarding, and null-thread proxy routing.
All 14 Messages HTTP/SSE integration tests pass on SQLite and PostgreSQL, including earlier-round fallback, citation retention in the next upstream request, truncated gateway calls versus truncated text, failed-publication telemetry, and JSON/SSE protocol-error rollback. Thread storage tests cover restart recovery, concurrent instances, cancellation, and fencing; stream tests cover missing-terminal/missing-stop rollback.
PostgreSQL storage (5 tests), migration/restart (5), tenant isolation (6), and schema verification (6), including removal of each required thread constraint and selection of a schema containing only the thread table.
All four real-vLLM cassettes replay successfully on both databases, comparing complete upstream requests and public output while normalizing only generated public message IDs. The captures were generated from vLLM
0.29.1rc1.dev391+g90531daf6servingQwen/Qwen3.8-27B-FP8; their original live scenarios passed on both databases.Real Claude Code 2.1.317 against the merged gateway and deterministic backend: repeated-field mode on SQLite and inherited-field mode on PostgreSQL. Both verify exactly two successful CLI requests, create then delta-only
tool_resultcontinuation, and startup configuration within the temporary gateway home. The existing user configuration hash remains unchanged.Claude Code read-only adversarial review, focused correction review, and current-main integration review; confirmed findings corrected and independently validated. The merge preserves main's typed item metadata/response snapshot contracts and migration 0009 plus migration 0010's removal of conversation revision, plus migration 0011's vector-search schema, with Messages in migration 0012. The SQLite CLI smoke test starts from main's migration 0011 and verifies migration 0012 preserves its existing conversation.
Provider evidence limitation: these recordings verify vLLM inference, gateway rehydration, and the gateway's public transcript. They are not Anthropic reference captures. Exact Anthropic error-envelope parity, optional fixed-field null/omission equivalence, paused-turn rejection, and continued-stream IDs have documentation/local-test evidence only. Null/omission equivalence is the gateway's current policy; no usable Anthropic API key was available here for a beta reference capture.