RFC: Decoupling Rust runtime storage for serverless execution

[likun666661]

RFC: Decoupling Maka Rust Runtime Storage for Serverless Execution

• Status: Draft for discussion
• Date: 2026-09-24
• Scope: Rust runtime storage architecture
• Source baseline: maka-runtime-host-rust, feat/runtime-host-rust, commit 1308b5150

Summary

Maka's Rust runtime currently persists its operational state in local SQLite databases under an exclusively owned state root. We want to explore a storage boundary that can support both local execution and a serverless deployment model. The working names are Store for the runtime-facing abstraction, LocalStore for the SQLite-backed implementation, and RemoteStore for a remote implementation.

The important qualification is that a remote implementation cannot simply replace SQLite with S3 or another file store while preserving the current behavior. Runtime storage performs transactional state transitions, consistency checks, compare-and-swap updates, and snapshot reads. A remote primary store will likely need a transactional database or an equivalent coordination service. Object storage may still be useful for large immutable payloads. This RFC does not select a vendor or prescribe an implementation.

Motivation

The current Host constructs EventLog and ConfigurationStore against a local RootOwner. This is appropriate for a single locally owned runtime but couples the runtime's persistence lifecycle to a local filesystem and SQLite connections. A serverless execution environment may create a new process for each request, so durable state and concurrency control must not depend on one process retaining the state-root lease indefinitely.

The intended decoupling has two aspects:

1. Runtime and context-building code should depend on explicit storage operations and guarantees, not SQLite connections or local file paths.
2. Deployment-specific storage implementations should provide those guarantees through suitable local or remote mechanisms.

This is a proposal to define the boundary, not a claim that the current EventLog API already is that boundary.

Current State (Code Baseline)

• EventLog::for_root opens runtime-rust.sqlite under RootOwner, and ConfigurationStore opens a separate configuration-rust.sqlite. The two databases have no shared atomic transaction. See crates/event-log/src/lib.rs, crates/event-log/src/root.rs, and crates/config/src/database.rs.
• An OwnedConnection serializes accepted writes on a dedicated owner thread and bounds independent read connections. Losing a caller does not necessarily cancel an accepted database operation. See crates/event-log/src/connection.rs.
• append_batch validates ordered facts, writes events and associated records, and advances session state inside one SQLite transaction. Exact event-ID retries are checked against persisted content; an uncertain commit is reported as an unknown outcome. See crates/event-log/src/append.rs.
• Context reads select and materialize multiple related records in a consistent transaction snapshot. See crates/event-log/src/context/read.rs.
• Session creation, message-queue edits, artifacts, and plugin data use additional transactional operations, revisions, receipts, or immutable-payload checks. See crates/event-log/src/sessions.rs, message_queue.rs, artifacts.rs, and plugins/storage.rs.
• The existing maka_plugins::storage::Store is a narrow plugin KV capability (scan, read, atomic CAS batch). It is not a runtime-wide storage interface. See crates/plugins/src/storage.rs.

Direction

Define a runtime-facing storage contract in terms of domain operations and observable consistency, rather than exposing SQL or requiring every backend to emulate SQLite internals. LocalStore may keep the existing SQLite implementation. RemoteStore may use a remote transactional database for authoritative mutable state and, where appropriate, object storage for large immutable content.

The contract should state, per operation, its atomicity boundary, concurrency preconditions, idempotency identity, read consistency, and behavior when the caller cannot determine whether a write committed. A single broad Store facade may be convenient for callers, but its internal capabilities can be separated by domain if one interface becomes unwieldy. The exact trait layout is not yet decided.

In particular, “S3-backed” should not imply that S3 alone provides the transactional semantics of the current event database. Nor should RemoteStore be required to mirror SQLite tables, queries, or connection ownership. The remote design must satisfy the required behavior with its own suitable primitives.

Questions to Resolve

1. Which existing operations must be atomic together, and which can be separated without changing observable runtime behavior?
2. What consistency and isolation do context reads, recovery, admission, queue edits, and plugin operations require across concurrent serverless workers?
3. What replaces the local root/writer lease for ownership, fencing, and concurrent execution of a Session?
4. Which identities and receipts let a new process reconcile a timed-out or otherwise uncertain write without repeating an external effect?
5. Which data belongs in the transactional primary store, and which immutable payloads can be externalized to object storage? How is referential integrity maintained across them?
6. Does configuration and credential storage share the deployment boundary, or need a separately managed service and security model?
7. What are the latency, cost, data locality, retention, and failure-mode requirements for local versus serverless deployments?
8. What migration or compatibility path, if any, is needed for existing local state roots?

Non-Goals for This Draft

• Selecting a cloud database, object store, schema, or Rust trait signature.
• Porting SQL queries to an object store or promising transparent interchangeability between arbitrary backends.
• Changing current storage code, recovery behavior, or model-context semantics.
• Producing a Chinese translation at this stage.

Suggested Next Step

Inventory the runtime-facing EventLog and ConfigurationStore operations by domain. For each operation, record the minimum behavioral contract and the SQLite transaction or snapshot it currently uses. Use that inventory to identify the smallest viable remote primary-store capability before choosing a backend or drafting traits.