Skip to content

Bucket Room Timeline projection history to bound resident RAM #1721

Description

@hmans

Problem

Room Timeline currently retains decoded timeline entries, message bodies, and derived indexes for every room for the lifetime of the process. Its RAM use therefore grows with the complete retained message history.

Direction

Keep one ordered Room Timeline projector, but divide its derived state into fixed calendar buckets. Lightweight bucket metadata remains resident, buckets intersecting an operator-configurable hot window are materialized eagerly, and older buckets are reconstructed from their recorded EVT sequences when a read needs them. The first version should expose safe acquire/release and size-accounting boundaries for a later byte-weighted LRU without implementing automatic eviction yet.

Acceptance criteria

  • Existing room timeline, point lookup, around-message, edit/retraction, secure-deletion, snapshot, realtime, and search-hydration semantics remain correct.
  • EVT remains the only authoritative backing store; bucket metadata and payloads remain disposable projection state.
  • Existing global stream-sequence cursors remain valid across bucket boundaries.
  • Operators can configure how many recent days are eagerly materialized.
  • Missing historical EVT facts fail explicitly except for deliberately secure-deleted message-body payloads.
  • Synthetic and restored real-history benchmarks demonstrate a meaningful retained-RAM reduction without an unacceptable historical-read penalty.
  • Architecture, configuration, and self-hosting documentation describe the new lifecycle and rollout implications.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions