Skip to content

refactor!: core transport on std:: containers and smart pointers (3/6) - #103

Open
Segfaultd wants to merge 14 commits into
refactor/modern-memory-2-allocatorfrom
refactor/modern-memory-3-core-transport
Open

Segfaultd wants to merge 14 commits into
refactor/modern-memory-2-allocatorfrom
refactor/modern-memory-3-core-transport

Conversation

@Segfaultd

Copy link
Copy Markdown
Member

Summary

The core transport moves to std:::

  • Reliability heaps: std::push_heap/pop_heap with FIFO tie-breaking
  • RangeList: ACK/NAK wire codec in range_list.h, wire-compatible
  • Command and packet queues: LockedQueue<T>, which removes the TCP queue data race TSan reported
  • InternalPacket allocation: ObjectPool<T> (InternalPacket only)
  • Receive loop: each message is owned by a unique_ptr

This also fixes a 94-byte ID_NEW_INCOMING_CONNECTION leak on an already-connected system that dates back to RakNet.

Notes

  • Benchmarks: no reliability-layer or RakPeer::Send benchmark is more than 5% slower than master; most are faster. RakPeer::Send now allocates its buffered command with make_unique instead of taking a pool slot.
  • Freeing data in RunUpdateCycle: master's three rakFree_Ex(data) calls there were removed during the rebase, because the unique_ptr already frees that buffer. A later change to that loop must not free data by hand.
  • GCC + ASan on arm64: this branch on its own is slow under GCC + ASan on arm64. The toolchain never releases ASan's use-after-return frames there, and the std:: heap makes many more instrumented calls, so Rm3ReceiveThread can stall. 6/6 turns that ASan check off for GCC on AArch64. CI's sanitizer jobs run on x86-64 and are not affected.

Test plan

  • Debug -Werror build; unit tests plus the Reliability/Simulated/RakPeer/Encrypt/Mtu/Split integration tests

Commits

  • refactor!: RangeList on std::vector in MafiaNet namespace
  • style: clang-format range_list.h
  • refactor: reliability heaps on std::push_heap with FIFO ties
  • test: extract ReliabilityLayerPair harness
  • refactor!: ReliabilityLayer containers on std::
  • fix: avoid shadowed loop index in UpdateInternal; rename split test
  • refactor!: LockedQueue replaces ThreadsafeAllocatingQueue and SingleProducerConsumer
  • refactor!: RakPeer containers and system-list API on std::vector
  • fix: make RemoteSystemStruct::connectMode atomic; drop its TSan suppression
  • bench: RakPeer send/receive throughput
  • perf: pool InternalPacket allocations
  • fix: free ID_NEW_INCOMING_CONNECTION on an already-connected system

Stack, merge in order: 1/6 safety net, 2/6 allocator, 3/6 core transport, 4/6 plugins, 5/6 Table + extensions, 6/6 delete DataStructures. Each PR targets the branch of the one before it. Breaking change for the next major version; no version bump yet.

@coderabbitai

coderabbitai Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review: 2

  • main
  • master

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: c7356b41-56e6-4bad-9815-43be4ace2370

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@Segfaultd
Segfaultd force-pushed the refactor/modern-memory-2-allocator branch from 78f100e to 39465ca Compare November 16, 2026 11:43
@Segfaultd
Segfaultd force-pushed the refactor/modern-memory-3-core-transport branch from e4ad4d4 to 2721afb Compare November 16, 2026 11:43
The two-ReliabilityLayer fake-socket harness was copied into the black-hole, split-packet and characterization tests. Move it to Tests/Support/ReliabilityLayerPair.h and migrate the three files to it, with no change in behaviour.
Every remaining DataStructures container in ReliabilityLayer moves to the
standard library: the Queue members (outputQueue, datagramHistory,
hasReceivedPacketQueue, delayList, BPSTracker::dataQueue) become std::deque,
the List members become std::vector, and splitPacketChannelList becomes
std::map<SplitPacketIdType, SplitPacketChannel *>. The dead commented-out
code that still named LinkedList, MLKeyRef, BPlusTree and the old
OrderedList comparator goes, with the ds_bplus_tree.h include and
RESEND_TREE_ORDER.

The plugin-list parameters ReliabilityLayer takes become
std::vector<PluginInterface2 *>&, so RakPeer's pluginListTS and
pluginListNTS move to std::vector as well.

Split reassembly keeps every check on network input: SplitPacketSort::Add
and the header bound in ReadFromBitStream are unchanged, and a channel is
still dropped when its first fragment is rejected.

The three MemoryPool members stay until the allocator decision in Task 3.6.

BREAKING CHANGE: SplitPacketChannelComp and RESEND_TREE_ORDER are removed,
BPSTracker::Reset takes no arguments, and the ReliabilityLayer members and
RakPeer plugin lists above change type.
RakPeer's remaining DataStructures members move to the standard library:
banList, pluginListNetworkThread, socketList, securityExceptionList and
SocketQueryOutput::sockets become std::vector; requestedConnectionQueue,
bufferedPacketsQueue and packetReturnQueue become std::deque; the
receive-buffer free pool becomes a std::vector stack. Existing mutex
guards are unchanged. RakString::freeList is a std::vector.

The Packet and RemoteSystemIndex MemoryPools are gone: both are plain
new/delete now. packetsOutstanding stays as an atomic leak counter. A
double allocation in ReferenceRemoteSystem that the pool used to absorb
is fixed.

RunUpdateCycle now checks requestedConnectionQueue for emptiness under
requestedConnectionQueueMutex; that unlocked read was the race behind the
Queue<RequestedConnectionStruct*>::Push TSan suppression, which is removed.

BREAKING CHANGE: GetSystemList, GetSockets, ReleaseSockets and
GetStatisticsList take std::vector out-params. RakNetSocket2 Send,
SendBatch and Bind, the Berkley bind helpers, Send_Windows_Linux and
RNS2EventHandler::AllocRNS2RecvStruct/DeallocRNS2RecvStruct no longer
take file/line arguments. peer_interface.h and string.h no longer pull
in ds_list.h.
…ession

Send()'s session-handshake gate (IsExchangingSessionData) reads
connectMode on the user thread while RunUpdateCycle writes it on the
network thread. The race:RakPeer::IsExchangingSessionData suppression
stopped matching once the function was inlined into Send, so
TypedSendLive, ComprehensiveConvert and Rm3ReceiveThread failed under
TSan. connectMode is now std::atomic<ConnectMode>; every use compiles
unchanged through implicit load/store, and the suppression is removed.
Two connected RakPeers on the tests' simulated network (no sockets, no
threads, virtual clock), so the benchmarks see RakPeer's own per-Send and
per-received-Packet work that the bare ReliabilityLayer benchmark misses.
Uses only Send/Receive/DeallocatePacket so it builds against older trees.
ReliabilityLayer's three DataStructures::MemoryPool members go. The
per-datagram MessageNumberNode and the shared-buffer
InternalPacketRefCountedData become plain new/delete; InternalPacket, taken
for every message sent, received, split or acked, is recycled through a new
ObjectPool<T> (raw-pointer Acquire/Release over a std::vector<std::unique_ptr<T>>
free list), because plain new/delete for it regressed the benchmarks past the
5% gate: ReliabilityLayer_SmallReliableOrdered/16 +22.6%, SplitMessage/16384
+13.0%, RakPeer_SendReceiveSmall/64 +5.8% against the pre-migration tree.

With the pool every ReliabilityLayer_* and RakPeer_* median is within 5% of
262799f3 (small-message paths 6-14% faster). Unlike MemoryPool::Clear, the
pool frees only released objects, so a packet the layer forgets to release
is now a visible leak instead of being reclaimed silently.
RakPeer::RunUpdateCycle dropped an ID_NEW_INCOMING_CONNECTION that arrived
for a remote system no longer in HANDLING_CONNECTION_REQUEST without freeing
it: the std::free had been commented out since RakNet. Both sides of a cross
(simultaneous) connection send one, so every such connect leaked the 94-byte
message. LeakSanitizer reports it in
CrossConnectionConvertSim.SimultaneousConnectWhenBothSidesAcceptIncoming and
ComprehensiveConvert.DoesALittleBitOfEverything; master leaks the same way.

The receive loop now holds each message ReliabilityLayer::Receive hands over
in a unique_ptr with a std::free deleter. It is freed when the iteration
ends, and the branches that build a Packet from it release() it to
AllocPacket. This replaces the 20 manual std::free calls, so a branch that
forgets one can no longer leak.
GetReceiveBufferSize casts the queue size to its unsigned int return type, and the integration tests keep the connection counts GetSystemList now reports as std::vector sizes in std::size_t. No assertions change.
@Segfaultd
Segfaultd force-pushed the refactor/modern-memory-2-allocator branch from 39465ca to d622b2d Compare November 16, 2026 13:58
@Segfaultd
Segfaultd force-pushed the refactor/modern-memory-3-core-transport branch from 2721afb to 1965815 Compare November 16, 2026 13:58

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant