Skip to content

in_unix_socket: fix DGRAM mode, the collector never started - #12830

Open
Noamshl wants to merge 3 commits into
fluent:masterfrom
Noamshl:in_unix_socket-dgram-register-once
Open

Noamshl wants to merge 3 commits into
fluent:masterfrom
Noamshl:in_unix_socket-dgram-register-once

Conversation

@Noamshl

@Noamshl Noamshl commented Nov 22, 2026 •

Copy link
Copy Markdown

in_unix_socket with socket_mode DGRAM reads nothing: the input fails to start its collector at startup.

[  Error] epoll_ctl: File exists, errno=17 at lib/monkey/mk_core/mk_event_epoll.c:140
[error] [input collector] COLLECT_EVENT registration failed
[error] [input] error starting collector #0: unix_socket.0
[error] could not start collectors for plugin 'unix_socket.0'

In DGRAM mode the plugin wraps the server socket in a dummy connection and passes it to unix_socket_conn_add(), which registered that socket in the engine event loop. The collector created by flb_input_set_collector_socket() registers the same socket when it starts, so epoll_ctl(ADD) fails with EEXIST. The DGRAM read path already uses the collector event (ctx->collector_event), so the dummy connection does not need an event of its own. This now matches in_udp, whose udp_conn_add() does not register the connection either.

Changes:

  • in_unix_socket: register only stream connections in the event loop: the fix.
  • in_unix_socket: remove the collector before releasing connections: once the collector runs, shutdown released the dummy connection first. That closes the server socket, so the collector removal then called mk_event_del() on a closed descriptor and logged [ warn] [input] cannot disable event for unix_socket.0. The collector is now removed first, the order in_tcp and in_syslog already use.
  • tests: runtime: in_unix_socket: add DGRAM and STREAM input tests: a new runtime test, flb-rt-in_unix_socket. It is built when FLB_IN_UNIX_SOCKET is on. On master the DGRAM case fails and the STREAM case passes.

STREAM mode does not change. There are no config or output format changes.

Fixes #8943


Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

  • Example configuration file for the change
  • Debug log output from testing the change
  • Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

  • [N/A] Run local packaging test showing all targets (including any new ones) build.
  • [N/A] Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

  • [N/A] Documentation required for this feature

Backporting

  • Backport to latest stable release.

Please include this fix in the 5.2.x series. master is the 5.2 line at the moment (5.2.1), so a merge here reaches 5.2.1; if a 5.2 branch is cut before this is merged, I will open the backport PR against it.


Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.


Testing details

Built on Debian trixie (gcc 14) with -DFLB_TESTS_RUNTIME=On -DFLB_TESTS_INTERNAL=On -DFLB_IN_UNIX_SOCKET=On.

Configuration:

service:
  flush: 1
  log_level: debug
  http_server: on
  http_listen: 127.0.0.1
  http_port: 2020

pipeline:
  inputs:
    - name: unix_socket
      tag: verify.unix
      socket_mode: DGRAM
      socket_path: /tmp/verify-dgram.sock
      format: json

  outputs:
    - name: stdout
      match: '*'

Three datagrams were sent with socket.sendto(b'{"message": "verify N"}', "/tmp/verify-dgram.sock").

Before (master 4aa8721): the collector fails, as shown above, and the client gets ConnectionRefusedError. No records arrive.

After:

[ info] [input:unix_socket:unix_socket.0] initializing
[ info] [input:unix_socket:unix_socket.0] storage_strategy='memory' (memory only)
[debug] [unix_socket:unix_socket.0] created event channels: read=21 write=22
[0] verify.unix: [[1795340400.774180701, {}], {"message"=>"verify 0"}]
[1] verify.unix: [[1795340400.774350890, {}], {"message"=>"verify 1"}]
[2] verify.unix: [[1795340400.774372278, {}], {"message"=>"verify 2"}]
[ warn] [engine] service will shutdown in max 5 seconds
[ info] [engine] service has stopped (0 pending tasks)

The metrics reported unix_socket.0 records 3 and stdout.0 proc_records 3. The log had no errors or other warnings, and the exit status was 0.

Tests:

$ ctest --test-dir build -R flb-rt-in_unix_socket --output-on-failure
1/1 Test #39: flb-rt-in_unix_socket ............   Passed    4.07 sec
100% tests passed, 0 tests failed out of 1

# the new test against master, without the fix:
Test unix_socket_dgram...   [  Error] epoll_ctl: File exists, errno=17 at lib/monkey/mk_core/mk_event_epoll.c:140
[ FAILED ]
Test unix_socket_stream...  [ OK ]

Valgrind:

$ valgrind --leak-check=full --error-exitcode=1 ./build/bin/flb-rt-in_unix_socket
Test unix_socket_dgram...                       [ OK ]
Test unix_socket_stream...                      [ OK ]
SUCCESS: All unit tests have passed.
==18== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

Integration, unix_socket_decode_status scenario (runs the real binary; it delivers records through in_unix_socket in STREAM mode and checks DGRAM only for a refused start on a socket that another process holds, so the DGRAM data path is covered by the runtime test above):

$ tests/integration/.venv/bin/python -m pytest scenarios/unix_socket_decode_status -q
19 passed in 18.69s

The same scenario under Valgrind:

$ VALGRIND=1 VALGRIND_STRICT=1 tests/integration/.venv/bin/python -m pytest scenarios/unix_socket_decode_status -q
19 passed in 54.57s

Summary by CodeRabbit

  • Bug Fixes
    • Improved handling of Unix-domain socket inputs in both stream and datagram modes, helping messages arrive reliably.
    • Improved cleanup when the input shuts down, including releasing its collector before tracked socket connections are cleared.
  • Tests
    • Added runtime coverage for receiving messages through Unix-domain datagram and stream inputs.

Noam Shlomi added 3 commits November 22, 2026 13:11
In DGRAM mode the plugin wraps the server socket in a connection and
passes it to unix_socket_conn_add(), which registered the socket in the
engine event loop. The collector set with flb_input_set_collector_socket()
registers the same socket when it starts, epoll_ctl() fails with EEXIST
and the input never reads a datagram:

  [error] [input collector] COLLECT_EVENT registration failed
  [error] [input] error starting collector #0: unix_socket.0
  [error] could not start collectors for plugin 'unix_socket.0'

The DGRAM path already reads through the collector event
(ctx->collector_event), so the wrapping connection needs no event of its
own. Only STREAM connections are registered now.

Signed-off-by: Noam Shlomi <noam.shlomi@audiocodes.com>
In DGRAM mode the dummy connection wraps the server socket, and
releasing it closes the socket. The exit callback released the
connections first and removed the collector afterwards. With the
collector running (previous commit), mk_event_del() then ran on a
closed descriptor and every shutdown logged:

  [ warn] [input] cannot disable event for unix_socket.0

The collector is now removed before the connections are released,
as in_tcp and in_syslog do before they destroy their downstream.

Signed-off-by: Noam Shlomi <noam.shlomi@audiocodes.com>
Each test sends one JSON map to the socket and waits for the record at
the lib output. The DGRAM test fails without the previous commit, since
the collector never starts. The tests are built when the plugin is
enabled (FLB_IN_UNIX_SOCKET).

Signed-off-by: Noam Shlomi <noam.shlomi@audiocodes.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 22, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ❌ Failed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ❌ Failed 2026-11-22T12:21:14.745726Z
We couldn’t complete this review. Comment @codex review to try again.
62d559e PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@coderabbitai

coderabbitai Bot commented Nov 22, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The Unix socket input now registers per-connection events only in stream mode and removes its collector during shutdown. Runtime tests cover datagram and stream input, and the runtime test configuration registers them.

Changes: Unix socket input

Layer / File(s) Summary
Socket event handling
plugins/in_unix_socket/unix_socket_conn.c, plugins/in_unix_socket/unix_socket.c
Datagram connections no longer register per-connection events. On exit, the input removes its registered collector and resets collector_id before releasing tracked connections.
Datagram and stream runtime coverage
tests/runtime/in_unix_socket.c, tests/runtime/CMakeLists.txt
Adds runtime tests that send a JSON record over datagram and stream sockets and check the output. Registers the tests when FLB_OUT_LIB is enabled.

Estimated code review effort: 2 (Simple) | ~12 minutes

Change: Bug fix · Severity of issue fixed: Medium

Merge Risk: Minimal · up to 62d55

This change fixes datagram-mode startup for the Unix socket input and leaves stream behavior unchanged. It adds runtime tests for both modes. No concrete merge-blocking risk was identified.

Pre-merge checks: Passed 4

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title names the affected plugin, the DGRAM mode, and the symptom (the collector never started). The changeset matches this: unix_socket_conn_add now registers only in stream mode, in_unix_socket_e…
Linked Issues check Passed The only linked issue, #8943, is closed and marked as historical context only. No active directly linked targets remain, so no linked-issue coding requirements apply. The assessment does not evaluate …
Out of Scope Changes check Passed All changes relate to the in_unix_socket DGRAM start failure described in the PR and in #8943, or to verifying it. (1) plugins/in_unix_socket/unix_socket_conn.c: registers the connection event only in…
Full details: Title check

Explanation

The title names the affected plugin, the DGRAM mode, and the symptom (the collector never started). The changeset matches this: unix_socket_conn_add now registers only in stream mode, in_unix_socket_exit removes the collector before releasing connections, and a runtime test covers DGRAM and STREAM. The title is specific and describes the main fix, so it passes.

Full details: Linked Issues check

Explanation

The only linked issue, #8943, is closed and marked as historical context only. No active directly linked targets remain, so no linked-issue coding requirements apply. The assessment does not evaluate the PR against the closed issue's bug report. The PR objectives also state that no active directly linked targets remain.

Full details: Out of Scope Changes check

Explanation

All changes relate to the in_unix_socket DGRAM start failure described in the PR and in #8943, or to verifying it. (1) plugins/in_unix_socket/unix_socket_conn.c: registers the connection event only in STREAM mode, so the DGRAM dummy connection no longer collides with the collector registration (EEXIST). Stream error handling is kept. (2) plugins/in_unix_socket/unix_socket.c: removes the collector before releasing connections at exit. This ordering change addresses the shutdown warning that follows from the DGRAM fix. It is in the same plugin and lifecycle path. (3) tests/runtime/in_unix_socket.c and tests/runtime/CMakeLists.txt: add DGRAM and STREAM runtime tests and register them under FLB_IN_UNIX_SOCKET. These tests support the fix. No config, output format, or unrelated plugin changes were identified in the PR summary.

Finishing Touches

Generate unit tests (beta)

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 8c65f24a-63f0-4061-8a3e-6300cb60e364
Commits

Reviewing files that changed from the base of the PR and between 4aa8721 and 62d559e.

Files selected for processing: 4

  • plugins/in_unix_socket/unix_socket.c
  • plugins/in_unix_socket/unix_socket_conn.c
  • tests/runtime/CMakeLists.txt
  • tests/runtime/in_unix_socket.c

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

epoll_ctl: File exists, errno=17 when using in_unix_socket with DGRAM option

1 participant