Conversation
Implement TCP as an alternative to UDP for remote syslog forwarding, providing reliable delivery for environments where message loss is unacceptable. Forwarding syntax: - @@host:port (double-at prefix) - tcp://host:port Receiving syntax: - listen tcp://addr:port Uses octet counting framing for sending per RFC 6587. Supports both octet counting and LF-delimited framing for receiving to interoperate with various syslog implementations. Also fixes a use-after-free bug in socket_poll() where LIST_FOREACH was used while callbacks could remove entries from the socket list. Changed to LIST_FOREACH_SAFE to handle this correctly. Fixes #91 Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Verify TCP forwarding between two syslogd instances using both the @@ and tcp:// configuration syntax. Tests that messages are correctly forwarded after config reload. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Update syslog.conf(5) with TCP forwarding syntax (@@, tcp://) and TCP listener configuration (listen tcp://). Add examples showing typical usage patterns. Update syslogd(8) STANDARDS section to reference RFC 6587 for syslog over TCP. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Implements cryptographic signing of syslog messages per RFC 5848, providing origin authentication, message integrity, replay resistance, and detection of missing messages. Requires OpenSSL >= 1.1.0 and ./configure --with-openssl to enable. New configuration options: sign_sg, sign_delim_sg2, sign_keyfile, sign_certfile. Follows NetBSD's configuration syntax for compatibility. Features: - Signature blocks (ssign) with SHA-256 hashes and digital signatures - Certificate blocks (ssign-cert) for public key distribution - Four signature group modes (SG=0,1,2,3) per RFC 5848 Section 4.2.5 - Compiles cleanly with stub macros when OpenSSL is not available Fixes #21 Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Implements TLS-encrypted syslog forwarding and receiving per RFC 5425, building on the existing TCP infrastructure (RFC 6587) and reusing OpenSSL from RFC 5848 signing support. Configuration syntax: - Forwarding: tls://host:port or @@@host:port - Listening: listen tls://addr:port - Global: tls_keyfile, tls_certfile, tls_cafile, tls_capath - Per-action: verify=off|optional|required|hostname, fingerprint=SHA256:... Key implementation details: - TLS 1.2 minimum per RFC 5425 requirements - Default port 6514 for TLS syslog - Certificate verification: chain, fingerprint, or hostname - Optional mutual TLS authentication with client certificates - Deferred TLS connection until after tls_init() completes - Reuses F_FORW_TLS/F_FORW_TLS_SUSP/F_FORW_TLS_UNKN state machine Fixes #22 Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Verifies TLS forwarding between two syslogd instances using both @@@ and tls:// configuration syntax. Generates self-signed test certificates and validates message delivery through encrypted channel. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Add documentation for TLS configuration options including new global settings: tls_keyfile, tls_certfile, tls_cafile, tls_capath, listener syntax: listen tls://, forwarding syntax: tls:// and @@@, and per-action options: verify=, fingerprint=. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
troglobit
force-pushed
the
next
branch
2 times, most recently
from
2a6c16a to
3b2c2ee
Compare
Inspired by OpenBSD syslogd, introduce a double-prefix syntax for block headers that stops rule evaluation once a matching rule fires, preventing a message from being logged twice. !!prog -- like !prog but stops on match ++host -- like +host but stops on match ::filter -- like :filter but stops on match (sysklogd extension) Use !*, +*, or :* as usual to reset and resume normal evaluation for subsequent blocks. The STOP_FLAG (0x040) is set on all filed entries created within a double-prefix block. The logmsg() dispatch loop breaks out of the SIMPLEQ_FOREACH as soon as it processes a rule carrying that flag, so no further rules see the message. Example -- route nftables firewall logs exclusively to their own file: ::msg, contains, "[nftables" kern.* -/var/log/nftables.log :* *.warn;authpriv.none -/var/log/syslog Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Stop-processing block prefixes (!!, ++, ::) can only suppress rules that follow in the parsed order. A snippet in /etc/syslog.d/ cannot gate rules that were already parsed from the main config file above the include line. Fix this by moving the include directive in the shipped syslog.conf to sit before the general catch-all rules, so snippets are evaluated first and their stop-blocks take effect. Document the ordering constraint in two places in syslog.conf(5): in the include description, where the interaction is explained and the shipped file's structure is called out; and in the Stop Processing example section, where a direct note warns users who place stop-blocks in syslog.d/ snippets. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
- Fix .Sh DESCRIPTIOMN typo in man page - Fix chomp() indentation, spaces -> tabs - Fix nslookup() EAI_SERVICE warning, drop hardcoded "/udp" suffix - Bump message buffer from 512 to 2048 bytes (MAXLINE), matching syslogd's own limit to avoid silent truncation of long messages Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
The -h option now accepts a transport prefix, matching the forwarding action syntax used in syslog.conf: logger -h tcp://logserver:514 "test" logger -h udp://logserver "test" # explicit UDP, same as before logger -h logserver "test" # plain host, UDP (backward compat) IPv6 addresses require bracket notation: tcp://[::1]:514 An optional port embedded in the URL overrides -P. The TCP path bypasses the syslog library entirely and handles DNS resolution (SOCK_STREAM), connection, message formatting (RFC5424 by default, RFC3164 with -b), and RFC 6587 octet-count framing directly, matching the framing used by syslogd's own TCP forwarding path. A tls:// prefix is recognised and rejected with a clear error for forward compatibility. Also change struct sockaddr to sockaddr_storage throughout to correctly handle IPv6 addresses in the UDP path as well. Note: bare IPv6 address literals (e.g. ::1) passed to -h are not supported; bracket notation must be used. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
When -V is given, diagnostic output is written to stderr showing
what logger is actually doing, making it easy to verify a remote
syslogd configuration:
TCP:
connecting to 192.0.2.1 port 514 (tcp) ... connected
sending (82 bytes): <28>1 2026-03-20T14:30:00.123456+01:00 ...
UDP:
sending to 192.0.2.1 port 514 (udp)
For TCP, the exact RFC 6587-framed syslog message is shown, allowing
the user to confirm the message content matches the filter rules in
syslog.conf. Connection failures are reported on the same line as
the connect attempt, giving a clear pass/fail result.
For UDP only the resolved peer is shown; delivery cannot be confirmed
since the transport is fire-and-forget.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Add two steps to the existing TCP forwarding test: - verify_direct_tcp: send directly from logger to the receiver's TCP listener using logger -h tcp://[::1]:PORT, bypassing the forwarding syslogd entirely. Confirms the RFC 6587 framing and message delivery end-to-end. - verify_verbose_tcp: invoke logger -V and check that "connected" appears on stderr, exercising the new verbose mode added alongside the TCP transport feature. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Verify that the :: (propfilter), !! (program), and ++ (hostname) stop-block prefixes prevent a matched message from reaching subsequent rules, while non-matching messages continue to fall through normally. Eight steps cover both positive (message lands in the right log) and negative (message absent from the wrong log) assertions for the propfilter and program-filter variants. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
When a TCP receiver goes down, the sender previously dropped all messages
for the full suspension window (default 180 s). This change retrofits a
per-destination FIFO queue so that messages accumulate during outages and
are flushed automatically on reconnect (inspired by the NetBSD GSoC 2008
work).
New data structures (syslogd.h):
- struct fwd_qentry / fwd_qhead -- RFC 6587 framed queue entries
- FORW_QUEUE_MAX_LEN (1000) and FORW_QUEUE_MAX_SIZE (1 MiB) limits
- Four new fields on struct filed: f_queue, f_qlen, f_qsize, f_qoverflow
New helpers (syslogd.c):
- tcp_build_frame() flatten iovec into a single "LEN SP MSG" frame
- forw_queue_enqueue() add a frame; evict oldest on overflow
- forw_queue_flush() drain the queue over a live socket
- forw_queue_clear() discard all entries (SIGHUP / shutdown)
Behaviour changes:
- F_FORW_TCP_SUSP: enqueue instead of drop during the backoff window
- F_FORW_TCP_UNKN: enqueue when DNS is still unresolved
- F_FORW_TCP: flush any backlog before/after a successful send
- close_open_log_files(): always call forw_queue_clear() to prevent leaks
New syslog.conf global directive:
tcp_suspend_time N override the per-TCP-destination backoff (default
INET_SUSPEND_TIME = 180 s); useful in tests and for
operators who want faster failover.
Two bugs found and fixed during testing:
1. fprintlog_first() was resetting f_time for all filed types except the
two original UDP suspended/unknown states (F_FORW_SUSP, F_FORW_UNKN).
The TCP and TLS equivalents were missing from the guard, so the
suspension start timestamp was clobbered on every arriving message,
making fwd_suspend always 0 and the backoff window never expire.
Fixed by extending the exclusion to F_FORW_TCP_SUSP/UNKN and
F_FORW_TLS_SUSP/UNKN.
2. The SUSP → reconnect path transitioned through F_FORW_TCP_UNKN /
forw_lookup(), which silently skips re-resolution (and thus
tcp_connect()) when timer_now() - f_time < INET_DNS_DELAY (60 s).
With a tcp_suspend_time shorter than 60 s the reconnect never fired.
Fixed by jumping directly to f_forw_tcp (bypassing forw_lookup) when
the suspension window expires; the stored f_addr is still valid.
New regression test: test/tcp-queue.sh
Kills the receiver, injects three messages during the outage, restarts
the receiver, sends a trigger after tcp_suspend_time expires, and verifies
that all four messages arrive in order.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
syslog.conf(5): - Add tcp_suspend_time to the synopsis block - Document the directive in the global options section: default value, suspension/queue/flush cycle, queue bounds (1000 msgs / 1 MiB), overflow eviction policy, and the SIGHUP/exit discard caveat - Add a paragraph in the Remote Machine action section explaining the queue from the operator's perspective when configuring @@ / tcp:// forwarding targets syslogd(8): - Note in the HUP signal entry that queued TCP messages are discarded on reload - Extend the RFC 6587 TCP sentence in STANDARDS to mention the per-destination send queue and cross-reference tcp_suspend_time Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
- bump GitHub Actions to Node.js 24 compatible versions - simplify, skip apt update - prevent duplicate jobs Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Drop one job when pushing to a branch that has a PR. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Many tests grep a logfile the instant logger returns, but syslogd drains its socket and writes asynchronously, so the line is often not there yet and the grep fails. Surfaced as an intermittent clang-only local.sh and gcc-only tag.sh ERROR in CI, but it is a timing race in the tests, not a compiler bug. Route the positive log checks through the existing tenacious() helper, as api.sh and mark.sh already do, so they poll instead of grepping once. This also replaces the fixed sleeps before the forwarding greps, which both slowed the suite and still raced. Absence checks (verify_not/check_not) keep the single-shot grep. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Verify that ;pri makes syslogd print the <PRI> code in the log line, and that the same message logged without ;pri does not carry it. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Verify that logger parses a leading <LEVEL> on messages read from stdin, overriding the severity and stripping the prefix, and that a <LEVEL> in a message given as an argument is left untouched. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
stop.sh exercised only the :: and !! prefixes; add a ++host block so all three documented stop-processing forms are covered. Reset each block (:* !* +*) before the next so its filter does not leak into the following rules, and poll for the log line via tenacious() instead of a fixed sleep. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
A :filter block heading more than one rule applied the filter only to the first rule; the rest were left unfiltered and syslogd logged "failed parsing property filter '(null)'" for each at startup. prop_filter_compile() tokenizes with strsep() and de-escapes the value in place, mutating the caller's buffer. cfparse() reuses that same buffer for the remaining rules in the block, which by then is truncated at the first delimiter. Compile on a private copy so the caller's buffer is left intact. Add a property.sh regression case: a :filter block over two rules, asserting the filter applies to both and a non-matching message reaches neither. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
The tcp4://, tcp6://, tls4://, and tls6:// URL prefixes were parsed but the 4/6 selector was discarded, so the address family was never forced and, e.g., tcp4://host could be delivered over IPv6. Thread a per-action address family through nslookup() and set it from the URL prefix. A per-action tcp4/6:// now overrides the global -4/-6. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Per-action tls_keyfile=/tls_certfile= were stored but never loaded, so a TLS forwarder presented no client certificate; and verify=optional and verify=required behaved identically (both SSL_VERIFY_PEER, with no SSL_get_verify_result() check). Load the per-action client certificate onto the connection, and split client verification into helpers shared by tls_connect() and tls_connect_continue(): required and hostname demand a trusted, valid chain (hostname also matches the name), optional verifies best-effort and proceeds with a warning. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
sign_send_sig_block() and sign_send_cert_block() assembled the [ssign] and [ssign-cert] structured data but discarded it, emitting only a status line, so no block ever reached a destination. Inject the block as an RFC5424 message via logmsg(), guarded against re-hashing its own output, and defer a full group's emission until after the destination loop. Advance the message counter once per message rather than once per destination; signature-group message numbers therefore differ from earlier 3.0 betas. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Pin a self-signed server certificate by its SHA-256 fingerprint (no CA, chain not validated) and verify the forward is accepted; a mismatching fingerprint must be rejected. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
A TLS receiver bound to [::1] only: tls6://localhost is delivered while tls4://localhost is blocked (forced to IPv4, where nothing listens). Mirrors the tcp4/6 family test; verify=off isolates family selection from certificate checks. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
The stderr writev was passed piovcnt + 1 iovecs, but only piovcnt are initialised, so it read one uninitialised iovec: nondeterministic, and -1 EFAULT under strace. Write exactly piovcnt. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
The PERROR/CONS echo skips the "MSGID SD " prefix to show only the message, using strlen() of the format string as the offset. When the SD holds conversions (e.g. iut="%d") the expanded prefix is a different length, so the echo started mid-SD. Measure the expanded prefix with vsnprintf(NULL, 0, ...) on a va_copy of the arguments instead. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
example.c barely scratched the API. Add basic.c (classic syslog(), priority mask, %m), structured.c (RFC5424 MSGID + structured data), reentrant.c (the _r API with a local struct syslog_data), and multicast.c (log_host/log_iface/log_ttl), wire them into example.mk and Makefile.am, and list them in the README. LOG_PERROR echoes each to stderr so they show output when run from a terminal. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Two features shipped without their user-facing docs: logger's <LEVEL> stdin priority prefix was missing from logger.1, and the stop-processing block prefixes (!!prog, ++host, ::filter) were absent from the README feature list. Also note logger's UDP/TCP/TLS transport and -V verbose mode in the README logger entry. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
The RFC 5848 section documented the sign_* options but never showed the emitted output; add a complete example (config plus sample [ssign-cert] and [ssign] blocks, which are themselves RFC5424 messages). Readers who scroll straight to EXAMPLES had no answer to "how do I make a change take effect?" Add an "Applying Changes" how-to at the top of the section (SIGHUP via the PID file, or systemctl reload), and name SIGHUP plus the reload command in syslogd.8. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Answer the questions newcomers actually ask: - a "TLS Certificates and Trust" how-to: what is private vs public, which option holds what (server identity, client CA, mutual TLS, fingerprint pinning), and a copy-paste openssl CA/server recipe. - a "Verifying signatures" note for RFC 5848: how a consumer detects tampering, the limitation that syslogd only produces signatures and ships no verifier (as in the NetBSD lineage it derives from), and that signing is orthogonal to TLS and works with remote forwarding. - note that NetBSD's '+' / '+-' destination prefix maps to the ;RFC5424 and ;pri options here; the '-' no-sync prefix is identical. Also, relocate and update the misplaced 224.0.0.0/4 interface route. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
The Introduction repeated itself across prose and a 30-item, changelog-ordered bullet list. Drop the hand-maintained table of contents (GitHub renders one), fold the duplicated daemon description into the component paragraph, condense the logger paragraph, and regroup the feature bullets by topic (transports, filtering, operation, build) without dropping a feature. Add a one-line config-reload tip after install. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
NetBSD prefixes a file destination with '+' to preserve the priority in the output (so RFC 5848 signatures can be verified) and '-' to skip the fsync, combined as '+-'. sysklogd honored only '-'; a NetBSD config using '+/path' failed to parse. Accept '+' and '-' in any order before a file or pipe action; '+' sets the priority flag (equivalent to the ;pri option). Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Embedded systems often keep logs in RAM to spare flash, leaving no way to read recent log activity without a file destination. Add an in-memory ring buffer, enabled with the new syslog.conf directive membuf PATH SIZE [USER:GROUP] and a logread(1) client, in the style of BusyBox and OpenWRT. Every message is captured in a size-bounded ring, oldest evicted first, and served to logread over a UNIX domain control socket at PATH. The buffer is disabled by default. Capture reuses the normal rule pipeline: a catch-all F_MEMBUF rule is prepended to the rule list so it sees every message ahead of any rule that stops processing, and the assembled line is appended to the ring in fprintlog_write(). Duplicate suppression is skipped for the buffer so logread sees every message verbatim. Access is the socket's ownership and permission. Connecting needs write permission, so the socket is mode 0660 owner:group; USER:GROUP sets the owner, defaulting to the adm group if present, else root only at 0600. An unresolvable user or group, or a failure to apply ownership, falls back to root-only rather than widening access. The example syslog.conf is generated from syslog.conf.in so the sample membuf path matches the build's runstatedir. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Verify the in-memory buffer captures messages, that logread dumps them, filters with -e, and follows with -f, and that the control socket is owned by root with no access for other. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
RFC 5848 signing and RFC 5425 TLS are core features now; default to building them. Mirror the logger/logread options: --without-openssl disables, and a missing library is an explicit error pointing at the opt-out rather than a silent downgrade. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Hardened builds (Buildroot) compile with _FORTIFY_SOURCE, which tags write/writev/read/system/truncate/ftruncate/chdir with warn_unused_result. A (void) cast does not silence that attribute, so these best-effort I/O calls warned despite -Wno-unused-result. Consume the result in a conditional instead. logger's rotate truncate now logs on failure, matching its sibling rotate path. The rotate gzip in syslogd stays silent on purpose: a WARN there re-enters logrotate before the file is truncated, recursing. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Testing of
next