Repository navigation
logd: fix ring buffers of 1 to 3 KiB, and smaller fixes - #11
Conversation
9a44712 to
8a2ba91
Compare
|
Gentle ping: #9 is merged and this follow-up is rebased on it (six commits). It fixes @robimarko, @Ansuel, could one of you take a look and merge it if it looks good? Thanks! |
|
It's hard for me to say anything useful by looking at the code, as I am unfamiliar with it, so I quickly ran it through Codex: Finding — P2: The new count < 0 check makes large valid lines requests return no entries. read_log() reads lines as a 32-bit unsigned value but stores it in an int. A request for 2147483648 therefore reaches log_list() as a negative count, setting back to zero. Previously, when that count exceeded the number of messages written, the request returned the available ring. Keep the count unsigned through the read path, or distinguish an omitted count without using its sign |
8a2ba91 to
153e0c5
Compare
|
Thanks for running it through Codex. Its finding is right that a The count is now unsigned from |
When a message does not fit before the end of the ring buffer, log_add() writes an end marker (an entry of size 0) where the current round ends, puts the message at the start of the buffer and looks for the new oldest entry: it walks the entries of the round that just ended from the start of the buffer, and stops at the first one that begins after the new message. With a buffer of 2 or 3 KiB (log_size 2 or 3), a round can be so short that all of its entries begin within the new message. The walk then stops at the end marker, which log_list() hands out as the oldest entry with a stale header, or it steps over the end marker into stale data of an earlier round. So logread gets entries with random sizes and timestamps, and on a 32-bit target a stale size can even wrap a pointer around, and logd can crash on it. Or the stale data sends the walk back to the start of the buffer, where it starts over and never ends: logd hangs in log_add() and stops reading /dev/log, and everything that logs blocks. Stop the walk at the end marker. When it gets there, nothing is left of the previous round, and the new message is the oldest entry. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Pierre-Yves Le Borgne <pylaterreur@gmail.com> Link: openwrt#11 Signed-off-by: Robert Marko <robimarko@gmail.com>
log_add() copies each message with strcpy() after the header of its entry, and relies on the entry fitting in the ring buffer with room for the header after it. The size of the buffer comes from "logd -S", in KiB (log_buffer_size or log_size in the system config, 64 by default), and anything from 1 up is taken. With a 1 KiB buffer, an entry with a message of more than 959 characters does not fit, and from 992 characters on strcpy() writes past the end of the buffer: a heap buffer overflow in logd, which any local user can trigger, as /dev/log is world-writable. Never make the buffer smaller than an entry with the longest message, LOG_LINE_SIZE bytes with the NUL, and the header after it: 1088 bytes with the 32-byte header of a 64-bit time_t, which is what "logd -S 1" now gets. Messages from /dev/log and ubus are cut to LOG_LINE_SIZE already, but a line from /proc/kmsg can be a little longer (up to 1023 characters from the kernel, plus the timestamp), so cut those too. A buffer this small relies on the previous commit: there, a long message often covers all of the previous round. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Pierre-Yves Le Borgne <pylaterreur@gmail.com> Link: openwrt#11 Signed-off-by: Robert Marko <robimarko@gmail.com>
log_add() strips trailing whitespace by walking back from the end of the message, and nothing stops it at the start. For a message that is empty or only whitespace, such as a datagram of a single space or newline on /dev/log, it reads the byte in front of the message, outside the buffer it was received in. If that byte is whitespace too, the walk goes on, logd writes the terminating NUL in front of the buffer and stores an entry of size 0, which looks like the end of a round. Stop at the first character. A message of only whitespace is still stored as an empty one. Fixes: 2c0d9cf ("logd: move stripping of newlines to log_add()") Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Pierre-Yves Le Borgne <pylaterreur@gmail.com> Link: openwrt#11 Signed-off-by: Robert Marko <robimarko@gmail.com>
Until the first message comes in, the ring buffer is all zeroes, and log_list() still returns its first slot as the oldest entry. So "logread" on a logd that has no message yet, e.g. right after "/etc/init.d/log restart", prints Thu Jan 1 00:00:00 1970 kern.emerg kernel: and "ubus call log read" returns that entry with a time of 0. Return nothing while the buffer is empty: "oldest" and "newest" are then both at its start, and they never meet again once an entry is in. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Pierre-Yves Le Borgne <pylaterreur@gmail.com> Link: openwrt#11 Signed-off-by: Robert Marko <robimarko@gmail.com>
The ids of the messages come from current_id, a signed int. After 2^31 messages, about eight months at 100 messages a second, incrementing it is undefined behaviour, and in practice it goes negative. log_list() then takes every entry for one of the last "count", so "logread -l 1" prints the whole buffer until 2^31 more messages have come in. When the counter is back at 0, the entries from before have the highest ids, and the same happens until they are overwritten. Make the counter unsigned, and tell the last "count" entries by how far their id is behind it, which works across the wrap around. The count is unsigned from the "lines" of the request on, so a value of 2^31 or more is a large count, not a negative one that gets no entries. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Pierre-Yves Le Borgne <pylaterreur@gmail.com> Link: openwrt#11 Signed-off-by: Robert Marko <robimarko@gmail.com>
Each entry starts right after the text of the one before, padded to 4 bytes. But the header holds a struct timespec, which needs 8-byte alignment on 64-bit targets and on most 32-bit ones with a 64-bit time_t, so about every second entry is misaligned. That is undefined behaviour, which UBSan reports as runtime error: member access within misaligned address ... for type 'struct log_head', which requires 8 byte alignment and on 64-bit MIPS and RISC-V, the time is read and written with 8-byte loads and stores, which trap on a misaligned address unless the CPU handles it in hardware. Pad the text of the entries to the alignment of their header. This costs at most 4 more bytes per entry, about 2% fewer entries in the buffer with messages of up to 100 characters, and nothing on 32-bit x86. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Pierre-Yves Le Borgne <pylaterreur@gmail.com> Link: openwrt#11 Signed-off-by: Robert Marko <robimarko@gmail.com>
153e0c5 to
9555709
Compare
|
Thanks! Rebased on top of master and merged! |
logd: fix ring buffers of 1 to 3 KiB, and smaller fixes
This follows #9, which is merged; this branch is rebased on it. With
log_buffer_size(orlog_size) set to 1, 2 or 3, logd breaks; the default of 64 KiB is not affected.-S 2logd got stuck at 100% CPU, stopped reading/dev/log(senders block) and stopped answering ubus.strcpy()write past the end (ASan: WRITE of size 1004 for a 1000-characterloggermessage);/dev/logis world-writable. The buffer is now never smaller than oneLOG_LINE_SIZEentry plus a header (1088 bytes). Kernel lines are cut toLOG_LINE_SIZE, which is the one change at normal sizes.Thu Jan 1 00:00:00 1970 kern.emerg kernel:line on an empty buffer;logread -l Nreturning everything after 2^31 messages; entries misaligned for their 8-byte-aligned header (UBSan; 8-byte loads on 64-bit MIPS and RISC-V). They are separate commits; any can be dropped.Tested:
log/syslog.cand checks every walk under ASan and UBSan, from 1088 bytes to 128 KiB, on 64-bit and on 32-bit with bothtime_tsizes. Before: hangs at 1-2 KiB, bad entries at 3 KiB, overflows at 1 KiB, crashes on 32-bit. After: 750+ runs clean.-S 1to-S 8and the FS#4118 reproducer at 16/64 KiB, with no bad line, crash or sanitizer report.log_addis unchanged.Behaviour changes to note:
-S 1gives a 1088-byte buffer; kernel lines over 1023 characters are now cut at every buffer size; 8-byte padding holds about 2% fewer entries with short messages.🤖 Generated with Claude Code
https://claude.ai/code/session_01GwUvhmzgfkiZ6vo4uGyesN