linux-ringbuf-driver

A multi-channel Linux character device driver that turns a circular buffer into /dev/ringbuf0 .. /dev/ringbuf3, with blocking and non-blocking I/O, epoll/select support, sysfs telemetry, an ioctl control interface and an integrity-verified test suite.

What it is

Four independent FIFO channels, each with its own ring buffer, lock, wait queues and sysfs node. Data written to one channel never blocks another. The driver is small enough to read in one sitting (~680 lines) but exercises the parts of the kernel that stream/telemetry drivers actually live in: wait queues, poll(), copy_to_user() on a faulting user buffer, an ioctl ABI that stays compatible between 32- and 64-bit user space, and module lifetime/refcounting.

Device nodes are created by udev as root:root; the default mode on most systems is 0600, so the demo below runs as root (sudo -i). To use another account instead, sudo chmod 666 /dev/ringbuf[0-3] or install a udev rule.

$ sudo make load                     # insmod + /dev/ringbuf[0-3]
$ echo "hello" > /dev/ringbuf0       # 6-byte message, buffered
$ dd if=/dev/ringbuf0 bs=6 count=1   # read it back: "hello"
$ cat /sys/class/ringbuf_class/ringbuf0/stats
Channel ID:          0
Capacity:            65536 bytes
Utilization:         0 bytes (0%)
Watermark:           80% (52428 bytes)
Head Offset:         6
Tail Offset:         6
Total Written:       6 bytes
Total Read:          6 bytes
Dropped Messages:    0
Dropped Bytes:       0
High Watermark Hit:  NO

What this demonstrates

Area	Where to look
Character device plumbing (cdev, dev_t, minor -> instance, class/device)	ringbuf_driver.c: ringbuf_init()
Blocking I/O with interruptible waits and lost-wakeup-safe re-checks	ringbuf_read(), ringbuf_write()
poll() / epoll readiness, edge-triggered draining, lock-free snapshots	ringbuf_poll()
ioctl ABI design: fixed-width UAPI types, _IOC_* validation, compat_ioctl	ringbuf_ioctl.h, ringbuf_ioctl()
sysfs attributes with input validation	stats_show(), watermark_store(), clear_store()
Concurrency reasoning (mutex vs spinlock, wait queues, atomic sizing)	docs/DESIGN.md
Defensive error handling and unwinding on init failures	ringbuf_init(), err_channels:
Kernel coding style + checkpatch --strict hygiene	make check
Measurement instead of guessing: throughput, CPU efficiency, syscalls	user_test.c section 5
Testing that can actually fail: sequence-verified integrity checks	user_test.c sections 1-4

Design in one picture

  /dev/ringbufN        ringbuf_channel                 waits on
  -------------        ---------------                 --------
  write(2)  -------->  +---------------------+ <------- write_queue: "space"
                        | buffer[capacity]    |
  read(2)   <--------  |  head / tail / count| <------- read_queue:  "data"
                        +---------------------+
  poll(2)   -------->  READ_ONCE(count) -> EPOLLIN/EPOLLOUT
  ioctl(2)  -------->  CLEAR | RESET | GET_STATUS
  sysfs     <--------> stats | watermark | clear

Highlights, with the reasoning in docs/DESIGN.md:

• Message-atomic writes. A write() either buffers the whole message or fails: -EMSGSIZE if it can never fit, -EAGAIN/blocking wait if it does not fit right now. Records are never torn, and readers never see half a record.
• No spurious "EOF". Blocking reads re-check the buffer under the lock after a wakeup, so a racing reader can never observe an empty channel and return 0 bytes (read(2) would interpret that as end of file).
• Lock-free poll(). Readiness is taken from a READ_ONCE() snapshot, so epoll_wait() never queues behind a writer that is faulting in a large copy_from_user(). Every state change is followed by a wakeup, so no readiness is lost.
• Power-of-two ring. Capacity is validated at load time and index arithmetic uses & mask, keeping a division out of the hot path; the high-watermark percentage is precomputed into a byte threshold for the same reason.
• Honest instrumentation. dropped_msgs/dropped_bytes count refused writes, total_written/total_read count accepted bytes, and CLEAR (drop data, keep counters) is distinct from RESET (drop data, zero counters).

Quick start

Requirements: kernel headers for the running kernel (linux-headers-* or a kernel source tree), make, a C compiler, and root to load modules.

$ make                # build the module and the harness
$ make selftest       # record codec tests, no module and no root needed
$ make check          # W=1, checkpatch --strict, -Wextra -Werror (+ sparse if installed)
$ sudo make load      # insmod, creates /dev/ringbuf[0-3]
$ sudo make test      # full validation suite (integrity, epoll, benchmark)
$ sudo make unload

Distributions that build their kernel with clang (this one does) are detected automatically and the module is built with LLVM=1; override with make KBUILD_ARGS=... if needed.

A five minute demo

# window 1
$ sudo make load && dmesg | tail -n 6

# window 2 - the full gate: error paths, ioctl semantics, 2 channels x
# 2 producers with sequence verification, epoll, benchmark, sysfs
$ sudo ./benchmark.sh

# window 3 - blocking I/O by hand
$ cat /dev/ringbuf0                 # blocks: the channel is empty
# window 2
$ printf 'hello ring buffer\n' > /dev/ringbuf0
# window 3 prints "hello ring buffer" and blocks again (Ctrl-C)

# window 3 - module lifetime: an open device pins the module
$ cat /dev/ringbuf0 &               # blocked reader holds the device
# window 2
$ sudo rmmod ringbuf_driver         # rmmod: ERROR: Module ringbuf_driver is in use
$ kill %1                           # close the reader
$ sudo rmmod ringbuf_driver         # now it unloads

Validation

The test harness does not count syscalls, it verifies data. Every record carries its channel, producer id and sequence number; the consumer parses each record and fails the run on a gap, a duplicate, a reordering or a corrupted payload. Because writes are message-atomic, a reader asking for exactly one record always gets exactly one record - so a short read is itself a bug and is reported.

Section	What it covers
1	Error paths: EAGAIN, EMSGSIZE, ENOTTY, exact-fit and full-buffer boundaries, drop accounting
2	ioctl semantics: CLEAR keeps counters, RESET zeroes them
3	Integrity: 2 channels x 2 producers -> 1 blocking consumer, per-producer sequence checks, byte totals cross-checked against the driver
4	epoll: EPOLLIN|EPOLLET reader with verification, wakeup count, EPOLLOUT empty -> full -> cleared transitions
5	Benchmark: timed 4 KiB record stream, MiB/s, CPU time, bytes per CPU-second, syscalls - all records still verified
6	sysfs: telemetry snapshot, watermark bounds, clear semantics

Every phase runs under a watchdog that interrupts the workers with SIGUSR1 and reports a failure instead of hanging. --selftest validates the verifier itself - record codec and sequence tracking - without the module, so CI catches a broken checker rather than a broken driver. It earned that on the first full run: the epoll reader expected a producer's first record to be sequence 1, so it reported its very first record as a gap even though all 4096 had arrived intact. The sequence bookkeeping now lives in one shared helper that --selftest covers.

Static gates on every push: build with W=1, harness with -Wall -Wextra -Werror, sparse, checkpatch --strict -f, self test. Runtime validation is local by design, because hosted CI runners cannot load modules - see docs/VALIDATION.md for what each check would catch, how to run KASAN/lockdep variants, and what is not covered.

Latest measured numbers: docs/RESULTS.md, including a repeatability table across three runs (969-993 MiB/s, 2.4% spread, no integrity violations in any run). Regenerate it with:

$ sudo ./benchmark.sh --bench-secs 5 --log bench-output-1.txt   # and -2, -3, ...
$ python3 tools/bench_report.py --write

Repository layout

ringbuf_driver.c        the driver (module lifecycle, data path, sysfs, ioctl)
ringbuf_ioctl.h         UAPI header shared by the driver and user space
user_test.c             validation harness (error paths, integrity, epoll, benchmark)
benchmark.sh            end-to-end script: build, self test, load, run, dmesg, unload
tools/bench_report.py   turns run logs into docs/RESULTS.md (repeatability table)
docs/DESIGN.md          design decisions and trade-offs
docs/VALIDATION.md      what is checked, how, and what it does not prove
Makefile                build, check, load, test, format targets
.clang-format           kernel style, so "make format" matches checkpatch

Limitations

• Capacity is capped at 16 MiB per channel and must be a power of two; the ring is a flat kmalloc() buffer, not vmalloc-backed pages.
• No mmap() interface, no DMA, no zero-copy paths.
• Concurrency is per channel: writers to the same channel are serialised by design (that is what makes records atomic); scale comes from multiple channels and message batching.
• Numbers in docs/RESULTS.md come from one machine and are meant for before/after comparisons, not as absolute claims.
• The module is unsigned, so a kernel with module signature enforcement logs module verification failed: ... tainting kernel at load time. That is the expected outcome for an out-of-tree build, not a driver bug: the module loads and produces no other kernel output. Sign it (scripts/sign-file) if a demo machine requires a clean log.

License

GPL-2.0. Kernel modules are derivative works of the kernel, and the driver uses GPL-only symbols; see LICENSE.