Skip to content

Pthread writer - #151

Open
ecbadeaux wants to merge 6 commits into
mainfrom
pthread-writer
Open

ecbadeaux wants to merge 6 commits into
mainfrom
pthread-writer

Conversation

@ecbadeaux

@ecbadeaux ecbadeaux commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Replaces the single shared buffer in the writer with per-thread buffering and one background sender thread. Recording metrics from several threads is now safe and fast in buffered mode. This also adds a configurable flush interval and full-buffer policy.

Changes

  • Per-thread buffered writer (libs/writer/writer_wrapper/buffered_writer.{h,cpp})
    • Each recording thread appends lines to its own buffer and hands it to a single sender thread when the buffer is full. Threads don't touch each other's buffers.
    • Datagrams contain only whole lines and are never larger than the buffer size. The one exception is a single line that is longer than the buffer, which goes out on its own.
    • Lines from one thread keep their order.
    • Each thread can have up to 8 full buffers waiting for the sender.
    • When a thread exits, its lines go to the sender. When the process shuts down, everything still buffered is sent.
  • New WriterConfig options (passed through Config → Registry → Writer::Initialize)
    • flushInterval: defaults to 3 s. The sender sends partly filled buffers at this interval so lines from quiet threads aren't held back. It must be between 0 (exclusive) and 30 s, or the constructor throws std::runtime_error.
    • fullBufferPolicy: what a thread does when it runs out of buffers.
      • Block (default): wait for the sender.
      • Drop: drop the line and log the drop count.
      • Send: the thread sends its oldest buffer itself.
  • Writer refactor: removes the old mutex/condition-variable buffering. Writer::Write is now static, and the meter headers call Writer::Write(line) directly.
  • UDSWriter: adds an optional receiverFullWait. On macOS a send can fail with "no buffer space"; the writer now retries with backoff for up to 250 ms in buffered mode instead of failing right away.
  • Performance test: adds the full-buffer policies and flush intervals to the scenarios, report tables and CSV output.
  • Docs: updates the top-level README.md and libs/writer/README.md to describe buffering behavior and settings.

Testing

  • New buffered_writer_test suite (test_buffered_writer.cpp)
  • Extended tests for WriterConfig, Config, Writer, UDSWriter and Registry
  • Updated the writer performance test to cover the new settings

Everett Badeaux and others added 6 commits November 5, 2026 11:47
The destructor of the thread key marked an exited thread in the key and
renewed the mark in every round of key destructors, so it also ran in the
last round. ThreadSanitizer on Linux lets go of the thread in that round,
and instrumented code that runs on the thread afterwards crashes.

Mark the thread in the key only on macOS, where thread_local storage does
not last through the rounds of key destructors. Elsewhere t_threadExited
lasts until the thread is gone, so the key is left empty and the
destructor runs once.

The test that records in the last round stops one round earlier under
ThreadSanitizer on Linux.
All registries of a process share one writer, and constructing a Registry
sets that writer up again. A second Registry replaces the writer that the
first one uses, and constructing one while another thread updates a meter
can crash the process.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant