Repository navigation
Pthread writer - #151
Open
ecbadeaux wants to merge 6 commits into
Open
Pthread writer#151ecbadeaux wants to merge 6 commits into
ecbadeaux wants to merge 6 commits into
Conversation
The destructor of the thread key marked an exited thread in the key and renewed the mark in every round of key destructors, so it also ran in the last round. ThreadSanitizer on Linux lets go of the thread in that round, and instrumented code that runs on the thread afterwards crashes. Mark the thread in the key only on macOS, where thread_local storage does not last through the rounds of key destructors. Elsewhere t_threadExited lasts until the thread is gone, so the key is left empty and the destructor runs once. The test that records in the last round stops one round earlier under ThreadSanitizer on Linux.
All registries of a process share one writer, and constructing a Registry sets that writer up again. A second Registry replaces the writer that the first one uses, and constructing one while another thread updates a meter can crash the process.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces the single shared buffer in the writer with per-thread buffering and one background sender thread. Recording metrics from several threads is now safe and fast in buffered mode. This also adds a configurable flush interval and full-buffer policy.
Changes
libs/writer/writer_wrapper/buffered_writer.{h,cpp})WriterConfigoptions (passed throughConfig→Registry→Writer::Initialize)flushInterval: defaults to 3 s. The sender sends partly filled buffers at this interval so lines from quiet threads aren't held back. It must be between 0 (exclusive) and 30 s, or the constructor throwsstd::runtime_error.fullBufferPolicy: what a thread does when it runs out of buffers.Block(default): wait for the sender.Drop: drop the line and log the drop count.Send: the thread sends its oldest buffer itself.Writerrefactor: removes the old mutex/condition-variable buffering.Writer::Writeis now static, and the meter headers callWriter::Write(line)directly.UDSWriter: adds an optionalreceiverFullWait. On macOS a send can fail with "no buffer space"; the writer now retries with backoff for up to 250 ms in buffered mode instead of failing right away.README.mdandlibs/writer/README.mdto describe buffering behavior and settings.Testing
buffered_writer_testsuite (test_buffered_writer.cpp)WriterConfig,Config,Writer,UDSWriterandRegistry