scrape-manager: add bounded concurrent and sequential scrape scheduler by bwplotka · Pull Request #49 · prometheus/scrape.rs

bwplotka · 2026-11-12T08:41:32Z

Follows up on prometheus/scrape.rs#48. Part of prometheus/scrape.rs#36.

Adds prometheus_scrape_manager::scrape::ScrapeManager with bounded concurrent jittered target scheduling alongside sequential modes. In SchedulingMode::Concurrent, all targets sit in a single min-heap ordered by deterministic 64-bit FNV-1a phase offsets modulo scrape_interval, while three separate limits decouple network wait concurrency from payload memory: max_in_flight bounds simultaneous HTTP requests waiting on TCP connect and response headers with zero payload buffers allocated, max_buffered_bodies bounds reusable Vec<u8> buffers (and paired resettable gzip inflate states) in BodyBufferPool acquired only after HTTP 200 OK headers arrive, and max_decoders bounds cooperative ArrowDecoder instances in DecoderPool. SchedulingMode::Sequential remains available with both a job-wide cycle tick via SequentialSchedule::CycleTick and per-target jittered offsets via SequentialSchedule::Jittered.

Initial commits in reading order (followed by incremental review-fix commits):

08a946b: add ScrapeConfig, BodyBufferPool, DecoderPool, and ReqwestFetcher with Prometheus scrape headers and streaming gzip size caps.
abba902: add TargetSchedule and ScrapeManager driving !Send-compatible FuturesUnordered concurrent and sequential loops.
aac8220: add benchfmt.rs scrape cycle benchmarks and document scheduling and pool contracts in specs/SPEC.md.

Benchmarks via benchstat

Base commit 159418e on main has an empty prometheus-scrape-manager crate, so benchstat -col /mode at 6906480 compares mode=sequential_tick as the baseline column against mode=concurrent_k8_b2_d1 (max_in_flight=8, max_buffered_bodies=2, max_decoders=1) and mode=concurrent_k8_b8_d4 (max_in_flight=8, max_buffered_bodies=8, max_decoders=4) across 16 targets serving alltypes.237mfs.prom.txt on a single-threaded Tokio runtime.

With ~2ms (header_wait=2ms) per-target HTTP header wait, concurrent_k8_b2_d1 cuts cycle time by 60.32% and raises throughput by 152.02% while matching concurrent_k8_b8_d4 within 3% using a single pooled decoder and at most two body buffers. The +7.34% delta in B/op and +0.22% delta in allocs/op (~7.7 KiB and 1.1 allocations per target scrape) come from the per-scrape FuturesUnordered task frame (including DecodeCursor state so PooledDecoder and PooledBodyBuffer release before final on_batch awaits) and OwnedSemaphorePermit lease rather than per-sample allocations.

goos: linux
goarch: amd64
pkg: prometheus-scrape-manager
cpu: Intel(R) Xeon(R) CPU @ 2.20GHz
                                          │ sequential_tick │        concurrent_k8_b2_d1         │        concurrent_k8_b8_d4         │
                                          │     sec/op      │   sec/op     vs base               │   sec/op     vs base               │
ScrapeCycle/targets=16/header_wait=none-2       19.58m ± 3%   19.51m ± 2%        ~ (p=1.000 n=6)   19.70m ± 1%        ~ (p=0.818 n=6)
ScrapeCycle/targets=16/header_wait=2ms-2        57.93m ± 2%   22.99m ± 3%  -60.32% (p=0.002 n=6)   22.30m ± 1%  -61.51% (p=0.002 n=6)
geomean                                         33.68m        21.18m       -37.12%                 20.96m       -37.76%

                                          │ sequential_tick │          concurrent_k8_b2_d1          │          concurrent_k8_b8_d4          │
                                          │       B/s       │      B/s       vs base                │      B/s       vs base                │
ScrapeCycle/targets=16/header_wait=none-2      148.3Mi ± 3%    148.8Mi ± 2%         ~ (p=1.000 n=6)    147.3Mi ± 1%         ~ (p=0.818 n=6)
ScrapeCycle/targets=16/header_wait=2ms-2       50.09Mi ± 2%   126.24Mi ± 2%  +152.02% (p=0.002 n=6)   130.14Mi ± 1%  +159.80% (p=0.002 n=6)
geomean                                        86.18Mi         137.0Mi        +59.03%                  138.4Mi        +60.65%

                                          │ sequential_tick │        concurrent_k8_b2_d1         │        concurrent_k8_b8_d4         │
                                          │      B/op       │     B/op      vs base              │     B/op      vs base              │
ScrapeCycle/targets=16/header_wait=none-2      1.651Mi ± 0%   1.772Mi ± 0%  +7.34% (p=0.002 n=6)   1.772Mi ± 0%  +7.34% (p=0.002 n=6)
ScrapeCycle/targets=16/header_wait=2ms-2       1.651Mi ± 0%   1.772Mi ± 0%  +7.33% (p=0.002 n=6)   1.772Mi ± 0%  +7.33% (p=0.002 n=6)
geomean                                        1.651Mi        1.772Mi       +7.34%                 1.772Mi       +7.34%

                                          │ sequential_tick │        concurrent_k8_b2_d1        │        concurrent_k8_b8_d4        │
                                          │    allocs/op    │  allocs/op   vs base              │  allocs/op   vs base              │
ScrapeCycle/targets=16/header_wait=none-2       8.385k ± 0%   8.404k ± 0%  +0.23% (p=0.002 n=6)   8.404k ± 0%  +0.23% (p=0.002 n=6)
ScrapeCycle/targets=16/header_wait=2ms-2        8.385k ± 0%   8.403k ± 0%  +0.21% (p=0.002 n=6)   8.403k ± 0%  +0.21% (p=0.002 n=6)
geomean                                         8.385k        8.403k       +0.22%                 8.403k       +0.22%

Comparison on header_wait=none between pre-review commit aac8220 (before.txt) and 6906480 (after.txt, +2.84% B/op is +3.1 KiB on the FuturesUnordered task state machine from holding DecodeCursor directly with 0 extra allocs/op):

goos: linux
goarch: amd64
pkg: prometheus-scrape-manager
cpu: Intel(R) Xeon(R) CPU @ 2.20GHz
                                                                   │ before.txt  │             after.txt             │
                                                                   │   sec/op    │   sec/op     vs base              │
ScrapeCycle/targets=16/header_wait=none/mode=sequential_tick-2       20.13m ± 9%   19.58m ± 3%       ~ (p=0.093 n=6)
ScrapeCycle/targets=16/header_wait=none/mode=concurrent_k8_b2_d1-2   20.08m ± 1%   19.51m ± 2%  -2.85% (p=0.002 n=6)
ScrapeCycle/targets=16/header_wait=none/mode=concurrent_k8_b8_d4-2   20.28m ± 9%   19.70m ± 1%  -2.83% (p=0.002 n=6)
geomean                                                              20.16m        19.60m       -2.82%

                                                                   │  before.txt  │             after.txt              │
                                                                   │     B/s      │     B/s       vs base              │
ScrapeCycle/targets=16/header_wait=none/mode=sequential_tick-2       144.1Mi ± 8%   148.3Mi ± 3%       ~ (p=0.093 n=6)
ScrapeCycle/targets=16/header_wait=none/mode=concurrent_k8_b2_d1-2   144.5Mi ± 1%   148.8Mi ± 2%  +2.94% (p=0.002 n=6)
ScrapeCycle/targets=16/header_wait=none/mode=concurrent_k8_b8_d4-2   143.1Mi ± 8%   147.3Mi ± 1%  +2.91% (p=0.002 n=6)
geomean                                                              143.9Mi        148.1Mi       +2.90%

                                                                   │  before.txt  │              after.txt               │
                                                                   │     B/op     │     B/op      vs base                │
ScrapeCycle/targets=16/header_wait=none/mode=sequential_tick-2       1.651Mi ± 0%   1.651Mi ± 0%       ~ (p=1.000 n=6) ¹
ScrapeCycle/targets=16/header_wait=none/mode=concurrent_k8_b2_d1-2   1.723Mi ± 0%   1.772Mi ± 0%  +2.84% (p=0.002 n=6)
ScrapeCycle/targets=16/header_wait=none/mode=concurrent_k8_b8_d4-2   1.723Mi ± 0%   1.772Mi ± 0%  +2.84% (p=0.002 n=6)
geomean                                                              1.698Mi        1.730Mi       +1.88%
¹ all samples are equal

                                                                   │ before.txt  │              after.txt              │
                                                                   │  allocs/op  │  allocs/op   vs base                │
ScrapeCycle/targets=16/header_wait=none/mode=sequential_tick-2       8.385k ± 0%   8.385k ± 0%       ~ (p=1.000 n=6) ¹
ScrapeCycle/targets=16/header_wait=none/mode=concurrent_k8_b2_d1-2   8.404k ± 0%   8.404k ± 0%       ~ (p=1.000 n=6) ¹
ScrapeCycle/targets=16/header_wait=none/mode=concurrent_k8_b8_d4-2   8.404k ± 0%   8.404k ± 0%       ~ (p=1.000 n=6) ¹
geomean                                                              8.398k        8.398k       +0.00%
¹ all samples are equal

Intentional differences from Go Prometheus

Go spawns one persistent goroutine, scrapeCache, and gzip.Reader per target in scrapePool.sync, and allocates response buffers from an uncapped sync.Pool in scrapeLoop.scrapeAndReport held until app.Commit() finishes. ScrapeManager instead keeps all target timers in a single BinaryHeap and bounds simultaneous body buffers, resettable gzip decoders, and ArrowDecoders via semaphore-gated pools, shrinking buffers above max_retained_body_bytes and resetting decoders above max_retained_decoder_bytes on drop so working-set memory stays bounded regardless of target count.
Go pins Target.offset to time.Now().UnixNano() modulo interval using xxhash64 over target labels. target_offset hashes jitter_seed, target URL, and sorted target labels with 64-bit FNV-1a relative to a monotonic tokio::time::Instant base so deterministic virtual-time tests and live reloads preserve exact intervals across wall-clock adjustments.
Go relies on go goroutines requiring thread-safe appenders, whereas ScrapeManager uses FuturesUnordered without tokio::spawn so !Send LocalSet runtimes in otel-arrow can pass Rc<RefCell<...>> sinks without synchronization overhead.

Add ScrapeTarget, ScrapeConfig, SchedulingMode, reusable BodyBufferPool and DecoderPool scratch pools with per-buffer retained capacity caps, and ReqwestFetcher with Prometheus Accept/timeout headers and bounded streaming gzip decompression. ReqwestFetcher delays leasing a PooledBodyBuffer until HTTP 2xx headers arrive so slow targets stalled on TCP connect or server-side metric generation hold zero response payload memory. Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

Add TargetSchedule and ScrapeManager to schedule targets across deterministic per-target FNV-1a phase offsets while decoupling max_in_flight HTTP header waits from max_buffered_bodies and max_decoders. Keep both cycle-tick and jittered sequential modes available for single-stream pipelines, and avoid tokio::spawn so thread-per-core !Send LocalSet runtimes can pass non-Send sinks directly. Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

Add a benchfmt.rs benchmark suite comparing sequential_tick against bounded concurrent_k8_b2_d1 and concurrent_k8_b8_d4 scrape passes with and without per-target HTTP header wait latency, and document the TargetSchedule and pool-admission contracts in specs/SPEC.md and crate READMEs. Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

nicolastakashi · 2026-11-12T13:31:28Z

+                                match cmd {
+                                    Some(ScrapeCommand::UpdateTargets(new_targets)) => {
+                                        targets = new_targets.into_iter().map(Arc::new).collect();
+                                        break;


issue: Target update mid-pass skips the rest of the CycleTick pass

manager.rs:670 drops the in-flight exec_one and breaks out of the loop when UpdateTargets arrives, which means every target after the current one waits a full interval for its next scrape. Jittered mode keeps in-flight scrapes across an update at manager.rs:750, could CycleTick apply the new list after the pass finishes instead?

nicolastakashi · 2026-11-12T13:31:28Z

+        }
+
+        let (body, wire_bytes) = tokio::time::timeout(remaining, async {
+            let mut body = pool.acquire(cap_hint).await;


issue: Waiting for a body buffer counts against scrape_timeout

http.rs:291 acquires the body permit inside the timeout(remaining, ...) block, and the permit stays held through decode and every sink.on_batch until manager.rs:358, so a slow sink makes healthy targets fail with TimedOut and up=false. In a local test with max_buffered_bodies=1, a 300ms timeout and a sink that sleeps 600ms, 3 of 4 instant targets timed out, while config.rs:259 documents scrape_timeout as the HTTP and body read timeout.

nicolastakashi · 2026-11-12T13:31:28Z

+            if !seen.insert(key.clone()) {
+                continue;
+            }
+            if self.index_by_key.contains_key(&key) {


issue: Re-adding a target that is still in flight allows overlap

schedule.rs:168 only checks index_by_key, which no longer holds the removed slot that is still scraping, so the re-add builds a fresh slot with in_flight: false and pop_due starts a second scrape of the same target while the first runs. SPEC.md:174 says an in-flight target skips its next tick, and a pod that flaps across two SD updates takes this path.

nicolastakashi · 2026-11-12T13:31:28Z

+            if !records.is_empty() {
+                batches_emitted += 1;
+                if sink
+                    .on_batch(target, fetched.scrape_timestamp_ms, records)


question: What should the sink do with batches from a failed scrape?

manager.rs:522 sends each chunk to on_batch as soon as it decodes, and a later decode error reports up=false, batches_emitted=0, so the sink already holds part of a scrape for a target marked down. Go rolls back the appender in this case (https://github.com/prometheus/prometheus/blob/v0.315.1/scrape/scrape.go#L1614-L1631), do you want a rollback signal on the sink, or a doc on on_scrape_complete saying sinks must drop those batches?

nicolastakashi · 2026-11-12T13:31:28Z

+pub fn build_accept_header(enable_open_metrics: bool, escaping: EscapingScheme) -> String {
+    let esc_param = match escaping {
+        EscapingScheme::EscapeUnderscores => String::new(),
+        other => format!(";escaping={}", other.as_str()),


issue: Accept header adds escaping to the 0.0.1 and 0.0.4 formats

http.rs:116 appends ;escaping= to every media type, while Go acceptHeader adds it only to the 1.0.0 formats because escaping is not valid for the older ones (https://github.com/prometheus/prometheus/blob/v0.315.1/scrape/scrape.go#L799-L802), which means with AllowUtf8 we send text/plain;version=0.0.4;escaping=allow-utf-8 even though SPEC.md:177 says this header matches targetScraper.scrape.

nicolastakashi · 2026-11-12T13:31:28Z

+            .map_err(|_| ScrapeError::TimedOut { timeout })??;
+
+        let status = response.status();
+        if !status.is_success() {


question: Accept any 2xx, or only 200 like Go?

http.rs:247 accepts any is_success() status, while Go readResponse fails anything other than 200 (https://github.com/prometheus/prometheus/blob/v0.315.1/scrape/scrape.go#L850), so a 204 or 206 passes here and fails in Prometheus. SPEC.md:175 says 2xx on purpose, should this go in the PR body's list of intentional differences?

nicolastakashi · 2026-11-12T13:31:28Z

+    pub escaping_scheme: EscapingScheme,
+    /// Negotiate and transparently decompress `Content-Encoding: gzip` responses when `true`.
+    pub enable_compression: bool,
+    /// Preserve explicit sample timestamps when `true`; override with scrape time when `false`.


issue: honor_timestamps=false is silently ignored

config.rs:273 says false replaces sample timestamps with the scrape time, but nothing reads the field yet (TODO at manager.rs:460), so a caller who sets it still gets the exposed timestamps with no error, could the doc say it is not wired yet, or the field wait until with_honor_timestamps lands?

nicolastakashi · 2026-11-12T13:31:29Z

+        // `self.body_pool` before `on_scrape_complete` awaits downstream work.
+        drop(fetched);
+
+        let (exit_early, report) = match decode_outcome {


nitpick: exit_early is always false

manager.rs:360 sets exit_early to false in both arms, so the exit_early || check at manager.rs:399 never fires and the tuple can go.

nicolastakashi · 2026-11-12T14:45:03Z

+        fetched: &FetchedScrape,
+        sink: &S,
+    ) -> ControlFlow<(), Result<(u64, usize), ScrapeError>> {
+        let mut decoder = self.decoder_pool.acquire().await;


issue: Decoder lease held across sink awaits caps default concurrency

manager.rs:452 takes the decoder before decode and holds it through every sink.on_batch(...).await at manager.rs:521, and concurrent() defaults to one decoder at config.rs:183, which means with a sink that sleeps 1ms per batch concurrent_k8_b2_d1 took 132.9ms/op against 133.2ms/op for sequential_tick (p=0.353) while b8_d8 took 21ms/op, and BlackBoxSink in the PR's bench never awaits so the PR's numbers can't show it, is one decoder the right default for sinks that await?

benchstat, darwin/arm64 at https://github.com/prometheus/scrape.rs/commit/aac82206fe8d695743cd2444b2a40e13399a5e88

│ sequential_tick │ concurrent_k8_b2_d1 │ concurrent_k8_b8_d4 │ concurrent_k8_b8_d8 │ │ sec/op │ sec/op vs base │ sec/op vs base │ sec/op vs base │ SlowSink/targets=16/sink_wait_per_batch=0-2 7.453m ± 1% 7.514m ± 3% +0.82% (p=0.011 n=10) 7.411m ± 1% -0.57% (p=0.000 n=10) 7.368m ± 0% -1.13% (p=0.000 n=10) SlowSink/targets=16/sink_wait_per_batch=1ms-2 133.20m ± 2% 132.92m ± 2% ~ (p=0.353 n=10) 38.87m ± 3% -70.82% (p=0.000 n=10) 20.97m ± 3% -84.25% (p=0.000 n=10)

nicolastakashi · 2026-11-12T14:45:03Z

+                    }
+                }
+                BodyEncoding::Gzip => {
+                    let mut decoder = GzDecoder::new(BoundedVecWriter {


suggestion: Reuse the gzip decoder across scrapes

http.rs:307 builds a new GzDecoder for every response and compression is on by default, so every scrape allocates 74 KiB in 3 allocations that a reset decoder avoids, which Go already does by resetting one gzip.Reader per target (https://github.com/prometheus/prometheus/blob/v0.315.1/scrape/scrape.go#L860-L874), could the inflate state live next to the pooled body buffer?

benchstat, darwin/arm64 at https://github.com/prometheus/scrape.rs/commit/aac82206fe8d695743cd2444b2a40e13399a5e88

│ new_decoder_per_scrape │ reset_reused_decoder_scratch32k │ │ B/op │ B/op vs base │ Gzip/body=alltypes.237mfs-2 74.29Ki ± 0% 0.00Ki ± 0% -100.00% (p=0.000 n=10) │ new_decoder_per_scrape │ reset_reused_decoder_scratch32k │ │ allocs/op │ allocs/op vs base │ Gzip/body=alltypes.237mfs-2 3.000 ± 0% 0.000 ± 0% -100.00% (p=0.000 n=10)

nicolastakashi · 2026-11-12T14:45:03Z

+        let mut seen = HashSet::with_capacity(targets.len());
+
+        for target in targets {
+            let key = TargetKey::from_target(&target);


suggestion: sync_targets blocks the scheduler for about 26ms at 10k targets

schedule.rs:164 clones each TargetKey twice and hashes its URL and labels on every update even when nothing changed, and an unchanged re-sync of 10k targets with 8 labels took 30.2ms against 4.6ms for cloning the input alone, which means every SD update stops timers and in-flight scrapes on that thread for about 26ms. Hashing Arc<ScrapeTarget> directly would drop the clones.

benchstat, darwin/arm64 at https://github.com/prometheus/scrape.rs/commit/aac82206fe8d695743cd2444b2a40e13399a5e88

│ input_clone_only │ unchanged_resync │ │ sec/op │ sec/op vs base │ SyncTargets/targets=1000/labels=8-2 444.9µ ± 0% 2071.1µ ± 0% +365.50% (p=0.000 n=10) SyncTargets/targets=10000/labels=8-2 4.578m ± 2% 30.241m ± 4% +560.56% (p=0.000 n=10)

nicolastakashi · 2026-11-12T14:45:03Z

+#[cfg(feature = "arrow")]
+impl Drop for PooledDecoder {
+    fn drop(&mut self) {
+        self.pool.active_leases.fetch_sub(1, Ordering::Relaxed);


question: Should DecoderPool cap retained memory like BodyBufferPool?

pool.rs:302 returns each decoder with all its scratch capacity while BodyBufferPool shrinks to max_retained_body_bytes at pool.rs:169, so in a local test retained_decoder_bytes went from 130 KB to 489 KB after one 1.4 MB scrape and stayed there after 50 small ones, which means the largest target ever scraped sets decoder memory for the life of the manager, do you want a retention cap here too?

nicolastakashi · 2026-11-12T14:45:03Z

+    /// Total target scrapes that failed HTTP fetch, size limits, or decoding.
+    pub scrapes_failed: u64,
+    /// Missed interval ticks caused by slow cycles or targets still in flight at their next tick.
+    pub scrapes_overrun: u64,


question: Can stats show when a pool limit is saturated?

ManagerStats at manager.rs:111 counts only scrapes_overrun, which needs a full missed interval, and the pools report peaks only, so in a paused-time test with max_in_flight=1 at 93% utilization scrapes started up to 3s late (p50 1.6s) while scrapes_overrun stayed at 0, would a wait count or start-lag counter fit here?

nicolastakashi · 2026-11-12T14:45:03Z

+
+    let mut suite = Suite::from_env("prometheus-scrape-manager").with_alloc(&ALLOC);
+
+    for (wait_label, header_wait) in [("none", Duration::ZERO), ("1ms", Duration::from_millis(1))] {


issue: header_wait=1ms rows measure about 2.3ms per fetch

benches/scrape.rs:143 sleeps Duration::from_millis(1) and tokio's sleep has millisecond granularity, so each fetch waits about 2.3ms, which the PR's own table shows as (55.85 - 19.22) / 16 = 2.29ms per target and means the -60% headline in the PR body is for a 2.3ms wait, could the label say 2ms?

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

…plete Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

…dule heap Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

…al counts Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

…to 2 decoders Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

… tests Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

…lSet loops Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

bwplotka added 3 commits November 12, 2026 08:40

bwplotka mentioned this pull request Nov 12, 2026

scenarios/otap-rust: enable scrape-manager, jitter, staleness, and relabeling bwplotka/prombenchy#20

Draft

nicolastakashi reviewed Nov 12, 2026

View reviewed changes

bwplotka mentioned this pull request Nov 12, 2026

scenarios: add otap-rust collector scenario bwplotka/prombenchy#19

Merged

nicolastakashi reviewed Nov 12, 2026

View reviewed changes

bwplotka added 23 commits November 12, 2026 16:48

cargo: remove unused bytes dep and scope tokio net/io-util to dev-deps

a5b1161

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

scrape-manager: remove dead exit_early flag in scrape_once_arrow

6d5eb28

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

scrape-manager: defer ScrapeConfig::honor_timestamps until #33 lands

0c13211

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

scrape-manager: mark public structs and enums non_exhaustive

e085497

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

scrape-manager: match Prometheus Accept header escaping and protocols

61a64cc

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

scrape-manager: require HTTP 200 OK on scrape responses

042e7ce

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

scrape-manager: redact URL passwords in ScrapeError and validate headers

e034adc

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

scrape-manager: harden Accept-Encoding and Content-Encoding handling

eb457dd

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

scrape-manager: unify scrape timeouts on tokio::time::Instant

7b204a0

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

scrape-manager: exclude BodyBufferPool permit wait from scrape_timeout

477d5ed

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

scrape-manager: pool reusable gzip decoder alongside body buffers

647385c

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

scrape-manager: support concatenated multi-member gzip streams

c1d1890

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

scrape-manager: cap idle ArrowDecoder scratch retention in DecoderPool

6ebe822

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

scrape-manager: avoid TargetKey clones in TargetSchedule::sync_targets

3c6e156

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

scrape-manager: prevent overlapping scrapes when flapping targets re-add

061a40d

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

scrape-manager: guard Instant deadline arithmetic against overflow

cfe0dd8

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

scrape-manager: count overruns and skip missed ticks on saturated com…

aa0ab04

…plete Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

scrape-manager: finish active CycleTick pass across UpdateTargets

590f541

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

scrape-manager: honor initial_delay on UpdateTargets and compact sche…

0044f7a

…dule heap Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

scrape-manager: document tentative on_batch contract and report parti…

f96a7cb

…al counts Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

scrape-manager: release finished decoder before on_batch and default …

ae09f73

…to 2 decoders Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

scrape-manager: add config validation and HTTP error unit tests

58cb1d6

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

scrape-manager: add cooperative Arrow decode, error report, and break…

cef34a1

… tests Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

bwplotka added 2 commits November 12, 2026 17:23

tests: synchronize HTTP header stall test with Barrier and cover Loca…

35ffd96

…lSet loops Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

benches: rename header_wait=1ms benchmark case to 2ms

6906480

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>


		let mut suite = Suite::from_env("prometheus-scrape-manager").with_alloc(&ALLOC);

		for (wait_label, header_wait) in [("none", Duration::ZERO), ("1ms", Duration::from_millis(1))] {

Conversation

bwplotka commented Nov 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

bwplotka commented Nov 12, 2026 •

edited

Loading