scrape.rs

Status: Experimental (Work in Progress)

Official Rust libraries for Prometheus and OpenMetrics metric collection pipelines: wire exposition format decoding (prometheus-expfmt), target and metric relabeling (prometheus-relabel), and service discovery and scrape management (prometheus-scrape-manager).

Overview

scrape.rs provides the building blocks required to discover targets, relabel labels, scrape HTTP endpoints, and decode Prometheus and OpenMetrics payloads in Rust:

Service discoveries
  -> Target relabeling
  -> Scrape manager / HTTP fetch
  -> Streaming expfmt parser
  -> Stage 1: family `__name__` filter pushdown
  -> Metric family assembly & string interning
  -> Stage 2: dictionary-vectorized relabeling & `honor_labels`
  -> Columnar Apache Arrow (OTAP) batches or Protobuf DTOs

The project serves two main use cases:

1. Columnar scraping for OpenTelemetry Arrow. open-telemetry/otel-arrow#3277 defines an OTAP-native prometheus_scrape receiver in Rust. scrape.rs streams scrape bytes through early __name__ family filtering, metric family assembly, and dictionary-vectorized relabeling directly into Apache Arrow (OtapArrowRecords) tables on thread-per-core !Send runtimes with bounded memory.
2. Standalone Rust Prometheus tooling. Validation harnesses such as OpenTelemetry Weaver live-checking /metrics endpoints against semantic conventions, CLIs, and custom collectors can depend on individual pure-CPU crates (prometheus-expfmt or prometheus-relabel) without pulling in async HTTP or service discovery dependencies.

Workspace crates

Crate	Path	Status	Specification	Scope
prometheus-expfmt	crates/expfmt	In progress	specs/SPEC.md	Content-Type and Accept negotiation, zero-copy streaming parsers for Prometheus Text 0.0.4, OpenMetrics 1.0, OpenMetrics 2.0, and Prometheus Protobuf, primary Apache Arrow / OTAP decoder (ArrowDecoder), and MetricFamily protobuf DTO decoder.
prometheus-relabel	crates/relabel	In progress	specs/SPEC.md	Prometheus-compatible target and metric relabeling (replace, keep, drop, hashmod, labelmap, labeldrop, labelkeep, lowercase, uppercase, keepequal, dropequal), Stage 1 __name__ filter pushdown, and Stage 2 dictionary-vectorized relabeling and honor_labels resolution.
prometheus-scrape-manager	crates/scrape-manager	Planned	TBD	Prometheus HTTP/TLS client configuration (http_config), service discovery mechanisms (discovery: static, file, HTTP, DNS, Kubernetes), and bounded target scrape pool scheduling (scrape).

Development and testing

Run the workspace unit and fixture test suite:

cargo test --workspace --all-features

Run the otel-arrow-dfe-pdata compatibility test suite:

cargo test --manifest-path tests/otap-compat/Cargo.toml

Run the Go differential fixture extractor and tests (requires Go):

cd tests/go-harness && go test ./...

Run the decoding and relabeling benchmarks in Go benchfmt format for benchstat:

cargo bench -p prometheus-expfmt --bench decode -- -bench '^BenchmarkDecode' -benchtime 1s -count 6 -cpu 2 -benchmem | tee decode.txt
cargo bench -p prometheus-relabel --bench relabel -- -bench '^BenchmarkRelabel' -benchtime 1s -count 6 -cpu 2 -benchmem | tee relabel.txt