Repository navigation
relabel: add Prometheus-compatible target and metric relabeling - #40
Conversation
| } | ||
|
|
||
| // Remove all `__meta_*` labels after target relabeling. | ||
| labels.retain(|k, _| !k.starts_with(META_LABEL_PREFIX)); |
There was a problem hiding this comment.
issue: populate_labels skips Go's scrape interval and timeout checks
relabel.rs:911 goes from the address check straight to stripping __meta_*, without the interval and timeout checks Go runs at https://github.com/prometheus/prometheus/blob/236a2b1fe8e0471586ff29cb8fd8929372643650/scrape/target.go#L657-L677, so a relabel that sets __scrape_interval__="15s" and __scrape_timeout__="30s" returns Ok(Some(..)) here where Prometheus returns "scrape timeout cannot be greater than scrape interval", could we port those checks?
There was a problem hiding this comment.
addressed in 2e56efb: ported Prometheus scrape interval, scrape timeout, and boolean scrape label checks into populate_labels.
| /// `relabel.NewRegexp`. | ||
| pub fn new(pattern: &str) -> Result<Self, RelabelError> { | ||
| let anchored = format!("^(?s:{pattern})$"); | ||
| let compiled = |
There was a problem hiding this comment.
issue: \d and \w in user regexes match non-ASCII characters
relabel.rs:250 compiles the pattern with the Unicode classes of the regex crate, while Go RE2 keeps \d, \w, \s and \b ASCII only, so keep with regex: '\d+' on the value "١٢٣" keeps the sample here and drops it in Prometheus, could these classes be ASCII only, like relabel.rs:217 already does by hand for \w?
There was a problem hiding this comment.
addressed in d947456: normalized \d, \D, \w, \W, \s, \S, \b, and \B (plus \Q...\E) to Go RE2 ASCII semantics in Regexp::new.
| .name_validation_scheme | ||
| .is_valid_label_name(&cfg.target_label) | ||
| { | ||
| let lower = val.to_lowercase(); |
There was a problem hiding this comment.
issue: lowercase and uppercase don't match Go's case mapping
relabel.rs:608 and relabel.rs:617 use str::to_lowercase and str::to_uppercase, which apply multi-character mappings that Go strings.ToUpper skips, so uppercase on "straße" writes "STRASSE" here and "STRAßE" in Prometheus, and the dictionary path at relabel.rs:1306 and relabel.rs:1322 does the same, could both paths use Go's simple per-rune mapping?
There was a problem hiding this comment.
addressed in 4762483: switched Lowercase and Uppercase on both paths to Go strings.ToLower / strings.ToUpper 1-to-1 per-rune mapping.
| /// `Some(exported_name)` for each renamed dictionary entry (resolved in ascending key-length | ||
| /// order matching Prometheus `resolveConflictingExposedLabels`) and `None` for unchanged entries, | ||
| /// and `true` is returned. Because resolution operates across the batch-wide `metric_keys` | ||
| /// dictionary, any `exported_<name>` key present in `metric_keys` is treated as occupied for the |
There was a problem hiding this comment.
question: The batch-wide exported_ rename differs from Go
relabel.rs:1077 treats an exported_<name> key from any series in the batch as taken, while Go resolveConflictingExposedLabels checks only the current series (https://github.com/prometheus/prometheus/blob/236a2b1fe8e0471586ff29cb8fd8929372643650/scrape/scrape.go#L682-L697), so with target job="a" a series {job="x"} becomes exported_exported_job when another series in the batch has exported_job, where Prometheus writes exported_job, could we list this in the PR body with the Go permalink, as AGENTS.md asks?
There was a problem hiding this comment.
addressed in c494f1c: documented the intentional batch-dictionary exported_ difference with the Go resolveConflictingExposedLabels permalink in specs/SPEC.md, src/relabel.rs, and the PR body.
|
|
||
| let mut end = 0; | ||
| for (idx, ch) in body.char_indices() { | ||
| if !ch.is_alphabetic() && !ch.is_numeric() && ch != '_' { |
There was a problem hiding this comment.
issue: $1² expands to an empty string
relabel.rs:732 uses char::is_numeric, which also accepts superscripts and other non-decimal numbers, while Go regexp.extract stops at the first rune that is not unicode.IsDigit, so the replacement "$1²" with group 1 "a" reads the name "1²", expands to "" and deletes the target label, where Prometheus writes "a²", could this stop where unicode.IsDigit does?
There was a problem hiding this comment.
addressed in a91842a: restricted template variable scanning to Unicode L, Nd, and '_' so $1² expands $1 and preserves '²'.
| 1. **Pre-scrape target discovery relabeling (`relabel_configs`)**: | ||
| - Runs once per discovered target endpoint (cold path) over `relabel::LabelSet` (`BTreeMap<String, String>`) via `populate_discovered_labels` and `populate_labels`. | ||
| - Populates reserved discovery labels (`__address__`, `__scheme__`, `__metrics_path__`, `__scrape_interval__`, `__scrape_timeout__`, `__param_*`, `job`), executes all 11 Prometheus relabel actions, strips `__meta_*` labels, and defaults `instance` to `__address__`. | ||
| - Non-reserved target labels (`PopulatedTarget::public_labels()`) attach once per scrape to the batch's `ResourceAttrs` (`job` -> `service.name`, `instance` -> `service.instance.id`) rather than duplicating target labels onto every data-point attribute row. |
There was a problem hiding this comment.
question: How do metric_relabel_configs rules on job or instance see target labels?
SPEC.md:138 moves target labels to ResourceAttrs, while Go mutateSampleLabels adds them to each series before metric_relabel_configs run (https://github.com/prometheus/prometheus/blob/236a2b1fe8e0471586ff29cb8fd8929372643650/scrape/scrape.go#L650-L675), so a drop with source_labels: [job] has nothing to read in the Stage 2 dictionaries, could the spec say how these rules get the target labels?
There was a problem hiding this comment.
addressed in 1b264d7: specified in specs/SPEC.md how Stage 2 resolves rules that read or override post-honor_labels public target labels.
| - Non-reserved target labels (`PopulatedTarget::public_labels()`) attach once per scrape to the batch's `ResourceAttrs` (`job` -> `service.name`, `instance` -> `service.instance.id`) rather than duplicating target labels onto every data-point attribute row. | ||
| 2. **Stage 1 pre-staging `__name__` filter pushdown (Layer 1 ingestion)**: | ||
| - `relabel::split_metric_name_pushdown` extracts the leading prefix of `metric_relabel_configs` rules (`keep`, `drop`, `keepequal`, `dropequal`) that depend exclusively on `__name__`. | ||
| - `ArrowDecoder` evaluates `relabel::keep_metric_name` once when transitioning to a new contiguous family / series name (~200 times per scrape instead of once per sample), discarding dropped families before parsing sample floats, unescaping labels, interning strings, or staging histogram buckets. |
There was a problem hiding this comment.
question: Does Stage 1 check the family name or the series name?
SPEC.md:141 says "family / series name", and Prometheus runs metric_relabel_configs on each series name like foo_bucket or foo_count (https://github.com/prometheus/prometheus/blob/236a2b1fe8e0471586ff29cb8fd8929372643650/scrape/scrape.go#L675), so if Stage 1 checks the family name foo, keep with __name__=~"foo_count" drops the whole family, could the spec say it uses the series name?
There was a problem hiding this comment.
addressed in 1c2ed48: clarified in specs/SPEC.md and src/relabel.rs that Stage 1 evaluates the wire series __name__ (such as foo_bucket or foo_count).
| }, | ||
| ) | ||
| .bench( | ||
| "Relabel/case=scrape_2000_series/mode=two_stage_dictionary", |
There was a problem hiding this comment.
suggestion: mode=two_stage_dictionary skips work that scalar_per_sample does
benches/relabel.rs:413 clones and merges a full label set for every sample, while the loop at benches/relabel.rs:486-518 only counts mask hits over pre-interned series and never builds output labels or applies renamed_keys, so the -97% in the PR body compares different work, could the PR body say that, or could this mode build the output label sets too?
There was a problem hiding this comment.
addressed in 55a8c6a: updated two_stage_dictionary to apply renamed_keys and materialize output (key_id, val_id) attribute rows and arena dictionary updates, and noted the LabelSet vs interned-buffer distinction in the PR body.
| .name_validation_scheme | ||
| .is_valid_label_name(&cfg.target_label) | ||
| { | ||
| let digest = md5_sum(val.as_bytes()); |
There was a problem hiding this comment.
suggestion: Share the hashmod calculation with the dictionary path
The digest to modulo code is copied in eval_single_label_value_transform at L1334, so a fix in one copy won't reach the other and the scalar and dictionary paths can drift apart, could both call a fn hashmod(val: &str, modulus: u64) -> String?
There was a problem hiding this comment.
addressed in 358ce0b: extracted hashmod_value shared by apply_single and eval_single_label_value_transform.
| // Fast path when `regex` is the default `(.*)` (which matches all strings with | ||
| // `$0 == $1 == val`) and `target_label` is static: skip regex capture allocation. | ||
| if cfg.regex.is_default() && !var_in_regex_template(&cfg.target_label) { | ||
| let direct_replacement = if !var_in_regex_template(&cfg.replacement) { |
There was a problem hiding this comment.
suggestion: Share the default regex replace fast path
The same fast path is copied at L1261 in eval_single_label_value_transform, so a change to one copy can make the two paths write different labels for the same input, and a small direct_replacement(cfg, val) helper would keep them in step.
There was a problem hiding this comment.
addressed in 4869816: extracted direct_replacement shared by apply_single and eval_single_label_value_transform.
| None | ||
| }; | ||
| if let Some(res) = direct_replacement { | ||
| return if res.is_empty() { |
There was a problem hiding this comment.
suggestion: Pull the KeepExisting / DeleteTarget / SetTarget decision into one helper
The same empty, in-place and unchanged checks repeat across the Replace, Lowercase and Uppercase arms of eval_one, which hides that these arms only differ in how they compute the new value, and a helper like classify(new_val, val, in_place, is_present) would leave each arm with just that computation.
There was a problem hiding this comment.
addressed in 723f466: extracted classify_value_transform so all transform arms share one classification helper.
| Action::LabelMap => { | ||
| let mut mapped = Vec::new(); | ||
| for (name, value) in labels.iter() { | ||
| if cfg.regex.is_match(name) |
There was a problem hiding this comment.
nitpick: LabelMap runs the regex twice per label
is_match followed by captures on the same name matches twice for every label that maps, and if let Some(caps) = cfg.regex.compiled.captures(name) alone gives the same result.
There was a problem hiding this comment.
addressed in 3531f9f: dropped the redundant is_match check in LabelMap now that captures_read reuses CaptureLocations.
| pub enum Action { | ||
| /// Empty/unset action (only produced when constructing an uninitialized [`Config`] manually; | ||
| /// deserialized configs default to [`Action::Replace`]). | ||
| Empty, |
There was a problem hiding this comment.
question: Do we need Action::Empty?
Deserializing gives Replace when the field is missing and an error for "" since 2b7baff, so Empty only shows up when someone builds a Config by hand while every match on Action still has to handle it, could we drop the variant along with is_empty_action and the check in validate?
There was a problem hiding this comment.
addressed in faedc22: removed Action::Empty along with is_empty_action and the empty-action check in Config::validate.
| } | ||
|
|
||
| /// Pure-Rust RFC 1321 MD5 implementation used by [`Action::HashMod`]. | ||
| fn md5_sum(input: &[u8]) -> [u8; 16] { |
There was a problem hiding this comment.
question: Hand-written MD5 instead of the md-5 crate
This is 85 lines of hash code we have to keep correct ourselves, was skipping the RustCrypto md-5 crate a deliberate choice to avoid a dependency?
There was a problem hiding this comment.
Yes, deliberate choice to keep dependencies minimal and unsafe-free: md-5 pulls in 7 transitive crates (digest, crypto-common, block-buffer, generic-array, typenum, version_check, cfg-if, including unsafe in generic-array/block-buffer), whereas RFC 1321 MD5 is frozen and only used here for non-cryptographic 64-bit hashmod sharding, tested against Prometheus vectors and multi-block padding boundaries.
| } | ||
|
|
||
| #[test] | ||
| fn relabel_matches_prometheus_relabel_test_suite() { |
There was a problem hiding this comment.
suggestion: Turn the ported TestRelabel cases into a table
tests/relabel.rs:41 runs about 365 lines of numbered cases as separate asserts, so a failure only reports a line number inside one test, and looping over a table like Go's TestRelabel with the case index in the assert message would point straight at the failing case.
There was a problem hiding this comment.
addressed in e643e2d: converted relabel_matches_prometheus_relabel_test_suite into a table-driven loop reporting the case index and name.
| false, | ||
| &mut renamed_keys, | ||
| ); | ||
| eval_key_keep_mask(&fixture.key_dict, &stage2[0], &mut key_keep_mask); |
There was a problem hiding this comment.
nitpick: Name the stage2 configs instead of indexing
stage2[0], stage2[1] and stage2[2] depend on the rule order in build_scrape_fixture, so reordering the rules there makes an unwrap() panic far from the cause, and binding them to names like label_drop_cfg makes that link explicit.
There was a problem hiding this comment.
addressed in 4d738be: bound [label_drop_cfg, keep_env_cfg, replace_pod_cfg] by name instead of indexing stage2.
| - Non-reserved target labels (`PopulatedTarget::public_labels()`) attach once per scrape to the batch's `ResourceAttrs` (`job` -> `service.name`, `instance` -> `service.instance.id`) rather than duplicating target labels onto every data-point attribute row. | ||
| 2. **Stage 1 pre-staging `__name__` filter pushdown (Layer 1 ingestion)**: | ||
| - `relabel::split_metric_name_pushdown` extracts the leading prefix of `metric_relabel_configs` rules (`keep`, `drop`, `keepequal`, `dropequal`) that depend exclusively on `__name__`. | ||
| - `ArrowDecoder` evaluates `relabel::keep_metric_name` once when transitioning to a new contiguous family / series name (~200 times per scrape instead of once per sample), discarding dropped families before parsing sample floats, unescaping labels, interning strings, or staging histogram buckets. |
There was a problem hiding this comment.
issue: ArrowDecoder doesn't call keep_metric_name yet
grep -rn keep_metric_name src/arrow finds nothing on this branch, so the spec describes Stage 1 pushdown as current behavior when only the primitive exists, could this move to the Phase 4 plan or use "will"?
There was a problem hiding this comment.
addressed in 3eab35c: updated specs/SPEC.md to use future tense and link ArrowDecoder Stage 1/2 wiring to Phase 4 (#35).
| - The metric family assembler groups classic histogram `_bucket`, `_sum`, and `_count` lines and summary `quantile`, `_sum`, and `_count` lines (plus OpenMetrics `_created` start timestamps) into complete metric families. | ||
| - Configurable translation policies control counter `_total` and unit suffix handling and untyped series handling. | ||
| - `ArrowDecoder` owns its scratch buffers and Arrow builders without internal locks or global state, so a per-core `!Send` receiver can reuse allocations across scrapes with bounded memory. | ||
|
|
||
| ### Target and metric relabeling architecture |
There was a problem hiding this comment.
suggestion (non-blocking): Give relabel its own spec
AGENTS.md names specs/expfmt/SPEC.md as the contract for expfmt, so with this section living there the relabel module has no contract of its own, could it move to specs/relabel/SPEC.md with the README row linking there?
There was a problem hiding this comment.
addressed in 1383fb5: renamed specs/expfmt/SPEC.md to specs/SPEC.md as a single unified spec across the crate, since relabel lives in the same crate and Stage 1/Stage 2 metric relabeling is tightly coupled to ArrowDecoder's series cursor and string interners.
| }; | ||
| } | ||
| } | ||
| let Some(caps) = cfg.regex.compiled.captures(val_str) else { |
There was a problem hiding this comment.
issue: captures allocates once per dictionary value
relabel.rs:1282 calls Regex::captures for every value, which allocates a new capture buffer even when the regex does not match, so the PR benchmark spends 128 of its 193 allocs/op there and a batch with 22,003 unique values spends 22,003 allocations per replace rule. Reusing one CaptureLocations with captures_read took two_stage_dictionary from 193 to 65 allocs/op and from 20.9µs to 17.8µs in a local run, and the scalar path has the same call at relabel.rs:593.
| - `relabel::split_metric_name_pushdown` extracts the leading prefix of `metric_relabel_configs` rules (`keep`, `drop`, `keepequal`, `dropequal`) that depend exclusively on `__name__`. | ||
| - `ArrowDecoder` evaluates `relabel::keep_metric_name` once when transitioning to a new contiguous family / series name (~200 times per scrape instead of once per sample), discarding dropped families before parsing sample floats, unescaping labels, interning strings, or staging histogram buckets. | ||
| 3. **Stage 2 dictionary-vectorized relabeling and Layer 3 DataFusion SIMD execution**: | ||
| - After `ArrowDecoder` interns label keys into `key_interner` ($U_{\text{keys}}$ unique keys, typically ~32) and label values into `val_interner` ($U_{\text{vals}}$ unique values, typically a few hundred), dictionary-level primitives in `relabel` avoid per-sample regex and string allocations: |
There was a problem hiding this comment.
question: Stage 2 runs each rule over every value in the batch
SPEC.md:143 expects a few hundred unique values, but val_interner holds the values of all keys (src/arrow/columns.rs:110) and relabel.rs:1197 runs the rule on each of them, which means a scrape where each series carries a few unique values like pod or request IDs makes the dictionary path slower than running the same rules per series. In a local run with one keep and one replace rule over 2,000 series, the dictionary path took 0.78x the per-series time at 2,003 values, 1.37x at 12,003 and 1.9x at 22,003, could the bench add a high-cardinality fixture, or could the rules run only on the values seen under source_labels[0]?
There was a problem hiding this comment.
addressed in 7b87a70: added lazy SingleLabelKeepCache and SingleLabelTransformCache to evaluate only val_ids seen under source_labels[0] on surviving series, and added a 22k-value high-cardinality benchmark fixture.
| } else if in_place && is_present && res.as_ref() == val_str { | ||
| DictionaryValueTransform::KeepExisting | ||
| } else { | ||
| DictionaryValueTransform::SetTarget(res.into_owned()) |
There was a problem hiding this comment.
suggestion: SetTarget allocates a String per matching value on every scrape
relabel.rs:1295 copies each result into a new String and transforms.clear() at relabel.rs:1351 frees them on the next call, so a replace rule over 2,000 unique pod values still allocates 2,000 strings per scrape after the capture fix, even though a $1 result is always a substring of the input value, could SetTarget hold a range into the value or a Cow tied to metric_values?
There was a problem hiding this comment.
addressed in 4ad29b6: added SetTargetSlice { start, end } so single-capture replacements like $1 borrow byte ranges from metric_values without allocating.
Add a `relabel` module and Cargo feature implementing Prometheus `relabel_configs` and `metric_relabel_configs` configuration schemas, validation rules, anchored regular expressions (`^(?s:<pattern>)$`), Go-compatible template expansion (`$1`, `${1}`, `$name`, `${name}`, `$$`), all 11 relabel actions (`replace`, `keep`, `drop`, `keepequal`, `dropequal`, `hashmod`, `labelmap`, `labeldrop`, `labelkeep`, `lowercase`, `uppercase`), target label population (`populate_discovered_labels`, `populate_labels`), and `honor_labels` sample label merging.
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
…to Unset Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
…_sum Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
…andString Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
… and validate static replace target Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
…gth order Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
…s, and template edge cases Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
…zed primitives Add Stage 1 metric-name filter pushdown (split_metric_name_pushdown and keep_metric_name) and Stage 2 dictionary-vectorized primitives (resolve_conflicting_exposed_keys, eval_key_keep_mask, eval_single_label_keep_mask, and eval_single_label_value_transform) so Layer 1/2 decoders can drop filtered metric families before parsing/interning and evaluate attribute relabeling and honor_labels once per unique dictionary entry. Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
… SPEC.md Extend specs/expfmt/SPEC.md and README.md with the three-tier relabeling architecture: pre-scrape target discovery relabeling on LabelSet, Stage 1 pre-staging __name__ family filter pushdown in ArrowDecoder, and Stage 2 dictionary-vectorized honor_labels and attribute relabeling paired with Layer 3 DataFusion SIMD execution. Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
…, and LabelMap Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
…ting_exposed_keys Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
…ercase/Uppercase Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
…nt, and legacy schemes Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
…ngths Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
…in keep_metric_name Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
…exposed_keys test Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
…ivalence check Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
…ike $1² Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
…ary paths Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
… captures Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
…acements Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
…rce values Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
…two_stage_dictionary Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
…ide target labels Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
… 4 (#35) Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
…s Go Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
6a64e62 to
65f1954
Compare
Adds a
relabelmodule and Cargo feature implementing Prometheusrelabel_configsandmetric_relabel_configsconfiguration schemas, validation rules, and two-stage / dictionary-vectorized execution primitives, towards prometheus/scrape.rs#35, and unifies the repository specification at specs/SPEC.md with the three-tier relabeling design.All resolved dependencies remain compatible with the crate's Rust 1.88 MSRV.
Intentional difference from Go: whereas Prometheus
resolveConflictingExposedLabels(matched by scalarmerge_target_and_metric_labels) checksexported_<name>occupancy per series, Stage 2resolve_conflicting_exposed_keysresolveshonor_labels = falsecollisions once across the batch-widemetric_keysdictionary so dictionary key IDs stay uniform across series; if any series in the batch already exposesexported_<name>,<name>is renamed toexported_exported_<name>across the batch dictionary.Included relabeling components
relabel::Config,Action,ValidationScheme(Utf8andLegacy), andRegexp(^(?s:<pattern>)$, with Go RE2 ASCII semantics for\d,\w,\s,\b, and\Q...\E) matching Prometheusmodel/relabeldefaults (separator: ";",regex: "(.*)",replacement: "$1",action: "replace") andConfig.Validaterules.processandprocess_in_placesupporting all 11 Prometheus relabel actions (replace,keep,drop,keepequal,dropequal,hashmodusing the last 8 big-endian bytes of RFC 1321 MD5,labelmap,labeldrop,labelkeep,lowercaseanduppercaseusing Gostrings.ToLower/strings.ToUpper1-to-1 per-rune mapping) and Goregexp.ExpandString-compatible template expansion ($1,${1},$name,${name},$$, UnicodeL | Nd | '_'variable boundaries).populate_discovered_labels,populate_labels(validating__address__,__scrape_interval__,__scrape_timeout__, and histogram scrape boolean labels), andmerge_target_and_metric_labelsmatching Prometheusscrape/target.goandscrape/scrape.go.__name__filter pushdown:split_metric_name_pushdownandkeep_metric_nameto evaluate leading__name__-onlykeep,drop,keepequal, anddropequalrules once per contiguous wire series-name transition.resolve_conflicting_exposed_keys,eval_key_keep_mask,eval_single_label_keep_mask/SingleLabelKeepCache, andeval_single_label_value_transform/SingleLabelTransformCache(reusingCaptureLocationsviacaptures_read, slicing single-capture replacements in-place viaSetTargetSlice, and evaluating lazily only onval_ids seen undersource_labels[0]on surviving series).benches/relabel.rsbenchmark suite usingbenchfmt.rs(SuiteandCountingAlloc) comparing Kubernetes target relabeling and 2,000-series scrape relabeling (both 128-value low cardinality and 22,012-value high cardinality) across per-sampleLabelSet(BTreeMap<String, String>) execution and two-stage interned(key_id, val_id)dictionary execution.Benchmarks (benchfmt.rs / benchstat)
Command:
Note on benchmark modes:
mode=scalar_per_sampleclones and mutates owned per-sampleLabelSet(BTreeMap<String, String>) maps viamerge_target_and_metric_labels+process_in_place, whereasmode=two_stage_dictionaryoperates onArrowDecoder-style pre-interned(key_id, val_id)series buffers and materializes the output(key_id, val_id)attribute rows and arena dictionary updates (including an activehonor_labels = falsecollision onmethod). Both low-cardinality (scrape_2000_series, 128 unique values) and high-cardinality (scrape_2000_series_22k_vals, 22,012 unique values across sharedval_dict) fixtures are benchmarked on65f1954:Review updates comparison (
6a64e62before vs65f1954after; notescrape_2000_seriesin65f1954now includes an activeexported_methodtarget-key collision and materializes output(key_id, val_id)attribute rows and dictionary arena updates, whiletwo_stage_dictionaryallocations dropped-77.20%viacaptures_read,SetTargetSlice, and lazySingleLabelTransformCache, andkubernetes_targetreflects single-passCaptureLocationsperLabelMaprule pluspopulate_labelsduration checks):