feat(trace-utils)!: add encoder v1 to v04 + refactor#2145
Open
anais-raison wants to merge 14 commits into
Open
feat(trace-utils)!: add encoder v1 to v04 + refactor#2145anais-raison wants to merge 14 commits into
anais-raison wants to merge 14 commits into
Conversation
Contributor
📚 Documentation Check Results📦
|
Contributor
Clippy Allow Annotation ReportComparing clippy allow annotations between branches:
Summary by Rule
Annotation Counts by File
Annotation Stats by Crate
About This ReportThis report tracks Clippy allow annotations for specific rules, showing how they've changed in this PR. Decreasing the number of these annotations generally improves code quality. |
Contributor
🔒 Cargo Deny Results📦
|
🎉 All green!🧪 All tests passed 🎯 Code Coverage (details) 🔗 Commit SHA: 67e02c2 | Docs | Datadog PR Page | Give us feedback! |
Contributor
Artifact Size Benchmark Reportaarch64-alpine-linux-musl
aarch64-unknown-linux-gnu
libdatadog-x64-windows
libdatadog-x86-windows
x86_64-alpine-linux-musl
x86_64-unknown-linux-gnu
|
yannham
reviewed
Jun 25, 2026
yannham
left a comment
yannham
left a comment
Contributor
There was a problem hiding this comment.
LGTM overall 👍 I think the naming convention brings more clarity indeed.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5fbf20172c
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Comment on lines
+242
to
+248
| let ctx = ChunkContext { | ||
| trace_id: &chunk.trace_id, | ||
| priority: chunk.priority, | ||
| origin: &chunk.origin, | ||
| sampling_mechanism: chunk.sampling_mechanism, | ||
| attributes: &chunk.attributes, | ||
| }; |
There was a problem hiding this comment.
When a native V1 TracerPayload carries tags at the payload level (for example payload.env, payload.app_version, or payload.attributes) and the individual spans do not duplicate them, this downgrade path never reads those fields: it builds the per-span context only from each chunk and then encodes spans from that context. Agents that lack /v1.0/traces will therefore receive /v0.4/traces payloads missing env/version/top-level attributes that the native V1 encoder would have sent, causing traces to be indexed without those tags during fallback.
Useful? React with 👍 / 👎.
Comment on lines
+239
to
+247
| for &(k, v) in &merged_attrs { | ||
| match v { | ||
| AttributeValue::String(s) => { | ||
| write_str(writer, k.borrow())?; | ||
| write_str(writer, s.borrow())?; | ||
| } | ||
| AttributeValue::Bool(b) => { | ||
| write_str(writer, k.borrow())?; | ||
| write_bool_as_str(writer, *b)?; |
There was a problem hiding this comment.
When a V1 span or chunk attribute uses a key that this downgrade also writes from a dedicated field (for example env, _dd.p.dm, _dd.origin, or _dd.p.tid), the attribute is still emitted here after the promoted entry, so the v0.4 meta map contains duplicate keys. Msgpack maps do not define duplicate-key precedence, and different consumers may keep different values, so fallback can send the wrong sampling/origin/trace-id tags for payloads that carry both the typed field and an attribute with the same Datadog key.
Useful? React with 👍 / 👎.
Contributor
Author
There was a problem hiding this comment.
Fixed the duplicate-key issue: attributes sharing a name with a dedicated field (env, version, component, span.kind, _dd.p.tid, _dd.origin, _dd.p.dm, _sampling_priority_v1) are now filtered out before flattening into meta/metrics. The dedicated field always wins and the colliding attribute is dropped. Also added a regression test covering span.env = "prod" vs attributes["env"] = "staging".
@ajgajg1134 does that seem good to you?
Contributor
There was a problem hiding this comment.
Yeah I think as long as we have at least one for each (when set) we should be good
Comment on lines
+240
to
+241
| write_array_len(writer, payload.chunks.len() as u32)?; | ||
| for chunk in &payload.chunks { |
There was a problem hiding this comment.
For a native V1 chunk with dropped_trace == true, the native V1 encoder preserves that sampler decision, but the v0.4 fallback has no equivalent marker and this loop still serializes every chunk as an ordinary /v0.4/traces trace. If such a dropped chunk contains spans, falling back to an agent without /v1.0/traces will ingest traces that the tracer marked as dropped instead of omitting them or otherwise preserving the drop semantics.
Useful? React with 👍 / 👎.
Contributor
There was a problem hiding this comment.
hmm I'm a bit confused by this comment. As the trace exporter we shouldn't be dropping trace chunks when dropped_trace is true, for v1 and for v4
anais-raison
changed the title
ajgajg1134
requested changes
Jun 29, 2026
ajgajg1134
left a comment
ajgajg1134
left a comment
Contributor
There was a problem hiding this comment.
Some of the function names are confusing me a bit / don't match the comments?
Also I think some of the v1 attribute types need to be "un-done" differently, specifically key values and lists shouldn't just get put in meta_struct and should be flattened into individual meta/metrics entries.
Once you have that changed, we should validate this with some integration tests against the real agent / intake to make sure the payloads match the agent decode and appear correctly in the UI
Aaalibaba42
reviewed
Jun 30, 2026
Aaalibaba42
left a comment
Aaalibaba42
left a comment
Contributor
There was a problem hiding this comment.
Mostly nits and/or styling, most of these can be disregarded
Comment on lines
+10
to
+20
| macro_rules! write_const_msg_pack_str { | ||
| ($writer:expr, $str:expr) => {{ | ||
| use rmp::encode::ValueWriteError; | ||
| const STRING_ENCODING_LEN: usize = super::msp_string_encoding_len($str); | ||
| const STRING_ENCODING: [u8; STRING_ENCODING_LEN] = super::msp_const_string_encoding($str); | ||
|
|
||
| $writer | ||
| .write_bytes(&STRING_ENCODING) | ||
| .map_err(ValueWriteError::InvalidDataWrite) | ||
| }}; | ||
| } |
Contributor
There was a problem hiding this comment.
Out of curiosity why does this have to be a macro ?
Contributor
Author
There was a problem hiding this comment.
This was already existing, I just moved it from span_v04.rs to mod.rs to be able to use it in span_v1.rs.
Comment on lines
+319
to
+320
| .map_err(super::super::flatten_value_write_infallible) | ||
| .unwrap_infallible(); |
Contributor
There was a problem hiding this comment.
Why do you need to unwrap_infallible here can't you propagate the Result up and only unwrap in the top level function ?
ajgajg1134
approved these changes
Jul 2, 2026
ajgajg1134
left a comment
ajgajg1134
left a comment
Contributor
There was a problem hiding this comment.
The changes look good! The new names are much clearer to me. I do think we need a better integration / system testing story here though, the unit tests do a good job asserting what we think is right here but alignment with the actual trace-agent is harder to be sure of here. Unless that testing step would happen as part of testing actual integration with something like the rust tracer?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Adds a V1 → v0.4 msgpack downgrade encoder so a tracer that emits V1 spans natively can still send traces to an agent that only advertises
/v0.4/traces.Also renames the existing encoder modules/functions so the convention is uniform: parent module = output wire format, file/function suffix = input span type. The four combinations now coexist:
v04/span_v04.rs— v0.4 → v0.4 (native)v04/span_v1.rs— v1 → v0.4 (downgrade, new)v1/span_v04.rs— v0.4 → V1 (upgrade)v1/span_v1.rs— v1 → V1 (native)Motivation
APMSP-2811