`component_sent_bytes_total` for the `loki` sink reports compressed bytes, unlike other sinks

[hillmandj]

A note for the community

Vector version: 0.57.0

Summary

The loki sink reports component_sent_bytes_total after compression. Other sinks, such as datadog_logs, report it before compression, which is what the component spec and #20610 say it should be.

The result is that the same logs sent to two sinks show very different sizes. That makes the metric misleading for capacity planning and cost comparisons.

What we see

We send the same log stream to both a datadog_logs sink and a loki sink (snappy compression). We compare component_sent_bytes_total to component_sent_event_bytes_total for each:

Sink	sent_bytes ÷ sent_event_bytes
datadog_logs	~1.0
loki	~0.2

• For datadog_logs, bytes sent roughly equals event bytes, which is what we expect.
• For loki, bytes sent is about 5× smaller, which fits the compressed payload size rather than the uncompressed one. Loki's own ingest metrics agree with the event-bytes figure.

Why it happens

The loki sink uses the shared default RequestBuilder::encode_events. That function compresses the payload first, then passes the compressed length into the slot meant for the uncompressed size:

• src/sinks/util/request_builder.rs#L99-L101: compressed_byte_size = payload.len() (already compressed) is passed as the uncompressed_byte_size argument of EncodeResult::compressed.

That value then flows into the metric:

1. src/sinks/util/metadata.rs#L93-L99: uncompressed_byte_size becomes request_encoded_size.
2. lib/vector-common/src/request_metadata.rs#L270-L310: it is stored and returned as-is.
3. src/sinks/loki/service.rs#L56-L57: bytes_sent() returns request_encoded_size().

By contrast, datadog_logs has its own builder. It measures the size before compressing:

• src/sinks/datadog/logs/sink.rs#L302-L313: uncompressed_size = buf.len() is taken before compression and passed to EncodeResult::compressed.

Expected

component_sent_bytes_total should report uncompressed bytes for every sink, including loki. Any other siult encode_events with compression turned on is probably affected too.

Suggested fix

In the default encode_events, record the payload length before compression and pass that as uncompressed_datadog_logs does.

Problem

component_sent_bytes_total for the loki sink reports compressed bytes, unlike other sinks

Configuration

Version

0.57.0

Debug Output

Example Data

No response

Additional Context

No response

References

No response