Summary
pup logs search and pup logs aggregate silently return partial results when the requested time range extends past the retention of the storage tier being queried. There is no warning, no metadata field indicating truncation, and no hint in --help that --storage is the knob to reach for.
Repro
On an org where the default (indexes) tier retains N days and Flex retains substantially more:
pup logs aggregate --query="service:some-service" --compute=count --from=30d
Returns counts consistent with only the last N days of data. Re-running the same query with --storage=flex returns the full 30-day count.
The output structure is identical in both cases — no truncated, partial, or retention_limit field on the response, and nothing on stderr.
Impact
For investigations that span more than the default tier's retention (incident forensics, audit lookbacks, long-horizon trend queries), the caller gets a plausible-looking answer that is silently incomplete. This is particularly risky for AI-agent callers, which do not know to cross-check against a second storage tier and will confidently report the truncated result as ground truth.
Suggested fixes
- stderr warning when --from reaches or exceeds detected retention for the selected tier, pointing at --storage.
- Response metadata — a retention_limited: true (or equivalent) field so programmatic callers can detect truncation.
- --storage=auto that picks the narrowest tier covering the requested range, and prints which tier it chose.
- --help cross-reference on logs search / logs aggregate noting that long lookbacks may require --storage=flex or --storage=online-archives.
Options 1 and 4 are the lowest-risk and would substantially close the gap on their own. Option 3 is the most user-friendly but is a larger design change.
Environment
pup 0.29.3
Behavior reproduced against the Datadog Logs v2 API path used by logs search and logs aggregate.
Summary
pup logs search and pup logs aggregate silently return partial results when the requested time range extends past the retention of the storage tier being queried. There is no warning, no metadata field indicating truncation, and no hint in --help that --storage is the knob to reach for.
Repro
On an org where the default (indexes) tier retains N days and Flex retains substantially more:
pup logs aggregate --query="service:some-service" --compute=count --from=30dReturns counts consistent with only the last N days of data. Re-running the same query with --storage=flex returns the full 30-day count.
The output structure is identical in both cases — no truncated, partial, or retention_limit field on the response, and nothing on stderr.
Impact
For investigations that span more than the default tier's retention (incident forensics, audit lookbacks, long-horizon trend queries), the caller gets a plausible-looking answer that is silently incomplete. This is particularly risky for AI-agent callers, which do not know to cross-check against a second storage tier and will confidently report the truncated result as ground truth.
Suggested fixes
Options 1 and 4 are the lowest-risk and would substantially close the gap on their own. Option 3 is the most user-friendly but is a larger design change.
Environment
pup 0.29.3
Behavior reproduced against the Datadog Logs v2 API path used by logs search and logs aggregate.