Publish enriched Lody ACP usage and compaction activity

[leynos]

Summary

This branch enriches VTCode's negotiated Lody ACP metadata so clients can
display provider-reported reasoning usage, resolved context capacity,
opt-in calculated costs and automatic compaction activity. It is stacked on
#14 and keeps rate-limit reporting deliberately conservative.

Baseten's nested reasoning-token field is preserved through the normalized
usage pipeline. Custom providers can supply per-million-token pricing at the
provider or model-profile level; VTCode then publishes costUSD alongside
the resolved contextWindow. Automatic compaction is represented by standard
ACP tool lifecycle updates with _meta.lody.activity.

Review walkthrough

• Start with the normalized provider usage contract and OpenAI-compatible parsing to see how nested reasoning counts are retained.
• Review custom-provider pricing and Lody usage projection for context-window resolution and calculated costs.
• Then inspect compaction lifecycle updates and their runtime integration.
• Finish with rate-limit notices and the ACP integration guide.

Validation

• ./scripts/check.sh: passed formatting, workflow security, structured
  logging, 72 ast-grep rule tests, Clippy with warnings denied, build,
  documentation and all release gates; 6,588 tests passed, 16 were skipped,
  and two pre-existing flaky tests passed on configured retry.
• cargo nextest run --locked -p vtcode-acp -E 'test(/lody_|compaction|advertised_capabilities|provider_error_telemetry/)': 23 passed.
• cargo nextest run --locked -p vtcode-config -E 'test(custom_provider)': 22 passed.
• cargo nextest run --locked -p vtcode-llm -E 'test(/parse_usage_openai_format|test_from_llm_usage_preserves_reasoning_output_tokens|openai_chat_stream_usage_obeys_profile_precedence_and_decodes_terminal_usage/)': 8 passed, including Baseten Wiremock playback.
• git diff --check: passed.

Notes

VTCode does not advertise _meta.lody.rateLimits: Baseten does not currently
provide a trustworthy quota window from which to calculate usage percentages
or reset times. Real HTTP 429 status, error detail and Retry-After values are
published through provider telemetry and negotiated Lody notices instead.

Summary by Sourcery

Enrich Lody ACP metadata with provider usage, cost, context, compaction, and conservative rate-limit activity reporting.

New Features:

• Publish enriched ACP usage updates with reasoning-token counts, resolved context windows, and optionally calculated USD costs.
• Advertise and report automatic context compaction through standard ACP tool lifecycle updates with activity metadata.

Bug Fixes:

• Preserve nested provider reasoning-token usage across OpenAI-compatible and Responses-style parsing and streaming paths.

Enhancements:

• Surface provider HTTP 429 diagnostics, retry timing, and Retry-After values through negotiated warning notices without advertising unreliable quota windows.
• Add opt-in provider and model-profile pricing configuration with validation and precedence handling.

Documentation:

• Document enriched ACP usage, compaction activity, conservative rate-limit reporting, and custom-provider pricing configuration.

Tests:

• Add coverage for reasoning usage parsing, pricing resolution, ACP usage projection, compaction lifecycle updates, and rate-limit notices.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Warning

Your free Security trial is over. An organization admin can activate Security or dismiss this notice.

---

_{Comment @coderabbitai help to get the list of available commands.}

[sourcery-ai]

Reviewer's Guide

This PR extends VTCode’s normalized usage pipeline to retain provider reasoning tokens, resolves optional context and pricing metadata for enriched Lody usage updates, adds standard ACP lifecycle reporting for automatic compaction, and surfaces conservative HTTP 429 notices while deliberately omitting untrustworthy quota metadata.

Sequence diagram for enriched Lody usage reporting

sequenceDiagram
    participant Provider
    participant Parser
    participant UsagePipeline
    participant ZedAgent
    participant ACPClient

    Provider->>Parser: usage response with reasoning_tokens
    Parser->>UsagePipeline: normalized Usage
    UsagePipeline->>ZedAgent: publish_lody_usage(provider, model, response)
    ZedAgent->>ZedAgent: lody_usage_profile(provider, model)
    ZedAgent->>UsagePipeline: estimate_session_costs_with_pricing
    ZedAgent->>ACPClient: _lody/session/usage_update
    Note over ACPClient: contextWindow, reasoningOutputTokens, costUSD

Loading

Sequence diagram for automatic compaction activity

sequenceDiagram
    participant ZedAgent
    participant CompactionActivity
    participant ACPClient

    ZedAgent->>CompactionActivity: begin(session_id, prompt_tokens)
    CompactionActivity-->>ZedAgent: started_update()
    ZedAgent->>ACPClient: tool_call: in_progress
    ZedAgent->>ZedAgent: compact conversation
    ZedAgent->>CompactionActivity: finished_update(used_tokens_after, failure_reason)
    CompactionActivity-->>ZedAgent: tool_call_update
    ZedAgent->>ACPClient: tool_call_update: completed or failed

Loading

Flow diagram for conservative rate-limit reporting

flowchart TD
    Error[Provider error] --> Metadata{HTTP 429 metadata?}
    Metadata -->|No| Telemetry[Provider telemetry only]
    Metadata -->|Yes| Notice[publish_rate_limit_notice]
    Notice --> Retry{Automatic retry scheduled?}
    Retry -->|Yes| Warning[Warning with Retry-After and retry delay]
    Retry -->|No| WarningNoRetry[Warning with no further retry]
    Warning --> ACPClient[ACP session_info_update]
    WarningNoRetry --> ACPClient
    Metadata -.->|No quota window calculated| Omit[Omit lody.rateLimits]

Loading

File-Level Changes

Change	Details	Files
Preserve provider-reported reasoning usage through normalization and expose it in ACP usage updates.	Add optional reasoning token accounting to the shared usage contract. Parse nested reasoning fields from OpenAI-compatible, Responses, OpenRouter, Hugging Face, and streaming payloads. Split visible output tokens from reasoning output tokens while retaining normalized model usage details.	crates/common/vtcode-commons/src/llm.rs crates/codegen/vtcode-llm/src/providers/common.rs crates/codegen/vtcode-llm/src/providers/openai/response_parser.rs crates/codegen/vtcode-llm/src/providers/openai/responses_api.rs crates/codegen/vtcode-llm/src/providers/openai/stream_decoder.rs crates/codegen/vtcode-llm/src/providers/shared/mod.rs crates/codegen/vtcode-llm/src/providers/shared/responses_stream.rs crates/codegen/vtcode-llm/src/providers/openrouter/stream_decoder.rs crates/codegen/vtcode-llm/src/providers/huggingface.rs crates/codegen/vtcode-llm/src/open_responses/usage.rs crates/codegen/vtcode-llm/src/providers/custom_provider.rs
Resolve context capacity and opt-in pricing, then project calculated costs into Lody usage metadata.	Add provider-level and sparse model-profile pricing configuration in USD per million tokens. Merge profile pricing over provider defaults and validate finite non-negative rates. Resolve context windows and pricing from custom-provider profiles or built-in model metadata. Emit optional contextWindow and costUSD, requiring resolved input and output pricing.	crates/codegen/vtcode-config/src/core/custom_provider.rs crates/codegen/vtcode-config/src/core/mod.rs crates/codegen/vtcode-config/src/lib.rs crates/codegen/vtcode-acp/src/zed/agent/lody_usage.rs crates/codegen/vtcode-config/src/loader/tests.rs README.md docs/config/CONFIG_FIELD_REFERENCE.md docs/config/config.md
Advertise and publish automatic compaction through standard ACP tool lifecycle updates.	Advertise Lody compaction capability version 1. Send in-progress and terminal tool updates with stable IDs and activity metadata. Include before/after token counts, duration, completion status, and failure reasons. Ensure update publication and serialization failures are logged without masking compaction results.	crates/codegen/vtcode-acp/src/zed/agent/lody_activity.rs crates/codegen/vtcode-acp/src/zed/agent/compaction.rs crates/codegen/vtcode-acp/src/zed/agent/handlers.rs
Publish conservative provider rate-limit notices and broaden provider error telemetry.	Extract telemetry diagnostics from all LLM error variants carrying metadata. Emit warning session-info notices for HTTP 429 errors and preserve provider Retry-After values. Include VTCode retry timing when applicable without advertising fabricated quota windows. Keep rateLimits capability metadata absent and cover the behavior with tests.	crates/codegen/vtcode-acp/src/zed/agent/lody_notice.rs crates/codegen/vtcode-acp/src/zed/agent/handlers.rs docs/acp/ACP_INTEGRATION.md docs/acp/ACP_QUICK_REFERENCE.md
Update configuration, usage, and provider tests for the enriched contract.	Adjust constructors and fixtures for the new optional reasoning and pricing fields. Add parser, pricing, ACP serialization, wire round-trip, capability, and telemetry coverage. Document the new ACP usage and compaction metadata semantics.	crates/codegen/vtcode-core/src/commands/ask.rs crates/codegen/vtcode-core/src/core/telemetry.rs crates/codegen/vtcode-core/src/llm/factory.rs crates/codegen/vtcode-core/src/llm/usage_cost.rs crates/codegen/vtcode-llm/src/model_resolver.rs crates/codegen/vtcode-llm/src/open_responses/bridge.rs crates/codegen/vtcode-llm/src/provider/response.rs crates/codegen/vtcode-llm/src/providers/anthropic/response_parser.rs crates/codegen/vtcode-llm/src/providers/anthropic/stream_decoder.rs crates/codegen/vtcode-llm/src/providers/evolink.rs crates/codegen/vtcode-llm/src/providers/gemini/helpers.rs crates/codegen/vtcode-llm/src/providers/ollama.rs crates/codegen/vtcode-llm/src/providers/openai/provider/harmony_client.rs crates/codegen/vtcode-llm/src/utils.rs src/agent/runloop/unified/context_manager_tests.rs src/agent/runloop/unified/turn/turn_loop/tests.rs vtcode.toml.example

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.