docs(adaptive): document opt-in tool result caching - #598
docs(adaptive): document opt-in tool result caching#598zhongxuanwang-nv wants to merge 5 commits into
Conversation
WalkthroughThe adaptive caching documentation now covers classified tool results alongside LLM responses. It adds tool-cache behavior, policies, observability, configuration, validation diagnostics, storage guidance, and Rust migration instructions. ChangesAdaptive cache documentation
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
willkill07
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/configure-plugins/adaptive/response-cache.mdx`:
- Around line 371-396: Update the response-cache documentation to ensure
unmatched tools always use an uncacheable tools.default policy. Remove or revise
any recommended broad catch-all configuration that enables caching without
explicit read-only classification, and document validation rejecting cacheable
global catch-all policies; apply the same correction to the referenced sections.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 64630caa-dd21-4130-847c-0360123523cf
📒 Files selected for processing (5)
crates/adaptive/README.mddocs/configure-plugins/adaptive/about.mdxdocs/configure-plugins/adaptive/configuration.mdxdocs/configure-plugins/adaptive/response-cache.mdxdocs/reference/migration-guides.mdx
📜 Review details
🧰 Additional context used
📓 Path-based instructions (16)
**/*.mdx
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/SKILL.md)
MDX top-of-file SPDX comments must use {/* ... */} delimiters instead of HTML comment delimiters (Must-Fix)
In MDX files, top-of-file comments must use JSX comment delimiters (
{/*to open and*/}to close); do not use HTML comments for MDX SPDX headers
Files:
docs/configure-plugins/adaptive/configuration.mdxdocs/configure-plugins/adaptive/about.mdxdocs/reference/migration-guides.mdxdocs/configure-plugins/adaptive/response-cache.mdx
**/*.{md,mdx}
📄 CodeRabbit inference engine (AGENTS.md)
Update
README.md,fern/, package READMEs, and binding-support notes when public behavior, package names, examples, or supported bindings change.
**/*.{md,mdx}: Prefer the documented public API, not internal shortcuts
Keep package names, repo references, and build commands current
Keep release-process and release-notes guidance in repo-maintainer docs such asRELEASING.md, not as user-facing docs pages orCHANGELOG.md
Keep stable user-facing wrappers atscripts/root in docs and examples; only point at namespaced helper paths when documenting internal maintenance work
When detailed dynamic plugin guides exist, keep Rust native plugin examples, Python worker plugin examples, andgrpc-v1protocol details on separate pagesIf links in documentation change, run
just docs-linkcheck.
Files:
docs/configure-plugins/adaptive/configuration.mdxdocs/configure-plugins/adaptive/about.mdxcrates/adaptive/README.mddocs/reference/migration-guides.mdxdocs/configure-plugins/adaptive/response-cache.mdx
**/*.{md,markdown,mdx}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Add the SPDX license header to all Markdown/MDX documentation files using the HTML comment block form.
Files:
docs/configure-plugins/adaptive/configuration.mdxdocs/configure-plugins/adaptive/about.mdxcrates/adaptive/README.mddocs/reference/migration-guides.mdxdocs/configure-plugins/adaptive/response-cache.mdx
{docs,examples}/**/*
📄 CodeRabbit inference engine (.agents/skills/rename-surfaces/SKILL.md)
Update docs and examples.
Files:
docs/configure-plugins/adaptive/configuration.mdxdocs/configure-plugins/adaptive/about.mdxdocs/reference/migration-guides.mdxdocs/configure-plugins/adaptive/response-cache.mdx
**/*
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
**/*: Format changed files with the language-native formatter before the final lint/test pass.
If dynamic plugin behavior changed, usemaintain-dynamic-pluginsand include the native SDK, worker protocol, Python SDK, docs, packaging, and Codecov surfaces in the validation plan.
If code changes alter APIs, bindings, commands, paths, packaging behavior, observability/adaptive semantics, or documented best practices, update any dependent maintainer or consumer skills in the same branch.
During iteration, preferuv run pre-commit run --files <changed files...>.
Before review or handoff, runuv run pre-commit run --all-files.
Files:
docs/configure-plugins/adaptive/configuration.mdxdocs/configure-plugins/adaptive/about.mdxcrates/adaptive/README.mddocs/reference/migration-guides.mdxdocs/configure-plugins/adaptive/response-cache.mdx
docs/**/*
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
If documentation examples or commands under
docs/change, run the targeted docs checks appropriate to the change.
Files:
docs/configure-plugins/adaptive/configuration.mdxdocs/configure-plugins/adaptive/about.mdxdocs/reference/migration-guides.mdxdocs/configure-plugins/adaptive/response-cache.mdx
**/*.{md,mdx,py,go,js,ts}
📄 CodeRabbit inference engine (.agents/skills/maintain-observability/SKILL.md)
Examples and documentation must use each exporter's documented flush/deregister order before shutdown.
Files:
docs/configure-plugins/adaptive/configuration.mdxdocs/configure-plugins/adaptive/about.mdxcrates/adaptive/README.mddocs/reference/migration-guides.mdxdocs/configure-plugins/adaptive/response-cache.mdx
{docs/**,README.md,CONTRIBUTING.md,RELEASING.md,SECURITY.md}
⚙️ CodeRabbit configuration file
{docs/**,README.md,CONTRIBUTING.md,RELEASING.md,SECURITY.md}: Review documentation for technical accuracy against the current API, command correctness, and consistency across language bindings.
Flag stale examples, missing SPDX headers where required, and instructions that no longer match CI or pre-commit behavior.
Files:
docs/configure-plugins/adaptive/configuration.mdxdocs/configure-plugins/adaptive/about.mdxdocs/reference/migration-guides.mdxdocs/configure-plugins/adaptive/response-cache.mdx
**/*.{md,rst,html,txt}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-brand-terminology.md)
**/*.{md,rst,html,txt}: Always spellNVIDIAin all caps. Do not useNvidia,nvidia,nVidia,nVIDIA, orNV.
Usean NVIDIAbefore a noun because the name starts with an 'en' sound.
Do not add a registered trademark symbol afterNVIDIAwhen referring to the company.
Use trademark symbols with product names only when the document type or legal guidance requires them.
Verify official capitalization, spacing, and hyphenation for product names.
Precede NVIDIA product names withNVIDIAon first mention when it is natural and accurate.
Do not rewrite product names for grammar or title-case rules.
Preserve third-party product names according to the owner's spelling.
Include the company name and full model qualifier on first use when it helps identify the model.
Preserve the official capitalization and punctuation of model names.
Use shorter family names only after the full name is established.
Spell out a term on first use and put the acronym in parentheses unless the acronym is widely understood by the intended audience.
Use the acronym on later mentions after it has been defined.
For long documents, reintroduce the full term if readers might lose context.
Form plurals of acronyms withs, not an apostrophe, such asGPUs.
In headings, common acronyms can remain abbreviated. Spell out the term in the first or second sentence of the body.
Common terms such asCPU,GPU,PC,API, andUIusually do not need to be spelled out for developer audiences.
Files:
crates/adaptive/README.md
**/*.{md,rst,html}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-brand-terminology.md)
Link the first mention of a product name when the destination helps the reader.
Files:
crates/adaptive/README.md
{crates/core,crates/adaptive}/**/*
📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)
Changes to
crates/coreorcrates/adaptivemust run the full language matrix
Files:
crates/adaptive/README.md
**/*.{md,rst,txt}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-guide.md)
Spell
NVIDIAin all caps. Do not useNvidia,nvidia, orNV.
Files:
crates/adaptive/README.md
**/*.{md,rst}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-guide.md)
**/*.{md,rst}: Format commands, code elements, expressions, package names, file names, and paths as inline code.
Use descriptive link text. Avoid raw URLs and weak anchors such as "here" or "read more."
Use title case consistently for technical documentation headings.
Introduce code blocks, lists, tables, and images with complete sentences.
Write procedures as imperative steps. Keep steps parallel and split long procedures into smaller tasks.
Prefer active voice, present tense, short sentences, contractions, and plain English.
Usecanfor possibility and reservemayfor permission.
Useafterfor temporal relationships instead ofonce.
Preferrefer tooverseewhen the wording points readers to another resource.
Avoid culture-specific idioms, unnecessary Latinisms, jokes, and marketing exaggeration in technical docs.
Spell out months in body text, avoid ordinal dates, and use clear time zones.
Spell out whole numbers from zero through nine unless they are technical values, parameters, versions, or UI values.
Use numerals for 10 or greater and include commas in thousands.
Do not add trademark symbols to learning-oriented docs unless the source, platform, or legal guidance explicitly requires them.
Files:
crates/adaptive/README.md
**/*.md
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-technical-docs.md)
**/*.md: Use title case consistently in technical documentation headings
Avoid quotation marks, ampersands, and exclamation marks in headings
Keep product, event, research, and whitepaper names in their official title case
Use title case for table headers
Do not force social-media sentence case into technical docs
Format code elements, commands, parameters, package names, and expressions in monospace
Format directories, file names, and paths in monospace using backticks
Use angle brackets inside monospace for variables inside paths, such as/home/<username>/.login
Format error messages and strings in quotation marks, keeping literal code strings in code formatting when clearer
Format UI buttons, menus, fields, and labels in bold
Use angle brackets between UI labels for menu paths, such as File > Save As
Use italics for new terms on first use, sparingly and only when introducing the term
Use italics for publication titles
Format keyboard shortcuts in plain text, such as Press Ctrl+Alt+Delete
Use owner/repo link text for GitHub repositories, preferring[NVIDIA/NeMo](link)over prose references like 'the GitHub repo'
Introduce every code block with a complete sentence
Do not make a code block complete the grammar of the previous sentence
Do not continue a sentence after a code block
Use syntax highlighting when the format supports it for code blocks
Avoid the word 'snippet' unless the surrounding docs already use it as a term of art
Keep inline method, function, and class references consistent with nearby docs, omitting empty parentheses for prose readability when no call is shown
Use descriptive anchor text that matches the destination title when possible for links
Avoid raw URLs in running text
Avoid generic anchor text such as 'here,' 'this page,' and 'read more'
Include acronyms in link text when a linked term includes an acronym
Do not link long sentences or multiple sentences
Avoid links that pull readers away from a procedure unless the link is a p...
Files:
crates/adaptive/README.md
crates/adaptive/**
📄 CodeRabbit inference engine (.agents/skills/maintain-optimizer/SKILL.md)
Keep
crates/adaptivealigned with the canonical adaptive config schema, built-in section helpers, plugin lifecycle, and validation/report behavior.
Files:
crates/adaptive/README.md
crates/{core,adaptive}/**/*
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
If
crates/coreorcrates/adaptivechanged, run the full validation matrix across Rust, Python, Go, and Node.js.
Files:
crates/adaptive/README.md
🪛 LanguageTool
docs/configure-plugins/adaptive/response-cache.mdx
[style] ~538-~538: This phrase is redundant. Consider writing “same”.
Context: ...onse_cache.tool_multiple_classes` | The same exact member or pattern appears in more than ...
(SAME_EXACT)
🔇 Additional comments (5)
crates/adaptive/README.md (1)
22-22: LGTM!docs/configure-plugins/adaptive/about.mdx (1)
25-25: LGTM!Also applies to: 55-57
docs/configure-plugins/adaptive/configuration.mdx (1)
36-36: LGTM!docs/configure-plugins/adaptive/response-cache.mdx (1)
3-25: LGTM!Also applies to: 296-298, 363-370, 398-466, 484-489, 501-510
docs/reference/migration-guides.mdx (1)
34-36: LGTM!
Signed-off-by: Zhongxuan Wang <daniewang@nvidia.com>
Signed-off-by: Zhongxuan Wang <daniewang@nvidia.com>
Signed-off-by: Zhongxuan Wang <daniewang@nvidia.com>
Signed-off-by: Zhongxuan Wang <daniewang@nvidia.com>
Signed-off-by: Zhongxuan Wang <daniewang@nvidia.com>
65a2072 to
ab8968c
Compare
|
Note GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer. |
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/configure-plugins/adaptive/response-cache.mdx`:
- Around line 364-397: Remove the undocumented tool-result caching section from
docs/configure-plugins/adaptive/response-cache.mdx (lines 364-397) until the
validator and tool-execution support exists. Remove the classified tool-result
caching capability claim from crates/adaptive/README.md (line 22), remove the
corresponding feature claim and page description from
docs/configure-plugins/adaptive/about.mdx (lines 25 and 55-57), and describe
response_cache as LLM-only in docs/configure-plugins/adaptive/configuration.mdx
(line 36); no runtime changes are requested.
In `@docs/reference/migration-guides.mdx`:
- Around line 53-54: Update the Windows path references in the migration guide
to use inline code with single backslashes, preserving the existing
`%ProgramData%` paths for config.toml and plugins.toml.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: cd200542-6e6e-402b-886b-9d0337ee36d6
📒 Files selected for processing (5)
crates/adaptive/README.mddocs/configure-plugins/adaptive/about.mdxdocs/configure-plugins/adaptive/configuration.mdxdocs/configure-plugins/adaptive/response-cache.mdxdocs/reference/migration-guides.mdx
📜 Review details
⏰ Context from checks skipped due to timeout. (2)
- GitHub Check: Check / Run
- GitHub Check: Preview docs
🧰 Additional context used
📓 Path-based instructions (22)
**/*.{md,rst,html,txt}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-brand-terminology.md)
**/*.{md,rst,html,txt}: Always spellNVIDIAin all caps. Do not useNvidia,nvidia,nVidia,nVIDIA, orNV.
Usean NVIDIAbefore a noun because the name starts with an 'en' sound.
Do not add a registered trademark symbol afterNVIDIAwhen referring to the company.
Use trademark symbols with product names only when the document type or legal guidance requires them.
Verify official capitalization, spacing, and hyphenation for product names.
Precede NVIDIA product names withNVIDIAon first mention when it is natural and accurate.
Do not rewrite product names for grammar or title-case rules.
Preserve third-party product names according to the owner's spelling.
Include the company name and full model qualifier on first use when it helps identify the model.
Preserve the official capitalization and punctuation of model names.
Use shorter family names only after the full name is established.
Spell out a term on first use and put the acronym in parentheses unless the acronym is widely understood by the intended audience.
Use the acronym on later mentions after it has been defined.
For long documents, reintroduce the full term if readers might lose context.
Form plurals of acronyms withs, not an apostrophe, such asGPUs.
In headings, common acronyms can remain abbreviated. Spell out the term in the first or second sentence of the body.
Common terms such asCPU,GPU,PC,API, andUIusually do not need to be spelled out for developer audiences.
Files:
crates/adaptive/README.md
**/*.{md,rst,html}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-brand-terminology.md)
Link the first mention of a product name when the destination helps the reader.
Files:
crates/adaptive/README.md
{crates/core,crates/adaptive}/**/*
📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)
Changes to
crates/coreorcrates/adaptivemust run the full language matrix
Files:
crates/adaptive/README.md
**/*.{md,rst,txt}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-guide.md)
Spell
NVIDIAin all caps. Do not useNvidia,nvidia, orNV.
Files:
crates/adaptive/README.md
**/*.{md,rst}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-guide.md)
**/*.{md,rst}: Format commands, code elements, expressions, package names, file names, and paths as inline code.
Use descriptive link text. Avoid raw URLs and weak anchors such as "here" or "read more."
Use title case consistently for technical documentation headings.
Introduce code blocks, lists, tables, and images with complete sentences.
Write procedures as imperative steps. Keep steps parallel and split long procedures into smaller tasks.
Prefer active voice, present tense, short sentences, contractions, and plain English.
Usecanfor possibility and reservemayfor permission.
Useafterfor temporal relationships instead ofonce.
Preferrefer tooverseewhen the wording points readers to another resource.
Avoid culture-specific idioms, unnecessary Latinisms, jokes, and marketing exaggeration in technical docs.
Spell out months in body text, avoid ordinal dates, and use clear time zones.
Spell out whole numbers from zero through nine unless they are technical values, parameters, versions, or UI values.
Use numerals for 10 or greater and include commas in thousands.
Do not add trademark symbols to learning-oriented docs unless the source, platform, or legal guidance explicitly requires them.
Files:
crates/adaptive/README.md
crates/adaptive/**
📄 CodeRabbit inference engine (.agents/skills/maintain-optimizer/SKILL.md)
Keep
crates/adaptivealigned with the canonical adaptive config schema, built-in section helpers, plugin lifecycle, and validation/report behavior.
Files:
crates/adaptive/README.md
**/*
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
**/*: Format changed files with the language-native formatter before the final lint/test pass.
If dynamic plugin behavior changed, usemaintain-dynamic-pluginsand include the native SDK, worker protocol, Python SDK, docs, packaging, and Codecov surfaces in the validation plan.
If code changes alter APIs, bindings, commands, paths, packaging behavior, observability/adaptive semantics, or documented best practices, update any dependent maintainer or consumer skills in the same branch.
During iteration, preferuv run pre-commit run --files <changed files...>.
Before review or handoff, runuv run pre-commit run --all-files.
**/*: Use release tags in raw Rust-compatible SemVer without a leadingv; tags such asv0.1.0are prohibited.
Use branch prefixesfeat/,fix/,docs/,test/, orrefactor/according to the change purpose.
Every commit in a pull request must include a DCOSigned-off-by:sign-off.
Before submitting a pull request, ensure pre-commit hooks, relevant tests, target-specific builds, documentation updates, and a rebase on the latestmainare complete.
Use commit messages in the formtype: short description, with a valid type and a first line under 72 characters.Run the prescribed plugin validation commands, including fixture building, focused Rust and Python package tests, integration tests, documentation checks, and the broader
validate-changematrix for broad runtime or public API changes.
**/*: Keep observability changes scoped, surface assumptions, and define focused validation before editing.
Run affected Rust tests and just test-rust for event-field changes; run just test-python, just test-go, and just test-node when binding-native configuration or lifecycle changes; update docs and examples in the same branch.
Files:
crates/adaptive/README.mddocs/configure-plugins/adaptive/configuration.mdxdocs/reference/migration-guides.mdxdocs/configure-plugins/adaptive/about.mdxdocs/configure-plugins/adaptive/response-cache.mdx
crates/{core,adaptive}/**/*
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
If
crates/coreorcrates/adaptivechanged, run the full validation matrix across Rust, Python, Go, and Node.js.For shared-semantics or broad runtime changes in the core or adaptive crates, run
just ci=true test-rust.
Files:
crates/adaptive/README.md
**/*.{md,mdx}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
If links in documentation change, run
just docs-linkcheck.Use documented public APIs and stable wrapper commands in examples and user-facing documentation; do not rely on internal helpers.
**/*.{md,mdx}: Prefer the documented public API over internal shortcuts in documentation and examples.
Keep package names, repository references, and build commands current.
Contribution workflow documentation must require an issue before external contribution pull requests and note that NVIDIA contributors may use a GitHub or Linear issue.
Update entry-point documentation when examples or reading paths change.
Keep release-process and release-notes guidance in maintainer documentation such asRELEASING.md, rather than user-facing documentation pages orCHANGELOG.md.
Use stable user-facing wrappers at thescripts/root in documentation and examples; reference namespaced helper paths only for internal maintenance documentation.
When detailed dynamic plugin guides exist, keep Rust native plugin examples, Python worker plugin examples, andgrpc-v1protocol details on separate pages.
Dynamic plugin manifests in documentation and examples should usecompat.relay = ">=0.5,<1.0"unless deliberately narrower.
Render images, diagrams, tables, and other visual content at representative page widths, ensuring legibility and complete access without clipping; use responsive scaling, reflow, or overflow as appropriate and scope visual styling narrowly.
Dynamic plugin entry pages should link to native, worker, Rust example, Python example, and protocol pages when those pages exist.
Images, diagrams, tables, and custom visual content must remain legible and fully accessible at representative desktop and narrow page widths.
Release-policy documentation must point to GitHub Releases as the only release-history source of truth.
Runjust docswhen the documentation site changes; retain./scripts/build-docs.sh htmlas the compatibility wrapper.
Files:
crates/adaptive/README.mddocs/configure-plugins/adaptive/configuration.mdxdocs/reference/migration-guides.mdxdocs/configure-plugins/adaptive/about.mdxdocs/configure-plugins/adaptive/response-cache.mdx
**/*.{rs,py,js,jsx,ts,tsx,go,c,h,cc,cpp,md,toml,yml,yaml,sh}
📄 CodeRabbit inference engine (AGENTS.md)
Keep SPDX headers on source, documentation, scripts, and configuration files; the project is Apache-2.0.
Files:
crates/adaptive/README.md
**/README.md
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Update the relevant crate or package README when that package surface changes.
Update relevant package or crate
README.mdfiles when examples or binding guidance changes.
Files:
crates/adaptive/README.md
**/*.{rs,py,go,js,jsx,ts,tsx,c,h,html,md,mdx,toml}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Include the appropriate SPDX copyright and Apache-2.0 license header in every source file.
Files:
crates/adaptive/README.mddocs/configure-plugins/adaptive/configuration.mdxdocs/reference/migration-guides.mdxdocs/configure-plugins/adaptive/about.mdxdocs/configure-plugins/adaptive/response-cache.mdx
**/{README.md,docs/index.md}
📄 CodeRabbit inference engine (.agents/skills/contribute-docs/SKILL.md)
Update
README.mdordocs/index.mdwhen documentation entry points change.
Files:
crates/adaptive/README.md
**/*.{md,mdx,rst}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-technical-docs.md)
**/*.{md,mdx,rst}: Use title case consistently for technical documentation headings and table headers; avoid quotation marks, ampersands, and exclamation marks in headings, while preserving official product, event, research, and whitepaper title case.
Format code elements, commands, parameters, package names, expressions, directories, file names, and paths in monospace; represent path placeholders with angle brackets inside monospace.
Format UI buttons, menus, fields, and labels in bold, and separate consecutive UI navigation labels with>.
Use quotation marks for error messages and strings when appropriate, italics for newly introduced terms and publication titles, and plain text for keyboard shortcuts.
Represent GitHub repositories with owner/repository link text, such as[NVIDIA/NeMo](link), rather than generic repository wording.
Introduce every code block with a complete sentence; do not let a code block complete or interrupt the grammar of surrounding prose; use syntax highlighting when supported.
Keep inline method, function, and class references consistent with nearby documentation; omit empty parentheses in prose when no call is shown.
Use descriptive link text matching the destination title when possible; avoid raw URLs, generic anchors, long-sentence links, and unnecessary links that distract from procedures.
Ensure lists have a complete lead-in sentence, more than one item, no more than two levels, parallel construction, one idea or action per item, and appropriate punctuation; use bullets for unordered items and numbers for ordered tasks.
Format definition lists with a bold term followed by a complete, parallel, punctuated definition.
Use tables for reference information, decision support, compatibility matrices, and comparable choices; flag one-row tables, missing captions or lead-ins, sentence-case headers where title case is expected, unexplained empty cells, and code or links that would be clearer as prose.
Write procedure steps as imperative ...
Files:
crates/adaptive/README.mddocs/configure-plugins/adaptive/configuration.mdxdocs/reference/migration-guides.mdxdocs/configure-plugins/adaptive/about.mdxdocs/configure-plugins/adaptive/response-cache.mdx
**/*.{toml,md,rs,py}
📄 CodeRabbit inference engine (.agents/skills/maintain-dynamic-plugins/SKILL.md)
Treat plugin Relay compatibility as normal SemVer; use
>=0.5,<1.0in examples unless a plugin intentionally declares a narrower range.
Files:
crates/adaptive/README.md
crates/{core,adaptive,plugin,worker,worker-proto,types}/**/*
📄 CodeRabbit inference engine (.agents/skills/test-rust-core/SKILL.md)
crates/{core,adaptive,plugin,worker,worker-proto,types}/**/*: For changes affectingcrates/core,crates/adaptive, or shared Rust runtime semantics, expand validation to the full binding matrix withvalidate-change.
Use narrower crate-specific tests only as a local debug loop, not as the final validation for a Rust change.
If a public API, event shape, middleware behavior, plugin semantics, orcrates/core/crates/adaptivebehavior changes, also runvalidate-change.
If the change is isolated to one binding wrapper while Rust semantics remain unchanged, prefer that binding's build/test skill instead.
Files:
crates/adaptive/README.md
**/*.mdx
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/SKILL.md)
MDX top-of-file SPDX comments must use {/* ... */} delimiters instead of HTML comment delimiters (Must-Fix)
**/*.mdx: In MDX files, top-of-file comments must use JSX comment delimiters ({/*and*/}); do not use HTML comments for MDX SPDX headers.
New or regenerated MDX files must use{/* ... */}for top-of-file SPDX comments.
Files:
docs/configure-plugins/adaptive/configuration.mdxdocs/reference/migration-guides.mdxdocs/configure-plugins/adaptive/about.mdxdocs/configure-plugins/adaptive/response-cache.mdx
{docs,examples}/**/*
📄 CodeRabbit inference engine (.agents/skills/rename-surfaces/SKILL.md)
Update docs and examples.
Files:
docs/configure-plugins/adaptive/configuration.mdxdocs/reference/migration-guides.mdxdocs/configure-plugins/adaptive/about.mdxdocs/configure-plugins/adaptive/response-cache.mdx
docs/**/*
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
If documentation examples or commands under
docs/change, run the targeted docs checks appropriate to the change.
Files:
docs/configure-plugins/adaptive/configuration.mdxdocs/reference/migration-guides.mdxdocs/configure-plugins/adaptive/about.mdxdocs/configure-plugins/adaptive/response-cache.mdx
docs/**/*.{md,mdx}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Update relevant reference documentation when public behavior or APIs change.
Files:
docs/configure-plugins/adaptive/configuration.mdxdocs/reference/migration-guides.mdxdocs/configure-plugins/adaptive/about.mdxdocs/configure-plugins/adaptive/response-cache.mdx
docs/**/*.mdx
📄 CodeRabbit inference engine (.agents/skills/test-python-binding/SKILL.md)
For documentation-only changes, prefer
contribute-docsplus targeted command checks.
Files:
docs/configure-plugins/adaptive/configuration.mdxdocs/reference/migration-guides.mdxdocs/configure-plugins/adaptive/about.mdxdocs/configure-plugins/adaptive/response-cache.mdx
{docs/**,README.md,CONTRIBUTING.md,RELEASING.md,SECURITY.md}
⚙️ CodeRabbit configuration file
{docs/**,README.md,CONTRIBUTING.md,RELEASING.md,SECURITY.md}: Review documentation for technical accuracy against the current API, command correctness, and consistency across language bindings.
Flag stale examples, missing SPDX headers where required, and instructions that no longer match CI or pre-commit behavior.
Files:
docs/configure-plugins/adaptive/configuration.mdxdocs/reference/migration-guides.mdxdocs/configure-plugins/adaptive/about.mdxdocs/configure-plugins/adaptive/response-cache.mdx
🪛 LanguageTool
docs/configure-plugins/adaptive/response-cache.mdx
[style] ~539-~539: This phrase is redundant. Consider writing “same”.
Context: ...onse_cache.tool_multiple_classes` | The same exact member or pattern appears in more than ...
(SAME_EXACT)
🔇 Additional comments (2)
docs/configure-plugins/adaptive/response-cache.mdx (1)
527-545: Reject Cacheable Catch-All Tool Policies.A cacheable
*member or override only emits a warning. It can still cache an unclassified write-capable tool and suppress its live callback. This repeats the prior catch-all policy finding.docs/reference/migration-guides.mdx (1)
16-20: LGTM!
| ## Tool-Result Cache | ||
|
|
||
| The same `response_cache` section can also cache results from | ||
| [managed tool calls](/instrument-applications/instrument-tool-call). This is a | ||
| separate, opt-in surface: it shares the configured store and namespace with | ||
| the LLM cache, but tool keys carry a distinct surface tag and cannot collide | ||
| with LLM keys. | ||
|
|
||
| Caching a tool call suppresses the real call. Cache only tools that are | ||
| read-only and stable for their TTL; do not cache a tool merely because it is | ||
| idempotent. A cache hit skips even an idempotent write. Classify each tool | ||
| explicitly before enabling the tool surface: | ||
|
|
||
| ```toml | ||
| [components.config.response_cache.tools] | ||
| enabled = true | ||
| cache_errors = false # default: do not store conventional in-band error results | ||
|
|
||
| [components.config.response_cache.tools.classes.read_only] | ||
| cacheable = true | ||
| ttl_seconds = 300 | ||
| arg_skip = ["trace_id"] | ||
| members = ["docs_*", "unit_convert"] | ||
|
|
||
| [components.config.response_cache.tools.overrides.docs_search] | ||
| tool_version = "v1" # identifies the deployed tool contract | ||
| ``` | ||
|
|
||
| An exact class member wins. Otherwise, Relay selects the most-specific matching | ||
| `*` pattern; an exact override wins over a wildcard override. Configuration | ||
| validation rejects overlapping wildcard class patterns or wildcard overrides | ||
| that disagree on `cacheable`, so a broad deny policy cannot silently become a | ||
| cacheable result. Unmatched tools use `tools.default`, which must remain | ||
| uncacheable; classify every cacheable tool through a named class or override. |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift
Implement The Tool-Cache Contract Or Remove Its Documentation.
The supplied validate_response_cache_section whitelist omits tools, and the supplied cache factory produces only LlmExecutionFn. The documented tool-cache API cannot be configured or intercepted through the supplied implementation.
docs/configure-plugins/adaptive/response-cache.mdx#L364-L397: Remove the tool-cache setup and policy documentation, or add the validator and tool-execution implementation first.crates/adaptive/README.md#L22-L22: Remove the classified tool-result caching capability claim until the runtime supports it.docs/configure-plugins/adaptive/about.mdx#L25-L25: Remove the tool-result caching feature claim until the runtime supports it.docs/configure-plugins/adaptive/about.mdx#L55-L57: Remove the tool-result cache page description until the runtime supports it.docs/configure-plugins/adaptive/configuration.mdx#L36-L36: Describeresponse_cacheas LLM-only until tool configuration is accepted and executed.
📍 Affects 4 files
docs/configure-plugins/adaptive/response-cache.mdx#L364-L397(this comment)crates/adaptive/README.md#L22-L22docs/configure-plugins/adaptive/about.mdx#L25-L25docs/configure-plugins/adaptive/about.mdx#L55-L57docs/configure-plugins/adaptive/configuration.mdx#L36-L36
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/configure-plugins/adaptive/response-cache.mdx` around lines 364 - 397,
Remove the undocumented tool-result caching section from
docs/configure-plugins/adaptive/response-cache.mdx (lines 364-397) until the
validator and tool-execution support exists. Remove the classified tool-result
caching capability claim from crates/adaptive/README.md (line 22), remove the
corresponding feature claim and page description from
docs/configure-plugins/adaptive/about.mdx (lines 25 and 55-57), and describe
response_cache as LLM-only in docs/configure-plugins/adaptive/configuration.mdx
(line 36); no runtime changes are requested.
| or `%ProgramData%\\nemo-relay\\config.toml` and | ||
| `%ProgramData%\\nemo-relay\\plugins.toml` on Windows, for system policy. System |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Format The Windows Paths As Literal Paths.
Use inline code for both paths. Remove the doubled separators. Markdown does not require escaping backslashes inside inline code.
Proposed fix
-or %ProgramData%\\nemo-relay\\config.toml and
-%ProgramData%\\nemo-relay\\plugins.toml on Windows, for system policy. System
+or `%ProgramData%\nemo-relay\config.toml` and
+`%ProgramData%\nemo-relay\plugins.toml` on Windows, for system policy. SystemAs per coding guidelines, “Format commands, code elements, expressions, package names, file names, and paths as inline code.”
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| or `%ProgramData%\\nemo-relay\\config.toml` and | |
| `%ProgramData%\\nemo-relay\\plugins.toml` on Windows, for system policy. System | |
| or `%ProgramData%\nemo-relay\config.toml` and | |
| `%ProgramData%\nemo-relay\plugins.toml` on Windows, for system policy. System |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/reference/migration-guides.mdx` around lines 53 - 54, Update the Windows
path references in the migration guide to use inline code with single
backslashes, preserving the existing `%ProgramData%` paths for config.toml and
plugins.toml.
Source: Coding guidelines
3 participants
Overview
Documents the opt-in Adaptive tool-result cache introduced in #597, including its safety constraints and configuration.
Details
nemo-relay doctoroutput, and validation diagnostics.just docs,just docs-linkcheck, anduv run pre-commit run --files docs/configure-plugins/adaptive/response-cache.mdx.Where should the reviewer start?
Start with the
Tool-Result Cachesection indocs/configure-plugins/adaptive/response-cache.mdx.Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)
Summary by CodeRabbit