Skip to content

fix: describe a draft data asset's files as readable - #45

Open
doronsl wants to merge 3 commits into
mainfrom
draft-readable-tool-desc-sc-200155
Open

fix: describe a draft data asset's files as readable#45
doronsl wants to merge 3 commits into
mainfrom
draft-readable-tool-desc-sc-200155

Conversation

@doronsl

@doronsl doronsl commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

get_data_asset_file_urls appended this to the SDK docstring: "Call only when the data asset is already created and in a ready state. If the asset may not yet be ready, first use wait_until_ready to poll until readiness, then retrieve the download URL."

A data asset in the draft state never reaches a ready state, so that advice is not merely conservative on a draft — it points the model at an event that cannot occur. Tool calls are bounded now, so it is not an endless wait, but the bound is half an hour, which is a hung turn as far as the user is concerned. The Aqua file-upload flow hands the agent a draft and asks it to read the files, so a draft is the common case for this tool rather than a corner of it.

The description now says what is true: a draft's files are readable right away, and readiness only concerns an asset still being created by one of the copy-based paths — a captured result, a connector, an import. Waiting therefore stays available exactly where waiting is correct. wait_until_ready gets the same point from the other side, since its own description offered "downloading files" as the reason to poll and so steered toward the same dead end; it now says a draft is not something to poll. The text ships on every invocation, so both additions are a sentence.

The misleading advice appeared on no other tool in tools/. download_and_read_a_file_from_data_asset, the sibling an agent is most likely to reach for, never carried it.

The joins

These descriptions are an SDK docstring concatenated with the server's own guidance, and reading the composed string rather than the source literal turned up a second, unrelated defect: get_data_asset_file_urls.__doc__ ends without trailing whitespace, so the model was served ...from an internal data asset.Call only when....

Rather than fix that one join, tests/test_tool_descriptions.py discovers them: it walks the SDK client classes for docstrings, finds every advertised description that is one of those docstrings plus appended text, and asserts the boundary carries whitespace. That covers 16 joins across all four files in tools/ and covers any future tool the moment someone appends guidance to it. A scan guard fails if the discovery matches nothing, since a silent zero-match would leave the boundary assertion vacuous.

Generalising it found one more genuine run-on, in a different file: get_capsule served ...capsule by its ID.Use only to fetch metadata.... Fixed here too.

Two joins in data_assets.py omit a separator of their own but are not run-ons, and are left as they are: wait_until_ready and create_data_asset sit against SDK docstrings ending in \n and \n\n, so the composed text breaks at a newline. Prepending a space would only put a stray space at the start of a line, and normalising to a single space would glue the guidance onto the last entry of a Raises: block. The boundary test is the durable guard for both — it fails if upstream ever reflows those docstrings, which an explicit space in the source would silently mask.

Release coupling

Based on main deliberately, not on file-upload-feature-sc-125554: the Aqua agent pins a released MCP server version, so a fix that exists only on the feature branch changes nothing at runtime and would hold back the thing waiting on it.

Next step is a separate bump-and-release PR to 0.12.3, following RELEASE.md, and then the agent's pin moves to it.

sc-200155's acceptance criteria are all end-to-end through the app and are pending that release plus the pin bump, so none is verified here:

  • AC1 — reading an unfinalized draft's file returns promptly, with no wait_until_ready in the agent log.
  • AC2 — a shared draft gets described without a readiness wait anywhere in the turn (sc-199898 AC1).
  • AC3 — a genuine readiness wait still works: reading from a captured result immediately after creating it.
  • AC4 — the fix is on main with a version bump and the agent's pin points at that version.

Story: sc-200155

🤖 Generated with Claude Code

`get_data_asset_file_urls` told the model to call it only on an asset in a
ready state, and to poll `wait_until_ready` first otherwise. A draft data
asset never reaches a ready state, so a model that follows this on a draft
waits for something that cannot happen: half an hour, now that tool calls are
bounded, which is a hung turn from the user's side. The Aqua upload flow hands
the agent a draft and asks it to read the files, so this is the common case,
not a corner.

Say what is true instead: a draft's files are readable as they are, and
readiness only concerns an asset still being created by a copy-based path, so
waiting stays available where waiting is right. `wait_until_ready` picks up
the same point from the other side, since its own description offered
downloading files as the reason to poll.

The appended guidance also ran into the SDK docstring's final word, because
the docstring ends without trailing whitespace. The new test reads the
composed description the server advertises rather than the source literal, so
it covers the join as well as the wording.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings September 8, 2026 18:05

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The changes are limited to tool-description wording plus focused regression tests, with only a minor test maintainability nit called out.

Pull request overview

Updates the MCP server’s tool descriptions to correctly communicate that draft data assets are immediately readable (and should not be polled for readiness), and adds regression tests to ensure the advertised tool descriptions match what models actually receive over stdio.

Changes:

  • Adjust get_data_asset_file_urls tool description to state drafts are readable immediately and readiness polling applies only to copy-based creation paths.
  • Update wait_until_ready tool description to explicitly rule out drafts and avoid prompting pointless polling.
  • Add tests that assert the composed (SDK docstring + server guidance) tool descriptions served via get_tools().
File summaries
File Description
src/codeocean_mcp_server/tools/data_assets.py Updates MCP tool descriptions for draft readability vs readiness polling.
tests/test_tool_descriptions.py Adds stdio-path regression tests for composed tool descriptions and spacing.
Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread tests/test_tool_descriptions.py Outdated
Comment on lines +12 to +14
def test_file_urls_description_joins_the_sdk_docstring_with_a_space():
"""The appended guidance does not run into the last word of the SDK docstring."""
assert "data asset. A draft" in DESCRIPTIONS["get_data_asset_file_urls"]

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, fixed in 94784bb. The test now locates the appended guidance and asserts that whatever precedes it ends in whitespace, so it tests the join and nothing about the SDK's own wording. Dropping the leading space in data_assets.py still fails it, so the regression it was written for is still covered.

The spacing test matched "data asset. A draft", which would fail on an
upstream docstring reword even though the server-side join was still right.
It now finds the appended guidance and checks that what precedes it ends in
whitespace, so only the join is under test. Dropping the leading space in the
source still fails it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The changes are limited to tool description text plus targeted tests that validate the advertised descriptions and prevent the reported regression.

Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

Generalising the join test to every advertised description turned up one more
run-on: `get_capsule` served "...capsule by its ID.Use only to fetch metadata
...", the same missing separator as `get_data_asset_file_urls`.

The test now discovers the joins instead of naming one. It walks the SDK
client classes for docstrings, finds every tool description that is one of
those docstrings plus appended text, and asserts the boundary carries
whitespace, so a tool gets covered the moment someone appends guidance to it.
A scan guard fails if the discovery matches nothing, since a silent zero-match
would make the boundary assertion vacuous.

`wait_until_ready` and `create_data_asset` omit a separator of their own but
are not run-ons: their SDK docstrings end in "\n" and "\n\n", so the composed
text breaks at a newline. Left as they are, because prepending a space would
put a stray space at the start of a line, and forcing a single space would
glue the guidance onto the last entry of a `Raises:` block. The boundary test
covers what actually matters, and covers it against an upstream reword too.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The changes are low-risk, limited to tool description text plus targeted regression tests that should prevent recurrence of the identified misleading guidance and join formatting defects.

Review details
  • Files reviewed: 2/3 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

@doronsl
doronsl requested a review from zvikagart September 9, 2026 12:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants