Skip to content

Feature request: find which capsules/pipelines a data asset is attached to #77

Description

@arielleleon

Problem

There is currently no way to ask the SDK "is this data asset attached to anything, and to
what?" This comes up whenever you are about to change or retire a data asset and need to know
what depends on it first:

  • before archiving or deleting a data asset, checking whether any capsule or pipeline still
    mounts it
  • auditing a project's assets to find orphaned ones that nothing references
  • tracing which pipelines consume a given dataset

Today the individual pieces exist but the caller has to know that a bare data asset ID works
as a search query, and has to drive pagination themselves for both resource types:

from codeocean.capsule import CapsuleSearchParams

capsules = list(client.capsules.search_capsules_iterator(
    CapsuleSearchParams(query=data_asset_id)
))
pipelines = list(client.pipelines.search_pipelines_iterator(
    CapsuleSearchParams(query=data_asset_id)
))

That the query field accepts a data asset ID at all is not discoverable from the
CapsuleSearchParams.query documentation, which lists the searchable fields as id, name,
doi, tag, field, affiliation, journal, article, and author plus free text.

The underlying API calls work today:

curl -X POST "https://{domain}/api/v1/capsules/search" \
  -u "$API_TOKEN:" \
  -H "Content-Type: application/json" \
  -d "{\"query\": \"${DATA_ASSET_ID}\"}"

curl -X POST "https://{domain}/api/v1/pipelines/search" \
  -u "$API_TOKEN:" \
  -H "Content-Type: application/json" \
  -d "{\"query\": \"${DATA_ASSET_ID}\"}"

Proposal

Add a named convenience method on each resource client that wraps the search and collects all
pages:

capsules  = client.capsules.search_capsules_by_data_asset(data_asset_id)   # -> list[Capsule]
pipelines = client.pipelines.search_pipelines_by_data_asset(data_asset_id) # -> list[Capsule]

if not capsules and not pipelines:
    client.data_assets.archive_data_asset(data_asset_id, archive=True)

An empty list means nothing accessible to the caller has the asset attached, so the same call
serves both the boolean "is it attached" check and the "what is it attached to" question.

This needs no new endpoint and no Min-Server-Version change, and it follows the existing
Capsules._route delegation so the pipeline variant is a one-line forwarder.

Alternatives considered

  • is_data_asset_attached(capsule_id, data_asset_id) -> bool â€â€� reads more literally, but it
    searches and then filters client-side, and throws away the more useful "which ones" answer.
  • client.data_assets.list_attached_capsules(data_asset_id) â€â€� arguably the most natural place
    to look for this, since the question starts from a data asset. It would put capsule and
    pipeline routes inside data_asset.py, which cuts against the module-per-route layout.

Happy to go with whichever shape you prefer.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions