Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ with the participant context, and v5 adds the endpoints of the features that onl
- discovery of the dataspace profiles usable with a counter-party (from its DID document and DSP version metadata):
`/v5/participants/{participantContextId}/discover`.

`/v4/secrets` and `/v4/catalogs` (the federated catalog) exist only in v4.
`/v4/secrets` exists only in v4.

The versions a runtime serves are listed by `GET /v1/version` on the default API context (`web.http.path`), with their
version number, path and maturity.
Expand Down Expand Up @@ -108,6 +108,7 @@ Scopes have the form `management-api[:resource]:action`, where `action` is `read
| `management-api:policies:read`, `:write` | policy definitions |
| `management-api:contractdefinitions:read`, `:write` | contract definitions |
| `management-api:catalog:read` | catalog and dataset requests |
| `management-api:catalogs:read` | the [Federated Catalog](../../federated-catalog/_index.md) cache |
| `management-api:negotiations:read`, `:write` | contract negotiations, including [approval](../policy-engine/negotiation-approval/_index.md) |
| `management-api:agreements:read`, `:write` | contract agreements, including retirement |
| `management-api:transfers:read`, `:write` | transfer processes |
Expand Down
37 changes: 21 additions & 16 deletions content/en/documentation/for-adopters/federated-catalog/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,27 +44,29 @@ The Federated Catalog is assembled from `federatedcatalog-base-bom` (crawler, qu
`federatedcatalog-dcp-bom` (with DCP, to authenticate to the crawled connectors) and `federatedcatalog-feature-sql-bom`
(PostgreSQL storage of the cache and of the crawl targets). The single participant control plane BOMs
(`controlplane-base-bom`, `controlplane-feature-sql-bom`) contain it too, so a connector can also act as a Federated
Catalog.
Catalog. In a multi-participant control plane, add the modules `catalog-crawler-core`, `federated-catalog-api` and,
for PostgreSQL, `federated-catalog-cache-sql` and `target-node-directory-sql`.

The crawler requests the catalogs as a single participant, the one configured with `edc.participant.id` (see
[Participant Contexts](../control-plane/participant-contexts/_index.md)): the Federated Catalog is not available in
multi-participant runtimes.
The crawl targets and the cache belong to a [participant context](../control-plane/participant-contexts/_index.md):
the crawler requests the catalogs of each participant context's targets with that participant's identity, and keeps
a separate cache for each. Deactivated participant contexts are skipped. A single participant runtime has one
participant context, configured with `edc.participant.id`.

### Crawl targets

The connectors to crawl are the _target nodes_ of the `TargetNodeDirectory`, each with a `name`, an `id` (the
participant id of the connector), a `url` (its DSP endpoint) and its `supportedProtocols`
(`dataspace-protocol-http:2025-1`). The default directory is in memory and empty; with
`federatedcatalog-feature-sql-bom`, the targets are read from the `edc_target_node_directory` table. There is no API to
participant id of the connector), a `url` (its DSP endpoint), its `supportedProtocols`
(`dataspace-protocol-http:2025-1`) and the `participantContextId` it is crawled for. The default directory is in memory
and empty; with the SQL store, the targets are read from the `edc_target_node_directory` table. There is no API to
register targets: insert them into that table, or provide a `TargetNodeDirectory` of your own in an extension, e.g. one
that reads the participants of the dataspace from a registry. The node whose `name` is the runtime id is skipped.

### Crawling

Every `edc.catalog.cache.execution.period.seconds`, the crawlers request the catalog of every target node with DSP
catalog requests, page by page, including the catalogs linked from it, and replace the node's entry in the cache. A
node that can't be reached is retried, then left out until the next run; its previous entry is removed at the start of
the next run.
Every `edc.catalog.cache.execution.period.seconds`, the crawlers request the catalog of every target node of every
active participant context with DSP catalog requests, page by page, including the catalogs linked from it, and replace
the node's entry in the cache. A node that can't be reached is retried, then left out until the next run; its previous
entry is removed at the start of the next run.

| Setting | Default | Description |
|---|---|---|
Expand All @@ -79,13 +81,16 @@ the next run.

### Querying the cache

Clients query the cached catalogs on the Management API context, which is secured like the Management API of a single
participant runtime (see [Management API](../control-plane/management-api/_index.md)):
Clients query the cached catalogs on the Management API (see
[Management API](../control-plane/management-api/_index.md)):

```http
POST /api/management/v4/catalogs/request?flatten=false
POST /api/management/v4/catalogs/request?flatten=false # single participant runtime
POST /api/management/v5/participants/{participantContextId}/catalogs/request?flatten=false # multi-participant runtime
```

The optional body is a `QuerySpec` to filter the catalogs. The answer is the list of the cached DSP catalogs, as they
were at the last crawl; with `flatten=true`, the datasets of linked catalogs are returned in their parent catalog.
In a multi-participant runtime, the v5 endpoint returns the cache of the participant context in the path and requires
the scope `management-api:catalogs:read`. The optional body is a `QuerySpec` to filter the catalogs. The answer is the
list of the cached DSP catalogs, as they were at the last crawl; with `flatten=true`, the datasets of linked catalogs
are returned in their parent catalog.

Loading