From 55c33674e77ce804a6babb4acb996fd94759cccd Mon Sep 17 00:00:00 2001 From: andreadimaio Date: Wed, 26 Aug 2026 11:00:58 +0200 Subject: [PATCH] Update documentation --- docs/content/migration/0.30.1-to-0.40.0.md | 51 +++++++++++++++------- docs/content/setup.md | 26 +++++------ 2 files changed, 49 insertions(+), 28 deletions(-) diff --git a/docs/content/migration/0.30.1-to-0.40.0.md b/docs/content/migration/0.30.1-to-0.40.0.md index 9b3d0164..c8deecd8 100644 --- a/docs/content/migration/0.30.1-to-0.40.0.md +++ b/docs/content/migration/0.30.1-to-0.40.0.md @@ -3,7 +3,42 @@ id: 0.30.1-to-0.40.0 title: 0.30.1 → 0.40.0 --- -This release renames the Model Gateway chat types, moves `EmptyChatResponseException` into a dedicated package, introduces `ModerationException` for responses blocked by the moderation system, and makes every collection the SDK exposes unmodifiable. +This release renames the Model Gateway chat types, moves `EmptyChatResponseException` into a dedicated package, introduces `ModerationException` for responses blocked by the moderation system, makes every collection the SDK exposes unmodifiable, and adds `DeveloperMessage`, `PartialResponseInterceptor`, and streaming support for `MessageInterceptor`. + +--- + +## What's new + +### `DeveloperMessage` + +`DeveloperMessage` is a new `ChatMessage` subtype that carries the OpenAI `developer` role. Newer models use it in place of `SystemMessage` to set the assistant's behavior. It is accepted only by the Model Gateway chat APIs (`ModelGatewayChatService` and `ModelGatewayTextChatRequest`). + +```java +import com.ibm.watsonx.ai.chat.model.DeveloperMessage; + +messages.add(DeveloperMessage.of("You are a helpful assistant")); +``` + +Passing a `DeveloperMessage` to `ChatService` or `DeploymentService` throws an `IllegalArgumentException`. Use `SystemMessage` there instead. + +--- + +### `PartialResponseInterceptor` and `MessageInterceptor` in streaming + +`MessageInterceptor` now fires in streaming mode as well. Previously it was applied only to non-streaming responses. In 0.40.0 it is invoked once per assistant message with the complete aggregated content, both when calling `chat(...)` and when calling `chatStreaming(...)`. + +A new `PartialResponseInterceptor` companion intercepts each individual content token before it is delivered to `ChatHandler.onPartialResponse`. Register both on `ChatService`, `DeploymentService`, and `ModelGatewayChatService` through their builders: + +```java +ChatService.builder() + // called once with the full message (streaming and non-streaming) + .messageInterceptor((ctx, content) -> content == null ? "" : content.strip()) + // called once per token in streaming mode + .partialResponseInterceptor((ctx, token) -> token.replace("foo", "bar")) + ... +``` + +`PartialResponseInterceptor` receives the token before it reaches `onPartialResponse`, so what the handler sees is already the transformed text. `ctx.response()` is always empty inside a `PartialResponseInterceptor` because no complete response exists at that point. See [Chat - Interceptors](/services/chat-service#interceptors) for the full reference. --- @@ -95,20 +130,6 @@ A `null` collection is still accepted and preserved as `null`, but a collection These do not break compilation, but they change what your code observes at runtime. -### Value types implement `equals`, `hashCode`, and `toString` - -Around a hundred public request, parameters, and value classes gained value-based `equals` and `hashCode`, plus a `toString` that prints their fields. Types such as `ChatParameters`, `ChatRequest`, `DeploymentChatRequest`, `FindByIdRequest`, the JSON schema classes, and the detector classes previously used identity comparison inherited from `Object`. - -Two distinct instances configured identically are now equal. Review any code that keeps these objects in a `HashSet` or uses them as `HashMap` keys and relies on identity semantics, and any log or test assertion that matched the old `ClassName@1a2b3c` output. - -### Streaming callbacks are fully serialized - -`onCompleteToolCall` used to be dispatched outside the sequential callback chain, so it could run concurrently with the other callbacks and handlers sharing mutable state had to synchronize access themselves. Within a single request every callback is now delivered sequentially in emission order and never concurrently with another one: all the `onPartialToolCall` fragments of a tool call reach the handler before its `onCompleteToolCall`, tool calls arrive in index order, and `onCompleteResponse` comes last. Handlers can accumulate into unsynchronized fields. - -The flip side is that a blocking callback now delays the ones after it in the same request. Move long-running work off the callback thread if that matters to you. See the threading note in [Chat](/services/chat-service#streaming-with-chathandler). - -Tool calls for functions that take no parameters are also reported consistently now. They produce a single `onPartialToolCall` with empty arguments followed by `onCompleteToolCall`, instead of the duplicate or missing fragments the previous implementation could emit. - ### Streams can be cancelled `chatStreaming` and `generateStreaming` return a `CompletableFuture` whose `cancel(...)` now aborts the response body subscription and closes the connection, so the model stops streaming. Previously cancelling the future left the stream running in the background. After cancellation no further callback reaches the handler, not even `onError`, and the future ends in the cancelled state so a later `get()` or `join()` throws a `CancellationException`. See [Chat](/services/chat-service#cancelling-a-stream). diff --git a/docs/content/setup.md b/docs/content/setup.md index 9095ae96..8104c3f0 100644 --- a/docs/content/setup.md +++ b/docs/content/setup.md @@ -40,7 +40,7 @@ Not every service needs every value: Create an IBM Cloud account and provision a watsonx.ai service instance. The free plan is enough to get started. -- 📖 [Signing up for watsonx.ai](https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/signup-wx.html?context=wx&audience=wdp) +- [Signing up for watsonx.ai](https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/signup-wx.html?context=wx&audience=wdp) --- @@ -50,7 +50,7 @@ The SDK authenticates to IBM Cloud by exchanging an API key for an IAM bearer to This is the value passed to `apiKey(...)` and referenced as `WATSONX_API_KEY` in the samples. -- 📖 [Managing user API keys](https://cloud.ibm.com/docs/account?topic=account-userapikey) +- [Managing user API keys](https://cloud.ibm.com/docs/account?topic=account-userapikey) --- @@ -58,8 +58,8 @@ This is the value passed to `apiKey(...)` and referenced as `WATSONX_API_KEY` in A **project** is the workspace where inference requests run. After creating one, open its **Manage → General** tab. The **Project ID** is listed there. This is the value for `projectId(...)` / `WATSONX_PROJECT_ID`. -- 📖 [Creating a project](https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/projects.html?context=wx&audience=wdp) -- 📖 [Working in projects](https://dataplatform.cloud.ibm.com/docs/content/wsj/manage-data/manage-projects.html?context=wx&audience=wdp) +- [Creating a project](https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/projects.html?context=wx&audience=wdp) +- [Working in projects](https://dataplatform.cloud.ibm.com/docs/content/wsj/manage-data/manage-projects.html?context=wx&audience=wdp) > Every service accepts either a `projectId` **or** a `spaceId`, so you do not need both. @@ -69,7 +69,7 @@ A **project** is the workspace where inference requests run. After creating one, A **deployment space** is a workspace for assets that are ready for testing or production. Most services can use a space instead of a project via `spaceId(...)`. Create one under **Deployments**, then find the **Space ID** in the space's **Manage** tab. -- 📖 [Deployment spaces](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/ml-spaces_local.html?context=wx&audience=wdp) +- [Deployment spaces](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/ml-spaces_local.html?context=wx&audience=wdp) --- @@ -77,7 +77,7 @@ A **deployment space** is a workspace for assets that are ready for testing or p The [Deployment Service](services/deployment-service/) targets a `WATSONX_DEPLOYMENT_ID` instead of a `modelId`. To obtain one, deploy an asset (a foundation model or a prompt template) into a deployment space. Once deployed, the deployment's unique ID is shown in the space. -- 📖 [Creating online deployments](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/deploy-online.html?context=wx&audience=wdp) +- [Creating online deployments](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/deploy-online.html?context=wx&audience=wdp) --- @@ -88,9 +88,9 @@ The [Text Extraction](services/document-processing/text-extraction-service/), [T 1. **Provision COS and create a bucket** - this gives you the `BUCKET_NAME` and the `cosUrl` (the regional S3 endpoint, e.g. `https://s3.us-south.cloud-object-storage.appdomain.cloud`). 2. **Create a connection asset** in your project or space - this yields the `CONNECTION_ID` the SDK uses to reference the bucket. -- 📖 [Provisioning Cloud Object Storage and creating buckets](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-provision) -- 📖 [Creating a Cloud Object Storage connection](https://dataplatform.cloud.ibm.com/docs/content/wsj/manage-data/conn-cos.html?context=wx&audience=wdp) -- 📖 [Adding connections to a project](https://dataplatform.cloud.ibm.com/docs/content/wsj/manage-data/create-conn.html?context=wx&audience=wdp) +- [Provisioning Cloud Object Storage and creating buckets](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-provision) +- [Creating a Cloud Object Storage connection](https://dataplatform.cloud.ibm.com/docs/content/wsj/manage-data/conn-cos.html?context=wx&audience=wdp) +- [Adding connections to a project](https://dataplatform.cloud.ibm.com/docs/content/wsj/manage-data/create-conn.html?context=wx&audience=wdp) --- @@ -112,8 +112,8 @@ The [Model Gateway Service](services/model-gateway/) routes requests to third-pa > Once the admin setup is complete, the standard `WATSONX_API_KEY` and `WATSONX_PROJECT_ID` are all end users need. -- 📖 [Setting up the Model Gateway in the UI](https://www.ibm.com/docs/en/watsonx/w-and-w/2.4.x?topic=gateway-setting-up-model-in-ui) -- 📖 [IBM watsonx.ai Model Gateway](https://www.ibm.com/docs/en/watsonx/w-and-w/2.4.x?topic=models-model-gateway) +- [Setting up the Model Gateway in the UI](https://www.ibm.com/docs/en/watsonx/w-and-w/2.4.x?topic=gateway-setting-up-model-in-ui) +- [IBM watsonx.ai Model Gateway](https://www.ibm.com/docs/en/watsonx/w-and-w/2.4.x?topic=models-model-gateway) --- @@ -123,7 +123,7 @@ Every service builder needs a `baseUrl`. On **IBM Cloud**, the SDK provides the > On **CP4D**, pass your instance URL as the `baseUrl` instead. The `CloudRegion` enum does not apply. See [On-premises (CP4D)](#on-premises-cp4d). -- 📖 [Endpoint URLs by region (apidocs)](https://cloud.ibm.com/apidocs/watsonx-ai#endpoint-url) +- [Endpoint URLs by region (apidocs)](https://cloud.ibm.com/apidocs/watsonx-ai#endpoint-url) --- @@ -136,7 +136,7 @@ The catalog groups models into two categories, and each is consumed through a di - **Provided with watsonx.ai (pay per token)** - models already hosted in watsonx.ai. Reference them directly by `modelId` through [Chat](services/chat-service/) and the other inference services. No deployment step is required. - **Deploy on demand (pay by the hour)** - models you first deploy into a deployment space from the **Resource Hub**. Once deployed, they are called by their `WATSONX_DEPLOYMENT_ID` through the [Deployment Service](services/deployment-service/). See [Deploy an asset](#5-deploy-an-asset-optional). -- 📖 [Supported foundation models](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-models.html?context=wx&audience=wdp) +- [Supported foundation models](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-models.html?context=wx&audience=wdp) ---