From 1ca672899125a7e36310a4910ef3164acb29a017 Mon Sep 17 00:00:00 2001 From: Ivo Bardarov Date: Thu, 8 Oct 2026 14:39:25 +0300 Subject: [PATCH 01/76] docs(iris): MOD-19267 document both https endpoints for the RAM FIPS posture A FIPS install needs an https Identity Service introspection URL and an https memory product validator URL; the chart refuses the in-cluster http defaults for both. Show both in the enablement steps, note that the hosting environment provides TLS, list allow_insecure_transport among the rejected settings, and add the matching troubleshooting rows. --- .../agent-memory/self-managed/operations.md | 37 +++++++++++++++++-- .../self-managed/prerequisites.md | 4 +- .../agent-memory/self-managed/reference.md | 4 +- 3 files changed, 39 insertions(+), 6 deletions(-) diff --git a/content/operate/iris/agent-memory/self-managed/operations.md b/content/operate/iris/agent-memory/self-managed/operations.md index 4b382cd4f0..35b09d2a9f 100644 --- a/content/operate/iris/agent-memory/self-managed/operations.md +++ b/content/operate/iris/agent-memory/self-managed/operations.md @@ -185,15 +185,36 @@ environments where the in-cluster test path is allowed. ## FIPS-oriented posture -The chart supports an opt-in FIPS-oriented posture for regulated environments: +The chart supports an opt-in FIPS-oriented posture for regulated environments. +Enabling it takes the profile plus two `https://` endpoints. The chart's +in-cluster defaults for both are plaintext `http://`, and the posture refuses +them: ```yaml security: profile: fips +memory: + auth: + agent_keys: + introspection: + # Data Plane to Identity Service agent-key introspection. + base_url: https:// +identityService: + productValidation: + products: + memory: + # Identity Service to Control Plane grant validation. + baseURL: https:// ``` +The chart doesn't terminate Transport Layer Security (TLS) for either endpoint. +Your hosting environment must provide both, for example with a service mesh, +ingress, or TLS-terminating proxy in front of the Identity Service and Control +Plane Services. The calling pods must trust both certificates. + You can also apply the bundled FIPS values overlay with the normal values file. -Download the chart to get `values-fips.yaml` from the chart root: +Download the chart to get `values-fips.yaml` from the chart root, and set the +two endpoints in `ram-values.yaml` or on the command line: ```bash helm pull redis-ai/redis-agent-memory --version 0.7.0 --untar @@ -202,7 +223,9 @@ helm upgrade --install redis-agent-memory redis-ai/redis-agent-memory \ --version \ --namespace \ -f ram-values.yaml \ - -f redis-agent-memory/values-fips.yaml + -f redis-agent-memory/values-fips.yaml \ + --set memory.auth.agent_keys.introspection.base_url=https:// \ + --set identityService.productValidation.products.memory.baseURL=https:// ``` When enabled, the chart sets `GODEBUG=fips140=on` on the Data Plane, worker, @@ -215,12 +238,18 @@ customer's compliance boundary. When the posture is active, the Data Plane and worker reject config that: -- enables `skip_verify` on outbound HTTP clients; or +- enables `skip_verify` on outbound HTTP clients; +- sets `allow_insecure_transport` on any authentication block, even a disabled + one; or - uses non-`rediss://` URLs for Redis connections covered by the posture. The Control Plane runs under the same posture and rejects non-`rediss://` `metadata.urls` or `databases."1".urls`. +The Identity Service sends the Control Plane internal token on every grant +validation. Under the posture it refuses to start unless each product validator +URL uses `https://`, and the chart refuses to render one that doesn't. + The Redis Agent Memory API listener itself speaks HTTP inside the cluster. Edge TLS termination is owned by the hosting environment, such as ingress, service mesh, or external load balancer. Outbound TLS to Redis, embedding providers, LLM providers, and diff --git a/content/operate/iris/agent-memory/self-managed/prerequisites.md b/content/operate/iris/agent-memory/self-managed/prerequisites.md index 8928d190db..dea10ed2f0 100644 --- a/content/operate/iris/agent-memory/self-managed/prerequisites.md +++ b/content/operate/iris/agent-memory/self-managed/prerequisites.md @@ -82,7 +82,9 @@ critical. Use persistent storage, Redis authentication, network isolation, and TLS where required. In FIPS-oriented deployments, Redis URLs covered by the posture must use -`rediss://`. Avoid eviction of metadata keys. Losing Metadata Redis removes Control +`rediss://`. The Identity Service and Control Plane also need `https://` +endpoints that your hosting environment provides. See +[FIPS-oriented posture](/content/operate/iris/agent-memory/self-managed/operations.md#fips-oriented-posture). Avoid eviction of metadata keys. Losing Metadata Redis removes Control Plane store records, and losing the Identity Service Metadata Redis removes agent-key records. ## Network access diff --git a/content/operate/iris/agent-memory/self-managed/reference.md b/content/operate/iris/agent-memory/self-managed/reference.md index 0f1bf4970b..d7ae21eff6 100644 --- a/content/operate/iris/agent-memory/self-managed/reference.md +++ b/content/operate/iris/agent-memory/self-managed/reference.md @@ -58,7 +58,9 @@ CSI-synced Secrets, make sure a Control Plane pod also mounts the corresponding | Data Plane health fails | Pod not ready, config invalid, Redis unavailable, or license invalid | Check pod logs and call `/health`, `/health/liveness`, and `/health/readiness`. | | Data Plane fails with `agent_keys.introspection.base_url is required` | Bring-your-own (BYO) Data Plane config uses agent-key auth, the default when no auth method is set, without the `auth.agent_keys.introspection` block | Add the `auth.agent_keys.introspection` block, or set `auth.method: none`. See [Configuration](/content/operate/iris/agent-memory/self-managed/configuration.md#bring-your-own-byo-configuration). | | Data Plane fails with `removed metadata configuration keys: …` | Config uses metadata keys that version 0.7.0 removed, such as `metadata.source` or `metadata.live` | Configure `metadata.urls`, `metadata.namespace`, and `databases`, then create stores through the Control Plane. | -| Pod fails to start in FIPS posture | A Redis URL is not `rediss://` or an outbound HTTP client uses `skip_verify: true` | Update Redis URLs and HTTP client config to satisfy the posture checks. | +| Pod fails to start in FIPS posture | A Redis URL is not `rediss://`, an outbound HTTP client uses `skip_verify: true`, an authentication block sets `allow_insecure_transport: true`, or the Identity Service logs `product_validation..base_url must be an https URL in FIPS profile` | Update Redis URLs, HTTP client config, authentication blocks, and the Identity Service product validator URL to satisfy the posture checks. The error names the exact config path. | +| `helm template` or `helm upgrade` fails with `security.profile=fips requires a TLS endpoint for Identity Service introspection` | FIPS is enabled and `memory.auth.agent_keys.introspection.base_url` uses the chart's in-cluster `http://` default | Set it to an `https://` Identity Service endpoint. See [FIPS-oriented posture](/content/operate/iris/agent-memory/self-managed/operations.md#fips-oriented-posture). | +| `helm template` or `helm upgrade` fails with `security.profile=fips requires identityService.productValidation.products.memory.baseURL to be an https:// URL` | FIPS is enabled and the memory product validator uses the chart's in-cluster `http://` Control Plane default, or an `http://` URL | Set `identityService.productValidation.products.memory.baseURL` to an `https://` Control Plane endpoint. See [FIPS-oriented posture](/content/operate/iris/agent-memory/self-managed/operations.md#fips-oriented-posture). | | Agent receives `401` | Key is missing, revoked, expired, or wrong | Check `Authorization` / `X-Api-Key` and key status. | | Agent receives `503` | Identity Service is unreachable, or the Data Plane's miss budget for rejected keys is exhausted | Check that the Identity Service pod is Ready and reachable from the Data Plane. | | Agent receives `403` | Key exists but lacks the required store grant or action | Update grants with `PATCH http://localhost:9200/v1/api-keys/{keyId}` and the Identity Service control token. | From b85be8130e93b9c4a8e74a55f95eeeb0cf00fa1b Mon Sep 17 00:00:00 2001 From: Zdravko Donev Date: Fri, 9 Oct 2026 11:27:50 +0300 Subject: [PATCH 02/76] Clarify RDI snapshot modes and missed deletes --- content/embeds/rdi-snapshot-gap.md | 30 ++++++ .../1.19.1/architecture/_index.md | 2 + .../1.19.1/data-pipelines/deploy.md | 17 +++- .../1.19.1/data-pipelines/pipeline-config.md | 96 +++++++++++++++++++ .../1.19.1/reference/cli/redis-di-reset.md | 10 +- .../1.19.1/reference/config-yaml-reference.md | 2 +- .../1.19.1/troubleshooting.md | 26 +++++ .../architecture/_index.md | 2 + .../data-pipelines/deploy.md | 17 +++- .../data-pipelines/multiple-sources.md | 40 +++++--- .../data-pipelines/pipeline-config.md | 96 +++++++++++++++++++ .../reference/cli/redis-di-reset.md | 10 +- .../reference/config-yaml-reference.md | 2 +- .../redis-data-integration/troubleshooting.md | 26 +++++ content/operate/rc/rdi/define.md | 9 +- content/operate/rc/rdi/faq.md | 19 +++- content/operate/rc/rdi/view-edit.md | 14 +-- layouts/shortcodes/embed-md.html | 5 +- 18 files changed, 376 insertions(+), 47 deletions(-) create mode 100644 content/embeds/rdi-snapshot-gap.md diff --git a/content/embeds/rdi-snapshot-gap.md b/content/embeds/rdi-snapshot-gap.md new file mode 100644 index 0000000000..65bbcbdf35 --- /dev/null +++ b/content/embeds/rdi-snapshot-gap.md @@ -0,0 +1,30 @@ +A new snapshot copies existing source rows. It does not replay missed deletes or remove +target records for rows absent from the snapshot. + +The timeline shows one row whose delete event is lost before collection resumes. +Time progresses from top to bottom. + +```mermaid +%%{init: {"sequence": {"actorMargin": 15, "diagramMarginX": 5, "width": 90, "mirrorActors": false}}}%% +sequenceDiagram + participant Source as Source
database + participant RDI as RDI
collector + participant Redis as Target
Redis + + Source->>RDI: Capture
row 42 + RDI->>Redis: Write
row 42 + Note over RDI: Collection stops + Note over Source: Delete row 42 + Note over Source,RDI: Required log
history expires + RDI->>Source: Restart with
when_needed + Note over Source,RDI: Saved position unavailable
New snapshot + Source->>RDI: Current rows
(without row 42) + RDI->>Redis: Write snapshot
rows + Note over Redis: Row 42 is stale
Delete missed +``` + +An interruption alone does not cause this gap. If the saved position and the required +log history are still available, the collector can resume and capture the delete. +In this example, the history expires before the collector resumes, and the new snapshot +contains no row 42 to overwrite or delete its target record. Streaming resumes after +the snapshot, but the stale record can remain in Redis. diff --git a/content/integrate/redis-data-integration/1.19.1/architecture/_index.md b/content/integrate/redis-data-integration/1.19.1/architecture/_index.md index 0f0a2b3ec7..5ab73690af 100644 --- a/content/integrate/redis-data-integration/1.19.1/architecture/_index.md +++ b/content/integrate/redis-data-integration/1.19.1/architecture/_index.md @@ -72,6 +72,8 @@ added to the target within a few seconds after capture. ## At-least-once delivery guarantee + + RDI guarantees *at-least-once delivery* to the target. This means that a given change will never be lost, but it might be added to the target more than once. Apart from a slight performance overhead, adding a diff --git a/content/integrate/redis-data-integration/1.19.1/data-pipelines/deploy.md b/content/integrate/redis-data-integration/1.19.1/data-pipelines/deploy.md index 841b6ec3b3..800b1c15dd 100644 --- a/content/integrate/redis-data-integration/1.19.1/data-pipelines/deploy.md +++ b/content/integrate/redis-data-integration/1.19.1/data-pipelines/deploy.md @@ -324,7 +324,9 @@ Use [`redis-di stop`](/content/integrate/redis-data-integration/1.19.1/reference to pause a running pipeline and [`redis-di start`](/content/integrate/redis-data-integration/1.19.1/reference/cli/redis-di-start.md) to resume it. Stopping a pipeline halts data processing without deleting the pipeline or its -configuration, so you can start it again later from where it left off. +configuration. A Debezium source restarts according to its +[snapshot mode](/content/integrate/redis-data-integration/1.19.1/data-pipelines/pipeline-config.md#choose-a-snapshot-mode) +and the availability of its saved source position. ```bash redis-di stop @@ -334,10 +336,15 @@ redis-di start ## Reset a pipeline Use [`redis-di reset`](/content/integrate/redis-data-integration/1.19.1/reference/cli/redis-di-reset.md) -to return a pipeline to initial full-sync mode. This reloads a fresh -[snapshot](/content/integrate/redis-data-integration/1.19.1/architecture/_index.md#overview) of the source -data and then resumes change data capture (CDC), which is useful when the source and target have -drifted out of sync. +to clear the pipeline's saved source positions. With the default `initial` snapshot +mode, a Debezium collector reloads selected source data and then resumes change data capture +(CDC). Reset does not change the configured +[snapshot mode](/content/integrate/redis-data-integration/1.19.1/data-pipelines/pipeline-config.md#choose-a-snapshot-mode). +Modes that skip existing rows do not reload them after a reset, and `initial_only` +does not capture subsequent changes. + +Reset does not flush the target Redis database. A new snapshot can overwrite target +records, but [missed deletes can leave stale records](/content/integrate/redis-data-integration/1.19.1/data-pipelines/pipeline-config.md#missed-deletes). ```bash redis-di reset diff --git a/content/integrate/redis-data-integration/1.19.1/data-pipelines/pipeline-config.md b/content/integrate/redis-data-integration/1.19.1/data-pipelines/pipeline-config.md index 7e1eb7876c..cc5e29d13a 100644 --- a/content/integrate/redis-data-integration/1.19.1/data-pipelines/pipeline-config.md +++ b/content/integrate/redis-data-integration/1.19.1/data-pipelines/pipeline-config.md @@ -160,6 +160,102 @@ configuration contains the following data: - `java_options`: controls the JAVA_OPTS environment variable (for RDI 1.15.1 and above). Use it to modify the default values for Java heap size and other Java options for the Debezium server. For example, set it to `"-Xmx2g -Xms512m"` to set the maximum heap size to 2 GB and the initial heap size to 512 MB. +### Choose a snapshot mode {#choose-a-snapshot-mode} + +Use `sources..advanced.source.snapshot.mode` to control when a +Debezium collector loads existing source data before streaming changes. This property +applies to sources with `type: cdc`, including MariaDB, MongoDB, MySQL, Oracle, +PostgreSQL, and SQL Server. It does not configure Spanner, Snowflake, or external +collectors. + +Add the property under the source you want to configure. For example, this file +uses `initial` for a MySQL source: + +```yaml +sources: + mysql: + type: cdc + connection: + type: mysql + host: + port: 3306 + user: ${SOURCE_DB_USERNAME} + password: ${SOURCE_DB_PASSWORD} + advanced: + source: + snapshot.mode: initial + +targets: + target: + connection: + type: redis + host: + port: 6379 + password: ${TARGET_DB_PASSWORD} +``` + +The collector saves an *offset*: its position in the source's change history. +A missing offset means there is no saved position, as on first deployment or after a +reset. An unavailable position means an offset exists, but the source no longer has +the log history needed to resume from it. + +With the default `initial` mode, the collector snapshots when there is no saved +position, then streams changes. A restart with a completed snapshot and a valid saved +position resumes streaming. Deployment or restart alone does not require a new snapshot. +If the saved position is no longer available, `initial` does not automatically +re-snapshot to recover, and the collector can report an error. + +With `when_needed`, the collector also takes a new snapshot when it detects that the +saved position is no longer available. This can reload all selected data after an +outage without a manual reset. The snapshot captures current rows, rather than +reconstructing every missed change. + +#### Missed deletes can leave stale target records {#missed-deletes} + +{{< embed-md "rdi-snapshot-gap.md" >}} + +#### Compare snapshot modes + +The available modes and their exact behavior depend on the source connector and its +version. The following table describes the six values listed in the RDI configuration +reference. A listed value is not necessarily supported by every connector. + +| Mode | Existing source data | Restart with a saved position | After a reset clears the position | +| :-- | :-- | :-- | :-- | +| `initial` (default) | Snapshot when no offset exists, then stream changes. | Resume after a completed snapshot if the position is valid. No automatic snapshot recovery for unavailable log history. | Snapshot, then stream changes. | +| `when_needed` | Snapshot when no offset exists, then stream changes. | Resume if the position is valid. Snapshot again if the connector detects an unavailable position. | Snapshot, then stream changes. | +| `always` | Snapshot on every collector start, then stream changes. | Take a new snapshot even if the position is valid. | Snapshot, then stream changes. | +| `initial_only` | Take an initial snapshot without streaming subsequent changes. | Restart an interrupted initial snapshot; do not capture subsequent changes after it completes. | Snapshot without ongoing change capture. | +| `no_data` | Skip existing rows. Initialize any required schema or source position, then stream changes. | Resume streaming if the position is valid. No data snapshot to recover unavailable history. | Skip existing rows and start streaming according to the connector's rules. | +| `never` | Skip the data snapshot where this mode is supported. | Resume according to the connector's rules; no data snapshot recovery. | Connector-specific startup behavior; existing rows are not reloaded. | + +An interrupted initial snapshot can restart when the collector starts again, including +with `initial` and `initial_only`. + +For Debezium 3.5, note these connector differences: + +- PostgreSQL `no_data` performs no snapshot. It resumes from a saved log sequence + number (LSN), or from the replication slot's creation position when no LSN is saved. + This requires the relevant write-ahead log (WAL) history to remain available. +- Other connectors can capture schema or initialize a position without copying + existing rows in `no_data` mode. +- `never` is a deprecated alias for `no_data` in PostgreSQL and MongoDB. MySQL and + MariaDB retain `never` with connector-specific prerequisites. Oracle and SQL Server + do not offer `never`; use `no_data` to skip existing rows. + +See the Debezium 3.5 connector references for [MariaDB](https://debezium.io/documentation/reference/3.5/connectors/mariadb.html), +[MongoDB](https://debezium.io/documentation/reference/3.5/connectors/mongodb.html), +[MySQL](https://debezium.io/documentation/reference/3.5/connectors/mysql.html), +[Oracle](https://debezium.io/documentation/reference/3.5/connectors/oracle.html), +[PostgreSQL](https://debezium.io/documentation/reference/3.5/connectors/postgresql.html), +and [SQL Server](https://debezium.io/documentation/reference/3.5/connectors/sqlserver.html). + +A reset clears saved positions without changing `snapshot.mode` or flushing the target +Redis database. If you need to reload existing rows, select a mode that takes a data +snapshot. A snapshot can overwrite target records, but it does not remove stale records +for rows that are absent from the source. See +[Recover from unavailable source log history](/content/integrate/redis-data-integration/1.19.1/troubleshooting.md#unavailable-source-history). + ### Targets Use this section to provide the connection details for the target Redis diff --git a/content/integrate/redis-data-integration/1.19.1/reference/cli/redis-di-reset.md b/content/integrate/redis-data-integration/1.19.1/reference/cli/redis-di-reset.md index 917ce3ac92..8c18f57588 100644 --- a/content/integrate/redis-data-integration/1.19.1/reference/cli/redis-di-reset.md +++ b/content/integrate/redis-data-integration/1.19.1/reference/cli/redis-di-reset.md @@ -8,9 +8,13 @@ categories: ["redis-di"] url: '/integrate/redis-data-integration/1.19.1/reference/cli/redis-di-reset/' --- -Resets a pipeline into initial full-sync mode, so it reloads a snapshot of the source data before -resuming change data capture. By default, the command waits for the pipeline to reach a terminal -state before returning. +Clears a pipeline's saved source positions without changing its configured snapshot mode +or flushing the target Redis database. With the default `initial` mode, the collector +takes a new data snapshot and then resumes change data capture (CDC). Other modes can +skip existing rows or omit ongoing change capture. See +[Choose a snapshot mode](/content/integrate/redis-data-integration/1.19.1/data-pipelines/pipeline-config.md#choose-a-snapshot-mode). + +By default, the command waits for the pipeline to reach a terminal state before returning. ## Usage diff --git a/content/integrate/redis-data-integration/1.19.1/reference/config-yaml-reference.md b/content/integrate/redis-data-integration/1.19.1/reference/config-yaml-reference.md index e2190ab93b..80103a06e8 100644 --- a/content/integrate/redis-data-integration/1.19.1/reference/config-yaml-reference.md +++ b/content/integrate/redis-data-integration/1.19.1/reference/config-yaml-reference.md @@ -459,7 +459,7 @@ redis.wait.retry.delay.ms: 1000 #### sources\.advanced\.source: Advanced source settings -Advanced configuration properties for the source database connection and CDC behavior. **Applies to the `cdc` and `flink` collector types.**

For the `cdc` collector type, available properties depend on the source database — refer to the relevant Debezium connector documentation: [MySQL](https://debezium.io/documentation/reference/stable/connectors/mysql.html), [MariaDB](https://debezium.io/documentation/reference/stable/connectors/mariadb.html), [PostgreSQL](https://debezium.io/documentation/reference/stable/connectors/postgresql.html), [Oracle](https://debezium.io/documentation/reference/stable/connectors/oracle.html), [SQL Server](https://debezium.io/documentation/reference/stable/connectors/sqlserver.html), [Db2](https://debezium.io/documentation/reference/stable/connectors/db2.html), [MongoDB](https://debezium.io/documentation/reference/stable/connectors/mongodb.html). When using a property from those pages, omit the `debezium.source.` prefix.

**The named properties below cover the most commonly tuned settings: `spanner.*` properties apply to the `flink` collector type, all others apply to the `cdc` collector type. Any other property from the Debezium documentation can still be set as a free-form key-value pair.** +Advanced configuration properties for the source database connection and CDC behavior. **Applies to the `cdc` and `flink` collector types.**

For the `cdc` collector type, available properties depend on the source database — refer to the relevant Debezium connector documentation: [MySQL](https://debezium.io/documentation/reference/stable/connectors/mysql.html), [MariaDB](https://debezium.io/documentation/reference/stable/connectors/mariadb.html), [PostgreSQL](https://debezium.io/documentation/reference/stable/connectors/postgresql.html), [Oracle](https://debezium.io/documentation/reference/stable/connectors/oracle.html), [SQL Server](https://debezium.io/documentation/reference/stable/connectors/sqlserver.html), [Db2](https://debezium.io/documentation/reference/stable/connectors/db2.html), [MongoDB](https://debezium.io/documentation/reference/stable/connectors/mongodb.html). When using a property from those pages, omit the `debezium.source.` prefix.

Set `snapshot.mode` as a free-form source property to control data snapshots. The default is `initial`; available modes and restart behavior depend on the connector. A reset clears the saved position without changing the mode. See [Choose a snapshot mode](/content/integrate/redis-data-integration/1.19.1/data-pipelines/pipeline-config.md#choose-a-snapshot-mode), including the [missed-delete timeline](/content/integrate/redis-data-integration/1.19.1/data-pipelines/pipeline-config.md#missed-deletes).

**The named properties below cover the most commonly tuned settings: `spanner.*` properties apply to the `flink` collector type, all others apply to the `cdc` collector type. Any other property from the Debezium documentation can still be set as a free-form key-value pair.** **Properties** diff --git a/content/integrate/redis-data-integration/1.19.1/troubleshooting.md b/content/integrate/redis-data-integration/1.19.1/troubleshooting.md index 7a967150ee..4c10e5f127 100644 --- a/content/integrate/redis-data-integration/1.19.1/troubleshooting.md +++ b/content/integrate/redis-data-integration/1.19.1/troubleshooting.md @@ -57,6 +57,32 @@ log analysis tools can use. > [Backpressure mechanism](/content/integrate/redis-data-integration/1.19.1/architecture/_index.md#backpressure-mechanism) > in the Architecture guide for more information. +## Recover from unavailable source log history {#unavailable-source-history} + +If the collector cannot resume from its saved position, check the collector source log +for unavailable or expired source log history. This differs from a missing offset, +which is expected on first deployment or after a reset. + +With `initial`, the collector can fail rather than start a new snapshot when the saved +position is unavailable. With `when_needed`, it can automatically snapshot the current +selected data and resume streaming. A new snapshot after a restart can therefore +indicate that the source history was no longer available. See +[Choose a snapshot mode](/content/integrate/redis-data-integration/1.19.1/data-pipelines/pipeline-config.md#choose-a-snapshot-mode) +for the connector-specific behavior. + +Before you use a new snapshot to recover, check for +[missed deletes and stale target records](/content/integrate/redis-data-integration/1.19.1/data-pipelines/pipeline-config.md#missed-deletes). +A snapshot does not replay lost history or delete target records that are absent from +the source. Resetting the pipeline also leaves target records in place. + +If the saved position is unavailable, work with Redis support to plan a target cleanup +or rebuild before relying on the target as a complete copy. Check whether the target +contains data from other sources or applications before deleting any records. To reload +existing source rows, configure a mode that takes a data snapshot, then +[reset the pipeline](/content/integrate/redis-data-integration/1.19.1/data-pipelines/deploy.md#reset-a-pipeline). +Review the source's log retention so the required history remains available during +future interruptions. + ## Dump support package If you need to send a comprehensive set of forensics data to Redis support, diff --git a/content/integrate/redis-data-integration/architecture/_index.md b/content/integrate/redis-data-integration/architecture/_index.md index c55e9db9f5..e1667e67f8 100644 --- a/content/integrate/redis-data-integration/architecture/_index.md +++ b/content/integrate/redis-data-integration/architecture/_index.md @@ -72,6 +72,8 @@ added to the target within a few seconds after capture. ## At-least-once delivery guarantee + + RDI guarantees *at-least-once delivery* to the target. This means that a given change will never be lost, but it might be added to the target more than once. Apart from a slight performance overhead, adding a diff --git a/content/integrate/redis-data-integration/data-pipelines/deploy.md b/content/integrate/redis-data-integration/data-pipelines/deploy.md index 438d886de6..d6debc601f 100644 --- a/content/integrate/redis-data-integration/data-pipelines/deploy.md +++ b/content/integrate/redis-data-integration/data-pipelines/deploy.md @@ -340,7 +340,9 @@ Use [`redis-di stop`](/content/integrate/redis-data-integration/reference/cli/re to pause a running pipeline and [`redis-di start`](/content/integrate/redis-data-integration/reference/cli/redis-di-start.md) to resume it. Stopping a pipeline halts data processing without deleting the pipeline or its -configuration, so you can start it again later from where it left off. +configuration. A Debezium source restarts according to its +[snapshot mode](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#choose-a-snapshot-mode) +and the availability of its saved source position. ```bash redis-di stop @@ -360,10 +362,15 @@ Note that a source can only run if its parent pipeline is running. See ## Reset a pipeline Use [`redis-di reset`](/content/integrate/redis-data-integration/reference/cli/redis-di-reset.md) -to return a pipeline to initial full-sync mode. This reloads a fresh -[snapshot](/content/integrate/redis-data-integration/architecture/_index.md#overview) of the source -data and then resumes change data capture (CDC), which is useful when the source and target have -drifted out of sync. +to clear the pipeline's saved source positions. With the default `initial` snapshot +mode, a Debezium collector reloads selected source data and then resumes change data capture +(CDC). Reset does not change the configured +[snapshot mode](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#choose-a-snapshot-mode). +Modes that skip existing rows do not reload them after a reset, and `initial_only` +does not capture subsequent changes. + +Reset does not flush the target Redis database. A new snapshot can overwrite target +records, but [missed deletes can leave stale records](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#missed-deletes). ```bash redis-di reset diff --git a/content/integrate/redis-data-integration/data-pipelines/multiple-sources.md b/content/integrate/redis-data-integration/data-pipelines/multiple-sources.md index 92bf0f4542..5773fe3dbf 100644 --- a/content/integrate/redis-data-integration/data-pipelines/multiple-sources.md +++ b/content/integrate/redis-data-integration/data-pipelines/multiple-sources.md @@ -207,10 +207,9 @@ collector and deletes that source's data from the RDI database, including its ch Debezium offsets, schema history, dead-letter queue entries, statistics, deduplication state, and record counters. The other sources keep their data, and RDI stops the whole pipeline while the deletion runs and starts it again afterwards. No further action is -needed for this cleanup, but it means that a source you add later under the same name starts -from a new -[initial snapshot](/content/integrate/redis-data-integration/architecture/_index.md) -rather than from the position it had reached. +needed for this cleanup, but a source you add later under the same name has no saved +position. With the default `initial` mode, it takes a new snapshot. Other modes follow +[their configured snapshot behavior](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#choose-a-snapshot-mode). The source's secrets are not deleted, so remove them yourself with [`redis-di delete-secret`](/content/integrate/redis-data-integration/reference/cli/redis-di-delete-secret.md) @@ -241,8 +240,9 @@ to removing the source and adding a new source with the new name. This implies i its `connection` section. - You must update `server_name` for every job that reads from the source. - The data present in the RDI database under the old name is deleted, as it is for any removed source. -- The source starts with a new - [initial snapshot](/content/integrate/redis-data-integration/architecture/_index.md). +- The new source has no saved position. With the default `initial` mode, it takes a + new snapshot. Other modes follow + [their configured snapshot behavior](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#choose-a-snapshot-mode). ## Start, stop, and reset a single source @@ -259,14 +259,23 @@ pipeline. Generally, stopping one source leaves the others running, and when one `external`. RDI creates no collector for this, so you cannot start or stop it. Stopping a source scales its collector down to zero replicas and leaves the rest of the -source's resources in place. RDI records a captured position for each source, so when you restart a collector, it resumes from where it stopped. +source's resources in place. When you restart a collector, its +[snapshot mode](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#choose-a-snapshot-mode) +and the availability of its saved position determine whether it resumes streaming or +takes a new snapshot. Resetting a single source deletes that source's data from the RDI database, including its change data streams, Debezium offsets, schema history, dead-letter queue entries, statistics, deduplication state, and record counters. -A new [initial snapshot](/content/integrate/redis-data-integration/architecture/_index.md) is then -taken for that source, while every other source keeps its data. RDI stops the whole pipeline while -the reset runs and starts it again afterwards, exactly as it does for a reset of the whole -pipeline. +The reset source then follows its configured +[snapshot mode](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#choose-a-snapshot-mode) +with no saved position. With the default `initial` mode, it takes a new snapshot and +resumes streaming. Every other source keeps its saved position, but all collectors +restart and follow their configured modes. For example, a source using `always` takes +another snapshot on restart even though its position was preserved. RDI stops the whole +pipeline while the reset runs and starts it again afterwards. + +Reset does not delete records from the target Redis database. A new snapshot can +[leave stale target records after missed deletes](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#missed-deletes). ## Monitor each source @@ -384,7 +393,8 @@ redis-di set-secret USERNAME --db mysql redis-di set-secret PASSWORD --db mysql ``` -The source then takes a fresh -[initial snapshot](/content/integrate/redis-data-integration/architecture/_index.md), because the position it had -reached was deleted along with the rest of its data. Records the pipeline already wrote to the -target database are not deleted, so the snapshot overwrites them. +The source has no saved position because its internal data was deleted. With the +default `initial` mode, it takes a fresh snapshot. Other modes follow +[their configured snapshot behavior](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#choose-a-snapshot-mode). +Records already written to the target remain. A new snapshot can overwrite them, but +it does not remove records for rows that are absent from the source. diff --git a/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md b/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md index 97fe54715c..ef6deb70da 100644 --- a/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md +++ b/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md @@ -170,6 +170,102 @@ The example configuration contains the following data: - `java_options`: controls the JAVA_OPTS environment variable (for RDI 1.15.1 and above). Use it to modify the default values for Java heap size and other Java options for the Debezium server. For example, set it to `"-Xmx2g -Xms512m"` to set the maximum heap size to 2 GB and the initial heap size to 512 MB. +### Choose a snapshot mode {#choose-a-snapshot-mode} + +Use `sources..advanced.source.snapshot.mode` to control when a +Debezium collector loads existing source data before streaming changes. This property +applies to sources with `type: cdc`, including MariaDB, MongoDB, MySQL, Oracle, +PostgreSQL, and SQL Server. It does not configure Spanner, Snowflake, or external +collectors. + +Add the property under the source you want to configure. For example, this file +uses `initial` for a MySQL source: + +```yaml +sources: + mysql: + type: cdc + connection: + type: mysql + host: + port: 3306 + user: ${MYSQL_DB_USERNAME} + password: ${MYSQL_DB_PASSWORD} + advanced: + source: + snapshot.mode: initial + +targets: + target: + connection: + type: redis + host: + port: 6379 + password: ${TARGET_DB_PASSWORD} +``` + +The collector saves an *offset*: its position in the source's change history. +A missing offset means there is no saved position, as on first deployment or after a +reset. An unavailable position means an offset exists, but the source no longer has +the log history needed to resume from it. + +With the default `initial` mode, the collector snapshots when there is no saved +position, then streams changes. A restart with a completed snapshot and a valid saved +position resumes streaming. Deployment or restart alone does not require a new snapshot. +If the saved position is no longer available, `initial` does not automatically +re-snapshot to recover, and the collector can report an error. + +With `when_needed`, the collector also takes a new snapshot when it detects that the +saved position is no longer available. This can reload all selected data after an +outage without a manual reset. The snapshot captures current rows, rather than +reconstructing every missed change. + +#### Missed deletes can leave stale target records {#missed-deletes} + +{{< embed-md "rdi-snapshot-gap.md" >}} + +#### Compare snapshot modes + +The available modes and their exact behavior depend on the source connector and its +version. The following table describes the six values listed in the RDI configuration +reference. A listed value is not necessarily supported by every connector. + +| Mode | Existing source data | Restart with a saved position | After a reset clears the position | +| :-- | :-- | :-- | :-- | +| `initial` (default) | Snapshot when no offset exists, then stream changes. | Resume after a completed snapshot if the position is valid. No automatic snapshot recovery for unavailable log history. | Snapshot, then stream changes. | +| `when_needed` | Snapshot when no offset exists, then stream changes. | Resume if the position is valid. Snapshot again if the connector detects an unavailable position. | Snapshot, then stream changes. | +| `always` | Snapshot on every collector start, then stream changes. | Take a new snapshot even if the position is valid. | Snapshot, then stream changes. | +| `initial_only` | Take an initial snapshot without streaming subsequent changes. | Restart an interrupted initial snapshot; do not capture subsequent changes after it completes. | Snapshot without ongoing change capture. | +| `no_data` | Skip existing rows. Initialize any required schema or source position, then stream changes. | Resume streaming if the position is valid. No data snapshot to recover unavailable history. | Skip existing rows and start streaming according to the connector's rules. | +| `never` | Skip the data snapshot where this mode is supported. | Resume according to the connector's rules; no data snapshot recovery. | Connector-specific startup behavior; existing rows are not reloaded. | + +An interrupted initial snapshot can restart when the collector starts again, including +with `initial` and `initial_only`. + +For Debezium 3.5, note these connector differences: + +- PostgreSQL `no_data` performs no snapshot. It resumes from a saved log sequence + number (LSN), or from the replication slot's creation position when no LSN is saved. + This requires the relevant write-ahead log (WAL) history to remain available. +- Other connectors can capture schema or initialize a position without copying + existing rows in `no_data` mode. +- `never` is a deprecated alias for `no_data` in PostgreSQL and MongoDB. MySQL and + MariaDB retain `never` with connector-specific prerequisites. Oracle and SQL Server + do not offer `never`; use `no_data` to skip existing rows. + +See the Debezium 3.5 connector references for [MariaDB](https://debezium.io/documentation/reference/3.5/connectors/mariadb.html), +[MongoDB](https://debezium.io/documentation/reference/3.5/connectors/mongodb.html), +[MySQL](https://debezium.io/documentation/reference/3.5/connectors/mysql.html), +[Oracle](https://debezium.io/documentation/reference/3.5/connectors/oracle.html), +[PostgreSQL](https://debezium.io/documentation/reference/3.5/connectors/postgresql.html), +and [SQL Server](https://debezium.io/documentation/reference/3.5/connectors/sqlserver.html). + +A reset clears saved positions without changing `snapshot.mode` or flushing the target +Redis database. If you need to reload existing rows, select a mode that takes a data +snapshot. A snapshot can overwrite target records, but it does not remove stale records +for rows that are absent from the source. See +[Recover from unavailable source log history](/content/integrate/redis-data-integration/troubleshooting.md#unavailable-source-history). + ### Targets Use this section to provide the connection details for the target Redis diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-reset.md b/content/integrate/redis-data-integration/reference/cli/redis-di-reset.md index 050fdcc4f5..4fe8da3624 100644 --- a/content/integrate/redis-data-integration/reference/cli/redis-di-reset.md +++ b/content/integrate/redis-data-integration/reference/cli/redis-di-reset.md @@ -9,9 +9,13 @@ aliases: - /integrate/redis-data-integration/ingest/reference/cli/redis-di-reset/ --- -Resets a pipeline into initial full-sync mode, so it reloads a snapshot of the source data before -resuming change data capture. By default, the command waits for the pipeline to reach a terminal -state before returning. +Clears a pipeline's saved source positions without changing its configured snapshot mode +or flushing the target Redis database. With the default `initial` mode, the collector +takes a new data snapshot and then resumes change data capture (CDC). Other modes can +skip existing rows or omit ongoing change capture. See +[Choose a snapshot mode](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#choose-a-snapshot-mode). + +By default, the command waits for the pipeline to reach a terminal state before returning. With `--source`, only the specified source is reset. See [Start, stop, and reset a single source](/content/integrate/redis-data-integration/data-pipelines/multiple-sources.md#start-stop-and-reset-a-single-source) for more information. diff --git a/content/integrate/redis-data-integration/reference/config-yaml-reference.md b/content/integrate/redis-data-integration/reference/config-yaml-reference.md index d63df53a7e..937f72c693 100644 --- a/content/integrate/redis-data-integration/reference/config-yaml-reference.md +++ b/content/integrate/redis-data-integration/reference/config-yaml-reference.md @@ -490,7 +490,7 @@ Advanced configuration properties for the source database connection and CDC beh |Name|Type|Description|Required| |----|----|-----------|--------| |**record\.processing\.threads**
(Record processing threads)|`integer`|Controls how many worker threads process captured records before they are written downstream.
Minimum: `1`
|| -|**snapshot\.mode**
(Snapshot mode)|`string`|Determines when the collector snapshots the source database. A pipeline reset clears the recorded offset, so this also decides whether the collector snapshots again after a reset.
Default: `"initial"`
Enum: `"always"`, `"initial"`, `"initial_only"`, `"never"`, `"no_data"`, `"when_needed"`
|| +|**snapshot\.mode**
(Snapshot mode)|`string`|Determines when a Debezium collector snapshots selected source data. Available modes and restart behavior depend on the source connector. A reset clears the saved offset without changing this mode or flushing the target. A new snapshot can leave stale target records after missed deletes. See [Choose a snapshot mode](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#choose-a-snapshot-mode) for the mode comparison and missed-delete timeline.
Default: `"initial"`
Enum: `"always"`, `"initial"`, `"initial_only"`, `"never"`, `"no_data"`, `"when_needed"`
|| |**snapshot\.max\.threads**
(Snapshot max threads)|`integer`|Sets the maximum number of threads used while taking the initial snapshot.
Default: `1`
Minimum: `1`
|| |**poll\.interval\.ms**
(Poll interval ms)|`integer`|Defines how often the collector polls the source for new changes.
Default: `500`
Minimum: `1`
|| |**snapshot\.fetch\.size**
(Snapshot fetch size)|`integer`|Defines how many rows are fetched per batch during the initial snapshot.
Minimum: `0`
|| diff --git a/content/integrate/redis-data-integration/troubleshooting.md b/content/integrate/redis-data-integration/troubleshooting.md index 4886cc33f9..bca017a36d 100644 --- a/content/integrate/redis-data-integration/troubleshooting.md +++ b/content/integrate/redis-data-integration/troubleshooting.md @@ -57,6 +57,32 @@ log analysis tools can use. > [Backpressure mechanism](/content/integrate/redis-data-integration/architecture/_index.md#backpressure-mechanism) > in the Architecture guide for more information. +## Recover from unavailable source log history {#unavailable-source-history} + +If the collector cannot resume from its saved position, check the collector source log +for unavailable or expired source log history. This differs from a missing offset, +which is expected on first deployment or after a reset. + +With `initial`, the collector can fail rather than start a new snapshot when the saved +position is unavailable. With `when_needed`, it can automatically snapshot the current +selected data and resume streaming. A new snapshot after a restart can therefore +indicate that the source history was no longer available. See +[Choose a snapshot mode](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#choose-a-snapshot-mode) +for the connector-specific behavior. + +Before you use a new snapshot to recover, check for +[missed deletes and stale target records](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#missed-deletes). +A snapshot does not replay lost history or delete target records that are absent from +the source. Resetting the pipeline also leaves target records in place. + +If the saved position is unavailable, work with Redis support to plan a target cleanup +or rebuild before relying on the target as a complete copy. Check whether the target +contains data from other sources or applications before deleting any records. To reload +existing source rows, configure a mode that takes a data snapshot, then +[reset the pipeline](/content/integrate/redis-data-integration/data-pipelines/deploy.md#reset-a-pipeline). +Review the source's log retention so the required history remains available during +future interruptions. + ## Dump support package If you need to send a comprehensive set of forensics data to Redis support, diff --git a/content/operate/rc/rdi/define.md b/content/operate/rc/rdi/define.md index 692bd553f7..654f150a19 100644 --- a/content/operate/rc/rdi/define.md +++ b/content/operate/rc/rdi/define.md @@ -124,6 +124,13 @@ Enter the selected source's database settings. The fields depend on the database Use **Collector properties** for additional source and sink settings. These settings apply to the selected source. See the [collector source properties](/content/integrate/redis-data-integration/reference/config-yaml-reference.md#sourcesadvancedsource-advanced-source-settings) and [collector sink properties](/content/integrate/redis-data-integration/reference/config-yaml-reference.md#sourcesadvancedsink-rdi-collector-stream-writer-configuration). +For Debezium sources, set `snapshot.mode` in **Collector source properties** to +control data snapshots. The default is `initial`. Set it for each source separately; +available modes depend on the source database. See [Choose a snapshot mode](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#choose-a-snapshot-mode). +With `when_needed`, an unavailable saved position can trigger a new snapshot, but +[missed deletes can leave stale target records](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#missed-deletes). +This property does not configure the Snowflake collector. + ![The Edit advanced properties control under Collector properties.](/images/rc/rdi/rdi-source-configuration-collector-properties.png) {width="500px"} @@ -179,7 +186,7 @@ For the Flink processor, source matchers can use lists or entries prefixed with Review every source's connection, selected data, and transformation jobs, together with the shared target and settings. Use the source actions to return to a section that needs changes. -Select **Deploy pipeline** to start the pipeline. Each source performs its initial snapshot and then captures ongoing changes. Snapshot duration depends on the amount of selected data and source performance. +Select **Deploy pipeline** to start the pipeline. With the default `initial` mode, each Debezium source without a saved position takes a snapshot and then captures ongoing changes. Other [snapshot modes](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#choose-a-snapshot-mode) can skip existing rows or omit ongoing change capture. Snapshot duration depends on the amount of selected data and source performance. ![The Deploy pipeline button.](/images/rc/rdi/rdi-confirm-deploy.png) {width="175px"} diff --git a/content/operate/rc/rdi/faq.md b/content/operate/rc/rdi/faq.md index 55c03028f3..39b4df6861 100644 --- a/content/operate/rc/rdi/faq.md +++ b/content/operate/rc/rdi/faq.md @@ -14,22 +14,31 @@ weight: 6 ### What happens when I reset the pipeline? {#reset-pipeline} -A pipeline reset clears the internal RDI state for all sources, including their saved positions. When the pipeline runs, RDI takes a new snapshot of the selected data from every source, applies the current transformations, and then resumes streaming changes. If you reset a stopped pipeline, it remains stopped until you start it. +A pipeline reset clears the internal RDI state for all sources, including their saved positions. When the pipeline runs, each source follows its configured initial data load settings. With the default `initial` mode, Debezium sources take new snapshots, apply the current transformations, and resume streaming. Other [snapshot modes](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#choose-a-snapshot-mode) can skip existing rows or omit ongoing change capture. If you reset a stopped pipeline, it remains stopped until you start it. A reset does **not** flush the target Redis database. Records already in the target remain until RDI overwrites or deletes them through normal processing. Keys that are no longer produced by the current dataset or transformations can remain in the target after a reset. For example, changing a transformation's key prefix and resetting creates keys with the new prefix without deleting keys with the old prefix. +A new snapshot can also [leave stale records after missed deletes](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#missed-deletes). +If a row was deleted while collection was interrupted and the required log history +expired, its delete event is not replayed by the snapshot. This applies to automatic +snapshots with `when_needed` as well as snapshots after a manual reset. + See [Reset data pipeline](/content/operate/rc/rdi/view-edit.md#reset-data-pipeline) for the steps. ### What happens when I flush the target database? **Flush target database** permanently deletes all data from the shared target, including data from every source and data written outside RDI. Stop the pipeline before flushing. The source databases are not changed. -Flushing does not clear RDI's saved source positions. If you only start the pipeline afterwards, it resumes from those positions. It does not automatically reload records that have not changed in the source. +Flushing does not clear RDI's saved source positions. Starting a Debezium source follows its [snapshot mode](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#choose-a-snapshot-mode). With the default `initial` mode and a valid saved position, it resumes streaming without reloading unchanged source rows. See [Flush the target database](/content/operate/rc/rdi/view-edit.md#flush-the-target-database). ### How do I reload data after a flush? {#reload-after-flush} +Before reloading, check each source's initial data load settings. For Debezium sources, +select a [snapshot mode](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#choose-a-snapshot-mode) that loads existing rows, such as +`initial`. Modes that skip data snapshots cannot refill the target's existing rows. + 1. Wait for the flush to finish. 1. [Reset the pipeline](/content/operate/rc/rdi/view-edit.md#reset-data-pipeline) while it is stopped, and wait for the reset to finish. 1. [Start the pipeline](/content/operate/rc/rdi/view-edit.md#stop-and-restart-data-pipeline) to take new snapshots of the selected data from all sources. @@ -41,7 +50,7 @@ RDI reloads data available in the source databases using the current dataset and ### What happens when I reset one source? {#reset-one-source} -RDI clears the selected source's internal state and takes a new snapshot of its selected data. The whole pipeline, including the other sources and the processor, restarts during this operation. Other sources keep their saved positions and resume streaming without a new snapshot. +RDI clears the selected source's internal state, including its saved position. With the default `initial` mode, a Debezium source takes a new data snapshot. Other [snapshot modes](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#choose-a-snapshot-mode) can skip existing rows or omit ongoing change capture. The whole pipeline, including the other sources and the processor, restarts. Other sources keep their saved positions but follow their configured modes on restart; `always` takes another snapshot even with a preserved position. All records already in the target remain, including records from the reset source. The new snapshot can overwrite that source's records. Resetting a source does not selectively delete its target data. @@ -51,7 +60,7 @@ See [Reset a source](/content/operate/rc/rdi/view-edit.md#reset-source). Currently, RDI cannot flush target data for a single source. Connect to the target Redis database and selectively delete the records you want to remove. Identify them from your key naming and transformation rules, and check that other sources do not write to the same keys. Do not use **Flush target database** for this purpose: it deletes data for all sources. -Stop the affected source before deleting its records and allow its pending records to finish processing. Starting the source resumes ingestion, and later changes can recreate records. To reload all its selected data, [reset that source](/content/operate/rc/rdi/view-edit.md#reset-source). +Stop the affected source before deleting its records and allow its pending records to finish processing. Starting the source resumes ingestion, and later changes can recreate records. To reload all its selected data, select a mode that takes a data snapshot, then [reset that source](/content/operate/rc/rdi/view-edit.md#reset-source). ### Does deleting a source delete its target data? @@ -63,7 +72,7 @@ No. Deleting a source removes its pipeline configuration and internal RDI state. Redis manages RDI upgrades in Redis Cloud. Maintenance follows your Redis Cloud Pro subscription's [maintenance window](/content/operate/rc/rdi/_index.md#maintenance-windows). -During an upgrade, monitoring may be temporarily unavailable, and ingestion pauses while the pipeline components restart. A streaming pipeline then resumes from its saved state and processes the changes accumulated during the interruption. +During an upgrade, monitoring may be temporarily unavailable, and ingestion pauses while the pipeline components restart. With the default `initial` mode and valid saved positions, Debezium sources resume streaming and process the accumulated changes. Other [snapshot modes](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#choose-a-snapshot-mode) or unavailable source history can change this behavior. A routine upgrade does not flush the target database or require you to reset the pipeline. Existing target records remain available, but they may temporarily lag behind the source data. diff --git a/content/operate/rc/rdi/view-edit.md b/content/operate/rc/rdi/view-edit.md index b438c306d8..5d950a7278 100644 --- a/content/operate/rc/rdi/view-edit.md +++ b/content/operate/rc/rdi/view-edit.md @@ -62,11 +62,11 @@ When extending an older single-source pipeline, preserve the original source nam 1. On **Dashboard**, open the source card's **More actions** menu. 1. Select **Stop source**, then confirm with **Stop source**. -Stopping one source pauses data capture from that source without stopping the other sources. Starting it again resumes from its saved position. To resume a source, open its **More actions** menu and select **Start source**, then confirm with **Start source**. A source processes data only while the pipeline is running. See [How does stopping sources affect billing?](/content/operate/rc/rdi/faq.md#stopping-and-billing). +Stopping one source pauses data capture from that source without stopping the other sources. Starting a Debezium source again follows its [snapshot mode](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#choose-a-snapshot-mode) and the availability of its saved position. To resume a source, open its **More actions** menu and select **Start source**, then confirm with **Start source**. A source processes data only while the pipeline is running. See [How does stopping sources affect billing?](/content/operate/rc/rdi/faq.md#stopping-and-billing). ### Reset a source {#reset-source} -Resetting one source starts a new snapshot and reprocesses its selected data. +Resetting one source clears its saved position. With the default `initial` mode, a Debezium source takes a new snapshot and reprocesses its selected data. Other [snapshot modes](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#choose-a-snapshot-mode) can skip existing rows or omit ongoing change capture. 1. On **Dashboard**, open the source card's **More actions** menu. 1. Select **Reset source**. @@ -75,7 +75,7 @@ Resetting one source starts a new snapshot and reprocesses its selected data. ![Reset source confirmation explaining that other sources retain their data and the pipeline temporarily stops.](/images/rc/rdi/rdi-2-reset-source.png) {width="600px"} -The reset clears that source's internal RDI streams, offsets, schema history, rejected records, and processing counters. The whole pipeline and all its sources restart during the reset. Other sources keep their saved positions and resume streaming. All records already in the shared target database remain, including those from the reset source. The new snapshot can overwrite records for that source. See [What happens when I reset one source?](/content/operate/rc/rdi/faq.md#reset-one-source). +The reset clears that source's internal RDI streams, offsets, schema history, rejected records, and processing counters. The whole pipeline and all its sources restart during the reset. Other sources keep their saved positions. Debezium sources restart according to their snapshot modes; for example, `always` takes another snapshot even when its saved position is preserved. All records already in the shared target database remain, including those from the reset source. The new snapshot can overwrite records for that source. See [What happens when I reset one source?](/content/operate/rc/rdi/faq.md#reset-one-source). ### Remove a source {#remove-source} @@ -100,7 +100,7 @@ You can change the target to another database in the same subscription. 1. Select the new target database. 1. Confirm with **Change target**. -All sources share the new target. Changing the target restarts the pipeline but does not automatically re-ingest existing records. [Reset the pipeline](#reset-data-pipeline) if you need to copy the existing source data to the new target. +All sources share the new target. Changing the target restarts the pipeline. Whether a Debezium source reloads existing rows depends on its [snapshot mode](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#choose-a-snapshot-mode). To copy existing source data to the new target, select modes that take data snapshots, then [reset the pipeline](#reset-data-pipeline). ## Metrics @@ -210,7 +210,7 @@ Select a source in the **Sources** list to view the schemas, tables, columns, an 1. Select **Save changes**. 1. Review the restart warning and select **Apply and restart**. -Dataset changes belong to the selected source. Applying changes restarts the pipeline and can temporarily interrupt processing. Use [Reset a source](#reset-source) when you need a new snapshot of that source. +Dataset changes belong to the selected source. Applying changes restarts the pipeline and can temporarily interrupt processing. To reload existing rows, select a snapshot mode that takes a data snapshot, then use [Reset a source](#reset-source). ## Transformations @@ -231,12 +231,12 @@ The Flink processor accepts lists and `regex:` entries in source selection patte ## Reset data pipeline -Resetting the whole pipeline clears its internal RDI state for all sources. A running pipeline restarts and takes a new snapshot for every source. A stopped pipeline remains stopped until you [start it](#stop-and-restart-data-pipeline). RDI reprocesses the selected data using the current transformations. Reset alone does not delete records from the target database. See [What happens when I reset the pipeline?](/content/operate/rc/rdi/faq.md#reset-pipeline). +Resetting the whole pipeline clears its internal RDI state for all sources. A running pipeline restarts each source according to its configured initial data load settings. With the default `initial` mode, Debezium sources take new snapshots. Other [snapshot modes](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#choose-a-snapshot-mode) can skip existing rows or omit ongoing change capture. A stopped pipeline remains stopped until you [start it](#stop-and-restart-data-pipeline). When a source takes a data snapshot, RDI reprocesses the selected data using the current transformations. Reset alone does not delete target records, and a new snapshot can [leave stale records after missed deletes](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#missed-deletes). See [What happens when I reset the pipeline?](/content/operate/rc/rdi/faq.md#reset-pipeline). 1. Open **Pipeline actions** and select **Reset pipeline**. 1. Review the confirmation and select **Reset data pipeline**. -To re-run the snapshot for just one source while preserving other sources' internal data, see [Reset a source](#reset-source). +To clear the saved position of one source while preserving other sources' internal data, see [Reset a source](#reset-source). ## Stop and restart data pipeline diff --git a/layouts/shortcodes/embed-md.html b/layouts/shortcodes/embed-md.html index be2deec933..7d79f4d15d 100644 --- a/layouts/shortcodes/embed-md.html +++ b/layouts/shortcodes/embed-md.html @@ -2,4 +2,7 @@ {{ $page := .Site.GetPage $file }} {{ with $page }} {{ .Content }} -{{ end }} \ No newline at end of file + {{ if .Store.Get "hasMermaid" }} + {{ $.Page.Store.Set "hasMermaid" true }} + {{ end }} +{{ end }} From 48a06a3d815d16bbc969c4a5cfcf010489c262c1 Mon Sep 17 00:00:00 2001 From: Zdravko Donev Date: Fri, 9 Oct 2026 12:43:49 +0300 Subject: [PATCH 03/76] Add missed-delete timeline to the RDI Cloud FAQ --- content/operate/rc/rdi/faq.md | 22 ++- layouts/shortcodes/rdi-snapshot-timeline.html | 23 +++ static/css/rdi-snapshot-timeline.css | 138 ++++++++++++++++++ 3 files changed, 178 insertions(+), 5 deletions(-) create mode 100644 layouts/shortcodes/rdi-snapshot-timeline.html create mode 100644 static/css/rdi-snapshot-timeline.css diff --git a/content/operate/rc/rdi/faq.md b/content/operate/rc/rdi/faq.md index 39b4df6861..0803963f3f 100644 --- a/content/operate/rc/rdi/faq.md +++ b/content/operate/rc/rdi/faq.md @@ -18,13 +18,25 @@ A pipeline reset clears the internal RDI state for all sources, including their A reset does **not** flush the target Redis database. Records already in the target remain until RDI overwrites or deletes them through normal processing. Keys that are no longer produced by the current dataset or transformations can remain in the target after a reset. For example, changing a transformation's key prefix and resetting creates keys with the new prefix without deleting keys with the old prefix. -A new snapshot can also [leave stale records after missed deletes](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#missed-deletes). -If a row was deleted while collection was interrupted and the required log history -expired, its delete event is not replayed by the snapshot. This applies to automatic -snapshots with `when_needed` as well as snapshots after a manual reset. - See [Reset data pipeline](/content/operate/rc/rdi/view-edit.md#reset-data-pipeline) for the steps. +### Why can records remain after a recovery snapshot? {#missed-deletes} + +A new snapshot can leave stale records in Redis. If a source row was deleted while +collection was interrupted and the required log history expired, the snapshot cannot +replay its delete event. This applies to Debezium sources with `type: cdc`, including +automatic snapshots with `when_needed` and snapshots after a manual reset. + +The timeline follows one row through an automatic recovery snapshot. Read the numbered +events in order. On wide screens, events alternate across a horizontal timeline. + +{{< rdi-snapshot-timeline >}} + +An interruption alone does not cause this gap. The saved position must become +unavailable. The snapshot does not clean up the target, and a reset does not flush it. +See the [detailed missed-delete sequence](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#missed-deletes) +and [recovery guidance](/content/integrate/redis-data-integration/troubleshooting.md#unavailable-source-history). + ### What happens when I flush the target database? **Flush target database** permanently deletes all data from the shared target, including data from every source and data written outside RDI. Stop the pipeline before flushing. The source databases are not changed. diff --git a/layouts/shortcodes/rdi-snapshot-timeline.html b/layouts/shortcodes/rdi-snapshot-timeline.html new file mode 100644 index 0000000000..33f821f25d --- /dev/null +++ b/layouts/shortcodes/rdi-snapshot-timeline.html @@ -0,0 +1,23 @@ +{{- $css := "css/rdi-snapshot-timeline.css" -}} + +
+
    + {{- range $i, $event := slice + (dict "title" "Row copied" "text" "Row 42 exists in the source and in Redis.") + (dict "title" "Collection stops" "text" "The collector saves its source position.") + (dict "title" "Row deleted" "text" "The source deletes row 42. Redis still has it.") + (dict "title" "History expires" "text" "The saved position is unavailable. The delete cannot be replayed.") + (dict "title" "New snapshot" "text" "when_needed copies current source rows. Row 42 is absent.") + (dict "title" "Stale record" "text" "Streaming resumes. Redis still has row 42: its delete was missed.") + }} +
  1. + +
    + {{ $event.title }} +

    {{ $event.text }}

    +
    +
    2. + {{- end }} +
+
A new snapshot copies existing source rows. It does not replay missed deletes or remove target records for rows absent from the snapshot.
+
diff --git a/static/css/rdi-snapshot-timeline.css b/static/css/rdi-snapshot-timeline.css new file mode 100644 index 0000000000..cf8537bb84 --- /dev/null +++ b/static/css/rdi-snapshot-timeline.css @@ -0,0 +1,138 @@ +.rdi-snapshot-timeline { + container-type: inline-size; + margin: 1.5rem 0; + padding: 1rem; + border: 1px solid #d5dce5; + border-radius: 8px; + background: #f8fafc; + color: #172b4d; +} + +.rdi-snapshot-timeline .rdi-snapshot-timeline__events { + position: relative; + display: grid; + gap: 1.5rem; + margin: 0; + padding: 0; + list-style: none; +} + +.rdi-snapshot-timeline__events::before { + position: absolute; + top: 16px; + bottom: 16px; + left: 15px; + width: 2px; + background: #64748b; + content: ""; +} + +.rdi-snapshot-timeline .rdi-snapshot-timeline__event { + --event-color: #b44724; + position: relative; + display: grid; + grid-template-columns: 32px 1fr; + gap: 1rem; + margin: 0; + padding: 0; +} + +.rdi-snapshot-timeline__event:nth-child(2) { --event-color: #925a00; } +.rdi-snapshot-timeline__event:nth-child(3) { --event-color: #247548; } +.rdi-snapshot-timeline__event:nth-child(4) { --event-color: #087b94; } +.rdi-snapshot-timeline__event:nth-child(5) { --event-color: #365caa; } +.rdi-snapshot-timeline__event:nth-child(6) { --event-color: #172b4d; } + +.rdi-snapshot-timeline__number { + position: relative; + display: grid; + place-items: center; + width: 32px; + height: 32px; + border-radius: 50%; + background: var(--event-color); + color: #fff; + font-size: 16px; + font-weight: 700; + line-height: 1; +} + +.rdi-snapshot-timeline__detail strong { + color: var(--event-color); + font-size: 16px; + line-height: 1.4; +} + +.rdi-snapshot-timeline .rdi-snapshot-timeline__detail p { + margin: 0.5rem 0 0; + font-size: 14px; + line-height: 1.5; +} + +.rdi-snapshot-timeline figcaption { + margin-top: 1.5rem; + padding-top: 1rem; + border-top: 1px solid #d5dce5; + color: #172b4d; + font-size: 14px; + font-weight: 600; + line-height: 1.5; +} + +@container (min-width: 550px) { + .rdi-snapshot-timeline .rdi-snapshot-timeline__events { + grid-template-columns: repeat(6, minmax(0, 1fr)); + gap: 0.5rem; + } + + .rdi-snapshot-timeline__events::before { + top: 50%; + right: 0; + bottom: auto; + left: 0; + width: auto; + height: 2px; + } + + .rdi-snapshot-timeline .rdi-snapshot-timeline__event { + grid-template-columns: 1fr; + grid-template-rows: 190px 32px 190px; + gap: 24px; + justify-items: center; + text-align: center; + } + + .rdi-snapshot-timeline__number { + grid-row: 2; + } + + .rdi-snapshot-timeline__detail { + position: relative; + grid-row: 1; + align-self: end; + } + + .rdi-snapshot-timeline__detail strong { + font-size: 14px; + } + + .rdi-snapshot-timeline__detail::after { + position: absolute; + bottom: -24px; + left: calc(50% - 1px); + width: 2px; + height: 24px; + background: var(--event-color); + content: ""; + } + + .rdi-snapshot-timeline__event:nth-child(even) .rdi-snapshot-timeline__detail { + grid-row: 3; + align-self: start; + } + + .rdi-snapshot-timeline__event:nth-child(even) .rdi-snapshot-timeline__detail::after { + top: -24px; + bottom: auto; + } +} From 2c2504a438647970704df64cadea8c634395897c Mon Sep 17 00:00:00 2001 From: Zdravko Donev Date: Fri, 9 Oct 2026 12:52:02 +0300 Subject: [PATCH 04/76] Address snapshot documentation review comments --- content/embeds/rdi-snapshot-modes.md | 55 +++++++++++++++++ .../1.19.1/architecture/_index.md | 11 +++- .../1.19.1/data-pipelines/pipeline-config.md | 59 ++---------------- .../architecture/_index.md | 11 +++- .../data-pipelines/pipeline-config.md | 60 ++----------------- .../reference/config-yaml-reference.md | 2 +- 6 files changed, 83 insertions(+), 115 deletions(-) create mode 100644 content/embeds/rdi-snapshot-modes.md diff --git a/content/embeds/rdi-snapshot-modes.md b/content/embeds/rdi-snapshot-modes.md new file mode 100644 index 0000000000..f8134d2cca --- /dev/null +++ b/content/embeds/rdi-snapshot-modes.md @@ -0,0 +1,55 @@ +The collector saves an *offset*: its position in the source's change history. +A missing offset means there is no saved position, as on first deployment or after a +reset. An unavailable position means an offset exists, but the source no longer has +the log history needed to resume from it. + +With the default `initial` mode, the collector snapshots when there is no saved +position, then streams changes. A restart with a completed snapshot and a valid saved +position resumes streaming. Deployment or restart alone does not require a new snapshot. +If the saved position is no longer available, `initial` does not automatically +re-snapshot to recover, and the collector can report an error. + +With `when_needed`, the collector also takes a new snapshot when it detects that the +saved position is no longer available. This can reload all selected data after an +outage without a manual reset. The snapshot captures current rows, rather than +reconstructing every missed change. + +#### Missed deletes can leave stale target records {#missed-deletes} + +{{< embed-md "rdi-snapshot-gap.md" >}} + +#### Compare snapshot modes + +The available modes and their exact behavior depend on the source connector and its +version. The following table describes the six values listed in the RDI configuration +reference. A listed value is not necessarily supported by every connector. + +| Mode | Existing source data | Restart with a saved position | After a reset clears the position | +| :-- | :-- | :-- | :-- | +| `initial` (default) | Snapshot when no offset exists, then stream changes. | Resume after a completed snapshot if the position is valid. No automatic snapshot recovery for unavailable log history. | Snapshot, then stream changes. | +| `when_needed` | Snapshot when no offset exists, then stream changes. | Resume if the position is valid. Snapshot again if the connector detects an unavailable position. | Snapshot, then stream changes. | +| `always` | Snapshot on every collector start, then stream changes. | Take a new snapshot even if the position is valid. | Snapshot, then stream changes. | +| `initial_only` | Take an initial snapshot without streaming subsequent changes. | Restart an interrupted initial snapshot; do not capture subsequent changes after it completes. | Snapshot without ongoing change capture. | +| `no_data` | Skip existing rows. Initialize any required schema or source position, then stream changes. | Resume streaming if the position is valid. No data snapshot to recover unavailable history. | Skip existing rows and start streaming according to the connector's rules. | +| `never` | Skip the data snapshot where this mode is supported. | Resume according to the connector's rules; no data snapshot recovery. | Connector-specific startup behavior; existing rows are not reloaded. | + +An interrupted initial snapshot can restart when the collector starts again, including +with `initial` and `initial_only`. + +For Debezium 3.5, note these connector differences: + +- PostgreSQL `no_data` performs no snapshot. It resumes from a saved log sequence + number (LSN), or from the replication slot's creation position when no LSN is saved. + This requires the relevant write-ahead log (WAL) history to remain available. +- Other connectors can capture schema or initialize a position without copying + existing rows in `no_data` mode. +- `never` is a deprecated alias for `no_data` in PostgreSQL and MongoDB. MySQL and + MariaDB retain `never` with connector-specific prerequisites. Oracle and SQL Server + do not offer `never`; use `no_data` to skip existing rows. + +See the Debezium 3.5 connector references for [MariaDB](https://debezium.io/documentation/reference/3.5/connectors/mariadb.html), +[MongoDB](https://debezium.io/documentation/reference/3.5/connectors/mongodb.html), +[MySQL](https://debezium.io/documentation/reference/3.5/connectors/mysql.html), +[Oracle](https://debezium.io/documentation/reference/3.5/connectors/oracle.html), +[PostgreSQL](https://debezium.io/documentation/reference/3.5/connectors/postgresql.html), +and [SQL Server](https://debezium.io/documentation/reference/3.5/connectors/sqlserver.html). diff --git a/content/integrate/redis-data-integration/1.19.1/architecture/_index.md b/content/integrate/redis-data-integration/1.19.1/architecture/_index.md index 5ab73690af..b6c8f5918a 100644 --- a/content/integrate/redis-data-integration/1.19.1/architecture/_index.md +++ b/content/integrate/redis-data-integration/1.19.1/architecture/_index.md @@ -72,8 +72,6 @@ added to the target within a few seconds after capture. ## At-least-once delivery guarantee - - RDI guarantees *at-least-once delivery* to the target. This means that a given change will never be lost, but it might be added to the target more than once. Apart from a slight performance overhead, adding a @@ -82,6 +80,15 @@ are [*idempotent*](https://en.wikipedia.org/wiki/Idempotence) (that is to say that all writes after the first one make no change to the overall state). +{{< note >}} +The delivery guarantee applies to change events captured by RDI. If a Debezium +collector's saved source position becomes unavailable, a new snapshot cannot replay +changes whose source log history has expired. Missed deletes can leave stale records +in the target. See the +[missed-delete timeline](/content/integrate/redis-data-integration/1.19.1/data-pipelines/pipeline-config.md#missed-deletes) +and [recovery guidance](/content/integrate/redis-data-integration/1.19.1/troubleshooting.md#unavailable-source-history). +{{< /note >}} + ## Checkpointing RDI uses Redis streams to store the sequence of change events diff --git a/content/integrate/redis-data-integration/1.19.1/data-pipelines/pipeline-config.md b/content/integrate/redis-data-integration/1.19.1/data-pipelines/pipeline-config.md index cc5e29d13a..fb962db493 100644 --- a/content/integrate/redis-data-integration/1.19.1/data-pipelines/pipeline-config.md +++ b/content/integrate/redis-data-integration/1.19.1/data-pipelines/pipeline-config.md @@ -194,61 +194,10 @@ targets: password: ${TARGET_DB_PASSWORD} ``` -The collector saves an *offset*: its position in the source's change history. -A missing offset means there is no saved position, as on first deployment or after a -reset. An unavailable position means an offset exists, but the source no longer has -the log history needed to resume from it. - -With the default `initial` mode, the collector snapshots when there is no saved -position, then streams changes. A restart with a completed snapshot and a valid saved -position resumes streaming. Deployment or restart alone does not require a new snapshot. -If the saved position is no longer available, `initial` does not automatically -re-snapshot to recover, and the collector can report an error. - -With `when_needed`, the collector also takes a new snapshot when it detects that the -saved position is no longer available. This can reload all selected data after an -outage without a manual reset. The snapshot captures current rows, rather than -reconstructing every missed change. - -#### Missed deletes can leave stale target records {#missed-deletes} - -{{< embed-md "rdi-snapshot-gap.md" >}} - -#### Compare snapshot modes - -The available modes and their exact behavior depend on the source connector and its -version. The following table describes the six values listed in the RDI configuration -reference. A listed value is not necessarily supported by every connector. - -| Mode | Existing source data | Restart with a saved position | After a reset clears the position | -| :-- | :-- | :-- | :-- | -| `initial` (default) | Snapshot when no offset exists, then stream changes. | Resume after a completed snapshot if the position is valid. No automatic snapshot recovery for unavailable log history. | Snapshot, then stream changes. | -| `when_needed` | Snapshot when no offset exists, then stream changes. | Resume if the position is valid. Snapshot again if the connector detects an unavailable position. | Snapshot, then stream changes. | -| `always` | Snapshot on every collector start, then stream changes. | Take a new snapshot even if the position is valid. | Snapshot, then stream changes. | -| `initial_only` | Take an initial snapshot without streaming subsequent changes. | Restart an interrupted initial snapshot; do not capture subsequent changes after it completes. | Snapshot without ongoing change capture. | -| `no_data` | Skip existing rows. Initialize any required schema or source position, then stream changes. | Resume streaming if the position is valid. No data snapshot to recover unavailable history. | Skip existing rows and start streaming according to the connector's rules. | -| `never` | Skip the data snapshot where this mode is supported. | Resume according to the connector's rules; no data snapshot recovery. | Connector-specific startup behavior; existing rows are not reloaded. | - -An interrupted initial snapshot can restart when the collector starts again, including -with `initial` and `initial_only`. - -For Debezium 3.5, note these connector differences: - -- PostgreSQL `no_data` performs no snapshot. It resumes from a saved log sequence - number (LSN), or from the replication slot's creation position when no LSN is saved. - This requires the relevant write-ahead log (WAL) history to remain available. -- Other connectors can capture schema or initialize a position without copying - existing rows in `no_data` mode. -- `never` is a deprecated alias for `no_data` in PostgreSQL and MongoDB. MySQL and - MariaDB retain `never` with connector-specific prerequisites. Oracle and SQL Server - do not offer `never`; use `no_data` to skip existing rows. - -See the Debezium 3.5 connector references for [MariaDB](https://debezium.io/documentation/reference/3.5/connectors/mariadb.html), -[MongoDB](https://debezium.io/documentation/reference/3.5/connectors/mongodb.html), -[MySQL](https://debezium.io/documentation/reference/3.5/connectors/mysql.html), -[Oracle](https://debezium.io/documentation/reference/3.5/connectors/oracle.html), -[PostgreSQL](https://debezium.io/documentation/reference/3.5/connectors/postgresql.html), -and [SQL Server](https://debezium.io/documentation/reference/3.5/connectors/sqlserver.html). +RDI 1.19.1 bundles Debezium 3.5. The comparison and connector notes in this +section apply to that release. + +{{< embed-md "rdi-snapshot-modes.md" >}} A reset clears saved positions without changing `snapshot.mode` or flushing the target Redis database. If you need to reload existing rows, select a mode that takes a data diff --git a/content/integrate/redis-data-integration/architecture/_index.md b/content/integrate/redis-data-integration/architecture/_index.md index e1667e67f8..8053c05634 100644 --- a/content/integrate/redis-data-integration/architecture/_index.md +++ b/content/integrate/redis-data-integration/architecture/_index.md @@ -72,8 +72,6 @@ added to the target within a few seconds after capture. ## At-least-once delivery guarantee - - RDI guarantees *at-least-once delivery* to the target. This means that a given change will never be lost, but it might be added to the target more than once. Apart from a slight performance overhead, adding a @@ -82,6 +80,15 @@ are [*idempotent*](https://en.wikipedia.org/wiki/Idempotence) (that is to say that all writes after the first one make no change to the overall state). +{{< note >}} +The delivery guarantee applies to change events captured by RDI. If a Debezium +collector's saved source position becomes unavailable, a new snapshot cannot replay +changes whose source log history has expired. Missed deletes can leave stale records +in the target. See the +[missed-delete timeline](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#missed-deletes) +and [recovery guidance](/content/integrate/redis-data-integration/troubleshooting.md#unavailable-source-history). +{{< /note >}} + ## Checkpointing RDI uses Redis streams to store the sequence of change events diff --git a/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md b/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md index ef6deb70da..33205175a5 100644 --- a/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md +++ b/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md @@ -204,61 +204,11 @@ targets: password: ${TARGET_DB_PASSWORD} ``` -The collector saves an *offset*: its position in the source's change history. -A missing offset means there is no saved position, as on first deployment or after a -reset. An unavailable position means an offset exists, but the source no longer has -the log history needed to resume from it. - -With the default `initial` mode, the collector snapshots when there is no saved -position, then streams changes. A restart with a completed snapshot and a valid saved -position resumes streaming. Deployment or restart alone does not require a new snapshot. -If the saved position is no longer available, `initial` does not automatically -re-snapshot to recover, and the collector can report an error. - -With `when_needed`, the collector also takes a new snapshot when it detects that the -saved position is no longer available. This can reload all selected data after an -outage without a manual reset. The snapshot captures current rows, rather than -reconstructing every missed change. - -#### Missed deletes can leave stale target records {#missed-deletes} - -{{< embed-md "rdi-snapshot-gap.md" >}} - -#### Compare snapshot modes - -The available modes and their exact behavior depend on the source connector and its -version. The following table describes the six values listed in the RDI configuration -reference. A listed value is not necessarily supported by every connector. - -| Mode | Existing source data | Restart with a saved position | After a reset clears the position | -| :-- | :-- | :-- | :-- | -| `initial` (default) | Snapshot when no offset exists, then stream changes. | Resume after a completed snapshot if the position is valid. No automatic snapshot recovery for unavailable log history. | Snapshot, then stream changes. | -| `when_needed` | Snapshot when no offset exists, then stream changes. | Resume if the position is valid. Snapshot again if the connector detects an unavailable position. | Snapshot, then stream changes. | -| `always` | Snapshot on every collector start, then stream changes. | Take a new snapshot even if the position is valid. | Snapshot, then stream changes. | -| `initial_only` | Take an initial snapshot without streaming subsequent changes. | Restart an interrupted initial snapshot; do not capture subsequent changes after it completes. | Snapshot without ongoing change capture. | -| `no_data` | Skip existing rows. Initialize any required schema or source position, then stream changes. | Resume streaming if the position is valid. No data snapshot to recover unavailable history. | Skip existing rows and start streaming according to the connector's rules. | -| `never` | Skip the data snapshot where this mode is supported. | Resume according to the connector's rules; no data snapshot recovery. | Connector-specific startup behavior; existing rows are not reloaded. | - -An interrupted initial snapshot can restart when the collector starts again, including -with `initial` and `initial_only`. - -For Debezium 3.5, note these connector differences: - -- PostgreSQL `no_data` performs no snapshot. It resumes from a saved log sequence - number (LSN), or from the replication slot's creation position when no LSN is saved. - This requires the relevant write-ahead log (WAL) history to remain available. -- Other connectors can capture schema or initialize a position without copying - existing rows in `no_data` mode. -- `never` is a deprecated alias for `no_data` in PostgreSQL and MongoDB. MySQL and - MariaDB retain `never` with connector-specific prerequisites. Oracle and SQL Server - do not offer `never`; use `no_data` to skip existing rows. - -See the Debezium 3.5 connector references for [MariaDB](https://debezium.io/documentation/reference/3.5/connectors/mariadb.html), -[MongoDB](https://debezium.io/documentation/reference/3.5/connectors/mongodb.html), -[MySQL](https://debezium.io/documentation/reference/3.5/connectors/mysql.html), -[Oracle](https://debezium.io/documentation/reference/3.5/connectors/oracle.html), -[PostgreSQL](https://debezium.io/documentation/reference/3.5/connectors/postgresql.html), -and [SQL Server](https://debezium.io/documentation/reference/3.5/connectors/sqlserver.html). +RDI 1.19.1 and RDI 2.0.0 bundle Debezium 3.5. The comparison and connector +notes in this section apply to those releases. RDI 2.2.0 is planned to use +Debezium 3.7. + +{{< embed-md "rdi-snapshot-modes.md" >}} A reset clears saved positions without changing `snapshot.mode` or flushing the target Redis database. If you need to reload existing rows, select a mode that takes a data diff --git a/content/integrate/redis-data-integration/reference/config-yaml-reference.md b/content/integrate/redis-data-integration/reference/config-yaml-reference.md index 937f72c693..1f90237053 100644 --- a/content/integrate/redis-data-integration/reference/config-yaml-reference.md +++ b/content/integrate/redis-data-integration/reference/config-yaml-reference.md @@ -490,7 +490,7 @@ Advanced configuration properties for the source database connection and CDC beh |Name|Type|Description|Required| |----|----|-----------|--------| |**record\.processing\.threads**
(Record processing threads)|`integer`|Controls how many worker threads process captured records before they are written downstream.
Minimum: `1`
|| -|**snapshot\.mode**
(Snapshot mode)|`string`|Determines when a Debezium collector snapshots selected source data. Available modes and restart behavior depend on the source connector. A reset clears the saved offset without changing this mode or flushing the target. A new snapshot can leave stale target records after missed deletes. See [Choose a snapshot mode](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#choose-a-snapshot-mode) for the mode comparison and missed-delete timeline.
Default: `"initial"`
Enum: `"always"`, `"initial"`, `"initial_only"`, `"never"`, `"no_data"`, `"when_needed"`
|| +|**snapshot\.mode**
(Snapshot mode)|`string`|Determines when a Debezium collector snapshots selected source data. Available modes and restart behavior depend on the source connector. A reset clears the saved offset without changing this mode or flushing the target. A new snapshot can leave stale target records after missed deletes. See [Choose a snapshot mode](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#choose-a-snapshot-mode) for the mode comparison.
Default: `"initial"`
Enum: `"always"`, `"initial"`, `"initial_only"`, `"never"`, `"no_data"`, `"when_needed"`
|| |**snapshot\.max\.threads**
(Snapshot max threads)|`integer`|Sets the maximum number of threads used while taking the initial snapshot.
Default: `1`
Minimum: `1`
|| |**poll\.interval\.ms**
(Poll interval ms)|`integer`|Defines how often the collector polls the source for new changes.
Default: `500`
Minimum: `1`
|| |**snapshot\.fetch\.size**
(Snapshot fetch size)|`integer`|Defines how many rows are fetched per batch during the initial snapshot.
Minimum: `0`
|| From f6d2bb1e26f121d97f3e0f71641a39cdacd49fe6 Mon Sep 17 00:00:00 2001 From: Zdravko Donev Date: Tue, 13 Oct 2026 08:44:20 +0300 Subject: [PATCH 05/76] Address RDI snapshot mode review comments --- content/embeds/rdi-snapshot-modes.md | 4 +- .../1.19.1/troubleshooting.md | 4 +- .../data-pipelines/deploy.md | 2 +- .../data-pipelines/multiple-sources.md | 2 +- .../data-pipelines/pipeline-config.md | 2 +- .../redis-data-integration/troubleshooting.md | 4 +- content/operate/rc/rdi/define.md | 4 +- content/operate/rc/rdi/faq.md | 12 +- content/operate/rc/rdi/view-edit.md | 2 +- layouts/shortcodes/rdi-snapshot-timeline.html | 23 --- static/css/rdi-snapshot-timeline.css | 138 ------------------ 11 files changed, 20 insertions(+), 177 deletions(-) delete mode 100644 layouts/shortcodes/rdi-snapshot-timeline.html delete mode 100644 static/css/rdi-snapshot-timeline.css diff --git a/content/embeds/rdi-snapshot-modes.md b/content/embeds/rdi-snapshot-modes.md index f8134d2cca..fd0e5af6a6 100644 --- a/content/embeds/rdi-snapshot-modes.md +++ b/content/embeds/rdi-snapshot-modes.md @@ -1,4 +1,4 @@ -The collector saves an *offset*: its position in the source's change history. +The collector saves an *offset*, which is its position in the source's change history. A missing offset means there is no saved position, as on first deployment or after a reset. An unavailable position means an offset exists, but the source no longer has the log history needed to resume from it. @@ -9,7 +9,7 @@ position resumes streaming. Deployment or restart alone does not require a new s If the saved position is no longer available, `initial` does not automatically re-snapshot to recover, and the collector can report an error. -With `when_needed`, the collector also takes a new snapshot when it detects that the +With `when_needed` mode, the collector also takes a new snapshot when it detects that the saved position is no longer available. This can reload all selected data after an outage without a manual reset. The snapshot captures current rows, rather than reconstructing every missed change. diff --git a/content/integrate/redis-data-integration/1.19.1/troubleshooting.md b/content/integrate/redis-data-integration/1.19.1/troubleshooting.md index 4c10e5f127..75ab72f723 100644 --- a/content/integrate/redis-data-integration/1.19.1/troubleshooting.md +++ b/content/integrate/redis-data-integration/1.19.1/troubleshooting.md @@ -63,8 +63,8 @@ If the collector cannot resume from its saved position, check the collector sour for unavailable or expired source log history. This differs from a missing offset, which is expected on first deployment or after a reset. -With `initial`, the collector can fail rather than start a new snapshot when the saved -position is unavailable. With `when_needed`, it can automatically snapshot the current +With `initial` mode, the collector can fail rather than start a new snapshot when the saved +position is unavailable. With `when_needed` mode, it can automatically snapshot the current selected data and resume streaming. A new snapshot after a restart can therefore indicate that the source history was no longer available. See [Choose a snapshot mode](/content/integrate/redis-data-integration/1.19.1/data-pipelines/pipeline-config.md#choose-a-snapshot-mode) diff --git a/content/integrate/redis-data-integration/data-pipelines/deploy.md b/content/integrate/redis-data-integration/data-pipelines/deploy.md index d6debc601f..02615595f9 100644 --- a/content/integrate/redis-data-integration/data-pipelines/deploy.md +++ b/content/integrate/redis-data-integration/data-pipelines/deploy.md @@ -366,7 +366,7 @@ to clear the pipeline's saved source positions. With the default `initial` snaps mode, a Debezium collector reloads selected source data and then resumes change data capture (CDC). Reset does not change the configured [snapshot mode](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#choose-a-snapshot-mode). -Modes that skip existing rows do not reload them after a reset, and `initial_only` +Modes that skip existing rows do not reload them after a reset, and `initial_only` mode does not capture subsequent changes. Reset does not flush the target Redis database. A new snapshot can overwrite target diff --git a/content/integrate/redis-data-integration/data-pipelines/multiple-sources.md b/content/integrate/redis-data-integration/data-pipelines/multiple-sources.md index 5773fe3dbf..5d8a132ee7 100644 --- a/content/integrate/redis-data-integration/data-pipelines/multiple-sources.md +++ b/content/integrate/redis-data-integration/data-pipelines/multiple-sources.md @@ -270,7 +270,7 @@ The reset source then follows its configured [snapshot mode](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#choose-a-snapshot-mode) with no saved position. With the default `initial` mode, it takes a new snapshot and resumes streaming. Every other source keeps its saved position, but all collectors -restart and follow their configured modes. For example, a source using `always` takes +restart and follow their configured modes. For example, a source using `always` mode takes another snapshot on restart even though its position was preserved. RDI stops the whole pipeline while the reset runs and starts it again afterwards. diff --git a/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md b/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md index 33205175a5..36acf0d4d0 100644 --- a/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md +++ b/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md @@ -179,7 +179,7 @@ PostgreSQL, and SQL Server. It does not configure Spanner, Snowflake, or externa collectors. Add the property under the source you want to configure. For example, this file -uses `initial` for a MySQL source: +uses `initial` mode for a MySQL source: ```yaml sources: diff --git a/content/integrate/redis-data-integration/troubleshooting.md b/content/integrate/redis-data-integration/troubleshooting.md index bca017a36d..d4159acee6 100644 --- a/content/integrate/redis-data-integration/troubleshooting.md +++ b/content/integrate/redis-data-integration/troubleshooting.md @@ -63,8 +63,8 @@ If the collector cannot resume from its saved position, check the collector sour for unavailable or expired source log history. This differs from a missing offset, which is expected on first deployment or after a reset. -With `initial`, the collector can fail rather than start a new snapshot when the saved -position is unavailable. With `when_needed`, it can automatically snapshot the current +With `initial` mode, the collector can fail rather than start a new snapshot when the saved +position is unavailable. With `when_needed` mode, it can automatically snapshot the current selected data and resume streaming. A new snapshot after a restart can therefore indicate that the source history was no longer available. See [Choose a snapshot mode](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#choose-a-snapshot-mode) diff --git a/content/operate/rc/rdi/define.md b/content/operate/rc/rdi/define.md index 654f150a19..bef1028d51 100644 --- a/content/operate/rc/rdi/define.md +++ b/content/operate/rc/rdi/define.md @@ -125,9 +125,9 @@ Enter the selected source's database settings. The fields depend on the database Use **Collector properties** for additional source and sink settings. These settings apply to the selected source. See the [collector source properties](/content/integrate/redis-data-integration/reference/config-yaml-reference.md#sourcesadvancedsource-advanced-source-settings) and [collector sink properties](/content/integrate/redis-data-integration/reference/config-yaml-reference.md#sourcesadvancedsink-rdi-collector-stream-writer-configuration). For Debezium sources, set `snapshot.mode` in **Collector source properties** to -control data snapshots. The default is `initial`. Set it for each source separately; +control data snapshots. The default mode is `initial`. Set it for each source separately; available modes depend on the source database. See [Choose a snapshot mode](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#choose-a-snapshot-mode). -With `when_needed`, an unavailable saved position can trigger a new snapshot, but +With `when_needed` mode, an unavailable saved position can trigger a new snapshot, but [missed deletes can leave stale target records](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#missed-deletes). This property does not configure the Snowflake collector. diff --git a/content/operate/rc/rdi/faq.md b/content/operate/rc/rdi/faq.md index 0803963f3f..617b8e13e8 100644 --- a/content/operate/rc/rdi/faq.md +++ b/content/operate/rc/rdi/faq.md @@ -27,10 +27,14 @@ collection was interrupted and the required log history expired, the snapshot ca replay its delete event. This applies to Debezium sources with `type: cdc`, including automatic snapshots with `when_needed` and snapshots after a manual reset. -The timeline follows one row through an automatic recovery snapshot. Read the numbered -events in order. On wide screens, events alternate across a horizontal timeline. +The sequence is: -{{< rdi-snapshot-timeline >}} +1. A source row is copied to Redis. +1. Collection stops, and the collector saves its source position. +1. The source deletes the row while Redis still has it. +1. The saved position becomes unavailable, so the delete cannot be replayed. +1. `when_needed` mode takes a new snapshot. The deleted row is absent. +1. Streaming resumes, but Redis still has the stale row because its delete was missed. An interruption alone does not cause this gap. The saved position must become unavailable. The snapshot does not clean up the target, and a reset does not flush it. @@ -62,7 +66,7 @@ RDI reloads data available in the source databases using the current dataset and ### What happens when I reset one source? {#reset-one-source} -RDI clears the selected source's internal state, including its saved position. With the default `initial` mode, a Debezium source takes a new data snapshot. Other [snapshot modes](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#choose-a-snapshot-mode) can skip existing rows or omit ongoing change capture. The whole pipeline, including the other sources and the processor, restarts. Other sources keep their saved positions but follow their configured modes on restart; `always` takes another snapshot even with a preserved position. +RDI clears the selected source's internal state, including its saved position. With the default `initial` mode, a Debezium source takes a new data snapshot. Other [snapshot modes](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#choose-a-snapshot-mode) can skip existing rows or omit ongoing change capture. The whole pipeline, including the other sources and the processor, restarts. Other sources keep their saved positions but follow their configured modes on restart; `always` mode takes another snapshot even with a preserved position. All records already in the target remain, including records from the reset source. The new snapshot can overwrite that source's records. Resetting a source does not selectively delete its target data. diff --git a/content/operate/rc/rdi/view-edit.md b/content/operate/rc/rdi/view-edit.md index 5d950a7278..1565673f16 100644 --- a/content/operate/rc/rdi/view-edit.md +++ b/content/operate/rc/rdi/view-edit.md @@ -75,7 +75,7 @@ Resetting one source clears its saved position. With the default `initial` mode, ![Reset source confirmation explaining that other sources retain their data and the pipeline temporarily stops.](/images/rc/rdi/rdi-2-reset-source.png) {width="600px"} -The reset clears that source's internal RDI streams, offsets, schema history, rejected records, and processing counters. The whole pipeline and all its sources restart during the reset. Other sources keep their saved positions. Debezium sources restart according to their snapshot modes; for example, `always` takes another snapshot even when its saved position is preserved. All records already in the shared target database remain, including those from the reset source. The new snapshot can overwrite records for that source. See [What happens when I reset one source?](/content/operate/rc/rdi/faq.md#reset-one-source). +The reset clears that source's internal RDI streams, offsets, schema history, rejected records, and processing counters. The whole pipeline and all its sources restart during the reset. Other sources keep their saved positions. Debezium sources restart according to their snapshot modes; for example, `always` mode takes another snapshot even when its saved position is preserved. All records already in the shared target database remain, including those from the reset source. The new snapshot can overwrite records for that source. See [What happens when I reset one source?](/content/operate/rc/rdi/faq.md#reset-one-source). ### Remove a source {#remove-source} diff --git a/layouts/shortcodes/rdi-snapshot-timeline.html b/layouts/shortcodes/rdi-snapshot-timeline.html deleted file mode 100644 index 33f821f25d..0000000000 --- a/layouts/shortcodes/rdi-snapshot-timeline.html +++ /dev/null @@ -1,23 +0,0 @@ -{{- $css := "css/rdi-snapshot-timeline.css" -}} - -
-
    - {{- range $i, $event := slice - (dict "title" "Row copied" "text" "Row 42 exists in the source and in Redis.") - (dict "title" "Collection stops" "text" "The collector saves its source position.") - (dict "title" "Row deleted" "text" "The source deletes row 42. Redis still has it.") - (dict "title" "History expires" "text" "The saved position is unavailable. The delete cannot be replayed.") - (dict "title" "New snapshot" "text" "when_needed copies current source rows. Row 42 is absent.") - (dict "title" "Stale record" "text" "Streaming resumes. Redis still has row 42: its delete was missed.") - }} -
  1. - -
    - {{ $event.title }} -

    {{ $event.text }}

    -
    -
    2. - {{- end }} -
-
A new snapshot copies existing source rows. It does not replay missed deletes or remove target records for rows absent from the snapshot.
-
diff --git a/static/css/rdi-snapshot-timeline.css b/static/css/rdi-snapshot-timeline.css deleted file mode 100644 index cf8537bb84..0000000000 --- a/static/css/rdi-snapshot-timeline.css +++ /dev/null @@ -1,138 +0,0 @@ -.rdi-snapshot-timeline { - container-type: inline-size; - margin: 1.5rem 0; - padding: 1rem; - border: 1px solid #d5dce5; - border-radius: 8px; - background: #f8fafc; - color: #172b4d; -} - -.rdi-snapshot-timeline .rdi-snapshot-timeline__events { - position: relative; - display: grid; - gap: 1.5rem; - margin: 0; - padding: 0; - list-style: none; -} - -.rdi-snapshot-timeline__events::before { - position: absolute; - top: 16px; - bottom: 16px; - left: 15px; - width: 2px; - background: #64748b; - content: ""; -} - -.rdi-snapshot-timeline .rdi-snapshot-timeline__event { - --event-color: #b44724; - position: relative; - display: grid; - grid-template-columns: 32px 1fr; - gap: 1rem; - margin: 0; - padding: 0; -} - -.rdi-snapshot-timeline__event:nth-child(2) { --event-color: #925a00; } -.rdi-snapshot-timeline__event:nth-child(3) { --event-color: #247548; } -.rdi-snapshot-timeline__event:nth-child(4) { --event-color: #087b94; } -.rdi-snapshot-timeline__event:nth-child(5) { --event-color: #365caa; } -.rdi-snapshot-timeline__event:nth-child(6) { --event-color: #172b4d; } - -.rdi-snapshot-timeline__number { - position: relative; - display: grid; - place-items: center; - width: 32px; - height: 32px; - border-radius: 50%; - background: var(--event-color); - color: #fff; - font-size: 16px; - font-weight: 700; - line-height: 1; -} - -.rdi-snapshot-timeline__detail strong { - color: var(--event-color); - font-size: 16px; - line-height: 1.4; -} - -.rdi-snapshot-timeline .rdi-snapshot-timeline__detail p { - margin: 0.5rem 0 0; - font-size: 14px; - line-height: 1.5; -} - -.rdi-snapshot-timeline figcaption { - margin-top: 1.5rem; - padding-top: 1rem; - border-top: 1px solid #d5dce5; - color: #172b4d; - font-size: 14px; - font-weight: 600; - line-height: 1.5; -} - -@container (min-width: 550px) { - .rdi-snapshot-timeline .rdi-snapshot-timeline__events { - grid-template-columns: repeat(6, minmax(0, 1fr)); - gap: 0.5rem; - } - - .rdi-snapshot-timeline__events::before { - top: 50%; - right: 0; - bottom: auto; - left: 0; - width: auto; - height: 2px; - } - - .rdi-snapshot-timeline .rdi-snapshot-timeline__event { - grid-template-columns: 1fr; - grid-template-rows: 190px 32px 190px; - gap: 24px; - justify-items: center; - text-align: center; - } - - .rdi-snapshot-timeline__number { - grid-row: 2; - } - - .rdi-snapshot-timeline__detail { - position: relative; - grid-row: 1; - align-self: end; - } - - .rdi-snapshot-timeline__detail strong { - font-size: 14px; - } - - .rdi-snapshot-timeline__detail::after { - position: absolute; - bottom: -24px; - left: calc(50% - 1px); - width: 2px; - height: 24px; - background: var(--event-color); - content: ""; - } - - .rdi-snapshot-timeline__event:nth-child(even) .rdi-snapshot-timeline__detail { - grid-row: 3; - align-self: start; - } - - .rdi-snapshot-timeline__event:nth-child(even) .rdi-snapshot-timeline__detail::after { - top: -24px; - bottom: auto; - } -} From e4bb092bf695db63ca6d3e8c55363f816d9ff4f0 Mon Sep 17 00:00:00 2001 From: Zdravko Donev Date: Tue, 13 Oct 2026 23:35:12 +0300 Subject: [PATCH 06/76] Address additional RDI review comments --- content/embeds/rdi-snapshot-modes.md | 2 +- .../redis-data-integration/1.19.1/data-pipelines/deploy.md | 3 ++- .../1.19.1/reference/config-yaml-reference.md | 2 +- 3 files changed, 4 insertions(+), 3 deletions(-) diff --git a/content/embeds/rdi-snapshot-modes.md b/content/embeds/rdi-snapshot-modes.md index fd0e5af6a6..b4c6b6ca1f 100644 --- a/content/embeds/rdi-snapshot-modes.md +++ b/content/embeds/rdi-snapshot-modes.md @@ -7,7 +7,7 @@ With the default `initial` mode, the collector snapshots when there is no saved position, then streams changes. A restart with a completed snapshot and a valid saved position resumes streaming. Deployment or restart alone does not require a new snapshot. If the saved position is no longer available, `initial` does not automatically -re-snapshot to recover, and the collector can report an error. +re-snapshot to recover, and the collector stops and reports an error. With `when_needed` mode, the collector also takes a new snapshot when it detects that the saved position is no longer available. This can reload all selected data after an diff --git a/content/integrate/redis-data-integration/1.19.1/data-pipelines/deploy.md b/content/integrate/redis-data-integration/1.19.1/data-pipelines/deploy.md index 800b1c15dd..cb1bcd3db3 100644 --- a/content/integrate/redis-data-integration/1.19.1/data-pipelines/deploy.md +++ b/content/integrate/redis-data-integration/1.19.1/data-pipelines/deploy.md @@ -327,6 +327,7 @@ to resume it. Stopping a pipeline halts data processing without deleting the pip configuration. A Debezium source restarts according to its [snapshot mode](/content/integrate/redis-data-integration/1.19.1/data-pipelines/pipeline-config.md#choose-a-snapshot-mode) and the availability of its saved source position. +If you keep the source stopped for long periods, the stored offset may be removed, making it impossible to resume streaming. ```bash redis-di stop @@ -388,4 +389,4 @@ redis-di set-secret SOURCE_DB_USERNAME newUsername --wait=false redis-di set-secret SOURCE_DB_PASSWORD newPassword ``` -The same applies to any set of changes that are only valid together. \ No newline at end of file +The same applies to any set of changes that are only valid together. diff --git a/content/integrate/redis-data-integration/1.19.1/reference/config-yaml-reference.md b/content/integrate/redis-data-integration/1.19.1/reference/config-yaml-reference.md index 80103a06e8..8ef36e6fd8 100644 --- a/content/integrate/redis-data-integration/1.19.1/reference/config-yaml-reference.md +++ b/content/integrate/redis-data-integration/1.19.1/reference/config-yaml-reference.md @@ -459,7 +459,7 @@ redis.wait.retry.delay.ms: 1000 #### sources\.advanced\.source: Advanced source settings -Advanced configuration properties for the source database connection and CDC behavior. **Applies to the `cdc` and `flink` collector types.**

For the `cdc` collector type, available properties depend on the source database — refer to the relevant Debezium connector documentation: [MySQL](https://debezium.io/documentation/reference/stable/connectors/mysql.html), [MariaDB](https://debezium.io/documentation/reference/stable/connectors/mariadb.html), [PostgreSQL](https://debezium.io/documentation/reference/stable/connectors/postgresql.html), [Oracle](https://debezium.io/documentation/reference/stable/connectors/oracle.html), [SQL Server](https://debezium.io/documentation/reference/stable/connectors/sqlserver.html), [Db2](https://debezium.io/documentation/reference/stable/connectors/db2.html), [MongoDB](https://debezium.io/documentation/reference/stable/connectors/mongodb.html). When using a property from those pages, omit the `debezium.source.` prefix.

Set `snapshot.mode` as a free-form source property to control data snapshots. The default is `initial`; available modes and restart behavior depend on the connector. A reset clears the saved position without changing the mode. See [Choose a snapshot mode](/content/integrate/redis-data-integration/1.19.1/data-pipelines/pipeline-config.md#choose-a-snapshot-mode), including the [missed-delete timeline](/content/integrate/redis-data-integration/1.19.1/data-pipelines/pipeline-config.md#missed-deletes).

**The named properties below cover the most commonly tuned settings: `spanner.*` properties apply to the `flink` collector type, all others apply to the `cdc` collector type. Any other property from the Debezium documentation can still be set as a free-form key-value pair.** +Advanced configuration properties for the source database connection and CDC behavior. **Applies to the `cdc` and `flink` collector types.**

For the `cdc` collector type, available properties depend on the source database — refer to the relevant Debezium connector documentation: [MySQL](https://debezium.io/documentation/reference/stable/connectors/mysql.html), [MariaDB](https://debezium.io/documentation/reference/stable/connectors/mariadb.html), [PostgreSQL](https://debezium.io/documentation/reference/stable/connectors/postgresql.html), [Oracle](https://debezium.io/documentation/reference/stable/connectors/oracle.html), [SQL Server](https://debezium.io/documentation/reference/stable/connectors/sqlserver.html), [MongoDB](https://debezium.io/documentation/reference/stable/connectors/mongodb.html). When using a property from those pages, omit the `debezium.source.` prefix.

Set `snapshot.mode` as a free-form source property to control data snapshots. The default is `initial`; available modes and restart behavior depend on the connector. A reset clears the saved position without changing the mode. See [Choose a snapshot mode](/content/integrate/redis-data-integration/1.19.1/data-pipelines/pipeline-config.md#choose-a-snapshot-mode), including the [missed-delete timeline](/content/integrate/redis-data-integration/1.19.1/data-pipelines/pipeline-config.md#missed-deletes).

**The named properties below cover the most commonly tuned settings: `spanner.*` properties apply to the `flink` collector type, all others apply to the `cdc` collector type. Any other property from the Debezium documentation can still be set as a free-form key-value pair.** **Properties** From d87ebb86e13654733d784c6b70151357fe8ef7d9 Mon Sep 17 00:00:00 2001 From: Zdravko Donev Date: Tue, 13 Oct 2026 23:41:57 +0300 Subject: [PATCH 07/76] Clarify RDI source restart behavior --- content/embeds/rdi-snapshot-modes.md | 8 ++++++++ .../redis-data-integration/1.19.1/troubleshooting.md | 8 ++++++++ .../redis-data-integration/troubleshooting.md | 10 +++++++++- content/operate/rc/rdi/view-edit.md | 7 ++++++- 4 files changed, 31 insertions(+), 2 deletions(-) diff --git a/content/embeds/rdi-snapshot-modes.md b/content/embeds/rdi-snapshot-modes.md index b4c6b6ca1f..84d463c4f6 100644 --- a/content/embeds/rdi-snapshot-modes.md +++ b/content/embeds/rdi-snapshot-modes.md @@ -14,6 +14,14 @@ saved position is no longer available. This can reload all selected data after a outage without a manual reset. The snapshot captures current rows, rather than reconstructing every missed change. +If the collector restarts during a snapshot, the snapshot starts again. Rows deleted +while the snapshot is interrupted can remain as stale records in the target because +the new snapshot does not replay their delete events. + +Use `when_needed` with monitoring and alerting. Recovery can start a long-running +snapshot, which can increase latency and leave stale target records until recovery +finishes. + #### Missed deletes can leave stale target records {#missed-deletes} {{< embed-md "rdi-snapshot-gap.md" >}} diff --git a/content/integrate/redis-data-integration/1.19.1/troubleshooting.md b/content/integrate/redis-data-integration/1.19.1/troubleshooting.md index 75ab72f723..545793a9ba 100644 --- a/content/integrate/redis-data-integration/1.19.1/troubleshooting.md +++ b/content/integrate/redis-data-integration/1.19.1/troubleshooting.md @@ -70,6 +70,14 @@ indicate that the source history was no longer available. See [Choose a snapshot mode](/content/integrate/redis-data-integration/1.19.1/data-pipelines/pipeline-config.md#choose-a-snapshot-mode) for the connector-specific behavior. +Possible causes include: + +- The collector was stopped long enough for the source change log to expire. +- A configuration change made the collector connect to another server with the same + data but different offsets. +- A cleanup process removed the change log while RDI was running. +- A crash and restore left the source without the saved offset. + Before you use a new snapshot to recover, check for [missed deletes and stale target records](/content/integrate/redis-data-integration/1.19.1/data-pipelines/pipeline-config.md#missed-deletes). A snapshot does not replay lost history or delete target records that are absent from diff --git a/content/integrate/redis-data-integration/troubleshooting.md b/content/integrate/redis-data-integration/troubleshooting.md index d4159acee6..5c254ef13c 100644 --- a/content/integrate/redis-data-integration/troubleshooting.md +++ b/content/integrate/redis-data-integration/troubleshooting.md @@ -64,12 +64,20 @@ for unavailable or expired source log history. This differs from a missing offse which is expected on first deployment or after a reset. With `initial` mode, the collector can fail rather than start a new snapshot when the saved -position is unavailable. With `when_needed` mode, it can automatically snapshot the current +position is unavailable. With `when_needed` mode, it will automatically snapshot the current selected data and resume streaming. A new snapshot after a restart can therefore indicate that the source history was no longer available. See [Choose a snapshot mode](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#choose-a-snapshot-mode) for the connector-specific behavior. +Possible causes include: + +- The collector was stopped long enough for the source change log to expire. +- A configuration change made the collector connect to another server with the same + data but different offsets. +- A cleanup process removed the change log while RDI was running. +- A crash and restore left the source without the saved offset. + Before you use a new snapshot to recover, check for [missed deletes and stale target records](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#missed-deletes). A snapshot does not replay lost history or delete target records that are absent from diff --git a/content/operate/rc/rdi/view-edit.md b/content/operate/rc/rdi/view-edit.md index 1565673f16..ffee8b653b 100644 --- a/content/operate/rc/rdi/view-edit.md +++ b/content/operate/rc/rdi/view-edit.md @@ -62,7 +62,12 @@ When extending an older single-source pipeline, preserve the original source nam 1. On **Dashboard**, open the source card's **More actions** menu. 1. Select **Stop source**, then confirm with **Stop source**. -Stopping one source pauses data capture from that source without stopping the other sources. Starting a Debezium source again follows its [snapshot mode](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#choose-a-snapshot-mode) and the availability of its saved position. To resume a source, open its **More actions** menu and select **Start source**, then confirm with **Start source**. A source processes data only while the pipeline is running. See [How does stopping sources affect billing?](/content/operate/rc/rdi/faq.md#stopping-and-billing). +Stopping one source pauses data capture from that source without stopping the other sources. When you start a source again, its behavior depends on the source type: + +- Debezium sources follow their [snapshot mode](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#choose-a-snapshot-mode) and the availability of their saved position. +- Snowflake sources use RIOT-X and Snowflake Streams. They resume from the saved stream offset when the stream retains the required changes. If the retention window expires while the source is stopped, changes can be lost. Set `clearOffset: true` to restart from the current state. + +To resume a source, open its **More actions** menu and select **Start source**, then confirm with **Start source**. A source processes data only while the pipeline is running. See [How does stopping sources affect billing?](/content/operate/rc/rdi/faq.md#stopping-and-billing). ### Reset a source {#reset-source} From a252aa427d006b49d62a706aa93ac87cd54c16f7 Mon Sep 17 00:00:00 2001 From: Zdravko Donev Date: Thu, 15 Oct 2026 13:56:00 +0300 Subject: [PATCH 08/76] DOC: clarify PostgreSQL publication setup for RDI --- .../aws-aurora-rds/aws-aur-pgsql.md | 12 +- .../data-pipelines/prepare-dbs/postgresql.md | 122 +++++++++++++++++- 2 files changed, 130 insertions(+), 4 deletions(-) diff --git a/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md b/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md index c18fcd7b05..d37ca6c598 100644 --- a/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md +++ b/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md @@ -150,8 +150,18 @@ to capture changes. Replace `` with the schema name and `` with the username of the Debezium user. 1. - Connect to your database as the `postgres` user and create a publication for the database: + Configure the PostgreSQL publication. For most RDI deployments, follow the + [PostgreSQL publication and replica identity guidance]({{< relref "/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql" >}}) + and use `publication.autocreate.mode: filtered` when RDI captures selected tables. + + If you intentionally want to publish every table in the database, first check that + every table has a primary key or another replica identity. Then connect to your database + as the `postgres` user and run: ```sql CREATE PUBLICATION dbz_publication FOR ALL TABLES; ``` + + Do not use this command without checking the replica identity of all tables. A later + `UPDATE` or `DELETE` on a published table without a replica identity can fail in the + application that writes to PostgreSQL. diff --git a/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md b/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md index 55ae12fc33..2e262aed33 100644 --- a/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md +++ b/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md @@ -39,6 +39,7 @@ checklist as you complete each step. - [ ] [Install the logical decoding output plug-in](#install-the-logical-decoding-output-plug-in) - [ ] [Configure the PostgreSQL server](#configure-the-postgresql-server) - [ ] [Set up permissions](#set-up-permissions) +- [ ] [Check replica identity and choose a publication mode](#check-replica-identity-and-choose-a-publication-mode) - [ ] [Set privileges for Debezium to create PostgreSQL publications with pgoutput](#set-privileges-for-debezium-to-create-postgresql-publications-with-pgoutput) - [ ] [Configure PostgreSQL for replication with the Debezium connector host](#configure-postgresql-for-replication-with-the-debezium-connector-host) ``` @@ -183,6 +184,120 @@ needs to perform replications, but you might also need to grant other permission CREATE ROLE REPLICATION LOGIN; ``` +## Check replica identity and choose a publication mode + +Before you start RDI, check the tables that RDI will capture and choose how PostgreSQL +should publish their changes. + +PostgreSQL needs a [replica identity](https://www.postgresql.org/docs/current/logical-replication-publication.html) +to identify the old row when it publishes an `UPDATE` or `DELETE`. By default, PostgreSQL +uses the table's primary key. A table without a primary key has no usable replica identity +unless you configure one. Inserts do not need a replica identity. + +Run the following query to find ordinary tables that do not have a primary key and use the +default identity, or that explicitly have no identity: + +```sql +SELECT n.nspname AS schema_name, + c.relname AS table_name, + CASE c.relreplident + WHEN 'd' THEN 'default' + WHEN 'n' THEN 'nothing' + WHEN 'f' THEN 'full' + WHEN 'i' THEN 'index' + END AS replica_identity +FROM pg_class AS c +JOIN pg_namespace AS n ON n.oid = c.relnamespace +WHERE c.relkind = 'r' + AND n.nspname NOT IN ('pg_catalog', 'information_schema') + AND ( + c.relreplident = 'n' + OR ( + c.relreplident = 'd' + AND NOT EXISTS ( + SELECT 1 + FROM pg_index AS i + WHERE i.indrelid = c.oid + AND i.indisprimary + ) + ) + ) +ORDER BY n.nspname, c.relname; +``` + +If this query returns a table that will be published, add a primary key where possible. +You can also configure a suitable unique index as the replica identity: + +```sql +ALTER TABLE REPLICA IDENTITY USING INDEX ; +``` + +As a fallback, you can record the full old row in the write-ahead log: + +```sql +ALTER TABLE REPLICA IDENTITY FULL; +``` + +`REPLICA IDENTITY FULL` can increase write-ahead log volume and reduce performance, so use +it only when a primary key or suitable unique index is not possible. The `keys` setting in +an RDI table definition creates a key for the Redis record. It does not create a PostgreSQL +primary key or replica identity. + +### Choose a publication mode + +The `publication.autocreate.mode` setting controls how RDI and Debezium manage the +PostgreSQL publication: + +| Mode | Behavior | Recommended use | +| --- | --- | --- | +| `filtered` | Creates or updates the publication to contain the tables configured for the RDI source. | Recommended when RDI captures selected tables. RDI manages the publication when the pipeline is redeployed with a changed table filter. | +| `disabled` | Does not create or change the publication. The publication must already exist. | Use when a database administrator manages the publication. | +| `all_tables` | Creates a `FOR ALL TABLES` publication when the publication does not exist. If it already exists, RDI uses the existing publication. | Use when all tables in the database should be published, after checking their replica identities. This is the RDI default. | + +The `all_tables` mode can publish changes for tables that are not listed in the RDI +pipeline. If an application later updates or deletes a row in a published table without +a replica identity, PostgreSQL can reject that application write. For this reason, +`filtered` is the recommended mode for a pipeline that captures only selected tables. + +For a customer-managed publication, create an explicit table list and disable automatic +publication management. For example: + +```sql +CREATE PUBLICATION rdi_publication +FOR TABLE public.customers, public.orders; +``` + +Then set the following source properties: + +```yaml +publication.name: rdi_publication +publication.autocreate.mode: disabled +``` + +### PostgreSQL publication error FAQ + +#### I get this PostgreSQL error. What should I do? + +You may see the following error when an application updates a published table without a +replica identity: + +```text +SQLSTATE 55000 +ERROR: cannot update table "example_table" because it does not have a replica identity and publishes updates +HINT: To enable updating the table, set REPLICA IDENTITY using ALTER TABLE. +``` + +Check whether the table should be captured by RDI. If it should, add a primary key or +configure a suitable replica identity. As a fallback, run: + +```sql +ALTER TABLE REPLICA IDENTITY FULL; +``` + +If the table should not be captured, use `publication.autocreate.mode: filtered` or remove +the table from the customer-managed publication. If you changed the RDI table filters or +publication mode, redeploy the pipeline so RDI applies the new configuration. + ## Set privileges for Debezium to create PostgreSQL publications with `pgoutput` The Debezium user needs specific permissions to work with the `pgoutput` plug-in. @@ -241,10 +356,11 @@ the replication group using the following commands: ALTER TABLE table_name OWNER TO replication_group; ``` -You must also set the value of the `publication.autocreate.mode` parameter to `filtered` -to allow Debezium to specify the publication configuration. See the +If you want Debezium to create or update a publication for the tables configured in the RDI +source, set `publication.autocreate.mode` to `filtered`. See the [Debezium documentation for `publication.autocreate.mode`](https://debezium.io/documentation/reference/stable/connectors/postgresql.html#postgresql-publication-autocreate-mode) -to learn more about this setting. +to learn more about this setting. If a database administrator manages an existing +publication, use `disabled` instead, as described in [Choose a publication mode](#choose-a-publication-mode). ## Configure PostgreSQL for replication with the Debezium connector host From 1aef8074ecaf6370ccd376b192ee338e243b4027 Mon Sep 17 00:00:00 2001 From: Zdravko Donev Date: Thu, 15 Oct 2026 15:11:22 +0300 Subject: [PATCH 09/76] DOC: update versioned RDI PostgreSQL guidance --- .../aws-aurora-rds/aws-aur-pgsql.md | 12 +- .../data-pipelines/prepare-dbs/postgresql.md | 122 +++++++++++++++++- 2 files changed, 130 insertions(+), 4 deletions(-) diff --git a/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md b/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md index 3254e9fd2f..a7df75cc8a 100644 --- a/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md +++ b/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md @@ -147,8 +147,18 @@ to capture changes. Replace `` with the schema name and `` with the username of the Debezium user. 1. - Connect to your database as the `postgres` user and create a publication for the database: + Configure the PostgreSQL publication. For most RDI deployments, follow the + [PostgreSQL publication and replica identity guidance]({{< relref "/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql" >}}) + and use `publication.autocreate.mode: filtered` when RDI captures selected tables. + + If you intentionally want to publish every table in the database, first check that + every table has a primary key or another replica identity. Then connect to your database + as the `postgres` user and run: ```sql CREATE PUBLICATION dbz_publication FOR ALL TABLES; ``` + + Do not use this command without checking the replica identity of all tables. A later + `UPDATE` or `DELETE` on a published table without a replica identity can fail in the + application that writes to PostgreSQL. diff --git a/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/postgresql.md b/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/postgresql.md index 31ccc7149f..4ce48b8e1e 100644 --- a/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/postgresql.md +++ b/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/postgresql.md @@ -36,6 +36,7 @@ checklist as you complete each step. - [ ] [Install the logical decoding output plug-in](#install-the-logical-decoding-output-plug-in) - [ ] [Configure the PostgreSQL server](#configure-the-postgresql-server) - [ ] [Set up permissions](#set-up-permissions) +- [ ] [Check replica identity and choose a publication mode](#check-replica-identity-and-choose-a-publication-mode) - [ ] [Set privileges for Debezium to create PostgreSQL publications with pgoutput](#set-privileges-for-debezium-to-create-postgresql-publications-with-pgoutput) - [ ] [Configure PostgreSQL for replication with the Debezium connector host](#configure-postgresql-for-replication-with-the-debezium-connector-host) ``` @@ -180,6 +181,120 @@ needs to perform replications, but you might also need to grant other permission CREATE ROLE REPLICATION LOGIN; ``` +## Check replica identity and choose a publication mode + +Before you start RDI, check the tables that RDI will capture and choose how PostgreSQL +should publish their changes. + +PostgreSQL needs a [replica identity](https://www.postgresql.org/docs/current/logical-replication-publication.html) +to identify the old row when it publishes an `UPDATE` or `DELETE`. By default, PostgreSQL +uses the table's primary key. A table without a primary key has no usable replica identity +unless you configure one. Inserts do not need a replica identity. + +Run the following query to find ordinary tables that do not have a primary key and use the +default identity, or that explicitly have no identity: + +```sql +SELECT n.nspname AS schema_name, + c.relname AS table_name, + CASE c.relreplident + WHEN 'd' THEN 'default' + WHEN 'n' THEN 'nothing' + WHEN 'f' THEN 'full' + WHEN 'i' THEN 'index' + END AS replica_identity +FROM pg_class AS c +JOIN pg_namespace AS n ON n.oid = c.relnamespace +WHERE c.relkind = 'r' + AND n.nspname NOT IN ('pg_catalog', 'information_schema') + AND ( + c.relreplident = 'n' + OR ( + c.relreplident = 'd' + AND NOT EXISTS ( + SELECT 1 + FROM pg_index AS i + WHERE i.indrelid = c.oid + AND i.indisprimary + ) + ) + ) +ORDER BY n.nspname, c.relname; +``` + +If this query returns a table that will be published, add a primary key where possible. +You can also configure a suitable unique index as the replica identity: + +```sql +ALTER TABLE REPLICA IDENTITY USING INDEX ; +``` + +As a fallback, you can record the full old row in the write-ahead log: + +```sql +ALTER TABLE REPLICA IDENTITY FULL; +``` + +`REPLICA IDENTITY FULL` can increase write-ahead log volume and reduce performance, so use +it only when a primary key or suitable unique index is not possible. The `keys` setting in +an RDI table definition creates a key for the Redis record. It does not create a PostgreSQL +primary key or replica identity. + +### Choose a publication mode + +The `publication.autocreate.mode` setting controls how RDI and Debezium manage the +PostgreSQL publication: + +| Mode | Behavior | Recommended use | +| --- | --- | --- | +| `filtered` | Creates or updates the publication to contain the tables configured for the RDI source. | Recommended when RDI captures selected tables. RDI manages the publication when the pipeline is redeployed with a changed table filter. | +| `disabled` | Does not create or change the publication. The publication must already exist. | Use when a database administrator manages the publication. | +| `all_tables` | Creates a `FOR ALL TABLES` publication when the publication does not exist. If it already exists, RDI uses the existing publication. | Use when all tables in the database should be published, after checking their replica identities. This is the RDI default. | + +The `all_tables` mode can publish changes for tables that are not listed in the RDI +pipeline. If an application later updates or deletes a row in a published table without +a replica identity, PostgreSQL can reject that application write. For this reason, +`filtered` is the recommended mode for a pipeline that captures only selected tables. + +For a customer-managed publication, create an explicit table list and disable automatic +publication management. For example: + +```sql +CREATE PUBLICATION rdi_publication +FOR TABLE public.customers, public.orders; +``` + +Then set the following source properties: + +```yaml +publication.name: rdi_publication +publication.autocreate.mode: disabled +``` + +### PostgreSQL publication error FAQ + +#### I get this PostgreSQL error. What should I do? + +You may see the following error when an application updates a published table without a +replica identity: + +```text +SQLSTATE 55000 +ERROR: cannot update table "example_table" because it does not have a replica identity and publishes updates +HINT: To enable updating the table, set REPLICA IDENTITY using ALTER TABLE. +``` + +Check whether the table should be captured by RDI. If it should, add a primary key or +configure a suitable replica identity. As a fallback, run: + +```sql +ALTER TABLE REPLICA IDENTITY FULL; +``` + +If the table should not be captured, use `publication.autocreate.mode: filtered` or remove +the table from the customer-managed publication. If you changed the RDI table filters or +publication mode, redeploy the pipeline so RDI applies the new configuration. + ## Set privileges for Debezium to create PostgreSQL publications with `pgoutput` The Debezium user needs specific permissions to work with the `pgoutput` plug-in. @@ -238,10 +353,11 @@ the replication group using the following commands: ALTER TABLE table_name OWNER TO replication_group; ``` -You must also set the value of the `publication.autocreate.mode` parameter to `filtered` -to allow Debezium to specify the publication configuration. See the +If you want Debezium to create or update a publication for the tables configured in the RDI +source, set `publication.autocreate.mode` to `filtered`. See the [Debezium documentation for `publication.autocreate.mode`](https://debezium.io/documentation/reference/stable/connectors/postgresql.html#postgresql-publication-autocreate-mode) -to learn more about this setting. +to learn more about this setting. If a database administrator manages an existing +publication, use `disabled` instead, as described in [Choose a publication mode](#choose-a-publication-mode). ## Configure PostgreSQL for replication with the Debezium connector host From 75545b24c904f4c5ef5d3351b75cdcfc01e3c811 Mon Sep 17 00:00:00 2001 From: Zdravko Donev Date: Thu, 15 Oct 2026 15:21:15 +0300 Subject: [PATCH 10/76] DOC: address PostgreSQL publication review comments --- .../data-pipelines/prepare-dbs/postgresql.md | 20 +++++++++++++------ .../data-pipelines/prepare-dbs/postgresql.md | 20 +++++++++++++------ 2 files changed, 28 insertions(+), 12 deletions(-) diff --git a/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/postgresql.md b/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/postgresql.md index 4ce48b8e1e..8e75a9e2f1 100644 --- a/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/postgresql.md +++ b/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/postgresql.md @@ -191,8 +191,10 @@ to identify the old row when it publishes an `UPDATE` or `DELETE`. By default, P uses the table's primary key. A table without a primary key has no usable replica identity unless you configure one. Inserts do not need a replica identity. -Run the following query to find ordinary tables that do not have a primary key and use the -default identity, or that explicitly have no identity: +Run the following query while connected to the database that RDI uses. You do not put the +database name in the query; select that database in your SQL editor or connection settings. +The query checks all non-system schemas. To check only one schema, uncomment the marked line +and replace `` with your schema name. ```sql SELECT n.nspname AS schema_name, @@ -207,6 +209,8 @@ FROM pg_class AS c JOIN pg_namespace AS n ON n.oid = c.relnamespace WHERE c.relkind = 'r' AND n.nspname NOT IN ('pg_catalog', 'information_schema') + -- Optional: uncomment and replace to check one schema only. + -- AND n.nspname = '' AND ( c.relreplident = 'n' OR ( @@ -247,9 +251,9 @@ PostgreSQL publication: | Mode | Behavior | Recommended use | | --- | --- | --- | +| **`all_tables` (default)** | Creates a `FOR ALL TABLES` publication when the publication does not exist. If it already exists, RDI uses the existing publication. | Use when all tables in the database should be published, after checking their replica identities. | | `filtered` | Creates or updates the publication to contain the tables configured for the RDI source. | Recommended when RDI captures selected tables. RDI manages the publication when the pipeline is redeployed with a changed table filter. | | `disabled` | Does not create or change the publication. The publication must already exist. | Use when a database administrator manages the publication. | -| `all_tables` | Creates a `FOR ALL TABLES` publication when the publication does not exist. If it already exists, RDI uses the existing publication. | Use when all tables in the database should be published, after checking their replica identities. This is the RDI default. | The `all_tables` mode can publish changes for tables that are not listed in the RDI pipeline. If an application later updates or deletes a row in a published table without @@ -264,11 +268,15 @@ CREATE PUBLICATION rdi_publication FOR TABLE public.customers, public.orders; ``` -Then set the following source properties: +Then set the following properties under the PostgreSQL source in your RDI `config.yaml`: ```yaml -publication.name: rdi_publication -publication.autocreate.mode: disabled +sources: + : + advanced: + source: + publication.name: rdi_publication + publication.autocreate.mode: disabled ``` ### PostgreSQL publication error FAQ diff --git a/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md b/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md index 2e262aed33..8da8fa6966 100644 --- a/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md +++ b/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md @@ -194,8 +194,10 @@ to identify the old row when it publishes an `UPDATE` or `DELETE`. By default, P uses the table's primary key. A table without a primary key has no usable replica identity unless you configure one. Inserts do not need a replica identity. -Run the following query to find ordinary tables that do not have a primary key and use the -default identity, or that explicitly have no identity: +Run the following query while connected to the database that RDI uses. You do not put the +database name in the query; select that database in your SQL editor or connection settings. +The query checks all non-system schemas. To check only one schema, uncomment the marked line +and replace `` with your schema name. ```sql SELECT n.nspname AS schema_name, @@ -210,6 +212,8 @@ FROM pg_class AS c JOIN pg_namespace AS n ON n.oid = c.relnamespace WHERE c.relkind = 'r' AND n.nspname NOT IN ('pg_catalog', 'information_schema') + -- Optional: uncomment and replace to check one schema only. + -- AND n.nspname = '' AND ( c.relreplident = 'n' OR ( @@ -250,9 +254,9 @@ PostgreSQL publication: | Mode | Behavior | Recommended use | | --- | --- | --- | +| **`all_tables` (default)** | Creates a `FOR ALL TABLES` publication when the publication does not exist. If it already exists, RDI uses the existing publication. | Use when all tables in the database should be published, after checking their replica identities. | | `filtered` | Creates or updates the publication to contain the tables configured for the RDI source. | Recommended when RDI captures selected tables. RDI manages the publication when the pipeline is redeployed with a changed table filter. | | `disabled` | Does not create or change the publication. The publication must already exist. | Use when a database administrator manages the publication. | -| `all_tables` | Creates a `FOR ALL TABLES` publication when the publication does not exist. If it already exists, RDI uses the existing publication. | Use when all tables in the database should be published, after checking their replica identities. This is the RDI default. | The `all_tables` mode can publish changes for tables that are not listed in the RDI pipeline. If an application later updates or deletes a row in a published table without @@ -267,11 +271,15 @@ CREATE PUBLICATION rdi_publication FOR TABLE public.customers, public.orders; ``` -Then set the following source properties: +Then set the following properties under the PostgreSQL source in your RDI `config.yaml`: ```yaml -publication.name: rdi_publication -publication.autocreate.mode: disabled +sources: + : + advanced: + source: + publication.name: rdi_publication + publication.autocreate.mode: disabled ``` ### PostgreSQL publication error FAQ From 7ec2c7d2ec6452b9fbc07f4efad556806658194d Mon Sep 17 00:00:00 2001 From: mich-elle-luna Date: Thu, 15 Oct 2026 14:32:20 -0700 Subject: [PATCH 11/76] Remove LangCache links and named mentions from Iris landing and concept pages The LangCache pages stay where they are; this only stops the Iris landing and concept pages from pointing at them. - Iris develop and operate landing pages: remove the LangCache tile, and on the operate page the Create and Self-managed LangCache links. Generic "semantic caching" wording stays. - Concepts page: drop LangCache concepts and the LangCache demo link from Next steps. - Request flow: name semantic caching instead of LangCache in the intro, relabel the diagram node "Cache hit?" with "Cache" in its descriptions, and drop the LangCache Next steps links. The node keeps its internal langcache ID, which the edges refer to, and its links to the LangCache demo and docs. - Preview banners on all five pages now name only Agent Memory and Context Retriever. The operate landing screenshot's alt text still lists LangCache, because the screenshot shows its card. The interactive demo page's description, its service table, and the demo script still include LangCache. Co-Authored-By: Claude Opus 5.5 (1M context) --- content/develop/ai/context-engine/_index.md | 3 +-- .../ai/context-engine/concepts/_index.md | 5 ++--- .../context-engine/concepts/interactive-demo.md | 2 +- .../ai/context-engine/concepts/request-flow.md | 17 ++++++++--------- content/operate/iris/_index.md | 5 +---- 5 files changed, 13 insertions(+), 19 deletions(-) diff --git a/content/develop/ai/context-engine/_index.md b/content/develop/ai/context-engine/_index.md index 0364923a73..893c7f51d0 100644 --- a/content/develop/ai/context-engine/_index.md +++ b/content/develop/ai/context-engine/_index.md @@ -9,7 +9,7 @@ hideListLinks: true linktitle: Redis Iris context engine title: Redis Iris context engine weight: 30 -bannerText: LangCache, Agent Memory, and Context Retriever are currently available in preview. Features and behavior are subject to change. +bannerText: Agent Memory and Context Retriever are currently available in preview. Features and behavior are subject to change. --- Give your AI agents the context layer they need to reliably act on business data. @@ -19,7 +19,6 @@ Redis Iris eliminates the infrastructure burden of building context-aware AI age
{{< tile-card icon="images/dev/icons/icon-redis-iris-64-duotone.png" title="Concepts" description="What happens when an agent asks Redis Iris for context" url="/develop/ai/context-engine/concepts" >}} {{< tile-card icon="images/dev/icons/icon-redis-agent-memory-64-duotone.png" title="Agent Memory" description="Persistent short-term and long-term memory across agent interactions" url="/develop/ai/context-engine/agent-memory" >}} - {{< tile-card icon="images/dev/icons/icon-redis-langcache-64-duotone.png" title="LangCache" description="Semantic caching to reduce LLM costs and improve response times" url="/develop/ai/context-engine/langcache" >}} {{< tile-card icon="images/dev/icons/icon-redis-context-retriever-64-duotone.png" title="Context Retriever" description="Governed, schema-first data access tools for agents" url="/develop/ai/context-engine/context-retriever" >}}
diff --git a/content/develop/ai/context-engine/concepts/_index.md b/content/develop/ai/context-engine/concepts/_index.md index 4bbacabd6b..585ac04de2 100644 --- a/content/develop/ai/context-engine/concepts/_index.md +++ b/content/develop/ai/context-engine/concepts/_index.md @@ -9,7 +9,7 @@ hideListLinks: true linktitle: Concepts title: How Redis Iris works weight: 5 -bannerText: LangCache, Agent Memory, and Context Retriever are currently available in preview. Features and behavior are subject to change. +bannerText: Agent Memory and Context Retriever are currently available in preview. Features and behavior are subject to change. --- Redis Iris sits between your agent and your data. It's one system. Your agent asks for context. Iris returns it through the fastest available path: a cached answer, a remembered fact, or a live lookup against your business data. That data can live in Redis or in another database you connect. You don't wire together separate lookups yourself. Iris decides which path answers the request. @@ -85,7 +85,6 @@ Run all three fully managed on Redis Cloud, or self-managed on your own infrastr ## Next steps - [How a request flows through Redis Iris]({{< relref "/develop/ai/context-engine/concepts/request-flow" >}}) -- [LangCache concepts]({{< relref "/develop/ai/context-engine/langcache/concepts" >}}) - [Agent Memory overview]({{< relref "/develop/ai/context-engine/agent-memory/overview" >}}) - [Context Retriever concepts]({{< relref "/develop/ai/context-engine/context-retriever/concepts" >}}) -- Interactive demos for [LangCache]({{< relref "/develop/ai/context-engine/langcache/interactive-demo" >}}), [Agent Memory]({{< relref "/develop/ai/context-engine/agent-memory/interactive-demo" >}}), and [Context Retriever]({{< relref "/develop/ai/context-engine/context-retriever/interactive-demo" >}}) that run in your browser +- Interactive demos for [Agent Memory]({{< relref "/develop/ai/context-engine/agent-memory/interactive-demo" >}}) and [Context Retriever]({{< relref "/develop/ai/context-engine/context-retriever/interactive-demo" >}}) that run in your browser diff --git a/content/develop/ai/context-engine/concepts/interactive-demo.md b/content/develop/ai/context-engine/concepts/interactive-demo.md index 3cd8f6b9eb..f96414e86c 100644 --- a/content/develop/ai/context-engine/concepts/interactive-demo.md +++ b/content/develop/ai/context-engine/concepts/interactive-demo.md @@ -9,7 +9,7 @@ hideListLinks: true linktitle: Interactive demo title: Redis Iris interactive demo weight: 10 -bannerText: LangCache, Agent Memory, and Context Retriever are currently available in preview. Features and behavior are subject to change. +bannerText: Agent Memory and Context Retriever are currently available in preview. Features and behavior are subject to change. aliases: - /develop/ai/context-engine/interactive-demo/ --- diff --git a/content/develop/ai/context-engine/concepts/request-flow.md b/content/develop/ai/context-engine/concepts/request-flow.md index 1dc3ac6b6b..00d1d43229 100644 --- a/content/develop/ai/context-engine/concepts/request-flow.md +++ b/content/develop/ai/context-engine/concepts/request-flow.md @@ -9,14 +9,14 @@ hideListLinks: true linktitle: Request flow title: How a request flows through Redis Iris weight: 6 -bannerText: LangCache, Agent Memory, and Context Retriever are currently available in preview. Features and behavior are subject to change. +bannerText: Agent Memory and Context Retriever are currently available in preview. Features and behavior are subject to change. --- [How Redis Iris works]({{< relref "/develop/ai/context-engine/concepts" >}}) covers the mental model. This page traces the same idea through an actual request, capability by capability. -Every capability in Iris manages the same limited resource: the model's context window. LangCache skips a model call when a similar one already ran. Agent Memory decides what's worth keeping from a conversation. Context Retriever returns exactly the data a tool call needs, instead of a raw query result. Treat "what goes into the next model call" as a budget. Manage it actively. It doesn't take care of itself once a request reaches Iris. +Every capability in Iris manages the same limited resource: the model's context window. Semantic caching skips a model call when a similar one already ran. Agent Memory decides what's worth keeping from a conversation. Context Retriever returns exactly the data a tool call needs, instead of a raw query result. Treat "what goes into the next model call" as a budget. Manage it actively. It doesn't take care of itself once a request reaches Iris. -These capabilities aren't a fixed sequence. A request can draw on any combination of them before a model call happens: [LangCache]({{< relref "/develop/ai/context-engine/langcache" >}}) for cached responses, [Agent Memory]({{< relref "/develop/ai/context-engine/agent-memory" >}}) for session and long-term recall, and [Context Retriever]({{< relref "/develop/ai/context-engine/context-retriever" >}}) for governed access to business data. [Data Integration]({{< relref "/develop/ai/context-engine/data-integration" >}}) runs in the background, keeping that business data fresh for Context Retriever to query. +These capabilities aren't a fixed sequence. A request can draw on any combination of them before a model call happens: semantic caching for responses to similar prompts, [Agent Memory]({{< relref "/develop/ai/context-engine/agent-memory" >}}) for session and long-term recall, and [Context Retriever]({{< relref "/develop/ai/context-engine/context-retriever" >}}) for governed access to business data. [Data Integration]({{< relref "/develop/ai/context-engine/data-integration" >}}) runs in the background, keeping that business data fresh for Context Retriever to query. Select a node in the diagram for a description of what that capability does, with a link to its docs. Select a scenario button to trace the path a request takes. @@ -33,12 +33,12 @@ nodes: The calling application or AI agent that sends a prompt. docsUrl: "/develop/ai/agent-builder" langcache: - label: "LangCache hit?" + label: "Cache hit?" type: decision col: 1 row: 0 description: | - LangCache: checks whether a similar prompt is already cached before calling the model. + Cache: checks whether a similar prompt is already cached before calling the model. links: demo: label: "Interactive demo" @@ -82,7 +82,7 @@ nodes: col: 2 row: 0 description: | - Return cached response: on a cache hit, LangCache returns the stored response directly, skipping the model call. + Return cached response: on a cache hit, Cache returns the stored response directly, skipping the model call. modelCall: label: "Model call" type: process @@ -163,7 +163,7 @@ paths: cacheHit: label: "Cache hit: fastest" description: | - LangCache finds a similar prompt already cached and returns it directly. No model call, so this path is the fastest and adds no LLM cost. + Cache finds a similar prompt already cached and returns it directly. No model call, so this path is the fastest and adds no LLM cost. memory: label: "Needs memory: recalls session or long-term info" description: | @@ -182,7 +182,6 @@ paths: ## Next steps -- [LangCache concepts]({{< relref "/develop/ai/context-engine/langcache/concepts" >}}) - [Agent Memory overview]({{< relref "/develop/ai/context-engine/agent-memory/overview" >}}) - [Context Retriever concepts]({{< relref "/develop/ai/context-engine/context-retriever/concepts" >}}) -- Interactive demos for [LangCache]({{< relref "/develop/ai/context-engine/langcache/interactive-demo" >}}), [Agent Memory]({{< relref "/develop/ai/context-engine/agent-memory/interactive-demo" >}}), and [Context Retriever]({{< relref "/develop/ai/context-engine/context-retriever/interactive-demo" >}}) that run in your browser +- Interactive demos for [Agent Memory]({{< relref "/develop/ai/context-engine/agent-memory/interactive-demo" >}}) and [Context Retriever]({{< relref "/develop/ai/context-engine/context-retriever/interactive-demo" >}}) that run in your browser diff --git a/content/operate/iris/_index.md b/content/operate/iris/_index.md index d8ee19646c..60ca4f2f85 100644 --- a/content/operate/iris/_index.md +++ b/content/operate/iris/_index.md @@ -9,7 +9,7 @@ categories: - iris hideListLinks: true weight: 45 -bannerText: LangCache, Agent Memory, and Context Retriever are currently available in preview. Features and behavior are subject to change. +bannerText: Agent Memory and Context Retriever are currently available in preview. Features and behavior are subject to change. --- Redis Iris context engine provides managed and self-managed services for building AI applications with persistent memory, semantic caching, and governed access to business data. @@ -22,7 +22,6 @@ Use this section to deploy, configure, and operate Redis Iris services. Develope
{{< tile-card icon="images/dev/icons/icon-redis-agent-memory-64-duotone.png" title="Redis Agent Memory" description="Create and manage a service on Redis Cloud or deploy it on your own infrastructure" url="/operate/iris/agent-memory" >}} {{< tile-card icon="images/dev/icons/icon-redis-context-retriever-64-duotone.png" title="Redis Context Retriever" description="Create and manage governed retrieval tools for AI agents" url="/operate/iris/context-retriever" >}} - {{< tile-card icon="images/dev/icons/icon-redis-langcache-64-duotone.png" title="LangCache" description="Create, configure, and monitor semantic caches for AI applications" url="/operate/iris/langcache" >}}
## Deployment options @@ -35,7 +34,6 @@ Create and manage Redis Iris services through the Redis Cloud console without de - [Create a Redis Agent Memory service]({{< relref "/operate/iris/agent-memory/create-service" >}}) - [Create a Redis Context Retriever service]({{< relref "/operate/iris/context-retriever/create-service" >}}) -- [Create a LangCache service]({{< relref "/operate/iris/langcache/create-service" >}}) ### Self-managed @@ -43,4 +41,3 @@ Deploy Redis Iris services on Kubernetes when you need to operate them on your o - [Self-managed Redis Agent Memory]({{< relref "/operate/iris/agent-memory/self-managed" >}}) - [Self-managed Redis Context Retriever]({{< relref "/operate/iris/context-retriever/self-managed" >}}) -- [Self-managed LangCache]({{< relref "/operate/iris/langcache/self-managed" >}}) From 1836664b2581a64c3bc9f222e54931ebf133c072 Mon Sep 17 00:00:00 2001 From: David Dougherty Date: Thu, 15 Oct 2026 15:16:04 -0700 Subject: [PATCH 12/76] DEV: misc. doc changes for Redis Search in 8.10 (#4279) --- content/commands/ft.hybrid.md | 63 +++++++++++++++++-- .../administration/configuration.md | 2 +- .../advanced-concepts/aggregations.md | 2 +- 3 files changed, 60 insertions(+), 7 deletions(-) diff --git a/content/commands/ft.hybrid.md b/content/commands/ft.hybrid.md index 1ebccc333d..8e4ac04ad1 100644 --- a/content/commands/ft.hybrid.md +++ b/content/commands/ft.hybrid.md @@ -167,6 +167,10 @@ arguments: name: sorting optional: true type: oneof +- name: explainscore + optional: true + token: EXPLAINSCORE + type: pure-token - arguments: - name: params token: PARAMS @@ -514,7 +518,7 @@ syntax_fmt: "FT.HYBRID index\n SEARCH query\n [SCORER scorer]\n [YIELD_SC \ nargs property [property ...]\n [GROUPBY nargs property [property ...]\n \ \ [REDUCE function nargs arg [arg ...] [AS name]\n [REDUCE function nargs arg\ \ [arg ...] [AS name] ...]] ...]]\n [APPLY expression AS name [APPLY expression\ - \ AS name ...]]\n [FILTER filter]\n PARAMS nargs vector_param vector_blob [name\ + \ AS name ...]]\n [FILTER filter]\n [EXPLAINSCORE]\n PARAMS nargs vector_param vector_blob [name\ \ value ...]\n [TIMEOUT timeout]" title: FT.HYBRID --- @@ -657,16 +661,26 @@ defines parameter substitution for the query. Parameters can be referenced in se Example: `PARAMS 4 min_price 50 max_price 200` - +returns an explanation of how each result's combined score was calculated. Available in Redis 8.10 and later. + +Each result gets a `score` field holding a two-element array: the combined score, followed by a nested explanation tree. You don't need `WITHSCORES`. The tree shows: + +- The final score formula and the fusion method with its parameters: `RRF` with `window` and `constant`, or `LINEAR` with `alpha`, `beta`, and `window`. +- The text branch: the scorer name, such as `BM25STD`, and the scorer's own breakdown. With `RRF`, it shows the text rank. With `LINEAR`, it shows the text contribution and the normalized text score. +- The vector branch: `KNN` or `RANGE` with its radius and epsilon. With `RRF`, it shows the vector rank. With `LINEAR`, it shows the vector contribution and the normalized vector score. `RANGE` also shows whether the document matched within the radius. + +A branch that didn't match the document shows ``. + +`EXPLAINSCORE` can't be combined with `GROUPBY`. The command returns the error `EXPLAINSCORE is not supported with GROUPBY`. +
TIMEOUT timeout -sets a runtime timeout for the query execution in milliseconds. +sets a runtime timeout for the query execution in milliseconds. When the query times out, the [`search-on-timeout`](/content/develop/ai/search-and-query/administration/configuration.md#search-on-timeout) policy applies. With `RETURN` or `RETURN_STRICT`, the command returns the results accumulated so far and reports the timeout in `warnings`. With `FAIL`, it returns an error.
- 6.7.1.RELEASE + 7.7.0.RELEASE @@ -81,7 +81,7 @@ If you are using [Gradle](https://gradle.org/), add the following dependencies to your `build.gradle` file: ```bash -compileOnly 'io.lettuce:lettuce-core:6.7.1.RELEASE' +compileOnly 'io.lettuce:lettuce-core:7.7.0.RELEASE' compileOnly 'ai.djl.huggingface:tokenizers:0.33.0' compileOnly 'ai.djl.pytorch:pytorch-model-zoo:0.33.0' compileOnly 'ai.djl:api:0.33.0' @@ -98,10 +98,10 @@ Import the following classes in your source file: When you store vectors in a hash object, or pass them as query parameters, you must encode the `float` components of the vector -array as a `byte` string. The helper method `floatArrayToByteBuffer()` +array as a `byte` string. The helper method `floatArrayToByteArray()` shown below does this for you: -{{< clients-example set="home_query_vec" step="helper_method" lang_filter="Java-Async,Java-Reactive" description="Foundational: Convert float arrays to ByteBuffer format for storing vectors in hash objects" difficulty="beginner" >}} +{{< clients-example set="home_query_vec" step="helper_method" lang_filter="Java-Async,Java-Reactive" description="Foundational: Convert float arrays to byte arrays for storing vectors in hash objects" difficulty="beginner" >}} {{< /clients-example >}} ## Create an embedding model instance @@ -126,20 +126,23 @@ creates an instance of `Predictor` that uses the `all-MiniLM-L6-v2` model: As noted in [Define a helper method](#define-a-helper-method) above, you must pass the embeddings to the hash and query commands as a binary string. -Lettuce has an option to specify a `ByteBufferCodec` for the connection to Redis. -This lets you construct binary strings for Redis keys and values conveniently using -the standard -[`ByteBuffer`](https://docs.oracle.com/javase/8/docs/api/java/nio/ByteBuffer.html) -class (see [Codecs](https://redis.github.io/lettuce/integration-extension/#codecs) -in the Lettuce documentation for more information). However, you will probably find -it more convenient to use the default `StringCodec` for commands that don't require binary strings. It is therefore helpful to have two connections available, one using `ByteBufferCodec` and one using `StringCodec`. +Lettuce lets you choose a codec for each connection to Redis (see +[Codecs](https://redis.github.io/lettuce/integration-extension/#codecs) +in the Lettuce documentation for more information). +`RedisCodec.of(StringCodec.UTF8, ByteArrayCodec.INSTANCE)` combines two codecs +so that keys, field names, and index names are `String` objects, while values are +`byte[]` arrays. The embeddings are binary strings, and you pass text values as +UTF-8 encoded `byte[]` arrays. However, you will probably find it more convenient +to use the default `StringCodec` for commands that don't require binary strings. +It is therefore helpful to have two connections available, one using the +composed codec and one using `StringCodec`. The code below shows how to declare one connection with the -`ByteBufferCodec` and another without in the try-with-resources +composed codec and another without in the try-with-resources block. You also need two separate instances of `RedisAsyncCommands` to use the two connections: -{{< clients-example set="home_query_vec" step="connect" lang_filter="Java-Async,Java-Reactive" description="Practical pattern: Create dual connections with ByteBufferCodec and StringCodec for vector operations" difficulty="intermediate" >}} +{{< clients-example set="home_query_vec" step="connect" lang_filter="Java-Async,Java-Reactive" description="Practical pattern: Create dual connections with a composed String/byte[] codec and StringCodec for vector operations" difficulty="intermediate" >}} {{< /clients-example >}} Next, create the index. @@ -171,19 +174,19 @@ you use the `doc:` prefix specified in the index definition. Use the `predict()` method of the `Predictor` object as shown below to create the embedding that represents the `content` field -and use the `floatArrayToByteBuffer()` helper method to convert it to a binary string. +and use the `floatArrayToByteArray()` helper method to convert it to a binary string. Use the binary string representation when you are indexing hash objects, but use an array of `float` for JSON objects (see [Differences with JSON objects](#differences-with-json-documents) below). -You must use instances of `Map` to supply the data to `hset()` -when using the `ByteBufferCodec` connection, which adds a little complexity. Note +You must use instances of `Map` to supply the data to `hset()` +when using the binary connection, so you encode the text values with `getBytes()`. Note that the `predict()` call is in a `try`/`catch` block because it will throw exceptions if it can't download the embedding model (you should add code to handle the exceptions in production code). -{{< clients-example set="home_query_vec" step="add_data" lang_filter="Java-Async,Java-Reactive" description="Foundational: Generate embeddings and store hash documents with vector data using ByteBufferCodec" difficulty="intermediate" >}} +{{< clients-example set="home_query_vec" step="add_data" lang_filter="Java-Async,Java-Reactive" description="Foundational: Generate embeddings and store hash documents with vector data using a String/byte[] codec" difficulty="intermediate" >}} {{< /clients-example >}} ### Bulk import with HIMPORT @@ -199,9 +202,8 @@ Create a `HashImport` object with the field names, then create each document with [`himportSet()`](/content/commands/himport-set.md), passing only its values, in the same order as the field names. Lettuce declares the field names with [`HIMPORT PREPARE`](/content/commands/himport-prepare.md) the first time you use -the `HashImport` object on a connection. On the `ByteBufferCodec` connection, -`HashImport.of()` takes `ByteBuffer` field names and a function that creates a -name for the field set from a numeric ID. Issue all the imports before you wait +the `HashImport` object on a connection. Because the binary connection uses +`String` keys, `HashImport.of()` takes the field names as strings. Issue all the imports before you wait for any reply, so that Lettuce [pipelines](/content/develop/clients/lettuce/transpipe.md) them instead of waiting for each reply in turn. With the reactive API, wait for the imports @@ -245,7 +247,6 @@ you run it for the first time (which happens because the model must download the generate the embeddings). When you run the code, it outputs the following result text: ``` -Results: ID: doc:1, Content: That is a very happy person, Distance: 0.114169836044 ID: doc:2, Content: That is a happy dog, Distance: 0.610845506191 ID: doc:3, Content: Today is a sunny day, Distance: 1.48624765873 @@ -275,7 +276,7 @@ the one created previously for hashes: An important difference with JSON indexing is that the vectors are specified using arrays of `float` instead of binary strings. This means -you don't need to use the `ByteBufferCodec` connection, and you can use +you don't need to use the binary connection, and you can use [`Arrays.toString()`](https://docs.oracle.com/javase/8/docs/api/java/util/Arrays.html#toString-float:A-) to convert the `float` array to a suitable JSON string. Use [`jsonSet()`](/content/commands/json.set.md) to add the data @@ -296,15 +297,13 @@ was specified as an array. {{< clients-example set="home_query_vec" step="json_query" lang_filter="Java-Async,Java-Reactive" description="Semantic search: Execute a KNN vector search on JSON documents with semantic similarity ranking" difficulty="advanced" >}} {{< /clients-example >}} -The distance values are not identical to the hash query because the -string representations of the vectors used here are stored with different -precisions. However, the relative order of the results is the same: +The embeddings are generated from the same text as the hash documents, so the +query returns the same distances: ``` -Results: -ID: jdoc:1, Content: That is a very happy person, Distance: 0.628328084946 -ID: jdoc:2, Content: That is a happy dog, Distance: 0.895147025585 -ID: jdoc:3, Content: Today is a sunny day, Distance: 1.49569523335 +ID: jdoc:1, Content: That is a very happy person, Distance: 0.114169836044 +ID: jdoc:2, Content: That is a happy dog, Distance: 0.610845506191 +ID: jdoc:3, Content: Today is a sunny day, Distance: 1.48624765873 ``` ## Learn more diff --git a/content/develop/clients/php/vecsearch.md b/content/develop/clients/php/vecsearch.md index 6d0ab0e333..3689edc357 100644 --- a/content/develop/clients/php/vecsearch.md +++ b/content/develop/clients/php/vecsearch.md @@ -91,10 +91,24 @@ The code below shows how to use the tokenizer to generate the embeddings. The vectors that represent the embeddings have 384 dimensions, regardless of the length of the input text. Here, the `pipeline()` call creates the `$extractor` function that -generates embeddings from text: +generates embeddings from text. + +The `all-MiniLM-L6-v2` model expects mean pooling followed by normalization +to unit length (see the +[`all-MiniLM-L6-v2`](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) +page for more information). In TransformersPHP v0.6.2, the `normalize: true` +option has no effect. The `$embed()` function below calls `$extractor()` with +mean pooling and then normalizes the vector itself: ```php $extractor = pipeline('embeddings', 'Xenova/all-MiniLM-L6-v2'); + +$embed = function (string $text) use ($extractor): array { + $emb = $extractor($text, pooling: 'mean')[0]; + $norm = sqrt(array_sum(array_map(fn($x) => $x * $x, $emb))); + + return array_map(fn($x) => $x / $norm, $emb); +}; ``` ## Create the index @@ -159,14 +173,8 @@ You can now supply the data objects, which will be indexed automatically when you add them with [`hmset()`](/content/commands/hset.md), as long as you use the `doc:` prefix specified in the index definition. -Use the `$extractor()` function as shown below to create the embedding that -represents the `content` field. Note that `$extractor()` can generate multiple -embeddings from multiple strings parameters at once, so it returns an array of -embedding vectors. Here, there is only one embedding in the returned array. -The `normalize:` and `pooling:` named parameters relate to details -of the embedding model (see the -[`all-MiniLM-L6-v2`](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) -page for more information). +Use the `$embed()` function as shown below to create the embedding that +represents the `content` field. To add an embedding as a field of a hash object, you must encode the vector array as a binary string. The built-in @@ -181,30 +189,30 @@ below). ```php $content = "That is a very happy person"; -$emb = $extractor($content, normalize: true, pooling: 'mean'); +$emb = $embed($content); $client->hmset("doc:0",[ "content" => $content, "genre" => "persons", - "embedding" => pack('g*', ...$emb[0]) + "embedding" => pack('g*', ...$emb) ]); $content = "That is a happy dog"; -$emb = $extractor($content, normalize: true, pooling: 'mean'); +$emb = $embed($content); $client->hmset("doc:1",[ "content" => $content, "genre" => "pets", - "embedding" => pack('g*', ...$emb[0]) + "embedding" => pack('g*', ...$emb) ]); $content = "Today is a sunny day"; -$emb = $extractor($content, normalize: true, pooling: 'mean'); +$emb = $embed($content); $client->hmset("doc:2",[ "content" => $content, "genre" => "weather", - "embedding" => pack('g*', ...$emb[0]) + "embedding" => pack('g*', ...$emb) ]); ``` @@ -231,16 +239,16 @@ $docs = [ ["doc:2", "Today is a sunny day", "weather"], ]; -$client->pipeline(function ($pipe) use ($docs, $extractor) { +$client->pipeline(function ($pipe) use ($docs, $embed) { $pipe->himport('PREPARE', "doc", ["content", "genre", "embedding"]); foreach ($docs as [$key, $content, $genre]) { - $emb = $extractor($content, normalize: true, pooling: 'mean'); + $emb = $embed($content); $pipe->himport('SET', $key, "doc", [ $content, $genre, - pack('g*', ...$emb[0]) + pack('g*', ...$emb) ]); } }); @@ -262,7 +270,7 @@ text. Redis calculates the vector distance between the query vector and each embedding vector in the index as it runs the query. You can request the results to be sorted to rank them in order of ascending distance. -The code below creates the query embedding using the `$extractor()` function, as with +The code below creates the query embedding using the `$embed()` function, as with the indexing, and passes it as a parameter when the query executes (see [Vector search](/content/develop/ai/search-and-query/query/vector-search.md) for more information about using query parameters with embeddings). @@ -277,16 +285,16 @@ the fields you requested (again as alternating key-value pairs). ```php $queryText = "That is a happy person"; -$queryEmb = $extractor($queryText, normalize: true, pooling: 'mean'); +$queryEmb = $embed($queryText); $result = $client->ftsearch( "vector_idx", '*=>[KNN 3 @embedding $vec AS vector_distance]', - new SearchArguments() + (new SearchArguments()) ->addReturn(1, "vector_distance") ->dialect("2") ->params([ - "vec", pack('g*', ...$queryEmb[0]) + "vec", pack('g*', ...$queryEmb) ]) ->sortBy("vector_distance") ); @@ -302,11 +310,11 @@ for ($i = 1; $i < ($numResults * 2 + 1); $i += 2) { echo "Field: {$fields[0]}, Value: {$fields[1]}" . PHP_EOL; } // >>> Key: doc:0 -// >>> Field: vector_distance, Value: 3.76152896881 +// >>> Field: vector_distance, Value: 0.133717402816 // >>> Key: doc:1 -// >>> Field: vector_distance, Value: 18.6544265747 +// >>> Field: vector_distance, Value: 0.621603608131 // >>> Key: doc:2 -// >>> Field: vector_distance, Value: 44.6189727783 +// >>> Field: vector_distance, Value: 1.46157670021 ``` Assuming you have added the code from the steps above to your source file, @@ -318,14 +326,16 @@ generate the embeddings). When you run the code, it outputs the following result ``` Number of results: 3 Key: doc:0 -Field: vector_distance, Value: 3.76152896881 +Field: vector_distance, Value: 0.133717402816 Key: doc:1 -Field: vector_distance, Value: 18.6544265747 +Field: vector_distance, Value: 0.621603608131 Key: doc:2 -Field: vector_distance, Value: 44.6189727783 +Field: vector_distance, Value: 1.46157670021 ``` -Note that the results are ordered according to the value of the `distance` +The exact distances may differ slightly on your system. + +Note that the results are ordered according to the value of the `vector_distance` field, with the lowest distance indicating the greatest similarity to the query. As you would expect, the text *"That is a very happy person"* (from the `doc:0` document) @@ -375,48 +385,48 @@ that specify the fields have roughly the same structure as the ones used for to generate a JSON string representation of the array. An important difference with JSON indexing is that the vectors are -specified using arrays instead of binary strings. Simply add the +specified using arrays instead of binary strings. Add the embedding as an array field without using the `pack()` function as you would with a hash. ```php $content = "That is a very happy person"; -$emb = $extractor($content, normalize: true, pooling: 'mean'); +$emb = $embed($content); $client->jsonset("jdoc:0", "$", json_encode( [ "content" => $content, "genre" => "persons", - "embedding" => $emb[0] + "embedding" => $emb ], JSON_THROW_ON_ERROR ) ); $content = "That is a happy dog"; -$emb = $extractor($content, normalize: true, pooling: 'mean'); +$emb = $embed($content); $client->jsonset("jdoc:1","$", json_encode( [ "content" => $content, "genre" => "pets", - "embedding" => $emb[0] + "embedding" => $emb ], JSON_THROW_ON_ERROR ) ); $content = "Today is a sunny day"; -$emb = $extractor($content, normalize: true, pooling: 'mean'); +$emb = $embed($content); $client->jsonset("jdoc:2", "$", json_encode( [ "content" => $content, "genre" => "weather", - "embedding" => $emb[0] + "embedding" => $emb ], JSON_THROW_ON_ERROR ) @@ -432,16 +442,16 @@ the `embedding` field of the JSON was specified as an array. ```php $queryText = "That is a happy person"; -$queryEmb = $extractor($queryText, normalize: true, pooling: 'mean'); +$queryEmb = $embed($queryText); $result = $client->ftsearch( "vector_json_idx", '*=>[KNN 3 @embedding $vec AS vector_distance]', - new SearchArguments() + (new SearchArguments()) ->addReturn(1, "vector_distance") ->dialect("2") ->params([ - "vec", pack('g*', ...$queryEmb[0]) + "vec", pack('g*', ...$queryEmb) ]) ->sortBy("vector_distance") ); @@ -453,11 +463,11 @@ query is the same as for hash: ``` Number of results: 3 Key: jdoc:0 -Field: vector_distance, Value: 3.76152896881 +Field: vector_distance, Value: 0.133717402816 Key: jdoc:1 -Field: vector_distance, Value: 18.6544265747 +Field: vector_distance, Value: 0.621603608131 Key: jdoc:2 -Field: vector_distance, Value: 44.6189727783 +Field: vector_distance, Value: 1.46157670021 ``` ## Learn more diff --git a/local_examples/client-specific/go/home_json_example_test.go b/local_examples/client-specific/go/home_json_example_test.go index 46ef638c65..bbdccf3541 100644 --- a/local_examples/client-specific/go/home_json_example_test.go +++ b/local_examples/client-specific/go/home_json_example_test.go @@ -220,7 +220,7 @@ func ExampleClient_search_json() { // STEP_END // Output: - // {1 [{user:3 map[$:{"age":35,"city":"Tel Aviv","email":"paul.zamir@example.com","name":"Paul Zamir"}] }]} + // {1 [{user:3 map[$:{"age":35,"city":"Tel Aviv","email":"paul.zamir@example.com","name":"Paul Zamir"}] }] []} // London // Tel Aviv // 0 @@ -376,5 +376,5 @@ func ExampleClient_search_hash() { // REMOVE_END // Output: - // {1 [{huser:3 map[age:35 city:Tel Aviv email:paul.zamir@example.com name:Paul Zamir] }]} + // {1 [{huser:3 map[age:35 city:Tel Aviv email:paul.zamir@example.com name:Paul Zamir] }] []} } diff --git a/local_examples/client-specific/go/home_query_vec.go b/local_examples/client-specific/go/home_query_vec.go index f2adced8a3..5da812ed6d 100644 --- a/local_examples/client-specific/go/home_query_vec.go +++ b/local_examples/client-specific/go/home_query_vec.go @@ -80,7 +80,7 @@ func main() { // STEP_START embedder // Create a Hugot session - session, err := hugot.NewGoSession() + session, err := hugot.NewGoSession(ctx) if err != nil { panic(err) } @@ -94,7 +94,7 @@ func main() { // Download the model downloadOptions := hugot.NewDownloadOptions() downloadOptions.OnnxFilePath = "onnx/model.onnx" // Specify which ONNX file to use - modelPath, err := hugot.DownloadModel("sentence-transformers/all-MiniLM-L6-v2", "./models/", downloadOptions) + modelPath, err := hugot.DownloadModel(ctx, "sentence-transformers/all-MiniLM-L6-v2", "./models/", downloadOptions) if err != nil { panic(err) } @@ -124,7 +124,7 @@ func main() { } // Generate embeddings using Hugot - embeddingResult, err := embeddingPipeline.RunPipeline(sentences) + embeddingResult, err := embeddingPipeline.RunPipeline(ctx, sentences) if err != nil { panic(err) } @@ -173,7 +173,7 @@ func main() { // STEP_START query // Generate query embedding using Hugot - queryResult, err := embeddingPipeline.RunPipeline([]string{ + queryResult, err := embeddingPipeline.RunPipeline(ctx, []string{ "That is a happy person", }) @@ -272,7 +272,7 @@ func main() { // STEP_START json_query // Generate query embedding for JSON search using Hugot - jsonQueryResult, err := embeddingPipeline.RunPipeline([]string{ + jsonQueryResult, err := embeddingPipeline.RunPipeline(ctx, []string{ "That is a happy person", }) diff --git a/local_examples/client-specific/lettuce-async/HomeQueryVecExample.java b/local_examples/client-specific/lettuce-async/HomeQueryVecExample.java index 231f9a69a9..9f455f4816 100644 --- a/local_examples/client-specific/lettuce-async/HomeQueryVecExample.java +++ b/local_examples/client-specific/lettuce-async/HomeQueryVecExample.java @@ -5,6 +5,9 @@ // Lettuce client and Redis Search classes. import io.lettuce.core.*; import io.lettuce.core.api.StatefulRedisConnection; +import io.lettuce.core.codec.ByteArrayCodec; +import io.lettuce.core.codec.RedisCodec; +import io.lettuce.core.codec.StringCodec; import io.lettuce.core.api.async.RedisAsyncCommands; import io.lettuce.core.search.arguments.*; import io.lettuce.core.search.SearchReply; @@ -34,12 +37,12 @@ public class HomeQueryVecExample { // STEP_START helper_method - private ByteBuffer floatArrayToByteBuffer(float[] vector) { + private byte[] floatArrayToByteArray(float[] vector) { ByteBuffer buffer = ByteBuffer.allocate(vector.length * 4).order(ByteOrder.LITTLE_ENDIAN); for (float value : vector) { buffer.putFloat(value); } - return (ByteBuffer) buffer.flip(); + return buffer.array(); } // STEP_END @@ -66,9 +69,10 @@ public void run() { RedisClient redisClient = RedisClient.create("redis://localhost:6379"); try (StatefulRedisConnection connection = redisClient.connect(); - StatefulRedisConnection binConnection = redisClient.connect(new ByteBufferCodec())) { + StatefulRedisConnection binConnection = redisClient + .connect(RedisCodec.of(StringCodec.UTF8, ByteArrayCodec.INSTANCE))) { RedisAsyncCommands asyncCommands = connection.async(); - RedisAsyncCommands binAsyncCommands = binConnection.async(); + RedisAsyncCommands binAsyncCommands = binConnection.async(); // ... // STEP_END // REMOVE_START @@ -106,17 +110,17 @@ VectorFieldArgs. builder().name("embedding").hnsw().type(VectorFieldArgs // STEP_START add_data String sentence1 = "That is a very happy person"; - Map doc1 = new HashMap<>(); - doc1.put(ByteBuffer.wrap("content".getBytes()), ByteBuffer.wrap(sentence1.getBytes())); - doc1.put(ByteBuffer.wrap("genre".getBytes()), ByteBuffer.wrap("persons".getBytes())); + Map doc1 = new HashMap<>(); + doc1.put("content", sentence1.getBytes(StandardCharsets.UTF_8)); + doc1.put("genre", "persons".getBytes(StandardCharsets.UTF_8)); try { - doc1.put(ByteBuffer.wrap("embedding".getBytes()), floatArrayToByteBuffer(predictor.predict(sentence1))); + doc1.put("embedding", floatArrayToByteArray(predictor.predict(sentence1))); } catch (Exception e) { // ... } - CompletableFuture addDoc1 = binAsyncCommands.hset(ByteBuffer.wrap("doc:1".getBytes()), doc1) + CompletableFuture addDoc1 = binAsyncCommands.hset("doc:1", doc1) .thenApply(result -> { // REMOVE_START assertThat(result).isEqualTo(3L); @@ -127,17 +131,17 @@ VectorFieldArgs. builder().name("embedding").hnsw().type(VectorFieldArgs String sentence2 = "That is a happy dog"; - Map doc2 = new HashMap<>(); - doc2.put(ByteBuffer.wrap("content".getBytes()), ByteBuffer.wrap(sentence2.getBytes())); - doc2.put(ByteBuffer.wrap("genre".getBytes()), ByteBuffer.wrap("pets".getBytes())); + Map doc2 = new HashMap<>(); + doc2.put("content", sentence2.getBytes(StandardCharsets.UTF_8)); + doc2.put("genre", "pets".getBytes(StandardCharsets.UTF_8)); try { - doc2.put(ByteBuffer.wrap("embedding".getBytes()), floatArrayToByteBuffer(predictor.predict(sentence2))); + doc2.put("embedding", floatArrayToByteArray(predictor.predict(sentence2))); } catch (Exception e) { // ... } - CompletableFuture addDoc2 = binAsyncCommands.hset(ByteBuffer.wrap("doc:2".getBytes()), doc2) + CompletableFuture addDoc2 = binAsyncCommands.hset("doc:2", doc2) .thenApply(result -> { // REMOVE_START assertThat(result).isEqualTo(3L); @@ -148,17 +152,17 @@ VectorFieldArgs. builder().name("embedding").hnsw().type(VectorFieldArgs String sentence3 = "Today is a sunny day"; - Map doc3 = new HashMap<>(); - doc3.put(ByteBuffer.wrap("content".getBytes()), ByteBuffer.wrap(sentence3.getBytes())); - doc3.put(ByteBuffer.wrap("genre".getBytes()), ByteBuffer.wrap("weather".getBytes())); + Map doc3 = new HashMap<>(); + doc3.put("content", sentence3.getBytes(StandardCharsets.UTF_8)); + doc3.put("genre", "weather".getBytes(StandardCharsets.UTF_8)); try { - doc3.put(ByteBuffer.wrap("embedding".getBytes()), floatArrayToByteBuffer(predictor.predict(sentence3))); + doc3.put("embedding", floatArrayToByteArray(predictor.predict(sentence3))); } catch (Exception e) { // ... } - CompletableFuture addDoc3 = binAsyncCommands.hset(ByteBuffer.wrap("doc:3".getBytes()), doc3) + CompletableFuture addDoc3 = binAsyncCommands.hset("doc:3", doc3) .thenApply(result -> { // REMOVE_START assertThat(result).isEqualTo(3L); @@ -177,20 +181,18 @@ VectorFieldArgs. builder().name("embedding").hnsw().type(VectorFieldArgs List> himportDocs = new ArrayList<>(); - try (HashImport fieldset = HashImport.of(id -> ByteBuffer.wrap(("himport:" + id).getBytes()), - ByteBuffer.wrap("content".getBytes()), ByteBuffer.wrap("genre".getBytes()), - ByteBuffer.wrap("embedding".getBytes()))) { + try (HashImport fieldset = HashImport.of("content", "genre", "embedding")) { for (String[] doc : docData) { - ByteBuffer docEmbedding = null; + byte[] docEmbedding = null; try { - docEmbedding = floatArrayToByteBuffer(predictor.predict(doc[1])); + docEmbedding = floatArrayToByteArray(predictor.predict(doc[1])); } catch (Exception e) { // ... } - himportDocs.add(binAsyncCommands.himportSet(ByteBuffer.wrap(doc[0].getBytes()), fieldset, - ByteBuffer.wrap(doc[1].getBytes()), ByteBuffer.wrap(doc[2].getBytes()), docEmbedding)); + himportDocs.add(binAsyncCommands.himportSet(doc[0], fieldset, doc[1].getBytes(StandardCharsets.UTF_8), + doc[2].getBytes(StandardCharsets.UTF_8), docEmbedding)); } } @@ -212,27 +214,21 @@ VectorFieldArgs. builder().name("embedding").hnsw().type(VectorFieldArgs // ... } - SearchArgs searchArgs = SearchArgs. builder() - .param(ByteBuffer.wrap("vec".getBytes()), floatArrayToByteBuffer(queryEmbedding)) - .returnField(ByteBuffer.wrap("content".getBytes())) - .returnField(ByteBuffer.wrap("vector_distance".getBytes())) - .sortBy(SortByArgs. builder().attribute(ByteBuffer.wrap("vector_distance".getBytes())).build()) - .build(); + SearchArgs searchArgs = SearchArgs. builder() + .param("vec", floatArrayToByteArray(queryEmbedding)).returnField("content").returnField("vector_distance") + .sortBy(SortByArgs. builder().attribute("vector_distance").build()).build(); - CompletableFuture> hashQuery = binAsyncCommands - .ftSearch(ByteBuffer.wrap("vector_idx".getBytes()), - ByteBuffer.wrap("*=>[KNN 3 @embedding $vec AS vector_distance]".getBytes()), searchArgs) + CompletableFuture> hashQuery = binAsyncCommands + .ftSearch("vector_idx", "*=>[KNN 3 @embedding $vec AS vector_distance]".getBytes(StandardCharsets.UTF_8), + searchArgs) .thenApply(result -> { - List> results = result.getResults(); + List> results = result.getResults(); results.forEach(r -> { - String id = StandardCharsets.UTF_8.decode(r.getId()).toString(); - String content = StandardCharsets.UTF_8 - .decode(r.getFields().get(ByteBuffer.wrap("content".getBytes()))).toString(); - String distance = StandardCharsets.UTF_8 - .decode(r.getFields().get(ByteBuffer.wrap("vector_distance".getBytes()))).toString(); + String content = new String(r.getFields().get("content"), StandardCharsets.UTF_8); + String distance = new String(r.getFields().get("vector_distance"), StandardCharsets.UTF_8); - System.out.println("ID: " + id + ", Content: " + content + ", Distance: " + distance); + System.out.println("ID: " + r.getId() + ", Content: " + content + ", Distance: " + distance); }); // >>> ID: doc:1, Content: That is a very happy person, Distance: 0.114169836044 // >>> ID: doc:2, Content: That is a happy dog, Distance: 0.610845506191 @@ -273,10 +269,10 @@ VectorFieldArgs. builder().name("$.embedding").as("embedding").hnsw() // STEP_START json_data JsonParser parser = asyncCommands.getJsonParser(); - String jSentence1 = "\"That is a very happy person\""; + String jSentence1 = "That is a very happy person"; JsonObject jDoc1 = parser.createJsonObject(); - jDoc1.put("content", parser.createJsonValue(jSentence1)); + jDoc1.put("content", parser.createJsonValue("\"" + jSentence1 + "\"")); jDoc1.put("genre", parser.createJsonValue("\"persons\"")); try { @@ -294,10 +290,10 @@ VectorFieldArgs. builder().name("$.embedding").as("embedding").hnsw() return result; }).toCompletableFuture(); - String jSentence2 = "\"That is a happy dog\""; + String jSentence2 = "That is a happy dog"; JsonObject jDoc2 = parser.createJsonObject(); - jDoc2.put("content", parser.createJsonValue(jSentence2)); + jDoc2.put("content", parser.createJsonValue("\"" + jSentence2 + "\"")); jDoc2.put("genre", parser.createJsonValue("\"pets\"")); try { @@ -315,10 +311,10 @@ VectorFieldArgs. builder().name("$.embedding").as("embedding").hnsw() return result; }).toCompletableFuture(); - String jSentence3 = "\"Today is a sunny day\""; + String jSentence3 = "Today is a sunny day"; JsonObject jDoc3 = parser.createJsonObject(); - jDoc3.put("content", parser.createJsonValue(jSentence3)); + jDoc3.put("content", parser.createJsonValue("\"" + jSentence3 + "\"")); jDoc3.put("genre", parser.createJsonValue("\"weather\"")); try { @@ -348,31 +344,25 @@ VectorFieldArgs. builder().name("$.embedding").as("embedding").hnsw() // ... } - SearchArgs jsonSearchArgs = SearchArgs. builder() - .param(ByteBuffer.wrap("vec".getBytes()), floatArrayToByteBuffer(jsonQueryEmbedding)) - .returnField(ByteBuffer.wrap("content".getBytes())) - .returnField(ByteBuffer.wrap("vector_distance".getBytes())) - .sortBy(SortByArgs. builder().attribute(ByteBuffer.wrap("vector_distance".getBytes())).build()) - .build(); + SearchArgs jsonSearchArgs = SearchArgs. builder() + .param("vec", floatArrayToByteArray(jsonQueryEmbedding)).returnField("content").returnField("vector_distance") + .sortBy(SortByArgs. builder().attribute("vector_distance").build()).build(); - CompletableFuture> jsonQuery = binAsyncCommands - .ftSearch(ByteBuffer.wrap("vector_json_idx".getBytes()), - ByteBuffer.wrap("*=>[KNN 3 @embedding $vec AS vector_distance]".getBytes()), jsonSearchArgs) + CompletableFuture> jsonQuery = binAsyncCommands + .ftSearch("vector_json_idx", "*=>[KNN 3 @embedding $vec AS vector_distance]".getBytes(StandardCharsets.UTF_8), + jsonSearchArgs) .thenApply(result -> { - List> results = result.getResults(); + List> results = result.getResults(); results.forEach(r -> { - String id = StandardCharsets.UTF_8.decode(r.getId()).toString(); - String content = StandardCharsets.UTF_8 - .decode(r.getFields().get(ByteBuffer.wrap("content".getBytes()))).toString(); - String distance = StandardCharsets.UTF_8 - .decode(r.getFields().get(ByteBuffer.wrap("vector_distance".getBytes()))).toString(); + String content = new String(r.getFields().get("content"), StandardCharsets.UTF_8); + String distance = new String(r.getFields().get("vector_distance"), StandardCharsets.UTF_8); - System.out.println("ID: " + id + ", Content: " + content + ", Distance: " + distance); + System.out.println("ID: " + r.getId() + ", Content: " + content + ", Distance: " + distance); }); - // >>> ID: jdoc:1, Content: "That is a very happy person", Distance:0.628328084946 - // >>> ID: jdoc:2, Content: That is a happy dog, Distance: 0.895147025585 - // >>> ID: jdoc:3, Content: "Today is a sunny day", Distance: 1.49569523335 + // >>> ID: jdoc:1, Content: That is a very happy person, Distance: 0.114169836044 + // >>> ID: jdoc:2, Content: That is a happy dog, Distance: 0.610845506191 + // >>> ID: jdoc:3, Content: Today is a sunny day, Distance: 1.48624765873 // REMOVE_START assertThat(result.getCount()).isEqualTo(3); diff --git a/local_examples/client-specific/lettuce-reactive/HomeQueryVecExample.java b/local_examples/client-specific/lettuce-reactive/HomeQueryVecExample.java index 78c94b1059..ecdce64dc6 100644 --- a/local_examples/client-specific/lettuce-reactive/HomeQueryVecExample.java +++ b/local_examples/client-specific/lettuce-reactive/HomeQueryVecExample.java @@ -5,6 +5,9 @@ // Lettuce client and Redis Search classes. import io.lettuce.core.*; import io.lettuce.core.api.StatefulRedisConnection; +import io.lettuce.core.codec.ByteArrayCodec; +import io.lettuce.core.codec.RedisCodec; +import io.lettuce.core.codec.StringCodec; import io.lettuce.core.api.reactive.RedisReactiveCommands; import io.lettuce.core.search.arguments.*; import io.lettuce.core.search.SearchReply; @@ -34,12 +37,12 @@ public class HomeQueryVecExample { // STEP_START helper_method - private ByteBuffer floatArrayToByteBuffer(float[] vector) { + private byte[] floatArrayToByteArray(float[] vector) { ByteBuffer buffer = ByteBuffer.allocate(vector.length * 4).order(ByteOrder.LITTLE_ENDIAN); for (float value : vector) { buffer.putFloat(value); } - return (ByteBuffer) buffer.flip(); + return buffer.array(); } // STEP_END @@ -66,9 +69,10 @@ public void run() { RedisClient redisClient = RedisClient.create("redis://localhost:6379"); try (StatefulRedisConnection connection = redisClient.connect(); - StatefulRedisConnection binConnection = redisClient.connect(new ByteBufferCodec())) { + StatefulRedisConnection binConnection = redisClient + .connect(RedisCodec.of(StringCodec.UTF8, ByteArrayCodec.INSTANCE))) { RedisReactiveCommands reactiveCommands = connection.reactive(); - RedisReactiveCommands binReactiveCommands = binConnection.reactive(); + RedisReactiveCommands binReactiveCommands = binConnection.reactive(); // ... // STEP_END // REMOVE_START @@ -98,17 +102,17 @@ VectorFieldArgs. builder().name("embedding").hnsw().type(VectorFieldArgs // STEP_START add_data String sentence1 = "That is a very happy person"; - Map doc1 = new HashMap<>(); - doc1.put(ByteBuffer.wrap("content".getBytes()), ByteBuffer.wrap(sentence1.getBytes())); - doc1.put(ByteBuffer.wrap("genre".getBytes()), ByteBuffer.wrap("persons".getBytes())); + Map doc1 = new HashMap<>(); + doc1.put("content", sentence1.getBytes(StandardCharsets.UTF_8)); + doc1.put("genre", "persons".getBytes(StandardCharsets.UTF_8)); try { - doc1.put(ByteBuffer.wrap("embedding".getBytes()), floatArrayToByteBuffer(predictor.predict(sentence1))); + doc1.put("embedding", floatArrayToByteArray(predictor.predict(sentence1))); } catch (Exception e) { // ... } - Mono addDoc1 = binReactiveCommands.hset(ByteBuffer.wrap("doc:1".getBytes()), doc1).doOnNext(result -> { + Mono addDoc1 = binReactiveCommands.hset("doc:1", doc1).doOnNext(result -> { System.out.println(result); // >>> 3 // REMOVE_START assertThat(result).isEqualTo(3L); @@ -117,17 +121,17 @@ VectorFieldArgs. builder().name("embedding").hnsw().type(VectorFieldArgs String sentence2 = "That is a happy dog"; - Map doc2 = new HashMap<>(); - doc2.put(ByteBuffer.wrap("content".getBytes()), ByteBuffer.wrap(sentence2.getBytes())); - doc2.put(ByteBuffer.wrap("genre".getBytes()), ByteBuffer.wrap("pets".getBytes())); + Map doc2 = new HashMap<>(); + doc2.put("content", sentence2.getBytes(StandardCharsets.UTF_8)); + doc2.put("genre", "pets".getBytes(StandardCharsets.UTF_8)); try { - doc2.put(ByteBuffer.wrap("embedding".getBytes()), floatArrayToByteBuffer(predictor.predict(sentence2))); + doc2.put("embedding", floatArrayToByteArray(predictor.predict(sentence2))); } catch (Exception e) { // ... } - Mono addDoc2 = binReactiveCommands.hset(ByteBuffer.wrap("doc:2".getBytes()), doc2).doOnNext(result -> { + Mono addDoc2 = binReactiveCommands.hset("doc:2", doc2).doOnNext(result -> { System.out.println(result); // >>> 3 // REMOVE_START assertThat(result).isEqualTo(3L); @@ -136,17 +140,17 @@ VectorFieldArgs. builder().name("embedding").hnsw().type(VectorFieldArgs String sentence3 = "Today is a sunny day"; - Map doc3 = new HashMap<>(); - doc3.put(ByteBuffer.wrap("content".getBytes()), ByteBuffer.wrap(sentence3.getBytes())); - doc3.put(ByteBuffer.wrap("genre".getBytes()), ByteBuffer.wrap("weather".getBytes())); + Map doc3 = new HashMap<>(); + doc3.put("content", sentence3.getBytes(StandardCharsets.UTF_8)); + doc3.put("genre", "weather".getBytes(StandardCharsets.UTF_8)); try { - doc3.put(ByteBuffer.wrap("embedding".getBytes()), floatArrayToByteBuffer(predictor.predict(sentence3))); + doc3.put("embedding", floatArrayToByteArray(predictor.predict(sentence3))); } catch (Exception e) { // ... } - Mono addDoc3 = binReactiveCommands.hset(ByteBuffer.wrap("doc:3".getBytes()), doc3).doOnNext(result -> { + Mono addDoc3 = binReactiveCommands.hset("doc:3", doc3).doOnNext(result -> { System.out.println(result); // >>> 3 // REMOVE_START assertThat(result).isEqualTo(3L); @@ -163,20 +167,18 @@ VectorFieldArgs. builder().name("embedding").hnsw().type(VectorFieldArgs List> himportDocs = new ArrayList<>(); - try (HashImport fieldset = HashImport.of(id -> ByteBuffer.wrap(("himport:" + id).getBytes()), - ByteBuffer.wrap("content".getBytes()), ByteBuffer.wrap("genre".getBytes()), - ByteBuffer.wrap("embedding".getBytes()))) { + try (HashImport fieldset = HashImport.of("content", "genre", "embedding")) { for (String[] doc : docData) { - ByteBuffer docEmbedding = null; + byte[] docEmbedding = null; try { - docEmbedding = floatArrayToByteBuffer(predictor.predict(doc[1])); + docEmbedding = floatArrayToByteArray(predictor.predict(doc[1])); } catch (Exception e) { // ... } - himportDocs.add(binReactiveCommands.himportSet(ByteBuffer.wrap(doc[0].getBytes()), fieldset, - ByteBuffer.wrap(doc[1].getBytes()), ByteBuffer.wrap(doc[2].getBytes()), docEmbedding)); + himportDocs.add(binReactiveCommands.himportSet(doc[0], fieldset, doc[1].getBytes(StandardCharsets.UTF_8), + doc[2].getBytes(StandardCharsets.UTF_8), docEmbedding)); } // Subscribe before the fieldset closes. @@ -194,27 +196,21 @@ VectorFieldArgs. builder().name("embedding").hnsw().type(VectorFieldArgs // ... } - SearchArgs searchArgs = SearchArgs. builder() - .param(ByteBuffer.wrap("vec".getBytes()), floatArrayToByteBuffer(queryEmbedding)) - .returnField(ByteBuffer.wrap("content".getBytes())) - .returnField(ByteBuffer.wrap("vector_distance".getBytes())) - .sortBy(SortByArgs. builder().attribute(ByteBuffer.wrap("vector_distance".getBytes())).build()) - .build(); + SearchArgs searchArgs = SearchArgs. builder() + .param("vec", floatArrayToByteArray(queryEmbedding)).returnField("content").returnField("vector_distance") + .sortBy(SortByArgs. builder().attribute("vector_distance").build()).build(); - Mono> hashQuery = binReactiveCommands - .ftSearch(ByteBuffer.wrap("vector_idx".getBytes()), - ByteBuffer.wrap("*=>[KNN 3 @embedding $vec AS vector_distance]".getBytes()), searchArgs) + Mono> hashQuery = binReactiveCommands + .ftSearch("vector_idx", "*=>[KNN 3 @embedding $vec AS vector_distance]".getBytes(StandardCharsets.UTF_8), + searchArgs) .doOnNext(result -> { - List> results = result.getResults(); + List> results = result.getResults(); results.forEach(r -> { - String id = StandardCharsets.UTF_8.decode(r.getId()).toString(); - String content = StandardCharsets.UTF_8 - .decode(r.getFields().get(ByteBuffer.wrap("content".getBytes()))).toString(); - String distance = StandardCharsets.UTF_8 - .decode(r.getFields().get(ByteBuffer.wrap("vector_distance".getBytes()))).toString(); + String content = new String(r.getFields().get("content"), StandardCharsets.UTF_8); + String distance = new String(r.getFields().get("vector_distance"), StandardCharsets.UTF_8); - System.out.println("ID: " + id + ", Content: " + content + ", Distance: " + distance); + System.out.println("ID: " + r.getId() + ", Content: " + content + ", Distance: " + distance); }); // >>> ID: doc:1, Content: That is a very happy person, Distance: 0.114169836044 // >>> ID: doc:2, Content: That is a happy dog, Distance: 0.610845506191 @@ -250,9 +246,9 @@ VectorFieldArgs. builder().name("$.embedding").as("embedding").hnsw() // STEP_START json_data JsonParser parser = reactiveCommands.getJsonParser(); - String jSentence1 = "\"That is a very happy person\""; + String jSentence1 = "That is a very happy person"; JsonObject jDoc1 = parser.createJsonObject(); - jDoc1.put("content", parser.createJsonValue(jSentence1)); + jDoc1.put("content", parser.createJsonValue("\"" + jSentence1 + "\"")); jDoc1.put("genre", parser.createJsonValue("\"persons\"")); try { @@ -268,9 +264,9 @@ VectorFieldArgs. builder().name("$.embedding").as("embedding").hnsw() // REMOVE_END }).then(); - String jSentence2 = "\"That is a happy dog\""; + String jSentence2 = "That is a happy dog"; JsonObject jDoc2 = parser.createJsonObject(); - jDoc2.put("content", parser.createJsonValue(jSentence2)); + jDoc2.put("content", parser.createJsonValue("\"" + jSentence2 + "\"")); jDoc2.put("genre", parser.createJsonValue("\"pets\"")); try { @@ -286,9 +282,9 @@ VectorFieldArgs. builder().name("$.embedding").as("embedding").hnsw() // REMOVE_END }).then(); - String jSentence3 = "\"Today is a sunny day\""; + String jSentence3 = "Today is a sunny day"; JsonObject jDoc3 = parser.createJsonObject(); - jDoc3.put("content", parser.createJsonValue(jSentence3)); + jDoc3.put("content", parser.createJsonValue("\"" + jSentence3 + "\"")); jDoc3.put("genre", parser.createJsonValue("\"weather\"")); try { @@ -316,31 +312,25 @@ VectorFieldArgs. builder().name("$.embedding").as("embedding").hnsw() // ... } - SearchArgs jsonSearchArgs = SearchArgs. builder() - .param(ByteBuffer.wrap("vec".getBytes()), floatArrayToByteBuffer(jsonQueryEmbedding)) - .returnField(ByteBuffer.wrap("content".getBytes())) - .returnField(ByteBuffer.wrap("vector_distance".getBytes())) - .sortBy(SortByArgs. builder().attribute(ByteBuffer.wrap("vector_distance".getBytes())).build()) - .build(); + SearchArgs jsonSearchArgs = SearchArgs. builder() + .param("vec", floatArrayToByteArray(jsonQueryEmbedding)).returnField("content").returnField("vector_distance") + .sortBy(SortByArgs. builder().attribute("vector_distance").build()).build(); - Mono> jsonQuery = binReactiveCommands - .ftSearch(ByteBuffer.wrap("vector_json_idx".getBytes()), - ByteBuffer.wrap("*=>[KNN 3 @embedding $vec AS vector_distance]".getBytes()), jsonSearchArgs) + Mono> jsonQuery = binReactiveCommands + .ftSearch("vector_json_idx", "*=>[KNN 3 @embedding $vec AS vector_distance]".getBytes(StandardCharsets.UTF_8), + jsonSearchArgs) .doOnNext(result -> { - List> results = result.getResults(); + List> results = result.getResults(); results.forEach(r -> { - String id = StandardCharsets.UTF_8.decode(r.getId()).toString(); - String content = StandardCharsets.UTF_8 - .decode(r.getFields().get(ByteBuffer.wrap("content".getBytes()))).toString(); - String distance = StandardCharsets.UTF_8 - .decode(r.getFields().get(ByteBuffer.wrap("vector_distance".getBytes()))).toString(); + String content = new String(r.getFields().get("content"), StandardCharsets.UTF_8); + String distance = new String(r.getFields().get("vector_distance"), StandardCharsets.UTF_8); - System.out.println("ID: " + id + ", Content: " + content + ", Distance: " + distance); + System.out.println("ID: " + r.getId() + ", Content: " + content + ", Distance: " + distance); }); - // >>> ID: jdoc:1, Content: "That is a very happy person", Distance: 0.628328084946 - // >>> ID: jdoc:2, Content: "That is a happy dog", Distance: 0.895147025585 - // >>> ID: jdoc:3, Content: "Today is a sunny day", Distance: 1.49569523335 + // >>> ID: jdoc:1, Content: That is a very happy person, Distance: 0.114169836044 + // >>> ID: jdoc:2, Content: That is a happy dog, Distance: 0.610845506191 + // >>> ID: jdoc:3, Content: Today is a sunny day, Distance: 1.48624765873 // REMOVE_START assertThat(result.getCount()).isEqualTo(3); // REMOVE_END From 143558e7ca24725c437cd887b0c2228fda2cff74 Mon Sep 17 00:00:00 2001 From: Zdravko Donev Date: Sun, 18 Oct 2026 02:02:34 +0300 Subject: [PATCH 21/76] Explain RDI CDC offsets and source retention --- .../redis-data-integration/_index.md | 5 +- .../architecture/_index.md | 4 + .../cdc-offsets-and-retention.md | 153 ++++++++++++++++++ .../data-pipelines/prepare-dbs/_index.md | 4 + .../data-pipelines/prepare-dbs/postgresql.md | 59 ++++++- static/images/rdi/cdc-retention/bookmark.svg | 43 +++++ static/images/rdi/cdc-retention/mongodb.svg | 43 +++++ .../rdi/cdc-retention/mysql-mariadb.svg | 43 +++++ static/images/rdi/cdc-retention/oracle.svg | 43 +++++ .../images/rdi/cdc-retention/postgresql.svg | 66 ++++++++ static/images/rdi/cdc-retention/snowflake.svg | 43 +++++ static/images/rdi/cdc-retention/spanner.svg | 44 +++++ .../images/rdi/cdc-retention/sql-server.svg | 44 +++++ 13 files changed, 585 insertions(+), 9 deletions(-) create mode 100644 content/integrate/redis-data-integration/cdc-offsets-and-retention.md create mode 100644 static/images/rdi/cdc-retention/bookmark.svg create mode 100644 static/images/rdi/cdc-retention/mongodb.svg create mode 100644 static/images/rdi/cdc-retention/mysql-mariadb.svg create mode 100644 static/images/rdi/cdc-retention/oracle.svg create mode 100644 static/images/rdi/cdc-retention/postgresql.svg create mode 100644 static/images/rdi/cdc-retention/snowflake.svg create mode 100644 static/images/rdi/cdc-retention/spanner.svg create mode 100644 static/images/rdi/cdc-retention/sql-server.svg diff --git a/content/integrate/redis-data-integration/_index.md b/content/integrate/redis-data-integration/_index.md index 6aeb833ebb..7780e71074 100644 --- a/content/integrate/redis-data-integration/_index.md +++ b/content/integrate/redis-data-integration/_index.md @@ -102,4 +102,7 @@ RDI can capture data from any of the following sources: ## Documentation -Learn more about RDI from the other pages in this section: \ No newline at end of file +Learn why a saved capture position can become unusable in +[Understand CDC offsets and source retention](/content/integrate/redis-data-integration/cdc-offsets-and-retention.md). + +Learn more about RDI from the other pages in this section: diff --git a/content/integrate/redis-data-integration/architecture/_index.md b/content/integrate/redis-data-integration/architecture/_index.md index c55e9db9f5..564ccb1a83 100644 --- a/content/integrate/redis-data-integration/architecture/_index.md +++ b/content/integrate/redis-data-integration/architecture/_index.md @@ -82,6 +82,10 @@ overall state). ## Checkpointing +The processor checkpoint is separate from the collector's position in source change +history. See [Understand CDC offsets and source retention](/content/integrate/redis-data-integration/cdc-offsets-and-retention.md) +for how source retention affects recovery after an interruption. + RDI uses Redis streams to store the sequence of change events captured from the sources. The events are then retrieved in order from the streams, processed, and written to the target. The stream diff --git a/content/integrate/redis-data-integration/cdc-offsets-and-retention.md b/content/integrate/redis-data-integration/cdc-offsets-and-retention.md new file mode 100644 index 0000000000..158a825f71 --- /dev/null +++ b/content/integrate/redis-data-integration/cdc-offsets-and-retention.md @@ -0,0 +1,153 @@ +--- +Title: Understand CDC offsets and source retention +alwaysopen: false +categories: +- docs +- integrate +- rs +- rdi +description: Learn why RDI can lose access to source change history during an interruption and how to plan for recovery. +group: di +hideListLinks: false +linkTitle: CDC offsets and retention +summary: Understand saved capture positions, source history, and how retention affects recovery after an interruption. +type: integration +weight: 31 +--- + +Redis Data Integration (RDI) can resume after an interruption only if the source still provides the change history and capture state it needs. Saving a position does not preserve that history. + +Imagine that RDI stops on Friday and starts again on Sunday. Your application keeps writing to Oracle during those two days. RDI remembers where it stopped, but Oracle has removed some of the redo logs needed to continue. The collector reports that its saved system change number (SCN) is no longer available. + +The source tables still exist. RDI has not forgotten its position. What is missing is the history between that position and the present. + +Two days is only an example. There is no universal expiration time for an RDI offset. The outcome depends on the database, its settings, the amount of change, and how far capture had already fallen behind. + +## Think of an offset as a bookmark + +Change data capture (CDC) lets RDI follow changes instead of repeatedly reading the entire source dataset. Four concepts explain how it works: + +- A **snapshot** reads the current dataset to establish a starting point. +- **Change history** records subsequent inserts, updates, and deletes. Depending on the database, this history lives in logs, change tables, or a change stream. +- An **offset** records the collector's progress through that history. Think of it as a bookmark, not a copy of the pages. +- **Retention** determines how long, or how much of, that history remains available. + +The source continues maintaining its history while RDI is disconnected. If the required history remains available, RDI can read the missed changes. If the source removes it, the bookmark alone cannot reconstruct those changes. + +![Two timelines show RDI resuming when its bookmark is inside retained history and failing when required history has been removed.](/images/rdi/cdc-retention/bookmark.svg) +{width="560"} + +*These timelines are illustrations, not database defaults. The important boundary is the oldest history still available when capture resumes.* + +The collector's offset is also different from the processor's **checkpoint**. The offset tracks capture from the source. The checkpoint tracks processing of events already captured into RDI. A processor can be behind even while capture is current. If the backlog eventually stops capture through [backpressure](/content/integrate/redis-data-integration/architecture/_index.md#backpressure-mechanism), source retention becomes important again. + +## See how each source keeps history + +The bookmark idea applies across sources, but the way each database retains history differs. The examples assume capture was working before the interruption. A source restart, replacement, or failover can introduce separate capture-state requirements. + +### Oracle: keep the required redo history + +Oracle records changes in redo logs. With LogMiner, RDI reads online and archived redo logs and tracks progress using SCNs. An SCN identifies a position in database history; it is not a timestamp with a fixed lifetime. + +Online redo logs are reused. Archived logs let a reader reach further back, but only while the required files remain available to the mining process. If cleanup removes a required archive during an outage, the collector can no longer continue from its saved position. Having newer logs does not fill the gap. + +![Oracle timelines show a saved SCN with all required redo available, then a saved SCN whose required archived redo has been removed.](/images/rdi/cdc-retention/oracle.svg) +{width="560"} + +Plan archive retention and storage with your database administrator. Check the archive destination RDI actually reads, not only whether backups exist elsewhere. Long-running transactions can also require older redo than the most recent visible activity suggests. + +XStream uses an Oracle capture process and an outbound server, with its own restart position and capture checkpoints. Preserve the history and state required by that capture process; do not apply a LogMiner error diagnosis unchanged to XStream. See [Prepare Oracle](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/oracle.md), [Oracle LogMiner](https://docs.oracle.com/en/database/oracle/oracle-database/19/sutil/oracle-logminer-utility.html), and [XStream restartability](https://docs.oracle.com/en/database/oracle/oracle-database/19/xstrm/xstream-out-concepts.html). + +### MySQL and MariaDB: keep the binary logs + +MySQL and MariaDB record changes in binary logs, often called *binlogs*. A resume position identifies the required place in this history. Using transaction identifiers does not preserve the underlying events after their logs are removed. + +Automatic expiration or a manual purge can remove a file that a disconnected collector still needs. Increasing retention after that purge does not bring the file back. + +![Binary-log timelines show successful resume while the required file remains, then failed resume after that file is purged.](/images/rdi/cdc-retention/mysql-mariadb.svg) +{width="560"} + +Set retention for the interruption you need to tolerate and leave enough disk space for peak writes. Coordinate manual cleanup with CDC readers. Managed services can expose different retention controls, so use the instructions for your deployment in [Prepare MySQL and MariaDB](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/my-sql-mariadb.md). The [MySQL purge reference](https://dev.mysql.com/doc/refman/8.0/en/purge-binary-logs.html) and [MariaDB binary-log guide](https://mariadb.com/docs/server/server-management/server-monitoring-logs/binary-log/using-and-maintaining-the-binary-log) describe the underlying cleanup behavior. + +### PostgreSQL: retain history without filling the disk + +PostgreSQL records changes in its write-ahead log (WAL). RDI reads through a logical replication slot and tracks a log sequence number (LSN). + +A slot can retain WAL that its reader still needs, even while RDI is stopped. This makes PostgreSQL different from a simple rolling time window: the immediate problem may be growing disk usage rather than removed history. + +Retention is not unconditional. A configured slot retention limit can allow required WAL to be removed. Newer PostgreSQL versions also support invalidating slots after an idle timeout. A dropped or invalidated slot cannot be treated as a healthy saved bookmark. + +![PostgreSQL timelines show WAL retained during an outage with growing storage, required WAL removed after a size limit, and an inactive slot invalidated by a timeout.](/images/rdi/cdc-retention/postgresql.svg) +{width="560"} + +Monitor slot health, retained WAL, and free disk together. Choose limits that balance recovery time with database storage capacity. Parameter names and available controls differ across PostgreSQL versions; see [Manage WAL retention](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md#manage-wal-retention). + +The same model applies to Supabase, Neon, AlloyDB for PostgreSQL, Amazon Aurora PostgreSQL, and other managed PostgreSQL services. Check provider-specific slot cleanup, storage, and failover policies rather than assuming that every service retains an idle slot indefinitely. + +### SQL Server: retain the captured change rows + +SQL Server CDC captures changes from the transaction log into change tables. RDI reads those captured changes using an LSN-based position. + +The CDC cleanup process removes old entries from the change tables. If RDI returns after its required entries have been cleaned up, keeping transaction-log backups alone does not make those entries available through the CDC reader. + +![SQL Server timelines show an LSN inside retained CDC change-table history, then outside the available range after cleanup.](/images/rdi/cdc-retention/sql-server.svg) +{width="560"} + +Plan CDC change-table retention for consumer interruptions. Monitor both capture progress into those tables and RDI's progress through them. Capture delays and reader delays are different problems. See [Prepare SQL Server](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/sql-server.md) and Microsoft's [CDC administration guide](https://learn.microsoft.com/en-us/sql/relational-databases/track-changes/administer-and-monitor-change-data-capture-sql-server). + +### MongoDB: keep enough operation-log history + +MongoDB change streams provide a **resume token** that identifies where reading can continue. Resuming requires enough history in the replica set's operation log, or *oplog*. + +For a size-limited oplog, a busy day can retain fewer hours of history than a quiet day. A collector that resumes successfully after one weekend might fail after another weekend with a large write burst. + +![MongoDB timelines show a resume token whose history remains in the oplog, then history overwritten during a write burst.](/images/rdi/cdc-retention/mongodb.svg) +{width="560"} + +Monitor the actual oplog time window. Size it for peak activity, and consider a minimum retention period where available. A minimum retention period can require the oplog to grow beyond its configured size, so account for storage too. Managed-service controls vary. See [Prepare MongoDB](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/mongodb.md) and the [MongoDB oplog guide](https://www.mongodb.com/docs/manual/core/replica-set-oplog/). + +### Spanner: distinguish two retention windows + +Spanner change streams retain change records for a configured period. RDI needs its streaming resume point to remain within the available change-stream history. Once those records expire, a later restart cannot read them from the stream. + +![Spanner timelines show a streaming resume point within change-stream retention and a point older than the retained records.](/images/rdi/cdc-retention/spanner.svg) +{width="560"} + +Spanner also has **database version retention**. That separate setting matters when RDI reads a consistent snapshot at a particular timestamp. Increasing version retention does not substitute for sufficient change-stream retention during streaming. + +Plan the change-stream window for interruptions and the version window for snapshot duration. See [Prepare Spanner](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/spanner.md) and Google's [change-stream overview](https://docs.cloud.google.com/spanner/docs/change-streams). + +### Snowflake: consume changes before the stream becomes stale + +Snowflake streams track an offset against source-table history. The stream does not hold an independent, permanent copy of every change. If the history needed by its offset is no longer retained, the stream becomes **stale**. + +![Snowflake timelines show a usable stream offset with retained table history and a stale stream after required history is no longer retained.](/images/rdi/cdc-retention/snowflake.svg) +{width="560"} + +Monitor stream staleness and the reported `STALE_AFTER` time. Account for table retention and Snowflake's retention-extension rules; do not assume that a stream keeps history indefinitely. Reading a stream with a query alone does not necessarily advance its offset: Snowflake distinguishes querying from consuming changes in a committed transaction. + +Snowflake support in RDI is in preview. See [Prepare Snowflake](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/snowflake.md) and Snowflake's [stream retention and staleness guidance](https://docs.snowflake.com/en/user-guide/streams-intro#data-retention-period-and-staleness). + +## Plan for the whole interruption + +RDI does not have to be completely down for history to become unavailable. A network failure, lost source access, or sustained capture lag can leave its position behind. A slow target can build a backlog until backpressure stops capture. Source cleanup continues independently. + +Use a conservative planning budget: + +**Existing capture lag + longest expected interruption + catch-up allowance + safety margin.** + +This is a planning aid, not a universal database setting. For example, a 48-hour outage budget needs more than 48 hours of retained history if capture was already behind. Catch-up also needs spare processing capacity while the application continues writing. + +Before maintenance, check capture progress, available history, and storage. For size-based limits, estimate how much history peak writes produce, not just average writes. Where history is retained by a slot, check how much additional disk an outage could consume. + +Alert while there is still time to act. Combine [RDI monitoring](/content/integrate/redis-data-integration/observability.md) with source-database checks. A running process or a saved offset does not prove that capture is advancing or that its history remains available. Confirm managed-service limits and cleanup policies with the database owner. + +## Recover when the history is gone + +First identify whether required history is missing or temporarily inaccessible. A permissions or archive-access problem needs a different correction from deleted history. + +If the database can make the original history available again through a supported procedure, resuming may still be possible. Otherwise, rebuilding from a fresh snapshot establishes a new starting point. Increasing retention protects future history; it cannot recreate removed records. + +A snapshot reads what exists now. It does not recover every intermediate event. Consider a row that existed in Redis on Friday but was deleted from the source on Saturday: Sunday's snapshot contains no row to overwrite that old Redis key. Recovery must account for such stale target data as well as current source rows. + +Use the [pipeline reset guidance](/content/integrate/redis-data-integration/data-pipelines/deploy.md#reset-a-pipeline) with your source's snapshot configuration and target reconciliation plan. Do not manually move an offset forward to silence an error: that can skip changes. Verify recovered inserts, updates, and deletes before relying on the target again. diff --git a/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/_index.md b/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/_index.md index 8a2a4e3b2e..816ffea2ca 100644 --- a/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/_index.md +++ b/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/_index.md @@ -22,6 +22,10 @@ generally, these mechanisms are not switched on by default. RDI's Debezium collector uses these mechanisms for change data capture (CDC), so you must prepare each source database before you can use it with RDI. +Plan how much change history your source needs to retain during interruptions. +See [Understand CDC offsets and source retention](/content/integrate/redis-data-integration/cdc-offsets-and-retention.md) +for a conceptual guide and examples for each source family. + A pipeline can capture from more than one source database, and you must prepare each of them separately. Follow the page for each source's database type. See [Multiple sources in one pipeline](/content/integrate/redis-data-integration/data-pipelines/multiple-sources.md) diff --git a/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md b/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md index 55ae12fc33..3833dbbdf7 100644 --- a/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md +++ b/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md @@ -147,18 +147,11 @@ wal_level = logical You can also set other PostgreSQL streaming replication parameters if you need them. For example, you can use `max_wal_senders` and `max_replication_slots` to increase -the number of connectors that can access the sending server concurrently, -and `wal_keep_size` to limit the maximum WAL size that a replication slot retains. +the number of connectors that can access the sending server concurrently. The [configuration parameters](https://www.postgresql.org/docs/current/runtime-config-replication.html#RUNTIME-CONFIG-REPLICATION-SENDER) documentation describes all the parameters you can use. -PostgreSQL’s logical decoding uses replication slots. These are guaranteed to retain all the WAL -segments that Debezium needs even when Debezium suffers an outage. You should monitor replication -slots carefully to avoid excessive disk consumption and other conditions such as catalog bloat that can arise -if a replication slot is used infrequently. See the PostgreSQL documentation about -[replication slots](https://www.postgresql.org/docs/current/warm-standby.html#STREAMING-REPLICATION-SLOTS) -for more information. If you are using a `synchronous_commit` setting other than `on`, then you should set `wal_writer_delay` to a value of about 10 milliseconds to ensure a low latency for change events. If you don't set this then the default value of about 200 milliseconds will apply. @@ -168,6 +161,56 @@ the default value of about 200 milliseconds will apply. > recommend you consult the [PostgreSQL write-ahead log](https://www.postgresql.org/docs/current/wal-configuration.html) > documentation to get a better understanding. +### Manage WAL retention + +Logical replication slots retain write-ahead log (WAL) needed by their readers, +including during an RDI outage. Retained WAL can grow until it fills the disk. +Configured limits or slot invalidation can also prevent RDI from resuming. +See [Understand CDC offsets and source retention](/content/integrate/redis-data-integration/cdc-offsets-and-retention.md#postgresql-retain-history-without-filling-the-disk) +for the conceptual explanation. + +Choose controls for your PostgreSQL version: + +| PostgreSQL version | Retention controls | +| :-- | :-- | +| 10–12 | `wal_keep_segments` specifies a **minimum** number of retained segments, not a maximum for a replication slot. These versions do not have `max_slot_wal_keep_size`. | +| 13–17 | `wal_keep_size` replaces `wal_keep_segments` and still specifies a **minimum**. `max_slot_wal_keep_size` limits how much WAL a slot can retain at checkpoint time. Its default, `-1`, allows unlimited retention. | +| 18 | The same size controls apply. `idle_replication_slot_timeout` can also invalidate an inactive slot at a checkpoint. Its default, `0`, disables this timeout. | + +See the [PostgreSQL 13 release notes](https://www.postgresql.org/docs/release/13.0/) +and [PostgreSQL 18 replication settings](https://www.postgresql.org/docs/18/runtime-config-replication.html) +for these version differences. Managed services can apply additional limits. + +Check the settings available on your server: + +```sql +SELECT name, setting, unit +FROM pg_settings +WHERE name IN ( + 'wal_keep_segments', 'wal_keep_size', + 'max_slot_wal_keep_size', 'idle_replication_slot_timeout' +); +``` + +Monitor slot state, retained WAL, and free disk together. The following query works +across these versions. Status fields that a version does not provide return `NULL`. + +```sql +SELECT slot_name, active, restart_lsn, confirmed_flush_lsn, + pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn) AS retained_bytes, + to_jsonb(s)->>'wal_status' AS wal_status, + to_jsonb(s)->>'invalidation_reason' AS invalidation_reason +FROM pg_replication_slots AS s +WHERE slot_type = 'logical'; +``` + +`retained_bytes` measures the log distance from the slot's oldest required position; +it is not the total disk usage of the WAL directory. Size retention for peak writes +and the interruption you need to tolerate. A small `wal_keep_size` does not cap +slot storage. A finite slot limit can protect disk space at the cost of losing +required history. Increasing the limit after that history is removed does not +restore it. Do not drop or recreate an RDI slot as a routine disk-cleanup step. + ## Set up permissions The Debezium connector needs a database user that has the REPLICATION and LOGIN roles so that it diff --git a/static/images/rdi/cdc-retention/bookmark.svg b/static/images/rdi/cdc-retention/bookmark.svg new file mode 100644 index 0000000000..d66da463cc --- /dev/null +++ b/static/images/rdi/cdc-retention/bookmark.svg @@ -0,0 +1,43 @@ + +A bookmark needs its history +Time moves from earlier to later. A vertical bookmark marks where capture stopped. In the first panel the required history is retained. In the second panel required history is removed. + + + +A bookmark needs its history +Illustration · no fixed expiration time + +1 History retained: resume succeeds +Capture stops; source writes continue + + +Saved offset + + +Restart + +Change history +Earlier + +Later +Required changes remain available. +RDI resumes from its saved position. + +2 History removed: cannot resume +Capture stops; source writes continue + + +Saved offset + + +Restart + +Removed +Retained +Earlier + +Later +Required changes have been removed. +The offset cannot recreate the gap. + + diff --git a/static/images/rdi/cdc-retention/mongodb.svg b/static/images/rdi/cdc-retention/mongodb.svg new file mode 100644 index 0000000000..c6638dd62d --- /dev/null +++ b/static/images/rdi/cdc-retention/mongodb.svg @@ -0,0 +1,43 @@ + +MongoDB: operation-log history +Time moves from earlier to later. A vertical bookmark marks where capture stopped. In the first panel the required history is retained. In the second panel required history is removed. + + + +MongoDB: operation-log history +Illustration · no fixed expiration time + +1 History retained: resume succeeds +Capture stops; source writes continue + + +Resume token + + +Restart + +Oplog history +Earlier + +Later +The token still has enough history. +The change stream can resume. + +2 History removed: cannot resume +Capture stops; source writes continue + + +Resume token + + +Restart + +Removed +Retained +Earlier + +Later +A write burst replaces old history. +The token alone cannot recover it. + + diff --git a/static/images/rdi/cdc-retention/mysql-mariadb.svg b/static/images/rdi/cdc-retention/mysql-mariadb.svg new file mode 100644 index 0000000000..0bfafc255a --- /dev/null +++ b/static/images/rdi/cdc-retention/mysql-mariadb.svg @@ -0,0 +1,43 @@ + +MySQL and MariaDB: binary logs +Time moves from earlier to later. A vertical bookmark marks where capture stopped. In the first panel the required history is retained. In the second panel required history is removed. + + + +MySQL and MariaDB: binary logs +Illustration · no fixed expiration time + +1 History retained: resume succeeds +Capture stops; source writes continue + + +Saved position + + +Restart + +Binary logs +Earlier + +Later +The required log file is retained. +Capture can read missed changes. + +2 History removed: cannot resume +Capture stops; source writes continue + + +Saved position + + +Restart + +Removed +Retained +Earlier + +Later +The required log file was purged. +Its events are no longer available. + + diff --git a/static/images/rdi/cdc-retention/oracle.svg b/static/images/rdi/cdc-retention/oracle.svg new file mode 100644 index 0000000000..eb975f07da --- /dev/null +++ b/static/images/rdi/cdc-retention/oracle.svg @@ -0,0 +1,43 @@ + +Oracle: required redo history +Time moves from earlier to later. A vertical bookmark marks where capture stopped. In the first panel the required history is retained. In the second panel required history is removed. + + + +Oracle: required redo history +Illustration · no fixed expiration time + +1 History retained: resume succeeds +Capture stops; source writes continue + + +Saved SCN + + +Restart + +Redo history +Earlier + +Later +Required redo is still available. +LogMiner can continue reading. + +2 History removed: cannot resume +Capture stops; source writes continue + + +Saved SCN + + +Restart + +Removed +Retained +Earlier + +Later +A required archive was removed. +Newer redo cannot fill the gap. + + diff --git a/static/images/rdi/cdc-retention/postgresql.svg b/static/images/rdi/cdc-retention/postgresql.svg new file mode 100644 index 0000000000..6be7311628 --- /dev/null +++ b/static/images/rdi/cdc-retention/postgresql.svg @@ -0,0 +1,66 @@ + +PostgreSQL: slots and WAL +Time moves from earlier to later. A vertical bookmark marks where capture stopped. In the first panel the required history is retained. In the second panel required history is removed. A third panel shows a slot invalidated by an idle timeout even if WAL remains on disk. + + + +PostgreSQL: slots and WAL +Illustration · no fixed expiration time + +1 History retained: resume succeeds +Capture stops; source writes continue + + +Saved position + + +Restart + +WAL retained by slot +Earlier + +Later +The slot keeps the required WAL. +Disk usage grows while RDI is stopped. +Disk used + + + + + + +2 History removed: cannot resume +Capture stops; source writes continue + + +Saved position + + +Restart + +Removed +Retained +Earlier + +Later +A slot limit allows WAL removal. +The invalidated slot cannot resume. + +3 Idle timeout: cannot resume +Capture stops; source writes continue + + +Saved position + + +Restart + +WAL may still exist +Earlier + +Later +PostgreSQL 18: enabled idle timeout +invalidates the slot at a checkpoint. +WAL availability alone is not enough. + + diff --git a/static/images/rdi/cdc-retention/snowflake.svg b/static/images/rdi/cdc-retention/snowflake.svg new file mode 100644 index 0000000000..4297f3c124 --- /dev/null +++ b/static/images/rdi/cdc-retention/snowflake.svg @@ -0,0 +1,43 @@ + +Snowflake: stream staleness +Time moves from earlier to later. A vertical bookmark marks where capture stopped. In the first panel the required history is retained. In the second panel required history is removed. + + + +Snowflake: stream staleness +Illustration · no fixed expiration time + +1 History retained: resume succeeds +Capture stops; source writes continue + + +Stream offset + + +Restart + +Source-table history +Earlier + +Later +Required table history remains. +The stream can provide changes. + +2 History removed: cannot resume +Capture stops; source writes continue + + +Stream offset + + +Restart + +Removed +Retained +Earlier + +Later +Required table history is gone. +The stream is stale. + + diff --git a/static/images/rdi/cdc-retention/spanner.svg b/static/images/rdi/cdc-retention/spanner.svg new file mode 100644 index 0000000000..de5e635cee --- /dev/null +++ b/static/images/rdi/cdc-retention/spanner.svg @@ -0,0 +1,44 @@ + +Spanner: change-stream history +Time moves from earlier to later. A vertical bookmark marks where capture stopped. In the first panel the required history is retained. In the second panel required history is removed. + + + +Spanner: change-stream history +Illustration · no fixed expiration time + +1 History retained: resume succeeds +Capture stops; source writes continue + + +Resume point + + +Restart + +Change records +Earlier + +Later +The resume point is still in range. +Streaming can continue. + +2 History removed: cannot resume +Capture stops; source writes continue + + +Resume point + + +Restart + +Removed +Retained +Earlier + +Later +Required change records expired. +Database version retention +does not extend this stream window. + + diff --git a/static/images/rdi/cdc-retention/sql-server.svg b/static/images/rdi/cdc-retention/sql-server.svg new file mode 100644 index 0000000000..f88562720c --- /dev/null +++ b/static/images/rdi/cdc-retention/sql-server.svg @@ -0,0 +1,44 @@ + +SQL Server: CDC change tables +Time moves from earlier to later. A vertical bookmark marks where capture stopped. In the first panel the required history is retained. In the second panel required history is removed. + + + +SQL Server: CDC change tables +Illustration · no fixed expiration time + +1 History retained: resume succeeds +Capture stops; source writes continue + + +Saved LSN + + +Restart + +Captured change rows +Earlier + +Later +Required change rows remain. +RDI continues reading CDC tables. + +2 History removed: cannot resume +Capture stops; source writes continue + + +Saved LSN + + +Restart + +Removed +Retained +Earlier + +Later +Cleanup removed required rows. +Transaction-log backups alone +do not restore this CDC history. + + From 9cfb187b1d414f9b3f5fb2ef2402f938cfa2da19 Mon Sep 17 00:00:00 2001 From: Zdravko Donev Date: Sun, 18 Oct 2026 02:12:19 +0300 Subject: [PATCH 22/76] Restyle CDC retention timelines with Redis docs design --- static/images/rdi/cdc-retention/bookmark.svg | 116 +++++++---- static/images/rdi/cdc-retention/mongodb.svg | 116 +++++++---- .../rdi/cdc-retention/mysql-mariadb.svg | 116 +++++++---- static/images/rdi/cdc-retention/oracle.svg | 116 +++++++---- .../images/rdi/cdc-retention/postgresql.svg | 183 ++++++++++++------ static/images/rdi/cdc-retention/snowflake.svg | 116 +++++++---- static/images/rdi/cdc-retention/spanner.svg | 118 +++++++---- .../images/rdi/cdc-retention/sql-server.svg | 118 +++++++---- 8 files changed, 654 insertions(+), 345 deletions(-) diff --git a/static/images/rdi/cdc-retention/bookmark.svg b/static/images/rdi/cdc-retention/bookmark.svg index d66da463cc..c437554191 100644 --- a/static/images/rdi/cdc-retention/bookmark.svg +++ b/static/images/rdi/cdc-retention/bookmark.svg @@ -1,43 +1,79 @@ - + A bookmark needs its history -Time moves from earlier to later. A vertical bookmark marks where capture stopped. In the first panel the required history is retained. In the second panel required history is removed. - - - -A bookmark needs its history -Illustration · no fixed expiration time - -1 History retained: resume succeeds -Capture stops; source writes continue - - -Saved offset - - -Restart - -Change history -Earlier - -Later -Required changes remain available. -RDI resumes from its saved position. - -2 History removed: cannot resume -Capture stops; source writes continue - - -Saved offset - - -Restart - -Removed -Retained -Earlier - -Later -Required changes have been removed. -The offset cannot recreate the gap. +Illustrative Friday-to-Sunday interruption. A red bookmark marks the saved capture position. Solid dark blocks show retained history; hatched blocks show removed history. The first timeline can resume. The second cannot resume because required history is missing. + + + + + + +RDI / SOURCE HISTORY +A bookmark needs history +Saved position ≠ saved history + + +01 +History retained +Resume succeeds +Saved offset +Restart + + + + + + + + + + + + + + + + + +Change history + +Friday +Sunday +Source keeps writing + + +Required changes remain available. +RDI resumes from its saved position. + + + + +02 +History removed +Cannot resume +Saved offset +Restart + + + + + + + + + + +Removed +Retained + +Friday +Sunday +Source keeps writing + + +Required changes have been removed. +The offset cannot recreate the gap. + + +Illustrative times · not retention defaults diff --git a/static/images/rdi/cdc-retention/mongodb.svg b/static/images/rdi/cdc-retention/mongodb.svg index c6638dd62d..9002b74f6c 100644 --- a/static/images/rdi/cdc-retention/mongodb.svg +++ b/static/images/rdi/cdc-retention/mongodb.svg @@ -1,43 +1,79 @@ - + MongoDB: operation-log history -Time moves from earlier to later. A vertical bookmark marks where capture stopped. In the first panel the required history is retained. In the second panel required history is removed. - - - -MongoDB: operation-log history -Illustration · no fixed expiration time - -1 History retained: resume succeeds -Capture stops; source writes continue - - -Resume token - - -Restart - -Oplog history -Earlier - -Later -The token still has enough history. -The change stream can resume. - -2 History removed: cannot resume -Capture stops; source writes continue - - -Resume token - - -Restart - -Removed -Retained -Earlier - -Later -A write burst replaces old history. -The token alone cannot recover it. +Illustrative Friday-to-Sunday interruption. A red bookmark marks the saved capture position. Solid dark blocks show retained history; hatched blocks show removed history. The first timeline can resume. The second cannot resume because required history is missing. + + + + + + +RDI / SOURCE HISTORY +MongoDB +Resume tokens and oplog history + + +01 +History retained +Resume succeeds +Resume token +Restart + + + + + + + + + + + + + + + + + +Oplog history + +Friday +Sunday +Source keeps writing + + +The token still has enough history. +The change stream can resume. + + + + +02 +History removed +Cannot resume +Resume token +Restart + + + + + + + + + + +Removed +Retained + +Friday +Sunday +Source keeps writing + + +A write burst replaces old history. +The token alone cannot recover it. + + +Illustrative times · not retention defaults diff --git a/static/images/rdi/cdc-retention/mysql-mariadb.svg b/static/images/rdi/cdc-retention/mysql-mariadb.svg index 0bfafc255a..c28d4c838c 100644 --- a/static/images/rdi/cdc-retention/mysql-mariadb.svg +++ b/static/images/rdi/cdc-retention/mysql-mariadb.svg @@ -1,43 +1,79 @@ - + MySQL and MariaDB: binary logs -Time moves from earlier to later. A vertical bookmark marks where capture stopped. In the first panel the required history is retained. In the second panel required history is removed. - - - -MySQL and MariaDB: binary logs -Illustration · no fixed expiration time - -1 History retained: resume succeeds -Capture stops; source writes continue - - -Saved position - - -Restart - -Binary logs -Earlier - -Later -The required log file is retained. -Capture can read missed changes. - -2 History removed: cannot resume -Capture stops; source writes continue - - -Saved position - - -Restart - -Removed -Retained -Earlier - -Later -The required log file was purged. -Its events are no longer available. +Illustrative Friday-to-Sunday interruption. A red bookmark marks the saved capture position. Solid dark blocks show retained history; hatched blocks show removed history. The first timeline can resume. The second cannot resume because required history is missing. + + + + + + +RDI / SOURCE HISTORY +MySQL and MariaDB +Positions in the binary logs + + +01 +History retained +Resume succeeds +Saved position +Restart + + + + + + + + + + + + + + + + + +Binary logs + +Friday +Sunday +Source keeps writing + + +The required log file is retained. +Capture can read missed changes. + + + + +02 +History removed +Cannot resume +Saved position +Restart + + + + + + + + + + +Removed +Retained + +Friday +Sunday +Source keeps writing + + +The required log file was purged. +Its events are no longer available. + + +Illustrative times · not retention defaults diff --git a/static/images/rdi/cdc-retention/oracle.svg b/static/images/rdi/cdc-retention/oracle.svg index eb975f07da..a891148e09 100644 --- a/static/images/rdi/cdc-retention/oracle.svg +++ b/static/images/rdi/cdc-retention/oracle.svg @@ -1,43 +1,79 @@ - + Oracle: required redo history -Time moves from earlier to later. A vertical bookmark marks where capture stopped. In the first panel the required history is retained. In the second panel required history is removed. - - - -Oracle: required redo history -Illustration · no fixed expiration time - -1 History retained: resume succeeds -Capture stops; source writes continue - - -Saved SCN - - -Restart - -Redo history -Earlier - -Later -Required redo is still available. -LogMiner can continue reading. - -2 History removed: cannot resume -Capture stops; source writes continue - - -Saved SCN - - -Restart - -Removed -Retained -Earlier - -Later -A required archive was removed. -Newer redo cannot fill the gap. +Illustrative Friday-to-Sunday interruption. A red bookmark marks the saved capture position. Solid dark blocks show retained history; hatched blocks show removed history. The first timeline can resume. The second cannot resume because required history is missing. + + + + + + +RDI / SOURCE HISTORY +Oracle +SCNs and required redo history + + +01 +History retained +Resume succeeds +Saved SCN +Restart + + + + + + + + + + + + + + + + + +Redo history + +Friday +Sunday +Source keeps writing + + +Required redo is still available. +LogMiner can continue reading. + + + + +02 +History removed +Cannot resume +Saved SCN +Restart + + + + + + + + + + +Removed +Retained + +Friday +Sunday +Source keeps writing + + +A required archive was removed. +Newer redo cannot fill the gap. + + +Illustrative times · not retention defaults diff --git a/static/images/rdi/cdc-retention/postgresql.svg b/static/images/rdi/cdc-retention/postgresql.svg index 6be7311628..06e5693518 100644 --- a/static/images/rdi/cdc-retention/postgresql.svg +++ b/static/images/rdi/cdc-retention/postgresql.svg @@ -1,66 +1,123 @@ - + PostgreSQL: slots and WAL -Time moves from earlier to later. A vertical bookmark marks where capture stopped. In the first panel the required history is retained. In the second panel required history is removed. A third panel shows a slot invalidated by an idle timeout even if WAL remains on disk. - - - -PostgreSQL: slots and WAL -Illustration · no fixed expiration time - -1 History retained: resume succeeds -Capture stops; source writes continue - - -Saved position - - -Restart - -WAL retained by slot -Earlier - -Later -The slot keeps the required WAL. -Disk usage grows while RDI is stopped. -Disk used - - - - - - -2 History removed: cannot resume -Capture stops; source writes continue - - -Saved position - - -Restart - -Removed -Retained -Earlier - -Later -A slot limit allows WAL removal. -The invalidated slot cannot resume. - -3 Idle timeout: cannot resume -Capture stops; source writes continue - - -Saved position - - -Restart - -WAL may still exist -Earlier - -Later -PostgreSQL 18: enabled idle timeout -invalidates the slot at a checkpoint. -WAL availability alone is not enough. +Illustrative Friday-to-Sunday interruption. A red bookmark marks the saved capture position. Solid dark blocks show retained history; hatched blocks show removed history. The first timeline can resume. The second cannot resume because required history is missing. PostgreSQL also shows growing WAL storage and a third timeline where an enabled idle timeout invalidates the slot even if WAL remains. + + + + + + +RDI / SOURCE HISTORY +PostgreSQL +Replication slots and WAL + + +01 +History retained +Resume succeeds +Saved position +Restart + + + + + + + + + + + + + + + + + +WAL retained by slot + +Friday +Sunday +Source keeps writing + + +The slot keeps the required WAL. +Disk usage grows while RDI is stopped. +WAL storage grows + + + + + + + + + + + +02 +History removed +Cannot resume +Saved position +Restart + + + + + + + + + + +Removed +Retained + +Friday +Sunday +Source keeps writing + + +A slot limit allows WAL removal. +The invalidated slot cannot resume. + + + + +03 +Slot invalidated +Cannot resume +Saved position +Restart + + + + + + + + + + + + + + + + + +WAL may still exist + +Friday +Sunday +Source keeps writing + + +PostgreSQL 18: enabled idle timeout +invalidates the slot at a checkpoint. +WAL availability alone is not enough. + + +Illustrative times · not retention defaults diff --git a/static/images/rdi/cdc-retention/snowflake.svg b/static/images/rdi/cdc-retention/snowflake.svg index 4297f3c124..9bfe75fdfb 100644 --- a/static/images/rdi/cdc-retention/snowflake.svg +++ b/static/images/rdi/cdc-retention/snowflake.svg @@ -1,43 +1,79 @@ - + Snowflake: stream staleness -Time moves from earlier to later. A vertical bookmark marks where capture stopped. In the first panel the required history is retained. In the second panel required history is removed. - - - -Snowflake: stream staleness -Illustration · no fixed expiration time - -1 History retained: resume succeeds -Capture stops; source writes continue - - -Stream offset - - -Restart - -Source-table history -Earlier - -Later -Required table history remains. -The stream can provide changes. - -2 History removed: cannot resume -Capture stops; source writes continue - - -Stream offset - - -Restart - -Removed -Retained -Earlier - -Later -Required table history is gone. -The stream is stale. +Illustrative Friday-to-Sunday interruption. A red bookmark marks the saved capture position. Solid dark blocks show retained history; hatched blocks show removed history. The first timeline can resume. The second cannot resume because required history is missing. + + + + + + +RDI / SOURCE HISTORY +Snowflake +Stream offsets and table history · Preview + + +01 +History retained +Resume succeeds +Stream offset +Restart + + + + + + + + + + + + + + + + + +Source-table history + +Friday +Sunday +Source keeps writing + + +Required table history remains. +The stream can provide changes. + + + + +02 +History removed +Cannot resume +Stream offset +Restart + + + + + + + + + + +Removed +Retained + +Friday +Sunday +Source keeps writing + + +Required table history is gone. +The stream is stale. + + +Illustrative times · not retention defaults diff --git a/static/images/rdi/cdc-retention/spanner.svg b/static/images/rdi/cdc-retention/spanner.svg index de5e635cee..b1ed8427cf 100644 --- a/static/images/rdi/cdc-retention/spanner.svg +++ b/static/images/rdi/cdc-retention/spanner.svg @@ -1,44 +1,80 @@ - + Spanner: change-stream history -Time moves from earlier to later. A vertical bookmark marks where capture stopped. In the first panel the required history is retained. In the second panel required history is removed. - - - -Spanner: change-stream history -Illustration · no fixed expiration time - -1 History retained: resume succeeds -Capture stops; source writes continue - - -Resume point - - -Restart - -Change records -Earlier - -Later -The resume point is still in range. -Streaming can continue. - -2 History removed: cannot resume -Capture stops; source writes continue - - -Resume point - - -Restart - -Removed -Retained -Earlier - -Later -Required change records expired. -Database version retention -does not extend this stream window. +Illustrative Friday-to-Sunday interruption. A red bookmark marks the saved capture position. Solid dark blocks show retained history; hatched blocks show removed history. The first timeline can resume. The second cannot resume because required history is missing. + + + + + + +RDI / SOURCE HISTORY +Spanner +Positions in change-stream history + + +01 +History retained +Resume succeeds +Resume point +Restart + + + + + + + + + + + + + + + + + +Change records + +Friday +Sunday +Source keeps writing + + +The resume point is still in range. +Streaming can continue. + + + + +02 +History removed +Cannot resume +Resume point +Restart + + + + + + + + + + +Removed +Retained + +Friday +Sunday +Source keeps writing + + +Required change records expired. +Database version retention +does not extend this stream window. + + +Illustrative times · not retention defaults diff --git a/static/images/rdi/cdc-retention/sql-server.svg b/static/images/rdi/cdc-retention/sql-server.svg index f88562720c..c8bb1e00d8 100644 --- a/static/images/rdi/cdc-retention/sql-server.svg +++ b/static/images/rdi/cdc-retention/sql-server.svg @@ -1,44 +1,80 @@ - + SQL Server: CDC change tables -Time moves from earlier to later. A vertical bookmark marks where capture stopped. In the first panel the required history is retained. In the second panel required history is removed. - - - -SQL Server: CDC change tables -Illustration · no fixed expiration time - -1 History retained: resume succeeds -Capture stops; source writes continue - - -Saved LSN - - -Restart - -Captured change rows -Earlier - -Later -Required change rows remain. -RDI continues reading CDC tables. - -2 History removed: cannot resume -Capture stops; source writes continue - - -Saved LSN - - -Restart - -Removed -Retained -Earlier - -Later -Cleanup removed required rows. -Transaction-log backups alone -do not restore this CDC history. +Illustrative Friday-to-Sunday interruption. A red bookmark marks the saved capture position. Solid dark blocks show retained history; hatched blocks show removed history. The first timeline can resume. The second cannot resume because required history is missing. + + + + + + +RDI / SOURCE HISTORY +SQL Server +Positions in CDC change tables + + +01 +History retained +Resume succeeds +Saved LSN +Restart + + + + + + + + + + + + + + + + + +Captured change rows + +Friday +Sunday +Source keeps writing + + +Required change rows remain. +RDI continues reading CDC tables. + + + + +02 +History removed +Cannot resume +Saved LSN +Restart + + + + + + + + + + +Removed +Retained + +Friday +Sunday +Source keeps writing + + +Cleanup removed required rows. +Transaction-log backups alone +do not restore this CDC history. + + +Illustrative times · not retention defaults From aa7ce3c8e93d0509cd618a1fb44c7f187fc9faf6 Mon Sep 17 00:00:00 2001 From: Zdravko Donev Date: Sun, 18 Oct 2026 02:29:55 +0300 Subject: [PATCH 23/76] Explain source-specific CDC history and state failures --- .../cdc-offsets-and-retention.md | 91 +++++---- .../data-pipelines/prepare-dbs/postgresql.md | 2 +- .../rdi/cdc-retention/mongodb-pre-images.svg | 33 ++++ static/images/rdi/cdc-retention/mongodb.svg | 119 ++++++------ .../rdi/cdc-retention/mysql-mariadb.svg | 138 +++++++------- .../rdi/cdc-retention/oracle-transactions.svg | 46 +++++ static/images/rdi/cdc-retention/oracle.svg | 124 ++++++------- .../cdc-retention/postgresql-slot-state.svg | 39 ++++ .../images/rdi/cdc-retention/postgresql.svg | 172 +++++++----------- static/images/rdi/cdc-retention/snowflake.svg | 130 ++++++------- static/images/rdi/cdc-retention/spanner.svg | 150 ++++++++------- .../images/rdi/cdc-retention/sql-server.svg | 134 +++++++------- 12 files changed, 641 insertions(+), 537 deletions(-) create mode 100644 static/images/rdi/cdc-retention/mongodb-pre-images.svg create mode 100644 static/images/rdi/cdc-retention/oracle-transactions.svg create mode 100644 static/images/rdi/cdc-retention/postgresql-slot-state.svg diff --git a/content/integrate/redis-data-integration/cdc-offsets-and-retention.md b/content/integrate/redis-data-integration/cdc-offsets-and-retention.md index 158a825f71..5af9fcf005 100644 --- a/content/integrate/redis-data-integration/cdc-offsets-and-retention.md +++ b/content/integrate/redis-data-integration/cdc-offsets-and-retention.md @@ -45,88 +45,103 @@ The collector's offset is also different from the processor's **checkpoint**. Th The bookmark idea applies across sources, but the way each database retains history differs. The examples assume capture was working before the interruption. A source restart, replacement, or failover can introduce separate capture-state requirements. -### Oracle: keep the required redo history +### Oracle: keep a complete, readable redo chain -Oracle records changes in redo logs. With LogMiner, RDI reads online and archived redo logs and tracks progress using SCNs. An SCN identifies a position in database history; it is not a timestamp with a fixed lifetime. +Oracle records changes in redo logs. With LogMiner, RDI reads online and archived redo and tracks system change numbers (SCNs). An SCN does not expire as a number. The problem is whether Oracle can provide the history needed from that position. -Online redo logs are reused. Archived logs let a reader reach further back, but only while the required files remain available to the mining process. If cleanup removes a required archive during an outage, the collector can no longer continue from its saved position. Having newer logs does not fill the gap. +Cleanup can remove an old archive, but a gap in the middle also matters. The file containing the saved SCN and the newest files can all exist while a required file between them is missing. With Oracle Real Application Clusters (RAC), mining needs the required redo from all relevant threads. -![Oracle timelines show a saved SCN with all required redo available, then a saved SCN whose required archived redo has been removed.](/images/rdi/cdc-retention/oracle.svg) +![Oracle archive files form a complete readable chain, or a missing middle file prevents continuation even though the saved SCN's file still exists.](/images/rdi/cdc-retention/oracle.svg) {width="560"} -Plan archive retention and storage with your database administrator. Check the archive destination RDI actually reads, not only whether backups exist elsewhere. Long-running transactions can also require older redo than the most recent visible activity suggests. +Unavailable does not always mean deleted. Archiving might be delayed, or the archive destination might be inaccessible. Check availability before choosing a fresh snapshot. See [Oracle LogMiner requirements](https://docs.oracle.com/en/database/oracle/oracle-database/19/sutil/oracle-logminer-utility.html). -XStream uses an Oracle capture process and an outbound server, with its own restart position and capture checkpoints. Preserve the history and state required by that capture process; do not apply a LogMiner error diagnosis unchanged to XStream. See [Prepare Oracle](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/oracle.md), [Oracle LogMiner](https://docs.oracle.com/en/database/oracle/oracle-database/19/sutil/oracle-logminer-utility.html), and [XStream restartability](https://docs.oracle.com/en/database/oracle/oracle-database/19/xstrm/xstream-out-concepts.html). +An outage is not the only risk. A long transaction can hold the restart position in older redo while other transactions continue. Quiet captured activity can also leave the saved offset behind. Monitor the saved position, not just recent events or process uptime. Use the [connector's offset and heartbeat guidance](https://debezium.io/documentation/reference/3.5/connectors/oracle.html#low-change-frequency-offset-management) for your Oracle configuration. -### MySQL and MariaDB: keep the binary logs +![A long-running Oracle transaction holds an older restart position while other transactions commit. Removing that older redo makes a later restart unsafe.](/images/rdi/cdc-retention/oracle-transactions.svg) +{width="560"} + +Two other cases need a different diagnosis. During a snapshot, a “snapshot too old” error concerns [undo history for consistent reads](https://docs.oracle.com/en/database/oracle/oracle-database/19/refrn/UNDO_RETENTION.html), not streaming redo retention. With XStream, recovery also depends on Oracle's capture checkpoints and outbound-server state; a saved client position alone is insufficient. See [Prepare Oracle](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/oracle.md) and [XStream restartability](https://docs.oracle.com/en/database/oracle/oracle-database/19/xstrm/xstream-out-concepts.html). + +### MySQL and MariaDB: preserve files and source continuity -MySQL and MariaDB record changes in binary logs, often called *binlogs*. A resume position identifies the required place in this history. Using transaction identifiers does not preserve the underlying events after their logs are removed. +MySQL and MariaDB record changes in binary logs, often called *binlogs*. A bookmark can name a file and a position inside it. Rotating to a new file is normal. Older files must remain available until capture has read them. -Automatic expiration or a manual purge can remove a file that a disconnected collector still needs. Increasing retention after that purge does not bring the file back. +Expiration or manual purging can remove required events. Resetting the binary logs can remove history immediately, regardless of the planned retention window. The source tables can remain intact throughout. -![Binary-log timelines show successful resume while the required file remains, then failed resume after that file is purged.](/images/rdi/cdc-retention/mysql-mariadb.svg) +![Binary-log file rotation preserves unread events, purging a required file removes them, and switching servers requires checking whether the saved position maps to the new source history.](/images/rdi/cdc-retention/mysql-mariadb.svg) {width="560"} -Set retention for the interruption you need to tolerate and leave enough disk space for peak writes. Coordinate manual cleanup with CDC readers. Managed services can expose different retention controls, so use the instructions for your deployment in [Prepare MySQL and MariaDB](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/my-sql-mariadb.md). The [MySQL purge reference](https://dev.mysql.com/doc/refman/8.0/en/purge-binary-logs.html) and [MariaDB binary-log guide](https://mariadb.com/docs/server/server-management/server-monitoring-logs/binary-log/using-and-maintaining-the-binary-log) describe the underlying cleanup behavior. +A source switch is a separate risk. File positions belong to a particular server's history. Global transaction identifiers (GTIDs) help identify transactions across servers; they do not recreate deleted events. Before a migration or failover, check that the replacement source has the required history and that your RDI configuration can resume from it. -### PostgreSQL: retain history without filling the disk +Size retention and storage for peak writes. Coordinate purges and log resets with capture. See [Prepare MySQL and MariaDB](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/my-sql-mariadb.md), [MySQL purging](https://dev.mysql.com/doc/refman/8.0/en/purge-binary-logs.html), [MySQL log resets](https://dev.mysql.com/doc/refman/8.4/en/reset-binary-logs-and-gtids.html), and [MariaDB transaction positioning](https://mariadb.com/docs/server/ha-and-performance/standard-replication/gtid). -PostgreSQL records changes in its write-ahead log (WAL). RDI reads through a logical replication slot and tracks a log sequence number (LSN). +### PostgreSQL: preserve WAL and slot state -A slot can retain WAL that its reader still needs, even while RDI is stopped. This makes PostgreSQL different from a simple rolling time window: the immediate problem may be growing disk usage rather than removed history. +PostgreSQL records changes in its write-ahead log (WAL). RDI uses a logical replication slot and tracks a log sequence number (LSN). A valid slot can retain needed WAL while RDI is stopped. Disk usage can therefore grow throughout an outage. A full disk is a storage failure, not an expired LSN. -Retention is not unconditional. A configured slot retention limit can allow required WAL to be removed. Newer PostgreSQL versions also support invalidating slots after an idle timeout. A dropped or invalidated slot cannot be treated as a healthy saved bookmark. +Two configured limits can break continuity differently. A finite slot size limit can allow required WAL to be removed at a checkpoint. PostgreSQL 18 can also invalidate an inactive slot at a checkpoint after an enabled idle timeout, even if WAL files remain. That timeout is disabled by default. -![PostgreSQL timelines show WAL retained during an outage with growing storage, required WAL removed after a size limit, and an inactive slot invalidated by a timeout.](/images/rdi/cdc-retention/postgresql.svg) +![PostgreSQL shows a slot retaining growing WAL, a size limit followed by checkpoint-time WAL removal, and a separate idle timer that invalidates the slot even when WAL remains.](/images/rdi/cdc-retention/postgresql.svg) {width="560"} -Monitor slot health, retained WAL, and free disk together. Choose limits that balance recovery time with database storage capacity. Parameter names and available controls differ across PostgreSQL versions; see [Manage WAL retention](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md#manage-wal-retention). +Keeping WAL is only part of recovery. Dropping and recreating a slot does not restore its old decoding state. After failover, the new primary needs usable slot state and the required history. A routine restart of the same database does not itself remove a persistent slot. -The same model applies to Supabase, Neon, AlloyDB for PostgreSQL, Amazon Aurora PostgreSQL, and other managed PostgreSQL services. Check provider-specific slot cleanup, storage, and failover policies rather than assuming that every service retains an idle slot indefinitely. +![An original PostgreSQL primary has saved slot state, while a replacement primary has WAL but lacks that slot state. Reusing the slot name does not restore it.](/images/rdi/cdc-retention/postgresql-slot-state.svg) +{width="560"} -### SQL Server: retain the captured change rows +Monitor slot health, retained WAL, and free disk together. See [Manage WAL retention](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md#manage-wal-retention) for version-specific controls and [PostgreSQL failover guidance](https://www.postgresql.org/docs/18/logical-replication-failover.html) for slot continuity. Supabase, Neon, AlloyDB, Amazon Aurora PostgreSQL, and other managed variants can have additional cleanup, storage, and failover policies. Do not assume they preserve every idle slot. -SQL Server CDC captures changes from the transaction log into change tables. RDI reads those captured changes using an LSN-based position. +### SQL Server: watch both capture stages -The CDC cleanup process removes old entries from the change tables. If RDI returns after its required entries have been cleaned up, keeping transaction-log backups alone does not make those entries available through the CDC reader. +SQL Server first captures transaction-log changes into CDC change tables. RDI then reads those tables using an LSN-based position. If source capture stops, newer changes cannot reach RDI yet and the transaction log can grow. If RDI falls behind while capture continues, CDC cleanup can remove rows it still needs. -![SQL Server timelines show an LSN inside retained CDC change-table history, then outside the available range after cleanup.](/images/rdi/cdc-retention/sql-server.svg) +![SQL Server copies the transaction log into CDC tables before RDI reads them. Cleanup can overtake RDI's saved LSN, or maintenance can delete the old capture instance and its table.](/images/rdi/cdc-retention/sql-server.svg) {width="560"} -Plan CDC change-table retention for consumer interruptions. Monitor both capture progress into those tables and RDI's progress through them. Capture delays and reader delays are different problems. See [Prepare SQL Server](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/sql-server.md) and Microsoft's [CDC administration guide](https://learn.microsoft.com/en-us/sql/relational-databases/track-changes/administer-and-monitor-change-data-capture-sql-server). +History can also disappear during maintenance. Disabling a capture instance deletes its change table, even inside the retention window. A new instance does not recreate those old rows. Follow the [schema-change guidance](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/sql-server.md#drop-the-old-capture-table) before removing an old instance. -### MongoDB: keep enough operation-log history +Monitor source capture and RDI reading separately. Keeping transaction-log backups alone does not make cleaned-up CDC rows available to this reader. See Microsoft's [CDC administration](https://learn.microsoft.com/en-us/sql/relational-databases/track-changes/administer-and-monitor-change-data-capture-sql-server) and [capture-instance removal](https://learn.microsoft.com/en-us/sql/relational-databases/system-stored-procedures/sys-sp-cdc-disable-table-transact-sql) references. -MongoDB change streams provide a **resume token** that identifies where reading can continue. Resuming requires enough history in the replica set's operation log, or *oplog*. +### MongoDB: account for write volume and old values -For a size-limited oplog, a busy day can retain fewer hours of history than a quiet day. A collector that resumes successfully after one weekend might fail after another weekend with a large write burst. +MongoDB change streams provide a **resume token**. Resuming requires enough history in the replica set's operation log, or *oplog*. For a size-limited oplog, a busy day covers fewer hours than a quiet day. The same weekend interruption can therefore succeed once and fail after a large write burst. -![MongoDB timelines show a resume token whose history remains in the oplog, then history overwritten during a write burst.](/images/rdi/cdc-retention/mongodb.svg) +![Two equal-size MongoDB oplogs cover different periods. Quiet writes preserve Friday's token through Sunday; busy writes leave only Sunday's history.](/images/rdi/cdc-retention/mongodb.svg) {width="560"} -Monitor the actual oplog time window. Size it for peak activity, and consider a minimum retention period where available. A minimum retention period can require the oplog to grow beyond its configured size, so account for storage too. Managed-service controls vary. See [Prepare MongoDB](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/mongodb.md) and the [MongoDB oplog guide](https://www.mongodb.com/docs/manual/core/replica-set-oplog/). +Some pipelines also need a document's old values, called a **pre-image**. Custom Redis keys can depend on fields that changed or were deleted. An event can remain in the oplog after its pre-image is gone. A usable resume token does not guarantee that all required old values remain. + +![MongoDB retains a change event in the oplog while its document pre-image is unavailable. This can affect pipelines using custom Redis keys without making the resume token invalid.](/images/rdi/cdc-retention/mongodb-pre-images.svg) +{width="560"} + +Monitor the actual oplog time window and any required pre-images. A minimum oplog retention period can require extra storage. Dropping or renaming watched objects can also [invalidate a stream, depending on its watch scope](https://www.mongodb.com/docs/manual/reference/change-events/invalidate/); this is separate from history expiry. See [Prepare MongoDB](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/mongodb.md), [oplog retention](https://www.mongodb.com/docs/manual/core/replica-set-oplog/), and [pre-image retention](https://www.mongodb.com/docs/manual/changestreams/#change-streams-with-document-pre--and-post-images). ### Spanner: distinguish two retention windows -Spanner change streams retain change records for a configured period. RDI needs its streaming resume point to remain within the available change-stream history. Once those records expire, a later restart cannot read them from the stream. +Spanner has two independent histories. **Database version retention** keeps older row versions for RDI's fixed-timestamp snapshot. A slow or interrupted snapshot can lose access to that timestamp. **Change-stream retention** keeps change records for streaming. Increasing one window does not extend the other. -![Spanner timelines show a streaming resume point within change-stream retention and a point older than the retained records.](/images/rdi/cdc-retention/spanner.svg) +![Spanner compares two separate histories: both can cover the work, snapshot row versions can expire while change records remain, or streaming history can expire after a completed snapshot.](/images/rdi/cdc-retention/spanner.svg) {width="560"} -Spanner also has **database version retention**. That separate setting matters when RDI reads a consistent snapshot at a particular timestamp. Increasing version retention does not substitute for sufficient change-stream retention during streaming. +Changes to the stream can remove history before an outage ends. Reducing retention makes records outside the new window unavailable immediately. Dropping a stream deletes its records; recreating the same name does not restore them. -Plan the change-stream window for interruptions and the version window for snapshot duration. See [Prepare Spanner](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/spanner.md) and Google's [change-stream overview](https://docs.cloud.google.com/spanner/docs/change-streams). +Also preserve what the stream watches. Suspending capture or excluding a required table or operation means those changes were never recorded. Longer retention cannot recover events that were never captured. -### Snowflake: consume changes before the stream becomes stale +Plan version history for snapshot duration and change-stream history for lag, interruptions, and catch-up. See [Prepare Spanner](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/spanner.md), [timestamp limits](https://docs.cloud.google.com/spanner/docs/timestamp-bounds#maximum_timestamp_staleness), and [change-stream management](https://docs.cloud.google.com/spanner/docs/change-streams/manage). -Snowflake streams track an offset against source-table history. The stream does not hold an independent, permanent copy of every change. If the history needed by its offset is no longer retained, the stream becomes **stale**. +### Snowflake: preserve table history and identity -![Snowflake timelines show a usable stream offset with retained table history and a stale stream after required history is no longer retained.](/images/rdi/cdc-retention/snowflake.svg) +Snowflake streams track an offset against source-table history. They do not hold a permanent copy of every change. If the required history is no longer retained, the stream becomes **stale**. Replacing the source table can also break continuity immediately: an old stream does not follow a new table that reuses the same name. + +![Snowflake retains table versions for a stream offset, loses required versions through retention, or replaces the source table with a new object that the old stream does not follow.](/images/rdi/cdc-retention/snowflake.svg) {width="560"} -Monitor stream staleness and the reported `STALE_AFTER` time. Account for table retention and Snowflake's retention-extension rules; do not assume that a stream keeps history indefinitely. Reading a stream with a query alone does not necessarily advance its offset: Snowflake distinguishes querying from consuming changes in a committed transaction. +Monitor staleness and the predicted `STALE_AFTER` time. A plain query does not advance a stream's offset. RDI manages its streams; do not manually consume or replace them to clear an error. + +RDI also checks whether streams have data. Snowflake can protect an empty stream from staleness when that check returns false. This does not protect pending changes during a collector outage. -Snowflake support in RDI is in preview. See [Prepare Snowflake](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/snowflake.md) and Snowflake's [stream retention and staleness guidance](https://docs.snowflake.com/en/user-guide/streams-intro#data-retention-period-and-staleness). +Snowflake support is in preview. See [Prepare Snowflake](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/snowflake.md), [stream staleness](https://docs.snowflake.com/en/user-guide/streams-intro#data-retention-period-and-staleness), and [empty-stream checks](https://docs.snowflake.com/en/sql-reference/functions/system_stream_has_data#usage-notes). ## Plan for the whole interruption @@ -138,7 +153,7 @@ Use a conservative planning budget: This is a planning aid, not a universal database setting. For example, a 48-hour outage budget needs more than 48 hours of retained history if capture was already behind. Catch-up also needs spare processing capacity while the application continues writing. -Before maintenance, check capture progress, available history, and storage. For size-based limits, estimate how much history peak writes produce, not just average writes. Where history is retained by a slot, check how much additional disk an outage could consume. +Before maintenance, check capture progress, available history, and storage. After restoring or replacing a source, also verify that the saved offset still belongs to its history and capture state. For size-based limits, estimate how much history peak writes produce, not just average writes. Where history is retained by a slot, check how much additional disk an outage could consume. Alert while there is still time to act. Combine [RDI monitoring](/content/integrate/redis-data-integration/observability.md) with source-database checks. A running process or a saved offset does not prove that capture is advancing or that its history remains available. Confirm managed-service limits and cleanup policies with the database owner. diff --git a/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md b/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md index 3833dbbdf7..171aa1b28c 100644 --- a/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md +++ b/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md @@ -166,7 +166,7 @@ the default value of about 200 milliseconds will apply. Logical replication slots retain write-ahead log (WAL) needed by their readers, including during an RDI outage. Retained WAL can grow until it fills the disk. Configured limits or slot invalidation can also prevent RDI from resuming. -See [Understand CDC offsets and source retention](/content/integrate/redis-data-integration/cdc-offsets-and-retention.md#postgresql-retain-history-without-filling-the-disk) +See [Understand CDC offsets and source retention](/content/integrate/redis-data-integration/cdc-offsets-and-retention.md#postgresql-preserve-wal-and-slot-state) for the conceptual explanation. Choose controls for your PostgreSQL version: diff --git a/static/images/rdi/cdc-retention/mongodb-pre-images.svg b/static/images/rdi/cdc-retention/mongodb-pre-images.svg new file mode 100644 index 0000000000..aff29561dd --- /dev/null +++ b/static/images/rdi/cdc-retention/mongodb-pre-images.svg @@ -0,0 +1,33 @@ + +MongoDB: A valid token is only one dependency +For custom Redis keys, RDI can need document pre-images. An event can remain in the oplog while its pre-image has expired or was never enabled. Resumable history does not guarantee availability of the old document values. + + + + + + +RDI / SOURCE HISTORY +MongoDB +A valid token is only one dependency + + +01 +Old values have their own lifetime +Custom keys can need document pre-images +Oplog + +Event still available +Pre-images + +Old value no longer available + + + +A usable token does not restore old values. +Plan pre-image retention for custom keys. + + +Illustrations · not retention defaults + + diff --git a/static/images/rdi/cdc-retention/mongodb.svg b/static/images/rdi/cdc-retention/mongodb.svg index 9002b74f6c..a5a8f55b73 100644 --- a/static/images/rdi/cdc-retention/mongodb.svg +++ b/static/images/rdi/cdc-retention/mongodb.svg @@ -1,79 +1,68 @@ - -MongoDB: operation-log history -Illustrative Friday-to-Sunday interruption. A red bookmark marks the saved capture position. Solid dark blocks show retained history; hatched blocks show removed history. The first timeline can resume. The second cannot resume because required history is missing. + +MongoDB: Write volume changes the oplog time window +Two equal-size oplogs at Sunday restart. With quiet writes, history still includes Friday’s resume token. With busy writes, the same capacity covers only Sunday and Friday’s position is no longer available. - - + + - + RDI / SOURCE HISTORY MongoDB -Resume tokens and oplog history +Write volume changes the oplog time window 01 -History retained -Resume succeeds -Resume token -Restart - - - - - - - - - - - - - - - - - -Oplog history - -Friday -Sunday -Source keeps writing - - -The token still has enough history. -The change stream can resume. +Quiet writes +Same example oplog capacity at restart + + + + + + + + + + + + +Friday token + + +Thursday +Sunday + + +Friday’s history still fits; resume is possible. - - + + 02 -History removed -Cannot resume -Resume token -Restart - - - - - - - - - - -Removed -Retained - -Friday -Sunday -Source keeps writing - - -A write burst replaces old history. -The token alone cannot recover it. +A burst of writes +Same example oplog capacity at restart + + + + + + + + + + + + +Friday token: already overwritten +Sunday morning +Sunday now + + +More writes fit fewer hours of history. +The token cannot recover overwritten events. - -Illustrative times · not retention defaults + +Illustrations · not retention defaults diff --git a/static/images/rdi/cdc-retention/mysql-mariadb.svg b/static/images/rdi/cdc-retention/mysql-mariadb.svg index c28d4c838c..2c1d2ea6d8 100644 --- a/static/images/rdi/cdc-retention/mysql-mariadb.svg +++ b/static/images/rdi/cdc-retention/mysql-mariadb.svg @@ -1,79 +1,87 @@ - -MySQL and MariaDB: binary logs -Illustrative Friday-to-Sunday interruption. A red bookmark marks the saved capture position. Solid dark blocks show retained history; hatched blocks show removed history. The first timeline can resume. The second cannot resume because required history is missing. + +MySQL and MariaDB: Files, positions, and source changes +Normal binary-log rotation keeps the saved file position usable while required files remain. Purging the required file removes its events. Switching to another server is a different case: file names and positions belong to that server’s history. - - + + - + RDI / SOURCE HISTORY MySQL and MariaDB -Positions in the binary logs +Files, positions, and source changes 01 -History retained -Resume succeeds -Saved position -Restart - - - - - - - - - - - - - - - - - -Binary logs - -Friday -Sunday -Source keeps writing - - -The required log file is retained. -Capture can read missed changes. +Rotation is normal +A bookmark names a file and a position + +File 041 + + +File 042 + + +File 043 +Position + + + + +Older files still supply the unread events. - - + + 02 -History removed -Cannot resume -Saved position -Restart - - - - - - - - - - -Removed -Retained - -Friday -Sunday -Source keeps writing - - -The required log file was purged. -Its events are no longer available. +A required file is purged +A bookmark names a file and a position + +File 041 + + +File 042 + + +File 043 +Position + + + + +Expiration or purge removes needed events. +A log reset can remove the whole chain. - -Illustrative times · not retention defaults + + + +03 +A different source +Same table rows ≠ same log positions + +Server A + +File 041 + +File 042 + + + + +Server B + +File 007 + +File 008 + + +No automatic mapping + + +Check source history and resume support. +Transaction IDs do not retain lost events. + + +Illustrations · not retention defaults diff --git a/static/images/rdi/cdc-retention/oracle-transactions.svg b/static/images/rdi/cdc-retention/oracle-transactions.svg new file mode 100644 index 0000000000..55891d4275 --- /dev/null +++ b/static/images/rdi/cdc-retention/oracle-transactions.svg @@ -0,0 +1,46 @@ + +Oracle: A running reader can have an old bookmark +A transaction starts on Thursday and stays open through Sunday. Other transactions commit on Friday and Saturday. The restart position can remain in Thursday redo. If cleanup removes that redo before a restart, recent activity does not make the restart safe. + + + + + + +RDI / SOURCE HISTORY +Oracle +A running reader can have an old bookmark + + +01 +An old restart position +Example: a long-running transaction +Transaction stays open + + + + +Thu +Fri +Sat +Sun +Other transactions commit + + + + +Redo available at restart + + + + +Required redo gone +Newer redo remains + + +Recent events do not prove restart safety. + + +Illustrations · not retention defaults + + diff --git a/static/images/rdi/cdc-retention/oracle.svg b/static/images/rdi/cdc-retention/oracle.svg index a891148e09..321e4df596 100644 --- a/static/images/rdi/cdc-retention/oracle.svg +++ b/static/images/rdi/cdc-retention/oracle.svg @@ -1,79 +1,73 @@ - -Oracle: required redo history -Illustrative Friday-to-Sunday interruption. A red bookmark marks the saved capture position. Solid dark blocks show retained history; hatched blocks show removed history. The first timeline can resume. The second cannot resume because required history is missing. + +Oracle: The whole redo chain must be readable +A saved SCN points into archive 101. A complete chain through archives 102 and 103 allows mining. A missing or inaccessible archive 102 interrupts that chain even if archive 101 and newer files exist. - - + + - + RDI / SOURCE HISTORY Oracle -SCNs and required redo history +The whole redo chain must be readable 01 -History retained -Resume succeeds -Saved SCN -Restart - - - - - - - - - - - - - - - - - -Redo history - -Friday -Sunday -Source keeps writing - - -Required redo is still available. -LogMiner can continue reading. +Complete redo chain +Required files must connect to newer redo + +Archive +101 + + +Archive +102 + + +Archive +103 + + +Online +redo +Saved SCN + + + + +Required redo is available for mining. - - + + 02 -History removed -Cannot resume -Saved SCN -Restart - - - - - - - - - - -Removed -Retained - -Friday -Sunday -Source keeps writing - - -A required archive was removed. -Newer redo cannot fill the gap. +A gap in the chain +Required files must connect to newer redo + +Archive +101 + + +Archive +102 + + +Archive +103 + + +Online +redo +Saved SCN + + +Unavailable + + +The saved SCN’s file can still exist. +One missing link stops continuation. - -Illustrative times · not retention defaults + +Illustrations · not retention defaults diff --git a/static/images/rdi/cdc-retention/postgresql-slot-state.svg b/static/images/rdi/cdc-retention/postgresql-slot-state.svg new file mode 100644 index 0000000000..0bd932874c --- /dev/null +++ b/static/images/rdi/cdc-retention/postgresql-slot-state.svg @@ -0,0 +1,39 @@ + +PostgreSQL: Preserve the slot’s state, not just its name +A slot on the original primary has saved decoding state. After a failover without matching slot state, the new primary may have WAL but no usable continuation. Dropping and recreating a slot under the same name also does not restore its old state. + + + + + + +RDI / SOURCE HISTORY +PostgreSQL +Preserve the slot’s state, not just its name + + +01 +Slot state not preserved +Example: failover without matching slot state + +Original primary +Original slot state + +New primary +WAL exists + + + + +Saved state exists + +Slot state missing + + +Reusing a slot name does not restore it. +Check your failover and hosting setup. + + +Illustrations · not retention defaults + + diff --git a/static/images/rdi/cdc-retention/postgresql.svg b/static/images/rdi/cdc-retention/postgresql.svg index 06e5693518..08e7c2d00e 100644 --- a/static/images/rdi/cdc-retention/postgresql.svg +++ b/static/images/rdi/cdc-retention/postgresql.svg @@ -1,123 +1,83 @@ - -PostgreSQL: slots and WAL -Illustrative Friday-to-Sunday interruption. A red bookmark marks the saved capture position. Solid dark blocks show retained history; hatched blocks show removed history. The first timeline can resume. The second cannot resume because required history is missing. PostgreSQL also shows growing WAL storage and a third timeline where an enabled idle timeout invalidates the slot even if WAL remains. + +PostgreSQL: History loss and slot loss are different +A valid replication slot pins WAL while disk use grows. With a configured slot size limit, a checkpoint can allow required WAL removal. Separately, PostgreSQL 18 can invalidate an inactive slot at a checkpoint after its enabled idle timeout, even if WAL files remain. - - + + - + RDI / SOURCE HISTORY PostgreSQL -Replication slots and WAL +History loss and slot loss are different 01 -History retained -Resume succeeds -Saved position -Restart - - - - - - - - - - - - - - - - - -WAL retained by slot - -Friday -Sunday -Source keeps writing - - -The slot keeps the required WAL. -Disk usage grows while RDI is stopped. -WAL storage grows - - - - - - - +A slot holds history +Capture is paused; source writes continue + +Valid slot + + +Required WAL retained +Disk used + + + + + + + +Outage begins +Later + + +Resume remains possible; disk can fill. - - + + 02 -History removed -Cannot resume -Saved position -Restart - - - - - - - - - - -Removed -Retained - -Friday -Sunday -Source keeps writing - - -A slot limit allows WAL removal. -The invalidated slot cannot resume. +A size limit is exceeded +PostgreSQL 13+ · only with a finite slot limit + + + +Configured limit + +Checkpoint + + +Required WAL removed + + +If required WAL is lost, resume fails. - - + + 03 -Slot invalidated -Cannot resume -Saved position -Restart - - - - - - - - - - - - - - - - - -WAL may still exist - -Friday -Sunday -Source keeps writing - - -PostgreSQL 18: enabled idle timeout -invalidates the slot at a checkpoint. -WAL availability alone is not enough. +An idle slot times out +PostgreSQL 18 · disabled by default + +Slot +inactive + + +Idle timeout +passes + + +Checkpoint + +Slot invalid + +WAL may still exist + + +Available WAL alone is not enough. - -Illustrative times · not retention defaults + +Illustrations · not retention defaults diff --git a/static/images/rdi/cdc-retention/snowflake.svg b/static/images/rdi/cdc-retention/snowflake.svg index 9bfe75fdfb..f3c50c58a2 100644 --- a/static/images/rdi/cdc-retention/snowflake.svg +++ b/static/images/rdi/cdc-retention/snowflake.svg @@ -1,79 +1,79 @@ - -Snowflake: stream staleness -Illustrative Friday-to-Sunday interruption. A red bookmark marks the saved capture position. Solid dark blocks show retained history; hatched blocks show removed history. The first timeline can resume. The second cannot resume because required history is missing. + +Snowflake: Table history and identity · Preview +A stream offset references retained versions of its source table. Removing required versions makes the stream stale. Replacing the source table is a separate failure: the old stream does not attach itself to a new table with the same name. - - + + - + RDI / SOURCE HISTORY Snowflake -Stream offsets and table history · Preview +Table history and identity · Preview 01 -History retained -Resume succeeds -Stream offset -Restart - - - - - - - - - - - - - - - - - -Source-table history - -Friday -Sunday -Source keeps writing - - -Required table history remains. -The stream can provide changes. +Table history available +A stream refers to history; it does not copy it + +Version 1 + + +Version 2 + + +Version 3 +Stream offset + + + + +Retained versions can supply changes. - - + + 02 -History removed -Cannot resume -Stream offset -Restart - - - - - - - - - - -Removed -Retained - -Friday -Sunday -Source keeps writing - - -Required table history is gone. -The stream is stale. +Required table history lost +A stream refers to history; it does not copy it + +Version 1 + + +Version 2 + + +Version 3 +Stream offset + + + + +The stream becomes stale. +Increasing retention cannot restore versions. - -Illustrative times · not retention defaults + + + +03 +The source table is replaced +Same name, different table object + +Old table: orders + +New table: orders + + + +Existing stream + + + + + +The old stream does not follow the new table. + + +Illustrations · not retention defaults diff --git a/static/images/rdi/cdc-retention/spanner.svg b/static/images/rdi/cdc-retention/spanner.svg index b1ed8427cf..b3245bb5c2 100644 --- a/static/images/rdi/cdc-retention/spanner.svg +++ b/static/images/rdi/cdc-retention/spanner.svg @@ -1,80 +1,98 @@ - -Spanner: change-stream history -Illustrative Friday-to-Sunday interruption. A red bookmark marks the saved capture position. Solid dark blocks show retained history; hatched blocks show removed history. The first timeline can resume. The second cannot resume because required history is missing. + +Spanner: Two histories, two independent windows +Three cases compare database row versions with change-stream records. Both histories can remain available. A fixed snapshot timestamp can become too old while change records remain. After a snapshot completes, streaming history can expire independently of database version history. - - + + - + RDI / SOURCE HISTORY Spanner -Positions in change-stream history +Two histories, two independent windows 01 -History retained -Resume succeeds -Resume point -Restart - - - - - - - - - - - - - - - - - -Change records - -Friday -Sunday -Source keeps writing - - -The resume point is still in range. -Streaming can continue. +Both windows cover the work +Snapshot reads and streaming need history +Row versions +Change records + + + + + + + + + + + + + + + +Saved point +Now + + +Required history remains in both windows. - - + + 02 -History removed -Cannot resume -Resume point -Restart - - - - - - - - - - -Removed -Retained - -Friday -Sunday -Source keeps writing - - -Required change records expired. -Database version retention -does not extend this stream window. +The snapshot takes too long +Change records do not replace old row versions +Row versions +Change records + + + + + + + + + + + + + + + + +Saved point +Now + + +Old snapshot reads fail; stream still exists. - -Illustrative times · not retention defaults + + + +03 +Streaming falls behind +A finished snapshot cannot replay the gap +Row versions +Change records +Snapshot finished + + + + + + + + + +Saved point +Now + + +The stream no longer covers missed changes. + + +Illustrations · not retention defaults diff --git a/static/images/rdi/cdc-retention/sql-server.svg b/static/images/rdi/cdc-retention/sql-server.svg index c8bb1e00d8..3736e892f0 100644 --- a/static/images/rdi/cdc-retention/sql-server.svg +++ b/static/images/rdi/cdc-retention/sql-server.svg @@ -1,80 +1,82 @@ - -SQL Server: CDC change tables -Illustrative Friday-to-Sunday interruption. A red bookmark marks the saved capture position. Solid dark blocks show retained history; hatched blocks show removed history. The first timeline can resume. The second cannot resume because required history is missing. + +SQL Server: Two readers, then a separate cleanup window +SQL Server capture copies the transaction log to CDC change tables; RDI reads those tables. Retained CDC rows permit resume. Cleanup can move the oldest available LSN past RDI. Disabling a capture instance deletes its old table independently of the retention window. - - + + - + RDI / SOURCE HISTORY SQL Server -Positions in CDC change tables +Two readers, then a separate cleanup window 01 -History retained -Resume succeeds -Saved LSN -Restart - - - - - - - - - - - - - - - - - -Captured change rows - -Friday -Sunday -Source keeps writing - - -Required change rows remain. -RDI continues reading CDC tables. +Two readers, two delays +SQL Server captures first; RDI reads second + +Transaction +log + + +SQL Server +capture + + +CDC +tables + + +RDI +reader + + +Saved LSN + + +Retained CDC rows allow continuation. - - + + 02 -History removed -Cannot resume -Saved LSN -Restart - - - - - - - - - - -Removed -Retained - -Friday -Sunday -Source keeps writing - - -Cleanup removed required rows. -Transaction-log backups alone -do not restore this CDC history. +Cleanup overtakes RDI +The available range moves forward + + +Saved LSN + + + +Oldest available row + +New changes + + +Required rows are outside the CDC range. - -Illustrative times · not retention defaults + + + +03 +Capture instance removed +Maintenance can remove history immediately + +Old CDC table +Deleted + +New CDC table +Starts later + + + +The retention window did not save these rows. + + +A replacement does not restore old rows. + + +Illustrations · not retention defaults From 9ebbfb7afd728b52c742cd3830dc90cf0c4c32d7 Mon Sep 17 00:00:00 2001 From: Zdravko Donev Date: Sun, 18 Oct 2026 02:44:31 +0300 Subject: [PATCH 24/76] Clarify MongoDB change streams and pre-image requirements --- .../data-pipelines/prepare-dbs/mongodb.md | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/mongodb.md b/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/mongodb.md index 02d2308a5c..746924996d 100644 --- a/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/mongodb.md +++ b/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/mongodb.md @@ -33,7 +33,7 @@ The following table summarizes the considerations to prepare a MongoDB database | MongoDB Topology | Replica Set, Sharded Cluster, or MongoDB Atlas | | User Roles | readAnyDatabase, clusterMonitor | | Oplog | Sufficient size for snapshot and streaming | -| Pre/Post Images | Enable on collections **only if using a custom key** | +| Pre/Post Images | Enable when the pipeline needs old document values, including custom keys and set output types | | Connection String | Must include all hosts, replicaSet (if applicable), authSource, credentials | | MongoDB Atlas | **[SSL required](https://debezium.io/documentation/reference/stable/connectors/mongodb.html#mongodb-property-mongodb-ssl-enabled)**, provide root CA as the source's `CACERT` secret in RDI | | MongoDB mTLS | X.509 authentication requires source TLS secrets and MongoDB SSL properties | @@ -48,7 +48,7 @@ complete each step. - [ ] [Configure oplog size](#1-configure-oplog-size) - [ ] [Create a MongoDB user for RDI](#2-create-a-mongodb-user-for-rdi) - [ ] [Connection string format](#3-connection-string-format) -- [ ] [Enable change streams and pre/post images (only if using a custom key)](#4-enable-change-streams-and-prepost-images-only-if-using-a-custom-key) +- [ ] [Enable pre-images when required](#4-enable-pre-images-when-required) - [ ] [MongoDB Atlas specific requirements](#5-mongodb-atlas-specific-requirements) - [ ] [Self-hosted MongoDB mTLS and X.509 authentication](#6-self-hosted-mongodb-mtls-and-x509-authentication) - [ ] [Network and security](#7-network-and-security) @@ -107,13 +107,15 @@ mongodb://${MONGODB_DB_USERNAME}:${MONGODB_DB_PASSWORD}@host:30000 - For Atlas, adjust the connection string accordingly (see example below). - Set `replicaSet` and `authSource` as appropriate for your deployment. -## 4. Enable change streams and pre/post images (only if using a custom key) + -Change Streams are required only if you are using a custom key in your RDI pipeline. Change streams are available by default on replica sets, sharded clusters, and MongoDB Atlas. +## 4. Enable pre-images when required -If your RDI pipeline uses a custom key, you must enable pre- and post-images on the relevant collections to capture the document state before and after updates or deletes. This allows RDI to access both the previous and updated versions of documents during change events, ensuring accurate synchronization. +RDI uses MongoDB change streams for ongoing replication, with both default and custom Redis keys. Change streams are available on replica sets, sharded clusters, and MongoDB Atlas. -Use the command below to enable change streams and pre/post images: +A **pre-image** contains a document's values before an update or deletion. For example, if a custom Redis key uses an email address, changing that address requires RDI to remove the old Redis key and write the new one. The pre-image supplies the old address. A **post-image** contains the document after an insert or update; a deleted document has no post-image. + +Jobs that use custom output keys or set output types require pre-images. Enable document pre- and post-images on the relevant collections with this command. It enables the extra document images, not change streams themselves: ```javascript db.runCommand({ @@ -122,6 +124,10 @@ db.runCommand({ }); ``` +Also set `capture.mode` in the source's advanced settings to a mode ending in `_with_pre_image`, such as `change_streams_update_full_with_pre_image`. Enabling images on the collection alone does not make the collector request them. See the [source configuration reference](/content/integrate/redis-data-integration/reference/config-yaml-reference.md#sourcesadvancedsource-advanced-source-settings). + +Keep required pre-images available for the lag and interruptions you need to tolerate. A change event can remain available after its pre-image is gone. See [MongoDB history and old values](/content/integrate/redis-data-integration/cdc-offsets-and-retention.md#mongodb-account-for-write-volume-and-old-values) and [MongoDB pre- and post-images](https://www.mongodb.com/docs/manual/changestreams/#change-streams-with-document-pre--and-post-images). + ## 5. MongoDB Atlas specific requirements MongoDB Atlas only supports secure connections via SSL. From 8458e460c0e218c56243f96b1747ff69eb020066 Mon Sep 17 00:00:00 2001 From: Zdravko Donev Date: Sun, 18 Oct 2026 02:55:49 +0300 Subject: [PATCH 25/76] Identify Oracle as the example source in CDC introduction --- .../redis-data-integration/cdc-offsets-and-retention.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/integrate/redis-data-integration/cdc-offsets-and-retention.md b/content/integrate/redis-data-integration/cdc-offsets-and-retention.md index 5af9fcf005..343783d349 100644 --- a/content/integrate/redis-data-integration/cdc-offsets-and-retention.md +++ b/content/integrate/redis-data-integration/cdc-offsets-and-retention.md @@ -17,7 +17,7 @@ weight: 31 Redis Data Integration (RDI) can resume after an interruption only if the source still provides the change history and capture state it needs. Saving a position does not preserve that history. -Imagine that RDI stops on Friday and starts again on Sunday. Your application keeps writing to Oracle during those two days. RDI remembers where it stopped, but Oracle has removed some of the redo logs needed to continue. The collector reports that its saved system change number (SCN) is no longer available. +Imagine that RDI stops on Friday and starts again on Sunday. In this example, Oracle is the source database. Your application keeps writing to it during those two days. RDI remembers where it stopped, but Oracle has removed some of the redo logs needed to continue. The collector reports that its saved system change number (SCN) is no longer available. The source tables still exist. RDI has not forgotten its position. What is missing is the history between that position and the present. From 9e5d6201cb62804da2e4ea739aef1b12ad05dbc9 Mon Sep 17 00:00:00 2001 From: Zdravko Donev Date: Sun, 18 Oct 2026 03:04:05 +0300 Subject: [PATCH 26/76] Introduce processing before linking to checkpointing --- .../redis-data-integration/cdc-offsets-and-retention.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/integrate/redis-data-integration/cdc-offsets-and-retention.md b/content/integrate/redis-data-integration/cdc-offsets-and-retention.md index 343783d349..c204291f73 100644 --- a/content/integrate/redis-data-integration/cdc-offsets-and-retention.md +++ b/content/integrate/redis-data-integration/cdc-offsets-and-retention.md @@ -39,7 +39,7 @@ The source continues maintaining its history while RDI is disconnected. If the r *These timelines are illustrations, not database defaults. The important boundary is the oldest history still available when capture resumes.* -The collector's offset is also different from the processor's **checkpoint**. The offset tracks capture from the source. The checkpoint tracks processing of events already captured into RDI. A processor can be behind even while capture is current. If the backlog eventually stops capture through [backpressure](/content/integrate/redis-data-integration/architecture/_index.md#backpressure-mechanism), source retention becomes important again. +RDI handles changes in two steps: the collector reads them from the source, then the processor transforms them and writes them to the target. This page focuses on the collector's bookmark. The processor separately tracks its progress so it can recover after a restart. See [Checkpointing](/content/integrate/redis-data-integration/architecture/_index.md#checkpointing) for that part of the pipeline. ## See how each source keeps history @@ -145,7 +145,7 @@ Snowflake support is in preview. See [Prepare Snowflake](/content/integrate/redi ## Plan for the whole interruption -RDI does not have to be completely down for history to become unavailable. A network failure, lost source access, or sustained capture lag can leave its position behind. A slow target can build a backlog until backpressure stops capture. Source cleanup continues independently. +RDI does not have to be completely down for history to become unavailable. A network failure, lost source access, or sustained capture lag can leave its position behind. A slow target can build a backlog until [backpressure](/content/integrate/redis-data-integration/architecture/_index.md#backpressure-mechanism) stops capture. Source cleanup continues independently. Use a conservative planning budget: From 83cd92a00906c4031a9ed72c1ebd027f907b0365 Mon Sep 17 00:00:00 2001 From: Zdravko Donev Date: Sun, 18 Oct 2026 03:14:15 +0300 Subject: [PATCH 27/76] Clarify source retention controls and replay limits --- .../cdc-offsets-and-retention.md | 34 +++++++++++++++++-- .../data-pipelines/prepare-dbs/postgresql.md | 7 ++++ 2 files changed, 38 insertions(+), 3 deletions(-) diff --git a/content/integrate/redis-data-integration/cdc-offsets-and-retention.md b/content/integrate/redis-data-integration/cdc-offsets-and-retention.md index c204291f73..b360e9a811 100644 --- a/content/integrate/redis-data-integration/cdc-offsets-and-retention.md +++ b/content/integrate/redis-data-integration/cdc-offsets-and-retention.md @@ -43,10 +43,12 @@ RDI handles changes in two steps: the collector reads them from the source, then ## See how each source keeps history -The bookmark idea applies across sources, but the way each database retains history differs. The examples assume capture was working before the interruption. A source restart, replacement, or failover can introduce separate capture-state requirements. +Each source has its own retention rules. These are source-database settings, not an RDI memory limit. A size limit usually means space on disk. The examples assume capture was working before the interruption. ### Oracle: keep a complete, readable redo chain +**Retention controls: cleanup rules and disk space.** There is no single "keep CDC for this many hours" setting. Backup and cleanup jobs can remove archived logs based on their age or backup status. Oracle can also free disk space in its fast recovery area by removing logs that its deletion rules allow. Check both the [archive deletion policy](https://docs.oracle.com/en/database/oracle/oracle-database/19/bradv/configuring-rman-client-basic.html) and any scheduled cleanup jobs. + Oracle records changes in redo logs. With LogMiner, RDI reads online and archived redo and tracks system change numbers (SCNs). An SCN does not expire as a number. The problem is whether Oracle can provide the history needed from that position. Cleanup can remove an old archive, but a gap in the middle also matters. The file containing the saved SCN and the newest files can all exist while a required file between them is missing. With Oracle Real Application Clusters (RAC), mining needs the required redo from all relevant threads. @@ -56,6 +58,8 @@ Cleanup can remove an old archive, but a gap in the middle also matters. The fil Unavailable does not always mean deleted. Archiving might be delayed, or the archive destination might be inaccessible. Check availability before choosing a fresh snapshot. See [Oracle LogMiner requirements](https://docs.oracle.com/en/database/oracle/oracle-database/19/sutil/oracle-logminer-utility.html). +The logs also need matching table definitions. For example, an old change might refer to a column that has since been renamed or removed. LogMiner can fail to read that change if it uses today's table definition. Keeping more logs does not fix this. See the [LogMiner strategy guidance](https://debezium.io/documentation/reference/3.5/connectors/oracle.html#oracle-logminer-mining-strategies). + An outage is not the only risk. A long transaction can hold the restart position in older redo while other transactions continue. Quiet captured activity can also leave the saved offset behind. Monitor the saved position, not just recent events or process uptime. Use the [connector's offset and heartbeat guidance](https://debezium.io/documentation/reference/3.5/connectors/oracle.html#low-change-frequency-offset-management) for your Oracle configuration. ![A long-running Oracle transaction holds an older restart position while other transactions commit. Removing that older redo makes a later restart unsafe.](/images/rdi/cdc-retention/oracle-transactions.svg) @@ -65,6 +69,8 @@ Two other cases need a different diagnosis. During a snapshot, a “snapshot too ### MySQL and MariaDB: preserve files and source continuity +**Retention controls: an age limit, automatic cleanup, and manual or scheduled purges.** [MySQL](https://dev.mysql.com/doc/refman/8.0/en/replication-options-binary-log.html#sysvar_binlog_expire_logs_seconds) and MariaDB can expire old log files. Cleanup happens at events such as server startup or log rotation, not necessarily at the exact expiry time. [MariaDB 11.4 and later](https://mariadb.com/docs/server/server-management/server-monitoring-logs/binary-log/using-and-maintaining-the-binary-log#limiting-the-binlog-size) can also limit the total log size on disk. A limit on the size of each individual file only controls rotation; it does not set the total retention window. + MySQL and MariaDB record changes in binary logs, often called *binlogs*. A bookmark can name a file and a position inside it. Rotating to a new file is normal. Older files must remain available until capture has read them. Expiration or manual purging can remove required events. Resetting the binary logs can remove history immediately, regardless of the planned retention window. The source tables can remain intact throughout. @@ -74,10 +80,14 @@ Expiration or manual purging can remove required events. Resetting the binary lo A source switch is a separate risk. File positions belong to a particular server's history. Global transaction identifiers (GTIDs) help identify transactions across servers; they do not recreate deleted events. Before a migration or failover, check that the replacement source has the required history and that your RDI configuration can resume from it. +Reading older changes also needs the right **schema history**: the record of how tables and columns changed. Rewinding across a column addition, rename, or removal can fail or produce wrong fields if that history does not match. Having the log files is not enough. See [how the connector uses schema history](https://debezium.io/documentation/reference/3.5/connectors/mysql.html#mysql-schema-history-topic). + Size retention and storage for peak writes. Coordinate purges and log resets with capture. See [Prepare MySQL and MariaDB](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/my-sql-mariadb.md), [MySQL purging](https://dev.mysql.com/doc/refman/8.0/en/purge-binary-logs.html), [MySQL log resets](https://dev.mysql.com/doc/refman/8.4/en/reset-binary-logs-and-gtids.html), and [MariaDB transaction positioning](https://mariadb.com/docs/server/ha-and-performance/standard-replication/gtid). ### PostgreSQL: preserve WAL and slot state +**Retention controls: slot progress, an optional disk-size limit, and an optional idle-time limit.** A slot normally keeps the history its reader still needs, rather than keeping a fixed number of days. Size limits and idle timeouts depend on the PostgreSQL version. They take effect at database checkpoints, when PostgreSQL saves recovery state. See [version-specific retention controls](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md#manage-wal-retention). + PostgreSQL records changes in its write-ahead log (WAL). RDI uses a logical replication slot and tracks a log sequence number (LSN). A valid slot can retain needed WAL while RDI is stopped. Disk usage can therefore grow throughout an outage. A full disk is a storage failure, not an expired LSN. Two configured limits can break continuity differently. A finite slot size limit can allow required WAL to be removed at a checkpoint. PostgreSQL 18 can also invalidate an inactive slot at a checkpoint after an enabled idle timeout, even if WAL files remain. That timeout is disabled by default. @@ -87,6 +97,8 @@ Two configured limits can break continuity differently. A finite slot size limit Keeping WAL is only part of recovery. Dropping and recreating a slot does not restore its old decoding state. After failover, the new primary needs usable slot state and the required history. A routine restart of the same database does not itself remove a persistent slot. +Resuming is also different from rewinding. The slot remembers which changes the collector has confirmed as received. Moving RDI's bookmark backward does not make that same slot send changes from before its confirmed position, even if the old WAL files still exist. See [the slot's confirmed position](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md#manage-wal-retention). + ![An original PostgreSQL primary has saved slot state, while a replacement primary has WAL but lacks that slot state. Reusing the slot name does not restore it.](/images/rdi/cdc-retention/postgresql-slot-state.svg) {width="560"} @@ -94,6 +106,8 @@ Monitor slot health, retained WAL, and free disk together. See [Manage WAL reten ### SQL Server: watch both capture stages +**Retention controls: a time window and a scheduled cleanup job.** The window is configured in minutes. The cleanup job removes old rows from the CDC change tables when it runs. This is not a memory-size limit; the tables also need enough disk space. See [CDC cleanup jobs](https://learn.microsoft.com/en-us/sql/relational-databases/track-changes/administer-and-monitor-change-data-capture-sql-server#cleanup-job). + SQL Server first captures transaction-log changes into CDC change tables. RDI then reads those tables using an LSN-based position. If source capture stops, newer changes cannot reach RDI yet and the transaction log can grow. If RDI falls behind while capture continues, CDC cleanup can remove rows it still needs. ![SQL Server copies the transaction log into CDC tables before RDI reads them. Cleanup can overtake RDI's saved LSN, or maintenance can delete the old capture instance and its table.](/images/rdi/cdc-retention/sql-server.svg) @@ -105,6 +119,8 @@ Monitor source capture and RDI reading separately. Keeping transaction-log backu ### MongoDB: account for write volume and old values +**Retention controls: log size, with an optional minimum time in hours.** MongoDB normally removes the oldest entries automatically as its operation log fills. A minimum retention time can make the log grow beyond its configured size. It is a minimum age to preserve, not a deadline to delete. Check the [oplog settings](https://www.mongodb.com/docs/manual/core/replica-set-oplog/) and your hosting service's limits. + MongoDB change streams provide a **resume token**. Resuming requires enough history in the replica set's operation log, or *oplog*. For a size-limited oplog, a busy day covers fewer hours than a quiet day. The same weekend interruption can therefore succeed once and fail after a large write burst. ![Two equal-size MongoDB oplogs cover different periods. Quiet writes preserve Friday's token through Sunday; busy writes leave only Sunday's history.](/images/rdi/cdc-retention/mongodb.svg) @@ -112,13 +128,19 @@ MongoDB change streams provide a **resume token**. Resuming requires enough hist Some pipelines also need a document's old values, called a **pre-image**. Custom Redis keys can depend on fields that changed or were deleted. An event can remain in the oplog after its pre-image is gone. A usable resume token does not guarantee that all required old values remain. +Pre-images have separate cleanup rules: they can expire by age, or disappear when the matching event leaves the oplog. Background cleanup removes them. + ![MongoDB retains a change event in the oplog while its document pre-image is unavailable. This can affect pipelines using custom Redis keys without making the resume token invalid.](/images/rdi/cdc-retention/mongodb-pre-images.svg) {width="560"} -Monitor the actual oplog time window and any required pre-images. A minimum oplog retention period can require extra storage. Dropping or renaming watched objects can also [invalidate a stream, depending on its watch scope](https://www.mongodb.com/docs/manual/reference/change-events/invalidate/); this is separate from history expiry. See [Prepare MongoDB](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/mongodb.md), [oplog retention](https://www.mongodb.com/docs/manual/core/replica-set-oplog/), and [pre-image retention](https://www.mongodb.com/docs/manual/changestreams/#change-streams-with-document-pre--and-post-images). +Reading an old event does not always return the document as it looked then. When the collector uses a [full-document lookup](https://www.mongodb.com/docs/manual/changestreams/#lookup-full-document-for-update-operations), it gets the document's values at lookup time. For example, an older update can return Sunday's values instead of Friday's. + +Monitor the actual oplog time window and any required pre-images. Dropping or renaming watched objects can also [invalidate a stream, depending on its watch scope](https://www.mongodb.com/docs/manual/reference/change-events/invalidate/); this is separate from history expiry. See [Prepare MongoDB](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/mongodb.md) and [pre-image retention](https://www.mongodb.com/docs/manual/changestreams/#change-streams-with-document-pre--and-post-images). ### Spanner: distinguish two retention windows +**Retention controls: two time windows, with automatic cleanup.** You set how long to keep change-stream records and, separately, old database versions. These are durations, not memory or log-size limits. Spanner manages cleanup; you do not schedule a cleanup job. See [change-stream retention](https://docs.cloud.google.com/spanner/docs/change-streams/manage) and [database version retention](https://docs.cloud.google.com/spanner/docs/timestamp-bounds#maximum_timestamp_staleness). + Spanner has two independent histories. **Database version retention** keeps older row versions for RDI's fixed-timestamp snapshot. A slow or interrupted snapshot can lose access to that timestamp. **Change-stream retention** keeps change records for streaming. Increasing one window does not extend the other. ![Spanner compares two separate histories: both can cover the work, snapshot row versions can expire while change records remain, or streaming history can expire after a completed snapshot.](/images/rdi/cdc-retention/spanner.svg) @@ -132,6 +154,8 @@ Plan version history for snapshot duration and change-stream history for lag, in ### Snowflake: preserve table history and identity +**Retention controls: time in days, with automatic management.** Table-history retention and the allowed extension for streams with unread changes determine the window. You do not set a stream memory limit or schedule its cleanup. Extra retained history uses storage. See [stream retention and staleness](https://docs.snowflake.com/en/user-guide/streams-intro#data-retention-period-and-staleness). + Snowflake streams track an offset against source-table history. They do not hold a permanent copy of every change. If the required history is no longer retained, the stream becomes **stale**. Replacing the source table can also break continuity immediately: an old stream does not follow a new table that reuses the same name. ![Snowflake retains table versions for a stream offset, loses required versions through retention, or replaces the source table with a new object that the old stream does not follow.](/images/rdi/cdc-retention/snowflake.svg) @@ -163,6 +187,10 @@ First identify whether required history is missing or temporarily inaccessible. If the database can make the original history available again through a supported procedure, resuming may still be possible. Otherwise, rebuilding from a fresh snapshot establishes a new starting point. Increasing retention protects future history; it cannot recreate removed records. +Changing an offset does not bring back missing history either. An offset can record both a log position and progress within a group of changes. Editing only part of it can skip or repeat changes. Rewinding can send events again; moving forward skips events. Restoring an old offset does not undo changes already written to Redis. + +A schema-history rebuild is different from a data snapshot. It reads current table definitions, not the rows. It cannot recreate older definitions needed to read old changes, or repair data already written with wrong fields. See the [connector's schema-recovery restrictions](https://debezium.io/documentation/reference/3.5/connectors/mysql.html#mysql-property-snapshot-mode). + A snapshot reads what exists now. It does not recover every intermediate event. Consider a row that existed in Redis on Friday but was deleted from the source on Saturday: Sunday's snapshot contains no row to overwrite that old Redis key. Recovery must account for such stale target data as well as current source rows. -Use the [pipeline reset guidance](/content/integrate/redis-data-integration/data-pipelines/deploy.md#reset-a-pipeline) with your source's snapshot configuration and target reconciliation plan. Do not manually move an offset forward to silence an error: that can skip changes. Verify recovered inserts, updates, and deletes before relying on the target again. +Use the [pipeline reset guidance](/content/integrate/redis-data-integration/data-pipelines/deploy.md#reset-a-pipeline) with your source's snapshot configuration and a plan to remove stale target data. Do not move an offset forward just to silence an error. Verify actual field values, inserts, updates, and deletes before relying on the target again; matching row counts alone is not enough. diff --git a/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md b/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md index 171aa1b28c..8adde9c1f9 100644 --- a/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md +++ b/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md @@ -211,6 +211,13 @@ slot storage. A finite slot limit can protect disk space at the cost of losing required history. Increasing the limit after that history is removed does not restore it. Do not drop or recreate an RDI slot as a routine disk-cleanup step. +The slot's `confirmed_flush_lsn` is the position the collector has acknowledged. +PostgreSQL starts logical streaming at the requested position or +`confirmed_flush_lsn`, whichever is later. Moving only RDI's saved offset backward +does not rewind the slot, even if older WAL remains on disk. Keeping unread history +for normal resume is different from replaying already acknowledged changes. +See the [PostgreSQL replication protocol](https://www.postgresql.org/docs/17/protocol-replication.html). + ## Set up permissions The Debezium connector needs a database user that has the REPLICATION and LOGIN roles so that it From d4cdff78caa7b426730fbf33482059aa0a75e12a Mon Sep 17 00:00:00 2001 From: Zdravko Donev Date: Sun, 18 Oct 2026 03:32:17 +0300 Subject: [PATCH 28/76] Clarify snapshot-only restarts and Snowflake recovery --- content/embeds/rdi-snapshot-modes.md | 12 ++++++++++-- content/operate/rc/rdi/view-edit.md | 15 ++++++++++++++- 2 files changed, 24 insertions(+), 3 deletions(-) diff --git a/content/embeds/rdi-snapshot-modes.md b/content/embeds/rdi-snapshot-modes.md index 84d463c4f6..e7a42995c1 100644 --- a/content/embeds/rdi-snapshot-modes.md +++ b/content/embeds/rdi-snapshot-modes.md @@ -19,8 +19,8 @@ while the snapshot is interrupted can remain as stale records in the target beca the new snapshot does not replay their delete events. Use `when_needed` with monitoring and alerting. Recovery can start a long-running -snapshot, which can increase latency and leave stale target records until recovery -finishes. +snapshot, which can increase latency. Missed deletes can leave stale target records +even after the snapshot finishes. #### Missed deletes can leave stale target records {#missed-deletes} @@ -41,6 +41,14 @@ reference. A listed value is not necessarily supported by every connector. | `no_data` | Skip existing rows. Initialize any required schema or source position, then stream changes. | Resume streaming if the position is valid. No data snapshot to recover unavailable history. | Skip existing rows and start streaming according to the connector's rules. | | `never` | Skip the data snapshot where this mode is supported. | Resume according to the connector's rules; no data snapshot recovery. | Connector-specific startup behavior; existing rows are not reloaded. | +{{< note >}} +On Kubernetes deployments, the collector exits after completing an `initial_only` +snapshot. Kubernetes restarts it, but the collector sees that the snapshot is complete +and exits again. This restart loop can make the source appear disconnected in +monitoring. Check snapshot completion separately from collector connectivity; +`initial_only` mode does not capture subsequent changes. +{{< /note >}} + An interrupted initial snapshot can restart when the collector starts again, including with `initial` and `initial_only`. diff --git a/content/operate/rc/rdi/view-edit.md b/content/operate/rc/rdi/view-edit.md index ffee8b653b..ba669f9700 100644 --- a/content/operate/rc/rdi/view-edit.md +++ b/content/operate/rc/rdi/view-edit.md @@ -65,7 +65,20 @@ When extending an older single-source pipeline, preserve the original source nam Stopping one source pauses data capture from that source without stopping the other sources. When you start a source again, its behavior depends on the source type: - Debezium sources follow their [snapshot mode](/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md#choose-a-snapshot-mode) and the availability of their saved position. -- Snowflake sources use RIOT-X and Snowflake Streams. They resume from the saved stream offset when the stream retains the required changes. If the retention window expires while the source is stopped, changes can be lost. Set `clearOffset: true` to restart from the current state. +- Snowflake sources use RIOT-X and Snowflake Streams. They resume from the saved stream offset when the stream retains the required changes. If the retention window expires while the source is stopped, changes can be lost. + +To recover a Snowflake source with an unusable saved position, set `clearOffset` to +`true` in that source's **Collector source properties**, then apply the changes. In the +pipeline configuration, the property is +`sources..advanced.riotx.clearOffset`. The Snowflake `snapshot` property +controls whether the collector reloads existing rows: + +- `INITIAL` (default): Load current source rows, then capture new changes. +- `NEVER`: Skip existing rows and capture only new changes. + +Clearing the offset does not replay missed events or remove stale target records. +After recovery, set `clearOffset` back to `false` so later collector starts preserve +the saved position. See [Edit collector properties](#edit-collector-properties). To resume a source, open its **More actions** menu and select **Start source**, then confirm with **Start source**. A source processes data only while the pipeline is running. See [How does stopping sources affect billing?](/content/operate/rc/rdi/faq.md#stopping-and-billing). From d303acb13bf1f5b00f7c098061e4b8077ff6d997 Mon Sep 17 00:00:00 2001 From: Zdravko Donev Date: Sun, 18 Oct 2026 03:56:52 +0300 Subject: [PATCH 29/76] DOC: align Cloud PostgreSQL publications and replica identity guidance --- .../data-pipelines/data-denormalization.md | 13 ++- .../aws-aurora-rds/aws-aur-pgsql.md | 28 ++++--- .../data-pipelines/prepare-dbs/postgresql.md | 82 ++++++++++++++++--- .../data-pipelines/prepare-dbs/supabase.md | 5 +- .../data-pipelines/data-denormalization.md | 13 ++- .../aws-aurora-rds/aws-aur-pgsql.md | 28 ++++--- .../data-pipelines/prepare-dbs/postgresql.md | 82 ++++++++++++++++--- .../data-pipelines/prepare-dbs/supabase.md | 5 +- content/operate/rc/rdi/define.md | 12 +++ content/operate/rc/rdi/quick-start.md | 4 + content/operate/rc/rdi/setup.md | 30 +++++++ 11 files changed, 244 insertions(+), 58 deletions(-) diff --git a/content/integrate/redis-data-integration/1.19.1/data-pipelines/data-denormalization.md b/content/integrate/redis-data-integration/1.19.1/data-pipelines/data-denormalization.md index 8fa42654e4..eacf01dd3e 100644 --- a/content/integrate/redis-data-integration/1.19.1/data-pipelines/data-denormalization.md +++ b/content/integrate/redis-data-integration/1.19.1/data-pipelines/data-denormalization.md @@ -217,11 +217,10 @@ There are several important things to note when you use nesting: ALTER TABLE REPLICA IDENTITY FULL; ``` - This configuration affects the information written to the write-ahead log (WAL) and whether it is available - for RDI to capture. By default, PostgreSQL only records - modified fields in the log, which means that it might omit the `parent_key`. This can cause incorrect updates to the - Redis key in the destination database. - See the - [Debezium PostgreSQL Connector Documentation](https://debezium.io/documentation/reference/connectors/postgresql.html#postgresql-replica-identity) - for more information about this. + Nesting needs the child's `parent_key` to identify the parent Redis document, including + when the child is deleted. The default replica identity does not provide the full old + row, so a non-key `parent_key` can be unavailable. `FULL` records all old column values + in the write-ahead log (WAL). It can increase WAL volume and reduce performance. + See [Which PostgreSQL row values are available](/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/postgresql.md#understand-which-row-values-are-available) + for the difference between old row values and unchanged large values. - Prior to RDI v1.12.1, there is a known limitation if you change the foreign key of a child object. In that scenario, the child object will be added to the new parent, but the old parent will not be updated. diff --git a/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md b/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md index a7df75cc8a..55def8274a 100644 --- a/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md +++ b/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md @@ -147,18 +147,24 @@ to capture changes. Replace `` with the schema name and `` with the username of the Debezium user. 1. - Configure the PostgreSQL publication. For most RDI deployments, follow the - [PostgreSQL publication and replica identity guidance]({{< relref "/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql" >}}) - and use `publication.autocreate.mode: filtered` when RDI captures selected tables. - - If you intentionally want to publish every table in the database, first check that - every table has a primary key or another replica identity. Then connect to your database - as the `postgres` user and run: + Connect to your database as the `postgres` user and create a publication containing + only the tables RDI should capture. First [check their replica identities](/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/postgresql.md#check-replica-identity-and-choose-a-publication-mode). + Replace the example table names with your selected tables: ```sql - CREATE PUBLICATION dbz_publication FOR ALL TABLES; + CREATE PUBLICATION rdi_publication + FOR TABLE public.customers, public.orders; ``` - Do not use this command without checking the replica identity of all tables. A later - `UPDATE` or `DELETE` on a published table without a replica identity can fail in the - application that writes to PostgreSQL. + Set `publication.name` to `rdi_publication` and `publication.autocreate.mode` to + `disabled` under the source's advanced source settings. On Redis Cloud, enter these + values under [**Collector properties**](/content/operate/rc/rdi/setup.md#postgresql-publications). + Update the publication's table list when you change the tables RDI captures. + + Alternatively, let RDI manage the publication with `publication.autocreate.mode: filtered` + after granting the [required ownership and creation permissions](/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/postgresql.md#set-privileges-for-debezium-to-create-postgresql-publications-with-pgoutput). + + Use `FOR ALL TABLES` only when you intend to publish every current and future table. + An application `UPDATE` or `DELETE` on any published table without replica identity can + fail, even if RDI does not capture that table. If you already created an all-tables + publication, follow [Replace an existing all-tables publication](/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/postgresql.md#replace-an-existing-all-tables-publication). diff --git a/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/postgresql.md b/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/postgresql.md index 8e75a9e2f1..9b348db69a 100644 --- a/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/postgresql.md +++ b/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/postgresql.md @@ -239,10 +239,27 @@ As a fallback, you can record the full old row in the write-ahead log: ALTER TABLE REPLICA IDENTITY FULL; ``` -`REPLICA IDENTITY FULL` can increase write-ahead log volume and reduce performance, so use -it only when a primary key or suitable unique index is not possible. The `keys` setting in -an RDI table definition creates a key for the Redis record. It does not create a PostgreSQL -primary key or replica identity. +`REPLICA IDENTITY FULL` can increase write-ahead log volume and reduce performance. +Use it when no suitable key is available or when RDI needs the full old row, such as +[denormalizing child tables with nesting](/content/integrate/redis-data-integration/1.19.1/data-pipelines/data-denormalization.md#using-nesting). +The `keys` setting in an RDI table definition creates a key for the Redis record. +It does not create a PostgreSQL primary key or replica identity. + +### Understand which row values are available + +Replica identity controls the old row values available to logical decoding for updates +and deletes. With the default identity, PostgreSQL uses the primary key; it does not +provide a complete old row. `REPLICA IDENTITY FULL` records all old column values. + +Update events are not generally limited to changed fields. However, PostgreSQL can omit +unchanged large values stored through The Oversized-Attribute Storage Technique (TOAST) +when they are not part of the replica identity. Debezium represents unavailable values +with a placeholder. With `FULL`, these values are available in the event's old and new row +images. See [Debezium's TOAST guidance](https://debezium.io/documentation/reference/3.0/connectors/postgresql.html#postgresql-toasted-values). + +For example, nesting a child record requires its `parent_key` to identify the parent +Redis document, including when the child is deleted. Set `REPLICA IDENTITY FULL` on +PostgreSQL child tables as required by the [nesting instructions](/content/integrate/redis-data-integration/1.19.1/data-pipelines/data-denormalization.md#using-nesting). ### Choose a publication mode @@ -256,12 +273,17 @@ PostgreSQL publication: | `disabled` | Does not create or change the publication. The publication must already exist. | Use when a database administrator manages the publication. | The `all_tables` mode can publish changes for tables that are not listed in the RDI -pipeline. If an application later updates or deletes a row in a published table without -a replica identity, PostgreSQL can reject that application write. For this reason, +pipeline, including tables created in the future. If an application later updates or +deletes a row in a published table without a replica identity, PostgreSQL can reject +that application write. For this reason, `filtered` is the recommended mode for a pipeline that captures only selected tables. +The `filtered` mode requires permission to create the publication and ownership rights on +the tables it manages. Use a separate publication for each source with different table +filters, so one source does not change another source's publication. + For a customer-managed publication, create an explicit table list and disable automatic -publication management. For example: +publication management. Replace the example table names with the tables you want to capture: ```sql CREATE PUBLICATION rdi_publication @@ -279,6 +301,43 @@ sources: publication.autocreate.mode: disabled ``` +For RDI on Redis Cloud, enter these properties under **Collector properties** in the +source configuration. See [Configure PostgreSQL publications on Redis Cloud](/content/operate/rc/rdi/setup.md#postgresql-publications). +With `disabled`, update the publication's table list yourself when changing the source's +selected tables. Selecting a table in RDI does not add it to a customer-managed publication. + +### Replace an existing all-tables publication + +Changing the RDI table selection does not narrow a `FOR ALL TABLES` publication. +Changing `publication.autocreate.mode` to `filtered` also does not convert it: +PostgreSQL rejects attempts to set, add, or remove individual tables in that publication. + +Have your database administrator plan the change with the owners of every consumer of +the publication: + +1. Check existing publications in the source database: + + ```sql + SELECT pubname, puballtables, pubupdate, pubdelete + FROM pg_publication; + ``` + +1. Identify every consumer of the broad publication and coordinate a maintenance window. +1. Stop the affected RDI sources before replacing the publication. +1. Replace the broad publication with an explicit table-list publication. You can recreate + it under the same name, or create a new publication and update each consumer's publication + name. For RDI, use `publication.autocreate.mode: disabled` for an administrator-managed + publication. On Redis Cloud, contact support if the running source's collector properties + cannot be edited in the console. +1. Remove the old broad publication only after coordinating all consumers. Leaving it in + place still subjects its tables to replica-identity requirements, even if RDI uses a + different publication or is stopped. +1. Resume the affected sources and verify inserts, updates, and deletes reach Redis. + +Keep the replication slots and saved RDI offsets during this change. Do not reset the +pipeline or drop its slot as a publication repair. If other consumers still need the broad +publication, configure replica identity on its affected tables instead of removing it. + ### PostgreSQL publication error FAQ #### I get this PostgreSQL error. What should I do? @@ -299,9 +358,12 @@ configure a suitable replica identity. As a fallback, run: ALTER TABLE REPLICA IDENTITY FULL; ``` -If the table should not be captured, use `publication.autocreate.mode: filtered` or remove -the table from the customer-managed publication. If you changed the RDI table filters or -publication mode, redeploy the pipeline so RDI applies the new configuration. +If the table should not be captured, remove it from an explicit table-list publication or +use `filtered` with the required permissions. For an existing `FOR ALL TABLES` publication, +follow [Replace an existing all-tables publication](#replace-an-existing-all-tables-publication). +Check all publications containing the table: another publication that publishes updates or +deletes can still require replica identity. After changing RDI configuration, redeploy the +pipeline so RDI applies it. ## Set privileges for Debezium to create PostgreSQL publications with `pgoutput` diff --git a/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/supabase.md b/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/supabase.md index 4e86b37b73..c4bacdd049 100644 --- a/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/supabase.md +++ b/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/supabase.md @@ -152,7 +152,10 @@ CREATE PUBLICATION rdi_publication Creating the publication explicitly avoids granting table ownership or broad publication-creation permissions to the RDI role and limits the publication's -table scope. +table scope. Before creating it, [check replica identity](/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/postgresql.md#check-replica-identity-and-choose-a-publication-mode) +for every included table. Updates and deletes on published tables require a usable replica +identity. Update the publication's table list when you add or remove captured tables; +`publication.autocreate.mode: disabled` leaves that list under your control. ## 5. Configure TLS diff --git a/content/integrate/redis-data-integration/data-pipelines/data-denormalization.md b/content/integrate/redis-data-integration/data-pipelines/data-denormalization.md index 6cacf08712..f09e8bf000 100644 --- a/content/integrate/redis-data-integration/data-pipelines/data-denormalization.md +++ b/content/integrate/redis-data-integration/data-pipelines/data-denormalization.md @@ -219,11 +219,10 @@ There are several important things to note when you use nesting: ALTER TABLE REPLICA IDENTITY FULL; ``` - This configuration affects the information written to the write-ahead log (WAL) and whether it is available - for RDI to capture. By default, PostgreSQL only records - modified fields in the log, which means that it might omit the `parent_key`. This can cause incorrect updates to the - Redis key in the destination database. - See the - [Debezium PostgreSQL Connector Documentation](https://debezium.io/documentation/reference/connectors/postgresql.html#postgresql-replica-identity) - for more information about this. + Nesting needs the child's `parent_key` to identify the parent Redis document, including + when the child is deleted. The default replica identity does not provide the full old + row, so a non-key `parent_key` can be unavailable. `FULL` records all old column values + in the write-ahead log (WAL). It can increase WAL volume and reduce performance. + See [Which PostgreSQL row values are available](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md#understand-which-row-values-are-available) + for the difference between old row values and unchanged large values. - Prior to RDI v1.12.1, there is a known limitation if you change the foreign key of a child object. In that scenario, the child object will be added to the new parent, but the old parent will not be updated. diff --git a/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md b/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md index d37ca6c598..5ee748a98d 100644 --- a/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md +++ b/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md @@ -150,18 +150,24 @@ to capture changes. Replace `` with the schema name and `` with the username of the Debezium user. 1. - Configure the PostgreSQL publication. For most RDI deployments, follow the - [PostgreSQL publication and replica identity guidance]({{< relref "/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql" >}}) - and use `publication.autocreate.mode: filtered` when RDI captures selected tables. - - If you intentionally want to publish every table in the database, first check that - every table has a primary key or another replica identity. Then connect to your database - as the `postgres` user and run: + Connect to your database as the `postgres` user and create a publication containing + only the tables RDI should capture. First [check their replica identities](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md#check-replica-identity-and-choose-a-publication-mode). + Replace the example table names with your selected tables: ```sql - CREATE PUBLICATION dbz_publication FOR ALL TABLES; + CREATE PUBLICATION rdi_publication + FOR TABLE public.customers, public.orders; ``` - Do not use this command without checking the replica identity of all tables. A later - `UPDATE` or `DELETE` on a published table without a replica identity can fail in the - application that writes to PostgreSQL. + Set `publication.name` to `rdi_publication` and `publication.autocreate.mode` to + `disabled` under the source's advanced source settings. On Redis Cloud, enter these + values under [**Collector properties**](/content/operate/rc/rdi/setup.md#postgresql-publications). + Update the publication's table list when you change the tables RDI captures. + + Alternatively, let RDI manage the publication with `publication.autocreate.mode: filtered` + after granting the [required ownership and creation permissions](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md#set-privileges-for-debezium-to-create-postgresql-publications-with-pgoutput). + + Use `FOR ALL TABLES` only when you intend to publish every current and future table. + An application `UPDATE` or `DELETE` on any published table without replica identity can + fail, even if RDI does not capture that table. If you already created an all-tables + publication, follow [Replace an existing all-tables publication](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md#replace-an-existing-all-tables-publication). diff --git a/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md b/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md index 8da8fa6966..295b93ae4a 100644 --- a/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md +++ b/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md @@ -242,10 +242,27 @@ As a fallback, you can record the full old row in the write-ahead log: ALTER TABLE REPLICA IDENTITY FULL; ``` -`REPLICA IDENTITY FULL` can increase write-ahead log volume and reduce performance, so use -it only when a primary key or suitable unique index is not possible. The `keys` setting in -an RDI table definition creates a key for the Redis record. It does not create a PostgreSQL -primary key or replica identity. +`REPLICA IDENTITY FULL` can increase write-ahead log volume and reduce performance. +Use it when no suitable key is available or when RDI needs the full old row, such as +[denormalizing child tables with nesting](/content/integrate/redis-data-integration/data-pipelines/data-denormalization.md#using-nesting). +The `keys` setting in an RDI table definition creates a key for the Redis record. +It does not create a PostgreSQL primary key or replica identity. + +### Understand which row values are available + +Replica identity controls the old row values available to logical decoding for updates +and deletes. With the default identity, PostgreSQL uses the primary key; it does not +provide a complete old row. `REPLICA IDENTITY FULL` records all old column values. + +Update events are not generally limited to changed fields. However, PostgreSQL can omit +unchanged large values stored through The Oversized-Attribute Storage Technique (TOAST) +when they are not part of the replica identity. Debezium represents unavailable values +with a placeholder. With `FULL`, these values are available in the event's old and new row +images. See [Debezium's TOAST guidance](https://debezium.io/documentation/reference/3.0/connectors/postgresql.html#postgresql-toasted-values). + +For example, nesting a child record requires its `parent_key` to identify the parent +Redis document, including when the child is deleted. Set `REPLICA IDENTITY FULL` on +PostgreSQL child tables as required by the [nesting instructions](/content/integrate/redis-data-integration/data-pipelines/data-denormalization.md#using-nesting). ### Choose a publication mode @@ -259,12 +276,17 @@ PostgreSQL publication: | `disabled` | Does not create or change the publication. The publication must already exist. | Use when a database administrator manages the publication. | The `all_tables` mode can publish changes for tables that are not listed in the RDI -pipeline. If an application later updates or deletes a row in a published table without -a replica identity, PostgreSQL can reject that application write. For this reason, +pipeline, including tables created in the future. If an application later updates or +deletes a row in a published table without a replica identity, PostgreSQL can reject +that application write. For this reason, `filtered` is the recommended mode for a pipeline that captures only selected tables. +The `filtered` mode requires permission to create the publication and ownership rights on +the tables it manages. Use a separate publication for each source with different table +filters, so one source does not change another source's publication. + For a customer-managed publication, create an explicit table list and disable automatic -publication management. For example: +publication management. Replace the example table names with the tables you want to capture: ```sql CREATE PUBLICATION rdi_publication @@ -282,6 +304,43 @@ sources: publication.autocreate.mode: disabled ``` +For RDI on Redis Cloud, enter these properties under **Collector properties** in the +source configuration. See [Configure PostgreSQL publications on Redis Cloud](/content/operate/rc/rdi/setup.md#postgresql-publications). +With `disabled`, update the publication's table list yourself when changing the source's +selected tables. Selecting a table in RDI does not add it to a customer-managed publication. + +### Replace an existing all-tables publication + +Changing the RDI table selection does not narrow a `FOR ALL TABLES` publication. +Changing `publication.autocreate.mode` to `filtered` also does not convert it: +PostgreSQL rejects attempts to set, add, or remove individual tables in that publication. + +Have your database administrator plan the change with the owners of every consumer of +the publication: + +1. Check existing publications in the source database: + + ```sql + SELECT pubname, puballtables, pubupdate, pubdelete + FROM pg_publication; + ``` + +1. Identify every consumer of the broad publication and coordinate a maintenance window. +1. Stop the affected RDI sources before replacing the publication. +1. Replace the broad publication with an explicit table-list publication. You can recreate + it under the same name, or create a new publication and update each consumer's publication + name. For RDI, use `publication.autocreate.mode: disabled` for an administrator-managed + publication. On Redis Cloud, contact support if the running source's collector properties + cannot be edited in the console. +1. Remove the old broad publication only after coordinating all consumers. Leaving it in + place still subjects its tables to replica-identity requirements, even if RDI uses a + different publication or is stopped. +1. Resume the affected sources and verify inserts, updates, and deletes reach Redis. + +Keep the replication slots and saved RDI offsets during this change. Do not reset the +pipeline or drop its slot as a publication repair. If other consumers still need the broad +publication, configure replica identity on its affected tables instead of removing it. + ### PostgreSQL publication error FAQ #### I get this PostgreSQL error. What should I do? @@ -302,9 +361,12 @@ configure a suitable replica identity. As a fallback, run: ALTER TABLE REPLICA IDENTITY FULL; ``` -If the table should not be captured, use `publication.autocreate.mode: filtered` or remove -the table from the customer-managed publication. If you changed the RDI table filters or -publication mode, redeploy the pipeline so RDI applies the new configuration. +If the table should not be captured, remove it from an explicit table-list publication or +use `filtered` with the required permissions. For an existing `FOR ALL TABLES` publication, +follow [Replace an existing all-tables publication](#replace-an-existing-all-tables-publication). +Check all publications containing the table: another publication that publishes updates or +deletes can still require replica identity. After changing RDI configuration, redeploy the +pipeline so RDI applies it. ## Set privileges for Debezium to create PostgreSQL publications with `pgoutput` diff --git a/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/supabase.md b/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/supabase.md index d559b06ab3..65b66ea348 100644 --- a/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/supabase.md +++ b/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/supabase.md @@ -151,7 +151,10 @@ CREATE PUBLICATION rdi_publication Creating the publication explicitly avoids granting table ownership or broad publication-creation permissions to the RDI role and limits the publication's -table scope. +table scope. Before creating it, [check replica identity](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md#check-replica-identity-and-choose-a-publication-mode) +for every included table. Updates and deletes on published tables require a usable replica +identity. Update the publication's table list when you add or remove captured tables; +`publication.autocreate.mode: disabled` leaves that list under your control. ## 5. Configure TLS diff --git a/content/operate/rc/rdi/define.md b/content/operate/rc/rdi/define.md index 692bd553f7..e74894a4d8 100644 --- a/content/operate/rc/rdi/define.md +++ b/content/operate/rc/rdi/define.md @@ -130,10 +130,18 @@ Use **Collector properties** for additional source and sink settings. These sett ![The advanced properties dialog with separate collector source and sink properties.](/images/rc/rdi/rdi-advanced-properties.png) {width="80%"} +For PostgreSQL, [configure the publication properties](/content/operate/rc/rdi/setup.md#postgresql-publications) +before deployment. Use `disabled` for an administrator-managed table-list publication, or +`filtered` with the required permissions for RDI to manage it. + ## Select data {#dataset} Select each source in the **Sources** list and choose the data to ingest from that source. +For PostgreSQL, this selection does not narrow an existing `FOR ALL TABLES` publication. +If your administrator manages the publication, update its table list to match the selection. +See [PostgreSQL publications](/content/operate/rc/rdi/setup.md#postgresql-publications). + > [!WARNING] > Do not write data directly to keys managed by RDI. Changes from another application can cause transformation failures or data inconsistencies, and RDI can overwrite them. A pipeline reset does not flush the target database. **Flush target database** is a separate action that deletes all target data, including data written outside RDI. See [Data recovery](/content/operate/rc/rdi/faq.md#data-recovery). @@ -156,6 +164,10 @@ Select each source in the **Sources** list and choose the data to ingest from th ![The key control beside the column used to identify a record.](/images/rc/rdi/rdi-unique-key-selected.png) {width="500px"} + For PostgreSQL, choosing a key here does not create a primary key or replica identity + in the source database. [Configure replica identity](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md#check-replica-identity-and-choose-a-publication-mode) + separately for tables that publish updates or deletes. + 1. Repeat for the other sources, then select **Continue**. The available schema, table, and column controls depend on the source type. Each source keeps its own selection, including when two sources contain tables with the same name. diff --git a/content/operate/rc/rdi/quick-start.md b/content/operate/rc/rdi/quick-start.md index 4c578cdc1b..15bf88fc32 100644 --- a/content/operate/rc/rdi/quick-start.md +++ b/content/operate/rc/rdi/quick-start.md @@ -129,6 +129,10 @@ If you lose any outputs, run `terraform output` to view them again. {width="80%"} 1. Under **Source configuration**, enter the `database` and `port` Terraform outputs in the matching fields. +1. Under **Collector properties**, set `publication.autocreate.mode` to `filtered` so RDI + creates a publication for the selected tables. This example uses the `postgres` user, + which has the required permissions. For an administrator-managed publication or an + existing all-tables publication, follow [PostgreSQL publications](/content/operate/rc/rdi/setup.md#postgresql-publications). 1. Select **Test source** and correct any validation errors, then select **Continue**. 1. In **Select data**, select the schemas, tables, and columns to ingest. Review the selected key for each table. diff --git a/content/operate/rc/rdi/setup.md b/content/operate/rc/rdi/setup.md index 7de00a64c0..ae4f7e8b26 100644 --- a/content/operate/rc/rdi/setup.md +++ b/content/operate/rc/rdi/setup.md @@ -33,6 +33,36 @@ See [Prepare source databases](/content/integrate/redis-data-integration/data-pi See the [RDI architecture overview](/content/integrate/redis-data-integration/architecture/_index.md#overview) for more information about CDC. +### PostgreSQL publications {#postgresql-publications} + +For PostgreSQL sources, including databases hosted on EC2, Aurora/RDS, and Supabase, +[check replica identity](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md#check-replica-identity-and-choose-a-publication-mode) +before deploying. Selecting tables in the console does not narrow an existing +`FOR ALL TABLES` publication. That publication includes current and future tables, and +updates or deletes on any published table without replica identity can fail in your +application. + +For a publication managed by your database administrator, create an explicit table list +as shown in the [PostgreSQL preparation guide](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md#choose-a-publication-mode). +In **Configure source**, under **Collector properties**, open **Edit advanced properties** +and enter these source properties: + +| Property | Value | +|:--|:--| +| `publication.name` | The name of the publication created by your administrator | +| `publication.autocreate.mode` | `disabled` | + +Keep the publication's table list aligned with the source's selected tables. With +`disabled`, RDI does not update the publication when you change the selection. + +To let RDI create and manage a publication for selected tables, use `filtered` instead. +This requires [publication creation and table ownership permissions](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md#set-privileges-for-debezium-to-create-postgresql-publications-with-pgoutput). +Use a separate publication for each source with different table selections. + +If a source already uses `FOR ALL TABLES`, follow +[Replace an existing all-tables publication](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md#replace-an-existing-all-tables-publication). +Changing the collector property alone does not convert that publication. + ### Supabase {#supabase} You can use a hosted [Supabase](https://supabase.com/) PostgreSQL database as From 731b4330e7088b64aa8873fdc1c40729e09a5ada Mon Sep 17 00:00:00 2001 From: Zdravko Donev Date: Sun, 18 Oct 2026 04:23:49 +0300 Subject: [PATCH 30/76] RDSC-6239: Clarify Aurora future table grants --- .../prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md | 10 ++++++---- .../prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md | 10 ++++++---- 2 files changed, 12 insertions(+), 8 deletions(-) diff --git a/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md b/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md index 3254e9fd2f..64830bc724 100644 --- a/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md +++ b/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md @@ -134,17 +134,19 @@ to capture changes. GRANT USAGE ON SCHEMA TO ; ``` - Replace `` with the schema name and `` with the username of the Debezium user."> + Replace `` with the schema name and `` with the username of the Debezium user. 1. - Connect to your database as the `postgres` user and grant the new user the necessary privileges for the future: + Connect to your database as the role that creates your application tables, such as your migration role, and grant the Debezium user access to that role's future tables: ```sql - ALTER DEFAULT PRIVILEGES IN SCHEMA + ALTER DEFAULT PRIVILEGES FOR ROLE IN SCHEMA GRANT SELECT ON TABLES TO ; ``` - Replace `` with the schema name and `` with the username of the Debezium user. + Replace `` with the role that runs `CREATE TABLE`, `` with the schema name, and `` with the Debezium user. You can also run this command as an administrator with permission to change that role's default privileges. + + Repeat for each table creator and schema. Defaults set for `postgres` apply only to tables created by `postgres`; a migration role does not inherit those defaults through role membership. This command does not grant access to existing tables, which require the earlier `GRANT SELECT` command. 1. Connect to your database as the `postgres` user and create a publication for the database: diff --git a/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md b/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md index c18fcd7b05..9d0fd2a18c 100644 --- a/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md +++ b/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md @@ -137,17 +137,19 @@ to capture changes. GRANT USAGE ON SCHEMA TO ; ``` - Replace `` with the schema name and `` with the username of the Debezium user."> + Replace `` with the schema name and `` with the username of the Debezium user. 1. - Connect to your database as the `postgres` user and grant the new user the necessary privileges for the future: + Connect to your database as the role that creates your application tables, such as your migration role, and grant the Debezium user access to that role's future tables: ```sql - ALTER DEFAULT PRIVILEGES IN SCHEMA + ALTER DEFAULT PRIVILEGES FOR ROLE IN SCHEMA GRANT SELECT ON TABLES TO ; ``` - Replace `` with the schema name and `` with the username of the Debezium user. + Replace `` with the role that runs `CREATE TABLE`, `` with the schema name, and `` with the Debezium user. You can also run this command as an administrator with permission to change that role's default privileges. + + Repeat for each table creator and schema. Defaults set for `postgres` apply only to tables created by `postgres`; a migration role does not inherit those defaults through role membership. This command does not grant access to existing tables, which require the earlier `GRANT SELECT` command. 1. Connect to your database as the `postgres` user and create a publication for the database: From fa635c7fb6fc7f71209c14a0e23250832f1b4ded Mon Sep 17 00:00:00 2001 From: Zdravko Donev Date: Sun, 18 Oct 2026 04:23:51 +0300 Subject: [PATCH 31/76] RDSC-6239: Correct database PrivateLink setup instructions --- content/operate/rc/rdi/setup.md | 52 +++++++++++++++++++-------------- 1 file changed, 30 insertions(+), 22 deletions(-) diff --git a/content/operate/rc/rdi/setup.md b/content/operate/rc/rdi/setup.md index 7de00a64c0..5203fec6fe 100644 --- a/content/operate/rc/rdi/setup.md +++ b/content/operate/rc/rdi/setup.md @@ -15,6 +15,8 @@ weight: 3 ## Prepare source database +Create the target Redis Cloud Pro database before starting the pipeline. See the [quickstart prerequisites](/content/operate/rc/rdi/quick-start.md#prerequisites). + Prepare every source database before adding it to a pipeline. Each source needs its own change data capture (CDC) configuration, connectivity, and credentials. See [Prerequisites](/content/operate/rc/rdi/_index.md#prerequisites) for a list of supported source databases and database versions. See [Prepare source databases](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/_index.md) to find steps for your database type: @@ -188,7 +190,7 @@ Select the steps for your database setup. To set up PrivateLink for a database hosted on an EC2 instance: -1. [Create a network load balancer](#create-network-load-balancer-ec2) that will route incoming HTTP requests to your database. +1. [Create a network load balancer](#create-network-load-balancer-ec2) that will route incoming TCP connections to your database. 1. [Create an endpoint service](#create-endpoint-service-ec2) through AWS PrivateLink. ### Create network load balancer {#create-network-load-balancer-ec2} @@ -199,29 +201,32 @@ In the [AWS Management Console](https://console.aws.amazon.com/), use the **Serv - **Scheme**: Select **Internal**. - **Load balancer IP address type**: Select **IPv4**. 1. In **Network mapping**, select the VPC and availability zone associated with your source database. -1. In **Security groups**, select the security group associated with your source database, or another security group that allows traffic from PrivateLink and allows traffic to the database. +1. Create a dedicated security group in the source database VPC for the network load balancer. Leave its inbound rules empty for this PrivateLink connection. Allow outbound TCP traffic to the database security group on the database port (`5432` for PostgreSQL unless you configured another port). +1. In the load balancer settings, select this group under **Security groups**. 1. In **Listeners and routing**: - 1. Select **Create target group** to [create a target group](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/create-target-group.html) with the following settings: + 1. Select **Create target group** to [create a target group](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/create-target-group.html) with the following settings: 1. In **Specify group details**: - **Target type**: Select **Instances**. - - **Protocol : Port**: Select **TCP**, and then enter the port number where your database is exposed. + - **Protocol : Port**: Select **TCP**, and then enter the database port (`5432` for PostgreSQL unless you configured another port). - The **IP address type** and **VPC** should be selected already and match the VPC you selected earlier. + 1. In **Health checks**, select **TCP** and use the traffic port. 1. In **Register targets**, select the EC2 instance that runs your source database, enter the port, and select **Include as pending below**. Then, select **Create target group** to create your target group. Return to **Listeners and routing** in the Network Load Balancer setup. 1. Set the following **Listener** properties: - **Protocol**: Select **TCP**. - - **Port**: Enter your source database's port. + - **Port**: Enter your source database's port (`5432` for PostgreSQL unless you configured another port), matching the target group. - **Default action**: Select the target group you created in the previous step. 1. Review the network load balancer settings, and then select **Create load balancer** to continue. 1. After the network load balancer is active, select **Security**. - If you selected the same security group as your source database, you must not enforce security group rules on PrivateLink traffic. Select **Edit** and then deselect **Enforce inbound rules on PrivateLink traffic**, and then select **Save changes**. + Select **Edit**, deselect **Enforce inbound rules on PrivateLink traffic**, and select **Save changes**. This allows accepted PrivateLink connections through the load balancer without an inbound CIDR rule. Configure allowed principals and connection acceptance in the endpoint service steps that follow. -1. Select the security group ID to open the Security group settings. +1. Open the security group attached to your source database. 1. Select **Edit inbound rules**, then **Add rule** to add a rule with the following settings: - - **Type**: Select **HTTP**. - - **Source**: Select **Anywhere - IPv4**. + - **Type**: Select **Custom TCP**. + - **Port range**: Enter the database port (`5432` for PostgreSQL unless you configured another port). + - **Source**: Select the network load balancer security group ID. This rule allows both database connections and the TCP health checks configured on the same port. Select **Save rules** to save your changes. -1. Select **Actions** > **Edit Load Balancer Attributes**. +1. Return to the network load balancer and select **Actions** > **Edit Load Balancer Attributes**. - Under **Load balancer targets selection policy** select **Enable cross-zone load balancing**. Click the **Save Changes** button. @@ -266,36 +271,39 @@ In the [AWS Management Console](https://console.aws.amazon.com/), use the **Serv - **Scheme**: Select **Internal**. - **Load balancer IP address type**: Select **IPv4**. 1. In **Network mapping**, select the VPC and availability zone associated with your source database. -1. In **Security groups**, select the security group associated with your source database, or another security group that allows traffic from PrivateLink and allows traffic to the database. +1. Create a dedicated security group in the source database VPC for the network load balancer. Leave its inbound rules empty for this PrivateLink connection. Allow outbound TCP traffic to the database security group on the database port (`5432` for PostgreSQL unless you configured another port). +1. In the load balancer settings, select this group under **Security groups**. 1. In **Listeners and routing**: - 1. Select **Create target group** to [create a target group](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/create-target-group.html) with the following settings: + 1. Select **Create target group** to [create a target group](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/create-target-group.html) with the following settings: 1. In **Specify group details**: - **Target type**: Select **IP Addresses**. - - **Protocol : Port**: Select **TCP**, and then enter the port number where your database is exposed. + - **Protocol : Port**: Select **TCP**, and then enter the database port (`5432` for PostgreSQL unless you configured another port). - The **IP address type** and **VPC** should be selected already and match the VPC you selected earlier. - 1. In **Register targets**, enter the static IP address of your database, enter the port, and select **Include as pending below**. Then, select **Create target group** to create your target group. Return to **Listeners and routing** in the Network Load Balancer setup. + 1. In **Health checks**, select **TCP** and use the traffic port. + 1. In **Register targets**, enter the currently resolved private IPv4 address of your database, enter the port, and select **Include as pending below**. Then, select **Create target group** to create your target group. Return to **Listeners and routing** in the Network Load Balancer setup. - To get the static IP address of your database, run the following command on an EC2 instance in the same VPC as the database: + To get the current private IPv4 address of your database, run the following command on an EC2 instance in the same VPC as the database: ```sh $ nslookup ``` - Replace `` with the endpoint of your RDS or Aurora database. + Replace `` with the writer endpoint of your RDS or Aurora database. The address can change after failover or maintenance. [Configure the existing failover Lambda](#setup-lambda-function) to update the target group when the writer address changes. 1. Set the following **Listener** properties: - **Protocol**: Select **TCP**. - - **Port**: Enter your source database's port. + - **Port**: Enter your source database's port (`5432` for PostgreSQL unless you configured another port), matching the target group. - **Default action**: Select the target group you created in the previous step. 1. Review the network load balancer settings, and then select **Create load balancer** to continue. 1. After the network load balancer is active, select **Security**. - If you selected the same security group as your source database, you must not enforce security group rules on PrivateLink traffic. Select **Edit** and then deselect **Enforce inbound rules on PrivateLink traffic**, and then select **Save changes**. + Select **Edit**, deselect **Enforce inbound rules on PrivateLink traffic**, and select **Save changes**. This allows accepted PrivateLink connections through the load balancer without an inbound CIDR rule. Configure allowed principals and connection acceptance in the endpoint service steps that follow. -1. Select the security group ID to open the Security group settings. +1. Open the security group attached to your source database. 1. Select **Edit inbound rules**, then **Add rule** to add a rule with the following settings: - - **Type**: Select **HTTP**. - - **Source**: Select **Anywhere - IPv4**. + - **Type**: Select **Custom TCP**. + - **Port range**: Enter the database port (`5432` for PostgreSQL unless you configured another port). + - **Source**: Select the network load balancer security group ID. This rule allows both database connections and the TCP health checks configured on the same port. Select **Save rules** to save your changes. -1. Select **Actions** > **Edit Load Balancer Attributes**. +1. Return to the network load balancer and select **Actions** > **Edit Load Balancer Attributes**. - Under **Load balancer targets selection policy** select **Enable cross-zone load balancing**. Click the **Save Changes** button. From 01a1a436ad427aa45197cc927c68bee480890617 Mon Sep 17 00:00:00 2001 From: Zdravko Donev Date: Sun, 18 Oct 2026 04:28:53 +0300 Subject: [PATCH 32/76] RDSC-6238: Check usable replica identity indexes in PostgreSQL guidance --- .../data-pipelines/prepare-dbs/postgresql.md | 14 +++++++++++--- .../data-pipelines/prepare-dbs/postgresql.md | 14 +++++++++++--- 2 files changed, 22 insertions(+), 6 deletions(-) diff --git a/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/postgresql.md b/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/postgresql.md index 9b348db69a..4bc8677325 100644 --- a/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/postgresql.md +++ b/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/postgresql.md @@ -214,19 +214,27 @@ WHERE c.relkind = 'r' AND ( c.relreplident = 'n' OR ( - c.relreplident = 'd' + c.relreplident IN ('d', 'i') AND NOT EXISTS ( SELECT 1 FROM pg_index AS i WHERE i.indrelid = c.oid - AND i.indisprimary + AND i.indislive + AND i.indisvalid + AND i.indisunique + AND i.indimmediate + AND i.indpred IS NULL + AND CASE WHEN c.relreplident = 'd' + THEN i.indisprimary + ELSE i.indisreplident + END ) ) ) ORDER BY n.nspname, c.relname; ``` -If this query returns a table that will be published, add a primary key where possible. +If this query returns a table that will be published, add a primary key that is not `DEFERRABLE` where possible. A deferrable primary key cannot serve as the replica identity. You can also configure a suitable unique index as the replica identity: ```sql diff --git a/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md b/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md index 295b93ae4a..1ae6637135 100644 --- a/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md +++ b/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md @@ -217,19 +217,27 @@ WHERE c.relkind = 'r' AND ( c.relreplident = 'n' OR ( - c.relreplident = 'd' + c.relreplident IN ('d', 'i') AND NOT EXISTS ( SELECT 1 FROM pg_index AS i WHERE i.indrelid = c.oid - AND i.indisprimary + AND i.indislive + AND i.indisvalid + AND i.indisunique + AND i.indimmediate + AND i.indpred IS NULL + AND CASE WHEN c.relreplident = 'd' + THEN i.indisprimary + ELSE i.indisreplident + END ) ) ) ORDER BY n.nspname, c.relname; ``` -If this query returns a table that will be published, add a primary key where possible. +If this query returns a table that will be published, add a primary key that is not `DEFERRABLE` where possible. A deferrable primary key cannot serve as the replica identity. You can also configure a suitable unique index as the replica identity: ```sql From e3bb66e5111044388259155e2eb0713dc1fae824 Mon Sep 17 00:00:00 2001 From: Zdravko Donev Date: Sun, 18 Oct 2026 04:42:56 +0300 Subject: [PATCH 33/76] Correct Snowflake timestamp recovery and retention guidance --- .../cdc-offsets-and-retention.md | 18 ++-- static/images/rdi/cdc-retention/snowflake.svg | 95 +++++++++---------- 2 files changed, 57 insertions(+), 56 deletions(-) diff --git a/content/integrate/redis-data-integration/cdc-offsets-and-retention.md b/content/integrate/redis-data-integration/cdc-offsets-and-retention.md index b360e9a811..ea8f671660 100644 --- a/content/integrate/redis-data-integration/cdc-offsets-and-retention.md +++ b/content/integrate/redis-data-integration/cdc-offsets-and-retention.md @@ -47,7 +47,7 @@ Each source has its own retention rules. These are source-database settings, not ### Oracle: keep a complete, readable redo chain -**Retention controls: cleanup rules and disk space.** There is no single "keep CDC for this many hours" setting. Backup and cleanup jobs can remove archived logs based on their age or backup status. Oracle can also free disk space in its fast recovery area by removing logs that its deletion rules allow. Check both the [archive deletion policy](https://docs.oracle.com/en/database/oracle/oracle-database/19/bradv/configuring-rman-client-basic.html) and any scheduled cleanup jobs. +**Retention controls: cleanup rules and disk space.** There is no single "keep CDC for this many hours" setting. Backup and cleanup jobs can remove archived logs based on their age or backup status. Oracle can also free disk space in its fast recovery area by removing logs that its deletion rules allow. Check both the [archive deletion policy](https://docs.oracle.com/en/database/oracle/oracle-database/19/bradv/configuring-rman-client-basic.html#GUID-FA142760-8ACB-43DA-939C-8407618D2D0F) and any scheduled cleanup jobs. Oracle records changes in redo logs. With LogMiner, RDI reads online and archived redo and tracks system change numbers (SCNs). An SCN does not expire as a number. The problem is whether Oracle can provide the history needed from that position. @@ -69,7 +69,7 @@ Two other cases need a different diagnosis. During a snapshot, a “snapshot too ### MySQL and MariaDB: preserve files and source continuity -**Retention controls: an age limit, automatic cleanup, and manual or scheduled purges.** [MySQL](https://dev.mysql.com/doc/refman/8.0/en/replication-options-binary-log.html#sysvar_binlog_expire_logs_seconds) and MariaDB can expire old log files. Cleanup happens at events such as server startup or log rotation, not necessarily at the exact expiry time. [MariaDB 11.4 and later](https://mariadb.com/docs/server/server-management/server-monitoring-logs/binary-log/using-and-maintaining-the-binary-log#limiting-the-binlog-size) can also limit the total log size on disk. A limit on the size of each individual file only controls rotation; it does not set the total retention window. +**Retention controls: an age limit, automatic cleanup, and manual or scheduled purges.** [MySQL](https://dev.mysql.com/doc/refman/8.0/en/replication-options-binary-log.html#sysvar_binlog_expire_logs_seconds) and MariaDB can expire old log files. Cleanup can happen at server startup or when the binary log is flushed, including during rotation. It does not necessarily happen at the exact expiry time. In MySQL 8.0.29 and later, `binlog_expire_logs_auto_purge` can disable automatic cleanup even when an age limit is set. [MariaDB 11.4 and later](https://mariadb.com/docs/server/server-management/server-monitoring-logs/binary-log/using-and-maintaining-the-binary-log#limiting-the-binlog-size) can also limit the total log size on disk. A limit on each individual file's size only controls rotation; it does not set the total retention window. MySQL and MariaDB record changes in binary logs, often called *binlogs*. A bookmark can name a file and a position inside it. Rotating to a new file is normal. Older files must remain available until capture has read them. @@ -143,6 +143,8 @@ Monitor the actual oplog time window and any required pre-images. Dropping or re Spanner has two independent histories. **Database version retention** keeps older row versions for RDI's fixed-timestamp snapshot. A slow or interrupted snapshot can lose access to that timestamp. **Change-stream retention** keeps change records for streaming. Increasing one window does not extend the other. +Keep both retention durations in [RDI's source configuration](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/spanner.md#5-configure-rdi-for-spanner) in sync with the database settings. RDI uses its configured durations for retention checks; changing only the database settings does not update RDI. + ![Spanner compares two separate histories: both can cover the work, snapshot row versions can expire while change records remain, or streaming history can expire after a completed snapshot.](/images/rdi/cdc-retention/spanner.svg) {width="560"} @@ -154,18 +156,20 @@ Plan version history for snapshot duration and change-stream history for lag, in ### Snowflake: preserve table history and identity -**Retention controls: time in days, with automatic management.** Table-history retention and the allowed extension for streams with unread changes determine the window. You do not set a stream memory limit or schedule its cleanup. Extra retained history uses storage. See [stream retention and staleness](https://docs.snowflake.com/en/user-guide/streams-intro#data-retention-period-and-staleness). +**Retention controls: time in days, with automatic management.** On the source table, `DATA_RETENTION_TIME_IN_DAYS` sets how long Snowflake keeps old table versions through its **Time Travel** feature. `MAX_DATA_EXTENSION_TIME_IN_DAYS` limits how far Snowflake can extend retention to protect streams; its default is 14 days. Check both settings and any extension held by RDI's existing stream. Do not assume that every saved timestamp has 14 days of protection. These controls use storage, not a stream memory limit. See [Snowflake retention rules](https://docs.snowflake.com/en/user-guide/streams-intro#data-retention-period-and-staleness). -Snowflake streams track an offset against source-table history. They do not hold a permanent copy of every change. If the required history is no longer retained, the stream becomes **stale**. Replacing the source table can also break continuity immediately: an old stream does not follow a new table that reuses the same name. +RDI's bookmark is a saved table timestamp. When RDI resumes, it creates a stream at that timestamp to read the changes since then. It also recreates the stream when fetching later batches. If the required history is gone, creating the stream at the saved timestamp fails. Creating a new stream does not bring back old history. -![Snowflake retains table versions for a stream offset, loses required versions through retention, or replaces the source table with a new object that the old stream does not follow.](/images/rdi/cdc-retention/snowflake.svg) +![Snowflake lets RDI recreate a stream when its saved timestamp is still available. Creation fails when that history has expired or when the timestamp predates a replacement table's history.](/images/rdi/cdc-retention/snowflake.svg) {width="560"} -Monitor staleness and the predicted `STALE_AFTER` time. A plain query does not advance a stream's offset. RDI manages its streams; do not manually consume or replace them to clear an error. +Replacing a table is another risk. RDI creates streams using the configured table name, which can now refer to the replacement. The name alone does not prove that the original history is still available. If the saved timestamp predates the replacement table's history, RDI cannot create the stream at that time. + +Monitor how old RDI's saved timestamp is against the history still available for that table. The existing stream's `STALE_AFTER` value is a useful warning, but it does not prove that RDI can recreate a stream from its saved timestamp. RDI manages its streams; do not manually consume or replace them to clear an error. RDI also checks whether streams have data. Snowflake can protect an empty stream from staleness when that check returns false. This does not protect pending changes during a collector outage. -Snowflake support is in preview. See [Prepare Snowflake](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/snowflake.md), [stream staleness](https://docs.snowflake.com/en/user-guide/streams-intro#data-retention-period-and-staleness), and [empty-stream checks](https://docs.snowflake.com/en/sql-reference/functions/system_stream_has_data#usage-notes). +Snowflake support is in private preview. See [Prepare Snowflake](/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/snowflake.md), [creating a stream at a past timestamp](https://docs.snowflake.com/en/sql-reference/sql/create-stream#usage-notes), and [empty-stream checks](https://docs.snowflake.com/en/sql-reference/functions/system_stream_has_data#usage-notes). ## Plan for the whole interruption diff --git a/static/images/rdi/cdc-retention/snowflake.svg b/static/images/rdi/cdc-retention/snowflake.svg index f3c50c58a2..b0f1f3ebc7 100644 --- a/static/images/rdi/cdc-retention/snowflake.svg +++ b/static/images/rdi/cdc-retention/snowflake.svg @@ -1,79 +1,76 @@ - -Snowflake: Table history and identity · Preview -A stream offset references retained versions of its source table. Removing required versions makes the stream stale. Replacing the source table is a separate failure: the old stream does not attach itself to a new table with the same name. + +Snowflake: Saved timestamps · Private preview +RDI saves a table timestamp and recreates a stream from it when fetching changes. Available table history allows resume. Expired history or a saved timestamp before a replacement table existed prevents stream creation. - - + + RDI / SOURCE HISTORY Snowflake -Table history and identity · Preview +Saved timestamps · Private preview 01 -Table history available -A stream refers to history; it does not copy it +Saved time is available +Resume needs history from the saved time -Version 1 +Friday -Version 2 +Saturday -Version 3 -Stream offset - - - - -Retained versions can supply changes. +Sunday +Saved time + + +RDI creates a stream at Friday’s timestamp. + + +The new stream can read missed changes. - + 02 -Required table history lost -A stream refers to history; it does not copy it +Saved time is too old +Resume needs history from the saved time -Version 1 +Friday -Version 2 +Saturday -Version 3 -Stream offset - - - - -The stream becomes stale. -Increasing retention cannot restore versions. +Sunday +Saved time + + +Friday’s table history is no longer available. + + +Stream creation at the saved time fails. - + 03 -The source table is replaced -Same name, different table object - -Old table: orders - -New table: orders - - - -Existing stream - - - - - -The old stream does not follow the new table. +The table is replaced +Same name does not preserve old history + +Old table: orders + +New table: orders + +RDI uses the table now at that name. +If its history starts after the saved time: + + +Stream creation at the saved time fails. - -Illustrations · not retention defaults + +Illustrations · not retention defaults From 41fd16975b2a3761ee2d4e446845da98ff82c801 Mon Sep 17 00:00:00 2001 From: zviRosenfeldRedis Date: Sun, 18 Oct 2026 15:40:08 +0300 Subject: [PATCH 34/76] Document that GET /v1/actions/bdb does not return completed actions --- .../operate/rs/references/rest-api/requests/actions/bdb.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/operate/rs/references/rest-api/requests/actions/bdb.md b/content/operate/rs/references/rest-api/requests/actions/bdb.md index e0e9757e2c..5131c89fee 100644 --- a/content/operate/rs/references/rest-api/requests/actions/bdb.md +++ b/content/operate/rs/references/rest-api/requests/actions/bdb.md @@ -21,7 +21,7 @@ weight: $weight GET /v1/actions/bdb/{bdb_uid} ``` -Get the status of all currently executing, pending, or completed state-machine-related actions for a specific database. This API tracks short-lived API requests that return `action_uid`. +Get the status of all currently executing or pending state-machine-related actions for a specific database. Completed actions are not returned, so an empty array means no actions are running for the database. This API tracks short-lived API requests that return `action_uid`. #### Required permissions @@ -45,7 +45,7 @@ GET /actions/bdb/1 ### Response {#get-response} -Returns an array of JSON objects with attributes from [actions]({{< relref "/operate/rs/references/rest-api/objects/action" >}}) and [state machines]({{< relref "/operate/rs/references/rest-api/objects/state-machine" >}}). +Returns an array of JSON objects with attributes from [actions]({{< relref "/operate/rs/references/rest-api/objects/action" >}}) and [state machines]({{< relref "/operate/rs/references/rest-api/objects/state-machine" >}}). Completed actions and state machines are not included. Each action contains the following attributes: `name`, `action_uid`, `status`, and `progress`. From e53775dde0ee7e39ebe327e7ffff583caa9fbd65 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Mon, 19 Oct 2026 06:16:54 -0700 Subject: [PATCH 35/76] update rc openapi spec (#4291) * Update content/operate/rc/api/api-reference/openapi.json * Update content/operate/rc/api/api-reference/openapi.json * Update content/operate/rc/api/api-reference/openapi.json --------- Co-authored-by: redisdocsapp[bot] <177626021+redisdocsapp[bot]@users.noreply.github.com> --- .../operate/rc/api/api-reference/openapi.json | 138 +++++++++--------- 1 file changed, 69 insertions(+), 69 deletions(-) diff --git a/content/operate/rc/api/api-reference/openapi.json b/content/operate/rc/api/api-reference/openapi.json index 70c2f2d172..9120d4ffe6 100644 --- a/content/operate/rc/api/api-reference/openapi.json +++ b/content/operate/rc/api/api-reference/openapi.json @@ -96,7 +96,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_2", + "operationId": "proxyDataIntegrationWorkspace_3", "parameters": [ { "name": "subscriptionId", @@ -166,7 +166,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_3", + "operationId": "proxyDataIntegrationWorkspace", "parameters": [ { "name": "subscriptionId", @@ -308,7 +308,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace", + "operationId": "proxyDataIntegrationWorkspace_4", "parameters": [ { "name": "subscriptionId", @@ -379,7 +379,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_4", + "operationId": "proxyDataIntegrationWorkspace_2", "parameters": [ { "name": "subscriptionId", @@ -452,7 +452,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_7", + "operationId": "proxyDataIntegrationWorkspace_8", "parameters": [ { "name": "subscriptionId", @@ -522,7 +522,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_8", + "operationId": "proxyDataIntegrationWorkspace_5", "parameters": [ { "name": "subscriptionId", @@ -664,7 +664,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_5", + "operationId": "proxyDataIntegrationWorkspace_9", "parameters": [ { "name": "subscriptionId", @@ -735,7 +735,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_9", + "operationId": "proxyDataIntegrationWorkspace_7", "parameters": [ { "name": "subscriptionId", @@ -8350,15 +8350,14 @@ "412": { "description": "Precondition Failed - Feature flag for this flow is off" }, - "200": { - "description": "Memory stores returned.", - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/MemoryStoreListResponse" - } - } - } + "401": { + "description": "Unauthorized - Authentication failed for requested resource" + }, + "404": { + "description": "Not Found - The resource you were trying to reach was not found or does not exist" + }, + "403": { + "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" }, "424": { "description": "Failed Dependency - The memory store's database is unavailable or out of memory, e.g. /errors/resource-unavailable or /errors/database-out-of-memory. When the error originates from the Agent Memory service the body is an application/problem+json document (MemoryProblemDetails) carrying the service's error type and detail", @@ -8370,14 +8369,15 @@ } } }, - "403": { - "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" - }, - "401": { - "description": "Unauthorized - Authentication failed for requested resource" - }, - "404": { - "description": "Not Found - The resource you were trying to reach was not found or does not exist" + "200": { + "description": "Memory stores returned.", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/MemoryStoreListResponse" + } + } + } }, "408": { "description": "Request Timeout - The server didn't receive a complete request message within the server's allotted timeout period" @@ -8490,6 +8490,15 @@ "412": { "description": "Precondition Failed - Feature flag for this flow is off" }, + "401": { + "description": "Unauthorized - Authentication failed for requested resource" + }, + "404": { + "description": "Not Found - The resource you were trying to reach was not found or does not exist" + }, + "403": { + "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" + }, "424": { "description": "Failed Dependency - The memory store's database is unavailable or out of memory, e.g. /errors/resource-unavailable or /errors/database-out-of-memory. When the error originates from the Agent Memory service the body is an application/problem+json document (MemoryProblemDetails) carrying the service's error type and detail", "content": { @@ -8500,15 +8509,6 @@ } } }, - "403": { - "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" - }, - "401": { - "description": "Unauthorized - Authentication failed for requested resource" - }, - "404": { - "description": "Not Found - The resource you were trying to reach was not found or does not exist" - }, "408": { "description": "Request Timeout - The server didn't receive a complete request message within the server's allotted timeout period" }, @@ -8565,15 +8565,11 @@ "412": { "description": "Precondition Failed - Feature flag for this flow is off" }, - "424": { - "description": "Failed Dependency - The memory store's database is unavailable or out of memory, e.g. /errors/resource-unavailable or /errors/database-out-of-memory. When the error originates from the Agent Memory service the body is an application/problem+json document (MemoryProblemDetails) carrying the service's error type and detail", - "content": { - "application/problem+json": { - "schema": { - "$ref": "#/components/schemas/MemoryProblemDetails" - } - } - } + "401": { + "description": "Unauthorized - Authentication failed for requested resource" + }, + "404": { + "description": "Not Found - The resource you were trying to reach was not found or does not exist" }, "201": { "description": "API key created. The generated key value is returned once in the response body.", @@ -8588,11 +8584,15 @@ "403": { "description": "Forbidden - Not allowed to create requested resource (primarily due to not active resource)" }, - "401": { - "description": "Unauthorized - Authentication failed for requested resource" - }, - "404": { - "description": "Not Found - The resource you were trying to reach was not found or does not exist" + "424": { + "description": "Failed Dependency - The memory store's database is unavailable or out of memory, e.g. /errors/resource-unavailable or /errors/database-out-of-memory. When the error originates from the Agent Memory service the body is an application/problem+json document (MemoryProblemDetails) carrying the service's error type and detail", + "content": { + "application/problem+json": { + "schema": { + "$ref": "#/components/schemas/MemoryProblemDetails" + } + } + } }, "408": { "description": "Request Timeout - The server didn't receive a complete request message within the server's allotted timeout period" @@ -10079,6 +10079,15 @@ "412": { "description": "Precondition Failed - Feature flag for this flow is off" }, + "401": { + "description": "Unauthorized - Authentication failed for requested resource" + }, + "404": { + "description": "Not Found - The resource you were trying to reach was not found or does not exist" + }, + "403": { + "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" + }, "200": { "description": "Memory store returned.", "content": { @@ -10099,15 +10108,6 @@ } } }, - "403": { - "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" - }, - "401": { - "description": "Unauthorized - Authentication failed for requested resource" - }, - "404": { - "description": "Not Found - The resource you were trying to reach was not found or does not exist" - }, "429": { "description": "Too Many Requests - Too many resources are concurrently created / updated / deleted in the account. Please re-submit API requests after resource changes are completed)" }, @@ -12804,6 +12804,18 @@ "412": { "description": "Precondition Failed - Feature flag for this flow is off" }, + "204": { + "description": "API key deleted." + }, + "401": { + "description": "Unauthorized - Authentication failed for requested resource" + }, + "404": { + "description": "Not Found - The resource you were trying to reach was not found or does not exist" + }, + "403": { + "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" + }, "424": { "description": "Failed Dependency - The memory store's database is unavailable or out of memory, e.g. /errors/resource-unavailable or /errors/database-out-of-memory. When the error originates from the Agent Memory service the body is an application/problem+json document (MemoryProblemDetails) carrying the service's error type and detail", "content": { @@ -12814,18 +12826,6 @@ } } }, - "403": { - "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" - }, - "204": { - "description": "API key deleted." - }, - "401": { - "description": "Unauthorized - Authentication failed for requested resource" - }, - "404": { - "description": "Not Found - The resource you were trying to reach was not found or does not exist" - }, "408": { "description": "Request Timeout - The server didn't receive a complete request message within the server's allotted timeout period" }, From cc5703f8cb78e13e226e065aae98bd8d5df0a81a Mon Sep 17 00:00:00 2001 From: Zdravko Donev Date: Mon, 19 Oct 2026 20:35:19 +0300 Subject: [PATCH 36/76] RDSC-6239: Cover future table grants for all PostgreSQL setups --- .../aws-aurora-rds/aws-aur-pgsql.md | 8 ++++---- .../data-pipelines/prepare-dbs/postgresql.md | 20 +++++++++++++++++++ .../aws-aurora-rds/aws-aur-pgsql.md | 8 ++++---- .../data-pipelines/prepare-dbs/postgresql.md | 20 +++++++++++++++++++ 4 files changed, 48 insertions(+), 8 deletions(-) diff --git a/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md b/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md index 64830bc724..ccfd4495ca 100644 --- a/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md +++ b/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md @@ -137,16 +137,16 @@ to capture changes. Replace `` with the schema name and `` with the username of the Debezium user. 1. - Connect to your database as the role that creates your application tables, such as your migration role, and grant the Debezium user access to that role's future tables: + Connect as the role that creates your application tables and grant the Debezium user access to future tables: ```sql - ALTER DEFAULT PRIVILEGES FOR ROLE IN SCHEMA + ALTER DEFAULT PRIVILEGES FOR ROLE IN SCHEMA GRANT SELECT ON TABLES TO ; ``` - Replace `` with the role that runs `CREATE TABLE`, `` with the schema name, and `` with the Debezium user. You can also run this command as an administrator with permission to change that role's default privileges. + Replace `` with the role that runs `CREATE TABLE`, often your migration role. Replace `` with the schema name and `` with the Debezium user. An administrator with permission to change that role's defaults can also run this command. - Repeat for each table creator and schema. Defaults set for `postgres` apply only to tables created by `postgres`; a migration role does not inherit those defaults through role membership. This command does not grant access to existing tables, which require the earlier `GRANT SELECT` command. + Repeat for each table creator and schema. This command covers future tables only. Keep the grants for existing tables. Running it as `postgres` without `FOR ROLE` covers only tables created by `postgres`. 1. Connect to your database as the `postgres` user and create a publication for the database: diff --git a/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/postgresql.md b/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/postgresql.md index 31ccc7149f..ab8ab63ad0 100644 --- a/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/postgresql.md +++ b/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/postgresql.md @@ -180,6 +180,26 @@ needs to perform replications, but you might also need to grant other permission CREATE ROLE REPLICATION LOGIN; ``` +### Grant access to existing and future tables + +The Debezium user needs read access to snapshot table data. If it does not already have this access, run these commands as an administrator with permission to grant access: + +```sql +GRANT USAGE ON SCHEMA TO ; +GRANT SELECT ON ALL TABLES IN SCHEMA TO ; +``` + +These grants cover existing tables. To grant access to future tables, connect as the role that creates your application tables and run: + +```sql +ALTER DEFAULT PRIVILEGES FOR ROLE IN SCHEMA + GRANT SELECT ON TABLES TO ; +``` + +Replace `` with the role that runs `CREATE TABLE`, often your migration role. Replace `` with the schema name and `` with the Debezium user. An administrator with permission to change that role's defaults can also run this command. + +Repeat for each table creator and schema. Running the command as `postgres` without `FOR ROLE` covers only tables created by `postgres`. Read access does not add a table to a publication or the RDI pipeline. + ## Set privileges for Debezium to create PostgreSQL publications with `pgoutput` The Debezium user needs specific permissions to work with the `pgoutput` plug-in. diff --git a/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md b/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md index 9d0fd2a18c..1f2cbed2e5 100644 --- a/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md +++ b/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md @@ -140,16 +140,16 @@ to capture changes. Replace `` with the schema name and `` with the username of the Debezium user. 1. - Connect to your database as the role that creates your application tables, such as your migration role, and grant the Debezium user access to that role's future tables: + Connect as the role that creates your application tables and grant the Debezium user access to future tables: ```sql - ALTER DEFAULT PRIVILEGES FOR ROLE IN SCHEMA + ALTER DEFAULT PRIVILEGES FOR ROLE IN SCHEMA GRANT SELECT ON TABLES TO ; ``` - Replace `` with the role that runs `CREATE TABLE`, `` with the schema name, and `` with the Debezium user. You can also run this command as an administrator with permission to change that role's default privileges. + Replace `` with the role that runs `CREATE TABLE`, often your migration role. Replace `` with the schema name and `` with the Debezium user. An administrator with permission to change that role's defaults can also run this command. - Repeat for each table creator and schema. Defaults set for `postgres` apply only to tables created by `postgres`; a migration role does not inherit those defaults through role membership. This command does not grant access to existing tables, which require the earlier `GRANT SELECT` command. + Repeat for each table creator and schema. This command covers future tables only. Keep the grants for existing tables. Running it as `postgres` without `FOR ROLE` covers only tables created by `postgres`. 1. Connect to your database as the `postgres` user and create a publication for the database: diff --git a/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md b/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md index 55ae12fc33..767673b3d6 100644 --- a/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md +++ b/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md @@ -183,6 +183,26 @@ needs to perform replications, but you might also need to grant other permission CREATE ROLE REPLICATION LOGIN; ``` +### Grant access to existing and future tables + +The Debezium user needs read access to snapshot table data. If it does not already have this access, run these commands as an administrator with permission to grant access: + +```sql +GRANT USAGE ON SCHEMA TO ; +GRANT SELECT ON ALL TABLES IN SCHEMA TO ; +``` + +These grants cover existing tables. To grant access to future tables, connect as the role that creates your application tables and run: + +```sql +ALTER DEFAULT PRIVILEGES FOR ROLE IN SCHEMA + GRANT SELECT ON TABLES TO ; +``` + +Replace `` with the role that runs `CREATE TABLE`, often your migration role. Replace `` with the schema name and `` with the Debezium user. An administrator with permission to change that role's defaults can also run this command. + +Repeat for each table creator and schema. Running the command as `postgres` without `FOR ROLE` covers only tables created by `postgres`. Read access does not add a table to a publication or the RDI pipeline. + ## Set privileges for Debezium to create PostgreSQL publications with `pgoutput` The Debezium user needs specific permissions to work with the `pgoutput` plug-in. From 70591390def5a8efcc8d11b15c4c9d5d03dce6a7 Mon Sep 17 00:00:00 2001 From: Zdravko Donev Date: Mon, 19 Oct 2026 20:38:06 +0300 Subject: [PATCH 37/76] RDSC-6239: Explain future table grants with three roles in docs --- .../aws-aurora-rds/aws-aur-pgsql.md | 16 ++++++++++++++-- .../data-pipelines/prepare-dbs/postgresql.md | 18 ++++++++++++++++-- .../aws-aurora-rds/aws-aur-pgsql.md | 16 ++++++++++++++-- .../data-pipelines/prepare-dbs/postgresql.md | 18 ++++++++++++++++-- 4 files changed, 60 insertions(+), 8 deletions(-) diff --git a/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md b/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md index ccfd4495ca..f858fa8925 100644 --- a/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md +++ b/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md @@ -137,7 +137,19 @@ to capture changes. Replace `` with the schema name and `` with the username of the Debezium user. 1. - Connect as the role that creates your application tables and grant the Debezium user access to future tables: + For example, your database setup might use three roles: + + - `postgres`: the administrator setting up RDI. + - `app_migrations`: the role that creates application tables. + - `debezium`: the user RDI uses to read those tables. + + These names are examples. Use your existing roles. + + The administrator can grant Debezium access to existing tables, so the initial snapshot works. Later, `app_migrations` creates a new table. Defaults set for `postgres` do not give Debezium access to that table, so reading it can fail. + + Set the defaults for the role that creates the tables. In this example, use `app_migrations` as `` and `debezium` as the Debezium user. + + Connect as the table creator and grant the Debezium user access to future tables: ```sql ALTER DEFAULT PRIVILEGES FOR ROLE IN SCHEMA @@ -146,7 +158,7 @@ to capture changes. Replace `` with the role that runs `CREATE TABLE`, often your migration role. Replace `` with the schema name and `` with the Debezium user. An administrator with permission to change that role's defaults can also run this command. - Repeat for each table creator and schema. This command covers future tables only. Keep the grants for existing tables. Running it as `postgres` without `FOR ROLE` covers only tables created by `postgres`. + Repeat for each table creator and schema. This command covers future tables only. Keep the grants for existing tables. 1. Connect to your database as the `postgres` user and create a publication for the database: diff --git a/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/postgresql.md b/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/postgresql.md index ab8ab63ad0..5014df9854 100644 --- a/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/postgresql.md +++ b/content/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs/postgresql.md @@ -189,7 +189,21 @@ GRANT USAGE ON SCHEMA TO ; GRANT SELECT ON ALL TABLES IN SCHEMA TO ; ``` -These grants cover existing tables. To grant access to future tables, connect as the role that creates your application tables and run: +These grants cover existing tables. + +For example, your database setup might use three roles: + +- `postgres`: the administrator setting up RDI. +- `app_migrations`: the role that creates application tables. +- `debezium`: the user RDI uses to read those tables. + +These names are examples. Use your existing roles. + +The administrator can grant Debezium access to existing tables, so the initial snapshot works. Later, `app_migrations` creates a new table. Defaults set for `postgres` do not give Debezium access to that table, so reading it can fail. + +Set the defaults for the role that creates the tables. In this example, use `app_migrations` as `` and `debezium` as the Debezium user. + +To grant access to future tables, connect as the table creator and run: ```sql ALTER DEFAULT PRIVILEGES FOR ROLE IN SCHEMA @@ -198,7 +212,7 @@ ALTER DEFAULT PRIVILEGES FOR ROLE IN SCHEMA Replace `` with the role that runs `CREATE TABLE`, often your migration role. Replace `` with the schema name and `` with the Debezium user. An administrator with permission to change that role's defaults can also run this command. -Repeat for each table creator and schema. Running the command as `postgres` without `FOR ROLE` covers only tables created by `postgres`. Read access does not add a table to a publication or the RDI pipeline. +Repeat for each table creator and schema. Read access does not add a table to a publication or the RDI pipeline. ## Set privileges for Debezium to create PostgreSQL publications with `pgoutput` diff --git a/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md b/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md index 1f2cbed2e5..a92ca119d3 100644 --- a/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md +++ b/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md @@ -140,7 +140,19 @@ to capture changes. Replace `` with the schema name and `` with the username of the Debezium user. 1. - Connect as the role that creates your application tables and grant the Debezium user access to future tables: + For example, your database setup might use three roles: + + - `postgres`: the administrator setting up RDI. + - `app_migrations`: the role that creates application tables. + - `debezium`: the user RDI uses to read those tables. + + These names are examples. Use your existing roles. + + The administrator can grant Debezium access to existing tables, so the initial snapshot works. Later, `app_migrations` creates a new table. Defaults set for `postgres` do not give Debezium access to that table, so reading it can fail. + + Set the defaults for the role that creates the tables. In this example, use `app_migrations` as `` and `debezium` as the Debezium user. + + Connect as the table creator and grant the Debezium user access to future tables: ```sql ALTER DEFAULT PRIVILEGES FOR ROLE IN SCHEMA @@ -149,7 +161,7 @@ to capture changes. Replace `` with the role that runs `CREATE TABLE`, often your migration role. Replace `` with the schema name and `` with the Debezium user. An administrator with permission to change that role's defaults can also run this command. - Repeat for each table creator and schema. This command covers future tables only. Keep the grants for existing tables. Running it as `postgres` without `FOR ROLE` covers only tables created by `postgres`. + Repeat for each table creator and schema. This command covers future tables only. Keep the grants for existing tables. 1. Connect to your database as the `postgres` user and create a publication for the database: diff --git a/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md b/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md index 767673b3d6..8dc5ff295f 100644 --- a/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md +++ b/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/postgresql.md @@ -192,7 +192,21 @@ GRANT USAGE ON SCHEMA TO ; GRANT SELECT ON ALL TABLES IN SCHEMA TO ; ``` -These grants cover existing tables. To grant access to future tables, connect as the role that creates your application tables and run: +These grants cover existing tables. + +For example, your database setup might use three roles: + +- `postgres`: the administrator setting up RDI. +- `app_migrations`: the role that creates application tables. +- `debezium`: the user RDI uses to read those tables. + +These names are examples. Use your existing roles. + +The administrator can grant Debezium access to existing tables, so the initial snapshot works. Later, `app_migrations` creates a new table. Defaults set for `postgres` do not give Debezium access to that table, so reading it can fail. + +Set the defaults for the role that creates the tables. In this example, use `app_migrations` as `` and `debezium` as the Debezium user. + +To grant access to future tables, connect as the table creator and run: ```sql ALTER DEFAULT PRIVILEGES FOR ROLE IN SCHEMA @@ -201,7 +215,7 @@ ALTER DEFAULT PRIVILEGES FOR ROLE IN SCHEMA Replace `` with the role that runs `CREATE TABLE`, often your migration role. Replace `` with the schema name and `` with the Debezium user. An administrator with permission to change that role's defaults can also run this command. -Repeat for each table creator and schema. Running the command as `postgres` without `FOR ROLE` covers only tables created by `postgres`. Read access does not add a table to a publication or the RDI pipeline. +Repeat for each table creator and schema. Read access does not add a table to a publication or the RDI pipeline. ## Set privileges for Debezium to create PostgreSQL publications with `pgoutput` From b3eac292043a71d0a648941888fe631713a4b529 Mon Sep 17 00:00:00 2001 From: Zdravko Donev Date: Mon, 19 Oct 2026 20:49:52 +0300 Subject: [PATCH 38/76] RDSC-6239: Show direct PrivateLink connectivity and Lambda updates --- content/operate/rc/rdi/rds-proxy.md | 5 ++ content/operate/rc/rdi/setup.md | 32 +++++---- .../rdi/rdi-setup-diagram-aurora-lambda.svg | 71 +++++++++++++++++++ 3 files changed, 94 insertions(+), 14 deletions(-) create mode 100644 static/images/rc/rdi/rdi-setup-diagram-aurora-lambda.svg diff --git a/content/operate/rc/rdi/rds-proxy.md b/content/operate/rc/rdi/rds-proxy.md index cfa0a29389..9d276c04e6 100644 --- a/content/operate/rc/rdi/rds-proxy.md +++ b/content/operate/rc/rdi/rds-proxy.md @@ -29,6 +29,11 @@ RDS Proxy is a fully managed, highly available database proxy for Amazon RDS. Wh - **Added complexity**: RDS Proxy adds an additional layer between RDI and your database. - **Lambda provides better failover**: The Lambda function approach handles failover scenarios more efficiently. +The diagram shows the RDS Proxy alternative. It does not apply to PostgreSQL sources. + +![RDI connects through PrivateLink and a network load balancer to RDS Proxy, which forwards connections to the RDS or Aurora database.](/images/rc/rdi/rdi-setup-diagram-aurora.png) +{width="100%"} + If you still need to use RDS Proxy, follow the instructions below. ## Prerequisites diff --git a/content/operate/rc/rdi/setup.md b/content/operate/rc/rdi/setup.md index 5203fec6fe..fb623d7e34 100644 --- a/content/operate/rc/rdi/setup.md +++ b/content/operate/rc/rdi/setup.md @@ -169,7 +169,7 @@ Repeat the preparation for each source. Keep track of which endpoint service and If your source database is not accessible via a public endpoint, you need to set up an endpoint service through AWS PrivateLink to be able to connect to it. For how traffic flows over PrivateLink, how to connect to an on-premises database over AWS Direct Connect, and how to keep the connection available during failover, see the [AWS PrivateLink reference](/content/operate/rc/rdi/networking/aws-privatelink.md). -The following diagrams show the network setup for the different database setups: +Choose the diagram and instructions for your source database: - Database hosted on an AWS EC2 instance: @@ -178,8 +178,10 @@ The following diagrams show the network setup for the different database setups: - Database hosted on AWS RDS or AWS Aurora: - ![The network setup for a database hosted on AWS RDS or AWS Aurora.](/images/rc/rdi/rdi-setup-diagram-aurora.png) - {width="80%"} + ![RDI connects to the RDS or Aurora writer through PrivateLink and an internal network load balancer. Lambda updates the target IP address after an RDS event.](/images/rc/rdi/rdi-setup-diagram-aurora-lambda.svg) + {width="100%"} + + Solid arrows show database connections. Dashed arrows show how Lambda updates the target group when the database address changes. Lambda does not carry database traffic. Select the steps for your database setup. @@ -201,7 +203,9 @@ In the [AWS Management Console](https://console.aws.amazon.com/), use the **Serv - **Scheme**: Select **Internal**. - **Load balancer IP address type**: Select **IPv4**. 1. In **Network mapping**, select the VPC and availability zone associated with your source database. -1. Create a dedicated security group in the source database VPC for the network load balancer. Leave its inbound rules empty for this PrivateLink connection. Allow outbound TCP traffic to the database security group on the database port (`5432` for PostgreSQL unless you configured another port). +1. Create a security group for the network load balancer in the source database VPC: + - **Inbound rules**: Leave empty. You will disable inbound rule checks for PrivateLink traffic in a later step. + - **Outbound rules**: Allow TCP to the database security group on the database port (`5432` for PostgreSQL unless you configured another port). 1. In the load balancer settings, select this group under **Security groups**. 1. In **Listeners and routing**: 1. Select **Create target group** to [create a target group](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/create-target-group.html) with the following settings: @@ -252,16 +256,14 @@ For more details on AWS PrivateLink, see [Share your services through AWS Privat --tab-sep-- -To set up PrivateLink for a database hosted on AWS RDS or AWS Aurora: - -To connect to your RDS or Aurora database, we recommend using a Lambda function approach. This provides a reliable and secure connection method for all database types. +Connect the network load balancer (NLB) directly to your RDS or Aurora database. The database address can change after failover or maintenance. We recommend using the Lambda function to keep the NLB target group up to date. -1. [Create a network load balancer](#create-network-load-balancer-rds) that will route incoming requests to your database. +1. [Create a network load balancer](#create-network-load-balancer-rds) that forwards TCP connections to your database. 1. [Create an endpoint service](#create-endpoint-service-rds) through AWS PrivateLink. -1. [Set up Lambda function connectivity](#setup-lambda-function) to route requests to your database. +1. [Set up the Lambda function](#setup-lambda-function) to update the target group when the database address changes. > [!NOTE] -> If you have specific requirements that necessitate using RDS Proxy instead of the recommended Lambda function approach, see the [RDS Proxy setup guide](/content/operate/rc/rdi/rds-proxy.md). Note that RDS Proxy is not recommended and does not work with PostgreSQL. +> RDS Proxy is not recommended for RDI and does not support PostgreSQL logical replication. If you need this alternative for a compatible database, see the [RDS Proxy setup guide](/content/operate/rc/rdi/rds-proxy.md). ### Create network load balancer {#create-network-load-balancer-rds} @@ -271,7 +273,9 @@ In the [AWS Management Console](https://console.aws.amazon.com/), use the **Serv - **Scheme**: Select **Internal**. - **Load balancer IP address type**: Select **IPv4**. 1. In **Network mapping**, select the VPC and availability zone associated with your source database. -1. Create a dedicated security group in the source database VPC for the network load balancer. Leave its inbound rules empty for this PrivateLink connection. Allow outbound TCP traffic to the database security group on the database port (`5432` for PostgreSQL unless you configured another port). +1. Create a security group for the network load balancer in the source database VPC: + - **Inbound rules**: Leave empty. You will disable inbound rule checks for PrivateLink traffic in a later step. + - **Outbound rules**: Allow TCP to the database security group on the database port (`5432` for PostgreSQL unless you configured another port). 1. In the load balancer settings, select this group under **Security groups**. 1. In **Listeners and routing**: 1. Select **Create target group** to [create a target group](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/create-target-group.html) with the following settings: @@ -327,12 +331,12 @@ After you create the endpoint service, you need to add Redis Cloud as an Allowed For more details on AWS PrivateLink, see [Share your services through AWS PrivateLink](https://docs.aws.amazon.com/vpc/latest/privatelink/privatelink-share-your-services.html). -### Set up Lambda function connectivity {#setup-lambda-function} +### Set up Lambda to update database addresses {#setup-lambda-function} > [!NOTE] -> Setting up the Lambda function is optional but recommended for production environments. The Lambda function provides automatic failover handling and a more robust connection to your RDS or Aurora database. +> The Lambda function is optional but recommended for production. Without it, you must update the target group yourself when the database address changes. -The Lambda function monitors RDS failover events and automatically updates the NLB Target Group to point to the new primary instance's IP address. This ensures RDI reconnects automatically after a failover. +RDS sends event notifications through Amazon Simple Notification Service (SNS). SNS invokes the Lambda function. Lambda looks up the current writer address using the Domain Name System (DNS) and updates the NLB target group. RDI can reconnect after the new database target is ready. #### Option 1: Use the Redis Terraform module diff --git a/static/images/rc/rdi/rdi-setup-diagram-aurora-lambda.svg b/static/images/rc/rdi/rdi-setup-diagram-aurora-lambda.svg new file mode 100644 index 0000000000..2bef6e34be --- /dev/null +++ b/static/images/rc/rdi/rdi-setup-diagram-aurora-lambda.svg @@ -0,0 +1,71 @@ + +RDI connects directly to RDS or Aurora through AWS PrivateLink +RDI connects through a VPC endpoint, an endpoint service, and an internal network load balancer to the database writer. The target group stores the writer IP address. On an RDS event, SNS invokes Lambda. Lambda resolves the writer endpoint with DNS and updates the target group. Lambda does not carry database traffic. The load balancer and database have separate security groups. SNS and Lambda are shown outside the customer VPC. + + + + + + +RDI with RDS or Aurora: direct connection and Lambda updates + +Customer AWS account + +Redis Cloud VPC + +Customer VPC + + +NLB security group +Database security group + +RDI + +VPC +endpoint + +Endpoint +service + +Internal network +load balancer + +IP target group +Writer IP + +RDS / Aurora +Database writer + + + + + +PrivateLink +PostgreSQL example: TCP 5432 for the listener, target and health check + +Amazon SNS +1. Invoke Lambda + +Lambda function +Update addresses + +DNS lookup +2. Resolve writer + +RDS event + + + +3. Update target IP +Lambda keeps the target group pointed at the current writer. +Database connections go through the NLB, not through Lambda. + +Database connection + +Failover update + From c13720f48e0e57b928183734ef2fedc17b0f316c Mon Sep 17 00:00:00 2001 From: Zdravko Donev Date: Mon, 19 Oct 2026 20:57:56 +0300 Subject: [PATCH 39/76] RDSC-6239: Match PrivateLink diagram to Redis visual style --- .../rdi/rdi-setup-diagram-aurora-lambda.svg | 162 +++++++++++------- 1 file changed, 104 insertions(+), 58 deletions(-) diff --git a/static/images/rc/rdi/rdi-setup-diagram-aurora-lambda.svg b/static/images/rc/rdi/rdi-setup-diagram-aurora-lambda.svg index 2bef6e34be..9635a9a9bb 100644 --- a/static/images/rc/rdi/rdi-setup-diagram-aurora-lambda.svg +++ b/static/images/rc/rdi/rdi-setup-diagram-aurora-lambda.svg @@ -1,71 +1,117 @@ - + RDI connects directly to RDS or Aurora through AWS PrivateLink RDI connects through a VPC endpoint, an endpoint service, and an internal network load balancer to the database writer. The target group stores the writer IP address. On an RDS event, SNS invokes Lambda. Lambda resolves the writer endpoint with DNS and updates the target group. Lambda does not carry database traffic. The load balancer and database have separate security groups. SNS and Lambda are shown outside the customer VPC. - - + + - + RDI with RDS or Aurora: direct connection and Lambda updates - + Customer AWS account - + Redis Cloud VPC - + Customer VPC - - -NLB security group -Database security group - -RDI - -VPC -endpoint - -Endpoint -service - -Internal network -load balancer - -IP target group -Writer IP - -RDS / Aurora -Database writer - - - - - + + +NLB security group +Database security group + +RDI + +VPC +endpoint + +Endpoint +service + +Internal network +load balancer + +IP target group +Writer IP + +RDS / Aurora +Database writer + + + + + PrivateLink PostgreSQL example: TCP 5432 for the listener, target and health check - -Amazon SNS -1. Invoke Lambda - -Lambda function -Update addresses - -DNS lookup -2. Resolve writer - -RDS event - - - -3. Update target IP -Lambda keeps the target group pointed at the current writer. -Database connections go through the NLB, not through Lambda. - -Database connection - -Failover update + +Amazon SNS +1. Invoke Lambda + +Lambda function +Update addresses + +DNS lookup +2. Resolve writer + +RDS event + + + +3. Update target IP +Lambda keeps the target group pointed at the current writer. +Database connections go through the NLB, not through Lambda. + +Database connection + +Failover update + + From f9d3c3e3087b757e758104978d92b1777f13e5d4 Mon Sep 17 00:00:00 2001 From: Zdravko Donev Date: Mon, 19 Oct 2026 21:19:38 +0300 Subject: [PATCH 40/76] RDSC-6239: Use original Redis icons in PrivateLink diagram --- .../rdi/rdi-setup-diagram-aurora-lambda.svg | 85 ++++++------------- 1 file changed, 26 insertions(+), 59 deletions(-) diff --git a/static/images/rc/rdi/rdi-setup-diagram-aurora-lambda.svg b/static/images/rc/rdi/rdi-setup-diagram-aurora-lambda.svg index 9635a9a9bb..74a01831ce 100644 --- a/static/images/rc/rdi/rdi-setup-diagram-aurora-lambda.svg +++ b/static/images/rc/rdi/rdi-setup-diagram-aurora-lambda.svg @@ -9,8 +9,6 @@ @@ -28,17 +26,17 @@ text { font-family: ui-monospace, SFMono-Regular, Consolas, "Liberation Mono", m RDI -VPC -endpoint +VPC +endpoint -Endpoint -service +Endpoint +service -Internal network -load balancer +Internal network +load balancer -IP target group -Writer IP +IP target group +Writer IP RDS / Aurora Database writer @@ -50,14 +48,14 @@ text { font-family: ui-monospace, SFMono-Regular, Consolas, "Liberation Mono", m PrivateLink PostgreSQL example: TCP 5432 for the listener, target and health check -Amazon SNS -1. Invoke Lambda +Amazon SNS +1. Invoke Lambda -Lambda function -Update addresses +Lambda function +Update addresses -DNS lookup -2. Resolve writer +DNS lookup +2. Resolve writer RDS event @@ -71,47 +69,16 @@ text { font-family: ui-monospace, SFMono-Regular, Consolas, "Liberation Mono", m Failover update - + + From 16a1043eb03e5a7fca0c9c917a14d6d4ec935574 Mon Sep 17 00:00:00 2001 From: Kaitlyn Michael <76962844+kaitlynmichael@users.noreply.github.com> Date: Mon, 19 Oct 2026 14:11:26 -0500 Subject: [PATCH 41/76] RS 8.0.20-127 and K8s 8.0.20-28 release notes (#4297) --- .../8-0-20-releases/8-0-20-28-october2026.md | 35 + .../release-notes/8-0-20-releases/_index.md | 2 +- .../rs-8-0-releases/rs-8-0-20-127.md | 731 ++++++++++++++++++ 3 files changed, 767 insertions(+), 1 deletion(-) create mode 100644 content/operate/kubernetes/release-notes/8-0-20-releases/8-0-20-28-october2026.md create mode 100644 content/operate/rs/release-notes/rs-8-0-releases/rs-8-0-20-127.md diff --git a/content/operate/kubernetes/release-notes/8-0-20-releases/8-0-20-28-october2026.md b/content/operate/kubernetes/release-notes/8-0-20-releases/8-0-20-28-october2026.md new file mode 100644 index 0000000000..08d13c3ada --- /dev/null +++ b/content/operate/kubernetes/release-notes/8-0-20-releases/8-0-20-28-october2026.md @@ -0,0 +1,35 @@ +--- +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: This is a maintenance release to support Redis Software version 8.0.20-127. +hideListLinks: true +linkTitle: 8.0.20-28 (October 2026) +title: Redis Software for Kubernetes 8.0.20-28 (October 2026) release notes +weight: 25 +--- + +## Highlights + +This is a maintenance release to support [Redis Software 8.0.20](/content/operate/rs/release-notes/rs-8-0-releases/_index.md) version 8.0.20-127. For version changes, supported distributions, and known limitations, see the [release notes for 8.0.18-11 (April 2026)](/content/operate/kubernetes/release-notes/8-0-18-releases/8-0-18-11-april2026.md). + +## Downloads + +- **Redis Enterprise**: `redislabs/redis:8.0.20-127.28` (`redislabs/redis:8.0.20-127.28.minimal` for minimal base image) +- **Operator**: `redislabs/operator:8.0.20-28` +- **Services Rigger**: `redislabs/k8s-controller:8.0.20-28` +- **Call Home Client**: `redislabs/re-call-home-client:8.0.20-28` + +### OpenShift downloads + +- **OLM operator bundle**: `8.0.20-28.2` +- **Redis Enterprise**: `registry.connect.redhat.com/redislabs/redis-enterprise:8.0.20-127.28` (`registry.connect.redhat.com/redislabs/redis-enterprise:8.0.20-127.28.minimal` for minimal base image) +- **Operator**: `registry.connect.redhat.com/redislabs/redis-enterprise-operator:8.0.20-28` +- **Services Rigger**: `registry.connect.redhat.com/redislabs/services-manager:8.0.20-28` +- **Call Home Client**: `registry.connect.redhat.com/redislabs/call-home-client:8.0.20-28` + +## Known limitations + +See [8.0.20 releases](/content/operate/kubernetes/release-notes/8-0-20-releases/_index.md) for information on known limitations. diff --git a/content/operate/kubernetes/release-notes/8-0-20-releases/_index.md b/content/operate/kubernetes/release-notes/8-0-20-releases/_index.md index 29c23495b4..29625a2817 100644 --- a/content/operate/kubernetes/release-notes/8-0-20-releases/_index.md +++ b/content/operate/kubernetes/release-notes/8-0-20-releases/_index.md @@ -11,7 +11,7 @@ title: Redis Enterprise for Kubernetes 8.0.20 release notes weight: 84 --- -Redis Enterprise for Kubernetes 8.0.20 is a maintenance release of [Redis Enterprise for Kubernetes 8.0.18](/content/operate/kubernetes/release-notes/8-0-18-releases/_index.md). The latest release is 8.0.20-26 with support for Redis Software version 8.0.20-96. +Redis Enterprise for Kubernetes 8.0.20 is a maintenance release of [Redis Enterprise for Kubernetes 8.0.18](/content/operate/kubernetes/release-notes/8-0-18-releases/_index.md). The latest release is 8.0.20-28 with support for Redis Software version 8.0.20-127. ## Detailed release notes diff --git a/content/operate/rs/release-notes/rs-8-0-releases/rs-8-0-20-127.md b/content/operate/rs/release-notes/rs-8-0-releases/rs-8-0-20-127.md new file mode 100644 index 0000000000..4c965a3d7c --- /dev/null +++ b/content/operate/rs/release-notes/rs-8-0-releases/rs-8-0-20-127.md @@ -0,0 +1,731 @@ +--- +Title: Redis Software release notes 8.0.20-127 (October 2026) +alwaysopen: false +categories: +- docs +- operate +- rs +compatibleOSSVersion: Redis 8.6.2, 8.4.0, 8.2.1, 8.0.2, 7.4.3, 7.2.7, 6.2.13 +description: Bug fixes for authentication after just-in-time user provisioning, Google Cloud Storage backup configuration, DMC proxy stability, periodic latency spikes, and cluster scaling. Internal fixes and improvements. +linkTitle: 8.0.20-127 (October 2026) +weight: 76 +--- + +This is a maintenance release for ​[​Redis Software version 8.0.20](https://redis.io/downloads/#Redis_Software). + +## Highlights + +This version offers: + +- Bug fix for authentication to ACL-only databases after just-in-time (JIT) user provisioning + +- Bug fix for Google Cloud Storage backup configuration in the Cluster Manager UI + +- Bug fixes for DMC proxy stability + +- Bug fixes for periodic latency spikes and cluster scaling + +- Internal fixes and improvements + +## New in this release + +### Redis database versions and feature sets + +Redis Software version 8.0.20 includes the following Redis database versions: 8.6.2, 8.4.0, 8.2.1, 8.0.2, 7.4.3, 7.2.7, and 6.2.13. + +The [default Redis database version](/content/operate/rs/databases/configure/db-defaults.md#database-version) is 8.6. + +Redis Software includes multiple feature sets, compatible with different Redis database versions. + +{{< collapsible "Show feature set details" >}} + +The following table shows which Redis modules are compatible with each Redis database version included in this release. + +| Redis database version | Compatible Redis modules | +|------------------------|--------------------------| +| 8.6 | RediSearch 8.6
RedisJSON 8.6
RedisTimeSeries 8.6
RedisBloom 8.6
See [What's new in Redis 8.6](/content/develop/whats-new/8-6.md) and [Redis Open Source 8.6 release notes](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisce/redisos-8.6-release-notes.md) | +| 8.4 | RediSearch 8.4
RedisJSON 8.4
RedisTimeSeries 8.4
RedisBloom 8.4
See [What's new in Redis 8.4](/content/develop/whats-new/8-4.md) and [Redis Open Source 8.4 release notes](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisce/redisos-8.4-release-notes.md) | +| 8.2 | RediSearch 8.2
RedisJSON 8.2
RedisTimeSeries 8.2
RedisBloom 8.2
See [What's new in Redis 8.2](/content/develop/whats-new/8-2.md) and [Redis Open Source 8.2 release notes](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisce/redisos-8.2-release-notes.md) | +| 8.0 | RediSearch 8.0
RedisJSON 8.0
RedisTimeSeries 8.0
RedisBloom 8.0
See [What's new in Redis 8.0](/content/develop/whats-new/8-0.md) and [Redis Open Source 8.0 release notes](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisce/redisos-8.0-release-notes.md) | +| 7.4 | [RediSearch 2.10](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisearch/redisearch-2.10-release-notes.md)
[RedisJSON 2.8](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisjson/redisjson-2.8-release-notes.md)
[RedisTimeSeries 1.12](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redistimeseries/redistimeseries-1.12-release-notes.md)
[RedisBloom 2.8](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisbloom/redisbloom-2.8-release-notes.md) | +| 7.2 | [RediSearch 2.8](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisearch/redisearch-2.8-release-notes.md)
[RedisJSON 2.6](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisjson/redisjson-2.6-release-notes.md)
[RedisTimeSeries 1.10](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redistimeseries/redistimeseries-1.10-release-notes.md)
[RedisBloom 2.6](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisbloom/redisbloom-2.6-release-notes.md) | +| 6.2 | [RediSearch 2.6](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisearch/redisearch-2.6-release-notes.md)
[RedisJSON 2.4](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisjson/redisjson-2.4-release-notes.md)
[RedisTimeSeries 1.8](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redistimeseries/redistimeseries-1.8-release-notes.md)
[RedisBloom 2.4](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisbloom/redisbloom-2.4-release-notes.md) | + +{{< /collapsible >}} + +### Resolved issues + +- RS192993: Fixed an issue where creating a new user with just-in-time (JIT) provisioning could cause authentication to fail for all databases that use ACL-only authentication. + +- RS194421: Fixed periodic traffic drops and latency spikes caused by internal scheduled tasks that all started at the same time. + +- RS196137: Fixed an issue where the Cluster Manager did not send the required `private_key` field when configuring Google Cloud Storage (GCS) backups. + +- RS211091: Fixed an issue where the cluster could become unresponsive after a newly joined node became the primary node while scaling the cluster. + +- RS215608: Fixed an issue where running `dmc-cli list bdb` or `dmc-cli list server` could corrupt memory and cause the DMC proxy to crash. + +- RS216045: Fixed an issue where the DMC proxy could crash repeatedly when a client sent a command without key arguments, such as `PUBSUB SHARDNUMSUB` with no channels, in a pipeline with another command. + +## Version changes + +### OpenSSL version on RHEL 9 + +Redis Software versions 8.0.16 through 8.0.20-68 required OpenSSL 3.3 or later on RHEL 9. That requirement no longer applies, because Redis Software is now built against an earlier OpenSSL version. + +On RHEL 9, you no longer need to upgrade the `openssl` and `openssl-libs` packages before you upgrade Redis Software. + +### Reserved ports + +Make sure the following ports are open before upgrading Redis Software. + +As of Redis Software version 8.0.20, port 3357 is optional rather than reserved: + +| Port | Process name | Usage | +|------|--------------|-------| +| 3357 | reconciliation_tree_grpc | Internal communication | + +Ports reserved as of Redis Software version 7.22.0: + +| Port | Process name | Usage | +|------|--------------|-------| +| 3346 | cluster_api_internal | Cluster API internal port | +| 3351 | cluster_watchdog_grpc_api | Cluster watchdog now supports gRPC | +| 3352 | grpc_service_mesh | gRPC communication between nodes | +| 3353 | local_grpc_service_mesh | Local gRPC services | +| 3354 | grpc_gossip_envoy | gRPC gossip protocol communication between nodes | +| 3355 | authentication_service | Authentication service internal port | + +Ports reserved as of Redis Software version 7.8.2: + +| Port | Process name | Usage | +|------|--------------|-------| +| 3347 | cert_exporter | Reports cluster certificate metrics | +| 3348 | process_exporter | Reports process metrics for DMC and Redis processes | +| 3349 | cluster_wd_exporter | Reports cluster watchdog metrics | +| 3350 | db_controller | Internode communication | +| 9091 | node_exporter | Reports host node metrics related to CPU, memory, disk, and more | +| 9125 | statsd_exporter | Reports push metrics related to the DMC and syncer, and some cluster and node metrics | + +See [Ports and port ranges used by Redis Software](/content/operate/rs/networking/port-configurations.md#ports-and-port-ranges-used-by-redis-enterprise-software) for a complete list. + +### Supported platforms + +The following table provides a snapshot of supported platforms as of this Redis Software release. See the [supported platforms reference](/content/operate/rs/references/supported-platforms.md) for more details about operating system compatibility. + +{{< collapsible "Show supported platforms" >}} + +✅ Supported – The platform is supported for this version of Redis Software and Redis Stack modules. + +:warning: Deprecation warning – The platform is still supported for this version of Redis Software, but support will be removed in a future release. + +| Redis Software
major versions | 8.0 | 7.22 | 7.8 | 7.4 | 7.2 | 6.4 | 6.2 | +|---------------------------------|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:| +| **Release date** | Oct 2025 | May 2025 | Nov 2024 | Feb 2024 | Aug 2023 | Feb 2023 | Aug 2021 | +| [**End-of-life date**](/content/operate/rs/installing-upgrading/product-lifecycle.md#endoflife-schedule) | Determined after
next major release | Oct 2027 | May 2027 | Nov 2026 | Feb 2026 | Aug 2025 | Feb 2025 | +| **Platforms** | | | | | | | | +| RHEL 9 &
compatible distros[1](#table-note-1) | ✅ | ✅ | ✅ | ✅ | – | – | – | +| RHEL 9
FIPS mode[5](#table-note-5) | ✅ | ✅ | ✅ | – | – | – | – | +| RHEL 8 &
compatible distros[1](#table-note-1) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| RHEL 7 &
compatible distros[1](#table-note-1) | – | – | – | – | :warning: | :warning: | :warning: | +| Ubuntu 22.04[2](#table-note-2) | ✅ | ✅ | ✅ | – | – | – | – | +| Ubuntu 20.04[2](#table-note-2) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | – | +| Ubuntu 18.04[2](#table-note-2) | – | – | – | :warning: | :warning: | :warning: | :warning: | +| Ubuntu 16.04[2](#table-note-2) | – | – | – | – | :warning: | :warning: | :warning: | +| Amazon Linux 2023[6](#table-note-6) | ✅ | – | – | – | – | – | – | +| Amazon Linux 2 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | – | +| Amazon Linux 1 | – | – | – | – | :warning: | :warning: | :warning: | +| Kubernetes[3](#table-note-3) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| Docker[4](#table-note-4) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | + +1. The RHEL-compatible distributions CentOS, CentOS Stream, Alma, and Rocky are supported if they have full RHEL compatibility. Oracle Linux running the Red Hat Compatible Kernel (RHCK) is supported, but the Unbreakable Enterprise Kernel (UEK) is not supported. + +2. The server version of Ubuntu is recommended for production installations. The desktop version is only recommended for development deployments. + +3. See the [Redis Enterprise for Kubernetes documentation](/content/operate/kubernetes/reference/supported_k8s_distributions.md) for details about support per version and Kubernetes distribution. + +4. [Docker images](/content/operate/rs/installing-upgrading/quickstarts/docker-quickstart.md) of Redis Software are certified for development and testing only. + +5. Supported only if [FIPS was enabled during RHEL installation](https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/9/html/security_hardening/switching-rhel-to-fips-mode_security-hardening#proc_installing-the-system-with-fips-mode-enabled_switching-rhel-to-fips-mode) to ensure FIPS compliance. + +6. Amazon Linux 2023 support was added in Redis Software version 8.0.20. + +{{< /collapsible >}} + +## Downloads + +The following table shows the SHA256 checksums for the available packages: + +| Package | SHA256 checksum (8.0.20-127 October release) | +|---------|---------------------------------------| +| Ubuntu 20 | 0719abd5ce45f7a31bc52b48a972d382699cc92b9ef6f120b3c615f0025f344d | +| Ubuntu 22 (amd64) | bf5b62cebe16a2d7f7d25c252de5a3bacb70c927177a1869f5c1257821bee8d8 | +| Ubuntu 22 (arm64) | 2d893ea4e48391458ed2615f451c114859abe3b9dc8c24beeee1b34c8f9d315b | +| Red Hat Enterprise Linux (RHEL) 8 | cbf8bcb1f082784a02076e2ffec39b19d3bf18c1a379de219c19faca68797a58 | +| Red Hat Enterprise Linux (RHEL) 9 (amd64) | 247a906a5f995f678f54feec198156c90fd39e74a54b970fea1b63b94bd78b87 | +| Red Hat Enterprise Linux (RHEL) 9 (arm64) | ca13be067048cc93660cffa7e1ed6769853f9503a6425abba350e4c97fa99c48 | +| Amazon Linux 2 | 1aa364cef82f0fd6a09c0a21cdfc727367f154a666e83edec04ef803b0a00a8e | +| Amazon Linux 2023 (amd64) | 4977a09448f7ebd4f1e00046677487e68a48f41e783643dbb9b73768bff42b4e | +| Amazon Linux 2023 (arm64) | c07dc53b1387bc6a2e220e8dacf30bac79af9a94e418ab25030e8ceff90a04a1 | + +## Known issues + +- RS155734: Endpoint availability metrics do not work as expected due to a calculation error. + +## Known limitations + +#### Redis Search query failures during rolling upgrades across the 8.4 version boundary + +Redis Search queries can fail during a rolling upgrade when a cluster contains shards running Redis versions earlier than 8.4 and shards running Redis version 8.4 or later, due to an internal protocol change introduced in version 8.4. This issue affects only clusters where `parallel_shards_upgrade` has been changed from its default value of `0`. If both conditions apply, expect Redis Search downtime until all nodes are upgraded. + +#### Trim ACKED not supported for Active-Active 8.4 databases + +For Active-Active databases running Redis database version 8.4, the `ACKED` option is not supported for trimming commands. + +#### Rolling upgrade limitation for clusters with custom or deprecated modules + +Due to module handling changes introduced in Redis Software version 8.0, upgrading a cluster that contains custom or deprecated modules, such as RedisGraph and RedisGears v2, can become stuck when adding a new node to the cluster during a rolling upgrade. + +#### Module commands limitation during Active-Active database upgrades to Redis 8.0 + +When upgrading an Active-Active database to Redis version 8.0, you cannot use module commands until all Active-Active database instances have been upgraded. Currently, these commands are not blocked automatically. + +#### Redis 8.0 database cannot be created with flash + +You cannot create a Redis 8.0 database with flash storage enabled. Create a Redis 8.0 database with RAM-only storage instead, or use Redis 8.2 for flash-enabled (Redis Flex) databases. + +#### Cluster Manager UI limitations + +The following legacy UI features are not yet available in the new Cluster Manager UI: + +- Purge an Active-Active instance. + + Use [`crdb-cli crdb purge-instance`](/content/operate/rs/references/cli-utilities/crdb-cli/crdb/purge-instance.md) instead. + +## Security + +#### Redis Open Source security fixes compatibility + +As part of Redis's commitment to security, Redis Software implements the latest [security fixes](https://github.com/redis/redis/releases) available with [Redis Open Source](https://github.com/redis/redis). Redis Software has already included the fixes for the relevant CVEs. + +Some CVEs announced for Redis Open Source do not affect Redis Software due to different or additional functionality available in Redis Software that is not available in Redis Open Source. + +Redis Software 8.0.20-127 supports Redis Open Source 8.6, 8.4, 8.2, 8.0, 7.4, 7.2, and 6.2. Below is the list of Redis Open Source CVEs and other security vulnerabilities fixed by version. + +{{< collapsible "Redis 8.6.x" >}} + +- Memory corruption in `SLOT_INFO` handling may allow an unauthenticated user to potentially run their own code. + +- A use-after-free in TLS pending-data processing may allow an authenticated user to cause a denial-of-service. + +- A vulnerability in the `RESTORE` command and in RDB loading allows an authenticated user to trigger a use-after-free in stream consumer groups via a specially crafted serialized payload, potentially resulting in remote code execution. + +- An ACL key-pattern bypass in `GEORADIUS` and `GEORADIUSBYMEMBER` may allow an authenticated user to write to keys outside their permitted key patterns. + +- An out-of-bounds read during command key extraction may cause the server to crash. + +- VSET: Missing `level` cap when loading a serialized HNSW graph may lead to resource exhaustion. + +- VSET: Missing validation of the parameter count when loading a serialized vector set may lead to oversized allocation and resource exhaustion. + +- Certificate common-name handling can allow an authentication bypass when `tls-auth-clients-user` is set to `CN`. + +- (CVE-2026-62356) An authenticated user may trigger a heap out-of-bounds write in RedisBloom by loading a specially crafted Count-Min Sketch (`CMSketch`) object, for example via the `RESTORE` command or a crafted RDB file. + +- RedisBloom: Out-of-bounds access in the `TopK` heap cleanup path. + +- VSET: Large `VADD REDUCE` values can lead to out-of-memory crashes or buffer overflow. + +- (CVE-2026-23479) When a blocked client is evicted while re-executing a blocked command, an authenticated user may trigger a use-after-free and potentially lead to remote code execution. + +- (CVE‑2026‑25243) A vulnerability in the Redis `RESTORE` command allows an authenticated user to trigger an invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE-2026-25588) A vulnerability in the `RESTORE` command, when used with the RedisTimeSeries module, allows an authenticated attacker to trigger invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE‑2026‑25589) A vulnerability in the `RESTORE` command, when used with the RedisBloom module, allows an authenticated attacker to trigger invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE-2026-23631) An authenticated user may exploit the synchronization mechanism of the master-replica and trigger a use-after-free vulnerability, potentially leading to remote code execution. The bug affects only replicas that are configured, or may be configured with `replica-read-only` disabled, and exists in all versions of Redis with Lua scripting. + +- Insufficient sanitization of Lua script error messages can allow an authenticated user with `EVAL` permission to inject arbitrary RESP protocol responses, leading to denial of service, data manipulation, and connection pool poisoning. + +- RedisBloom: Integer overflow in `TopK` RDB loading may lead to heap out-of-bounds access. + +- RedisBloom: Missing validation in `td_histogram_t` RDB loading may lead to heap buffer overflow. + +- RedisBloom: Missing validation in `CuckooFilter` RDB loading may lead to heap out-of-bounds access. + +- RedisBloom: Missing validation in `SBChain` and `CMSketch` RDB loading may lead to heap out-of-bounds write. + +- Added validation of limits during RDB load and data creation to avoid resource exhaustion and Denial-of-Service (DoS) vulnerabilities. + +{{< /collapsible >}} + +{{< collapsible "Redis 8.4.x" >}} + +- Memory corruption in `SLOT_INFO` handling may allow an unauthenticated user to potentially run their own code. + +- A use-after-free in TLS pending-data processing may allow an authenticated user to cause a denial-of-service. + +- A vulnerability in the `RESTORE` command and in RDB loading allows an authenticated user to trigger a use-after-free in stream consumer groups via a specially crafted serialized payload, potentially resulting in remote code execution. + +- An ACL key-pattern bypass in `GEORADIUS` and `GEORADIUSBYMEMBER` may allow an authenticated user to write to keys outside their permitted key patterns. + +- An out-of-bounds read during command key extraction may cause the server to crash. + +- VSET: Missing `level` cap when loading a serialized HNSW graph may lead to resource exhaustion. + +- VSET: Missing validation of the parameter count when loading a serialized vector set may lead to oversized allocation and resource exhaustion. + +- (CVE-2026-62356) An authenticated user may trigger a heap out-of-bounds write in RedisBloom by loading a specially crafted Count-Min Sketch (`CMSketch`) object, for example via the `RESTORE` command or a crafted RDB file. + +- RedisBloom: Out-of-bounds access in the `TopK` heap cleanup path. + +- VSET: Large `VADD REDUCE` values can lead to out-of-memory crashes or buffer overflow. + +- (CVE-2026-23479) When a blocked client is evicted while re-executing a blocked command, an authenticated user may trigger a use-after-free and potentially lead to remote code execution. + +- (CVE‑2026‑25243) A vulnerability in the Redis `RESTORE` command allows an authenticated user to trigger an invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE-2026-25588) A vulnerability in the `RESTORE` command, when used with the RedisTimeSeries module, allows an authenticated attacker to trigger invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE‑2026‑25589) A vulnerability in the `RESTORE` command, when used with the RedisBloom module, allows an authenticated attacker to trigger invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE-2026-23631) An authenticated user may exploit the synchronization mechanism of the master-replica and trigger a use-after-free vulnerability, potentially leading to remote code execution. The bug affects only replicas that are configured, or may be configured with `replica-read-only` disabled, and exists in all versions of Redis with Lua scripting. + +- Insufficient sanitization of Lua script error messages can allow an authenticated user with `EVAL` permission to inject arbitrary RESP protocol responses, leading to denial of service, data manipulation, and connection pool poisoning. + +- RedisBloom: Integer overflow in `TopK` RDB loading may lead to heap out-of-bounds access. + +- RedisBloom: Missing validation in `td_histogram_t` RDB loading may lead to heap buffer overflow. + +- RedisBloom: Missing validation in `CuckooFilter` RDB loading may lead to heap out-of-bounds access. + +- RedisBloom: Missing validation in `SBChain` and `CMSketch` RDB loading may lead to heap out-of-bounds write. + +- Added validation of limits during RDB load and data creation to avoid resource exhaustion and Denial-of-Service (DoS) vulnerabilities. + +{{< /collapsible >}} + +{{< collapsible "Redis 8.2.x" >}} + +- Memory corruption in `SLOT_INFO` handling may allow an unauthenticated user to potentially run their own code. + +- A use-after-free in TLS pending-data processing may allow an authenticated user to cause a denial-of-service. + +- A vulnerability in the `RESTORE` command and in RDB loading allows an authenticated user to trigger a use-after-free in stream consumer groups via a specially crafted serialized payload, potentially resulting in remote code execution. + +- An ACL key-pattern bypass in `GEORADIUS` and `GEORADIUSBYMEMBER` may allow an authenticated user to write to keys outside their permitted key patterns. + +- An out-of-bounds read during command key extraction may cause the server to crash. + +- VSET: Missing `level` cap when loading a serialized HNSW graph may lead to resource exhaustion. + +- VSET: Missing validation of the parameter count when loading a serialized vector set may lead to oversized allocation and resource exhaustion. + +- (CVE-2026-62356) An authenticated user may trigger a heap out-of-bounds write in RedisBloom by loading a specially crafted Count-Min Sketch (`CMSketch`) object, for example via the `RESTORE` command or a crafted RDB file. + +- RedisBloom: Out-of-bounds access in the `TopK` heap cleanup path. + +- VSET: Large `VADD REDUCE` values can lead to out-of-memory crashes or buffer overflow. + +- (CVE-2026-23479) When a blocked client is evicted while re-executing a blocked command, an authenticated user may trigger a use-after-free and potentially lead to remote code execution. + +- (CVE‑2026‑25243) A vulnerability in the Redis `RESTORE` command allows an authenticated user to trigger an invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE-2026-25588) A vulnerability in the `RESTORE` command, when used with the RedisTimeSeries module, allows an authenticated attacker to trigger invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE‑2026‑25589) A vulnerability in the `RESTORE` command, when used with the RedisBloom module, allows an authenticated attacker to trigger invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE-2026-23631) An authenticated user may exploit the synchronization mechanism of the master-replica and trigger a use-after-free vulnerability, potentially leading to remote code execution. The bug affects only replicas that are configured, or may be configured with `replica-read-only` disabled, and exists in all versions of Redis with Lua scripting. + +- Insufficient sanitization of Lua script error messages can allow an authenticated user with `EVAL` permission to inject arbitrary RESP protocol responses, leading to denial of service, data manipulation, and connection pool poisoning. + +- RedisBloom: Integer overflow in `TopK` RDB loading may lead to heap out-of-bounds access. + +- RedisBloom: Missing validation in `td_histogram_t` RDB loading may lead to heap buffer overflow. + +- RedisBloom: Missing validation in `CuckooFilter` RDB loading may lead to heap out-of-bounds access. + +- RedisBloom: Missing validation in `SBChain` and `CMSketch` RDB loading may lead to heap out-of-bounds write. + +- Added validation of limits during RDB load and data creation to avoid resource exhaustion and Denial-of-Service (DoS) vulnerabilities. + +- RedisBloom: Restore invalid filter. + +- (CVE-2025-62507) A user can run the `XACKDEL` command with multiple IDs and trigger a stack buffer overflow, which can potentially lead to remote code execution. + +- The `HGETEX` command can lead to a buffer overflow. + +- Integer overflow in `hllPatLen`. + +- RedisBloom: Cuckoo filter counter overflow. + +- RedisBloom: Invalid Bloom filters can cause arbitrary memory reads and writes. + +- RedisBloom: Reachable assert in `TopK_Create` + +- RedisBloom: Out-of-bounds access with empty Bloom chains. + +- RedisBloom: Division by zero in Cuckoo filter insertion. + +- (CVE-2025-46818) An authenticated user may use a specially crafted Lua script to manipulate different LUA objects and potentially run their own code in the context of another user. + +- (CVE-2025-46819) An authenticated user may use a specially crafted LUA script to read out-of-bound data or crash the server and lead to subsequent denial of service. + +- (CVE-2025-46817) An authenticated user may use a specially crafted Lua script to cause an integer overflow and potentially lead to remote code execution. + +- (CVE-2025-49844) An authenticated user may use a specially crafted Lua script to manipulate the garbage collector, trigger a use-after-free, and potentially lead to remote code execution. + +{{< /collapsible >}} + +{{< collapsible "Redis 8.0.x" >}} + +- Memory corruption in `SLOT_INFO` handling may allow an unauthenticated user to potentially run their own code. + +- A use-after-free in TLS pending-data processing may allow an authenticated user to cause a denial-of-service. + +- A vulnerability in the `RESTORE` command and in RDB loading allows an authenticated user to trigger a use-after-free in stream consumer groups via a specially crafted serialized payload, potentially resulting in remote code execution. + +- An ACL key-pattern bypass in `GEORADIUS` and `GEORADIUSBYMEMBER` may allow an authenticated user to write to keys outside their permitted key patterns. + +- An out-of-bounds read during command key extraction may cause the server to crash. + +- (CVE-2026-62356) An authenticated user may trigger a heap out-of-bounds write in RedisBloom by loading a specially crafted Count-Min Sketch (`CMSketch`) object, for example via the `RESTORE` command or a crafted RDB file. + +- RedisBloom: Out-of-bounds access in the `TopK` heap cleanup path. + +- VSET: Large `VADD REDUCE` values can lead to out-of-memory crashes or buffer overflow. + +- (CVE-2026-23479) When a blocked client is evicted while re-executing a blocked command, an authenticated user may trigger a use-after-free and potentially lead to remote code execution. + +- (CVE‑2026‑25243) A vulnerability in the Redis `RESTORE` command allows an authenticated user to trigger an invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE-2026-25588) A vulnerability in the `RESTORE` command, when used with the RedisTimeSeries module, allows an authenticated attacker to trigger invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE‑2026‑25589) A vulnerability in the `RESTORE` command, when used with the RedisBloom module, allows an authenticated attacker to trigger invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE-2026-23631) An authenticated user may exploit the synchronization mechanism of the master-replica and trigger a use-after-free vulnerability, potentially leading to remote code execution. The bug affects only replicas that are configured, or may be configured with `replica-read-only` disabled, and exists in all versions of Redis with Lua scripting. + +- Insufficient sanitization of Lua script error messages can allow an authenticated user with `EVAL` permission to inject arbitrary RESP protocol responses, leading to denial of service, data manipulation, and connection pool poisoning. + +- RedisBloom: Integer overflow in `TopK` RDB loading may lead to heap out-of-bounds access. + +- RedisBloom: Missing validation in `td_histogram_t` RDB loading may lead to heap buffer overflow. + +- RedisBloom: Missing validation in `CuckooFilter` RDB loading may lead to heap out-of-bounds access. + +- RedisBloom: Missing validation in `SBChain` and `CMSketch` RDB loading may lead to heap out-of-bounds write. + +- Added validation of limits during RDB load and data creation to avoid resource exhaustion and Denial-of-Service (DoS) vulnerabilities. + +- RedisBloom: Restore invalid filter. + +- The `HGETEX` command can lead to a buffer overflow. + +- Integer overflow in `hllPatLen`. + +- RedisBloom: Cuckoo filter counter overflow. + +- RedisBloom: Invalid Bloom filters can cause arbitrary memory reads and writes. + +- RedisBloom: Reachable assert in `TopK_Create` + +- RedisBloom: Out-of-bounds access with empty Bloom chains. + +- RedisBloom: Division by zero in Cuckoo filter insertion. + +- (CVE-2025-46818) An authenticated user may use a specially crafted Lua script to manipulate different LUA objects and potentially run their own code in the context of another user. + +- (CVE-2025-46819) An authenticated user may use a specially crafted LUA script to read out-of-bound data or crash the server and lead to subsequent denial of service. + +- (CVE-2025-46817) An authenticated user may use a specially crafted Lua script to cause an integer overflow and potentially lead to remote code execution. + +- (CVE-2025-49844) An authenticated user may use a specially crafted Lua script to manipulate the garbage collector, trigger a use-after-free, and potentially lead to remote code execution. + +{{< /collapsible >}} + +{{< collapsible "Redis 7.4.x" >}} + +- Memory corruption in `SLOT_INFO` handling may allow an unauthenticated user to potentially run their own code. + +- A use-after-free in TLS pending-data processing may allow an authenticated user to cause a denial-of-service. + +- A vulnerability in the `RESTORE` command and in RDB loading allows an authenticated user to trigger a use-after-free in stream consumer groups via a specially crafted serialized payload, potentially resulting in remote code execution. + +- An ACL key-pattern bypass in `GEORADIUS` and `GEORADIUSBYMEMBER` may allow an authenticated user to write to keys outside their permitted key patterns. + +- An out-of-bounds read during command key extraction may cause the server to crash. + +- (CVE-2026-62356) An authenticated user may trigger a heap out-of-bounds write in RedisBloom by loading a specially crafted Count-Min Sketch (`CMSketch`) object, for example via the `RESTORE` command or a crafted RDB file. + +- RedisBloom: Out-of-bounds access in the `TopK` heap cleanup path. + +- (CVE-2026-23479) When a blocked client is evicted while re-executing a blocked command, an authenticated user may trigger a use-after-free and potentially lead to remote code execution. + +- (CVE‑2026‑25243) A vulnerability in the Redis `RESTORE` command allows an authenticated user to trigger an invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE-2026-25588) A vulnerability in the `RESTORE` command, when used with the RedisTimeSeries module, allows an authenticated attacker to trigger invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE‑2026‑25589) A vulnerability in the `RESTORE` command, when used with the RedisBloom module, allows an authenticated attacker to trigger invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE-2026-23631) An authenticated user may exploit the synchronization mechanism of the master-replica and trigger a use-after-free vulnerability, potentially leading to remote code execution. The bug affects only replicas that are configured, or may be configured with `replica-read-only` disabled, and exists in all versions of Redis with Lua scripting. + +- Insufficient sanitization of Lua script error messages can allow an authenticated user with `EVAL` permission to inject arbitrary RESP protocol responses, leading to denial of service, data manipulation, and connection pool poisoning. + +- RedisBloom: Integer overflow in `TopK` RDB loading may lead to heap out-of-bounds access. + +- RedisBloom: Missing validation in `td_histogram_t` RDB loading may lead to heap buffer overflow. + +- RedisBloom: Missing validation in `CuckooFilter` RDB loading may lead to heap out-of-bounds access. + +- RedisBloom: Missing validation in `SBChain` and `CMSketch` RDB loading may lead to heap out-of-bounds write. + +- Added validation of limits during RDB load and data creation to avoid resource exhaustion and Denial-of-Service (DoS) vulnerabilities. + +- RedisBloom: Restore invalid filter. + +- Integer overflow in `hllPatLen`. + +- RedisBloom: Cuckoo filter counter overflow. + +- RedisBloom: Invalid Bloom filters can cause arbitrary memory reads and writes. + +- RedisBloom: Reachable assert in `TopK_Create` + +- RedisBloom: Out-of-bounds access with empty Bloom chains. + +- RedisBloom: Division by zero in Cuckoo filter insertion. + +- (CVE-2025-46818) An authenticated user may use a specially crafted Lua script to manipulate different LUA objects and potentially run their own code in the context of another user. + +- (CVE-2025-46819) An authenticated user may use a specially crafted LUA script to read out-of-bound data or crash the server and lead to subsequent denial of service. + +- (CVE-2025-46817) An authenticated user may use a specially crafted Lua script to cause an integer overflow and potentially lead to remote code execution. + +- (CVE-2025-49844) An authenticated user may use a specially crafted Lua script to manipulate the garbage collector, trigger a use-after-free, and potentially lead to remote code execution. + +- (CVE-2025-32023) An authenticated user can use a specially crafted string to trigger a stack/heap out-of-bounds write on HyperLogLog operations, which can lead to remote code execution. + +- (CVE-2025-21605) An unauthenticated client can cause unlimited growth of output buffers until the server runs out of memory or is terminated, which can lead to denial-of-service. + +{{< /collapsible >}} + +{{< collapsible "Redis 7.2.x" >}} + +- Memory corruption in `SLOT_INFO` handling may allow an unauthenticated user to potentially run their own code. + +- A use-after-free in TLS pending-data processing may allow an authenticated user to cause a denial-of-service. + +- A vulnerability in the `RESTORE` command and in RDB loading allows an authenticated user to trigger a use-after-free in stream consumer groups via a specially crafted serialized payload, potentially resulting in remote code execution. + +- An ACL key-pattern bypass in `GEORADIUS` and `GEORADIUSBYMEMBER` may allow an authenticated user to write to keys outside their permitted key patterns. + +- An out-of-bounds read during command key extraction may cause the server to crash. + +- (CVE-2026-62356) An authenticated user may trigger a heap out-of-bounds write in RedisBloom by loading a specially crafted Count-Min Sketch (`CMSketch`) object, for example via the `RESTORE` command or a crafted RDB file. + +- RedisBloom: Out-of-bounds access in the `TopK` heap cleanup path. + +- (CVE-2026-23479) When a blocked client is evicted while re-executing a blocked command, an authenticated user may trigger a use-after-free and potentially lead to remote code execution. + +- (CVE‑2026‑25243) A vulnerability in the Redis `RESTORE` command allows an authenticated user to trigger an invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE-2026-25588) A vulnerability in the `RESTORE` command, when used with the RedisTimeSeries module, allows an authenticated attacker to trigger invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE‑2026‑25589) A vulnerability in the `RESTORE` command, when used with the RedisBloom module, allows an authenticated attacker to trigger invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE-2026-23631) An authenticated user may exploit the synchronization mechanism of the master-replica and trigger a use-after-free vulnerability, potentially leading to remote code execution. The bug affects only replicas that are configured, or may be configured with `replica-read-only` disabled, and exists in all versions of Redis with Lua scripting. + +- Insufficient sanitization of Lua script error messages can allow an authenticated user with `EVAL` permission to inject arbitrary RESP protocol responses, leading to denial of service, data manipulation, and connection pool poisoning. + +- RedisBloom: Integer overflow in `TopK` RDB loading may lead to heap out-of-bounds access. + +- RedisBloom: Missing validation in `td_histogram_t` RDB loading may lead to heap buffer overflow. + +- RedisBloom: Missing validation in `CuckooFilter` RDB loading may lead to heap out-of-bounds access. + +- RedisBloom: Missing validation in `SBChain` and `CMSketch` RDB loading may lead to heap out-of-bounds write. + +- Added validation of limits during RDB load and data creation to avoid resource exhaustion and Denial-of-Service (DoS) vulnerabilities. + +- RedisBloom: Restore invalid filter. + +- Integer overflow in `hllPatLen`. + +- RedisBloom: Cuckoo filter counter overflow. + +- RedisBloom: Invalid Bloom filters can cause arbitrary memory reads and writes. + +- RedisBloom: Reachable assert in `TopK_Create` + +- RedisBloom: Out-of-bounds access with empty Bloom chains. + +- RedisBloom: Division by zero in Cuckoo filter insertion. + +- (CVE-2025-46818) An authenticated user may use a specially crafted Lua script to manipulate different LUA objects and potentially run their own code in the context of another user. + +- (CVE-2025-46819) An authenticated user may use a specially crafted LUA script to read out-of-bound data or crash the server and lead to subsequent denial of service. + +- (CVE-2025-46817) An authenticated user may use a specially crafted Lua script to cause an integer overflow and potentially lead to remote code execution. + +- (CVE-2025-49844) An authenticated user may use a specially crafted Lua script to manipulate the garbage collector, trigger a use-after-free, and potentially lead to remote code execution. + +- (CVE-2025-32023) An authenticated user can use a specially crafted string to trigger a stack/heap out-of-bounds write on HyperLogLog operations, which can lead to remote code execution. + +- (CVE-2025-21605) An unauthenticated client can cause unlimited growth of output buffers until the server runs out of memory or is terminated, which can lead to denial-of-service. + +- (CVE-2024-31449) An authenticated user may use a specially crafted Lua script to trigger a stack buffer overflow in the bit library, which may potentially lead to remote code execution. + +- (CVE-2024-31228) An authenticated user can trigger a denial-of-service by using specially crafted, long string match patterns on supported commands such as `KEYS`, `SCAN`, `PSUBSCRIBE`, `FUNCTION LIST`, `COMMAND LIST`, and ACL definitions. Matching of extremely long patterns may result in unbounded recursion, leading to stack overflow and process crashes. + +- (CVE-2023-41056) In some cases, Redis may incorrectly handle resizing of memory buffers, which can result in incorrect accounting of buffer sizes and lead to heap overflow and potential remote code execution. + +- (CVE-2023-41053) Redis does not correctly identify keys accessed by `SORT_RO` and, as a result, may grant users executing this command access to keys that are not explicitly authorized by the ACL configuration. (Redis 7.2.1) + +{{< /collapsible >}} + +{{< collapsible "Redis 7.0.x" >}} + +- (CVE-2024-31449) An authenticated user may use a specially crafted Lua script to trigger a stack buffer overflow in the bit library, which may potentially lead to remote code execution. + +- (CVE-2024-31228) An authenticated user can trigger a denial-of-service by using specially crafted, long string match patterns on supported commands such as `KEYS`, `SCAN`, `PSUBSCRIBE`, `FUNCTION LIST`, `COMMAND LIST`, and ACL definitions. Matching of extremely long patterns may result in unbounded recursion, leading to stack overflow and process crashes. + +- (CVE-2023-41056) In some cases, Redis may incorrectly handle resizing of memory buffers, which can result in incorrect accounting of buffer sizes and lead to heap overflow and potential remote code execution. + +- (CVE-2023-41053) Redis does not correctly identify keys accessed by `SORT_RO` and, as a result, may grant users executing this command access to keys that are not explicitly authorized by the ACL configuration. (Redis 7.0.13) + +- (CVE-2023-36824) Extracting key names from a command and a list of arguments may, in some cases, trigger a heap overflow and result in reading random heap memory, heap corruption, and potentially remote code execution. Specifically: using `COMMAND GETKEYS*` and validation of key names in ACL rules. (Redis 7.0.12) + +- (CVE-2023-28856) Authenticated users can use the `HINCRBYFLOAT` command to create an invalid hash field that will crash Redis on access. (Redis 7.0.11) + +- (CVE-2023-28425) Specially crafted `MSETNX` commands can lead to assertion and denial-of-service. (Redis 7.0.10) + +- (CVE-2023-25155) Specially crafted `SRANDMEMBER`, `ZRANDMEMBER`, and `HRANDFIELD` commands can trigger an integer overflow, resulting in a runtime assertion and termination of the Redis server process. (Redis 7.0.9) + +- (CVE-2023-22458) Integer overflow in the Redis `HRANDFIELD` and `ZRANDMEMBER` commands can lead to denial-of-service. (Redis 7.0.8) + +- (CVE-2022-36021) String matching commands (like `SCAN` or `KEYS`) with a specially crafted pattern to trigger a denial-of-service attack on Redis can cause it to hang and consume 100% CPU time. (Redis 7.0.9) + +- (CVE-2022-35977) Integer overflow in the Redis `SETRANGE` and `SORT`/`SORT_RO` commands can drive Redis to OOM panic. (Redis 7.0.8) + +- (CVE-2022-35951) Executing an `XAUTOCLAIM` command on a stream key in a specific state, with a specially crafted `COUNT` argument, may cause an integer overflow, a subsequent heap overflow, and potentially lead to remote code execution. The problem affects Redis versions 7.0.0 or newer. (Redis 7.0.5) + +- (CVE-2022-31144) A specially crafted `XAUTOCLAIM` command on a stream key in a specific state may result in heap overflow and potentially remote code execution. The problem affects Redis versions 7.0.0 or newer. (Redis 7.0.4) + +- (CVE-2022-24834) A specially crafted Lua script executing in Redis can trigger a heap overflow in the cjson and cmsgpack libraries, and result in heap corruption and potentially remote code execution. The problem exists in all versions of Redis with Lua scripting support, starting from 2.6, and affects only authenticated and authorized users. (Redis 7.0.12) + +- (CVE-2022-24736) An attacker attempting to load a specially crafted Lua script can cause NULL pointer dereference which will result in a crash of the `redis-server` process. This issue affects all versions of Redis. (Redis 7.0.0) + +- (CVE-2022-24735) By exploiting weaknesses in the Lua script execution environment, an attacker with access to Redis can inject Lua code that will execute with the (potentially higher) privileges of another Redis user. (Redis 7.0.0) + +{{< /collapsible >}} + +{{< collapsible "Redis 6.2.x" >}} + +- Memory corruption in `SLOT_INFO` handling may allow an unauthenticated user to potentially run their own code. + +- A use-after-free in TLS pending-data processing may allow an authenticated user to cause a denial-of-service. + +- A vulnerability in the `RESTORE` command and in RDB loading allows an authenticated user to trigger a use-after-free in stream consumer groups via a specially crafted serialized payload, potentially resulting in remote code execution. + +- An ACL key-pattern bypass in `GEORADIUS` and `GEORADIUSBYMEMBER` may allow an authenticated user to write to keys outside their permitted key patterns. + +- An out-of-bounds read during command key extraction may cause the server to crash. + +- (CVE-2026-62356) An authenticated user may trigger a heap out-of-bounds write in RedisBloom by loading a specially crafted Count-Min Sketch (`CMSketch`) object, for example via the `RESTORE` command or a crafted RDB file. + +- RedisBloom: Out-of-bounds access in the `TopK` heap cleanup path. + +- (CVE-2026-23479) When a blocked client is evicted while re-executing a blocked command, an authenticated user may trigger a use-after-free and potentially lead to remote code execution. + +- (CVE‑2026‑25243) A vulnerability in the Redis `RESTORE` command allows an authenticated user to trigger an invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE-2026-25588) A vulnerability in the `RESTORE` command, when used with the RedisTimeSeries module, allows an authenticated attacker to trigger invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE‑2026‑25589) A vulnerability in the `RESTORE` command, when used with the RedisBloom module, allows an authenticated attacker to trigger invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE-2026-23631) An authenticated user may exploit the synchronization mechanism of the master-replica and trigger a use-after-free vulnerability, potentially leading to remote code execution. The bug affects only replicas that are configured, or may be configured with `replica-read-only` disabled, and exists in all versions of Redis with Lua scripting. + +- Insufficient sanitization of Lua script error messages can allow an authenticated user with `EVAL` permission to inject arbitrary RESP protocol responses, leading to denial of service, data manipulation, and connection pool poisoning. + +- RedisBloom: Integer overflow in `TopK` RDB loading may lead to heap out-of-bounds access. + +- RedisBloom: Missing validation in `td_histogram_t` RDB loading may lead to heap buffer overflow. + +- RedisBloom: Missing validation in `CuckooFilter` RDB loading may lead to heap out-of-bounds access. + +- RedisBloom: Missing validation in `SBChain` and `CMSketch` RDB loading may lead to heap out-of-bounds write. + +- Added validation of limits during RDB load and data creation to avoid resource exhaustion and Denial-of-Service (DoS) vulnerabilities. + +- RedisBloom: Restore invalid filter. + +- Integer overflow in `hllPatLen`. + +- RedisBloom: Cuckoo filter counter overflow. + +- RedisBloom: Invalid Bloom filters can cause arbitrary memory reads and writes. + +- RedisBloom: Reachable assert in `TopK_Create` + +- RedisBloom: Out-of-bounds access with empty Bloom chains. + +- RedisBloom: Division by zero in Cuckoo filter insertion. + +- (CVE-2025-46818) An authenticated user may use a specially crafted Lua script to manipulate different LUA objects and potentially run their own code in the context of another user. + +- (CVE-2025-46819) An authenticated user may use a specially crafted LUA script to read out-of-bound data or crash the server and lead to subsequent denial of service. + +- (CVE-2025-46817) An authenticated user may use a specially crafted Lua script to cause an integer overflow and potentially lead to remote code execution. + +- (CVE-2025-49844) An authenticated user may use a specially crafted Lua script to manipulate the garbage collector, trigger a use-after-free, and potentially lead to remote code execution. + +- (CVE-2025-32023) An authenticated user can use a specially crafted string to trigger a stack/heap out-of-bounds write on HyperLogLog operations, which can lead to remote code execution. + +- (CVE-2025-21605) An unauthenticated client can cause unlimited growth of output buffers until the server runs out of memory or is terminated, which can lead to denial-of-service. + +- (CVE-2024-31449) An authenticated user may use a specially crafted Lua script to trigger a stack buffer overflow in the bit library, which may potentially lead to remote code execution. + +- (CVE-2024-31228) An authenticated user can trigger a denial-of-service by using specially crafted, long string match patterns on supported commands such as `KEYS`, `SCAN`, `PSUBSCRIBE`, `FUNCTION LIST`, `COMMAND LIST`, and ACL definitions. Matching of extremely long patterns may result in unbounded recursion, leading to stack overflow and process crashes. + +- (CVE-2023-28856) Authenticated users can use the `HINCRBYFLOAT` command to create an invalid hash field that will crash Redis on access. (Redis 6.2.12) + +- (CVE-2023-25155) Specially crafted `SRANDMEMBER`, `ZRANDMEMBER`, and `HRANDFIELD` commands can trigger an integer overflow, resulting in a runtime assertion and termination of the Redis server process. (Redis 6.2.11) + +- (CVE-2023-22458) Integer overflow in the Redis `HRANDFIELD` and `ZRANDMEMBER` commands can lead to denial-of-service. (Redis 6.2.9) + +- (CVE-2022-36021) String matching commands (like `SCAN` or `KEYS`) with a specially crafted pattern to trigger a denial-of-service attack on Redis can cause it to hang and consume 100% CPU time. (Redis 6.2.11) + +- (CVE-2022-35977) Integer overflow in the Redis `SETRANGE` and `SORT`/`SORT_RO` commands can drive Redis to OOM panic. (Redis 6.2.9) + +- (CVE-2022-24834) A specially crafted Lua script executing in Redis can trigger a heap overflow in the cjson and cmsgpack libraries, and result in heap corruption and potentially remote code execution. The problem exists in all versions of Redis with Lua scripting support, starting from 2.6, and affects only authenticated and authorized users. (Redis 6.2.13) + +- (CVE-2022-24736) An attacker attempting to load a specially crafted Lua script can cause NULL pointer dereference which will result in a crash of the `redis-server` process. This issue affects all versions of Redis. (Redis 6.2.7) + +- (CVE-2022-24735) By exploiting weaknesses in the Lua script execution environment, an attacker with access to Redis can inject Lua code that will execute with the (potentially higher) privileges of another Redis user. (Redis 6.2.7) + +- (CVE-2021-41099) Integer to heap buffer overflow handling certain string commands and network payloads, when `proto-max-bulk-len` is manually configured to a non-default, very large value. (Redis 6.2.6) + +- (CVE-2021-32762) Integer to heap buffer overflow issue in `redis-cli` and `redis-sentinel` parsing large multi-bulk replies on some older and less common platforms. (Redis 6.2.6) + +- (CVE-2021-32761) An integer overflow bug in Redis version 2.2 or newer can be exploited using the `BITFIELD` command to corrupt the heap and potentially result with remote code execution. (Redis 6.2.5) + +- (CVE-2021-32687) Integer to heap buffer overflow with intsets, when `set-max-intset-entries` is manually configured to a non-default, very large value. (Redis 6.2.6) + +- (CVE-2021-32675) Denial Of Service when processing RESP request payloads with a large number of elements on many connections. (Redis 6.2.6) + +- (CVE-2021-32672) Random heap reading issue with Lua Debugger. (Redis 6.2.6) + +- (CVE-2021-32628) Integer to heap buffer overflow handling ziplist-encoded data types, when configuring a large, non-default value for `hash-max-ziplist-entries`, `hash-max-ziplist-value`, `zset-max-ziplist-entries` or `zset-max-ziplist-value`. (Redis 6.2.6) + +- (CVE-2021-32627) Integer to heap buffer overflow issue with streams, when configuring a non-default, large value for `proto-max-bulk-len` and `client-query-buffer-limit`. (Redis 6.2.6) + +- (CVE-2021-32626) Specially crafted Lua scripts may result with Heap buffer overflow. (Redis 6.2.6) + +- (CVE-2021-32625) An integer overflow bug in Redis version 6.0 or newer can be exploited using the STRALGO LCS command to corrupt the heap and potentially result with remote code execution. This is a result of an incomplete fix by CVE-2021-29477. (Redis 6.2.4) + +- (CVE-2021-29478) An integer overflow bug in Redis 6.2 could be exploited to corrupt the heap and potentially result with remote code execution. The vulnerability involves changing the default set-max-intset-entries configuration value, creating a large set key that consists of integer values and using the COPY command to duplicate it. The integer overflow bug exists in all versions of Redis starting with 2.6, where it could result with a corrupted RDB or DUMP payload, but not exploited through COPY (which did not exist before 6.2). (Redis 6.2.3) + +- (CVE-2021-29477) An integer overflow bug in Redis version 6.0 or newer could be exploited using the STRALGO LCS command to corrupt the heap and potentially result in remote code execution. The integer overflow bug exists in all versions of Redis starting with 6.0. (Redis 6.2.3) + +{{< /collapsible >}} From fdad3ab9b4a2672aa9c9df37a76da8a3aa6a88e0 Mon Sep 17 00:00:00 2001 From: Kaitlyn Michael <76962844+kaitlynmichael@users.noreply.github.com> Date: Mon, 19 Oct 2026 14:11:36 -0500 Subject: [PATCH 42/76] RS: fix release weights (#4298) --- content/operate/rs/release-notes/rs-8-0-releases/rs-8-0-2-17.md | 2 +- content/operate/rs/release-notes/rs-8-0-releases/rs-8-0-2-41.md | 2 +- content/operate/rs/release-notes/rs-8-0-releases/rs-8-0-6-54.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/content/operate/rs/release-notes/rs-8-0-releases/rs-8-0-2-17.md b/content/operate/rs/release-notes/rs-8-0-releases/rs-8-0-2-17.md index 205d430022..35fc588dca 100644 --- a/content/operate/rs/release-notes/rs-8-0-releases/rs-8-0-2-17.md +++ b/content/operate/rs/release-notes/rs-8-0-releases/rs-8-0-2-17.md @@ -8,7 +8,7 @@ categories: compatibleOSSVersion: Redis 8.2.1, 8.0.2, 7.4.3, 7.2.7, 6.2.13 description: Redis Software 8! The most performant, most secure, and richest version so far. Built for performance, scale, and reliability to power modern ML and AI applications. linkTitle: 8.0.2-17 (October 2025) -weight: 90 +weight: 91 --- ​[​Redis Software version 8.0.2](https://redis.io/downloads/#Redis_Software) is now available! diff --git a/content/operate/rs/release-notes/rs-8-0-releases/rs-8-0-2-41.md b/content/operate/rs/release-notes/rs-8-0-releases/rs-8-0-2-41.md index 6f0fa85265..3537fe3320 100644 --- a/content/operate/rs/release-notes/rs-8-0-releases/rs-8-0-2-41.md +++ b/content/operate/rs/release-notes/rs-8-0-releases/rs-8-0-2-41.md @@ -8,7 +8,7 @@ categories: compatibleOSSVersion: Redis 8.2.1, 8.0.2, 7.4.3, 7.2.7, 6.2.13 description: Bug fixes for cluster_wd configuration after restarts, internode encryption certificate storage and retrieval, database creation, Lua script migration issues, and database endpoint assignment after node removal. linkTitle: 8.0.2-41 (November 2025) -weight: 89 +weight: 90 aliases: - /operate/rs/release-notes/rs-8-0-releases/rs-8-0-2-tba/ --- diff --git a/content/operate/rs/release-notes/rs-8-0-releases/rs-8-0-6-54.md b/content/operate/rs/release-notes/rs-8-0-releases/rs-8-0-6-54.md index b573cdac00..d7a02bfc7d 100644 --- a/content/operate/rs/release-notes/rs-8-0-releases/rs-8-0-6-54.md +++ b/content/operate/rs/release-notes/rs-8-0-releases/rs-8-0-6-54.md @@ -8,7 +8,7 @@ categories: compatibleOSSVersion: Redis 8.2.1, 8.0.2, 7.4.3, 7.2.7, 6.2.13 description: Single sign-on for the Cluster Manager UI. Error reports for Replica Of migration status. Automatically download and install user-defined modules during bootstrapping. linkTitle: 8.0.6-54 (December 2025) -weight: 88 +weight: 89 --- ​[​Redis Software version 8.0.6](https://redis.io/downloads/#Redis_Software) is now available! This release includes API enhancements that warranted a new minor version instead of a maintenance release for version 8.0.2. However, you can upgrade from 8.0.2 to 8.0.6 without issue. From 6bd6d869bb00847b133911e57d041c8cc7a0acc9 Mon Sep 17 00:00:00 2001 From: Kaitlyn Michael <76962844+kaitlynmichael@users.noreply.github.com> Date: Mon, 19 Oct 2026 14:13:26 -0500 Subject: [PATCH 43/76] K8s: 8.2.0 version (#4276) --- build/test_version_archiver.py | 42 + build/version_archiver.py | 27 +- content/operate/kubernetes/8.2.0/_index.md | 114 + .../kubernetes/8.2.0/active-active/_index.md | 92 + .../8.2.0/active-active/create-aa-crdb-cli.md | 217 + .../8.2.0/active-active/create-reaadb.md | 145 + .../8.2.0/active-active/edit-clusters.md | 164 + .../8.2.0/active-active/edit-rerc.md | 88 + .../8.2.0/active-active/global-config.md | 178 + .../8.2.0/active-active/global-db-secret.md | 76 + .../8.2.0/active-active/prepare-clusters.md | 147 + .../kubernetes/8.2.0/architecture/_index.md | 160 + .../8.2.0/architecture/deployment-options.md | 56 + .../kubernetes/8.2.0/deployment/_index.md | 62 + .../8.2.0/deployment/container-images.md | 259 + .../kubernetes/8.2.0/deployment/helm.md | 313 ++ .../8.2.0/deployment/openshift/_index.md | 43 + .../deployment/openshift/openshift-cli.md | 224 + .../openshift/openshift-operatorhub.md | 67 + .../8.2.0/deployment/quick-start.md | 277 + .../operate/kubernetes/8.2.0/flex/_index.md | 117 + .../kubernetes/8.2.0/flex/auto-tiering.md | 94 + .../kubernetes/8.2.0/flex/get-started.md | 135 + content/operate/kubernetes/8.2.0/flex/plan.md | 122 + .../operate/kubernetes/8.2.0/flex/scale.md | 146 + .../kubernetes/8.2.0/kubernetes-archive.md | 30 + .../operate/kubernetes/8.2.0/logs/_index.md | 48 + .../kubernetes/8.2.0/logs/collect-logs.md | 64 + .../kubernetes/8.2.0/networking/_index.md | 54 + .../8.2.0/networking/cluster-aware-clients.md | 91 + .../8.2.0/networking/database-connectivity.md | 304 + .../kubernetes/8.2.0/networking/ingress.md | 229 + .../8.2.0/networking/ingressorroutespec.md | 91 + .../8.2.0/networking/istio-ingress.md | 158 + .../kubernetes/8.2.0/networking/routes.md | 59 + .../kubernetes/8.2.0/re-clusters/_index.md | 65 + .../8.2.0/re-clusters/cluster-recovery.md | 53 + .../connect-prometheus-operator.md | 75 + .../re-clusters/connect-to-admin-console.md | 57 + .../re-clusters/delete-custom-resources.md | 136 + .../8.2.0/re-clusters/expand-pvc.md | 124 + .../8.2.0/re-clusters/multi-namespace.md | 156 + .../kubernetes/8.2.0/re-databases/_index.md | 58 + .../8.2.0/re-databases/db-controller.md | 126 + .../re-databases/enrich-metrics-with-tags.md | 137 + .../kubernetes/8.2.0/re-databases/modules.md | 348 ++ .../8.2.0/re-databases/replica-redb.md | 264 + .../8.2.0/recommendations/_index.md | 34 + .../8.2.0/recommendations/node-resources.md | 69 + .../8.2.0/recommendations/node-selection.md | 248 + .../recommendations/persistent-volumes.md | 103 + .../8.2.0/recommendations/pod-stability.md | 115 + .../recommendations/sizing-on-kubernetes.md | 121 + .../kubernetes/8.2.0/reference/_index.md | 115 + .../kubernetes/8.2.0/reference/api/_index.md | 63 + .../kubernetes-api-reference-template.tmpl | 105 + .../reference/api/redis_enterprise_acl_api.md | 128 + ...s_enterprise_active_active_database_api.md | 1906 +++++++ .../api/redis_enterprise_cluster_api.md | 4993 +++++++++++++++++ .../api/redis_enterprise_cluster_empty_fields | 8 + .../api/redis_enterprise_cluster_role_api.md | 179 + ...dis_enterprise_cluster_role_binding_api.md | 175 + .../api/redis_enterprise_database_api.md | 2118 +++++++ .../redis_enterprise_remote_cluster_api.md | 182 + .../api/redis_enterprise_role_api.md | 299 + .../api/redis_enterprise_role_binding_api.md | 175 + .../api/redis_enterprise_user_api.md | 501 ++ .../reference/supported_k8s_distributions.md | 78 + .../kubernetes/8.2.0/reference/yaml/_index.md | 89 + .../8.2.0/reference/yaml/active-active.md | 94 + .../8.2.0/reference/yaml/basic-deployment.md | 93 + .../reference/yaml/log-collector-rbac.md | 155 + .../8.2.0/reference/yaml/multi-namespace.md | 68 + .../8.2.0/reference/yaml/rack-awareness.md | 132 + .../kubernetes/8.2.0/security/_index.md | 39 + .../8.2.0/security/access-control/_index.md | 150 + .../security/access-control/manage-acls.md | 121 + .../access-control/manage-bindings.md | 174 + .../security/access-control/manage-roles.md | 200 + .../security/access-control/manage-users.md | 199 + .../migrate-rolespermissions.md | 194 + .../security/allow-resource-adjustment.md | 128 + .../8.2.0/security/authentication/_index.md | 47 + .../authentication/configuration-secrets.md | 156 + .../8.2.0/security/authentication/ldap.md | 169 + .../authentication/manage-rec-credentials.md | 134 + .../authentication/manage-redb-credentials.md | 70 + .../8.2.0/security/authentication/sso.md | 370 ++ .../8.2.0/security/certificates/_index.md | 48 + .../certificates/add-client-certificates.md | 45 + .../security/certificates/cert-manager.md | 363 ++ .../certificates/internode-encryption.md | 137 + .../certificates/manage-rec-certificates.md | 130 + .../operate/kubernetes/8.2.0/security/fips.md | 44 + .../kubernetes/8.2.0/security/vault.md | 558 ++ .../kubernetes/8.2.0/upgrade/_index.md | 53 + .../kubernetes/8.2.0/upgrade/openshift-cli.md | 285 + .../kubernetes/8.2.0/upgrade/upgrade-olm.md | 188 + .../8.2.0/upgrade/upgrade-redis-cluster.md | 310 + .../operate/kubernetes/kubernetes-archive.md | 1 + 100 files changed, 22745 insertions(+), 6 deletions(-) create mode 100644 content/operate/kubernetes/8.2.0/_index.md create mode 100644 content/operate/kubernetes/8.2.0/active-active/_index.md create mode 100644 content/operate/kubernetes/8.2.0/active-active/create-aa-crdb-cli.md create mode 100644 content/operate/kubernetes/8.2.0/active-active/create-reaadb.md create mode 100644 content/operate/kubernetes/8.2.0/active-active/edit-clusters.md create mode 100644 content/operate/kubernetes/8.2.0/active-active/edit-rerc.md create mode 100644 content/operate/kubernetes/8.2.0/active-active/global-config.md create mode 100644 content/operate/kubernetes/8.2.0/active-active/global-db-secret.md create mode 100644 content/operate/kubernetes/8.2.0/active-active/prepare-clusters.md create mode 100644 content/operate/kubernetes/8.2.0/architecture/_index.md create mode 100644 content/operate/kubernetes/8.2.0/architecture/deployment-options.md create mode 100644 content/operate/kubernetes/8.2.0/deployment/_index.md create mode 100644 content/operate/kubernetes/8.2.0/deployment/container-images.md create mode 100644 content/operate/kubernetes/8.2.0/deployment/helm.md create mode 100644 content/operate/kubernetes/8.2.0/deployment/openshift/_index.md create mode 100644 content/operate/kubernetes/8.2.0/deployment/openshift/openshift-cli.md create mode 100644 content/operate/kubernetes/8.2.0/deployment/openshift/openshift-operatorhub.md create mode 100644 content/operate/kubernetes/8.2.0/deployment/quick-start.md create mode 100644 content/operate/kubernetes/8.2.0/flex/_index.md create mode 100644 content/operate/kubernetes/8.2.0/flex/auto-tiering.md create mode 100644 content/operate/kubernetes/8.2.0/flex/get-started.md create mode 100644 content/operate/kubernetes/8.2.0/flex/plan.md create mode 100644 content/operate/kubernetes/8.2.0/flex/scale.md create mode 100644 content/operate/kubernetes/8.2.0/kubernetes-archive.md create mode 100644 content/operate/kubernetes/8.2.0/logs/_index.md create mode 100644 content/operate/kubernetes/8.2.0/logs/collect-logs.md create mode 100644 content/operate/kubernetes/8.2.0/networking/_index.md create mode 100644 content/operate/kubernetes/8.2.0/networking/cluster-aware-clients.md create mode 100644 content/operate/kubernetes/8.2.0/networking/database-connectivity.md create mode 100644 content/operate/kubernetes/8.2.0/networking/ingress.md create mode 100644 content/operate/kubernetes/8.2.0/networking/ingressorroutespec.md create mode 100644 content/operate/kubernetes/8.2.0/networking/istio-ingress.md create mode 100644 content/operate/kubernetes/8.2.0/networking/routes.md create mode 100644 content/operate/kubernetes/8.2.0/re-clusters/_index.md create mode 100644 content/operate/kubernetes/8.2.0/re-clusters/cluster-recovery.md create mode 100644 content/operate/kubernetes/8.2.0/re-clusters/connect-prometheus-operator.md create mode 100644 content/operate/kubernetes/8.2.0/re-clusters/connect-to-admin-console.md create mode 100644 content/operate/kubernetes/8.2.0/re-clusters/delete-custom-resources.md create mode 100644 content/operate/kubernetes/8.2.0/re-clusters/expand-pvc.md create mode 100644 content/operate/kubernetes/8.2.0/re-clusters/multi-namespace.md create mode 100644 content/operate/kubernetes/8.2.0/re-databases/_index.md create mode 100644 content/operate/kubernetes/8.2.0/re-databases/db-controller.md create mode 100644 content/operate/kubernetes/8.2.0/re-databases/enrich-metrics-with-tags.md create mode 100644 content/operate/kubernetes/8.2.0/re-databases/modules.md create mode 100644 content/operate/kubernetes/8.2.0/re-databases/replica-redb.md create mode 100644 content/operate/kubernetes/8.2.0/recommendations/_index.md create mode 100644 content/operate/kubernetes/8.2.0/recommendations/node-resources.md create mode 100644 content/operate/kubernetes/8.2.0/recommendations/node-selection.md create mode 100644 content/operate/kubernetes/8.2.0/recommendations/persistent-volumes.md create mode 100644 content/operate/kubernetes/8.2.0/recommendations/pod-stability.md create mode 100644 content/operate/kubernetes/8.2.0/recommendations/sizing-on-kubernetes.md create mode 100644 content/operate/kubernetes/8.2.0/reference/_index.md create mode 100644 content/operate/kubernetes/8.2.0/reference/api/_index.md create mode 100644 content/operate/kubernetes/8.2.0/reference/api/kubernetes-api-reference-template.tmpl create mode 100644 content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_acl_api.md create mode 100644 content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_active_active_database_api.md create mode 100644 content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md create mode 100644 content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_empty_fields create mode 100644 content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_role_api.md create mode 100644 content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_role_binding_api.md create mode 100644 content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md create mode 100644 content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_remote_cluster_api.md create mode 100644 content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_role_api.md create mode 100644 content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_role_binding_api.md create mode 100644 content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_user_api.md create mode 100644 content/operate/kubernetes/8.2.0/reference/supported_k8s_distributions.md create mode 100644 content/operate/kubernetes/8.2.0/reference/yaml/_index.md create mode 100644 content/operate/kubernetes/8.2.0/reference/yaml/active-active.md create mode 100644 content/operate/kubernetes/8.2.0/reference/yaml/basic-deployment.md create mode 100644 content/operate/kubernetes/8.2.0/reference/yaml/log-collector-rbac.md create mode 100644 content/operate/kubernetes/8.2.0/reference/yaml/multi-namespace.md create mode 100644 content/operate/kubernetes/8.2.0/reference/yaml/rack-awareness.md create mode 100644 content/operate/kubernetes/8.2.0/security/_index.md create mode 100644 content/operate/kubernetes/8.2.0/security/access-control/_index.md create mode 100644 content/operate/kubernetes/8.2.0/security/access-control/manage-acls.md create mode 100644 content/operate/kubernetes/8.2.0/security/access-control/manage-bindings.md create mode 100644 content/operate/kubernetes/8.2.0/security/access-control/manage-roles.md create mode 100644 content/operate/kubernetes/8.2.0/security/access-control/manage-users.md create mode 100644 content/operate/kubernetes/8.2.0/security/access-control/migrate-rolespermissions.md create mode 100644 content/operate/kubernetes/8.2.0/security/allow-resource-adjustment.md create mode 100644 content/operate/kubernetes/8.2.0/security/authentication/_index.md create mode 100644 content/operate/kubernetes/8.2.0/security/authentication/configuration-secrets.md create mode 100644 content/operate/kubernetes/8.2.0/security/authentication/ldap.md create mode 100644 content/operate/kubernetes/8.2.0/security/authentication/manage-rec-credentials.md create mode 100644 content/operate/kubernetes/8.2.0/security/authentication/manage-redb-credentials.md create mode 100644 content/operate/kubernetes/8.2.0/security/authentication/sso.md create mode 100644 content/operate/kubernetes/8.2.0/security/certificates/_index.md create mode 100644 content/operate/kubernetes/8.2.0/security/certificates/add-client-certificates.md create mode 100644 content/operate/kubernetes/8.2.0/security/certificates/cert-manager.md create mode 100644 content/operate/kubernetes/8.2.0/security/certificates/internode-encryption.md create mode 100644 content/operate/kubernetes/8.2.0/security/certificates/manage-rec-certificates.md create mode 100644 content/operate/kubernetes/8.2.0/security/fips.md create mode 100644 content/operate/kubernetes/8.2.0/security/vault.md create mode 100644 content/operate/kubernetes/8.2.0/upgrade/_index.md create mode 100644 content/operate/kubernetes/8.2.0/upgrade/openshift-cli.md create mode 100644 content/operate/kubernetes/8.2.0/upgrade/upgrade-olm.md create mode 100644 content/operate/kubernetes/8.2.0/upgrade/upgrade-redis-cluster.md diff --git a/build/test_version_archiver.py b/build/test_version_archiver.py index 08ddeccf87..55a56a428a 100644 --- a/build/test_version_archiver.py +++ b/build/test_version_archiver.py @@ -131,6 +131,46 @@ def test_other_products_use_their_own_prefix(): print("✓ redis-data-integration (integrate prefix) is versioned") +def test_links_into_older_versions_untouched(): + """A link from latest into an older snapshot must keep pointing there. + + Archiving kubernetes 8.2.0 turned /content/operate/kubernetes/7.22/... into + /content/operate/kubernetes/8.2.0/7.22/..., which does not exist. + """ + both = ('[m]({{< relref "/operate/kubernetes/7.22/re-clusters/auto-tiering" >}})\n' + '[n](/content/operate/kubernetes/7.22/re-clusters/auto-tiering.md)') + out = archive("kubernetes", "8.2.0", DEEP, both) + assert out == both, out + print("✓ links into older version directories are left alone") + + +def test_agent_and_os_files_not_copied(): + """Agent instruction files, .DS_Store and hidden dirs (.claude/) stay out of a version.""" + cwd = os.getcwd() + with tempfile.TemporaryDirectory() as tmp: + os.chdir(tmp) + try: + archiver = VersionArchiver("kubernetes", "9.9") + sub = os.path.join(archiver.latest, "flex") + os.makedirs(sub) + for name in ("AGENTS.md", "CLAUDE.md", "CLAUDE.local.md", ".DS_Store", "_index.md"): + open(os.path.join(archiver.latest, name), "w").close() + open(os.path.join(sub, "AGENTS.md"), "w").close() + open(os.path.join(sub, "plan.md"), "w").close() + hidden = os.path.join(sub, ".claude") + os.makedirs(hidden) + open(os.path.join(hidden, "state"), "w").close() + archiver.archive_version() + copied = sorted( + os.path.relpath(os.path.join(r, f), archiver.new_directory) + for r, _, fs in os.walk(archiver.new_directory) for f in fs + ) + finally: + os.chdir(cwd) + assert copied == ["_index.md", os.path.join("flex", "plan.md")], copied + print("✓ agent instruction and OS files are not copied into the version") + + def main(): tests = [ test_relref_is_versioned, @@ -141,6 +181,8 @@ def main(): test_already_versioned_is_idempotent, test_other_product_and_external_urls_untouched, test_other_products_use_their_own_prefix, + test_links_into_older_versions_untouched, + test_agent_and_os_files_not_copied, ] try: for t in tests: diff --git a/build/version_archiver.py b/build/version_archiver.py index d2f1abb7f7..22389f2bd1 100644 --- a/build/version_archiver.py +++ b/build/version_archiver.py @@ -1,5 +1,15 @@ import os, shutil, re, argparse +# Files that live in the latest tree but are not documentation. Agent +# instruction files are loaded by directory, so a frozen copy inside a version +# would shadow the live rules for anyone editing that version. +EXCLUDED_FILES = {"AGENTS.md", "CLAUDE.md", "CLAUDE.local.md", ".DS_Store"} + + +def is_version_dir(name): + """Same rule archive_version uses to skip other releases: any digit.""" + return any(char.isdigit() for char in name) + class VersionArchiver: def __init__(self, product, version): @@ -24,7 +34,9 @@ def archive_version(self): dirs[:] = [ d for d in dirs - if not any(char.isdigit() for char in d) and d != "release-notes" + if not is_version_dir(d) + and d != "release-notes" + and not d.startswith(".") # local tool state such as .claude/ ] # Create the corresponding destination directory if it does not exist @@ -34,6 +46,8 @@ def archive_version(self): os.makedirs(dest_dir) for file in files: + if file in EXCLUDED_FILES: + continue src_file = os.path.join(root, file) dest_file = os.path.join(dest_dir, file) shutil.copy2(src_file, dest_file) @@ -74,11 +88,12 @@ def replace_link(match): full_match = match.group(0) # The entire match link = match.group(1) # The actual relref link - # Check if the link contains 'release-notes' and whether the replacement has already happened - if ( - "release-notes" not in link - and f"/{self.prefix}/{product}/{version}" not in link - ): + # Leave release notes alone, and any link that already points + # into a version directory: this one (idempotency) or an older + # snapshot such as 7.22/, which must not become /7.22/. + after_product = link.split(f"/{self.prefix}/{product}/", 1)[-1] + first_segment = after_product.split("/", 1)[0] + if "release-notes" not in link and not is_version_dir(first_segment): # Otherwise, replace it new_link = link.replace( f"/{self.prefix}/{product}/", diff --git a/content/operate/kubernetes/8.2.0/_index.md b/content/operate/kubernetes/8.2.0/_index.md new file mode 100644 index 0000000000..ad6693bf53 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/_index.md @@ -0,0 +1,114 @@ +--- +Title: Redis Enterprise for Kubernetes +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Deploy and manage Redis Enterprise on Kubernetes with the Redis Enterprise operator. +hideListLinks: true +linkTitle: 8.2.0 +weight: 50 +bannerText: This documentation applies to version 8.2.0. For documentation on the latest version, see [redis.io/docs/latest/operate/kubernetes/](https://redis.io/docs/latest/operate/kubernetes/). +bannerChildren: true +url: '/operate/kubernetes/8.2.0/' +--- + +Redis Enterprise for Kubernetes brings Redis Enterprise to Kubernetes environments through the Redis Enterprise operator. You can deploy, scale, and manage Redis Enterprise clusters and databases by using native Kubernetes resources and workflows. + +Redis Enterprise for Kubernetes provides all the enterprise features of Redis Software: + +- Linear scalability with Redis clustering +- High availability with automatic failover +- Active-Active geo-distribution +- Redis Flex for cost optimization +- Enterprise-grade security and encryption +- 24/7 support + +The Redis Enterprise operator simplifies deployment and management by providing custom resource definitions (CRDs) for Redis Enterprise clusters (REC) and databases (REDB). This approach enables GitOps workflows and Kubernetes-native operations. + +Redis Enterprise for Kubernetes is compatible with [CNCF-conformant](https://www.cncf.io/training/certification/software-conformance/) Kubernetes platforms. The operator follows standard Kubernetes APIs and practices and is designed to run consistently across certified Kubernetes environments. + +## Get started + +Deploy Redis Enterprise on your Kubernetes cluster and create your first database. + +- [Quick start deployment](/content/operate/kubernetes/8.2.0/deployment/quick-start.md) +- [Deploy with Helm](/content/operate/kubernetes/8.2.0/deployment/helm.md) +- [Deploy on OpenShift](/content/operate/kubernetes/8.2.0/deployment/openshift/_index.md) +- [Supported Kubernetes distributions](/content/operate/kubernetes/8.2.0/reference/supported_k8s_distributions.md) + +## Redis Enterprise clusters (REC) + +Create and manage [Redis Enterprise clusters](/content/operate/kubernetes/8.2.0/re-clusters/_index.md) on Kubernetes. + +- [Connect to admin console](/content/operate/kubernetes/8.2.0/re-clusters/connect-to-admin-console.md) +- [Redis Flex](/content/operate/kubernetes/8.2.0/flex/_index.md) +- [Multi-namespace deployment](/content/operate/kubernetes/8.2.0/re-clusters/multi-namespace.md) +- [Cluster recovery](/content/operate/kubernetes/8.2.0/re-clusters/cluster-recovery.md) +- [REC API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md) + +## Redis Enterprise databases (REDB) + +Create and manage [Redis Enterprise databases](/content/operate/kubernetes/8.2.0/re-databases/_index.md) using Kubernetes resources. + +- [Database controller](/content/operate/kubernetes/8.2.0/re-databases/db-controller.md) +- [Create replica databases](/content/operate/kubernetes/8.2.0/re-databases/replica-redb.md) +- [REDB API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md) + +## Active-Active databases + +Set up globally distributed [Active-Active databases](/content/operate/kubernetes/8.2.0/active-active/_index.md) across multiple Kubernetes clusters. + +- [Prepare participating clusters](/content/operate/kubernetes/8.2.0/active-active/prepare-clusters.md) +- [Create Active-Active database](/content/operate/kubernetes/8.2.0/active-active/create-reaadb.md) +- [Global configuration](/content/operate/kubernetes/8.2.0/active-active/global-config.md) +- [REAADB API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_active_active_database_api.md) +- [Remote cluster API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_remote_cluster_api.md) + +## Security + +Manage [secure connections](/content/operate/kubernetes/8.2.0/security/_index.md) and access control for your Redis Enterprise deployment. + +- [Access control](/content/operate/kubernetes/8.2.0/security/access-control/_index.md) +- [Manage REC credentials](/content/operate/kubernetes/8.2.0/security/authentication/manage-rec-credentials.md) +- [Manage REC certificates](/content/operate/kubernetes/8.2.0/security/certificates/manage-rec-certificates.md) +- [Internode encryption](/content/operate/kubernetes/8.2.0/security/certificates/internode-encryption.md) +- [LDAP authentication](/content/operate/kubernetes/8.2.0/security/authentication/ldap.md) + +## Reference + +Use the Kubernetes API and command-line tools to manage your Redis Enterprise deployment. + +- [Redis Enterprise cluster API (REC)](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md) +- [Redis Enterprise database API (REDB)](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md) +- [Active-Active database API (REAADB)](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_active_active_database_api.md) +- [Remote cluster API (RERC)](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_remote_cluster_api.md) + +## Logs & monitoring + +Monitor and troubleshoot your Redis Enterprise deployment. + +- [Collect logs](/content/operate/kubernetes/8.2.0/logs/collect-logs.md) +- [Connect to Prometheus operator](/content/operate/kubernetes/8.2.0/re-clusters/connect-prometheus-operator.md) + +## Upgrade + +Keep your Redis Enterprise deployment up to date. + +- [Upgrade Redis cluster](/content/operate/kubernetes/8.2.0/upgrade/upgrade-redis-cluster.md) +- [Upgrade with OpenShift CLI](/content/operate/kubernetes/8.2.0/upgrade/openshift-cli.md) +- [Upgrade with OLM](/content/operate/kubernetes/8.2.0/upgrade/upgrade-olm.md) + +## Release notes + +Stay informed about new features, enhancements, and fixes. + +- [Release notes](/content/operate/kubernetes/release-notes/_index.md) + +## Related info + +- [Redis Enterprise Software](/content/operate/rs/_index.md) +- [Redis Cloud](/content/operate/rc/_index.md) +- [Redis Open Source](/content/operate/oss_and_stack/_index.md) +- [Glossary](/content/glossary/_index.md) \ No newline at end of file diff --git a/content/operate/kubernetes/8.2.0/active-active/_index.md b/content/operate/kubernetes/8.2.0/active-active/_index.md new file mode 100644 index 0000000000..bc561fd03d --- /dev/null +++ b/content/operate/kubernetes/8.2.0/active-active/_index.md @@ -0,0 +1,92 @@ +--- +Title: Active-Active databases +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Create and manage Active-Active Redis Enterprise databases across multiple Kubernetes clusters. +hideListLinks: true +linkTitle: Active-Active databases +weight: 40 +url: '/operate/kubernetes/8.2.0/active-active/' +--- + +Redis Enterprise [Active-Active](/content/operate/rs/databases/active-active/_index.md) databases on Kubernetes provide read and write access to the same dataset from different Kubernetes clusters. This enables globally distributed applications with local read and write access, automatic conflict resolution, and seamless failover capabilities. + +Active-Active databases use multi-master replication to keep data synchronized across participating clusters, allowing applications to read and write data locally while maintaining global consistency. + +## Active-Active setup methods + +There are two methods for creating an Active-Active database with Redis Enterprise for Kubernetes: + +- The `RedisEnterpriseActiveActiveDatabase` (REAADB) custom resource is available for versions 6.4.2 and later. +- The `crdb-cli` method is available for versions 6.4.2 or earlier. + + +We recommend creating new Active-Active databases using the RedisEnterpriseActiveActiveDatabase (REAADB) custom resource. This allows you to manage your Active-Active database with the operator and ensures you have the latest features and functionality. + +### Active-Active controller method + +Versions 6.4.2-6 or later fully support the Active-Active controller. Some of these features were available as a preview in 6.4.2-4 and 6.4.2-5. Please upgrade to 6.4.2-6 for the full set of general availability features and bug fixes. + +This setup method includes the following steps: + +1. Gather REC credentials and [prepare participating clusters](/content/operate/kubernetes/8.2.0/active-active/prepare-clusters.md). +2. Create [`RedisEnterpriseRemoteCluster` (RERC)](/content/operate/kubernetes/8.2.0/active-active/create-reaadb.md#create-rerc) resources. +3. Create [`RedisEnterpriseActiveActiveDatabase` (REAADB)](/content/operate/kubernetes/8.2.0/active-active/create-reaadb.md#create-reaadb) resource. + +### `crdb-cli` method + +For versions 6.4.2 or earlier, this Active-Active setup method includes the following steps: + +1. Install and configure an ingress. +2. Gather configuration details. +3. Add the `ActiveActive` field to the REC spec. +4. Create the database with the `crdb-cli` tool. + +## Redis Enterprise Active-Active controller for Kubernetes + +> [!NOTE] +> These features are supported for general availability in releases 6.4.2-6 and later. + +[Active-Active](/content/operate/rs/databases/active-active/_index.md) databases give you read-and-write access to Redis Enterprise clusters (REC) in different Kubernetes clusters or namespaces. Active-Active deployments managed by the Redis Enterprise operator require two additional custom resources: Redis Enterprise Active-Active database (REAADB) and Redis Enterprise remote cluster (RERC). + +To create an Active-Active Redis Enterprise deployment for Kubernetes with these new features, first [prepare participating clusters](/content/operate/kubernetes/8.2.0/active-active/prepare-clusters.md) then [create an Active-Active database](/content/operate/kubernetes/8.2.0/active-active/create-reaadb.md). + +### REAADB custom resource + +Redis Enterprise Active-Active database (REAADB) contains a link to the RERC for each participating cluster, and provides configuration and status to the management plane. + +For a full list of fields and options, see the [REAADB API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_active_active_database_api.md). + +For examples, see the [YAML examples](/content/operate/kubernetes/8.2.0/reference/yaml/active-active.md) section. + +### RERC custom resource + +Redis Enterprise remote cluster (RERC) custom resource contains configuration details for all the participating clusters. + +For a full list of fields and options, see the [RERC API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_remote_cluster_api.md). + +For examples, see the [YAML examples](/content/operate/kubernetes/8.2.0/reference/yaml/active-active.md) section. + +### Manage certificates + +The operator automates Active-Active certificate updates. When you update the proxy or syncer certificate secret on a participating cluster's REC, the operator detects the change and propagates the new certificate to the other participating clusters. + +For details, see [Manage REC certificates](/content/operate/kubernetes/8.2.0/security/certificates/manage-rec-certificates.md) and [cert-manager integration](/content/operate/kubernetes/8.2.0/security/certificates/cert-manager.md). + +### Limitations + +* Existing Redis databases cannot be migrated to a REAADB. (DOC-3594) +* Admission is not blocking REAADB with `shardCount` which exceeds license quota. (RED-96301) + Workaround: Fix the problems with the REAADB and reapply. +* The `/` value must be unique for each RERC resource. (RED-96302) +* Only global database options are supported, no support for specifying configuration per location. +* No support for migration from old (`crdb-cli`) Active-Active database method to new Active-Active controller. +* No support for REAADB with participating clusters co-located within the same Kubernetes cluster, except for a single designated local participating cluster. +* Active-Active databases cannot be configured as [Redis Flex](/content/operate/kubernetes/8.2.0/flex/_index.md) deployments. + +## More info + +For more general information about Active-Active, see the [Redis Enterprise Software docs](/content/operate/rs/databases/active-active/_index.md). diff --git a/content/operate/kubernetes/8.2.0/active-active/create-aa-crdb-cli.md b/content/operate/kubernetes/8.2.0/active-active/create-aa-crdb-cli.md new file mode 100644 index 0000000000..3cc2e2ed99 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/active-active/create-aa-crdb-cli.md @@ -0,0 +1,217 @@ +--- +Title: Create Active-Active databases with crdb-cli +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: This section shows how to set up an Active-Active Redis Enterprise database + on Kubernetes using the Redis Enterprise Software operator. +linkTitle: Create Active-Active with crdb-cli +weight: 99 +url: '/operate/kubernetes/8.2.0/active-active/create-aa-crdb-cli/' +--- +> [!NOTE] +> Versions 6.4.2 and later support the Active-Active database controller. This controller allows you to create Redis Enterprise Active-Active databases (REAADB) and Redis Enterprise remote clusters (RERC) with custom resources. We recommend using the [REAADB method for creating Active-Active databases](/content/operate/kubernetes/8.2.0/active-active/create-reaadb.md). + +On Kubernetes, Redis Enterprise [Active-Active](/content/operate/rs/databases/active-active/_index.md) databases provide read-and-write access to the same dataset from different Kubernetes clusters. For more general information about Active-Active, see the [Redis Enterprise Software docs](/content/operate/rs/databases/active-active/_index.md). + +Creating an Active-Active database requires routing [network access](/content/operate/kubernetes/8.2.0/networking/_index.md) between two Redis Enterprise clusters residing in different Kubernetes clusters. Without the proper access configured for each cluster, syncing between the databases instances will fail. + +This process consists of: + +1. Documenting values to be used in later steps. It's important these values are correct and consistent. +1. Editing the Redis Enterprise cluster (REC) spec file to include the `ActiveActive` section. This will be slightly different depending on the K8s distribution you are using. +1. Creating the database with the `crdb-cli` command. These values must match up with values in the REC resource spec. + +## Prerequisites + +Before creating Active-Active databases, you'll need admin access to two or more working Kubernetes clusters that each have: + +- Routing for external access with an [ingress resources](/content/operate/kubernetes/8.2.0/networking/ingress.md) (or [route resources](/content/operate/kubernetes/8.2.0/networking/routes.md) on OpenShift). +- A working [Redis Enterprise cluster (REC)](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md) with a unique name. +- Enough memory resources available for the database (see [hardware requirements](/content/operate/rs/installing-upgrading/install/plan-deployment/hardware-requirements.md)). + +> [!NOTE] +> The `activeActive` field and the `ingressOrRouteSpec` field cannot coexist in the same REC. If you configured your ingress via the `ingressOrRouteSpec` field in the REC, create your Active-Active database with the RedisEnterpriseActiveActiveDatabase (REAADB) custom resource. + +## Document required parameters + +The most common mistake when setting up Active-Active databases is incorrect or inconsistent parameter values. The values listed in the resource file must match those used in the crdb-cli command. + +- **Database name** ``: + - Description: Combined with ingress suffix to create the Active-Active database hostname + - Format: string + - Example value: `myaadb` + - How you get it: you choose + - The database name requirements are: + - Maximum of 63 characters + - Only letter, number, or hyphen (-) characters + - Starts with a letter; ends with a letter or digit. + - Database name is not case-sensitive + +You'll need the following information for each participating Redis Enterprise cluster (REC): + +> [!NOTE] +> You'll need to create DNS aliases to resolve your API hostname ``,``, `` to the IP address for the ingress controller’s LoadBalancer (or routes in Openshift) for each database. To avoid entering multiple DNS records, you can use a wildcard in your alias (such as *.ijk.example.com). + +- **REC hostname** ``: + - Description: Hostname used to identify your Redis Enterprise cluster in the `crdb-cli` command. This MUST be different from other participating clusters. + - Format: `..svc.cluster.local` + - Example value: `rec01.ns01.svc.cluster.local` + - How to get it: List all your Redis Enterprise clusters + ```bash + kubectl get rec + ``` +- **API hostname** ``: + - Description: Hostname used to access the Redis Enterprise cluster API from outside the K8s cluster + - Format: string + - Example value: `api.ijk.example.com` +- **Ingress suffix** ``: + - Description: Combined with database name to create the Active-Active database hostname + - Format: string + - Example value: `-cluster.ijk.example.com` +- [**REC admin credentials**](/content/operate/kubernetes/8.2.0/security/authentication/manage-rec-credentials.md) ` `: + - Description: Admin username and password for the REC stored in a secret + - Format: string + - Example value: username: `user@example.com`, password: `something` + - How to get them: + ```sh + kubectl get secret \ + -o jsonpath='{.data.username}' | base64 --decode + kubectl get secret \ + -o jsonpath='{.data.password}' | base64 --decode + ``` +- **Replication hostname** ``: + - Description: Hostname used inside the ingress for the database + - Format: `` + - Example value: `myaadb-cluster.ijk.example.com` + - How to get it: Combine `` and ` values you documented above. +- **Replication endpoint** ``: + - Description: Endpoint used externally to contact the database + - Format: `:443` + - Example value: `myaadb-cluster.ijk.example.com:443` + - How to get it:`:443` + +## Add `activeActive` section to the REC resource file + +From inside your K8s cluster, edit your Redis Enterprise cluster (REC) resource to add the following to the `spec` section. Do this for each participating cluster. + + The operator uses the API hostname (``) to create an ingress to the Redis Enterprise cluster's API; this only happens once per cluster. Every time a new Active-Active database instance is created on this cluster, the operator creates a new ingress route to the database with the ingress suffix (``). The hostname for each new database will be in the format ``. + +### Using ingress controller + +1. If your cluster uses an [ingress controller](/content/operate/kubernetes/8.2.0/networking/ingress.md), add the following to the `spec` section of your REC resource file. + + Nginx: + + ```sh + activeActive: + apiIngressUrl: + dbIngressSuffix: + ingressAnnotations: + kubernetes.io/ingress.class: nginx + nginx.ingress.kubernetes.io/backend-protocol: HTTPS + nginx.ingress.kubernetes.io/ssl-passthrough: "true" + method: ingress + ``` + +HAproxy: + + ```sh + activeActive: + apiIngressUrl: + dbIngressSuffix: + ingressAnnotations: + kubernetes.io/ingress.class: haproxy + ingress.kubernetes.io/ssl-passthrough: "true" + method: ingress + ``` + +2. After the changes are saved and applied, you can verify a new ingress was created for the API. + + ```sh + $ kubectl get ingress + NAME HOSTS ADDRESS PORTS AGE + rec01 api.abc.cde.example.com 225161f845b278-111450635.us.cloud.com 80 24h + ``` + +3. Verify you can access the API from outside the K8s cluster. + + ```sh + curl -k -L -i -u : https:///v1/cluster + ``` + + If the API call fails, create a DNS alias that resolves your API hostname (``) to the IP address for the ingress controller's LoadBalancer. + +4. Make sure you have DNS aliases for each database that resolve your API hostname ``,``, `` to the IP address of the ingress controller’s LoadBalancer. To avoid entering multiple DNS records, you can use a wildcard in your alias (such as `*.ijk.example.com`). + +#### If using Istio Gateway and VirtualService + +No changes are required to the REC spec if you are using [Istio](/content/operate/kubernetes/8.2.0/networking/istio-ingress.md) in place of an ingress controller. The `activeActive` section added above creates ingress resources. The two custom resources used to configure Istio (Gateway and VirtualService) replace the need for ingress resources. + +> [!WARNING] +> These custom resources are not controlled by the operator and will need to be configured and maintained manually. + +For each cluster, verify the VirtualService resource has two `- match:` blocks in the `tls` section. The hostname under `sniHosts:` should match your ``. + +### Using OpenShift routes + +1. Make sure your Redis Enterprise cluster (REC) has a different name (``) than any other participating clusters. If not, you'll need to manually rename the REC or move it to a different namespace. + You can check your new REC name with: + ```sh + oc get rec -o jsonpath='{.items[0].metadata.name}' + ``` + + If the rec name was modified, reapply [scc.yaml](https://github.com/RedisLabs/redis-enterprise-k8s-docs/blob/master/openshift/scc.yaml) to the namespace to reestablish security privileges. + + ```sh + oc apply -f scc.yaml + oc adm policy add-scc-to-group redis-enterprise-scc-v2 system:serviceaccounts: + ``` + + Releases before 6.4.2-6 use the earlier version of the SCC, named `redis-enterprise-scc`. + +1. Make sure you have DNS aliases for each database that resolve your API hostname ``,``, `` to the route IP address. To avoid entering multiple DNS records, you can use a wildcard in your alias (such as `*.ijk.example.com`). + +1. If your cluster uses [OpenShift routes](/content/operate/kubernetes/8.2.0/networking/routes.md), add the following to the `spec` section of your Redis Enterprise cluster (REC) resource file. + + ```sh + activeActive: + apiIngressUrl: + dbIngressSuffix: + method: openShiftRoute + ``` + +1. Make sure you have DNS aliases that resolve to the routes IP for both the API hostname (``) and the replication hostname (``) for each database. To avoid entering each database individually, you can use a wildcard in your alias (such as `*.ijk.example.com`). + +1. After the changes are saved and applied, you can see that a new route was created for the API. + + ```sh + $ oc get route + NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD + rec01 api-openshift.apps.abc.example.com rec01 api passthrough None + ``` + +## Create an Active-Active database with `crdb-cli` + +The `crdb-cli` command can be run from any Redis Enterprise pod hosted on any participating K8s cluster. You'll need the values for the [required parameters](/content/operate/kubernetes/8.2.0/active-active/create-aa-crdb-cli.md#document-required-parameters) for each Redis Enterprise cluster. + +```sh +crdb-cli crdb create \ + --name \ + --memory-size \ + --encryption yes \ + --instance fqdn=,url=https://,username=,password=,replication_endpoint=,replication_tls_sni= \ + --instance fqdn=,url=https://,username=,password=,replication_endpoint=,replication_tls_sni= +``` + +To create a database that syncs between more than two instances, add additional `--instance` arguments. + +See the [`crdb-cli` reference](/content/operate/rs/references/cli-utilities/crdb-cli/_index.md) for more options. + +## Test your database + +The easiest way to test your Active-Active database is to set a key-value pair in one database and retrieve it from the other. + +You can connect to your databases with the instructions in [Manage databases](/content/operate/kubernetes/8.2.0/re-databases/db-controller.md#connect-to-a-database). Set a test key with `SET foo bar` in the first database. If your Active-Active deployment is working properly, when connected to your second database, `GET foo` should output `bar`. diff --git a/content/operate/kubernetes/8.2.0/active-active/create-reaadb.md b/content/operate/kubernetes/8.2.0/active-active/create-reaadb.md new file mode 100644 index 0000000000..c5cfdd7d9e --- /dev/null +++ b/content/operate/kubernetes/8.2.0/active-active/create-reaadb.md @@ -0,0 +1,145 @@ +--- +Title: Create Active-Active database (REAADB) +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: null +linkTitle: Create database +weight: 30 +url: '/operate/kubernetes/8.2.0/active-active/create-reaadb/' +--- + + + +## Prerequisites + +To create an Active-Active database, make sure you've completed all the following steps and have gathered the information listed below each step. + +1. Configure the [admission controller and ValidatingWebhook](/content/operate/kubernetes/8.2.0/deployment/quick-start.md#enable-the-admission-controller/). + > [!NOTE] + > These are installed and enabled by default on clusters created via the OpenShift OperatorHub. + +2. Create two or more [RedisEnterpriseCluster (REC) custom resources](/content/operate/kubernetes/8.2.0/deployment/quick-start.md#create-a-redis-enterprise-cluster-rec) with enough [memory resources](/content/operate/rs/installing-upgrading/install/plan-deployment/hardware-requirements.md). + * Name of each REC (``) + * Namespace for each REC (``) + +3. Configure the REC [`ingressOrRoutes` field](/content/operate/kubernetes/8.2.0/networking/ingressorroutespec.md) and [create DNS records](/content/operate/kubernetes/8.2.0/networking/ingressorroutespec.md#configure-dns/). + * REC API hostname (`api--.`) + * Database hostname suffix (`.db--.`) + +4. [Prepare participating clusters](/content/operate/kubernetes/8.2.0/active-active/prepare-clusters.md) + * RERC name (`) + * RERC secret name (`redis-enterprise-`) + +For a list of example values used throughout this article, see the [Example values](#example-values) section. + +## Create RERC custom resources {#create-rerc} + +1. Create a `RedisEnterpriseRemoteCluster` (RERC) custom resource file for each participating Redis Enterprise cluster (REC). + +Below are examples of RERC resources for two participating clusters. Substitute your own values to create your own resource. + +Example RERC (`rerc-ohare`) for the REC named `rec-chicago` in the namespace `ns-illinois`: + +{{}} + +Example RERC (`rerc-raegan`) for the REC named `rec-arlington` in the namespace `ns-virginia`: + +{{}} + +For more details on RERC fields, see the [RERC API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_remote_cluster_api.md). + +2. Create a Redis Enterprise remote cluster from each RERC custom resource file. + +```sh +kubectl create -f +``` + +3. Check the status of your RERC. If `STATUS` is `Active` and `SPEC STATUS` is `Valid`, then your configurations are correct. + +```sh +kubectl get rerc +``` + +The output should look similar to: + +```sh +kubectl get rerc rerc-ohare + +NAME STATUS SPEC STATUS LOCAL +rerc-ohare Active Valid true +``` + +In case of errors, review the RERC custom resource events and the Redis Enterprise operator logs. + +## Create the REAADB {#create-reaadb} + +1. Create a `RedisEnterpriseActiveActiveDatabase` (REAADB) custom resource file meeting the naming requirements and listing the names of the RERC custom resources created in the last step. + +Naming requirements: +* less than 63 characters +* contains only lowercase letters, numbers, or hyphens +* starts with a letter +* ends with a letter or digit +* **must be unique across all participating clusters** + + The admission controller prevents duplicate database names when databases are created via the Kubernetes operator. Ensure database names are unique across all participating clusters to avoid service deletion and database unavailability. + +Example REAADB named `reaadb-boeing` linked to the REC named `rec-chicago` with two participating clusters and a global database configuration with shard count set to 3: + +{{}} + +> [!NOTE] +> Sharding is disabled on Active-Active databases created with a `shardCount` of 1. Sharding cannot be enabled after database creation. + +For more details on RERC fields, see the [RERC API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_remote_cluster_api.md). + +2. Create a Redis Enterprise Active-Active database from the REAADB custom resource file. + +```sh +kubectl create -f +``` + +3. Check the status of your RERC. If `STATUS` is `Active` and `SPEC STATUS` is `Valid`, your configurations are correct. + +```sh +kubectl get reaadb +``` + +The output should look similar to: + +```sh +kubectl get reaadb reaadb-boeing + +NAME STATUS SPEC STATUS LINKED REDBS REPLICATION STATUS +reaadb-boeing active Valid up +``` + +In case of errors, review the REAADB custom resource events and the Redis Enterprise operator logs. + +To add tags to an Active-Active database and expose them as metric labels, see [Enrich database metrics with tags](/content/operate/kubernetes/8.2.0/re-databases/enrich-metrics-with-tags.md). + +## Example values + +This article uses the example values listed below. You can also find them in the [YAML examples](/content/operate/kubernetes/8.2.0/reference/yaml/active-active.md) section. + +Example cluster 1: + +* REC name: `rec-chicago` +* REC namespace: `ns-illinois` +* RERC name: `rerc-ohare` +* RERC secret name: `redis-enterprise-rerc-ohare` +* API FQDN: `api-rec-chicago-ns-illinois.example.com` +* DB FQDN suffix: `.db-rec-chicago-ns-illinois.example.com` + +Example cluster 2: + +* REC name: `rec-arlington` +* REC namespace: `ns-virginia` +* RERC name: `rerc-raegan` +* RERC secret name: `redis-enterprise-rerc-reagan` +* API FQDN: `api-rec-arlington-ns-virginia.example.com` +* DB FQDN suffix: `.db-rec-arlington-ns-virginia.example.com` + diff --git a/content/operate/kubernetes/8.2.0/active-active/edit-clusters.md b/content/operate/kubernetes/8.2.0/active-active/edit-clusters.md new file mode 100644 index 0000000000..593f53a72a --- /dev/null +++ b/content/operate/kubernetes/8.2.0/active-active/edit-clusters.md @@ -0,0 +1,164 @@ +--- +Title: Edit participating clusters for Active-Active database +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Steps to add or remove a participating cluster to an existing Active-Active + database with Redis Enterprise for Kubernetes. +linkTitle: Edit participating clusters +weight: 40 +url: '/operate/kubernetes/8.2.0/active-active/edit-clusters/' +--- +> [!NOTE] +> This feature is supported for general availability in releases 6.4.2-6 and later. Some of these features were available as a preview in 6.4.2-4 and 6.4.2-5. Please upgrade to 6.4.2-6 for the full set of general availability features and bug fixes. and later. + +## Add a participating cluster + +Use the following steps to add a participating cluster to an existing Redis Enterprise Active-Active database (REAADB) for Kubernetes. + +### Prerequisites + +To prepare the Redis Enterprise cluster (REC) to participate in an Active-Active database, perform the following tasks from [Prepare participating clusters](/content/operate/kubernetes/8.2.0/active-active/prepare-clusters.md): + +- Make sure the cluster meets the hardware and naming requirements. +- Enable the Active-Active controllers. +- Configure external routing. +- Configure `ValidatingWebhookConfiguration`. + +### Collect REC credentials + +To communicate with other clusters, all participating clusters need access to the admin credentials for all other clusters. + +1. Get the REC credentials secret for the new participating cluster. + + ```sh + kubectl get secret -o yaml + ``` + + This example shows an admin credentials secret for an REC named `rec-boston`: + + ```yaml + apiVersion: v1 + data: + password: ABcdef12345 + username: GHij56789 + kind: Secret + metadata: + name: rec-boston + type: Opaque + ``` + +1. Create a secret for the new participating cluster named `redis-enterprise-` and add the username and password. + + The example below shows a secret file for a remote cluster named `rerc-logan` . + + ```yaml + apiVersion: v1 + data: + password: ABcdef12345 + username: GHij56789 + kind: Secret + metadata: + name: redis-enterprise-rerc-logan + type: Opaque + ``` + +1. Apply the file of collected secrets to every participating REC. + + ```sh + kubectl apply -f + ``` + + If the admin credentials for any of the clusters change, update and reapply the file to all clusters. + +### Create RERC + +1. From one of the existing participating clusters, create a `RedisEnterpriseRemoteCluster` (RERC) custom resource for the new participating cluster. + + This example (rerc-logan) shows an RERC custom resource for an REC named `rec-boston` in the namespace `ns-massachusetts`. + + ```yaml + apiVersion: app.redislabs.com/v1alpha1 + kind: RedisEnterpriseRemoteCluster + metadata: + name: rerc-logan + spec: + recName: rec-boston + recNamespace: ns-massachusetts + apiFqdnUrl: test-example-api-rec-boston-ns-massachusetts.example.com + dbFqdnSuffix: .example-cluster-rec-boston-ns-massachusetts.example.com + secretName: redis-enterprise-rerc-logan + ``` + +1. Create the RERC custom resource. + + ```sh + kubectl create -f + ``` + +1. Check the status of the newly created RERC custom resource. + + ```sh + kubectl get rerc + ``` + + The output should look like this: + + ```sh + NAME STATUS SPEC STATUS LOCAL + rerc-logan Active Valid true + ``` + +### Edit REAADB spec + +1. Patch the REAADB spec to add the new RERC name to the `participatingClusters`, replacing `` and `` with your own values. + + ```sh + kubectl patch reaadb < --type merge --patch '{"spec": {"participatingClusters": [{"name": ""}]}}' + ``` + +1. View the REAADB `participatingClusters` status to verify the cluster was added. + + ```sh + kubectl get reaadb -o=jsonpath='{.status.participatingClusters}' + ``` + + The output should look like this: + + ```sh + [{"id":1,"name":"rerc-ohare"},{"id":2,"name":"rerc-reagan"},{"id":3,"name":"rerc-logan"}] + ``` + +## Remove a participating cluster + +1. On an existing participating cluster,remove the desired cluster from the `participatingCluster` section of the REAADB spec. + + ```sh + kubectl edit reaadb + ``` + +1. On each of the other participating clusters, verify the status is `active` and the spec status is `Valid` and the cluster was removed. + + ```sh + kubectl get reaadb -o=jsonpath='{.status}' + ``` + + The output should look like this: + + ```sh + {... ,"participatingClusters":[{"id":1,"name":"rerc1"},{"id":2,"name":"rerc2"}],"redisEnterpriseCluster":"rec1","specStatus":"Valid","status":"active"} + ``` + +1. On the removed participating cluster, list all REAADB resources on the cluster to verify they were deleted. + + ```sh + kubectl get reaadb -o jsonpath='{range .items[*]}{.metadata.name}{"\n"}{end}' + ``` + +1. List the status of all the participating clusters. + +```sh +kubectl get reaadb -o=jsonpath='{.status.participatingClusters}' +``` diff --git a/content/operate/kubernetes/8.2.0/active-active/edit-rerc.md b/content/operate/kubernetes/8.2.0/active-active/edit-rerc.md new file mode 100644 index 0000000000..57484479cc --- /dev/null +++ b/content/operate/kubernetes/8.2.0/active-active/edit-rerc.md @@ -0,0 +1,88 @@ +--- +Title: Edit Redis Enterprise remote clusters +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Edit the configuration details of an existing RERC with Redis Enterprise + for Kubernetes. +linkTitle: Edit RERC +weight: 60 +url: '/operate/kubernetes/8.2.0/active-active/edit-rerc/' +--- +> [!NOTE] +> This feature is supported for general availability in releases 6.4.2-6 and later. Some of these features were available as a preview in 6.4.2-4 and 6.4.2-5. Please upgrade to 6.4.2-6 for the full set of general availability features and bug fixes. and later. + +Before a RedisEnterpriseCluster (REC) can participate in an Active-Active database, it needs an accompanying RedisEnterpriseRemoteCluster (RERC) custom resource. The RERC contains details allowing the REC to link to the RedisEnterpriseActiveActiveDatabase (REAADB). The RERC resource is listed in the REAADB resource to become a participating cluster for the Active-Active database. + +The RERC controller periodically connects to the local REC endpoint via its external address, to ensure it’s setup correctly. For this to work, the external load balancer must support [NAT hairpinning](https://en.wikipedia.org/wiki/Network_address_translation#NAT_loopback). In some cloud environments, this may involve disabling IP preservation for the load balancer target groups. + +For more details, see the [RERC API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_remote_cluster_api.md). + +## Edit RERC + +Use the `kubectl patch rerc --type merge --patch` command to patch the local RERC custom resource with your changes. For a full list of available fields, see the [RERC API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_remote_cluster_api.md). + +The following example edits the `dbFqdnSuffix` field for the RERC named `rerc-ohare`. + +```sh +kubectl patch rerc rerc-ohare --type merge --patch \ +'{"spec":{"dbFqdnSuffix": ".example2-cluster-rec-chicago-ns-illinois.example.com"}}' +``` + +## Update RERC secret + +If the credentials are changed or updated for a REC participating cluster, you need to manually edit the RERC secret and apply it to all participating clusters. + +1. On the local cluster, update the secret with new credentials and name it with the following convention: `redis-enterprise-`. + + A secret for a remote cluster named `rerc-ohare` would be similar to the following: + + ```yaml + apiVersion: v1 + data: + password: PHNvbWUgcGFzc3dvcmQ+ + username: PHNvbWUgdXNlcj4 + kind: Secret + metadata: + name: redis-enterprise-rerc-ohare + type: Opaque + ``` + +1. Apply the file. + + ```sh + kubectl apply -f + ``` + +1. Watch the RERC to verify the status is "Active" and the spec status is "Valid." + + ```sh + kubectl get rerc + ``` + + The output should look like this: + + ```sh + NAME STATUS SPEC STATUS LOCAL + rerc-ohare Active Valid true + ``` + + To troubleshoot invalid configurations, view the RERC custom resource events and the [Redis Enterprise operator logs](/content/operate/kubernetes/8.2.0/logs/_index.md). + +1. Verify the status of each REAADB using that RERC is "Active" and the spec status is "Valid." + + ```sh + kubectl get reaadb reaadb-boeing + ``` + The output should look similar to: + + ```sh + NAME STATUS SPEC STATUS LINKED REDBS REPLICATION STATUS + reaadb-boeing active Valid up + ``` + + To troubleshoot invalid configurations, view the RERC custom resource events and the [Redis Enterprise operator logs](/content/operate/kubernetes/8.2.0/logs/_index.md). + +1. Repeat the above steps on all other participating clusters. \ No newline at end of file diff --git a/content/operate/kubernetes/8.2.0/active-active/global-config.md b/content/operate/kubernetes/8.2.0/active-active/global-config.md new file mode 100644 index 0000000000..6790507edd --- /dev/null +++ b/content/operate/kubernetes/8.2.0/active-active/global-config.md @@ -0,0 +1,178 @@ +--- +Title: Set global database configurations +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: The REAADB contains the field '.spec.globalConfigurations' and through + this the database configurations are set. +linkTitle: Global configuration +weight: 50 +url: '/operate/kubernetes/8.2.0/active-active/global-config/' +--- + + +The Redis Enterprise Active-Active database (REAADB) custom resource contains the field `.spec.globalConfigurations`. This field sets configurations for the Active-Active database across all participating clusters, such as memory size, shard count, and the global database secrets. + +The [REAADB API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_active_active_database_api.md) contains a full list of available fields. + +## Edit global configurations + +1. Edit or patch the REAADB custom resource with your global configuration changes. + + The example command below patches the REAADB named `reaadb-boeing` to set the global memory size to 200MB: + + ```sh + kubectl patch reaadb reaadb-boeing --type merge --patch \ + '{"spec": {"globalConfigurations": {"memorySize": "200mb"}}}' + ``` + +1. Verify the status is `active` and the spec status is `Valid`. + + This example shows the status for the `reaadb-boeing` database. + + ```sh + kubectl get reaadb reaadb-boeing + + NAME STATUS SPEC STATUS GLOBAL CONFIGURATIONS REDB LINKED REDBS + reaadb-boeing active Valid + ``` + +1. View the global configurations on each participating cluster to verify they are synced. + + ```sh + kubectl get reaadb -o yaml + ``` + +## Edit global configuration secrets + +This section edits the secrets under the REAADB `.spec.globalConfigurations` section. For more information and all available fields, see the [REAADB API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_active_active_database_api.md). + +1. On an existing participating cluster, generate a YAML file containing the database secret with the relevant data. + + This example shoes a secret named `my-db-secret` with the password `my-password` encoded in base 64. + + ```yaml + apiVersion: v1 + data: + password: bXktcGFzcw + kind: Secret + metadata: + name: my-db-secret + type: Opaque + ``` + +1. Apply the secret file from the previous step, substituting your own value for ``. + + ```sh + kubectl apply -f + ``` + +1. Patch the REAADB custom resource to specify the database secret, substituting your own values for `` and ``. + + ```sh + kubectl patch reaadb --type merge --patch \ + '{"spec": {"globalConfigurations": {"databaseSecretName": "secret-name"}}}' + ``` + +1. Check the REAADB status for an `active` status and `Valid` spec status. + + ```sh + kubectl get reaadb + + NAME STATUS SPEC STATUS GLOBAL CONFIGURATIONS REDB LINKED REDBS + reaadb-boeing active Valid + ``` + +1. On each other participating cluster, check the secret status. + + ```sh + kubectl get reaadb -o=jsonpath='{.status.secretsStatus}' + ``` + + The output should show the status as `Invalid`. + + ```sh + [{"name":"my-db-secret","status":"Invalid"}] + ``` + +1. Sync the secret on each participating cluster. + + ```sh + kubectl apply -f + ``` + +1. Repeat the previous two steps on every participating cluster. + +## Configure role permissions + +You can configure role-based access control (RBAC) permissions for Active-Active databases using the `rolesPermissions` field in the REAADB `.spec.globalConfigurations` section. The role permissions configuration is propagated across all participating clusters, but the underlying roles and Redis ACLs must be created on each cluster. + +You can define those roles and ACLs as Kubernetes resources with the [`RedisEnterpriseRole`](/content/operate/kubernetes/8.2.0/security/access-control/manage-roles.md) and [`RedisEnterpriseACL`](/content/operate/kubernetes/8.2.0/security/access-control/manage-acls.md) custom resources, or create them directly through the Redis Enterprise admin console or REST API. For an overview of the Kubernetes-native model, see [Access control](/content/operate/kubernetes/8.2.0/security/access-control/_index.md). + +> [!NOTE] +> You must manually create the specified roles and Redis ACLs on all participating clusters before configuring role permissions. The operator only propagates the role permissions configuration—it does not create the underlying roles and ACLs. If roles or ACLs are missing on any cluster, the operator will log errors and dispatch an Event associated with the REAADB object until they are manually created. + +### Prerequisites + +Before configuring role permissions: + +1. Create the required roles and Redis ACLs on all participating clusters, either as [`RedisEnterpriseRole`](/content/operate/kubernetes/8.2.0/security/access-control/manage-roles.md) and [`RedisEnterpriseACL`](/content/operate/kubernetes/8.2.0/security/access-control/manage-acls.md) resources or through the Redis Enterprise admin console or REST API. +2. Ensure role and ACL names match exactly across all clusters (names are case-sensitive). +3. Verify that roles and ACLs are properly configured on each cluster. + +> [!WARNING] +> The operator does not automatically create or synchronize roles and ACLs across clusters. You are responsible for manually creating identical roles and ACLs on each participating cluster. + +### Add role permissions to REAADB + +1. Create or update your REAADB custom resource to include `rolesPermissions` in the global configurations. + + Example REAADB with role permissions: + + ```yaml + apiVersion: app.redislabs.com/v1alpha1 + kind: RedisEnterpriseActiveActiveDatabase + metadata: + name: reaadb-boeing + spec: + globalConfigurations: + databaseSecretName: + memorySize: 200MB + shardCount: 3 + rolesPermissions: + - role: + acl: + type: redis-enterprise + participatingClusters: + - name: rerc-ohare + - name: rerc-reagan + ``` + + Replace `` and `` with the exact names of your Redis Enterprise role and ACL. + +2. Apply the REAADB custom resource: + + ```sh + kubectl apply -f + ``` + + Alternatively, patch an existing REAADB to add role permissions: + + ```sh + kubectl patch reaadb --type merge --patch \ + '{"spec": {"globalConfigurations": {"rolesPermissions": [{"role": "", "acl": "", "type": "redis-enterprise"}]}}}' + ``` + +3. After the REAADB is active and its replication status is "Up", verify role permissions are applied to the local database using the Redis Enterprise REST API. See [Database requests](/content/operate/rs/references/rest-api/requests/bdbs/_index.md#get-bdbs) for details. + +### Troubleshooting role permissions + +If you encounter issues with role permissions: + +- **Missing role or ACL errors**: Manually create the specified roles and ACLs on all participating clusters with exact name matches. The operator cannot create these automatically. +- **Permission propagation failures**: Verify that the roles and ACLs are properly configured and accessible on each cluster. Remember that you must manually create identical roles and ACLs on every participating cluster. +- **Case sensitivity issues**: Verify that role and ACL names match exactly, including capitalization, across all clusters. + +For more details on the `rolesPermissions` field structure, see the [REAADB API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_active_active_database_api.md#specglobalconfigurationsrolespermissions). \ No newline at end of file diff --git a/content/operate/kubernetes/8.2.0/active-active/global-db-secret.md b/content/operate/kubernetes/8.2.0/active-active/global-db-secret.md new file mode 100644 index 0000000000..3ccaadd455 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/active-active/global-db-secret.md @@ -0,0 +1,76 @@ +--- +Title: Set global database secret +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: The REAADB contains the field '.spec.globalConfigurations' to set the + global database secret. +linkTitle: Global database secret +weight: 50 +url: '/operate/kubernetes/8.2.0/active-active/global-db-secret/' +--- + + +## Set global database secret + +One of the fields available for `globalConfigurations` is `databaseSecretName` which can point to a secret containing the database password. To set the database secret name and sync the data to all participating clusters, follow the steps below. + +To edit other global configruations, see [global configuration](/content/operate/kubernetes/8.2.0/active-active/global-config.md) + +1. On an existing participating cluster, generate a YAML file containing the database secret with the database password. + + This example shoes a secret named `my-db-secret` with the password `my-password` encoded in base 64. + + ```yaml + apiVersion: v1 + data: + password: bXktcGFzcw + kind: Secret + metadata: + name: my-db-secret + type: Opaque + ``` + +1. Apply the secret file from the previous step, substituting your own value for ``. + + ```sh + kubectl apply -f + ``` + +1. Patch the REAADB custom resource to specify the database secret, substituting your own values for `` and ``. + + ```sh + kubectl patch reaadb --type merge --patch \ + '{"spec": {"globalConfigurations": {"databaseSecretName": "secret-name"}}}' + ``` + +1. Check the REAADB status for an `active` status and `Valid` spec status. + + ```sh + kubectl get reaadb + + NAME STATUS SPEC STATUS GLOBAL CONFIGURATIONS REDB LINKED REDBS + example-aadb-1 active Valid + ``` + +1. On each other participating cluster, check the secret status. + + ``sh + kubectl get reaadb -o=jsonpath='{.status.secretsStatus}' + ``` + + The output should show the status as `Invalid`. + + ```sh + [{"name":"my-db-secret","status":"Invalid"}] + ``` + +1. Sync the secret on each participating cluster. + + ```sh + kubectl apply -f + ``` + +1. Repeat the previous two steps on every participating cluster. diff --git a/content/operate/kubernetes/8.2.0/active-active/prepare-clusters.md b/content/operate/kubernetes/8.2.0/active-active/prepare-clusters.md new file mode 100644 index 0000000000..3f556552ea --- /dev/null +++ b/content/operate/kubernetes/8.2.0/active-active/prepare-clusters.md @@ -0,0 +1,147 @@ +--- +Title: Prepare participating clusters +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Prepare your participating RECs to be part of an Active-Active database + deployment. +linkTitle: Prepare clusters +weight: 10 +url: '/operate/kubernetes/8.2.0/active-active/prepare-clusters/' +--- + +## Prepare participating clusters + +Before you prepare your clusters to participate in an Active-Active database, make sure you've completed all the following steps and have gathered the information listed below each step. + +1. Configure the [admission controller and ValidatingWebhook](/content/operate/kubernetes/8.2.0/deployment/quick-start.md#enable-the-admission-controller/). + +2. Create two or more [RedisEnterpriseCluster (REC) custom resources](/content/operate/kubernetes/8.2.0/deployment/quick-start.md#create-a-redis-enterprise-cluster-rec) with enough [memory resources](/content/operate/rs/installing-upgrading/install/plan-deployment/hardware-requirements.md). + * Name of each REC (``) + * Namespace for each REC (``) + +3. Configure the REC [`ingressOrRoutes` field](/content/operate/kubernetes/8.2.0/networking/ingressorroutespec.md) and [create DNS records](/content/operate/kubernetes/8.2.0/networking/ingressorroutespec.md#configure-dns/). + * REC API hostname (`api--.`) + * Database hostname suffix (`.db--.`) + +Next you'll [collect credentials](#collect-rec-credentials) for your participating clusters and create secrets for the RedisEnterprsieRemoteCluster (RERC) to use. + +For a list of example values used throughout this article, see the [Example values](#example-values) section. + +## Collect REC credentials + +To communicate with other clusters, all participating clusters will need access to the admin credentials for all other clusters. + +1. Create a file to hold the admin credentials for all participating RECs (such as `all-rec-secrets.yaml`). + +1. Within that file, create a new secret for each participating cluster named `redis-enterprise-`. + + The example below shows a file (`all-rec-secrets.yaml`) holding secrets for two participating clusters: + + ```yaml + apiVersion: v1 + data: + password: + username: + kind: Secret + metadata: + name: redis-enterprise-rerc-ohare + type: Opaque + + --- + + apiVersion: v1 + data: + password: + username: + kind: Secret + metadata: + name: redis-enterprise-rerc-reagan + type: Opaque + + ``` + +1. Get the REC credentials secret for each participating cluster. + + ```sh + kubectl get secret -o yaml + ``` + + The admin credentials secret for an REC named `rec-chicago` would be similar to this: + + ```yaml + apiVersion: v1 + data: + password: ABcdef12345 + username: GHij56789 + kind: Secret + metadata: + name: rec-chicago + type: Opaque + ``` + + > [!NOTE] + > The `username` and `password` values should be base64 encoded, not plain text. + +1. Add the username and password to the new secret for that REC and namespace. + + This example shows the collected secrets file (`all-rec-secrets.yaml`) for `rerc-ohare` (representing `rec-chicago` in namespace `ns-illinois`) and `rerc-reagan` (representing `rec-arlington` in namespace `ns-virginia`). + + ```yaml + apiVersion: v1 + data: + password: ABcdef12345 + username: GHij56789 + kind: Secret + metadata: + name: redis-enterprise-rerc-ohare + type: Opaque + + --- + + apiVersion: v1 + data: + password: KLmndo123456 + username: PQrst789010 + kind: Secret + metadata: + name: redis-enterprise-rerc-reagan + type: Opaque + + ``` + +1. Apply the file of collected secrets to every participating REC. + + ```sh + kubectl apply -f + ``` + + If the admin credentials for any of the clusters changes, the file will need to be updated and reapplied to all clusters. + +## Next steps + +Now you are ready to [create your Redis Enterprise Active-Active database](/content/operate/kubernetes/8.2.0/active-active/create-reaadb.md). + +## Example values + +This article uses the example values listed below. They can also be found in the [YAML examples](/content/operate/kubernetes/8.2.0/reference/yaml/active-active.md) section. + +Example cluster 1: + +* REC name: `rec-chicago` +* REC namespace: `ns-illinois` +* RERC name: `rerc-ohare` +* RERC secret name: `redis-enterprise-rerc-ohare` +* API FQDN: `api-rec-chicago-ns-illinois.example.com` +* DB FQDN suffix: `.db-rec-chicago-ns-illinois.example.com` + +Example cluster 2: + +* REC name: `rec-arlington` +* REC namespace: `ns-virginia` +* RERC name: `rerc-raegan` +* RERC secret name: `redis-enterprise-rerc-reagan` +* API FQDN: `api-rec-arlington-ns-virginia.example.com` +* DB FQDN suffix: `.db-rec-arlington-ns-virginia.example.com` diff --git a/content/operate/kubernetes/8.2.0/architecture/_index.md b/content/operate/kubernetes/8.2.0/architecture/_index.md new file mode 100644 index 0000000000..9aa04e252e --- /dev/null +++ b/content/operate/kubernetes/8.2.0/architecture/_index.md @@ -0,0 +1,160 @@ +--- +Title: Redis Enterprise for Kubernetes architecture +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Overview of the architecture and components of Redis Enterprise for Kubernetes. +hideListLinks: true +linkTitle: Architecture +weight: 1 +aliases: [/operate/kubernetes/architecture/operator/, /operate/kubernetes/architecture/operator-architecture/, /operate/kubernetes/architecture/operator-architecture] +url: '/operate/kubernetes/8.2.0/architecture/' +--- + +Redis Enterprise for Kubernetes gives you the speed and durability of [Redis Enterprise](https://redis.io/redis-enterprise/advantages/), with the flexibility and ease of [Kubernetes (K8s)](https://kubernetes.io/). Redis Enterprise for Kubernetes uses the Kubernetes operator pattern and custom controllers to bring the best of Redis Enterprise to the Kubernetes platform. + +## Lifecycle + +Redis Enterprise for Kubernetes is compatible with [CNCF-conformant](https://www.cncf.io/training/certification/software-conformance/) Kubernetes platforms. The operator follows standard Kubernetes APIs and practices and is designed to run consistently across certified Kubernetes environments. + +Each version of Redis Enterprise for Kubernetes is tested to ensure the version of Redis Enterprise works with the [supported Kubernetes distributions](/content/operate/kubernetes/8.2.0/reference/supported_k8s_distributions.md) at the time. Both the Kubernetes version and the Redis Enterprise version must be supported for the operator to function correctly. We encourage you to upgrade Redis Enterprise for Kubernetes frequently, not only to get the benefit of enhancements and bug fixes, but to keep your software supported. + +Supported platforms are listed in the [release notes](/content/operate/kubernetes/release-notes/_index.md) and in the [supported platforms reference.](/content/operate/kubernetes/8.2.0/reference/supported_k8s_distributions.md) + +## Architecture + +The image below illustrates the components of a single namespace, three node deployment. + +![](/images/k8s/k8s-arch-v4.png) + +## Operator + +An [operator](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/) is a custom extension of the Kubernetes API designed to manage complex, stateful applications and their components. This operator pattern is commonly used by databases and other applications to extend the cluster's behavior without changing its underlying code. Kubernetes.io/docs has a great explanation of the [operator pattern](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/). + +The operator is a deployment that runs within a [namespace](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/) and uses [controllers](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/#custom-controllers) to manage [custom resources (CRs)](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/), ensuring these resources are continuously monitored and maintained. + +When the operator is installed, the following resources are created: + +- [service account](https://kubernetes.io/docs/concepts/security/service-accounts/) under which the operator will run +- set of [roles](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#role-and-clusterrole) to define the privileges necessary for the operator to perform its tasks +- set of [role bindings](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#rolebinding-and-clusterrolebinding) to authorize the service account +- [CustomResourceDefinition (CRD)](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/#customresourcedefinitions) for each Redis Enterprise custom resource +- the operator deployment + +## Namespace + +The Redis Enterprise [operator](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/) is deployed within a [namespace](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/). Each namespace can host only one operator and one [RedisEnterpriseCluster (REC)](#redisenterprisecluster-rec). Namespaces create logical boundaries between resources, allowing organization and security. Some resources are limited to a namespace, while others are cluster-wide. + +Redis Enterprise for Kubernetes also supports [multi-namespace deployments](/content/operate/kubernetes/8.2.0/architecture/deployment-options.md), meaning the operator can monitor other namespaces (that host applications) for custom resources and apply any changes. + +## Custom resources + +Kubernetes [custom resources (CRs)](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/) are commonly used by databases and other applications to extend the cluster's behavior without changing its underlying code. [Custom resources (CRs)](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/) extend the Kubernetes API, enabling users to manage Redis databases the Kubernetes way. Custom resources are created and managed using YAML configuration files. + +This [declarative configuration approach](https://kubernetes.io/docs/tasks/manage-kubernetes-objects/declarative-config/) allows you to specify the desired state for your resources, and the operator makes the necessary changes to achieve that state. This simplifies [installation](/content/operate/kubernetes/8.2.0/deployment/_index.md), [upgrades](/content/operate/kubernetes/8.2.0/upgrade/_index.md), and [scaling](/content/operate/kubernetes/8.2.0/recommendations/sizing-on-kubernetes.md) both vertically and horizontally. + +The operator continuously monitors CRs for changes, automatically reconciling any differences between the desired state you specified in your YAML configuration file, and the actual state of your resources. Custom resources can also reside in separate namespaces from the operator managing them, such as in [multi-namespace installations](/content/operate/kubernetes/8.2.0/re-clusters/multi-namespace.md). + +## Custom resource definitions + +A [custom resource definition (CRD)](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/#customresourcedefinitions) is a cluster-wide resource that specifies which settings can be configured via custom resource files. Any setting not defined by the CRD is not managed by the operator. You can still make changes to these unmanaged settings using standard Redis Enterprise Software methods. + +For settings managed by the operator, any changes made outside of the CR YAML files (e.g., through the management UI) will be overwritten by the operator. Ensure that all operator-managed settings are updated using the CR YAML files to prevent conflicts. + +## RedisEnterpriseCluster REC + +A Redis Enterprise cluster is a set of Redis Enterprise nodes pooling resources. Each node is capable of running multiple Redis instances ([shards](/content/operate/rs/references/terminology.md)). + +![](/images/k8s/k8s-node-arch.png) + +A Redis cluster is created and managed by the [RedisEnterpriseCluster (REC)](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md) [custom resource](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/). Changes to the REC configuration file prompt the operator to make changes to the cluster. The REC is required for both standard databases ([REDB](#redisenterprisedatabase-redb)) and Active-Active databases ([REAADB](#redisenterpriseactiveactivedatabase-reaadb)). + +See the [RedisEnterpriseCluster API Reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md) for a full list of fields and settings. + +## RedisEnterpriseDatabase REDB + +A Redis Enterprise database is a logical entity that manages your entire dataset across multiple Redis instances. A Redis instance is a single-threaded database process ([commonly referred to as a shard](/content/operate/rs/references/terminology.md)). + +Redis databases are created and managed by the [RedisEnterpriseDatabase (REDB)](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md) [custom resource (CR)](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/). Changes to the REDB YAML configuration file prompt the operator to make changes to the database. + +An operator can manage a database in the same namespace, or a different namespace. See ["Flexible deployment"](/content/operate/kubernetes/8.2.0/architecture/deployment-options.md) options and ["Manage databases in multiple namespaces"](/content/operate/kubernetes/8.2.0/re-clusters/multi-namespace.md) for more information. + +See the [RedisEnterpriseDatabase (REDB) API Reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md) for a full list of fields and settings. + +## Security + +Redis Enterprise for Kubernetes uses [secrets](https://kubernetes.io/docs/concepts/configuration/secret/) to manage your cluster credentials, cluster certificates, and client certificates. You can configure [LDAP](/content/operate/kubernetes/8.2.0/security/authentication/ldap.md) and [internode encryption](/content/operate/kubernetes/8.2.0/security/certificates/internode-encryption.md) using the [RedisEnterpriseCluster (REC)](#redisenterprisecluster-rec) spec. + +### REC credentials + +Redis Enterprise for Kubernetes uses the [RedisEnterpriseCluster (REC)](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md) [custom resource](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/) to create a Redis Enterprise cluster. During creation it generates random credentials for the operator to use. The credentials are saved in a Kubernetes (K8s) [secret](https://kubernetes.io/docs/concepts/configuration/secret/). The secret name defaults to the name of the cluster. + +See [Manage REC credentials](/content/operate/kubernetes/8.2.0/security/authentication/manage-rec-credentials.md) for more details. + +### REC certificates + +By default, Redis Enterprise Software for Kubernetes generates TLS certificates for the cluster during creation. These self-signed certificates are generated on the first node of each Redis Enterprise cluster (REC) and are copied to all other nodes in the cluster. + +See [Manage REC certificates](/content/operate/kubernetes/8.2.0/security/certificates/manage-rec-certificates.md) for more details. + +### Client certificates + +For each client certificate you want to use, you need to create a [Kubernetes secret](https://kubernetes.io/docs/concepts/configuration/secret/) to hold it. You can then reference that secret in your [Redis Enterprise database (REDB)](#redisenterprisedatabase-redb) custom resource. + +See [Add client certificates](/content/operate/kubernetes/8.2.0/security/certificates/add-client-certificates.md) for more details. + +## Storage + +[Persistent storage is mandatory for Redis Enterprise.](/content/operate/rs/installing-upgrading/install/plan-deployment/persistent-ephemeral-storage.md) Redis Enterprise for Kubernetes [requires network-attached storage](https://en.wikipedia.org/wiki/Network-attached_storage). + +Redis Enterprise for Kubernetes uses [PersistentVolumeClaims (PVC)](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#expanding-persistent-volumes-claims) to manage storage resources. The PVC is an abstract representation of the [PersistentVolume (PV)](https://kubernetes.io/docs/concepts/storage/persistent-volumes) resources used by your Redis pods. PVCs are created by the Redis Enterprise operator and used by the [RedisEnterpriseCluster (REC)](#redisenterprisecluster-rec). + +PVCs are created with a specific size and [can be expanded](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#expanding-persistent-volumes-claims), if the underlying [storage class](https://kubernetes.io/docs/concepts/storage/storage-classes/) supports it. + +### Redis Flex + +Redis Enterprise Software for Kubernetes supports [Redis Flex](/content/operate/kubernetes/8.2.0/flex/_index.md) (previously known as Redis on Flash), which extends your node memory to use both RAM and flash storage. SSDs (solid state drives) can store infrequently used (warm) keys and values while frequently used (hot) keys and values are stored in RAM. This improves performance and lowers costs for large datasets. + +NVMe (non-volatile memory express) SSDs are strongly recommended to achieve the best performance. + +## Networking + +By default, Kubernetes doesn't allow you to access your Redis database from outside your K8s cluster. Redis Enterprise for Kubernetes supports several ways to route external traffic to your [Redis Enterprise cluster (REC)](#redisenterprisecluster-rec): + +- Ingress controllers [HAProxy](https://haproxy-ingress.github.io/) and [NGINX](https://kubernetes.github.io/ingress-nginx/) require an `ingress` API resource. +- [Istio](https://istio.io/latest/docs/setup/getting-started/) requires `Gateway` and `VirtualService` API resources. +- OpenShift uses [routes](/content/operate/kubernetes/8.2.0/networking/routes.md) to route external traffic. + +The [Active-Active databases](#active-active-databases) require one of above routing methods to be configured in the REC with the [ingressOrRouteSpec field](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#specingressorroutespec). + +## Services Rigger + +The services rigger is responsible for creating and updating services related to database objects. It identifies database objects within the cluster and creates services in accordance with [`redisEnterpriseCluster.Spec.servicesRiggerSpec` setting](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#specservicesriggerspec) to allow access to those databases. By default, each database has two services, a `cluster_ip` Service with the same name as the database and a `headless` Service with the same name as the database suffixed with `-headless`. It also creates other types of Services such as Ingress Services or OpenshiftRoutes (defined in `redisEnterpriseCluster.Spec.ingressOrRouteSpec`) meant to provide access to REAADB objects. + +You can view a list of services with the `kubectl get services` command. + +Kubernetes is a dynamic environment, with nodes and pods changing as needed. The services rigger monitors the cluster for these changes and updates the database services to ensure reliable communication with the databases. + +## Active-Active databases + +On Kubernetes, Redis Enterprise [Active-Active](/content/operate/rs/databases/active-active/_index.md) databases provide read and write access to the same dataset from different Kubernetes clusters. Creating an Active-Active database requires routing [network access](/content/operate/kubernetes/8.2.0/networking/_index.md) between two Redis Enterprise clusters residing in different Kubernetes clusters. Without the proper access configured for each cluster, syncing between the databases instances will fail.The admission controller is also required for Active-Active databases on Kubernetes, to validate changes to the custom resources. + +For more details and installation information, see [Active-Active databases](/content/operate/kubernetes/8.2.0/active-active/_index.md). For more general information about Active-Active, see the [Redis Enterprise Software docs](/content/operate/rs/databases/active-active/_index.md). + +## RedisEnterpriseRemoteCluster RERC + +The [RedisEnterpriseRemoteCluster (RERC)](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_remote_cluster_api.md) contains details allowing the REC to link to the RedisEnterpriseActiveActiveDatabase (REAADB). The RERC resource is listed in the [REAADB](#redisenterpriseactiveactivedatabase-reaadb) resource to become a participating cluster for the Active-Active database. + +See the [RERC API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_remote_cluster_api.md) for a full list of fields and settings. + +## RedisEnterpriseActiveActiveDatabase REAADB + +The RedisEnterpriseActiveActiveDatabase (REAADB) resource creates and manages a database that spans more than one Kubernetes cluster. An REAADB requires [external routing](#networking), at least two [RECs](#redisenterprisecluster-rec), and at least two [RERCs](#redisenterpriseremotecluster-rerc). + +See the [REAADB API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_active_active_database_api.md) for a full list of fields and settings. + +## Metrics + +To collect metrics data from your databases and Redis Enterprise cluster (REC), you can [connect your Prometheus](/content/operate/kubernetes/8.2.0/re-clusters/connect-prometheus-operator.md) server to an endpoint exposed on your REC. Redis Enterprise for Kubernetes creates a dedicated service to expose the `prometheus` port (8070) for data collection. A custom resource called `ServiceMonitor` allows the [Prometheus operator](https://github.com/prometheus-operator/prometheus-operator/tree/main/Documentation) to connect to this port and collect data from Redis Enterprise. diff --git a/content/operate/kubernetes/8.2.0/architecture/deployment-options.md b/content/operate/kubernetes/8.2.0/architecture/deployment-options.md new file mode 100644 index 0000000000..c9848a8bbe --- /dev/null +++ b/content/operate/kubernetes/8.2.0/architecture/deployment-options.md @@ -0,0 +1,56 @@ +--- +Title: Flexible deployment options +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Redis Enterprise for Kubernetes allows you to deploy to multiple namespaces. + This article describes flexible deployment options you can use to meet your specific + needs. +linkTitle: Deployment options +aliases: [/operate/kubernetes/deployment/deployment-options/] +url: '/operate/kubernetes/8.2.0/architecture/deployment-options/' +--- +You can deploy Redis Enterprise for Kubernetes in several different ways depending on your database needs. + +Multiple RedisEnterpriseDatabase (REDB) resources can be associated with a single Redis Enterprise cluster resource (REC) even if they reside in different namespaces. + +The Redis Enterprise cluster (REC) custom resource must reside in the same namespace as the Redis Enterprise operator. + +## Single REC and single namespace (one-to-one) + +The standard and simplest deployment deploys your Redis Enterprise databases (REDB) in the same namespace as the Redis Enterprise cluster (REC). No additional configuration is required for this, since there is no communication required to cross namespaces. See [Deploy Redis Enterprise for Kubernetes](/content/operate/kubernetes/8.2.0/deployment/quick-start.md). + +![](/images/k8s/k8s-deploy-one-to-one.png) + +## Single REC and multiple namespaces (one-to-many) + +Multiple Redis Enterprise databases (REDB) spread across multiple namespaces within the same K8s cluster can be associated with the same Redis Enterprise cluster (REC). See [Manage databases in multiple namespaces](/content/operate/kubernetes/8.2.0/re-clusters/multi-namespace.md) for more information. + +![](/images/k8s/k8s-deploy-one-to-many.png) + +## Multiple RECs and multiple namespaces (many-to-many) + +A single Kubernetes cluster can contain multiple Redis Enterprise clusters (REC), as long as they reside in different namespaces. Each namespace can host only one REC and each operator can only manage one REC. + +You have the flexibility to create databases in separate namespaces, or in the same namespace as the REC, or combine any of the supported deployment options above. This configuration is geared towards use cases that require multiple Redis Enterprise clusters with greater isolation or different cluster configurations. + +See [Manage databases in multiple namespaces](/content/operate/kubernetes/8.2.0/re-clusters/multi-namespace.md) for more information. + + +![](/images/k8s/k8s-deploy-many-to-many-new.png) + +## Unsupported deployment patterns + +### Cross-cluster operations + +Redis Enterprise for Kubernetes does not support operations that cross Kubernetes clusters. Redis Enterprise clusters (REC) work inside a single K8s cluster. Crossing clusters could result in functional and security issues. + +![](/images/k8s/k8s-deploy-cross-namespaces.png) + +### Multiple RECs in one namespace + +Redis Enterprise for Kubernetes does not support multiple Redis Enterprise clusters (REC) in the same namespace. Creating more than one REC in the same namespace will result in errors. + +![](/images/k8s/k8s-deploy-multicluster-antipattern.png) diff --git a/content/operate/kubernetes/8.2.0/deployment/_index.md b/content/operate/kubernetes/8.2.0/deployment/_index.md new file mode 100644 index 0000000000..6e01abb70c --- /dev/null +++ b/content/operate/kubernetes/8.2.0/deployment/_index.md @@ -0,0 +1,62 @@ +--- +Title: Deployment +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Deploy Redis Enterprise for Kubernetes using the Redis Enterprise operator on various Kubernetes distributions. +hideListLinks: true +linkTitle: Deployment +weight: 11 +url: '/operate/kubernetes/8.2.0/deployment/' +--- + +Deploy Redis Enterprise for Kubernetes by using the Redis Enterprise operator. The operator provides a simple way to deploy and manage Redis Enterprise clusters on various Kubernetes distributions, both on-premises and in the cloud. + +The Redis Enterprise operator uses custom resource definitions (CRDs) to manage Redis Enterprise clusters (REC) and databases (REDB) as native Kubernetes resources. This approach enables GitOps workflows and Kubernetes-native operations. + +## Quick start + +Get started quickly with a basic Redis Enterprise deployment: + +- [Deploy Redis Enterprise for Kubernetes](/content/operate/kubernetes/8.2.0/deployment/quick-start.md) - Step-by-step guide for most Kubernetes distributions +- [Deploy on OpenShift](/content/operate/kubernetes/8.2.0/deployment/openshift/_index.md) - Specific instructions for OpenShift environments + +## Deployment methods + +Choose the deployment method that best fits your environment: + +- [Deploy with Helm](/content/operate/kubernetes/8.2.0/deployment/helm.md) - Use Helm charts for simplified deployment and management +- [Deploy with operator bundle](/content/operate/kubernetes/8.2.0/deployment/quick-start.md) - Direct deployment using kubectl and operator manifests + +## Container images + +Understand the container images used by the Redis Enterprise operator: + +- [Container images](/content/operate/kubernetes/8.2.0/deployment/container-images.md) - Details about Redis Enterprise container images and registries + +## Compatibility + +Before installing, verify compatibility with your environment: + +- [Supported Kubernetes distributions](/content/operate/kubernetes/8.2.0/reference/supported_k8s_distributions.md) - Check which Redis Enterprise operator version supports your Kubernetes distribution + +## Prerequisites + +Before deploying Redis Enterprise for Kubernetes, ensure you have: + +- A Kubernetes cluster running a [supported distribution](/content/operate/kubernetes/8.2.0/reference/supported_k8s_distributions.md) +- Minimum of three worker nodes for high availability +- Kubernetes client (kubectl) configured to access your cluster +- Access to container registries (DockerHub, Red Hat Container Catalog, or private registry) +- Sufficient resources as outlined in [sizing recommendations](/content/operate/kubernetes/8.2.0/recommendations/sizing-on-kubernetes.md) + +## Next steps + +After deployment, you can: + +- [Create a Redis Enterprise cluster (REC)](/content/operate/kubernetes/8.2.0/re-clusters/_index.md) +- [Create Redis Enterprise databases (REDB)](/content/operate/kubernetes/8.2.0/re-databases/_index.md) +- [Configure networking](/content/operate/kubernetes/8.2.0/networking/_index.md) +- [Set up security](/content/operate/kubernetes/8.2.0/security/_index.md) \ No newline at end of file diff --git a/content/operate/kubernetes/8.2.0/deployment/container-images.md b/content/operate/kubernetes/8.2.0/deployment/container-images.md new file mode 100644 index 0000000000..f99bdcfd76 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/deployment/container-images.md @@ -0,0 +1,259 @@ +--- +Title: Use a private registry for container images +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: This section details how the Redis Enterprise Software and Kubernetes + operator images can be configured to be pulled from a variety of sources. This page + describes how to configure alternate private repositories for images, plus some + techniques for handling public repositories with rate limiting. +linktitle: Use a private container registry +weight: 92 +url: '/operate/kubernetes/8.2.0/deployment/container-images/' +--- + +Redis Enterprise Software, its Kubernetes operator, and the Service Rigger +are all distributed as separate container images. +Your Kubernetes deployment will pull these images as needed. +You can control where these images are +pulled from within the operator deployment and also via the +Redis Enterprise custom resources. + +The operator image also includes the admission controller, which runs as part of the operator container and provides validation for Redis Enterprise database resources. + +For security reasons (e.g., in air-gapped environments), you may want to pull the images +from a public registry once and then push them to a private registry under +your control. + +> [!WARNING] +> It is very important that the images you are pushing to the private registry have the same exact version tag as the original images. + +Furthermore, because [Docker rate limits public pulls](https://www.docker.com/blog/scaling-docker-to-serve-millions-more-developers-network-egress/), +you may want to consider pulling images from a +private registry to avoid deployment failures when you hit your DockerHub rate limit. + +The information below will help you track and configure where your deployments pull container images. + +> [!NOTE] +> **IMPORTANT** +> * Each version of the Redis Enterprise operator is mapped to a specific version of Redis Enterprise Software. The semantic versions always match (for example, 7.22.0), although the specific release numbers may be different (for example, 7.22.0-7 is the operator version for Redis Enterprise Software 7.22.0-28). +> * A specific operator version only supports a specific Redis Enterprise version. Other combinations of operator and Redis Enterprise versions are **not supported**. + + +## Find container sources + +Every pod in your deployed application has a source registry. Any image not prefixed by a registry domain name (e.g., "gcr.io") will pull from the default registry for the Kubernetes cluster (i.e., DockerHub). You can use the commands below to discover the pull sources for the images on your cluster. + +To list all the images used by your cluster: + +```sh +kubectl get pods --all-namespaces -o jsonpath="{..image}" |tr -s '[[:space:]]' '\n' | uniq -c +``` + +To specifically determine the pull source for the Redis Enterprise operator itself, run the following command: + +```sh +kubectl get pods --all-namespaces -o jsonpath="{..image}" |tr -s '[[:space:]]' '\n' | uniq -c | grep redislabs +``` + +You can limit this command to specific namespaces by replacing the `--all-namespaces` parameter with +a set of `-n {namespace}` parameters, where each `{namespace}` is a specific +namespace of interest on your cluster. + +## Create a private container registry + +You can set up a private container registry in a couple of ways: + +* On-premise via [Docker registry](https://docs.docker.com/registry/deploying/), [Red Hat Quay](https://www.redhat.com/en/technologies/cloud-computing/quay), or other providers +* Cloud provider based registries (e.g., Azure Container Registry, Google Container Registry, etc.). + +Once you have set up a private container registry, you will identify the container registry using: + +* A domain name +* A port (optional) +* A repository path (optional) + +## Push images to a private container registry + +Important images for a Redis Enterprise Software deployment include: + +* Redis Enterprise Software +* Bootstrapping a Redis Enterprise cluster node (in the operator image) +* The Service Rigger +* The Redis Enterprise Software operator (which also includes the admission controller) + +You will need to push all these images to your private container registry. In general, +to push the images you must: + + 1. [Pull](https://docs.docker.com/engine/reference/commandline/pull/) the various images locally for the Redis Enterprise Software, the Service Rigger, and the operator. + 2. Tag the local images with the private container registry, repository, and version tag. + 3. [Push](https://docs.docker.com/engine/reference/commandline/push/) the newly tagged images. + +The example below shows the commands for pushing the images for Redis Enterprise Software and its operator to a private container registry: + +```sh +PRIVATE_REPO=...your repo... +RS_VERSION=7.22.0-28 +OPERATOR_VERSION=7.22.0-7 +docker pull redislabs/redis:${RS_VERSION} +docker pull redislabs/operator:${OPERATOR_VERSION} +docker pull redislabs/k8s-controller:${OPERATOR_VERSION} +docker tag redislabs/redis:${RS_VERSION} ${PRIVATE_REPO}/redislabs/redis:${RS_VERSION} +docker tag redislabs/operator:${OPERATOR_VERSION} ${PRIVATE_REPO}/redislabs/operator:${OPERATOR_VERSION} +docker tag redislabs/k8s-controller:${OPERATOR_VERSION} ${PRIVATE_REPO}/redislabs/k8s-controller:${OPERATOR_VERSION} +docker push ${PRIVATE_REPO}/redislabs/redis:${RS_VERSION} +docker push ${PRIVATE_REPO}/redislabs/operator:${OPERATOR_VERSION} +docker push ${PRIVATE_REPO}/redislabs/k8s-controller:${OPERATOR_VERSION} +``` + +## Configure deployments to use a private container registry + +Once you push your images to your private container registry, you need to +configure your deployments to use that registry for Redis Enterprise Software and operator +deployments. The operator container image is configured directly by the operator deployment +bundle. The Redis Enterprise cluster pod (RS and bootstrapper) and Service Rigger +images are configured in the Redis Enterprise custom resource. + +Depending on your Kubernetes platform, your private container registry may +require authentication. If you do need authentication, add a [pull secret](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/) to your namespace. Then you'll need to configure Kubernetes and the operator to use the pull secret. The two following sections have examples of adding the `imagePullSecrets` to the operator deployment and `pullSecrets` to the cluster custom resource. + +### Specify the operator image source + +The operator bundle contains the operator deployment and the reference to the operator image (`redislabs/operator`). This image includes both the operator functionality and the admission controller. To use a private container registry, you must +change this image reference in your operator deployment file **before** you deploy the operator. If you apply this change to modify an existing operator deployment, the operator's pod will restart. + +In the operator deployment file, 'containers:image' should point to the same repository and tag you used when [pushing](/content/operate/kubernetes/8.2.0/deployment/container-images.md#push-images-to-a-private-container-registry) to the private container registry: + +```sh +${PRIVATE_REPO}/redislabs/operator:${OPERATOR_VERSION} +``` + +The example below specifies a 7.22.0-7 operator image in a Google Container Registry: + +```YAML +apiVersion: apps/v1 +kind: Deployment +metadata: + name: redis-enterprise-operator +spec: + replicas: 1 + selector: + matchLabels: + name: redis-enterprise-operator + template: + metadata: + labels: + name: redis-enterprise-operator + spec: + serviceAccountName: redis-enterprise-operator + containers: + - name: redis-enterprise-operator + image: gcr.io/yourproject/redislabs/operator:7.22.0-7 +... +``` + +If your container registry requires a pull secret, configure `imagePullSecrets` on the operator deployment: + +```YAML +spec: + template: + spec: + imagePullSecrets: + - name: regcred +``` + +### Specify the Redis Enterprise cluster images source + +A Redis Enterprise cluster managed by the operator consists of three +container images: + +* **`redislabs/redis`**: the Redis Enterprise Software container image +* **`redislabs/operator`**: the bootstrapper is packaged within the operator container image +* **`redislabs/k8s-controller`**: the Service Rigger container image + +By default, a new Redis Enterprise cluster is created using the +container images listed above. These container images are pulled from the K8s cluster's default +container registry. + +To [pull](https://docs.docker.com/engine/reference/commandline/pull/) the Redis Enterprise container images from +a private container registry, you must specify them in the +Redis Enterprise cluster custom resource. + +Add the following sections to the `spec` section of your RedisEnterpriseCluster resource file: + + * **`redisEnterpriseImageSpec`**: controls the Redis Enterprise Software container image. *The version should match the RS version associated with the operator version*. + * **`bootstrapperImageSpec`**": controls the bootstrapper container image. *The version must match the operator version*. + * **`redisEnterpriseServicesRiggerImageSpec`**: controls the Service Rigger container image. *The version must match the operator version*. + +The REC custom resource example below pulls all three container images from a GCR private registry: + +```YAML +apiVersion: app.redislabs.com/v1 +kind: RedisEnterpriseCluster +metadata: + name: rec +spec: + nodes: 3 + redisEnterpriseImageSpec: + imagePullPolicy: IfNotPresent + repository: gcr.io/yourproject/redislabs/redis + versionTag: 7.22.0-28 + bootstrapperImageSpec: + imagePullPolicy: IfNotPresent + repository: gcr.io/yourproject/redislabs/operator + versionTag: 7.22.0-7 + redisEnterpriseServicesRiggerImageSpec: + imagePullPolicy: IfNotPresent + repository: gcr.io/yourproject/redislabs/k8s-controller + versionTag: 7.22.0-7 +``` + +If your private container registry requires pull secrets, you must add `pullSecrets` +to the `spec` section: + +```YAML +apiVersion: app.redislabs.com/v1 +kind: RedisEnterpriseCluster +metadata: + name: rec +spec: + nodes: 3 + pullSecrets: + - name: regcred + redisEnterpriseImageSpec: + imagePullPolicy: IfNotPresent + repository: gcr.io/yourproject/redislabs/redis + versionTag: 7.22.0-28 + bootstrapperImageSpec: + imagePullPolicy: IfNotPresent + repository: gcr.io/yourproject/redislabs/operator + versionTag: 7.22.0-7 + redisEnterpriseServicesRiggerImageSpec: + imagePullPolicy: IfNotPresent + repository: gcr.io/yourproject/redislabs/k8s-controller + versionTag: 7.22.0-7 +``` + +## Admission controller + +The admission controller is included as part of the operator container image and does not require a separate container image. When you configure a private container registry for the operator image, the admission controller functionality is automatically included. + +The admission controller runs within the operator pod and provides validation for Redis Enterprise database resources. It exposes an HTTPS endpoint on port 8443 that Kubernetes uses to validate resource configurations before they are applied to the cluster. + +For more information about configuring the admission controller, see [Enable the admission controller](/content/operate/kubernetes/8.2.0/deployment/quick-start.md#enable-the-admission-controller). + +## Rate limiting with DockerHub + +Docker has [rate limits for image pulls](https://www.docker.com/blog/scaling-docker-to-serve-millions-more-developers-network-egress/). +Anonymous users are allowed a certain number of pulls every 6 hours. For authenticated users, the limit is larger. +These rate limits may affect your Kubernetes cluster in a number of ways: + +* The cluster nodes will likely be treated as a single anonymous user. +* The number of pulls during a deployment might exceed the rate limit for other deployment dependencies, including the operator, Redis Enterprise Software, or other non-Redis pods. +* Pull failures may prevent your deployment from downloading the required images in a timely manner. Delays here can affect the stability of deployments used by the Redis Enterprise operator. + +For these reasons, you should seriously consider where your images +are pulled from to **avoid failures caused by rate limiting**. The easiest solution +is to push the required images to a private container registry under your control. diff --git a/content/operate/kubernetes/8.2.0/deployment/helm.md b/content/operate/kubernetes/8.2.0/deployment/helm.md new file mode 100644 index 0000000000..3cbc82f82a --- /dev/null +++ b/content/operate/kubernetes/8.2.0/deployment/helm.md @@ -0,0 +1,313 @@ +--- +Title: Install Redis Enterprise Helm chart +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Install Redis Enterprise for Kubernetes version 7.8.6 using Helm charts. +linkTitle: Helm +weight: 11 +url: '/operate/kubernetes/8.2.0/deployment/helm/' +--- +Helm charts provide a simple way to install the Redis Enterprise for Kubernetes operator in just a few steps. For more information about Helm, go to [https://helm.sh/docs/](https://helm.sh/docs/). + +## Prerequisites + +- A [supported distribution](/content/operate/kubernetes/8.2.0/reference/supported_k8s_distributions.md) of Kubernetes. +- At least three worker nodes. +- [Kubernetes client (kubectl)](https://kubernetes.io/docs/tasks/tools/). +- [Helm 3.10 or later](https://helm.sh/docs/intro/install/) + or 3.18 for migrating from a non-Helm installation. + +If you suspect your file descriptor limits are below 100,000, you must either manually increase limits or [Allow automatic resource adjustment](/content/operate/kubernetes/8.2.0/security/allow-resource-adjustment.md). Most major cloud providers and standard container runtime configurations set default file descriptor limits well above the minimum required by Redis Enterprise. In these environments, you can safely run without enabling automatic resource adjustment. + +### Example values + +The steps below use the following placeholders to indicate command line parameters you must provide: + +- `` is the name you give a specific installation of the Helm chart (example: `my-redis-enterprise-operator`) +- `` is the version of the Helm chart you are installing (example: `7.8.6-2`). Verify that the version you specify is listed in the [Redis Helm repository](https://helm.redis.io/). Using an invalid version number causes installation failures. +- `` is the name of the new namespace the Redis operator will run in (example: `ns1`) +- `` is the filepath to the Helm chart, if it is stored in a local directory (example: `/home/charts/redis-enterprise-operator`) + +## Install the operator + +Below are several ways to install the operator using Helm. To create the REC in the same step, see [Create the REC at install time](#create-the-rec-at-install-time). + +- [Install from the Redis Helm repository](#install-from-the-redis-helm-repository) +- [Install from local directory](#install-from-local-directory) +- [Specify values during install](#specify-values-during-install) +- [Install with values file](#install-with-values-file) + +### Install from the Redis Helm repository + +1. Add the Redis repository. + + ```sh + helm repo add redis https://helm.redis.io + ``` + +2. Install the Helm chart into a new namespace. + +```sh +helm install redis/redis-enterprise-operator \ + --version \ + --namespace \ + --create-namespace +``` + +To install with Openshift, add `--set openshift.mode=true`. + +To monitor the installation add the `--debug` flag. The installation runs several jobs synchronously and may take a few minutes to complete. + +### Install from local directory + +If you need a local copy of the chart, for example to inspect it or customize values, use one of the following methods. + +**Pull the packaged chart (recommended).** `helm pull` fetches the exact packaged chart the Helm repository serves for the version you specify, matching what `helm install` from the repository would deploy. + +1. Add the Redis repository, if you haven't already. + + ```sh + helm repo add https://helm.redis.io + ``` + +2. Pull and unpack the chart. This creates a `redis-enterprise-operator` directory in your current location. + + ```sh + helm pull /redis-enterprise-operator --version --untar + ``` + +3. Install the Helm chart from the local directory. + + ```sh + helm install ./redis-enterprise-operator \ + --namespace \ + --create-namespace + ``` + +**Download the source code.** Alternatively, download the chart source from GitHub. + +1. Find the latest release on the [redis-enterprise-k8s-docs](https://github.com/RedisLabs/redis-enterprise-k8s-docs/releases) repo and download the `tar.gz` source code into a local directory. + +2. Install the Helm chart from your local directory. + + ```sh + helm install \ + --namespace \ + --create-namespace + ``` + +To install with Openshift, add `--set openshift.mode=true`. + +To monitor the installation add the `--debug` flag. The installation runs several jobs synchronously and may take a few minutes to complete. + +### Specify values during install + +1. View configurable values with `helm show values redis/redis-enterprise-operator`. + +2. Install the Helm chart, overriding specific value defaults using `--set`. + +```sh +helm install redis/redis-enterprise-operator \ + --version \ + --namespace \ + --create-namespace + --set = \ + --set = +``` + +### Install with values file + +1. View configurable values with `helm show values redis/redis-enterprise-operator`. + +2. Create a YAML file to specify the values you want to configure. + +3. Install the chart with the `--values` option. + +```sh +helm install redis/redis-enterprise-operator \ + --version \ + --namespace \ + --create-namespace \ + --values +``` + +## Create the REC at install time + +The chart can create the `RedisEnterpriseCluster` (REC) custom resource at install time, so a single `helm install` deploys both the operator and the cluster. To enable this, set `cluster.create` to `true` and define the cluster under `cluster.spec` in your values file. The `spec` field accepts any field from the `RedisEnterpriseCluster` CRD. For the full list, see the [RedisEnterpriseCluster API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md). + +> [!NOTE] +> `cluster.create` defaults to `false`. When `false`, the chart installs only the operator and you create the REC yourself. + +1. Create a values file with a `cluster` section. For example: + + ```yaml + cluster: + create: true + spec: + nodes: 3 + redisEnterpriseImageSpec: + repository: redislabs/redis + redisEnterpriseServicesRiggerImageSpec: + repository: redislabs/k8s-controller + bootstrapperImageSpec: + repository: redislabs/operator + ``` + +2. Install the chart with the values file. + + ```sh + helm install redis/redis-enterprise-operator \ + --version \ + --namespace \ + --create-namespace \ + --values + ``` + + The chart runs a Kubernetes Job to create the REC. The Job deletes itself on success. + +3. Confirm the operator deployment is running. + + ```sh + kubectl get deployment redis-enterprise-operator --namespace + ``` + +4. Wait for the REC to reach the `Running` state. Initialization takes about a minute. + + ```sh + kubectl get rec --namespace + ``` + +To update or uninstall an REC created by the chart, see [Update an REC created by the chart](#update-an-rec-created-by-the-chart) and [Uninstall the REC and operator together](#uninstall-the-rec-and-operator-together). + +## Upgrade the chart + +To upgrade an existing Helm chart installation: + +```sh +helm upgrade redis/redis-enterprise-operator --version +``` + +You can also upgrade from a local directory: + +```sh +helm upgrade +``` + +For example, to upgrade a chart with the release name `my-redis-enterprise` from the chart's root directory: + +```sh +helm upgrade my-redis-enterprise . +``` + +To upgrade with OpenShift, add `--set openshift.mode=true`. + +The upgrade process automatically updates the operator and its components, including the Custom Resource Definitions (CRDs). The CRDs are versioned and update only if the new version is higher than the existing version. + +After you upgrade the operator, you might need to upgrade your Redis Enterprise clusters, depending on the Redis software version bundled with the operator. For detailed information about the upgrade process, see [Redis Enterprise for Kubernetes upgrade documentation](https://redis.io/docs/latest/operate/kubernetes/upgrade/). + +> [!NOTE] +> If your databases use user-defined modules (custom non-bundled modules), you must take additional steps during the upgrade process. See [Upgrade with user-defined modules](/content/operate/kubernetes/8.2.0/upgrade/upgrade-redis-cluster.md#user-defined-modules) for details. + +For more information and options when upgrading charts, see [helm upgrade](https://helm.sh/docs/helm/helm_upgrade/). + +### Update an REC created by the chart + +If you used `cluster.create: true`, change REC settings by editing `cluster.spec` in your values file and running `helm upgrade`. + +1. Update your values file. For example, to add labels: + + ```yaml + cluster: + create: true + spec: + nodes: 3 + redisEnterpriseImageSpec: + repository: redislabs/redis + redisEnterpriseServicesRiggerImageSpec: + repository: redislabs/k8s-controller + bootstrapperImageSpec: + repository: redislabs/operator + extraLabels: + environment: production + ``` + +2. Apply the changes. + + ```sh + helm upgrade redis/redis-enterprise-operator \ + --namespace \ + --values + ``` + + Some REC fields trigger a StatefulSet rolling update. Others, such as `extraLabels`, take effect immediately. + +3. Confirm the change. + + ```sh + kubectl get rec --namespace -o yaml + ``` + +## Uninstall + +1. Delete any custom resources managed by the operator. See [Delete custom resources](/content/operate/kubernetes/8.2.0/re-clusters/delete-custom-resources.md) for detailed steps. You must delete custom resources in the correct order to avoid errors. + +2. Uninstall the Helm chart. + +```sh +helm uninstall +``` + +This removes all Kubernetes resources associated with the chart and deletes the release. + +> [!NOTE] +> Custom Resource Definitions (CRDs) installed by the chart are not removed during chart uninstallation. To remove them manually after uninstalling the chart, run `kubectl delete crds -l app=redis-enterprise`. + +### Uninstall the REC and operator together + +When `cluster.create` is `true`, `helm uninstall` removes both the REC and the operator. You don't need to delete the REC first. + +```sh +helm uninstall --namespace +``` + +Confirm all resources are gone. + +```sh +kubectl get all --namespace +``` + +> [!NOTE] +> This applies only to RECs the chart created. If you created the REC outside the chart, follow [Delete custom resources](/content/operate/kubernetes/8.2.0/re-clusters/delete-custom-resources.md) before you run `helm uninstall`. + +## Migrate from a non-Helm installation + +To migrate an existing non-Helm installation of the Redis Enterprise operator to a Helm-based installation: + +1. [Upgrade](/content/operate/kubernetes/8.2.0/upgrade/_index.md) your existing Redis Enterprise operator to match the version of the Helm chart you want to install. Use the same non-Helm method you used for the original installation. + +2. [Install](#install-the-operator) the Helm chart adding the `--take-ownership` flag: + + ```sh + helm install redis/redis-enterprise-operator --take-ownership + ``` + + - The `--take-ownership` flag is available with Helm versions 3.18 or later. + - This flag is only needed for the first installation of the chart. Subsequent upgrades don't require this flag. + - Use the `helm install` command, not `helm upgrade`. + +3. Delete the old `ValidatingWebhookConfiguration` object from the previous non-Helm installation: + + ```sh + kubectl delete validatingwebhookconfiguration redis-enterprise-admission + ``` + + This step is only needed when the `admission.limitToNamespace` chart value is set to `true` (the default). In this case, the webhook object installed by the chart is named `redis-enterprise-admission-`, and the original webhook object, named `redis-enterprise-admission`, becomes redundant. If `admission.limitToNamespace` is set to `false`, the webhook installed by the chart is named `redis-enterprise-admission`, and the existing webhook object is reused. + +## Known limitations + +- Custom resources other than the REC must still be created using the standard process. The chart doesn't manage them. +- The chart doesn't include configuration options for multiple namespaces, rack-awareness, and Vault integration. The steps for configuring these options remain the same. +- The chart has had limited testing in advanced setups, including Active-Active configurations, air-gapped deployments, and IPv6/dual-stack environments. \ No newline at end of file diff --git a/content/operate/kubernetes/8.2.0/deployment/openshift/_index.md b/content/operate/kubernetes/8.2.0/deployment/openshift/_index.md new file mode 100644 index 0000000000..af47bb11af --- /dev/null +++ b/content/operate/kubernetes/8.2.0/deployment/openshift/_index.md @@ -0,0 +1,43 @@ +--- +Title: Deploy Redis Enterprise for Kubernetes with OpenShift +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: A quick introduction to the steps necessary to get a Redis Enterprise + cluster installed in your OpenShift Kubernetes cluster +hideListLinks: true +linkTitle: OpenShift +weight: 11 +url: '/operate/kubernetes/8.2.0/deployment/openshift/' +--- +The deployment of Redis Enterprise clusters is managed with the Redis Enterprise operator that you deploy in the namespace for your project. +To create a database that your application +workloads can use: + +1. Install the Redis Enterprise operator. + +1. Create a Redis Enterprise CRD to describe your desired cluster. + +1. The operator reads this cluster description and deploys the various components on your K8s cluster. + +1. Once running, use the Redis Enterprise cluster to create a database. + +1. The operator automatically exposes the new database as a K8s service. + +## For OpenShift via the OperatorHub + +To [create a database on an OpenShift 4.x cluster via the OperatorHub](/content/operate/kubernetes/8.2.0/deployment/openshift/openshift-operatorhub.md) you only need to have the [OpenShift 4.x cluster installed](https://docs.openshift.com/container-platform/4.3/welcome/index.html) with at least three nodes that each meet the [minimum requirements for a development installation](/content/operate/rs/installing-upgrading/install/plan-deployment/hardware-requirements.md). + +## For OpenShift via the CLI + +To [create a database on an OpenShift cluster via the CLI](/content/operate/kubernetes/8.2.0/deployment/openshift/openshift-cli.md), you need: + +1. An [OpenShift cluster installed](https://docs.openshift.com/container-platform/4.3/welcome/index.html) with at least three nodes that each meet the [minimum requirements for a development installation](/content/operate/rs/installing-upgrading/install/plan-deployment/hardware-requirements.md). +1. The [kubectl package installed](https://kubernetes.io/docs/tasks/tools/install-kubectl/) at version 1.9 or higher +1. The [OpenShift cli installed](https://docs.openshift.com/container-platform/4.2/cli_reference/openshift_cli/getting-started-cli.html) + +## Version compatibility + +To see which version of Redis Enterprise for Kubernetes supports your OpenShift version, see [Supported Kubernetes distributions](/content/operate/kubernetes/8.2.0/reference/supported_k8s_distributions.md). \ No newline at end of file diff --git a/content/operate/kubernetes/8.2.0/deployment/openshift/openshift-cli.md b/content/operate/kubernetes/8.2.0/deployment/openshift/openshift-cli.md new file mode 100644 index 0000000000..1d81360bc4 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/deployment/openshift/openshift-cli.md @@ -0,0 +1,224 @@ +--- +Title: Deployment with OpenShift CLI for Redis Enterprise for Kubernetes +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Redis Enterprise for Kubernetes and cluster can be installed via CLI + tools OpenShift +linkTitle: OpenShift CLI +weight: 60 +url: '/operate/kubernetes/8.2.0/deployment/openshift/openshift-cli/' +--- + +Use these steps to set up a Redis Enterprise Software cluster with OpenShift. + +## Prerequisites + +- [OpenShift cluster](https://docs.openshift.com/container-platform/4.8/installing/index.html) with at least 3 nodes (each meeting the [minimum requirements for a development installation](/content/operate/rs/installing-upgrading/install/plan-deployment/hardware-requirements.md)) +- [OpenShift CLI](https://docs.openshift.com/container-platform/latest/cli_reference/openshift_cli/getting-started-cli.html) + +To see which version of Redis Enterprise for Kubernetes supports your OpenShift version, see [Supported Kubernetes distributions](/content/operate/kubernetes/8.2.0/reference/supported_k8s_distributions.md). + +> [!NOTE] +> If you suspect your file descriptor limits are below 100,000, you must either manually increase limits or [Allow automatic resource adjustment](/content/operate/kubernetes/8.2.0/security/allow-resource-adjustment.md). Most major cloud providers and standard container runtime configurations set default file descriptor limits well above the minimum required by Redis Enterprise. In these environments, you can safely run without enabling automatic resource adjustment. + +## Deploy the operator + +1. Create a new project. + + ```sh + oc new-project + ``` + +1. Verify the newly created project. + + ```sh + oc project + ``` + +1. Get the deployment files. + + ```sh + git clone https://github.com/RedisLabs/redis-enterprise-k8s-docs + ``` + +1. Deploy the OpenShift operator bundle. + + If you are using version 6.2.18-41 or earlier, you must [apply the security context constraint](#install-security-context-constraint) before the operator bundle. + + ```sh + oc apply -f openshift.bundle.yaml + ``` + + > [!WARNING] + > Changes to the `openshift.bundle.yaml` file can cause unexpected results. + > + +1. Verify that your `redis-enterprise-operator` deployment is running. + + ```sh + oc get deployment + ``` + + A typical response looks like this: + + ```sh + NAME READY UP-TO-DATE AVAILABLE AGE + redis-enterprise-operator 1/1 1 1 0m36s + ``` + + > [!WARNING] + > DO NOT modify or delete the StatefulSet created during the deployment process. Doing so could destroy your Redis Enterprise cluster (REC). + > + +## Security context constraints + +Versions 7.22.0-6 and later run in without permissions to [allow automatic resource adjustment](/content/operate/kubernetes/8.2.0/security/allow-resource-adjustment.md). If you use the recommended default security constraints, remove the existing `redis-enterprise-scc-v2` SCC and unbind it from the REC service account after upgrading. + +## Create a Redis Enterprise cluster custom resource + +1. Download the `RedisEnterpriseCluster` resource file ([rec.yaml](https://github.com/RedisLabs/redis-enterprise-k8s-docs/blob/master/examples/v1/rec.yaml)). + + You can rename the file to `.yaml`, but it is not required. Examples below use `rec.yaml`. [Options for Redis Enterprise clusters](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md) has more info about the Redis Enterprise cluster (REC) custom resource, or see the [Redis Enterprise cluster API](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md) for a full list of options. + + > [!NOTE] + > If you suspect your file descriptor limits are below 100,000, you must either manually increase limits or [Allow automatic resource adjustment](/content/operate/kubernetes/8.2.0/security/allow-resource-adjustment.md). Most major cloud providers and standard container runtime configurations set default file descriptor limits well above the minimum required by Redis Enterprise. In these environments, you can safely run without enabling automatic resource adjustment. + > + + The REC name cannot be changed after cluster creation. + + > [!NOTE] + > Each Redis Enterprise cluster requires at least 3 nodes. Single-node RECs are not supported. + > + +2. Apply the custom resource file to create your Redis Enterprise cluster. + + > [!NOTE] + > If you enabled automatic resource adjustment in your configuration, this step will trigger the operator to apply elevated capabilities. Ensure your security context allows it. + > + + ```sh + oc apply -f rec.yaml + ``` + + The operator typically creates the REC within a few minutes. + +1. Check the cluster status. + + ```sh + oc get pod + ``` + + You should receive a response similar to the following: + + ```sh + NAME | READY | STATUS | RESTARTS | AGE | + | -------------------------------- | ----- | ------- | -------- | --- | + | rec-name-0 | 2/2 | Running | 0 | 1m | + | rec-name-1 | 2/2 | Running | 0 | 1m | + | rec-name-2 | 2/2 | Running | 0 | 1m | + | rec-name-controller-x-x | 1/1 | Running | 0 | 1m | + | Redis-enterprise-operator-x-x | 1/1 | Running | 0 | 5m | + ``` + +## Configure the admission controller + +{{< embed-md "k8s-admission-webhook-cert.md" >}} + +### Limit the webhook to relevant namespaces + +If not limited, the webhook intercepts requests from all namespaces. If you have several REC objects in your Kubernetes cluster, limit the webhook to the relevant namespaces. If you aren't using multiple namespaces, skip this step. + +1. Verify your namespace is labeled and the label is unique to this namespace, as shown in the next example. + + ```sh + apiVersion: v1 + kind: Namespace + metadata: + labels: + namespace-name: staging + name: staging + ``` + +1. Patch the webhook spec with the `namespaceSelector` field. + + ```sh + cat > modified-webhook.yaml < [!NOTE] + > For releases before 6.4.2-4, use this command instead: + > + > ```sh + > oc patch ValidatingWebhookConfiguration \ + > redb-admission --patch "$(cat modified-webhook.yaml)" + > ``` + > + > The 6.4.2-4 release introduces a new `ValidatingWebhookConfiguration` to replace `redb-admission`. See the [6.4.2-4 release notes](/content/operate/kubernetes/release-notes/previous-releases/_index.md). + > + +### Verify admission controller installation + +Apply an invalid resource as shown below to force the admission controller to reject it. If it applies successfully, the admission controller is not installed correctly. + +```sh +oc apply -f - << EOF +apiVersion: app.redislabs.com/v1alpha1 +kind: RedisEnterpriseDatabase +metadata: + name: redis-enterprise-database +spec: + evictionPolicy: illegal +EOF +``` + +You should see this error from the admission controller webhook `redisenterprise.admission.redislabs`. + +```sh +Error from server: error when creating "STDIN": admission webhook "redisenterprise.admission.redislabs" denied the request: eviction_policy: u'illegal' is not one of [u'volatile-lru', u'volatile-ttl', u'volatile-random', u'allkeys-lru', u'allkeys-random', u'noeviction', u'volatile-lfu', u'allkeys-lfu'] +``` + +## Create a Redis Enterprise database custom resource + +The operator uses the instructions in the Redis Enterprise database (REDB) custom resources to manage databases on the Redis Enterprise cluster. + +1. Create a `RedisEnterpriseDatabase` custom resource. + + This example creates a test database. For production databases, see [create a database](/content/operate/kubernetes/8.2.0/re-databases/db-controller.md#create-a-database) and [RedisEnterpriseDatabase API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md). + + ```sh + cat << EOF > /tmp/redis-enterprise-database.yml + apiVersion: app.redislabs.com/v1alpha1 + kind: RedisEnterpriseDatabase + metadata: + name: redis-enterprise-database + spec: + memorySize: 100MB + EOF + ``` + +1. Apply the newly created REDB resource. + + ```sh + oc apply -f /tmp/redis-enterprise-database.yml + ``` + +## More info + +- [Redis Enterprise cluster API](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md) +- [Redis Enterprise database API](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md) diff --git a/content/operate/kubernetes/8.2.0/deployment/openshift/openshift-operatorhub.md b/content/operate/kubernetes/8.2.0/deployment/openshift/openshift-operatorhub.md new file mode 100644 index 0000000000..532d445940 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/deployment/openshift/openshift-operatorhub.md @@ -0,0 +1,67 @@ +--- +Title: Deploy Redis Enterprise with OpenShift OperatorHub +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: OpenShift provides the OperatorHub where you can install the Redis Enterprise + operator from the administrator user interface. +linkTitle: OpenShift OperatorHub +weight: 70 +url: '/operate/kubernetes/8.2.0/deployment/openshift/openshift-operatorhub/' +--- +You can deploy Redis Enterprise for Kubernetes from the Red Hat OpenShift CLI. You can also use a UI, [OperatorHub](https://docs.openshift.com/container-platform/4.11/operators/index.html) (Red Hat) to install operators and create custom resources. + +> [!NOTE] +> If you suspect your file descriptor limits are below 100,000, you must either manually increase limits or [Allow automatic resource adjustment](/content/operate/kubernetes/8.2.0/security/allow-resource-adjustment.md). Most major cloud providers and standard container runtime configurations set default file descriptor limits well above the minimum required by Redis Enterprise. In these environments, you can safely run without enabling automatic resource adjustment. + +To see which version of Redis Enterprise for Kubernetes supports your OpenShift version, see [Supported Kubernetes distributions](/content/operate/kubernetes/8.2.0/reference/supported_k8s_distributions.md). + +## Install the Redis Enterprise operator + +> [!WARNING] +> If using version 6.2.18-41 or earlier, [Install the security context constraint](#install-security-context-constraint) before installing the operator. + +1. Select **Operators > OperatorHub**. + +2. Search for _Redis Enterprise_ in the search dialog and select the **Redis Enterprise Operator provided by Redis** marked as **Certified**. + + By default, the image is pulled from Red Hat's registry. + +3. On the **Install Operator** page, specify the namespace for the operator. + + Only one namespace per operator is supported. + +4. Update the **channel** with the version you're installing. + + For more information about specific versions, see the [release notes](/content/operate/kubernetes/release-notes/_index.md). + +5. Choose an approval strategy. + + Use **Manual** for production systems to ensure the operator is only upgraded by approval. + +6. Select **Install** and approve the install plan. + + You can monitor the subscription status in **Operators > Installed Operators**. + +> [!WARNING] +> DO NOT modify or delete the StatefulSet created during the deployment process. Doing so could destroy your Redis Enterprise cluster (REC). + +## Security context constraints + +Versions 7.22.0-6 and later run in without permissions to [allow automatic resource adjustment](/content/operate/kubernetes/8.2.0/security/allow-resource-adjustment.md). If you use the recommended default security constraints, remove the existing `redis-enterprise-scc-v2` SCC and unbind it from the REC service account after upgrading. + +## Create Redis Enterprise custom resources + +The **Installed Operators**->**Operator details** page shows the provided APIs: **RedisEnterpriseCluster** and **RedisEnterpriseDatabase**. You can select **Create instance** to create custom resources using the OperatorHub interface. + + +Use the YAML view to create a custom resource file or let OperatorHub generate the YAML file for you by specifying your configuration options in the form view. + +> [!NOTE] +> If you suspect your file descriptor limits are below 100,000, you must either manually increase limits or [Allow automatic resource adjustment](/content/operate/kubernetes/8.2.0/security/allow-resource-adjustment.md). Most major cloud providers and standard container runtime configurations set default file descriptor limits well above the minimum required by Redis Enterprise. In these environments, you can safely run without enabling automatic resource adjustment. + + The REC name cannot be changed after cluster creation. + +For more information on creating and maintaining Redis Enterprise custom resources, see [Redis Enterprise clusters (REC)](/content/operate/kubernetes/8.2.0/re-clusters/_index.md) and [Redis Enterprise databases (REDB)](/content/operate/kubernetes/8.2.0/re-databases/_index.md). diff --git a/content/operate/kubernetes/8.2.0/deployment/quick-start.md b/content/operate/kubernetes/8.2.0/deployment/quick-start.md new file mode 100644 index 0000000000..c03fad451b --- /dev/null +++ b/content/operate/kubernetes/8.2.0/deployment/quick-start.md @@ -0,0 +1,277 @@ +--- +Title: Deploy Redis Enterprise Software for Kubernetes +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: How to install Redis Enterprise Software for Kubernetes. +linkTitle: Kubernetes +weight: 10 +url: '/operate/kubernetes/8.2.0/deployment/quick-start/' +--- +To deploy Redis Enterprise Software for Kubernetes and start your Redis Enterprise cluster (REC), you need to do the following: + +- Create a new namespace in your Kubernetes cluster. +- Download the operator bundle. +- Apply the operator bundle and verify it's running. +- Create a Redis Enterprise cluster (REC). + +This guide works with most supported Kubernetes distributions. If you're using OpenShift, see [Redis Enterprise on OpenShift](/content/operate/kubernetes/8.2.0/deployment/openshift/_index.md). For details on what is currently supported, see [supported distributions](/content/operate/kubernetes/8.2.0/reference/supported_k8s_distributions.md). + +## Prerequisites + +To deploy Redis Enterprise for Kubernetes, you'll need: + +- Kubernetes cluster in a [supported distribution](/content/operate/kubernetes/8.2.0/reference/supported_k8s_distributions.md) +- minimum of three worker nodes +- Kubernetes client (kubectl) +- access to DockerHub, RedHat Container Catalog, or a private repository that can hold the required images. + +If you suspect your file descriptor limits are below 100,000, you must either manually increase limits or [Allow automatic resource adjustment](/content/operate/kubernetes/8.2.0/security/allow-resource-adjustment.md). Most major cloud providers and standard container runtime configurations set default file descriptor limits well above the minimum required by Redis Enterprise. In these environments, you can safely run without enabling automatic resource adjustment. + +> [!NOTE] +> If you are applying version 7.8.2-6 or above, check if the [OS](https://redis.io/docs/latest/operate/kubernetes/release-notes/7-8-2-releases/7-8-2-6-nov24/#breaking-changes) installed on the node is supported. + +### Create a new namespace + +**Important:** Each namespace can only contain one Redis Enterprise cluster. Multiple RECs with different operator versions can coexist on the same Kubernetes cluster, as long as they are in separate namespaces. + +Throughout this guide, each command is applied to the namespace in which the Redis Enterprise cluster operates. + +1. Create a new namespace + + ```sh + kubectl create namespace + ``` + +1. Change the namespace context to make the newly created namespace default for future commands. + + ```sh + kubectl config set-context --current --namespace= + ``` + +You can use an existing namespace as long as it does not contain any existing Redis Enterprise cluster resources. It's best practice to create a new namespace to make sure there are no Redis Enterprise resources that could interfere with the deployment. + +## Install the operator + +Redis Enterprise for Kubernetes bundle is published as a container image. A list of required images is available in the [release notes](/content/operate/kubernetes/release-notes/_index.md) for each version. + +The operator [definition and reference materials](https://github.com/RedisLabs/redis-enterprise-k8s-docs) are available on GitHub. The operator definitions are [packaged as a single generic YAML file](https://github.com/RedisLabs/redis-enterprise-k8s-docs/blob/master/bundle.yaml). + +> [!NOTE] +> If you do not pull images from DockerHub or another public registry, you need to use a [private container registry](/content/operate/kubernetes/8.2.0/deployment/container-images.md#manage-image-sources). + +### Download the operator bundle + +Pull the latest version of the operator bundle: + +```sh +VERSION=`curl --silent https://api.github.com/repos/RedisLabs/redis-enterprise-k8s-docs/releases/latest | grep tag_name | awk -F'"' '{print $4}'` +``` + + If you need a different release, replace `VERSION` with a specific release tag. + + Check version tags listed with the [operator releases on GitHub](https://github.com/RedisLabs/redis-enterprise-k8s-docs/releases) or by [using the GitHub API](https://docs.github.com/en/rest/reference/repos#releases) to ensure the version of the bundle is correct. + +### Deploy the operator bundle + +Apply the operator bundle in your REC namespace: + +```sh +kubectl apply -f https://raw.githubusercontent.com/RedisLabs/redis-enterprise-k8s-docs/$VERSION/bundle.yaml +``` + + You should see a result similar to this: + + ```sh + role.rbac.authorization.k8s.io/redis-enterprise-operator created + serviceaccount/redis-enterprise-operator created + rolebinding.rbac.authorization.k8s.io/redis-enterprise-operator created + customresourcedefinition.apiextensions.k8s.io/redisenterpriseclusters.app.redislabs.com configured + customresourcedefinition.apiextensions.k8s.io/redisenterprisedatabases.app.redislabs.com configured + deployment.apps/redis-enterprise-operator created + ``` + +> [!WARNING] +> DO NOT modify or delete the StatefulSet created during the deployment process. Doing so could destroy your Redis Enterprise cluster (REC). + +#### Verify the operator is running + +Check the operator deployment to verify it's running in your namespace: + +```sh +kubectl get deployment redis-enterprise-operator +``` + +You should see a result similar to this: + +```sh +NAME READY UP-TO-DATE AVAILABLE AGE +redis-enterprise-operator 1/1 1 1 0m36s +``` + +## Create a Redis Enterprise cluster (REC) + +A Redis Enterprise cluster (REC) is created from a `RedisEnterpriseCluster` custom resource +that contains cluster specifications. + +The following example creates a minimal Redis Enterprise cluster. See the [RedisEnterpriseCluster API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md) for more information on the various options available. + +> [!NOTE] +> If you suspect your file descriptor limits are below 100,000, you must either manually increase limits or [Allow automatic resource adjustment](/content/operate/kubernetes/8.2.0/security/allow-resource-adjustment.md). Most major cloud providers and standard container runtime configurations set default file descriptor limits well above the minimum required by Redis Enterprise. In these environments, you can safely run without enabling automatic resource adjustment. + +1. Create a file that defines a Redis Enterprise cluster with three nodes. + + > [!NOTE] + > The REC name (`my-rec` in this example) cannot be changed after cluster creation. + > + + ```sh + cat < my-rec.yaml + apiVersion: "app.redislabs.com/v1" + kind: "RedisEnterpriseCluster" + metadata: + name: my-rec + spec: + nodes: 3 + EOF + ``` + + This will request a cluster with three Redis Enterprise nodes using the default requests (i.e., 2 CPUs and 4GB of memory per node). + + To test with a larger configuration, use the example below to add node resources to the `spec` section of your test cluster (`my-rec.yaml`). + + ```sh + redisEnterpriseNodeResources: + limits: + cpu: 2000m + memory: 16Gi + requests: + cpu: 2000m + memory: 16Gi + ``` + + > [!NOTE] + > Each cluster must have at least 3 nodes. Single-node RECs are not supported. + > + + See the [Redis Enterprise hardware requirements](/content/operate/rs/installing-upgrading/install/plan-deployment/hardware-requirements.md) for more information on sizing Redis Enterprise node resource requests. + + > [!NOTE] + > If you enabled automatic resource adjustment in your configuration, this step will trigger the operator to apply elevated capabilities. Ensure your security context allows it. + > + +1. Apply your custom resource file in the same namespace as `my-rec.yaml`. + + ```sh + kubectl apply -f my-rec.yaml + ``` + + You should see a result similar to this: + + ```sh + redisenterprisecluster.app.redislabs.com/my-rec created + ``` + +1. You can verify the creation of the cluster with: + + ```sh + kubectl get rec + ``` + + You should see a result similar to this: + + ```sh + NAME AGE + my-rec 1m + ``` + + At this point, the operator will go through the process of creating various services and pod deployments. + + You can track the progress by examining the StatefulSet associated with the cluster: + + ```sh + kubectl rollout status sts/my-rec + ``` + + or by looking at the status of all of the resources in your namespace: + + ```sh + kubectl get all + ``` + +## Enable the admission controller + +The admission controller dynamically validates REDB resources configured by the operator. It is strongly recommended that you use the admission controller on your Redis Enterprise Cluster (REC). The admission controller only needs to be configured once per operator deployment. + +As part of the REC creation process, the operator stores the admission controller certificate in a Kubernetes secret called `admission-tls`. You may have to wait a few minutes after creating your REC to see the secret has been created. + +{{< embed-md "k8s-admission-webhook-cert" >}} + +### Limit the webhook to the relevant namespaces {#webhook} + +The operator bundle includes a webhook file. The webhook will intercept requests from all namespaces unless you edit it to target a specific namespace. You can do this by adding the `namespaceSelector` section to the webhook spec to target a label on the namespace. + +1. Make sure the namespace has a unique `namespace-name` label. + + ```sh + apiVersion: v1 + kind: Namespace + metadata: + name: example-ns + labels: + namespace-name: example-ns + ``` + +1. Patch the webhook to add the `namespaceSelector` section. + + ```sh + cat > modified-webhook.yaml < [!WARNING] +> Flex does not replace long-term data persistence. For workloads that require durability and recovery across restarts or failures, use Redis persistence features like [AOF (Append-Only File)](/content/operate/oss_and_stack/management/persistence.md#append-only-file), [RDB snapshots](/content/operate/oss_and_stack/management/persistence.md#snapshotting), or both. For more information, see [Database persistence](/content/operate/rs/databases/configure/database-persistence.md). + +## Flex and Auto Tiering + +Flex replaces [Auto Tiering](/content/operate/kubernetes/7.22/re-clusters/auto-tiering.md) (formerly known as Redis on Flash). Redis Enterprise selects the implementation based on your database version: + +| Redis database version | Operator version | Redis Flex | Auto Tiering | +|------------------------|------------------|------------|--------------| +| 8.0 and later | 8.0.2-2 and later | ✅ | ❌ | +| 7.4 | 7.8.2-6 and later | ✅ | ✅ | +| 7.2 and earlier | 7.22.2-22 and earlier | ❌ | ✅ | + +For Redis Enterprise for Kubernetes version 7.22.2-22 or earlier, see [Auto Tiering](/content/operate/kubernetes/7.22/re-clusters/auto-tiering.md). + +### Differences between Flex and Auto Tiering + +- Key and value offloading + - Auto Tiering offloads only values to flash while keys remain in RAM. + - Flex offloads both keys and values, which increases dataset density per node and reduces RAM consumption. +- RAM requirements + - Auto Tiering requires enough RAM to hold all keys. + - Flex has no requirement to keep keys in RAM. Minimum RAM is based on performance goals, not key count. The minimum RAM-to-flash ratio is 10%, which enables high flash utilization without compromising architecture. +- RAM population strategy + - Auto Tiering fills all available RAM before offloading data to flash. This maximizes hot-data performance but can cause non-linear performance changes at high utilization. + - Flex uses utilization-aware RAM population. +- Storage engine + - Auto Tiering uses either RocksDB or Speedb as the storage engine. + - Flex uses Speedb only. +- Hot data caching and dataset size + - Auto Tiering is limited by key memory overhead, which restricts maximum dataset size. + - Flex is more efficient because RAM holds hot keys and values, supporting larger datasets. +- Performance consistency + - Auto Tiering depends on RAM pressure and key count. + - Flex provides higher RAM hit-rate and more stable latency. + +## Next steps + +- [Plan your deployment](/content/operate/kubernetes/8.2.0/flex/plan.md): Review hardware requirements, sizing guidelines, and limitations. +- [Get started](/content/operate/kubernetes/8.2.0/flex/get-started.md): Configure Flex on your cluster. +- [Scale your deployment](/content/operate/kubernetes/8.2.0/flex/scale.md): Learn scaling strategies and best practices. diff --git a/content/operate/kubernetes/8.2.0/flex/auto-tiering.md b/content/operate/kubernetes/8.2.0/flex/auto-tiering.md new file mode 100644 index 0000000000..dd6807a88f --- /dev/null +++ b/content/operate/kubernetes/8.2.0/flex/auto-tiering.md @@ -0,0 +1,94 @@ +--- +Title: Use Auto Tiering on Kubernetes +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Deploy a cluster with Auto Tiering on Kubernetes. +linkTitle: Auto Tiering +weight: 40 +url: '/operate/kubernetes/8.2.0/flex/auto-tiering/' +--- + +> [!NOTE] +> This page applies to Redis database version 7.4 and earlier. If you use version 8.0 or later, see [Redis Flex](https://redis.io/docs/latest/operate/kubernetes/re-clusters/redis-flex). + +## Prerequisites + +Redis Enterprise Software for Kubernetes supports using Auto Tiering (previously known as Redis on Flash), which extends your node memory to use both RAM and flash storage. SSDs (solid state drives) can store infrequently used (warm) values while your keys and frequently used (hot) values are still stored in RAM. This improves performance and lowers costs for large datasets. + +> [!NOTE] +> NVMe (non-volatile memory express) SSDs are strongly recommended to achieve the best performance. + +Before creating your Redis clusters or databases, these SSDs must be: + +- [locally attached to worker nodes in your Kubernetes cluster](https://kubernetes.io/docs/concepts/storage/volumes/#local) +- formatted and mounted on the nodes that will run Redis Enterprise pods +- dedicated to Auto Tiering and not shared with other parts of the database, (e.g. durability, binaries) +- [provisioned as local persistent volumes](https://kubernetes.io/docs/concepts/storage/volumes/#local) + - You can use a [local volume provisioner](https://github.com/kubernetes-sigs/sig-storage-local-static-provisioner/blob/master/README.md) to do this [dynamically](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#dynamic) +- a [StorageClass](https://kubernetes.io/docs/concepts/storage/storage-classes/#local) resource with a unique name + +For more information on node storage, see [Node persistent and ephemeral storage](/content/operate/rs/installing-upgrading/install/plan-deployment/persistent-ephemeral-storage.md). + +## Create a Redis Enterprise cluster + +To deploy a Redis Enterprise cluster (REC) with Auto Tiering, you'll need to specify the following in the `redisOnFlashSpec` section of your [REC custom resource](/content/operate/kubernetes/7.22/reference/api/redis_enterprise_cluster_api.md): + +- enable Auto Tiering (`enabled: true`) +- flash storage driver (`bigStoreDriver`) + - `rocksdb` or `speedb`(default) +- storage class name (`storageClassName`) +- minimal flash disk size (`flashDiskSize`) + +> [!NOTE] +> Clusters upgraded to version 7.2.4-2 from an earlier version will change the `bigStoreDriver` (previously called `flashStorageEngine`) to the new default `speedb`, regardless of previous configuration. + +> [!WARNING] +> Switching between storage engines (`speedb` and `rocksdb`) requires guidance by Redis Support or your Account Manager. + +> [!WARNING] +> PVC expansion is not supported when using Auto Tiering. Do not enable `enablePersistentVolumeResize` in the REC `persistentSpec` if you are using `redisOnFlashSpec` as this will result in conflicts. + +Here is an example of an REC custom resource with these attributes: + +```YAML +apiVersion: app.redislabs.com/v1 +kind: RedisEnterpriseCluster +metadata: + name: "rec" +spec: + + nodes: 3 + redisOnFlashSpec: + enabled: true + bigStoreDriver: speedb + storageClassName: local-scsi + flashDiskSize: 100G +``` + +### Create a Redis Enterprise database + +By default, any new database will use RAM only. To create a Redis Enterprise database (REDB) that can use flash storage, specify the following in the `redisEnterpriseCluster` section of the REDB custom resource definition: + +- `isRof: true` enables Auto Tiering +- `rofRamSize` defines the RAM capacity for the database + +Below is an example REDB custom resource: + +```YAML +apiVersion: app.redislabs.com/v1alpha1 +kind: RedisEnterpriseDatabase +metadata: + name: autoteiring-redb +spec: + redisEnterpriseCluster: + name: rec + isRof: true + memorySize: 2GB + rofRamSize: 0.5GB +``` + +> [!NOTE] +> This example defines both `memorySize` and `rofRamSize`. When using Auto Tiering, `memorySize` refers to the total combined memory size (RAM + flash) allocated for the database. `rofRamSize` specifies only the RAM capacity for the database. `rofRamSize` must be at least 10% of `memorySize`. diff --git a/content/operate/kubernetes/8.2.0/flex/get-started.md b/content/operate/kubernetes/8.2.0/flex/get-started.md new file mode 100644 index 0000000000..7cae218b14 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/flex/get-started.md @@ -0,0 +1,135 @@ +--- +title: Get started with Flex on Kubernetes +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Configure Flex on your Redis Enterprise cluster for Kubernetes. +hideListLinks: true +linkTitle: Get started +weight: 20 +aliases: /operate/kubernetes/re-clusters/redis-flex/ +url: '/operate/kubernetes/8.2.0/flex/get-started/' +--- + +> [!NOTE] +> This page applies to Redis database version 7.4 and earlier using Auto Tiering. If you use version 8.0 or later, see [Redis Flex](https://redis.io/docs/latest/operate/kubernetes/flex/). + +Flex extends your database capacity by combining RAM and flash (SSD) storage. This tiered architecture keeps frequently accessed (hot) data in RAM for sub-millisecond latency while storing less active (warm) data on flash to reduce costs and increase capacity. + +## Prerequisites + +Before you begin, verify that you have: + +- Redis Enterprise for Kubernetes version 8.0.2-2 or later installed +- A running `RedisEnterpriseCluster` (REC) resource +- Redis database version 7.4 or later +- Locally attached NVMe SSDs on your worker nodes +- A StorageClass configured for flash storage with a unique name + +For hardware requirements and sizing guidelines, see [Plan your deployment](/content/operate/kubernetes/8.2.0/flex/plan.md). + +## Configure the REC for Flex + +To enable Flex, configure your `RedisEnterpriseCluster` (REC) resource with flash storage settings. Add the [`redisOnFlashSpec`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#specredisonflashspec) section to your REC specification. + +Key fields in `redisOnFlashSpec`: + +| Field | Description | +|-------------------------|--------------------------------------------------| +| `enabled` | Set to `true` to enable Flex. | +| `flashStorageClassName` | Name of the StorageClass for flash storage. | +| `flashDiskSize` | Size of the flash storage per node. | +| `storageEngine` | Storage engine. Set to `speedb` for Flex. | + +For all available fields, see the [REC API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#specredisonflashspec). + +> [!WARNING] +> PVC expansion is not supported when using Flex. Do not enable `enablePersistentVolumeResize` in the REC `persistentSpec` if you are using `redisOnFlashSpec` as this will result in conflicts. + +1. Create a REC specification file with flash storage settings similar to the following example: + + ```yaml + apiVersion: app.redislabs.com/v1 + kind: RedisEnterpriseCluster + metadata: + name: rec + spec: + nodes: 3 + redisOnFlashSpec: + enabled: true + flashStorageClassName: local-flash + flashDiskSize: 100Gi + storageEngine: speedb + ``` + +1. Apply the REC configuration: + + ```sh + kubectl apply -f rec.yaml + ``` + +1. Verify that the cluster is ready: + + ```sh + kubectl get rec + ``` + +## Create a Flex database + +After you configure the cluster, create a `RedisEnterpriseDatabase` (REDB) resource with Flex enabled. + +Key fields in the REDB: + +| Field | Description | +|--------------|--------------------------------------------------------| +| `memorySize` | Total database size (RAM + flash). | +| `isRof` | Set to `true` to enable Flex for this database. | +| `rofRamSize` | Amount of RAM allocated to the database. | + +For all available fields, see the [REDB API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md). + +1. Create a database specification file with `isRof` set to `true` similar to the following example: + + ```yaml + apiVersion: app.redislabs.com/v1alpha1 + kind: RedisEnterpriseDatabase + metadata: + name: flex-db + spec: + memorySize: 10Gi + isRof: true + rofRamSize: 2Gi + ``` + +1. Apply the REDB: + + ```sh + kubectl apply -f redb.yaml + ``` + +1. Verify the database status: + + ```sh + kubectl get redb + ``` + +## Verify Flex is active + +To confirm that Flex is working: + +1. Check the database status: + + ```sh + kubectl get redb -o yaml + ``` + +2. Look for `isRof: true` in the status section. + +3. Connect to the database and verify that data operations work correctly. + +## Next steps + +- [Scale your deployment](/content/operate/kubernetes/8.2.0/flex/scale.md): Learn how to scale volume, throughput, and infrastructure. +- [Plan your deployment](/content/operate/kubernetes/8.2.0/flex/plan.md): Review sizing guidelines and best practices. diff --git a/content/operate/kubernetes/8.2.0/flex/plan.md b/content/operate/kubernetes/8.2.0/flex/plan.md new file mode 100644 index 0000000000..bc85ffd420 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/flex/plan.md @@ -0,0 +1,122 @@ +--- +title: Plan a Redis Flex deployment on Kubernetes +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Hardware requirements, sizing guidelines, and limitations for Redis Flex on Kubernetes. +hideListLinks: true +linkTitle: Plan +weight: 10 +url: '/operate/kubernetes/8.2.0/flex/plan/' +--- + +Review the hardware requirements, sizing guidelines, and known limitations before you deploy Redis Flex. + +## Version requirements + +Redis Flex requires: + +- Redis Enterprise for Kubernetes version 8.0.2-2 or later +- Redis database version 8.2 or later + +## Hardware requirements + +### Flash storage requirements + +Your SSDs must meet these requirements: + +- **Locally attached** to worker nodes in your Kubernetes cluster. Network-attached storage (NAS), storage area networks (SAN), and cloud block storage (like AWS EBS) are not supported. +- **Dedicated** to Redis Flex. Don't share Flash storage with other database components like durability, binaries, or persistence. +- **Formatted and mounted** on the nodes that run Redis Enterprise pods. +- **Provisioned as local persistent volumes**. You can use a [local volume provisioner](https://github.com/kubernetes-sigs/sig-storage-local-static-provisioner) for dynamic provisioning. +- **Configured with a StorageClass** resource with a unique name. + +For more information about storage configuration, see [Kubernetes local volumes](https://kubernetes.io/docs/concepts/storage/volumes/#local). + +### Flash capacity sizing + +Provision Flash capacity that exceeds your total database size. The extra space accounts for: + +- Write buffers +- Space amplification +- Operational overhead + +### Recommended SSD types + +For best performance, use NVMe SSDs: + +| SSD type | Recommendation | +|-----------------------|--------------------------------| +| NVMe Gen 5 or Gen 4 | Recommended | +| NVMe Gen 3 | Supported (lower performance) | + +## Sizing guidelines + +### Shard fundamentals + +Redis Flex databases consist of shards—independent data partitions that handle a portion of the dataset and request load. Each shard runs as a self-contained Redis process with dedicated memory, CPU, and Flash resources. + +### Recommended shard size + +Use these standard building blocks for capacity planning: + +- **50 GB per shard**: Standard capacity unit +- **1 vCPU per shard**: Baseline compute allocation + +For example, a 1 TB dataset requires 20 shards (20 × 50 GB). + +### RAM-to-flash ratio + +The RAM-to-flash ratio directly affects throughput and latency: + +- More RAM: Higher throughput and lower latency +- Less RAM: Lower throughput, higher latency, and lower cost + +## Module compatibility + +### Supported + +Redis Flex supports: + +- All standard Redis data types +- Redis JSON +- Redis probabilistic data structures (Bloom filters, Count-Min Sketch, Top-K) + +### Not supported + +Redis Flex doesn't support: + +- Redis Query Engine (RQE) +- RedisTimeSeries +- Active-Active databases + +## Best practices + +- Store small keys and values (under 10 KB) when possible. Large objects reduce performance, and objects larger than 4 GB can't be stored in Flash. +- Enable replication before populating the database with data. If you enable replication after the database is created, recovery can fail because required persistence files might be missing. +- Increase RAM percentage if 99th-percentile latency exceeds your target or cache hit rates drop. +- Decrease RAM percentage if memory utilization is high but latency remains stable to lower cost and improve efficiency. +- Re-evaluate the ratio of RAM to Flash periodically as dataset size and access patterns evolve. + +## Known limitations + +- **Flash storage**: Must be locally attached. Network storage isn't supported. +- **Active-Active**: Not supported with Flex. +- **PVC expansion**: Not supported with `redisOnFlashSpec`. Don't enable `enablePersistentVolumeResize` in the REC `persistentSpec`. +- **Maximum object size**: Keys or values larger than 4 GB remain in RAM only. + +Avoid Flex in the following situations: + +- Your working set is significantly smaller than the total dataset and access is biased toward recent data (high RAM hit rate). +- Your workload uses very long key names, broad or random access patterns, or a very large working set. These patterns reduce Flex benefits and increase flash I/O. + +### Deprecated fields + +The `flashStorageEngine` field is deprecated. Use `bigStoreDriver` instead. + +## Next steps + +- [Get started](/content/operate/kubernetes/8.2.0/flex/get-started.md): Configure Redis Flex on your cluster. +- [Scale your deployment](/content/operate/kubernetes/8.2.0/flex/scale.md): Learn scaling strategies. diff --git a/content/operate/kubernetes/8.2.0/flex/scale.md b/content/operate/kubernetes/8.2.0/flex/scale.md new file mode 100644 index 0000000000..ba1e22c177 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/flex/scale.md @@ -0,0 +1,146 @@ +--- +title: Scale Redis Flex on Kubernetes +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Scaling strategies and methods for Redis Flex deployments on Kubernetes. +hideListLinks: true +linkTitle: Scale +weight: 30 +url: '/operate/kubernetes/8.2.0/flex/scale/' +--- + +This guide shows you how to scale Flex databases on Kubernetes to meet changing workload demands. + +## Choose a scaling strategy + +Use the following table to determine the best scaling strategy for your Flex deployment: + +| Goal | Recommended action | +|------|--------------------| +| Increase data capacity only without adding CPU | [Increase `memorySize` and decrease RAM percentage](#decrease-ram-to-flash-ratio) | +| Increase throughput only | [Add shards and vCPU](#add-shards-or-nodes) | +| Increase data capacity and throughput | [Add shards](#add-shards) | +| Improve latency under higher load | [Increase RAM percentage](#increase-ram-to-flash-ratio) | +| Reduce cost while maintaining performance | [Tune RAM-to-flash ratio](#decrease-ram-to-flash-ratio) | + +## Prerequisites + +Before you scale a Flex deployment, verify that your cluster has sufficient resources, such as memory, disk, and vCPU. + +## Scale volume + +If your dataset requires more capacity while maintaining performance, you can prepare a Flex database to store more data using one of the following options: + +1. Increase `memorySize` and [add shards](#add-shards). + +1. Increase `memorySize` and [decrease the RAM-to-flash ratio](#decrease-ram-to-flash-ratio). + +### Add shards + +You can add more shards to expand dataset capacity while maintaining the existing RAM-to-flash ratio. Throughput capacity also typically increases as a result of additional shards and infrastructure. This strategy is recommended when the dataset size and traffic are expected to grow together. + +Before you increase the dataset capacity and add shards, you need to add more RAM and vCPUs to handle the increased number of shards. + +To increase the dataset capacity and shards: + +1. Edit your REDB custom resource: + + ```sh + kubectl edit redb + ``` + +1. Increase `memorySize` in the `spec` section. + +1. Increase `shardCount` in the `spec` section. + +1. Save and close the file. + +### Decrease RAM-to-flash ratio + +You can allocate more data to the flash tier to increase the database capacity while keeping the same amount of RAM, shards, and vCPU. This strategy is recommended when scaling for volume only and SSD resources are underutilized. This increases capacity without adding CPU or RAM but can lower RAM hit-rate and increase p99 latency; monitor metrics before and after the change. + +To increase the dataset capacity and decrease the RAM-to-flash ratio: + +1. Edit your REDB custom resource: + + ```sh + kubectl edit redb + ``` + +1. Increase `memorySize` in the `spec` section. + +1. Decrease `rofRamSize` in the `redisOnFlashSpec` section. + +1. Save and close the file. + +## Scale throughput + +If your workload's read/write rate increases and latency starts to rise, you can prepare the database to handle more traffic using one of the following strategies: + +1. [Add shards or nodes](#add-shards-or-nodes). + +1. [Increase the RAM-to-flash ratio](#increase-ram-to-flash-ratio). + +### Add shards or nodes + +You can add more shards or nodes to distribute traffic and increase throughput without changing the RAM-to-flash ratio. Dataset size capacity also typically increases as a result of additional shards and infrastructure. This strategy is recommended when the dataset size and traffic are expected to grow together. + +Before you add shards or nodes, you need to add more RAM and vCPUs to handle the increased number of shards or nodes. This increases capacity and potential throughput but requires more RAM, vCPUs, and a rebalance operation. + +To add shards: + +1. Edit your REDB custom resource: + + ```sh + kubectl edit redb + ``` + +1. Increase `shardCount` in the `spec` section. + +1. Save and close the file. + +To add nodes to the cluster, increase the `nodes` count in your RedisEnterpriseCluster (REC) custom resource: + +```sh +kubectl edit rec +``` + +### Increase RAM-to-flash ratio + +To improve throughput and lower latency, you can expand the in-memory tier to serve a higher proportion of requests directly from RAM. This strategy is recommended when low latency is your primary goal and you don't need to increase the dataset size. + +Before increasing the RAM-to-flash ratio, you might need to add more nodes to accommodate additional RAM. This improves throughput and lowers latency by serving more requests from RAM, at the cost of higher RAM usage. + +To increase the RAM-to-flash ratio: + +1. Edit your REDB custom resource: + + ```sh + kubectl edit redb + ``` + +1. Increase `rofRamSize` in the `redisOnFlashSpec` section. + +1. Save and close the file. + +## Scale infrastructure + +You can increase or adjust the underlying resources supporting the database, such as CPU, memory, and disk. + +For Flex deployments on Kubernetes, ensure the cluster has sufficient physical resources before scaling. The cluster requires: + +- Enough RAM to support the desired in-memory dataset size. + +- Enough SSD capacity for flash-tier data. + +- Adequate vCPU to support increased shard count or throughput. + +> [!WARNING] +> Scaling operations will fail or underperform if the underlying cluster is resource-constrained. + +PVC expansion is not supported with `redisOnFlashSpec`. Plan flash storage capacity upfront and don't enable `enablePersistentVolumeResize` in the REC `persistentSpec`. + +See Flex [hardware requirements](/content/operate/kubernetes/8.2.0/flex/plan.md#hardware-requirements) for more information. diff --git a/content/operate/kubernetes/8.2.0/kubernetes-archive.md b/content/operate/kubernetes/8.2.0/kubernetes-archive.md new file mode 100644 index 0000000000..8737391563 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/kubernetes-archive.md @@ -0,0 +1,30 @@ +--- +Title: Archive +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Describes where to view the archive of Redis Enterprise for Kubernetes + documentation. +linkTitle: Archive +weight: 99999999999 +url: '/operate/kubernetes/8.2.0/kubernetes-archive/' +--- + +The Redis Enterprise for Kubernetes documentation is versioned. You are looking at the documentation for the latest release. + +- View the [8.0.18 docs](https://redis.io/docs/latest/operate/kubernetes/8.0.18/) for Redis Enterprise for Kubernetes versions 8.0.18-11 and 8.0.20-21. +- View the [8.0 docs](https://redis.io/docs/latest/operate/kubernetes/8.0/) for Redis Enterprise for Kubernetes versions 8.0.2-2 through 8.0.10-23. +- View the [7.22 docs](https://redis.io/docs/latest/operate/kubernetes/7.22/) for Redis Enterprise for Kubernetes versions 7.22.0-7 through 7.22.2-40. +- View the [7.8.6 docs](https://redis.io/docs/latest/operate/kubernetes/7.8.6/) for Redis Enterprise for Kubernetes versions 7.8.6-1 through 7.8.6-16. +- View the [7.8.4 docs](https://redis.io/docs/latest/operate/kubernetes/7.8.4/) for Redis Enterprise for Kubernetes versions 7.8.2-6, 7.8.4-8, and 7.8.4-9. +- View the [7.4.6 docs](https://redis.io/docs/latest/operate/kubernetes/7.4.6/) for Redis Enterprise for Kubernetes versions 7.4.6-2 and 7.4.6-6. + +Older versions of Redis Enterprise for Kubernetes documentation are available on the archived web site: + +- [Redis Enterprise for Kubernetes v7.4 documentation archive](https://docs.redis.com/7.4/kubernetes/) + +- [Redis Enterprise for Kubernetes v7.2 documentation archive](https://docs.redis.com/7.2/kubernetes/)   + +- [Redis Enterprise for Kubernetes v6.x documentation archive](https://docs.redis.com/6.4/kubernetes/) diff --git a/content/operate/kubernetes/8.2.0/logs/_index.md b/content/operate/kubernetes/8.2.0/logs/_index.md new file mode 100644 index 0000000000..60044e2b42 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/logs/_index.md @@ -0,0 +1,48 @@ +--- +Title: Logs +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Access and manage Redis Enterprise logs on Kubernetes for monitoring and troubleshooting. +hideListLinks: true +linkTitle: Logs +weight: 60 +url: '/operate/kubernetes/8.2.0/logs/' +--- + +Access and manage Redis Enterprise logs on Kubernetes for monitoring, troubleshooting, and debugging your Redis Enterprise deployment. Logs provide valuable insights into cluster operations, database performance, and system health. + +## Log collection and access + +Learn how to collect and access logs from your Redis Enterprise deployment: + +- [Collect logs](/content/operate/kubernetes/8.2.0/logs/collect-logs.md) - Methods for collecting logs from Redis Enterprise pods and containers + +## Log storage and access + +Each Redis Enterprise container stores its logs under `/var/opt/redislabs/log`. When using persistent storage, this path is automatically mounted to the `redis-enterprise-storage` volume, making logs accessible through sidecar containers or external log collection tools. + +For example, in the REC (Redis Enterprise Cluster) spec you can add a sidecar container, such as a busybox, and mount the logs to there: + +```yaml +sideContainersSpec: + - name: busybox + image: busybox + args: + - /bin/sh + - -c + - while true; do echo "hello"; sleep 1; done + + volumeMounts: + - name: redis-enterprise-storage + mountPath: /home/logs + subPath: logs +``` + +Now the logs can be accessed from in the sidecar. For example by running + +```kubectl exec -it -c busybox -- tail home/logs/supervisord.log``` + +The sidecar container is user determined and can be used to format, process and share logs in a specified format and protocol. diff --git a/content/operate/kubernetes/8.2.0/logs/collect-logs.md b/content/operate/kubernetes/8.2.0/logs/collect-logs.md new file mode 100644 index 0000000000..c85b09c0d1 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/logs/collect-logs.md @@ -0,0 +1,64 @@ +--- +Title: Collect logs +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Run the log collector script to package relevant logs into a tar.gz file + to send to Redis Support for help troubleshooting your Kubernetes environment. +linkTitle: Collect logs +weight: 89 +url: '/operate/kubernetes/8.2.0/logs/collect-logs/' +--- + +The Redis Enterprise cluster (REC) log collector script ([`log_collector.py`](https://github.com/RedisLabs/redis-enterprise-k8s-docs/blob/master/log_collector/log_collector.py)) creates and fills a directory with the relevant logs for your environment. These logs will help the support team with troubleshooting. + +The log collector tool has two modes: + +- **restricted** collects only resources and logs created by the operator and Redis Enterprise deployments +- **all** collects everything from your environment + +> [!NOTE] +> This script requires Python 3.11 or later. + +## Prerequisites + +Before running the log collector, ensure you have the appropriate RBAC permissions configured. See [Log collector RBAC examples](/content/operate/kubernetes/8.2.0/reference/yaml/log-collector-rbac.md) for detailed RBAC configuration instructions. + +## Collect logs + +1. Download the latest [`log_collector.py`](https://github.com/RedisLabs/redis-enterprise-k8s-docs/blob/master/log_collector/log_collector.py) file. + +1. Ensure your `kubectl` or `oc` CLI is configured to access the Kubernetes cluster you want to collect logs from. + +1. Have a K8s administrator run the script on the system that runs your `kubectl` or `oc` commands. + + ```bash + python log_collector.py + ``` + +## Options + +You can run `log_collector.py` with the following options: + +| Option | Description | +|--------|-------------| +| `-n`, `--namespace` | Sets the namespace(s) to collect from. Can be set to a single namespace, or multiple namespaces (comma-separated). When left empty, will use the current context's namespace from kubeconfig. | +| `-o`, `--output_dir` | Sets the output directory. Defaults to current working directory. | +| `-a`, `--logs_from_all_pods` | Collect logs from all pods in the selected namespace(s), and otherwise collect only from the operator and pods run by the operator. | +| `-t`, `--timeout` | Time to wait for external commands to finish execution (Linux only). Defaults to 180s. Specify 0 to disable timeout. | +| `--k8s_cli` | The K8s cli client to use (kubectl/oc/auto-detect). Defaults to auto-detect (chooses between 'kubectl' and 'oc'). Full paths can also be used. | +| `-m`, `--mode` | Controls which resources are collected. In 'restricted' mode, only resources associated with the operator and have the label 'app=redis-enterprise' are collected. In 'all' mode, all resources are collected. Defaults to 'restricted' mode. | +| `--collect_istio` | Collect data from istio-system namespace to debug potential problems related to istio ingress method. | +| `--collect_empty_files` | Collect empty log files for missing resources. | +| `--helm_release_name` | Collect resources related to the given Helm release name. | +| `--collect_rbac_resources` | Temporary development flag. Collect all role based access control related custom resources. | +| `-h`, `--help` | Show help message and exit. | + +> [!NOTE] +> If you get an error because the yaml module is not found, install the pyYAML module with `pip install pyyaml`. + +1. Upload the resulting `tar.gz` file containing all the logs to [Redis Support](https://support.redislabs.com/). + + diff --git a/content/operate/kubernetes/8.2.0/networking/_index.md b/content/operate/kubernetes/8.2.0/networking/_index.md new file mode 100644 index 0000000000..627d8582de --- /dev/null +++ b/content/operate/kubernetes/8.2.0/networking/_index.md @@ -0,0 +1,54 @@ +--- +Title: Networking +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Configure networking and external access for Redis Enterprise clusters and databases on Kubernetes. +hideListLinks: true +linkTitle: Networking +weight: 40 +url: '/operate/kubernetes/8.2.0/networking/' +--- + +Configure networking and external access for your Redis Enterprise deployment on Kubernetes. By default, Kubernetes doesn't allow external access to your Redis databases. Redis Enterprise for Kubernetes provides several methods to route external traffic to your clusters and databases. + +## Database connectivity + +Connect applications to your Redis Enterprise databases: + +- [Database connectivity](/content/operate/kubernetes/8.2.0/networking/database-connectivity.md) - Comprehensive guide to in-cluster and external database access, service discovery, and credentials management. + +## External routing methods + +Choose the appropriate method for your environment to enable external access: + +- [Ingress routing](/content/operate/kubernetes/8.2.0/networking/ingress.md) - Use HAProxy ingress controller with `ingress` API resources (the community NGINX ingress controller is retired) +- [Istio ingress routing](/content/operate/kubernetes/8.2.0/networking/istio-ingress.md) - Use Istio service mesh with `Gateway` and `VirtualService` API resources +- [OpenShift routes](/content/operate/kubernetes/8.2.0/networking/routes.md) - Use OpenShift-specific route resources for external traffic + +## Automatic ingress configuration + +For Active-Active databases, configure automatic ingress creation: + +- [REC external routing](/content/operate/kubernetes/8.2.0/networking/ingressorroutespec.md) - Use `ingressOrRouteSpec` field in RedisEnterpriseCluster (REC) for automatic ingress creation + +## `ingressOrRouteSpec` for Active-Active databases + +Versions 6.4.2 or later of Redis Enterprise for Kubernetes include a feature for ingress configuration. The `ingressOrRouteSpec` field is available in the RedisEnterpriseCluster spec to automatically create an Ingress (or route) for the API service and databases (REAADB) on that REC. See [REC external routing](/content/operate/kubernetes/8.2.0/networking/ingressorroutespec.md) for more details. + +This feature only supports automatic Ingress creation for Active-Active databases created and managed with the RedisEnterpriseActiveActiveDatabase (REAADB) custom resource. Use with the standard Redis Enterprise database (REDB) is not currently supported. + +## OSS Cluster API support + +[OSS Cluster API](/content/operate/kubernetes/8.2.0/networking/cluster-aware-clients.md) is supported for both internal and external clients: + +- **Internal clients** running on pods within the same Kubernetes cluster can connect directly to Redis Enterprise pods using pod IPs. +- **External clients** outside the Kubernetes cluster can connect through LoadBalancer services when `enableExternalAccess: true` is configured in the database specification. This creates a separate LoadBalancer service for each Redis Enterprise pod. Note that LoadBalancers are resources that can significantly increase operational costs. + +See [Enable cluster-aware clients (OSS Cluster API)](/content/operate/kubernetes/8.2.0/networking/cluster-aware-clients.md) for configuration details. + +## REC domain name + +The RedisEnterpriseCluster does not support custom domain names. Domain names for the REC are in the following format: `..svc.cluster.local`. diff --git a/content/operate/kubernetes/8.2.0/networking/cluster-aware-clients.md b/content/operate/kubernetes/8.2.0/networking/cluster-aware-clients.md new file mode 100644 index 0000000000..597513832e --- /dev/null +++ b/content/operate/kubernetes/8.2.0/networking/cluster-aware-clients.md @@ -0,0 +1,91 @@ +--- +Title: Enable cluster-aware clients (OSS Cluster API) +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Improve performance with cluster-aware clients by enabling the OSS Cluster API for your Redis Enterprise for Kubernetes database. +linkTitle: Cluster-aware clients +weight: 99 +url: '/operate/kubernetes/8.2.0/networking/cluster-aware-clients/' +--- + +The OSS Cluster API improves performance by allowing cluster-aware Redis clients to discover database topology and route requests directly. This feature supports cluster-aware clients running on the same Kubernetes cluster (internal) and cluster-aware clients outside the Kubernetes cluster (external). + +> [!NOTE] +> Enabling external access for OSS Cluster API creates a separate LoadBalancer service for each Redis Enterprise pod in addition to the LoadBalancer service for the cluster endpoint. LoadBalancers are resources that can significantly increase operational costs; plan your infrastructure budget accordingly. + +## Prerequisites + +- RedisEnterpriseCluster (REC) running version 8.0.10-21 or later. +- Proxy policy is set to `all-master-shards` or `all-nodes`. +- Modules used by the database (if any) are bundled modules. +- The database is not an Active-Active database. + +## Limitations + +- OSS Cluster databases with external access must be created and managed using the RedisEnterpriseDatabase (REDB) custom resource. You cannot create or configure these databases using the Redis Enterprise REST API directly. + +## Enable OSS Cluster API + +To enable cluster-aware clients, edit the REC and REDB custom resources with the following fields. + +### Edit REC (RedisEnterpriseCluster) {#edit-rec} + +1. Edit the RedisEnterpriseCluster (REC) custom resource to add the `ossClusterSettings` section to the `spec` section. + +Set `externalAccessType` to `LoadBalancer` and add any `serviceAnnotations`, if required by your service provider. + +```yaml +ossClusterSettings: + externalAccessType: LoadBalancer + loadBalancer: + serviceAnnotations: +``` + +2. If you are using internal clients (within the Kubernetes cluster), you can specify their IP ranges in the `podCIDRs` field to improve performance. Clients from these ranges are routed directly to the Redis Enterprise pods, without going through the load balancer. + +The following is an example `podCIDRs` field using example values; replace with your own unique CIDRs. + +```yaml + podCIDRs: + - "192.0.2.0/24" + - "198.51.100.0/24" + - "203.0.113.0/24" +``` + +### Edit REDB (RedisEnterpriseDatabase) {#edit-redb} + +3. Edit the REDB custom resource to add the following fields and values to the `spec` section. + +Set `enableExternalAccess: true` to allow external clients to connect. This provisions a LoadBalancer service for each Redis Enterprise pod running a node, which increases infrastructure costs. + +```yaml + ossCluster: true + + ossClusterSettings: + enableExternalAccess: true +``` + +## Connect clients + +Use the `kubectl get svc` command to view the services created by the OSS Cluster API. It should look similar to this example: + +```sh +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +admission ClusterIP 10.0.1.10 443/TCP 3h28m +oss-cluster ClusterIP 10.0.1.20 11712/TCP 3h6m +oss-cluster-headless ClusterIP None 11712/TCP 3h6m +oss-cluster-load-balancer LoadBalancer 10.0.1.30 203.0.113.10 11712:30245/TCP 3h6m +rec ClusterIP 10.0.1.40 9443/TCP,8001/TCP 3h27m +rec-cluster LoadBalancer 10.0.1.50 203.0.113.20 11712:32559/TCP 3h6m +rec-1-lb LoadBalancer 10.0.1.60 203.0.113.30 11712:31976/TCP 3h6m +rec-2-lb LoadBalancer 10.0.1.70 203.0.113.40 11712:31972/TCP 3h6m +rec-oss ClusterIP None 8070/TCP 3h27m +rec-ui ClusterIP 10.0.1.80 8443/TCP 3h27m +``` + +To connect an external client (outside the Kubernetes cluster), use the `EXTERNAL-IP` address for the `oss-cluster-load-balancer` service. + +To connect an internal client (listed in the `podCIDRs` field), use the `CLUSTER-IP` address for the `oss-cluster` service. diff --git a/content/operate/kubernetes/8.2.0/networking/database-connectivity.md b/content/operate/kubernetes/8.2.0/networking/database-connectivity.md new file mode 100644 index 0000000000..507ccb22e7 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/networking/database-connectivity.md @@ -0,0 +1,304 @@ +--- +Title: Database connectivity +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Connect applications to Redis Enterprise databases in Kubernetes clusters with in-cluster and external access patterns. +linkTitle: Database connectivity +weight: 1 +url: '/operate/kubernetes/8.2.0/networking/database-connectivity/' +--- + +Connecting applications to Redis Enterprise databases in Kubernetes involves understanding service discovery, credentials management, and access patterns. This guide covers the essential connectivity aspects unique to Kubernetes deployments. + +## Service types and access patterns + +When you create a RedisEnterpriseDatabase (REDB), the Redis Enterprise operator automatically creates Kubernetes services to route traffic to your database. Understanding these service types is crucial for proper connectivity. + +### Default service creation + +By default, the operator creates two services for each database: + +- ClusterIP service - Provides a stable cluster-internal IP address. +- Headless service - Enables direct pod-to-pod communication and service discovery. + +Both services are created in the same namespace as your database and follow predictable naming conventions. + +### Service types + +Redis Enterprise supports three service types for database access: + +| Service Type | Rigger Spec Exact Name | Access Scope | Use Case | +|--------------|--------------|--------------|----------| +| `ClusterIP` | `cluster_ip` | Cluster-internal only | Applications running within the same Kubernetes cluster | +| `headless` | `headless` | Cluster-internal only | Direct pod access, service discovery, StatefulSet scenarios | +| `LoadBalancer` | `load_balancer` | External access | Applications outside the Kubernetes cluster | + +To configure the service type, use the `databaseServiceType` field in your REC's `servicesRiggerSpec`. + +For example, to enable the three Service options, you may use: +``` + {"spec":{"servicesRiggerSpec":{"databaseServiceType":"cluster_ip,headless,load_balancer"}}} +``` + +## In-cluster database access + +For applications running within your Kubernetes cluster, use the automatically created services to connect to your databases. + +### Retrieve connection information + +Database connection details are stored in a Kubernetes secret maintained by the database controller. This secret contains: + +- Database port (`port`) +- Service names (`service_names`) +- Database password (`password`) + +1. Get the secret name from your database resource: + + ```sh + kubectl get redb -o jsonpath="{.spec.databaseSecretName}" + ``` + + The secret name typically follows the pattern `redb-`. + +2. Retrieve the database port: + + ```sh + kubectl get secret redb- -o jsonpath="{.data.port}" | base64 --decode + ``` + +3. Get available service names: + + ```sh + kubectl get secret redb- -o jsonpath="{.data.service_names}" | base64 --decode + ``` + +4. Retrieve the database password: + + ```sh + kubectl get secret redb- -o jsonpath="{.data.password}" | base64 --decode + ``` + +### Service naming and DNS resolution + +Services follow standard Kubernetes DNS naming conventions: + +- Service FQDN: `..svc.cluster.local` +- Short name: `` (within the same namespace) + +For a database named `mydb` in namespace `production`, the service names would be: +- ClusterIP service: `mydb.production.svc.cluster.local` +- Headless service: `mydb-headless.production.svc.cluster.local` + +### Connect from within the cluster + +Use any service name from the `service_names` list to connect: + +```sh +redis-cli -h -p +``` + +Then authenticate with the retrieved password: + +```sh +auth +``` + +## External database access + +To access databases from outside the Kubernetes cluster, you need to configure external routing. Currently supported methods for external access are ingress controllers or OpenShift routes. + +### Ingress controllers + +Redis Enterprise for Kubernetes only supports the following ingress controllers for external database access: + +- HAProxy Ingress - Built-in SSL passthrough support. +- Istio Gateway - Service mesh integration with advanced traffic management. +- Ingress-NGINX - SSL passthrough is off by default; start the controller with `--enable-ssl-passthrough`. The community `kubernetes/ingress-nginx` project is retired (maintenance ended March 2026); existing deployments only. + +See [Ingress routing](/content/operate/kubernetes/8.2.0/networking/ingress.md) for detailed configuration steps. + +### OpenShift routes + +OpenShift users can leverage routes for external access: + +```yaml +apiVersion: route.openshift.io/v1 +kind: Route +metadata: + name: redis-route +spec: + to: + kind: Service + name: + port: + targetPort: redis + tls: + termination: passthrough +``` + +See [OpenShift routes](/content/operate/kubernetes/8.2.0/networking/routes.md) for complete setup instructions. + +## Service ports and configuration + +### Default port behavior + +- Redis Enterprise databases use dynamic port allocation. +- Port numbers are assigned automatically during database creation. +- The actual port is stored in the database secret. + +### Custom port configuration + +You can specify custom ports using the `databasePort` field in your REDB specification: + +```yaml +apiVersion: app.redislabs.com/v1alpha1 +kind: RedisEnterpriseDatabase +metadata: + name: mydb +spec: + memorySize: 256MB + databasePort: 6379 +``` + +Custom ports replace the default service port and are reflected in the database secret. + +## Credentials and secrets management + +### Database secrets structure + +Each database has an associated Kubernetes secret containing connection details: + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: redb- +type: Opaque +data: + port: + service_names: + password: +``` + +### Using secrets in applications + +Reference database secrets in your application deployments: + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: my-app +spec: + template: + spec: + containers: + - name: app + image: my-app:latest + env: + - name: REDIS_HOST + value: "" + - name: REDIS_PORT + valueFrom: + secretKeyRef: + name: redb- + key: port + - name: REDIS_PASSWORD + valueFrom: + secretKeyRef: + name: redb- + key: password +``` + +### Rotate the database password + +To rotate a database password, update the `password` key in the database secret. See [Change the REDB password](/content/operate/kubernetes/8.2.0/security/authentication/manage-redb-credentials.md#change-the-redb-password). + +### Default user configuration + +By default, databases create a default user with full access. You can disable this behavior: + +```yaml +apiVersion: app.redislabs.com/v1alpha1 +kind: RedisEnterpriseDatabase +metadata: + name: mydb +spec: + memorySize: 256MB + defaultUser: false +``` + +When `defaultUser` is disabled, the database secret is not created, and you must configure custom authentication. + +## Connection examples + +### Python application + +```python +import redis +import base64 +import os + +# Read from Kubernetes secret (mounted as environment variables) +host = os.getenv('REDIS_HOST') +port = int(os.getenv('REDIS_PORT')) +password = os.getenv('REDIS_PASSWORD') + +# Create Redis connection +r = redis.Redis(host=host, port=port, password=password, decode_responses=True) + +# Test connection +r.ping() +``` + +### Node.js application + +```javascript +const redis = require('redis'); + +const client = redis.createClient({ + host: process.env.REDIS_HOST, + port: process.env.REDIS_PORT, + password: process.env.REDIS_PASSWORD +}); + +client.on('connect', () => { + console.log('Connected to Redis'); +}); +``` + +## Troubleshooting connectivity + +### Common issues + +1. Connection refused - Verify service names and ports from the database secret. +2. Authentication failed - Check password encoding and special characters. +3. DNS resolution - Ensure applications use correct service FQDNs. +4. Network policies - Verify Kubernetes network policies allow traffic. + +### Debugging steps + +1. Verify service creation: + ```sh + kubectl get services -l app=redis-enterprise + ``` + +2. Check service endpoints: + ```sh + kubectl get endpoints + ``` + +3. Test connectivity from within cluster: + ```sh + kubectl run redis-test --image=redis:latest -it --rm -- redis-cli -h -p + ``` + +## Related topics + +- [Ingress routing](/content/operate/kubernetes/8.2.0/networking/ingress.md) - Configure external access with ingress controllers +- [OpenShift routes](/content/operate/kubernetes/8.2.0/networking/routes.md) - External access using OpenShift routes +- [Database controller](/content/operate/kubernetes/8.2.0/re-databases/db-controller.md) - Database lifecycle management +- [Security](/content/operate/kubernetes/8.2.0/security/_index.md) - TLS configuration and access control diff --git a/content/operate/kubernetes/8.2.0/networking/ingress.md b/content/operate/kubernetes/8.2.0/networking/ingress.md new file mode 100644 index 0000000000..739620a883 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/networking/ingress.md @@ -0,0 +1,229 @@ +--- +Title: Configure Ingress for external routing +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Configure an ingress controller to access your Redis Enterprise databases + from outside the Kubernetes cluster. +linkTitle: Ingress routing +weight: 5 +url: '/operate/kubernetes/8.2.0/networking/ingress/' +--- + +> [!WARNING] +> The community [Ingress-NGINX controller](https://github.com/kubernetes/ingress-nginx) (`kubernetes/ingress-nginx`) is retired. Best-effort maintenance ended in March 2026 and the project no longer ships releases, bug fixes, or security updates. If you are not already using it, use HAProxy or Istio as shown below. + +## Prerequisites + +Before creating an Ingress, you'll need: + + - A RedisEnterpriseDatabase (REDB) with TLS enabled for client connections + - An Ingress controller with `ssl-passthrough` enabled. Options include: + - [HAProxy Ingress](https://haproxy-ingress.github.io/docs/getting-started/) + - [Ingress-NGINX Controller](https://kubernetes.github.io/ingress-nginx/deploy/) (retired; existing deployments only) + - To use Istio for your Ingress resources, see [Configure Istio for external routing](/content/operate/kubernetes/8.2.0/networking/istio-ingress.md) + +> [!NOTE] +> Make sure your Ingress controller has `ssl-passthrough` enabled. It is on by default for HAProxy and off by default for Ingress-NGINX. For Ingress-NGINX, start the controller with the `--enable-ssl-passthrough` flag. See the [Ingress-NGINX TLS guide](https://kubernetes.github.io/ingress-nginx/user-guide/tls/#ssl-passthrough) for details. + +## Create an Ingress resource + +1. Retrieve the hostname of your Ingress controller's `LoadBalancer` service. + + ``` sh + $ kubectl get svc \ + -n + ``` + + Below is example output for an HAProxy running on a K8s cluster hosted by AWS. + + ``` sh + NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE + haproxy-ingress LoadBalancer 10.43.62.53 a56e24df8c6173b79a63d5da54fd9cff-676486416.us-east-1.elb.amazonaws.com 80:30610/TCP,443:31597/TCP 21m + ``` + +1. Choose the hostname you will use to access your database (this value will be represented in this article with ``). + +1. Create a DNS entry that resolves your chosen database hostname to the IP address for the Ingress controller's LoadBalancer. + +1. Create the Ingress resource YAML file. + + ``` YAML + apiVersion: networking.k8s.io/v1 + kind: Ingress + metadata: + name: rec-ingress + annotations: + + spec: + ingressClassName: + rules: + - host: + http: + paths: + - path: / + pathType: ImplementationSpecific + backend: + service: + name: + port: + name: redis + ``` + + Set `ingressClassName` to match the `IngressClass` your controller installed. The deprecated `kubernetes.io/ingress.class` annotation is no longer accepted by recent controller versions, so use `ingressClassName` instead. + + For HAProxy, add the following annotation: + + ``` YAML + ingress.kubernetes.io/ssl-passthrough: "true" + ``` + + For Ingress-NGINX, add the following annotation: + + ``` YAML + nginx.ingress.kubernetes.io/ssl-passthrough: "true" + ``` + + The `ssl-passthrough` annotation is required to allow access to the database. The exact format depends on your Ingress controller and any additional customizations. See [Ingress-NGINX annotations](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/) and [HAProxy Ingress configuration keys](https://haproxy-ingress.github.io/docs/configuration/keys/) for current annotation formats. + +## Test your external access + +To test your external access to the database, you need a client that supports [TLS](https://en.wikipedia.org/wiki/Transport_Layer_Security) and [SNI](https://en.wikipedia.org/wiki/Server_Name_Indication). + +#### Test your access with Openssl + +1. Get the default CA certificate from the `redis-enterprise-node` container on any of the Redis Enterprise pods. + + ``` sh + $ kubectl exec -it -c redis-enterprise-node \ + -- cat /etc/opt/redislabs/proxy_cert.pem + ``` + +1. Run the following `openssl` command, substituting your own values for ``. + + ``` sh + $ openssl s_client \ + -connect :443 \ + -crlf -CAfile ./proxy_cert.pem \ + -servername + ``` + + If you are connected to the database, you will receive `PONG` back, as shown below: + + ``` sh + ... + Verify return code: 0 (ok) + --- + + PING + +PONG + ``` + +#### Test your access with Python + +You can use the code below to test your access with Python, substituting your own values for `` and ``. + +``` python +import redis + +r = redis.Redis(host='', + port=443, db=0, ssl=True, + ssl_ca_certs='//proxy_cert.pem') + + +print(r.info()) +``` + +Your output should look something like this: + +``` sh +$ /Users/example-user/Documents/Projects/test_client/venv/bin/python \ + /Users/example-user/Documents/Projects/test_client/test_ssl.py +{ + 'redis_version': '5.0.5', + 'redis_git_sha1': 0, + 'redis_git_dirty': 0, + 'redis_build_id': 0, + 'redis_mode': 'standalone', + 'os': 'Linux 4.14.154-128.181.amzn2.x86_64 x86_64', + 'arch_bits': 64, + 'multiplexing_api': 'epoll', + 'gcc_version': '7.4.0', + 'process_id': 1, + 'run_id': '3ce7721b096517057d28791aab555ed8ac02e1de', + 'tcp_port': 10811, + 'uptime_in_seconds': 316467, + 'uptime_in_days': 3, + 'hz': 10, + 'lru_clock': 0, + 'config_file': '', + 'connected_clients': 1, + 'client_longest_output_list': 0, + 'client_biggest_input_buf': 0, + 'blocked_clients': 0, + 'used_memory': 12680016, + 'used_memory_human': '12.9M', + 'used_memory_rss': 12680016, + 'used_memory_peak': 13452496, + 'used_memory_peak_human': '12.82M', + 'used_memory_lua': 151552, + 'mem_fragmentation_ratio': 1, + 'mem_allocator': 'jemalloc-5.1.0', + 'loading': 0, + 'rdb_changes_since_last_save': 0, + 'rdb_bgsave_in_progress': 0, + 'rdb_last_save_time': 1577753916, + 'rdb_last_bgsave_status': 'ok', + 'rdb_last_bgsave_time_sec': 0, + 'rdb_current_bgsave_time_sec': -1, + 'aof_enabled': 0, + 'aof_rewrite_in_progress': 0, + 'aof_rewrite_scheduled': 0, + 'aof_last_rewrite_time_sec': -1, + 'aof_current_rewrite_time_sec': -1, + 'aof_last_bgrewrite_status': 'ok', + 'aof_last_write_status': 'ok', + 'total_connections_received': 4, + 'total_commands_processed': 6, + 'instantaneous_ops_per_sec': 14, + 'total_net_input_bytes': 0, + 'total_net_output_bytes': 0, + 'instantaneous_input_kbps': 0.0, + 'instantaneous_output_kbps': 0.0, + 'rejected_connections': 0, + 'sync_full': 1, + 'sync_partial_ok': 0, + 'sync_partial_err': 0, + 'expired_keys': 0, + 'evicted_keys': 0, + 'keyspace_hits': 0, + 'keyspace_misses': 0, + 'pubsub_channels': 0, + 'pubsub_patterns': 0, + 'latest_fork_usec': 0, + 'migrate_cached_sockets': 0, + 'role': 'master', + 'connected_slaves': 1, + 'slave0': { + 'ip': '0.0.0.0', + 'port': 0, + 'state': 'online', + 'offset': 0, + 'lag': 0 + }, + 'master_repl_offset': 0, + 'repl_backlog_active': 0, + 'repl_backlog_size': 1048576, + 'repl_backlog_first_byte_offset': 0, + 'repl_backlog_histlen': 0, + 'used_cpu_sys': 0.0, + 'used_cpu_user': 0.0, + 'used_cpu_sys_children': 0.0, + 'used_cpu_user_children': 0.0, + 'cluster_enabled': 0 +} + +Process finished with exit code 0 +``` diff --git a/content/operate/kubernetes/8.2.0/networking/ingressorroutespec.md b/content/operate/kubernetes/8.2.0/networking/ingressorroutespec.md new file mode 100644 index 0000000000..1c683e2df4 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/networking/ingressorroutespec.md @@ -0,0 +1,91 @@ +--- +Title: Establish external routing on the RedisEnterpriseCluster +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: null +linkTitle: REC external routing +weight: 30 +url: '/operate/kubernetes/8.2.0/networking/ingressorroutespec/' +--- +An Ingress is an API resource that provides a standardized and flexible way to manage external access to services running within a Kubernetes cluster. + +> [!WARNING] +> The community [Ingress-NGINX controller](https://github.com/kubernetes/ingress-nginx) (`kubernetes/ingress-nginx`) is retired. Best-effort maintenance ended in March 2026 and the project no longer ships releases, bug fixes, or security updates. If you are not already using it, use HAProxy or Istio. + +## Install Ingress controller + +Redis Enterprise for Kubernetes supports the Ingress controllers below: +* [HAProxy](https://haproxy-ingress.github.io/) +* [Istio](https://istio.io/latest/docs/setup/getting-started/) +* [Ingress-NGINX](https://kubernetes.github.io/ingress-nginx/) (retired; existing deployments only) + +OpenShift users can use [routes](/content/operate/kubernetes/8.2.0/networking/routes.md) instead of an Ingress. + +Install your chosen Ingress controller, making sure `ssl-passthrough` is enabled. It is on by default for HAProxy and off by default for Ingress-NGINX. For Ingress-NGINX, start the controller with the `--enable-ssl-passthrough` flag. + +## Configure DNS + +1. Choose the API hostname and database hostname suffix you will use, replacing `` with your own values. The recommended formats are: + + * REC API hostname (`apiFqdnUrl`): `api--.` + * Database hostname suffix (`dbFqdnSuffix`): `.db--.` + + The operator appends each database name to `dbFqdnSuffix` to build the per-database hostname. For example, a database named `mydb` with the suffix above resolves to `mydb.db--.`. For the wildcard DNS record, replace the database name with `*`, which resolves to `*.db--.`. + +1. Retrieve the `EXTERNAL-IP` of your Ingress controller's `LoadBalancer` service. + + ``` sh + $ kubectl get svc \ + -n + ``` + + Below is example output for an HAProxy ingress controller running on a K8s cluster hosted by AWS. + + ``` sh + NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE + haproxy-ingress LoadBalancer 10.43.62.53 a56e24df8c6173b79a63d5da54fd9cff-676486416.us-east-1.elb.amazonaws.com 80:30610/TCP,443:31597/TCP 21m + ``` + +1. Create DNS records to resolve your chosen REC API hostname and database hostname to the `EXTERNAL-IP` found in the previous step. + +## Edit the REC spec + +Edit the RedisEnterpriseCluster (REC) spec to add the `ingressOrRouteSpec` field, replacing `` below with your own values. + +### NGINX or HAproxy ingress controllers + +* Define the REC API hostname (`apiFqdnUrl`) and database hostname suffix (`dbFqdnSuffix`) you chose when configuring DNS. +* Set `method` to `ingress`. +* Set `ssl-passthrough` to "true". +* Add any additional annotations required for your ingress controller. See [NGINX docs](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/) or [HAproxy docs](https://haproxy-ingress.github.io/docs/configuration/keys/) for more information. + +```sh +kubectl patch rec --type merge --patch "{\"spec\": \ + {\"ingressOrRouteSpec\": \ + {\"apiFqdnUrl\": \"api--.example.com\", \ + \"dbFqdnSuffix\": \".db--.example.com\", \ + \"ingressAnnotations\": \ + {\"kubernetes.io/ingress.class\": \"\", \ + \"/ssl-passthrough\": \"true\"}, \ + \"method\": \"ingress\"}}}" +``` + +Set `` to `haproxy` or `nginx`. The operator validates that `ingressAnnotations` includes `kubernetes.io/ingress.class` and is the only annotation key it requires; add any other controller-specific annotations alongside it. + +### OpenShift routes + +* Define the REC API hostname (`apiFqdnUrl`) and database hostname suffix (`dbFqdnSuffix`) you chose when configuring DNS. +* Set `method` to `openShiftRoute`. + +```sh +kubectl patch rec --type merge --patch "{\"spec\": \ + {\"ingressOrRouteSpec\": \ + {\"apiFqdnUrl\": \"api--.example.com\" \ + \"dbFqdnSuffix\": \".db--.example.com\", \ + \"method\": \"openShiftRoute\"}}}" +``` + +OpenShift routes do not require any `ingressAnnotations` in the `ingressOrRouteSpec`. \ No newline at end of file diff --git a/content/operate/kubernetes/8.2.0/networking/istio-ingress.md b/content/operate/kubernetes/8.2.0/networking/istio-ingress.md new file mode 100644 index 0000000000..b6cb42413f --- /dev/null +++ b/content/operate/kubernetes/8.2.0/networking/istio-ingress.md @@ -0,0 +1,158 @@ +--- +Title: Configure Istio for external routing +alwaysOpen: false +categories: +- docs +- operate +- kubernetes +description: Configure Istio as an ingress controller for access to your Redis Enterprise + databases from outside the Kubernetes cluster. +linkTitle: Istio ingress routing +weight: 20 +url: '/operate/kubernetes/8.2.0/networking/istio-ingress/' +--- + +Redis Enterprise for Kubernetes has the ability to use an Istio Ingress gateway as an alternative to NGINX or HaProxy Ingress controllers. + +Istio can also understand Ingress resources, but using that mechanism takes away the advantages and options that the native Istio resources provide. Istio offers its own configuration methods using custom resources. + +To configure Istio to work with the Redis Kubernetes operator, we will use two custom resources: a `Gateway` and a `VirtualService`. Then you'll be able to establish external access to your database. + +> [!NOTE] +> When using Istio ingress for Redis on Kubernetes, avoid partial wildcard hostnames in your REC configuration. Use only valid FQDNs or leftmost wildcards for Istio compatibility and stable external routing. For more information, see [Istio Ingress Fails for Redis on Kubernetes with Partial Wildcard Hostnames](https://support.redislabs.com/hc/en-us/articles/30446436267666-Istio-Ingress-Fails-for-Redis-on-Kubernetes-with-Partial-Wildcard-Hostnames). + +## Install and configure Istio for Redis Enterprise + +1. [Download](https://istio.io/latest/docs/setup/getting-started/) and [install](https://istio.io/latest/docs/setup/getting-started/) Istio (see instructions from Istio's [Getting Started](https://istio.io/latest/docs/setup/getting-started/) guide). + + Once the installation is complete, all the deployments, pods, and services will be deployed in a namespace called `istio-system`. This namespace contains a `LoadBalancer` type service called `service/istio-ingressgateway` that exposes the external IP address. + +1. Find the `EXTERNAL-IP` for the `istio-ingressgateway` service. + + ```sh + kubectl get svc istio-ingressgateway -n istio-system + + NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE + istio-ingressgateway LoadBalancer 10.34.67.89 10.145.78.91 15021:12345/TCP,80:67891/TCP,443:23456/TCP,31400:78901/TCP,15443:10112/TCP 3h8m + ``` + +1. Create a DNS entry that resolves your chosen database hostname (or a wildcard `*` followed by your domain) to the Istio `EXTERNAL-IP`. Use this hostname to access your database from outside the cluster. + + In this example, any hostname that ends with `.istio.k8s.my.example.com` will resolve to the Istio LoadBalancer's external IP of `10.145.78.91`. Substitute your own values accordingly. + +1. Verify the record was created successfully. + + ```sh + dig api.istio.k8s.my.example.com + ``` + + Look in the `ANSWER SECTION` for the record you just created. + + ```sh + ;; ANSWER SECTION: + api.istio.k8s.my.example.com 0 IN A 10.145.78.91 + ``` + +## Create custom resources + +### `Gateway` custom resource + +1. On a different namespace from `istio-system`, create a `Gateway` custom resource file (`redis-gateway.yaml` in this example). + + - Replace `.istio.k8s.my.example.com` with the domain that matches your DNS record. + - Replace `` with the label set on your Istio ingress gateway pod. A default `istioctl install` deploys the gateway with label `istio: ingressgateway`. + - TLS passthrough mode is required to allow secure access to the database. + + ```yaml + apiVersion: networking.istio.io/v1beta1 + kind: Gateway + metadata: + name: redis-gateway + spec: + selector: + istio: + servers: + - hosts: + - '*.istio.k8s.my.example.com' + port: + name: tls + number: 443 + protocol: TLS + tls: + mode: PASSTHROUGH + ``` + +1. Apply the `Gateway` custom resource file to create the Ingress gateway. + + ```sh + kubectl apply -f redis-gateway.yaml + ``` + +1. Verify the gateway was created successfully. + + ```sh + kubectl get gateway + + NAME AGE + redis-gateway 3h33m + ``` + +### `VirtualService` custom resource + +1. On a different namespace than `istio-system`, create the `VirtualService` custom resource file (`redis-vs.yaml` in this example). + + ```yaml + apiVersion: networking.istio.io/v1beta1 + kind: VirtualService + metadata: + name: redis-vs + spec: + gateways: + - redis-gateway + hosts: + - "*.istio.k8s.my.example.com" + tls: + - match: + - port: 443 + sniHosts: + - api.istio.k8s.my.example.com + route: + - destination: + host: rec1 + port: + number: 9443 + - match: + - port: 443 + sniHosts: + - db1.istio.k8s.my.example.com + route: + - destination: + host: db1 + ``` + + This creates both a route to contact the API server on the REC (`rec1`) and a route to contact one of the databases (`db1`). + + - Replace `.istio.k8s.my.example.com` with the domain that matches your DNS record. + - The gateway's metadata name must be similar to the gateway's spec name (`redis-gateway` in this example). +1. Apply `VirtualService` custom resource file to create the virtual service. + + ```sh + kubectl apply -f redis-vs.yaml + ``` + +1. Verify the virtual service was created successfully. + + ```sh + kubectl get vs + + NAME GATEWAYS HOSTS AGE + redis-vs ["redis-gateway"] ["*.istio.k8s.my.example.com"] 3h33m + ``` + +1. [Deploy the operator](/content/operate/kubernetes/8.2.0/deployment/quick-start.md), Redis Enterprise Cluster (REC), and Redis Enterprise Database (REDB) on the same namespace as the gateway and virtual service. + +## Test your external access to the database + +To [test your external access](/content/operate/kubernetes/8.2.0/networking/ingress.md) to the database, you need a client that supports [TLS](https://en.wikipedia.org/wiki/Transport_Layer_Security) and [SNI](https://en.wikipedia.org/wiki/Server_Name_Indication). + +See [Test your access with Openssl](/content/operate/kubernetes/8.2.0/networking/ingress.md#test-your-access-with-openssl) or [Test your access with Python](/content/operate/kubernetes/8.2.0/networking/ingress.md#test-your-access-with-python) for more info. diff --git a/content/operate/kubernetes/8.2.0/networking/routes.md b/content/operate/kubernetes/8.2.0/networking/routes.md new file mode 100644 index 0000000000..5439ec167e --- /dev/null +++ b/content/operate/kubernetes/8.2.0/networking/routes.md @@ -0,0 +1,59 @@ +--- +Title: Use OpenShift routes for external database access +alwaysOpen: false +categories: +- docs +- operate +- kubernetes +description: null +linkTitle: OpenShift routes +weight: 15 +url: '/operate/kubernetes/8.2.0/networking/routes/' +--- + +OpenShift routes allow requests to be routed to the database or cluster API from outside the cluster. For more information about routes, see [OpenShift documentation](https://docs.openshift.com/container-platform/4.13/networking/routes/route-configuration.html). + +## Prerequisites + +* Before you can connect to your database from outside the cluster, you'll need the root CA certificate of the DMC Proxy server to validate the server certificate. + + By default, the DMC Proxy uses a self-signed certificate. You can retrieve it from the Redis Enterprise admin console, or from the machine at `/etc/opt/redislabs/proxy_cert.pem` (default location), and save it as a file (for example, named "ca.pem") on the client machine. + +* Your database also needs TLS encryption enabled. + +## Create OpenShift route + +1. Select the **Networking/Routes** section of the OpenShift web console. + +1. Select **Create route** and fill out the following fields: + + * **Name**: Choose any name you want as the first part of your generated hostname + * **Hostname**: Leave blank + * **Path**: Leave blank + * **Service**: Select the service for the database you want to access + * **TLS Termination**: Choose "passthrough" + * **Insecure Traffic**: Select "None" + +1. Select **Create**. + +1. Find the hostname for your new route. After route creation, it appears in the "Host" field. + +1. Verify you have a DNS entry to resolve the hostname for your new route to the cluster's load balancer. + +## Access database + +Access the database from outside the cluster using `redis-cli` or `openssl`. + +To connect with `redis-cli`: + + ```sh + redis-cli -h -p 443 --tls --cacert ./ca.pem --sni + ``` + +Replace the `` value with the hostname for your new route. + +To connect with `openssl`: + + ```sh + openssl s_client -connect :443 -crlf -CAfile ./ca.pem -servername + ``` diff --git a/content/operate/kubernetes/8.2.0/re-clusters/_index.md b/content/operate/kubernetes/8.2.0/re-clusters/_index.md new file mode 100644 index 0000000000..0a38e5c870 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/re-clusters/_index.md @@ -0,0 +1,65 @@ +--- +Title: Redis Enterprise clusters (REC) +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Create and manage Redis Enterprise clusters (REC) on Kubernetes using the Redis Enterprise operator. +hideListLinks: true +linkTitle: Redis Enterprise clusters (REC) +weight: 30 +url: '/operate/kubernetes/8.2.0/re-clusters/' +--- + +A Redis Enterprise cluster (REC) is a custom Kubernetes resource that represents a Redis Enterprise cluster deployment. The Redis Enterprise operator manages the lifecycle of REC resources, including deployment, scaling, upgrades, and recovery operations. + +REC resources define the cluster configuration, including node specifications, storage requirements, security settings, and networking configuration. After you deploy the cluster, it provides a foundation for creating and managing Redis Enterprise databases (REDB). + +## Cluster management + +Manage your Redis Enterprise cluster lifecycle and configuration: + +- [Connect to admin console](/content/operate/kubernetes/8.2.0/re-clusters/connect-to-admin-console.md) - Access the Redis Enterprise web UI for cluster management +- [Multi-namespace deployment](/content/operate/kubernetes/8.2.0/re-clusters/multi-namespace.md) - Deploy clusters across multiple Kubernetes namespaces +- [Delete custom resources](/content/operate/kubernetes/8.2.0/re-clusters/delete-custom-resources.md) - Safely remove REC and related resources + +## Storage and performance + +Optimize storage and performance for your Redis Enterprise cluster: + +- [Redis Flex](/content/operate/kubernetes/8.2.0/flex/_index.md) - Configure automatic data tiering between RAM and flash storage +- [Expand PVC](/content/operate/kubernetes/8.2.0/re-clusters/expand-pvc.md) - Expand persistent volume claims for additional storage + +## Monitoring and observability + +Monitor cluster health and performance: + +- [Connect to Prometheus operator](/content/operate/kubernetes/8.2.0/re-clusters/connect-prometheus-operator.md) - Integrate with Prometheus for metrics collection and monitoring + +### Call home client + +The call home client sends health or error data from your deployment(s) back to Redis. You can disable it by adding the following to your REC specification: + +```yaml +spec: + usageMeter: + callHomeClient: + disabled: true +``` + +> [!NOTE] +> The REST API approach used for Redis Software deployments will have no effect on Kubernetes deployments. You must use the REC specification method shown above. + +## Recovery and troubleshooting + +Handle cluster recovery and troubleshooting scenarios: + +- [Cluster recovery](/content/operate/kubernetes/8.2.0/re-clusters/cluster-recovery.md) - Recover from cluster failures and restore operations + +## Related topics + +- [Redis Enterprise databases (REDB)](/content/operate/kubernetes/8.2.0/re-databases/_index.md) - Create and manage databases on your cluster +- [Security](/content/operate/kubernetes/8.2.0/security/_index.md) - Configure security settings for your cluster +- [Networking](/content/operate/kubernetes/8.2.0/networking/_index.md) - Set up networking and ingress for cluster access +- [REC API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md) - Complete API specification for REC resources diff --git a/content/operate/kubernetes/8.2.0/re-clusters/cluster-recovery.md b/content/operate/kubernetes/8.2.0/re-clusters/cluster-recovery.md new file mode 100644 index 0000000000..a6fc0b54ff --- /dev/null +++ b/content/operate/kubernetes/8.2.0/re-clusters/cluster-recovery.md @@ -0,0 +1,53 @@ +--- +Title: Recover a Redis Enterprise cluster on Kubernetes +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: This task describes how to recover a Redis Enterprise cluster on Kubernetes. +linkTitle: Recover a Redis cluster +weight: 20 +url: '/operate/kubernetes/8.2.0/re-clusters/cluster-recovery/' +--- +When a Redis Enterprise cluster loses contact with more than half of its nodes either because of failed nodes or network split, +the cluster stops responding to client connections. +When this happens, you must recover the cluster to restore the connections. + +You can also perform cluster recovery to reset cluster nodes, to troubleshoot issues, or in a case of active/passive failover. + +The Redis Enterprise for Kubernetes automates these recovery steps: + +1. Recreates a fresh Redis Enterprise cluster +1. Mounts the persistent storage with the recovery files from the original cluster to the nodes of the new cluster +1. Recovers the cluster configuration on the first node in the new cluster +1. Joins the remaining nodes to the new cluster. + +> [!WARNING] +> Redis Enterprise for Kubernetes 7.2.4-2 introduces a new limitation. You cannot recover or upgrade your cluster if there are databases with old module versions or manually uploaded modules. See the [Redis Enterprise Software 7.2.4 known limitations](/content/operate/rs/release-notes/rs-7-2-4-releases/rs-7-2-4-52.md#cluster-recovery-with-manually-uploaded-modules) for more details. + +> [!NOTE] +> If your cluster uses user-defined modules, the recovery process doesn't block on module validation errors (such as URL or credential issues). The cluster can recover successfully, and you can resolve any module configuration issues after recovery is complete. See [User-defined modules](/content/operate/kubernetes/8.2.0/re-databases/modules.md#user-defined-modules) for more information. + +## Prerequisites + +- For cluster recovery, the cluster must be [deployed with persistence](/content/operate/kubernetes/8.2.0/recommendations/persistent-volumes.md). + +## Recover a cluster + +1. Edit the REC resource to set the `clusterRecovery` flag to `true`. + + ```sh + kubectl patch rec -n --type merge --patch '{"spec":{"clusterRecovery":true}}' + ``` + + +1. Wait for the cluster to recover until it is in the "Running" state. + + To see the state of the cluster, run: + + ```sh + watch "kubectl describe rec | grep State" + ``` + +1. To recover the database, see [Recover a failed database](/content/operate/rs/databases/recover.md). \ No newline at end of file diff --git a/content/operate/kubernetes/8.2.0/re-clusters/connect-prometheus-operator.md b/content/operate/kubernetes/8.2.0/re-clusters/connect-prometheus-operator.md new file mode 100644 index 0000000000..7cb5595454 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/re-clusters/connect-prometheus-operator.md @@ -0,0 +1,75 @@ +--- +Title: Connect the Prometheus operator to Redis Enterprise for Kubernetes +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: This article describes how to configure a Prometheus operator custom + resource to allow it to export metrics from Redis Enterprise for Kubernetes. +linkTitle: Export metrics to Prometheus +weight: 92 +url: '/operate/kubernetes/8.2.0/re-clusters/connect-prometheus-operator/' +--- + +To collect metrics data from your databases and Redis Enterprise cluster (REC), you can connect your [Prometheus](https://prometheus.io/) server to an endpoint exposed on your REC. Redis Enterprise for Kubernetes creates a dedicated service to expose the `prometheus` port (8070) for data collection. A custom resource called `ServiceMonitor` allows the [Prometheus operator](https://github.com/prometheus-operator/prometheus-operator/tree/main/Documentation) to connect to this port and collect data from Redis Enterprise. + +To expose database tags as metric labels so you can filter and group metrics by attributes such as environment or team, see [Enrich database metrics with tags](/content/operate/kubernetes/8.2.0/re-databases/enrich-metrics-with-tags.md). + +## Prerequisites + +Before connecting Redis Enterprise to Prometheus on your Kubernetes cluster, make sure you've done the following: + +- [Deploy Redis Enterprise for Kubernetes](/content/operate/kubernetes/8.2.0/deployment/quick-start.md) (version 6.2.10-4 or newer) +- [Deploy the Prometheus operator](https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/getting-started/introduction.md) (version 0.19.0 or newer) +- [Create a Redis Enterprise cluster](/content/operate/kubernetes/8.2.0/deployment/quick-start.md#create-a-redis-enterprise-cluster-rec) + +## Create a `ServiceMonitor` custom resource + +Below is an example `ServiceMonitor` custom resource file. By specifying the service label (`app: redis.io/service=prom-metrics`) in the `selector.matchLabels` section, you can point the Prometheus operator to the correct Redis Enterprise service (`-prom`). + +You'll need to configure the following fields to connect Prometheus to Redis Enterprise: + +| Section | Field | Value | +|---|---|---| +| `spec.endpoints` | `port` | Name of exposed port (`prometheus`) | +| `spec.namespaceSelector` | `matchNames` | Namespace for your REC | +| `spec.selector` | `matchLabels` | REC service label (`app: redis.io/service=prom-metrics`) | + +Apply the file in the same namespace as your Redis Enterprise cluster (REC). + +> [!NOTE] +> If Redis Enterprise and Prometheus are deployed in different namespaces, you'll also need to add the [`serviceMonitorNamespaceSelector`](https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/api-reference/api.md) field to your Prometheus resource. See the [Prometheus operator documentation](https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/user-guides/running-exporters.md) for more details on cross-namespace `ServiceMonitor` configuration. + + +```YAML +apiVersion: monitoring.coreos.com/v1 +kind: ServiceMonitor +metadata: + name: redis-enterprise +spec: + endpoints: + - interval: 15s + port: prometheus + scheme: https + tlsConfig: + insecureSkipVerify: true + namespaceSelector: + matchNames: + - + selector: + matchLabels: + redis.io/service: prom-metrics +``` + +For more info about configuring the `ServiceMonitor` resource, see the [`ServiceMonitorSpec` API documentation](https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/api-reference/api.md#monitoring.coreos.com/v1.ServiceMonitorSpec). + +## More info + +- github.com/prometheus-operator + - [Getting started](https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/getting-started/introduction.md) + - [Running exporters](https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/user-guides/running-exporters.md) + - [Troubleshooting ServiceMonitor changes](https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/platform/troubleshooting.md#troubleshooting-servicemonitor-changes) +- redis.io/docs + - [Metrics in Prometheus](/content/integrate/prometheus-with-redis-enterprise/prometheus-metrics-definitions.md) + - [Monitoring and metrics](/content/operate/rs/monitoring/_index.md) diff --git a/content/operate/kubernetes/8.2.0/re-clusters/connect-to-admin-console.md b/content/operate/kubernetes/8.2.0/re-clusters/connect-to-admin-console.md new file mode 100644 index 0000000000..0f50ac8cb8 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/re-clusters/connect-to-admin-console.md @@ -0,0 +1,57 @@ +--- +Title: Connect to the admin console +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Connect to the Redis Enterprise admin console to manage your Redis Enterprise + cluster. +linkTitle: Connect to the admin console +weight: 10 +url: '/operate/kubernetes/8.2.0/re-clusters/connect-to-admin-console/' +--- + +The username and password for the Redis Enterprise Software [admin console](/content/operate/rs/_index.md) are stored in a Kubernetes [secret](https://kubernetes.io/docs/concepts/configuration/secret/). After retrieving your credentials, you can use port forwarding to connect to the admin console. + +> [!NOTE] +> There are several methods for accessing the admin console. Port forwarding is the simplest, but not the most efficient method for long-term use. You could also use a load balancer service or Ingress. + +1. Switch to the namespace with your Redis Enterprise cluster (REC). + + ```sh + kubectl config set-context --current --namespace= + ``` + +1. Find your cluster name from your list of secrets. + + ```sh + kubectl get secret + ``` + + In this example, the cluster name is `rec`. + +1. Extract and decode your credentials from the secret. + + ```sh + kubectl get secret -o jsonpath='{.data.username}' | base64 --decode + kubectl get secret -o jsonpath='{.data.password}' | base64 --decode + ``` + +1. Find the port for the REC UI service in the `spec:ports` section of the service definition file. + + ```sh + kubectl get service/-ui -o yaml + ``` + + > [!NOTE] + > The default port is 8443. + +1. Use `kubectl port-forward` to forward your local port to the service port. + + ```sh + kubectl port-forward service/-ui : + ``` + +1. View the admin console from a web browser on your local machine at `https://localhost:8443`. + diff --git a/content/operate/kubernetes/8.2.0/re-clusters/delete-custom-resources.md b/content/operate/kubernetes/8.2.0/re-clusters/delete-custom-resources.md new file mode 100644 index 0000000000..63f2e5bb61 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/re-clusters/delete-custom-resources.md @@ -0,0 +1,136 @@ +--- +Title: Delete custom resources +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: This article explains how to delete Redis Enterprise clusters and Redis + Enterprise databases from your Kubernetes environment. +linkTitle: Delete custom resources +weight: 70 +aliases: [/operate/kubernetes/delete-custom-resources/] +url: '/operate/kubernetes/8.2.0/re-clusters/delete-custom-resources/' +--- + +## Multi-namespace management + +Before you can delete a namespace, you must remove the Redis custom resources ([REDB](#delete-a-database-redb), [REC](#delete-a-redis-enterprise-cluster-rec), and [operator](#delete-operator-from-one-namespace)) residing within a namespace before you can delete that namespace. Deleting namespace containing an REDB may cause errors or data loss. + +## Delete a database (REDB) + +To delete a database managed by the Redis Enterprise Kubernetes operator, run `kubectl delete redb ` from your K8s cluster. + +When you delete a database, your data and the REDB custom resource are also deleted. + +## Delete a Redis Enterprise cluster (REC) + +To delete a Redis Enterprise cluster managed by the operator: + +1. Delete all the databases in your cluster. + +2. Run `kubectl delete rec ` from your K8s cluster. + +When you delete your cluster, your databases and the REC custom resource are also deleted. However, persistent volume claims (PVCs) for your cluster are not deleted in the process. If you want to delete your PVCs, you'll have to delete them manually. + +## Delete the operator + +There are two ways to delete the operator. You can delete the entire operator bundle, but this will also delete any cluster-wide resources created by the operator. To only delete resources within a specific namespace and leave other cluster-wide resources intact, you need to delete each resource manually. + +Whether you decide to delete the entire bundle or just resources from a specific namespace, it's important to delete the operator ConfigMap (`operator-environment-config`). + +```sh +kubectl delete cm operator-environment-config +``` + +### Delete operator bundle + +To delete the operator from your K8s cluster, you can delete the operator bundle with: + +- `kubectl delete -f bundle.yaml` for non-OpenShift K8s deployments +- `kubectl delete -f openshift.bundle.yaml` for OpenShift deployments + +This will remove the operator and its custom resource definitions (CRDs) from your K8s cluster. + +> [!WARNING] +> The Redis Enterprise CRDs are non-namespaced resources, meaning they are shared across your entire K8s cluster. Deleting CRDs in one namespace will delete custom resources in every other namespace across the K8s cluster. + +### Delete operator from one namespace + +If you have Redis Enterprise clusters running in different namespaces on the same K8s cluster, deleting the entire operator bundle might cause data loss. + +To safely delete the operator from one namespace without affecting the others, delete the operator files individually, excluding the CRD files: + +```sh +kubectl delete -f role.yaml +kubectl delete -f role_binding.yaml +kubectl delete -f service_account.yaml +kubectl delete -f admission-service.yaml +kubectl delete -f operator.yaml +``` + +You will also need to remove [the `namespaceSelector` section from the validating webhook](/content/operate/kubernetes/8.2.0/deployment/quick-start.md#webhook). + +## Delete an Active-Active database (REAADB) + +1. On one of the existing participating clusters, delete the REEADB (substituting `` with your database name). + + ```sh + kubectl delete reaadb + ``` + +2. Verify the REAADB no longer exists. + + ```sh + kubectl get reaadb -o=jsonpath='{range .items[*]}{.metadata.name}' + ``` + +## Delete a remote cluster (RERC) + +1. Verify the RERC you want to delete isn't listed as a participating cluster in any REAADB resources. + + If an RERC is still listed as a participating cluster in any database, the deletion will be blocked. + +2. On one of the existing participating clusters, delete the RERC (substituting `` with your database name). + + ```sh + kubectl delete rerc + ``` + +3. Verify the RERC no longer exists. + + ```sh + kubectl get rerc -o=jsonpath='{range .items[*]}{.metadata.name}' + ``` + +## Troubleshoot REDB deletion + +The operator attaches a finalizer to the Redis Enterprise database (REDB) object. This makes sure the database is deleted before the REDB custom resource is removed from the K8s cluster. + +If the operator isn't running, or some other fatal error occurs, the finalizer isn't removed automatically by the operator. In this case, you won't be able to delete your REDB resource. + +If this happens, you can remove the finalizer manually. + +> [!WARNING] +> If you remove the finalizer manually, there is no guarantee that the underlying REDB has been deleted. This may cause resource issues and require manual intervention. + +```sh +kubectl patch redb --type=json -p \ + '[{"op":"remove","path":"/metadata/finalizers","value":"finalizer.redisenterprisedatabases.app.redislabs.com"}]' +``` + +## Troubleshoot REC deletion + +The operator attaches a finalizer to the Redis Enterprise cluster (REC) object. This makes sure the Redis cluster is deleted before the REC custom resource is removed from the K8s cluster. + +If the operator isn't running, or some other fatal error occurs, the finalizer isn't removed automatically by the operator. In this case, you won't be able to delete your REC resource. + +If this happens, you can remove the finalizer manually. + +> [!WARNING] +> If you remove the finalizer manually, there is no guarantee that the underlying REC has been deleted. This may cause resource issues and require manual intervention. + +```sh +kubectl patch rec --type=json -p \ + '[{"op":"remove","path":"/metadata/finalizers","value":"redbfinalizer.redisenterpriseclusters.app.redislabs.com"}]' +``` diff --git a/content/operate/kubernetes/8.2.0/re-clusters/expand-pvc.md b/content/operate/kubernetes/8.2.0/re-clusters/expand-pvc.md new file mode 100644 index 0000000000..ee14a40f0c --- /dev/null +++ b/content/operate/kubernetes/8.2.0/re-clusters/expand-pvc.md @@ -0,0 +1,124 @@ +--- +Title: Expand PersistentVolumeClaim (PVC) +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Expand your persistent volume claim by editing the REC. +linkTitle: Expand PVC +weight: 82 +url: '/operate/kubernetes/8.2.0/re-clusters/expand-pvc/' +--- + +This article outlines steps to increase the size of the persistent volume claim for your Redis Enterprise cluster (REC). + +> [!NOTE] +> This feature is only supported in versions 7.4.2-12 and above. + +[PersistentVolumeClaims (PVC)](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#expanding-persistent-volumes-claims) are created by the Redis Enterprise operator and used by the RedisEnterpriseCluster (REC). PVCs are created with a specific size and [can be expanded](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#expanding-persistent-volumes-claims) with the following steps, if the underlying [storage class](https://kubernetes.io/docs/concepts/storage/storage-classes/) supports it. + +This process involves deleting and recreating the REC StatefulSet with a larger persistent volume size. The pods owned by the StatefulSet are not restarted or affected by the deletion and recreation process, except when they are left without an owner momentarily. + +> [!NOTE] +> Shrinking (reducing the size) of your PVC is not allowed. This process only allows you to expand (size up) your PVC. + +### Default PVC size + +By default, if you omit [`spec.persistentSpec.volumeSize`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#specpersistentspec), the operator allocates a persistent volume that is five times (5x) the Redis Enterprise node memory request defined in [`spec.redisEnterpriseNodeResources.requests.memory`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#specredisenterprisenoderesources) (per node). This 5x ratio is the recommended minimum capacity. + +- If you set [`spec.persistentSpec.volumeSize`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#specpersistentspec) explicitly, that exact size is used and the 5x default does not apply. +- Changing node memory requests does not automatically resize existing PVCs. Use the procedure below to expand the PVC if you want to maintain the 5x ratio after changing memory. +- If you omit [`spec.redisEnterpriseNodeResources.requests.memory`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#specredisenterprisenoderesources), the operator uses its default memory request (4Gi). With `volumeSize` omitted, this results in a default PVC size of approximately 20Gi per node (5 × 4Gi). See [sizing on Kubernetes](/content/operate/kubernetes/8.2.0/recommendations/sizing-on-kubernetes.md) for defaults. + +- See the [volume size recommendations](/content/operate/kubernetes/8.2.0/recommendations/persistent-volumes.md#volume-size) and [hardware requirements](/content/operate/rs/installing-upgrading/install/plan-deployment/hardware-requirements.md) for guidance. + +## Prerequisites + +> [!WARNING] +> Do not change any other REC fields related to the StatefulSet while resizing is in progress. + +- PVC expansion must be supported and enabled by the StorageClass and underlying storage driver of the REC PVCs. + - The relevant StorageClass is the one associated with the REC PVCs. The StorageClass for existing PVCs cannot be changed. +- The StorageClass must be configured with `allowVolumeExpansion: true`. +- Your storage driver must support online expansion. +- We highly recommend you backup your databases before beginning this PVC expansion process. + +> [!WARNING] +> OpenShift users should be aware that (`ClusterResourceQuota`) can limit the PVC expansion. Check your quota before resizing using `oc describe clusterresourcequota `. + +> [!WARNING] +> PVC expansion is not supported when using Redis Flex (previously Redis on Flash). Do not enable `enablePersistentVolumeResize` if your REC uses `redisOnFlashSpec` as this will result in conflicts. + +## Expand REC PVC + +1. Enable the REC persistent volume resize flag. + + ```YAML + spec: + persistentSpec: + enablePersistentVolumeResize: true + ``` + +1. Set the value of `volumeSize` to your desired size. + + ```YAML + spec: + persistentSpec: + enablePersistentVolumeResize: true + volumeSize: Gi + ``` + +1. Apply the changes to the REC, replacing `` with the name of your REC. + + ```sh + kubectl apply -f + ``` + +After applying the REC changes, the PVCs will begin to expand to the new size. + +Once all the PVCs finish the resizing process, the operator will delete and recreate the StatefulSet with the new volume size. + +### Track progress + +You can track the progress by monitoring the status of the REC and PersistentVolumeClaim objects. + +The REC status will correspond to the status of one or more PVCs, and will reflect if the resizing is successful or failed. + +While the resizing is in progress, the status will be: + +```yaml +status: + persistenceStatus: + status: Resizing + succeeded: 2/3 +``` + +When the resizing is complete, the status becomes Provisioned and the new volume size is available for use by the REC pods. + +```yaml +status: + persistenceStatus: + status: Provisioned + succeeded: 3/3 +``` + +### Troubleshooting + +If an error occurs during this process: + +- Examine the status and events of the REC and PVC objects. + + ```sh + kubectl describe pvc + ``` + + ```sh + kubectl get events + ``` + +- Examine the logs of the operator pods. + + ```sh + kubectl logs + ``` diff --git a/content/operate/kubernetes/8.2.0/re-clusters/multi-namespace.md b/content/operate/kubernetes/8.2.0/re-clusters/multi-namespace.md new file mode 100644 index 0000000000..24a9c1d38f --- /dev/null +++ b/content/operate/kubernetes/8.2.0/re-clusters/multi-namespace.md @@ -0,0 +1,156 @@ +--- +Title: Manage databases in multiple namespaces +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Redis Enterprise for Kubernetes allows you to deploy to multiple namespaces + within your Kubernetes cluster. This article shows you how to configure your Redis + Enterprise cluster to connect to databases in multiple namespaces +linktitle: Manage multiple namespaces +weight: 17 +url: '/operate/kubernetes/8.2.0/re-clusters/multi-namespace/' +--- + +Multiple Redis Enterprise database resources (REDBs) can be associated with a single Redis Enterprise cluster resource (REC) even if they reside in different namespaces. + +To learn more about designing a multi-namespace Redis Enterprise cluster, see [flexible deployment options](/content/operate/kubernetes/8.2.0/architecture/deployment-options.md). + +> [!NOTE] +> Multi-namespace installations now support Active-Active databases (REAADB) with certain configuration requirements. For details, see [Multi-namespace Active-Active databases](#multi-namespace-active-active-databases). + +## Prerequisites + +Before configuring a multi-namespace deployment, you must have a running [Redis Enterprise cluster (REC)](/content/operate/kubernetes/8.2.0/deployment/quick-start.md). See more information in the [deployment](/content/operate/kubernetes/8.2.0/deployment/_index.md) section. + +## Create role and role binding for managed namespaces + +Both the operator and the RedisEnterpriseCluster (REC) resource need access to each namespace the REC will manage. For each **managed** namespace, create a `consumer_role.yaml` and `consumer_role_binding.yaml` file within the managed namespace, as shown in the examples below. + +> [!NOTE] +> These will need to be reapplied each time you [upgrade](/content/operate/kubernetes/8.2.0/upgrade/upgrade-redis-cluster.md). + +Replace `` with the namespace the REC resides in. +Replace `` with your own value (defaults to the REC name). + +`consumer_role.yaml` example: + +{{}} + +`consumer_role_binding.yaml` example: + +{{}} + +> [!NOTE] +> **Alternative approach**: Instead of creating individual `Role` objects for each namespace, you can create a single `ClusterRole` and bind it with multiple `RoleBinding` objects. This reduces the number of objects and simplifies role management. +> +> To use this approach: +> 1. Change `kind: Role` to `kind: ClusterRole` in the role definition above +> 2. Change `roleRef.kind: Role` to `roleRef.kind: ClusterRole` in the role binding definition above +> 3. Apply the ClusterRole once globally, then apply a RoleBinding in each managed namespace + +Apply the files, replacing `` with your own values: + +```sh +kubectl apply -f consumer_role.yaml -n +kubectl apply -f consumer_role_binding.yaml -n +``` + +> [!NOTE] +> If the REC is configured to watch a namespace without setting the role and role binding permissions, or a namespace that is not yet created, the operator will fail and halt normal operations. + +## Update Redis Enterprise operator ConfigMap + +There are two methods of updating the operator ConfigMap (`operator-environment-config`) to specify which namespaces to manage. + +- Method 1: Configure the operator to watch for a namespace label and add this label to managed namespaces (available in versions 6.4.2-4 or later). +- Method 2: Configure the operator with an explicit list of namespaces to manage. + +You can create this ConfigMap manually before deployment, or it will be created automatically after the operator was deployed. + +> [!WARNING] +> Only configure the operator to watch a namespace after the namespace is created and configured with the role/role_binding as explained above. If configured to watch a namespace without setting those permissions or a namespace that is not created yet, the operator will fail and not perform normal operations. + +### Method 1: Namespace label (available in versions 6.4.2-4 or later) + +1. Create the `cluster_role_binding.yaml` and `cluster_role.yaml` files. Replace the `` with the namespace the Redis Enterprise cluster (REC) resides in. + + `operator_cluster_role.yaml` example: + +{{}} + + `operator_cluster_role_binding.yaml` example: + +{{}} + +2. Apply the files. + + ```sh + kubectl apply -f operator_cluster_role.yaml + kubectl apply -f operator_cluster_role_binding.yaml + ``` + +3. Patch the ConfigMap in the REC namespace (``) to identify the managed namespaces with your label (``). + + ```sh + kubectl patch ConfigMap/operator-environment-config \ + -n \ + --type merge \ + -p '{"data": {"REDB_NAMESPACES_LABEL": ""}}' + ``` + +4. For each managed namespace, apply the same label. Replace `` with the namespace the REC will manage. Replace `` with the value used in the previous step. If you specify a value for ``, both the label name and value in managed namespaces must match to be detected by the operator. If the `` is empty, only the label name needs to match on managed namespaces and the value is disregarded. + + + ```sh + kubectl label namespace = + ``` + +> [!NOTE] +> The operator restarts when it detects a namespace label was added or removed. + +### Method 2: Explicit namespace list + +Patch the `operator-environment-config` in the REC namespace with a new environment variable (`REDB_NAMESPACES`). + +```sh +kubectl patch ConfigMap/operator-environment-config \ +-n \ +--type merge \ +-p '{"data":{"REDB_NAMESPACES": " [!NOTE] +> Apply the REAADB object to only one Kubernetes cluster. Based on the specified participating clusters and namespaces, the operator automatically creates the necessary resources in the other clusters. + +For example: + +```yaml +apiVersion: app.redislabs.com/v1alpha1 +kind: RedisEnterpriseActiveActiveDatabase +metadata: + name: consumer-reaadb + namespace: consumer-namespace-main +spec: + participatingClusters: + - name: participating-cluster-main + - name: participating-cluster-peer + namespace: consumer-namespace-peer + globalConfigurations: + +``` + +> [!WARNING] +> Configure the operator to watch a namespace only after the namespace exists and the required `Role` and `RoleBinding` resources have been applied. If the operator is configured to watch a namespace that lacks these permissions or does not exist, it will fail and halt normal operations. diff --git a/content/operate/kubernetes/8.2.0/re-databases/_index.md b/content/operate/kubernetes/8.2.0/re-databases/_index.md new file mode 100644 index 0000000000..d1055890f9 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/re-databases/_index.md @@ -0,0 +1,58 @@ +--- +Title: Redis Enterprise databases (REDB) +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Create and manage Redis Enterprise databases (REDB) on Kubernetes using the Redis Enterprise operator. +hideListLinks: true +linkTitle: Redis Enterprise databases (REDB) +weight: 31 +url: '/operate/kubernetes/8.2.0/re-databases/' +--- + +A Redis Enterprise database (REDB) is a custom Kubernetes resource that represents a Redis database running on a Redis Enterprise cluster. The Redis Enterprise operator manages REDB resources and handles database creation, configuration, scaling, and lifecycle operations. + +REDB resources define database specifications including memory limits, persistence settings, security configurations, networking options, and Redis modules. You can deploy databases on existing Redis Enterprise clusters (REC) and manage them by using standard Kubernetes tools and workflows. + +## Database management + +Create and manage Redis Enterprise databases on your cluster: + +- [Database controller](/content/operate/kubernetes/8.2.0/re-databases/db-controller.md) - Understand how the database controller manages REDB resources and database lifecycle + +## Replication and high availability + +Set up database replication for high availability and disaster recovery: + +- [Create replica databases](/content/operate/kubernetes/8.2.0/re-databases/replica-redb.md) - Configure replica databases for read scaling and disaster recovery scenarios + +## Advanced database configurations + +Explore advanced database features and configurations: + +- [Active-Active databases](/content/operate/kubernetes/8.2.0/active-active/_index.md) - Set up globally distributed Active-Active databases across multiple Kubernetes clusters + +## Database connectivity + +Connect applications to your Redis Enterprise databases: + +- [Database connectivity](/content/operate/kubernetes/8.2.0/networking/database-connectivity.md) - Comprehensive guide to in-cluster and external database access, service discovery, and credentials management. +- [Manage REDB passwords](/content/operate/kubernetes/8.2.0/security/authentication/manage-redb-credentials.md) - Retrieve and rotate Redis Enterprise database (REDB) passwords. +- [Networking](/content/operate/kubernetes/8.2.0/networking/_index.md) - Configure ingress, routes, and service exposure for database access +- [Security](/content/operate/kubernetes/8.2.0/security/_index.md) - Set up TLS, authentication, and access control for secure database connections + +## Monitoring and troubleshooting + +Monitor database performance and troubleshoot issues: + +- [Logs](/content/operate/kubernetes/8.2.0/logs/_index.md) - Collect and analyze database logs for troubleshooting +- [Connect to Prometheus operator](/content/operate/kubernetes/8.2.0/re-clusters/connect-prometheus-operator.md) - Monitor database metrics with Prometheus + +## Related topics + +- [Redis Enterprise clusters (REC)](/content/operate/kubernetes/8.2.0/re-clusters/_index.md) - Manage the underlying cluster infrastructure +- [REDB API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md) - Complete API specification for REDB resources +- [Active-Active database API](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_active_active_database_api.md) - API reference for Active-Active databases +- [Remote cluster API](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_remote_cluster_api.md) - API reference for remote cluster configurations diff --git a/content/operate/kubernetes/8.2.0/re-databases/db-controller.md b/content/operate/kubernetes/8.2.0/re-databases/db-controller.md new file mode 100644 index 0000000000..d98ca81fd0 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/re-databases/db-controller.md @@ -0,0 +1,126 @@ +--- +Title: Manage Redis Enterprise databases for Kubernetes +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: This section describes how the database controller provides the ability + to create, manage, and use databases via a database custom resource. +linkTitle: Manage databases +weight: 5 +url: '/operate/kubernetes/8.2.0/re-databases/db-controller/' +--- +## Redis Enterprise database (REDB) lifecycle + +A Redis Enterprise database (REDB) is created with a custom resource file. The custom resource defines the size, name, and other specifications for the REDB. The database is created when you apply the custom resource file. + +The database controller in Redis Enterprise for Kubernetes: + +- Discovers the custom resource +- Makes sure the REDB is created in the same namespace as the Redis Enterprise cluster (REC) +- Maintains consistency between the custom resource and the REDB + +The database controller recognizes the new custom resource and validates the specification. +If valid, the controller combines the values specified in +the custom resource with default values to create a full specification. It then uses this full specification to create the +database on the specified Redis Enterprise cluster (REC). + +> [!NOTE] +> The admission controller prevents duplicate database names when databases are created via the Kubernetes operator. + +Once the database is created, it is exposed with the same service mechanisms by the service rigger for the Redis Enterprise cluster. +If the database [custom resource is deleted](/content/operate/kubernetes/8.2.0/re-clusters/delete-custom-resources.md), the database and its services are deleted from the cluster. + +### Flexible deployment options + +Databases in multiple namespaces can be managed by the same operator. To learn more, see [Manage databases in multiple namespaces](/content/operate/kubernetes/8.2.0/re-clusters/multi-namespace.md). + +To learn more about designing a multi-namespace Redis Enterprise cluster, see [flexible deployment options](/content/operate/kubernetes/8.2.0/architecture/deployment-options.md). + +## Create a database + +Your Redis Enterprise database custom resource must be of the `kind: RedisEnterpriseDatabase` and have values for `name` and `memorySize`. All other values are optional and will be defaults if not specified. + +1. Create a file (in this example mydb.yaml) that contains your database custom resource. + + ```YAML + apiVersion: app.redislabs.com/v1alpha1 + kind: RedisEnterpriseDatabase + metadata: + name: mydb + spec: + memorySize: 1GB + ``` + + To create a REDB in a different namespace from your REC, you need to specify the cluster with `redisEnterpriseCluster` in the `spec` section of your RedisEnterpriseDatabase custom resource. + + ```YAML + redisEnterpriseCluster: + name: rec + ``` + +1. Apply the file in the namespace you want your database to be in. + + ```sh + kubectl apply -f mydb.yaml + ``` + +1. Check the status of your database. + + ```sh + kubectl get redb mydb -o jsonpath="{.status.status}" + ``` + + When the status is `active`, the database is ready to use. + +## Modify a database + +The custom resource defines the properties of the database. +To change the database, you can edit your original specification and apply the change or use `kubectl edit`. + +To modify the database: + +1. Edit the definition: + + ```sh + kubectl edit redb mydb + ``` + +1. Change the specification (only properties in `spec` section) and save the changes. + For more details, see [Options for Redis Enterprise databases](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md). + +1. Monitor the status to see when the changes take effect: + + ```sh + kubectl get redb mydb -o jsonpath="{.status.status}" + ``` + + When the status is `active`, the database is ready for use. + +To add tags to a database and expose them as metric labels, see [Enrich database metrics with tags](/content/operate/kubernetes/8.2.0/re-databases/enrich-metrics-with-tags.md). + +## Delete a database + +The database exists as long as the custom resource exists. +If you delete the custom resource, the database controller deletes the database. +The database controller removes the database and its services from the cluster. + +To delete a database, run: + +```sh +kubectl delete redb mydb +``` + +## Connect to a database + +After the database controller creates a database, services for accessing the database are automatically created in the same namespace. Connection information is stored in a Kubernetes [secret](https://kubernetes.io/docs/concepts/configuration/secret/) maintained by the database controller. + +For comprehensive information about connecting to your database, including: + +- Service types and access patterns +- Retrieving connection information from secrets +- In-cluster and external access methods +- Connection examples and troubleshooting + +See [Database connectivity](/content/operate/kubernetes/8.2.0/networking/database-connectivity.md). diff --git a/content/operate/kubernetes/8.2.0/re-databases/enrich-metrics-with-tags.md b/content/operate/kubernetes/8.2.0/re-databases/enrich-metrics-with-tags.md new file mode 100644 index 0000000000..e43823e777 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/re-databases/enrich-metrics-with-tags.md @@ -0,0 +1,137 @@ +--- +Title: Enrich database metrics with tags +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Expose Redis Enterprise database tags as metric labels so you can filter + dashboards and alerts by attributes such as environment, team, or application. +linkTitle: Enrich metrics with tags +weight: 20 +url: '/operate/kubernetes/8.2.0/re-databases/enrich-metrics-with-tags/' +--- + +Database tags are key-value attributes that you attach to your databases. You can expose selected tag keys as metric labels. Your monitoring stack can then filter and group metrics by those attributes. For example, you can build dashboards scoped to a team or environment and route alerts with context already attached. + +Tag-derived metrics are controlled in two places: + +- **Cluster (REC):** An administrator uses [`spec.metricsConfig`](#configure-the-cluster) on the Redis Enterprise cluster (REC) to turn tag-derived metrics on or off and choose which tag keys are eligible. +- **Database (REDB / REAADB):** Application teams set tag values on individual databases with [`spec.tags`](#tag-a-database) (REDB) or [`spec.globalConfigurations.tags`](#tag-an-active-active-database) (REAADB). + +A tag becomes a metric label only when both of these are true: + +- The tag key is in the cluster's allowlist. +- The tag is set with a value on the database. + +Keys that aren't in the allowlist never appear as labels, even if they're set on a database. + +## How it works + +1. On the REC, set `metricsConfig.exposeDatabaseTags` to `true`. In `metricsConfig.metricsTagKeysExposed`, list the tag keys you want available for metrics. +2. On each database, set the tag values with `spec.tags` (REDB) or `spec.globalConfigurations.tags` (REAADB). +3. The operator syncs the cluster configuration and the database tags to Redis Enterprise. Only keys in the allowlist that are present on a database appear as metric labels. + +The REC defines the metrics configuration. The operator overwrites any changes made directly through the Redis Enterprise REST API. + +## Configure the cluster + +Add `metricsConfig` to the REC `spec` to turn tag-derived metrics on and choose which tag keys are eligible: + +```YAML +apiVersion: app.redislabs.com/v1alpha1 +kind: RedisEnterpriseCluster +metadata: + name: rec +spec: + nodes: 3 + metricsConfig: + exposeDatabaseTags: true + metricsTagKeysExposed: + - env + - team +``` + +In this example, only the `env` and `team` tag keys can appear as metric labels. A tag such as `app` is not exposed even if it is set on a database, because it is not in the allowlist. + +`metricsConfig` supports the following fields. All fields are optional. The operator applies the listed default when a field is unset. + +| Field | Type | Default | Description | +|---|---|---|---| +| `exposeDatabaseTags` | boolean | `false` | Enables or disables tag-derived metrics for the cluster. When `true`, the tag keys listed in `metricsTagKeysExposed` can be used as metric labels. When `false`, no database tag keys are exposed as labels, even if they are listed. | +| `metricsTagKeysExposed` | array of strings | `[]` | Allowlist of database tag keys eligible to be exposed as metric labels. Keys not listed here are never exposed, even if present on a database. | +| `maxRequestsInFlight` | integer | `2` | Maximum number of concurrent metrics scrape requests. Set to `0` to disable the limit. | +| `keyDistributionEnabled` | boolean | `false` | When `true`, export Prometheus histogram metrics for key size and key item distributions. | +| `keySizeBuckets` | string | `""` | Comma-separated bucket boundaries for key size distribution histograms, for example `"128M,512M"`. | +| `keyItemsBuckets` | string | `""` | Comma-separated bucket boundaries for key item distribution histograms, for example `"1M,8M"`. | +| `localStorageMaxSizeMb` | integer | `1024` | Maximum size, in MB, for local metrics storage. | +| `localStorageRetentionDays` | integer | `8` | Number of days to retain local metrics data. | + +Tag key allowlist entries (`metricsTagKeysExposed`) must follow these rules: + +- Keys are lowercased on the cluster. +- Each key must be 1–64 characters and cannot be whitespace only. +- Keys may contain ASCII letters, digits, spaces, and the characters `-`, `_`, `.`, `+`, `@`, and `:`. +- Duplicate keys are rejected. The operator rejects the resource if the same key appears more than once, ignoring case. +- A maximum of 50 keys is allowed. + +## Tag a database + +Set tags on a Redis Enterprise database (REDB) with `spec.tags`, a set of key-value pairs: + +```YAML +apiVersion: app.redislabs.com/v1alpha1 +kind: RedisEnterpriseDatabase +metadata: + name: orders-cache +spec: + memorySize: 1GB + tags: + env: prod + team: payments + app: orders-service +``` + +The operator merges these tags into the database's tags on the cluster. With the cluster configuration shown above, only `env` and `team` can appear as metric labels. The `app` tag is stored on the database but not exposed in metrics. + +`spec.tags` is optional. For details on how tags are merged, updated, and removed, see [Managed tags and deletion behavior](#managed-tags-and-deletion-behavior). + +## Tag an Active-Active database + +For an Active-Active database (REAADB), set tags under `spec.globalConfigurations.tags`, also a set of key-value pairs: + +```YAML +apiVersion: app.redislabs.com/v1alpha1 +kind: RedisEnterpriseActiveActiveDatabase +metadata: + name: orders-aa +spec: + globalConfigurations: + tags: + env: prod + team: payments +``` + +The operator applies these tags to the Active-Active database's default configuration and to every participating cluster's instance. The cluster-level allowlist still applies. Configure `metricsConfig` on each participating REC to add the tag keys you want exposed to the allowlist. + +## Managed tags and deletion behavior + +The operator tracks the tag keys it manages for each database. This lets it apply changes, including deletions, without disturbing tags set outside Kubernetes. + +- **External tags are preserved.** Tags that exist on the database but were never declared in `spec.tags` (or `spec.globalConfigurations.tags`) are left in place. +- **The custom resource wins on collision.** If a key exists both on the database and in the custom resource, the operator applies the value from the custom resource. +- **`status.managedTags` records what the operator owns.** After a successful sync, the operator records the tag keys it applied in `status.managedTags`. This record lets it tell a key you removed apart from a key it never managed. +- **Removing a key deletes it.** If you remove a key that was recorded in `status.managedTags`, the operator removes that tag from the database on the next reconciliation. Keys that were never in `status.managedTags`, such as tags set only through the Redis Enterprise REST API, aren't deleted. +- **Reserved keys are rejected.** Keys the operator reserves for its own internal use cannot be set in the spec. For REDB, these are `managed_by`, `redb_name`, `redb_namespace`, `db_service_port`, and `oss_cluster_access`. For REAADB, they also include `redb_resource`, `redb_resource_name`, `global_configurations_spec`, `replication_endpoint_port`, `rerc_name`, and the operator's internal secret-tracking keys (`redis.io/db-sec-name`, `redis.io/db-sec-latest`, `redis.io/certs-latest`, `redis.io/bu-sec-name`, and `redis.io/bu-sec-latest`). The admission controller rejects a custom resource that uses one. + +> [!NOTE] +> When you upgrade to a release with this feature, `status.managedTags` starts empty for databases that already existed. The operator does not drop existing tags. It reapplies only the keys currently in the spec, so the upgrade has no effect on databases that never used tags. + +## View tagged metrics in Prometheus + +For instructions on connecting Prometheus to Redis Enterprise for Kubernetes, see [Export metrics to Prometheus](/content/operate/kubernetes/8.2.0/re-clusters/connect-prometheus-operator.md). + +## Limitations + +- **Allowlist size.** The cluster allows at most 50 tag keys in `metricsTagKeysExposed`. The operator doesn't cap the list itself; the cluster enforces the limit. Keep the allowlist small: each exposed key adds a label dimension to the affected metrics, which increases cardinality in your monitoring stack. +- **When changes take effect.** Changes to `metricsConfig`, `spec.tags`, and `spec.globalConfigurations.tags` apply on the operator's next reconciliation of the resource. No pod or cluster restart is required. Exposed tag labels appear in metrics on the next scrape after the cluster applies the change. diff --git a/content/operate/kubernetes/8.2.0/re-databases/modules.md b/content/operate/kubernetes/8.2.0/re-databases/modules.md new file mode 100644 index 0000000000..d602ea8aa3 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/re-databases/modules.md @@ -0,0 +1,348 @@ +--- +Title: Configure modules +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Deploy Redis Enterprise databases with modules using the Redis Enterprise operator for Kubernetes. +linkTitle: Configure modules +weight: 15 +url: '/operate/kubernetes/8.2.0/re-databases/modules/' +--- + +Redis Enterprise modules extend Redis functionality with additional data types, commands, and capabilities. Redis Enterprise versions 8.0.2 and later include several bundled modules that are automatically enabled for compatible database types. You can also add user-defined modules for additional functionality. + +## Prerequisites + +Before you begin, verify the following: + +- [Redis Enterprise operator deployed](/content/operate/kubernetes/8.2.0/deployment/quick-start.md) in your Kubernetes cluster +- [Redis Enterprise cluster (REC)](/content/operate/kubernetes/8.2.0/re-clusters/_index.md) running and in a healthy state + +## Bundled modules + +Redis Enterprise includes several bundled modules that extend Redis functionality with additional data types, commands, and capabilities. Starting with Redis Enterprise version 8.0.2, these modules are automatically included and immediately available for use. + +### Available bundled modules + +| Module | Name | Description | Capabilities | +|--------|------|-------------|--------------| +| **[RediSearch](/content/develop/ai/search-and-query/_index.md)** | `search` | Full-text search and secondary indexing | Query, aggregation, full-text search, vector similarity search | +| **[RedisJSON](/content/develop/data-types/json/_index.md)** | `ReJSON` | JSON data type support | Store, update, and query JSON documents | +| **[RedisTimeSeries](/content/develop/data-types/timeseries/_index.md)** | `timeseries` | Time series data structures | Ingest and query time series data with downsampling and aggregation | +| **[RedisBloom](/content/develop/data-types/probabilistic/_index.md)** | `bf` | Probabilistic data structures | Bloom filters, Cuckoo filters, Count-Min Sketch, Top-K | + +> [!NOTE] +> When configuring databases with modules, use the `NAME` field (for example, `search` or `ReJSON`) instead of the `DISPLAY_NAME` field. + +### Automatic enablement in Redis 8 and later + +For databases created with or upgraded to Redis version 8 or later, bundled modules are automatically enabled based on the database type. You don't need to specify them in the `spec.moduleList` field unless you want to use a specific version. + +{{}} + +For databases using Redis versions earlier than 8, explicitly specify bundled modules in the `spec.moduleList` field when you create the database. + +### Check available module versions + +To see which bundled module versions are available in your cluster, run the following command: + +```bash +kubectl get rec -o jsonpath='{.status.modules}' | jq +``` + +This command displays all modules (both bundled and user-defined) installed in the cluster and their available versions. + +### Bundled vs. user-defined modules + +The following table shows the key differences between bundled and user-defined modules: + +| Aspect | Bundled modules | User-defined modules | +|--------|----------------|---------------------| +| **Installation** | Pre-installed with Redis Enterprise | Must be added to the REC spec | +| **Availability** | Immediately available | Available after you add them to the REC | +| **Versions** | Bundled with the Redis Enterprise version | Specified by URL in the REC spec | +| **Examples** | RediSearch, RedisJSON, RedisTimeSeries, RedisBloom | RedisGears, custom modules | +| **Redis 8 and later behavior** | Automatically enabled for compatible database types | Must be explicitly specified | + +## User-defined modules + +User-defined modules are custom Redis modules that extend Redis functionality beyond the bundled modules. User-defined modules can include third-party modules like RedisGears or custom in-house modules developed for specific use cases. + +**Limitations:** + +- **Active-Active databases**: User-defined modules are not supported with Active-Active databases. Only bundled modules (RediSearch, RedisJSON, RedisTimeSeries, RedisBloom) can be used with Active-Active databases. + +- **Redis on Flash**: User-defined modules are fully supported with Redis on Flash databases. + +### Add user-defined modules to the REC + +To use user-defined modules with your databases, first add them to the Redis Enterprise cluster (REC) custom resource. This enables the operator to validate the modules and make them available for database creation. + +> [!WARNING] +> Add user-defined modules to the REC **before** you create any databases that use them. The admission controller validates that modules exist in the REC before allowing REDB creation. + +1. Edit your REC custom resource: + + ```sh + kubectl edit rec + ``` + +2. Add the `userDefinedModules` section to the `spec`: + + ```yaml + spec: + userDefinedModules: + - name: "custom-module" + source: + https: + url: "https://modules.company.com/custom-module-v1.0.zip" + credentialsSecret: "module-repo-creds" + ``` + +3. If your module repository requires authentication, create a secret with your credentials: + + ```sh + kubectl create secret generic module-repo-creds \ + --from-literal=username= \ + --from-literal=password= + ``` + +If you use [HashiCorp Vault integration](/content/operate/kubernetes/8.2.0/security/vault.md), store the module credentials in Vault instead of creating a Kubernetes secret. The `credentialsSecret` field will reference the secret name in Vault. + +### Module naming requirements + +The `name` field in the REC spec must match either the `module_name` or `display_name` value from the module's manifest file (`module.json` inside the module zip file). This enables the operator to validate the module. + +For example, if your module manifest contains the following: + +```json +{ + "module_name": "rg", + "display_name": "RedisGears", + "semantic_version": "1.2.5" +} +``` + +You can use either `"rg"` or `"RedisGears"` as the `name` value in your REC spec. + +> [!NOTE] +> If the names don't match, the operator can't validate the module. This can lead to preventable errors during database creation or upgrades. + +### Edit user-defined modules + +To modify the user-defined modules list, complete the following steps: + +1. Edit the REC custom resource: + + ```sh + kubectl edit rec + ``` + +1. Update the `userDefinedModules` section: + - **Add new modules**: Append them to the list. + - **Update module URLs**: Change the `url` field for existing modules. + - **Update credentials**: Change the `credentialsSecret` reference. + +1. Save your changes. The operator validates and applies the updates. + +> [!WARNING] +> Don't remove modules that are currently in use by any database. The operator rejects the change and puts the REC into an error state. + +> [!NOTE] +> Changes to the `userDefinedModules` list trigger a rolling restart of the Redis Enterprise cluster pods. Plan module updates during a maintenance window to minimize potential impact on your databases. + +### Verify user-defined modules + +After you add user-defined modules to the REC, verify that they're available: + +```sh +kubectl get rec -o jsonpath='{.spec.userDefinedModules}' | jq +``` + +You can also check the REC status for validation errors: + +```sh +kubectl describe rec +``` + +Look for events or status messages related to module validation in the output. + +## Upgrade with modules + +The upgrade process differs depending on whether you use bundled modules or user-defined modules. + +### Module version selection + +When multiple versions of a module are available in the cluster, Redis Enterprise selects the appropriate version based on the `compatible_redis_version` field in the module's manifest file (`module.json`). This field must match the Redis OSS version that the database is using. + +For example, if your database uses Redis 7.2, Redis Enterprise selects the module version whose `compatible_redis_version` is `7.2`. If no matching version is found, the module cannot be loaded. + +### Upgrade with bundled modules + +For databases using bundled modules (RediSearch, RedisJSON, RedisTimeSeries, RedisBloom): + +- **Redis 8 and later**: Bundled modules are automatically enabled and upgraded when you upgrade the database to Redis version 8 or later. You don't need to take any additional action. The module version is automatically selected based on the database's Redis version. + +- **Redis versions earlier than 8**: Bundled modules are upgraded automatically when you upgrade the Redis Enterprise cluster. The bundled module versions are tied to the Redis Enterprise version, and the appropriate version is selected based on the database's Redis version. + +### Upgrade with user-defined modules + +For databases using user-defined modules, you must take additional steps during cluster upgrades: + +1. Set `spec.upgradeSpec.autoUpgradeRedisEnterprise` to `false` in your REC before upgrading. + +1. Add or update the `userDefinedModules` list in the REC spec with the new module versions before or during the cluster upgrade. Ensure that the new module versions include a `compatible_redis_version` field that matches the Redis version your databases will use after the upgrade. + +1. After the cluster upgrade completes, you can re-enable `autoUpgradeRedisEnterprise` if desired. + +For detailed upgrade instructions, see the following: + +- [Upgrade a Redis Enterprise cluster (REC)](/content/operate/kubernetes/8.2.0/upgrade/upgrade-redis-cluster.md) +- [Upgrade Redis Enterprise on OpenShift](/content/operate/kubernetes/8.2.0/upgrade/openshift-cli.md) + +## Troubleshooting + +This section covers common issues you might encounter when working with user-defined modules. + +### Module validation errors + +Module validation errors occur when the operator can't validate a user-defined module. Common causes include incorrect URLs, authentication failures, or invalid module manifests. + +**Symptoms:** + +- REC status shows validation errors +- Events indicate module download or validation failures +- Databases fail to create with module-related errors + +**Diagnosis:** + +Check the REC status for validation errors: + +```sh +kubectl describe rec +``` + +Look for error messages related to modules in the Events section. + +**Resolution:** + +1. **Verify the module URL is accessible:** + + ```sh + curl -I + ``` + +2. **Check credentials secret exists and has correct values:** + + ```sh + kubectl get secret -o yaml + ``` + +3. **Verify the module manifest (`module.json`) is valid:** + + Download the module zip file and check that it contains a valid `module.json` file with required fields: `module_name`, `display_name`, `semantic_version`, `commands`, and `compatible_redis_version`. + +4. **Ensure the `name` field in the REC spec matches the module manifest:** + + The `name` must match either `module_name` or `display_name` from the module's `module.json` file. See [Module naming requirements](#module-naming-requirements) for details. + +### Bootstrap failures + +Bootstrap failures occur when the Redis Enterprise cluster fails to start due to module-related issues. + +**Symptoms:** + +- REC pods fail to reach Running state +- Operator logs show module-related errors during bootstrap +- Cluster remains in a non-ready state + +**Diagnosis:** + +Check the operator logs: + +```sh +kubectl logs -l name=redis-enterprise-operator -n +``` + +Check the REC pod logs: + +```sh +kubectl logs -n +``` + +**Resolution:** + +1. **Remove problematic modules from the REC spec:** + + Edit the REC and remove or comment out the problematic module from the `userDefinedModules` list: + + ```sh + kubectl edit rec + ``` + +2. **Wait for the cluster to recover:** + + ```sh + kubectl get rec -w + ``` + +3. **Fix the module configuration and re-add it:** + + After the cluster is running, correct the module URL, credentials, or manifest issues, then add the module back to the REC spec. + +### Module not found errors + +Module not found errors occur when you try to create a database that uses a module that isn't defined in the REC. + +**Symptoms:** + +- REDB creation fails with admission webhook errors +- Error message indicates the module is not found in the REC +- Database remains in a pending or failed state + +**Diagnosis:** + +Check the REDB creation error: + +```sh +kubectl describe redb +``` + +Verify which modules are defined in the REC: + +```sh +kubectl get rec -o jsonpath='{.spec.userDefinedModules}' | jq +``` + +**Resolution:** + +1. **Add the missing module to the REC:** + + See [Add user-defined modules to the REC](#add-user-defined-modules-to-the-rec) for detailed instructions. + +2. **Wait for the module to be validated:** + + ```sh + kubectl describe rec + ``` + + Look for successful validation in the Events section. + +3. **Retry database creation:** + + After the module is available in the REC, the database creation should succeed automatically, or you can delete and recreate the REDB. + +## Related information + +- [Redis modules documentation](/content/develop/reference/modules/_index.md) - Official Redis modules documentation +- [REDB API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md) - Complete API specification for REDB resources +- [REAADB API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_active_active_database_api.md) - API reference for Active-Active databases + +### Redis Software documentation + +- [Add modules to a cluster](/content/operate/rs/installing-upgrading/modules/add-module-to-cluster.md) - Install module packages on Redis Enterprise Software clusters +- [Enable modules for a database](/content/operate/rs/installing-upgrading/modules/add-module-to-database.md) - Add modules to databases in Redis Enterprise Software +- [Upgrade modules](/content/operate/rs/installing-upgrading/modules/upgrade-module.md) - Upgrade module versions in Redis Enterprise Software +- [Module lifecycle](/content/operate/rs/installing-upgrading/modules-lifecycle.md) - Module versioning and end-of-life schedule diff --git a/content/operate/kubernetes/8.2.0/re-databases/replica-redb.md b/content/operate/kubernetes/8.2.0/re-databases/replica-redb.md new file mode 100644 index 0000000000..da8afb154a --- /dev/null +++ b/content/operate/kubernetes/8.2.0/re-databases/replica-redb.md @@ -0,0 +1,264 @@ +--- +Title: Create replica databases on Kubernetes +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: How to create and automate database replicas using the database controller +linkTitle: Create replica databases +weight: 42 +url: '/operate/kubernetes/8.2.0/re-databases/replica-redb/' +--- + +You can configure a replica of a database by creating an item in +the [`replicaSources`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md#specreplicasources) section of the RedisEnterpriseDatabase (REDB) custom resource. + +> [!NOTE] +> If the source cluster is reached through an ingress route (for example, an OpenShift route or any TLS pass-through that relies on SNI), set [`tlsSniName`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md#specreplicasources) on each `replicaSources` item to the SNI hostname configured on the ingress. Without it, the syncer cannot route TLS to the source database's proxy and replication fails. + +A secret must be created with the `stringData` section containing the replica source URI as follows: + +Create a secret with the replica source URI listed in the `stringData` field as follows: + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: my-replica-source +stringData: + uri: +``` + +The replica source URL can be retrieved by going to "UI > database > configuration > Press the button Get Replica of source URL" +in the administrative UI. But, this information can also be retrieved directly from +the REST API as well. + +A replica of database CR simply uses the secret in the `replicaSources` section: + +```yaml +apiVersion: app.redislabs.com/v1alpha1 +kind: RedisEnterpriseDatabase +metadata: + name: name-of-replica +spec: + redisEnterpriseCluster: + name: name-of-cluster + replicaSources: + - replicaSourceType: SECRET + replicaSourceName: my-replica-source + # Required when the source cluster is reached via an ingress route + # tlsSniName: source-db.example.com +``` + +In the above, `name-of-replica` database will be created as a replica of the +source database as long as the source database exists on the source cluster +and the secret contains the correct replica source URL for that database. + +## Retrieving the replica source URL via kubectl + +You will need `kubectl`, `curl`, and `jq` installed for this procedure. + +1. Set your metadata: + + ```js + CLUSTER_NAME=test + SOURCE_DB=db1 + TARGET_DB=db2 + TARGET_CLUSTER_NAME=test + ``` + +1. Retrieve the cluster authentication: + + ```js + CLUSTER_USER=`kubectl get secret/${CLUSTER_NAME} -o json | jq -r .data.username | base64 -d` + CLUSTER_PASSWORD=`kubectl get secret/${CLUSTER_NAME} -o json | jq -r .data.password | base64 -d` + ``` + +1. Forward the port of the REST API service for your source cluster: + + ```sh + kubectl port-forward pod/${CLUSTER_NAME}-0 9443 + ``` + +1. Request the information from the REST API: + + ```js + JQ='.[] | select(.name=="' + JQ+="${SOURCE_DB}" + JQ+='") | ("redis://admin:" + .authentication_admin_pass + "@"+.name+":"+(.endpoints[0].port|tostring))' + URI=`curl -sf -k -u "$CLUSTER_USER:$CLUSTER_PASSWORD" "https://localhost:9443/v1/bdbs?fields=uid,name,endpoints,authentication_admin_pass" | jq "$JQ" | sed 's/"//g'` + ``` + + Note: URI now contains the replica source URI. + +1. Construct the secret for the replica: + + ```yaml + cat << EOF > secret.yaml + apiVersion: v1 + kind: Secret + metadata: + name: ${SOURCE_DB}-url + stringData: + uri: ${URI} + EOF + kubectl apply -f secret.yaml + ``` + +1. Create the replica database: + + ```yaml + cat << EOF > target.yaml + apiVersion: app.redislabs.com/v1alpha1 + kind: RedisEnterpriseDatabase + metadata: + name: ${TARGET_DB} + spec: + redisEnterpriseCluster: + name: ${TARGET_CLUSTER_NAME} + replicaSources: + - replicaSourceType: SECRET + replicaSourceName: ${SOURCE_DB}-url + # Required when the source cluster is reached via an ingress route + # tlsSniName: source-db.example.com + EOF + kubectl apply -f target.yaml + ``` + +## Automating the creation via a job + +The following procedure uses a ConfigMap and a Job to construct the replica +source URL secret from the source database and configure the target database. + +There are four parameters: + +- `source` - the name of the source database +- `cluster` - the name of the cluster for the source database +- `target` - the name of the target database +- `targetCluster` - the name of the cluster for the target database + +These parameters can be set by: + +```sh +kubectl create configmap replica-of-database-parameters \ +--from-literal=source=name-of-source \ +--from-literal=cluster=name-of-cluster \ +--from-literal=target=name-of-target \ +--from-literal=targetCluster=name-of-cluster +``` + +where "name-of-..." is replaced with the database source, source cluster, +database target, and target cluster names. + +The Job and ConfigMap below, when submitted, will create the secret and +replica database: + +```yaml +apiVersion: batch/v1 +kind: Job +metadata: + name: replica-of-database +spec: + backoffLimit: 4 + template: + spec: + serviceAccountName: redis-enterprise-operator + restartPolicy: Never + volumes: + - name: scripts + configMap: + name: replica-of-database + containers: + - name: createdb + image: debian:stable-slim + env: + - name: MY_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + - name: SCRIPT + value: create.sh + - name: SOURCE_DB + valueFrom: + configMapKeyRef: + name: replica-of-database-parameters + key: source + - name: TARGET_DB + valueFrom: + configMapKeyRef: + name: replica-of-database-parameters + key: target + - name: CLUSTER_SERVICE + value: .svc.cluster.local + - name: CLUSTER_NAME + valueFrom: + configMapKeyRef: + name: replica-of-database-parameters + key: cluster + - name: CLUSTER_PORT + value: "9443" + - name: TARGET_CLUSTER_NAME + valueFrom: + configMapKeyRef: + name: replica-of-database-parameters + key: targetCluster + volumeMounts: + - mountPath: /opt/scripts/ + name: scripts + command: + - /bin/bash + - -c + - | + apt-get update; apt-get install -y curl jq apt-transport-https gnupg2 + apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 6A030B21BA07F4FB + curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - + echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | tee -a /etc/apt/sources.list.d/kubernetes.list + apt-get update + apt-get install -y kubectl + bash /opt/scripts/$SCRIPT +--- +apiVersion: v1 +kind: ConfigMap +metadata: + name: replica-of-database +data: + create.sh: | + CLUSTER_USER=`kubectl get secret/${CLUSTER_NAME} -o json | jq -r .data.username | base64 -d` + CLUSTER_PASSWORD=`kubectl get secret/${CLUSTER_NAME} -o json | jq -r .data.password | base64 -d` + CLUSTER_HOST=${CLUSTER_NAME}.${MY_NAMESPACE}${CLUSTER_SERVICE} + JQ='.[] | select(.name=="' + JQ+="${SOURCE_DB}" + JQ+='") | ("redis://admin:" + .authentication_admin_pass + "@"+.endpoints[0].dns_name+":"+(.endpoints[0].port|tostring))' + URI=`curl -sf -k -u "$CLUSTER_USER:$CLUSTER_PASSWORD" "https://${CLUSTER_HOST}:${CLUSTER_PORT}/v1/bdbs?fields=uid,name,endpoints,authentication_admin_pass" | jq "$JQ" | sed 's/"//g'` + echo "URL: ${URL}" + echo "" + cat << EOF > /tmp/secret.yaml + apiVersion: v1 + kind: Secret + metadata: + name: ${SOURCE_DB}-url + stringData: + uri: ${URI} + EOF + cat /tmp/secret.yaml + cat << EOF > /tmp/target.yaml + apiVersion: app.redislabs.com/v1alpha1 + kind: RedisEnterpriseDatabase + metadata: + name: ${TARGET_DB} + spec: + redisEnterpriseCluster: + name: ${TARGET_CLUSTER_NAME} + replicaSources: + - replicaSourceType: SECRET + replicaSourceName: ${SOURCE_DB}-url + # Required when the source cluster is reached via an ingress route + # tlsSniName: source-db.example.com + EOF + echo "---" + cat /tmp/target.yaml + echo "" + kubectl -n ${MY_NAMESPACE} apply -f /tmp/secret.yaml + kubectl -n ${MY_NAMESPACE} apply -f /tmp/target.yaml +``` diff --git a/content/operate/kubernetes/8.2.0/recommendations/_index.md b/content/operate/kubernetes/8.2.0/recommendations/_index.md new file mode 100644 index 0000000000..f0f12b9c4c --- /dev/null +++ b/content/operate/kubernetes/8.2.0/recommendations/_index.md @@ -0,0 +1,34 @@ +--- +Title: Configuration recommendations +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Best practices and configuration recommendations for Redis Enterprise on Kubernetes deployments. +hideListLinks: false +linkTitle: Recommendations +weight: 80 +url: '/operate/kubernetes/8.2.0/recommendations/' +--- + +Follow these best practices and configuration recommendations to optimize your Redis Enterprise deployment on Kubernetes for performance, reliability, and scalability. + +## Infrastructure recommendations + +Configure your Kubernetes infrastructure for optimal Redis Enterprise performance: + +- [Node resources](/content/operate/kubernetes/8.2.0/recommendations/node-resources.md) - CPU, memory, and resource allocation recommendations +- [Node selection](/content/operate/kubernetes/8.2.0/recommendations/node-selection.md) - Best practices for selecting and configuring Kubernetes nodes +- [Persistent volumes](/content/operate/kubernetes/8.2.0/recommendations/persistent-volumes.md) - Storage configuration and persistent volume recommendations + +## Deployment recommendations + +Optimize your Redis Enterprise deployment configuration: + +- [Sizing on Kubernetes](/content/operate/kubernetes/8.2.0/recommendations/sizing-on-kubernetes.md) - Guidelines for sizing clusters and databases +- [Pod stability](/content/operate/kubernetes/8.2.0/recommendations/pod-stability.md) - Ensure stable pod operations and prevent disruptions + +## Performance optimization + +Configure your deployment for optimal performance and reliability based on your workload requirements and infrastructure capabilities. diff --git a/content/operate/kubernetes/8.2.0/recommendations/node-resources.md b/content/operate/kubernetes/8.2.0/recommendations/node-resources.md new file mode 100644 index 0000000000..3901bef8b1 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/recommendations/node-resources.md @@ -0,0 +1,69 @@ +--- +Title: Manage node resources +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Recommendations on managing node resources and setting eviction thresholds + with Redis Enterprise for Kubernetes. +linktitle: Manage resources +weight: 17 +url: '/operate/kubernetes/8.2.0/recommendations/node-resources/' +--- + + + +## Node eviction thresholds + + Kubernetes uses [node-pressure eviction](https://kubernetes.io/docs/concepts/scheduling-eviction/node-pressure-eviction/) to terminate pods and free up node resources. Redis Enterprise for Kubernetes works best when you use the recommendations below. + +Eviction thresholds are managed by [kubelet arguments](https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/). + +Redis recommends setting a high [soft eviction threshold](https://kubernetes.io/docs/concepts/scheduling-eviction/node-pressure-eviction/#soft-eviction-thresholds). This changes the node condition early enough to alert the administrator. + +We also recommend setting the [`eviction-soft-grace-period`](https://kubernetes.io/docs/concepts/scheduling-eviction/node-pressure-eviction/#soft-eviction-thresholds) high enough for the administrator to scale the cluster. + +For more information about configuring eviction thresholds on specific platforms, see [Freeing node resources](https://docs.openshift.com/container-platform/4.9/nodes/nodes/nodes-nodes-garbage-collection.html) for OpenShift or [Cluster architecture](https://cloud.google.com/kubernetes-engine/docs/concepts/cluster-architecture#node_allocatable) for GKE. + +### Monitor node conditions + +Redis recommends monitoring the node conditions. If both `MemoryPressure` and `DiskPressure` are true, the eviction threshold was met. + +```sh +> kubectl get nodes -o jsonpath='{range .items[*]}name:{.metadata.name}{"\t"}MemoryPressure:{.status.conditions[?(@.type == "MemoryPressure")].status}{"\t"}DiskPressure:{.status.conditions[?(@.type == "DiskPressure")].status}{"\n"}{end}' +name:gke-55d1ac88-213c MemoryPressure:False DiskPressure:False +name:gke-55d1ac88-vrpp MemoryPressure:False DiskPressure:False +name:gke-7253cc19-42g0 MemoryPressure:False DiskPressure:False +``` + +For more information about monitoring node conditions, see [Node conditions](https://kubernetes.io/docs/concepts/scheduling-eviction/node-pressure-eviction/#node-conditions) on [kubernetes.io](https://kubernetes.io/docs/home/). + +## Transparent Huge Pages + +Redis recommends disabling Transparent Huge Pages (THP) on every Kubernetes node that hosts Redis Software pods. THP causes a latency penalty and increases memory usage, and Redis logs a warning at startup when it is enabled. + +The operator does not manage this setting. Kubernetes has no API object for kernel tunables, so you configure THP on the nodes themselves. + +Check the current setting on your nodes and disable THP where it is enabled. For instructions, see your platform's documentation. For background on how THP affects Redis, see [Latency induced by transparent huge pages](/content/operate/oss_and_stack/management/optimization/latency.md#latency-induced-by-transparent-huge-pages). + +## Resource quotas + +Kubernetes uses the `ResourceQuota` object to limit resource consumption per namespace. This lets you limit the number of objects created by a namespace or the amount of compute resources consumed by a namespace. + +The resource settings for Redis Enterprise for Kubernetes are defined in the `operator.yaml` and the [`RedisEnterpriseCluster`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#spec) custom resource. + +The following settings are the minimum workloads for the operator to function. + +```yaml + resources: + limits: + cpu: 0.5 + memory: 256Mi + requests: + cpu: 0.5 + memory: 256Mi +``` +For more details on using resource quotas, see the [Kubernetes documentation](https://kubernetes.io/docs/concepts/policy/resource-quotas/). + + diff --git a/content/operate/kubernetes/8.2.0/recommendations/node-selection.md b/content/operate/kubernetes/8.2.0/recommendations/node-selection.md new file mode 100644 index 0000000000..55d75f22b2 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/recommendations/node-selection.md @@ -0,0 +1,248 @@ +--- +Title: Control node selection +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: This section provides information about how Redis Enterprise cluster + pods can be scheduled to only be placed on specific nodes or node pools. +linkTitle: Node selection +weight: 80 +url: '/operate/kubernetes/8.2.0/recommendations/node-selection/' +--- + +Kubernetes clusters often include nodes with different CPU and memory profiles. You control where Redis Enterprise cluster (REC) pods run by setting fields in the REC custom resource (CRD). + +A Redis Enterprise cluster (REC) runs as a StatefulSet. The Kubernetes scheduler assigns nodes when you create or resize the cluster, or when a pod restarts. + +Use these options to control pod placement: + +## Use node selectors + +The [`nodeSelector`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#spec) field matches the Kubernetes [`nodeSelector`](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector) syntax. +Label the nodes you want to target. For example, if nodes 'n1' and 'n2' are labeled with `memory=high`: + +```sh +kubectl label nodes n1 memory=high +kubectl label nodes n2 memory=high +``` + +The Redis Enterprise cluster CRD can request to be scheduled on these nodes: + +```yaml +apiVersion: app.redislabs.com/v1 +kind: RedisEnterpriseCluster +metadata: + name: rec +spec: + nodes: 3 + nodeSelector: + memory: high +``` + +The operator copies [`nodeSelector`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#spec) into the pod spec. The scheduler places pods only on nodes that match the selector. + +## Use node pools + +Node pools group similar nodes. Providers label nodes by pool. + +Use [`nodeSelector`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#spec) to target a pool by label. For example, on GKE: + +```yaml +apiVersion: app.redislabs.com/v1 +kind: RedisEnterpriseCluster +metadata: + name: rec +spec: + nodes: 3 + nodeSelector: + cloud.google.com/gke-nodepool: 'high-memory' +``` + +### Provider resources + +Cloud providers label nodes by pool. See links below for specific documentation. + +- GKE: + - [Create and manage cluster and node pool labels](https://cloud.google.com/kubernetes-engine/docs/how-to/creating-managing-labels) + - [Update node labels and taints for existing node pools](https://cloud.google.com/kubernetes-engine/docs/how-to/update-existing-nodepools) +- AKS: + - [Use labels in an AKS cluster](https://learn.microsoft.com/en-us/azure/aks/use-labels) + - [Manage node pools in AKS](https://learn.microsoft.com/en-us/azure/aks/manage-node-pools) +- EKS: + - [Create a managed node group with labels (AWS CLI)](https://docs.aws.amazon.com/cli/latest/reference/eks/create-nodegroup.html) + - [Update a managed node group to add labels (AWS CLI)](https://docs.aws.amazon.com/cli/latest/reference/eks/update-nodegroup-config.html) + + +## Use node taints + +Use node taints and pod tolerations to control REC pod scheduling. Set tolerations with [`spec.podTolerations`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#specpodtolerations) (standard [Kubernetes tolerations](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/#concepts)). + +Example: on a single node pool, reserve nodes n1–n3 for REC by adding taints: + +```sh +kubectl taint nodes n1 db=rec:NoSchedule +kubectl taint nodes n2 db=rec:NoSchedule +kubectl taint nodes n3 db=rec:NoSchedule +``` + +This blocks pods unless they tolerate the `db=rec` taint. + +Then add a matching toleration to the REC: + +```yaml +apiVersion: app.redislabs.com/v1 +kind: RedisEnterpriseCluster +metadata: + name: rec +spec: + nodes: 3 + podTolerations: + - key: db + operator: Equal + value: rec + effect: NoSchedule +``` + +A set of taints can also handle more complex use cases. +For example, a `role=test` or `role=dev` taint can be used to designate a node as dedicated for testing or development workloads via pod tolerations. + +## Use pod anti-affinity + +By default, REC node pods are not scheduled on the same node within the same cluster: + +```yaml +podAntiAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + - labelSelector: + matchLabels: + app: redis-enterprise + redis.io/cluster: rec + redis.io/role: node + topologyKey: kubernetes.io/hostname +``` + +Each pod has these labels. `redis.io/cluster` is your cluster name. + +Modify this rule to widen or narrow placement. For example, remove the `redis.io/cluster` label to prevent pods from different clusters from sharing a node: + +```yaml +apiVersion: app.redislabs.com/v1 +kind: RedisEnterpriseCluster +metadata: + name: rec +spec: + nodes: 3 + podAntiAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + - labelSelector: + matchLabels: + app: redis-enterprise + redis.io/role: node + topologyKey: kubernetes.io/hostname +``` + +To avoid co-locating with other database workloads, label those pods `local/role: database` and add anti-affinity to keep one database per node: + +```yaml +apiVersion: app.redislabs.com/v1 +kind: RedisEnterpriseCluster +metadata: + name: rec +spec: + nodes: 3 + extraLabels: + local/role: database + podAntiAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + - labelSelector: + matchLabels: + local/role: database + app: redis-enterprise + redis.io/cluster: rec + redis.io/role: node + topologyKey: kubernetes.io/hostname +``` + +Kubernetes will not schedule two pods with label `local/role: database` on the same node. + +## Enable rack awareness + +Enable rack-zone awareness to improve availability during rack or zone failures. + +> [!NOTE] +> When creating your rack-zone ID, there are some constraints to consider; see [rack-zone awareness](/content/operate/rs/clusters/configure/rack-zone-awareness.md#rack-zone-id-rules) for more info. + +Configure it with [`spec.rackAwarenessNodeLabel`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#spec) in the REC. + +### Choose a node label + +The most common label used for rack-zone awareness is topology.kubernetes.io/zone, a standard Kubernetes label that shows the zone a node runs in. Many Kubernetes platforms add this label to nodes by default, as noted in the [Kubernetes documentation](https://kubernetes.io/docs/setup/best-practices/multiple-zones/#nodes-are-labeled). + +If your platform doesn’t set this label automatically, you can use any custom label that describes the node’s topology (such as rack, zone, or region). + +### Label all eligible nodes + +> [!WARNING] +> All eligible nodes **must** have the label for rack awareness to work. The operator requires every node that might run Redis Enterprise pods to be labeled. If any nodes are missing the label, reconciliation fails. + +Eligible nodes are nodes where REC pods can run. By default, this means all worker nodes. You can limit eligibility with [`spec.nodeSelector`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#spec). + +Give each eligible node a label value that reflects its rack, zone, or region. + +Check node label values: + +```sh +kubectl get nodes -o custom-columns="name:metadata.name","rack\\zone:metadata.labels.topology\.kubernetes\.io/zone" + +name rack\zone +ip-10-0-x-a.eu-central-1.compute.internal eu-central-1a +ip-10-0-x-b.eu-central-1.compute.internal eu-central-1a +ip-10-0-x-c.eu-central-1.compute.internal eu-central-1b +ip-10-0-x-d.eu-central-1.compute.internal eu-central-1b +``` + +### Enable the cluster role + +Grant the operator read access to node labels with a ClusterRole and ClusterRoleBinding. + +ClusterRole: + +{{}} + +Bind to the `redis-enterprise-operator` service account: + +{{}} + +Apply these files with `kubectl apply`. For example: + +```sh +kubectl apply -f rack-aware-cluster-role.yaml +kubectl apply -f rack-aware-cluster-role-binding.yaml +``` + +After you apply the role and binding, you can configure rack awareness. + +### Configure rack awareness + +Set [`spec.rackAwarenessNodeLabel`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#spec) to the node label to use: + +{{}} + +> [!NOTE] +> When you set [`spec.rackAwarenessNodeLabel`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#spec), the operator sets the anti-affinity `topologyKey` to that label unless you define [`spec.podAntiAffinity`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#specpodantiaffinity). If you define both, make sure `topologyKey` matches your node label. + +### Rack awareness limitations + +> [!WARNING] +> **Pod restart distribution maintenance**: When rack awareness is enabled, node pods and shards are initially deployed based on rack constraints to ensure proper distribution across zones. However, Redis Enterprise does not automatically maintain this distribution when node pods are restarted. +> +> After pod restarts, the rack awareness policy may be violated, requiring manual intervention to restore proper shard distribution. While Redis Enterprise provides tools to identify shards that need to be moved to restore correct rack distribution, it does not provide automated orchestration to perform these moves. + +**Important considerations for production deployments:** + +- At scale, manual shard redistribution can become operationally challenging +- This limitation is particularly important for edge deployments where automated recovery is preferred +- Plan for operational procedures to handle rack awareness policy violations after pod restarts +- Consider monitoring tools to detect when rack awareness constraints are violated diff --git a/content/operate/kubernetes/8.2.0/recommendations/persistent-volumes.md b/content/operate/kubernetes/8.2.0/recommendations/persistent-volumes.md new file mode 100644 index 0000000000..da8049a46c --- /dev/null +++ b/content/operate/kubernetes/8.2.0/recommendations/persistent-volumes.md @@ -0,0 +1,103 @@ +--- +Title: Use persistent volumes in Redis Enterprise clusters +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: This section covers details about how persistent volumes are sized and + specified for Redis Enterprise cluster deployments. +linkTitle: Persistent volumes +weight: 40 +url: '/operate/kubernetes/8.2.0/recommendations/persistent-volumes/' +--- +## Storage types + +Redis Enterprise for Kubernetes can only use storage classes supported by block storage. Block storage is mounted at the Kubernetes node level and utilizes EXT4 or XFS file systems. It can be sourced from enterprise-grade SANs or cloud environments such as EBS, Azure Managed Disks, or GCP persistent disks. + +> [!WARNING] +> NFS, NFS-like, and multi-read-write/shared storage options are not supported. These types of storage are often slow and can cause locking behaviors that are incompatible with the requirements of database storage. + +## REC `persistentSpec` field +To deploy a Redis Enterprise cluster with Redis Enterprise operator the +spec should include a *persistentSpec* section, in the +*redis-enterprise-cluster.yaml* file: + + spec: + nodes: 3 + persistentSpec: + enabled: true + storageClassName: "standard" + volumeSize: "23Gi" #optional + +Persistence storage is a requirement for production deployments. + +> [!NOTE] +> For **production deployments** of Redis Enterprise Cluster on Kubernetes, +> the Redis Enterprise Cluster (REC) must be deployed with persistence enabled. +> The REC deployment files in the [Kubernetes documentation](https://github.com/RedisLabs/redis-enterprise-k8s-docs) contain this declaration by default. + +## Volume size + +*volumeSize* is an optional definition. By default, if the definition is +omitted, operator allocates five times (5x) the amount of memory (RAM) +defined for nodes (see example below), which is the recommended +persistent storage size as described in the [Hardware +requirements](/content/operate/rs/installing-upgrading/install/plan-deployment/hardware-requirements.md) article. + +To explicitly specify the persistent storage size, use the *volumeSize* +property as described in the example above. + +Persistent volume claims can be expanded, but not reduced after creation. See [Expand PersistentVolumeClaim (PVC)](/content/operate/kubernetes/8.2.0/re-clusters/expand-pvc.md) for details. + +> [!NOTE] +> We recommend that you omit the volumeSize definition from the REC declaration +> so that the Redis Enterprise Cluster deployment on Kubernetes use the default volume size. + +## Storage class name + +*storageClassName* determines the Storage Class resource, which is +defined by the Kubernetes cluster administrator, to be used for +persistent storage. + +Different Kubernetes distributions and different deployments use +different Storage Class resources. + +In order to determine the Storage Class resources available for your K8s +deployment, use the following command: + + kubectl get StorageClass + +Typically, AWS provides "gp3" as the Storage Class name while GKE uses "standard." +Azure provides two Storage Classes: "default" using HDDs, and "managed-premium" using SSDs. + +Below is an example of a response to the command. + +| | | +| ----------------------- | ------------------------------------------------------- | +| *Name:* | *gp3* | +| *IsDefaultClass:* | *Yes* | +| *Annotations:* | *storageclass.beta.kubernetes.io/is-default-class=true* | +| *Provisioner:* | *kubernetes.io/aws-ebs* | +| *Parameters:* | *encrypted=false,kmsKeyId=,type=gp3* | +| *AllowVolumeExpansion:* | *\* | +| *MountOptions:* | *\* | +| *ReclaimPolicy:* | *Delete* | +| *VolumeBindingMode:* | *Immediate* | +| *Events:* | *\* | + +> [!NOTE] +> storageClassName must be specified for this deployment type. + +> [!WARNING] +> The storage class cannot be changed after deployment. Trying to change this value after deployment could result in unexpected and potentially damaging behavior. + +Example of the redisEnterpriseNodeResources definition: + + redisEnterpriseNodeResources: + limits: + cpu: "4000m" + memory: 4Gi + requests: + cpu: "4000m" + memory: 4Gi diff --git a/content/operate/kubernetes/8.2.0/recommendations/pod-stability.md b/content/operate/kubernetes/8.2.0/recommendations/pod-stability.md new file mode 100644 index 0000000000..bb36f8c16d --- /dev/null +++ b/content/operate/kubernetes/8.2.0/recommendations/pod-stability.md @@ -0,0 +1,115 @@ +--- +Title: Manage pod stability +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: This section provides information about how you can use quality of service, + priority class, eviction thresholds and resource monitoring to maintain cluster + node pod stability. +linkTitle: Pod stability +weight: 90 +url: '/operate/kubernetes/8.2.0/recommendations/pod-stability/' +--- + +Kubernetes clusters manage the allocation of system resources and can evict pods to release system resources. +Here are some ways that you can configure the Redis Enterprise node pods to maintain pod stability: + +## Guaranteed quality of service + +A running pod has a quality of service measure assigned to it that is +one of three [quality of service classes](https://kubernetes.io/docs/tasks/configure-pod-container/quality-service-pod/): +Guaranteed, Burstable, and Best Effort. +You can assure the Guaranteed class is assigned to the Redis Enterprise node pods +by following the right guidelines. + +To get a Guaranteed quality of service class assigned: + + * Every container in the pod must have a memory limit and a memory request, and they must be the same. + * Every container in the pod must have a CPU limit and a CPU request, and they must be the same. + +If resources limits and requests are not specified in the Redis Enterprise CRD, +these requirements are met in the default version created by the operator. of your Redis Enterprise cluster CRD, +Otherwise, you must set the limits and requests to the same value for memory and CPU in the `redisEnterpriseNodeResources` +section of the CRD. + +Sidecar containers also impact the quality of service class assignment for the pod. + +To check the quality of service class of any running Redis Enterprise node pod, run: + +```sh +kubectl get pod rec-0 --o jsonpath="{.status.qosClass}" +``` + +where `rec-0` is the name of one of the pods associated with the Redis Enterprise cluster. + +## Using priority to protect from preemption + +When a Redis Enterprise node pod is scheduled, it can be assigned a +[priority class](https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/) +with the `priorityClassName` property. This property value is the name of +a priority class that must already exist within the cluster. + +A sufficiently high priority will prevent other workloads with a lower +priority from preempting the scheduling of Redis Enterprise Nodes. Similarly, +a high value may also prevent eviction when lower priority workloads +are deployed on the same cluster. + +The successful use of this strategy involves first creating a priority class with a very large priority value: + +```yaml +apiVersion: scheduling.k8s.io/v1 +kind: PriorityClass +metadata: + name: redis-enterprise-priority +value: 1000000000 +globalDefault: false +description: "This priority class should be used for Redis Enterprise pods only." +``` + +Then, you refer to the priority class by name in your Redis Enterprise cluster CRD: + +```yaml +apiVersion: app.redislabs.com/v1 +kind: RedisEnterpriseCluster +metadata: + name: example-redisenterprisecluster +spec: + size: 3 + priorityClassName: "redis-enterprise-priority" +``` + +Alternatively, you can also [disable preemption entirely](https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/#how-to-disable-preemption). + +## Managing eviction thresholds + +Eviction thresholds are typically managed by kubelet arguments. +You can set the thresholds: + +- On OpenShift - In the [config file](https://docs.openshift.com/container-platform/3.11/admin_guide/out_of_resource_handling.html#out-of-resource-create-config). +- On GKE - In the [managed settings](https://cloud.google.com/kubernetes-engine/docs/concepts/cluster-architecture#node_allocatable). + +We recommend that you: + + * Set the [soft eviction threshold](https://kubernetes.io/docs/tasks/administer-cluster/out-of-resource/#soft-eviction-thresholds) + to be higher than the [hard eviction threshold](https://kubernetes.io/docs/tasks/administer-cluster/out-of-resource/#hard-eviction-thresholds). + The high soft threshold makes the node condition change earlier, and alerts the administrator. + * Set `eviction-max-pod-grace-period` high enough to allow the RS pods to migrate the Redis databases before the pods are force killed. + * Set the `eviction-soft-grace-period` high enough that the administrator (or a k8s auto-scaling mechanism) scales k8s up or out. + +## Monitoring for memory and disk usage + +We recommend that you monitor the node for MemoryPressure and DiskPressure. +When both of these conditions are true, then an [eviction threshold](https://kubernetes.io/docs/tasks/administer-cluster/out-of-resource/#eviction-thresholds) +is met and the pod is evicted. + +To retrieve the flags, run the command: + +```sh +$kubectl get nodes -o jsonpath='{range .items[*]}name:{.metadata.name}{"\t"}MemoryPressure:{.status.conditions[?(@.type == "MemoryPressure")].status}{"\t"}DiskPressure:{.status.conditions[?(@.type == "DiskPressure")].status}{"\n"}{end}' + +name:gke-55d1ac88-213c MemoryPressure:False DiskPressure:False +name:gke-55d1ac88-vrpp MemoryPressure:False DiskPressure:False +name:gke-7253cc19-42g0 MemoryPressure:False DiskPressure:False +``` diff --git a/content/operate/kubernetes/8.2.0/recommendations/sizing-on-kubernetes.md b/content/operate/kubernetes/8.2.0/recommendations/sizing-on-kubernetes.md new file mode 100644 index 0000000000..770f018456 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/recommendations/sizing-on-kubernetes.md @@ -0,0 +1,121 @@ +--- +Title: Size and scale a Redis Enterprise cluster deployment on Kubernetes +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: This section provides information about sizing and scaling Redis Enterprise + in a Kubernetes deployment. +linkTitle: Sizing and scaling +weight: 50 +url: '/operate/kubernetes/8.2.0/recommendations/sizing-on-kubernetes/' +--- +The following article reviews the mechanism and methods available for sizing +and scaling a Redis Enterprise cluster deployment. + +For minimum and recommended sizing, always follow the sizing guidelines +detailed in the [Redis Enterprise hardware requirements](/content/operate/rs/installing-upgrading/install/plan-deployment/hardware-requirements.md). + +## Sizing and scaling cluster nodes + +### Setting the number of cluster nodes + +Define the number of cluster nodes in redis-enterprise-cluster.yaml file. + + spec: + + nodes: 3 + +The number of nodes in the cluster must be an uneven number +equal to or greater than 3. Refer to the article [Highly Available Redis](https://redislabs.com/redis-enterprise/technology/highly-available-redis/) +for a detailed explanation on this topic. + +Set the number of cluster nodes during deployment +by editing the redis-enterprise-cluster.yaml file and +applying the file by running: + + kubectl apply -f redis-enterprise-cluster.yaml + +### Scaling out + +To scale out a Redis Enterprise Cluster deployment, increase the number of nodes +in the spec. For example, to scale the cluster out from 3 nodes to 5 nodes, +edit the redis-enterprise-cluster.yaml file with the following: + + spec: + + nodes: 5 + +To apply the new cluster configuration run: + + kubectl apply -f redis-enterprise-cluster.yaml + +> [!WARNING] +> Decreasing the number of nodes is not supported. + +### Sizing compute resources + +To set the compute resources required for each node, +use the redisEnterpriseNodeResources spec field. + +Under redisEnterpriseNodeResources spec, set the following fields +according to the provided guidelines. + +- limits – specifies the maximum compute resources for a Redis node +- requests – specifies the minimum compute resources for a Redis node + +For example: + + redisEnterpriseNodeResources: + + limits: + + cpu: "2000m" + + memory: 4Gi + + requests: + + cpu: "2000m" + + memory: 4Gi + +The default values, if unspecified, are 2 cores (2000m) and 4GB (4Gi). + +Set the compute resources for cluster nodes during deployment +by editing the redis-enterprise-cluster.yaml file and +applying the file by running: + + kubectl apply -f redis-enterprise-cluster.yaml + +### Scaling up node compute resources + +To scale up nodes in an existing Redis Enterprise Cluster deployment, +adjust the cpu and memory parameters in the spec. For example, +to scale nodes up to the recommended amount of compute resources, +edit the redis-enterprise-cluster.yaml file with the following: + +redisEnterpriseNodeResources: + + limits: + + cpu: "8000m" + + memory: 30Gi + + requests + + cpu: "8000m" + + memory: 30Gi + +Then, apply the file by running: + + kubectl apply -f redis-enterprise-cluster.yaml + +> [!WARNING] +> When adjusting compute resources, make sure the ratio of persistent volume size and the new memory size are in accordance to the [Hardware +> requirements](/content/operate/rs/installing-upgrading/install/plan-deployment/hardware-requirements.md). +> +> Persistent volume claims can be expanded, but not reduced after creation. See [Expand PersistentVolumeClaim (PVC)](/content/operate/kubernetes/8.2.0/re-clusters/expand-pvc.md) for details. diff --git a/content/operate/kubernetes/8.2.0/reference/_index.md b/content/operate/kubernetes/8.2.0/reference/_index.md new file mode 100644 index 0000000000..20e51a9570 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/reference/_index.md @@ -0,0 +1,115 @@ +--- +Title: Reference +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Resources to help you manage Redis Enterprise custom resources on Kubernetes. +hideListLinks: true +linkTitle: Reference +weight: 89 +url: '/operate/kubernetes/8.2.0/reference/' +--- + +This reference documentation covers Redis Enterprise custom resources, API specifications, and practical instructions for creating, configuring, and managing Redis Enterprise deployments on Kubernetes. + +## Work with custom resources + +Redis Enterprise for Kubernetes uses custom resources to manage clusters and databases. Use standard Kubernetes tools to create, modify, and delete these resources. + +### Create custom resources + +Use `kubectl apply` with YAML manifests to create custom resources: + +```bash +kubectl apply -f my-redis-cluster.yaml +kubectl apply -f my-redis-database.yaml +``` + +### View custom resources + +Use these commands to list and inspect existing custom resources: + +```bash +# List Redis Enterprise clusters +kubectl get rec + +# List Redis Enterprise databases +kubectl get redb + +# List Active-Active databases +kubectl get reaadb + +# List remote clusters +kubectl get rerc + +# Get detailed information about a specific resource +kubectl describe rec my-cluster +kubectl describe redb my-database +``` + +### Modify custom resources + +Edit the YAML manifest and reapply to update custom resources: + +```bash +# Edit and apply updated manifest +kubectl apply -f updated-redis-cluster.yaml + +# Or edit directly (not recommended for production) +kubectl edit rec my-cluster +kubectl edit redb my-database +``` + +## YAML examples + +Find complete YAML examples for common deployment scenarios: + +- [YAML examples](/content/operate/kubernetes/8.2.0/reference/yaml/_index.md) - Ready-to-use YAML configurations for different deployment types + +### Example categories + +- [Basic deployment](/content/operate/kubernetes/8.2.0/reference/yaml/basic-deployment.md) - Essential YAML files for simple Redis Enterprise deployment +- [Rack awareness](/content/operate/kubernetes/8.2.0/reference/yaml/rack-awareness.md) - YAML examples for rack-aware deployments across availability zones +- [Active-Active](/content/operate/kubernetes/8.2.0/reference/yaml/active-active.md) - YAML examples for Active-Active databases across multiple clusters +- [Multi-namespace](/content/operate/kubernetes/8.2.0/reference/yaml/multi-namespace.md) - YAML examples for deploying across multiple namespaces + +## API reference + +Review complete API specifications for all Redis Enterprise custom resources: + +### Core resources + +- [Redis Enterprise cluster API (REC)](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md) - Manage Redis Enterprise clusters +- [Redis Enterprise database API (REDB)](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md) - Manage Redis databases + +### Active-Active resources + +- [Active-Active database API (REAADB)](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_active_active_database_api.md) - Manage Active-Active databases +- [Remote cluster API (RERC)](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_remote_cluster_api.md) - Configure remote cluster connections + +### Access control resources + +- [RedisEnterpriseUser API (REUSER)](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_user_api.md) - Manage users for access control +- [RedisEnterpriseRole API (REROLE)](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_role_api.md) - Define roles scoped to selected databases +- [RedisEnterpriseRoleBinding API (REROLEBINDING)](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_role_binding_api.md) - Bind users to a scoped role +- [RedisEnterpriseClusterRole API (RECROLE)](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_role_api.md) - Define cluster-scoped roles +- [RedisEnterpriseClusterRoleBinding API (RECROLEBINDING)](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_role_binding_api.md) - Bind users to a cluster-scoped role +- [RedisEnterpriseACL API (REACL)](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_acl_api.md) - Define access control lists + +## Compatibility + +Check supported Kubernetes distributions and versions: + +- [Supported Kubernetes distributions](/content/operate/kubernetes/8.2.0/reference/supported_k8s_distributions.md) - Compatible Kubernetes platforms and versions + +## Best practices + +Follow these best practices when working with custom resources: + +- **Use version control**: Store your YAML manifests in version control systems +- **Validate before applying**: Use `kubectl apply --dry-run=client` to validate changes +- **Monitor resource status**: Check resource status after you apply changes +- **Follow naming conventions**: Use consistent naming for easier management +- **Document configurations**: Add annotations and labels to describe resource purpose \ No newline at end of file diff --git a/content/operate/kubernetes/8.2.0/reference/api/_index.md b/content/operate/kubernetes/8.2.0/reference/api/_index.md new file mode 100644 index 0000000000..991c69ac0a --- /dev/null +++ b/content/operate/kubernetes/8.2.0/reference/api/_index.md @@ -0,0 +1,63 @@ +--- +title: API reference +categories: +- docs +- operate +- kubernetes +linkTitle: API reference +description: Reference documentation for Redis Enterprise operator APIs +weight: 30 +alwaysopen: false +hideListLinks: true +aliases: +url: '/operate/kubernetes/8.2.0/reference/api/' +--- + +The Redis Enterprise operator provides Kubernetes custom resource definitions (CRDs) that let you manage Redis Enterprise clusters and databases declaratively. This section contains the complete API reference for all operator resources. + +## API versions and stability + +The operator uses different API versions to indicate stability and feature maturity: + +- **`app.redislabs.com/v1`** - Stable APIs for production use +- **`app.redislabs.com/v1alpha1`** - Alpha APIs that may change in future releases + +## Custom resources + +| Resource | API Version | Purpose | +|----------|-------------|---------| +| [RedisEnterpriseCluster (REC)](redis_enterprise_cluster_api) | `v1` | Manages Redis Enterprise cluster deployments | +| [RedisEnterpriseDatabase (REDB)](redis_enterprise_database_api) | `v1alpha1` | Creates and configures Redis databases | +| [RedisEnterpriseActiveActiveDatabase (REAADB)](redis_enterprise_active_active_database_api) | `v1alpha1` | Sets up active-active databases across clusters | +| [RedisEnterpriseRemoteCluster (RERC)](redis_enterprise_remote_cluster_api) | `v1alpha1` | Defines remote cluster connections for active-active | +| [RedisEnterpriseUser (REUSER)](redis_enterprise_user_api) | `v1alpha1` | Defines users for cluster access control | +| [RedisEnterpriseRole (REROLE)](redis_enterprise_role_api) | `v1alpha1` | Defines a role scoped to selected databases | +| [RedisEnterpriseRoleBinding (REROLEBINDING)](redis_enterprise_role_binding_api) | `v1alpha1` | Binds users to a RedisEnterpriseRole | +| [RedisEnterpriseClusterRole (RECROLE)](redis_enterprise_cluster_role_api) | `v1alpha1` | Defines a cluster-scoped role | +| [RedisEnterpriseClusterRoleBinding (RECROLEBINDING)](redis_enterprise_cluster_role_binding_api) | `v1alpha1` | Binds users to a RedisEnterpriseClusterRole | +| [RedisEnterpriseACL (REACL)](redis_enterprise_acl_api) | `v1alpha1` | Defines an access control list (ACL) for a database | + +## Working with the APIs + +### Using kubectl + +Manage all resources using standard `kubectl` commands: + +```bash +# List all Redis Enterprise clusters +kubectl get rec + +# Get detailed information about a specific database +kubectl describe redb my-database + +# Apply a configuration from a YAML file +kubectl apply -f my-redis-config.yaml +``` + +### Resource relationships + +- Create a `RedisEnterpriseCluster` (REC) first to provide the Redis Enterprise infrastructure +- Create `RedisEnterpriseDatabase` (REDB) resources within a cluster to provision individual databases +- Use `RedisEnterpriseActiveActiveDatabase` (REAADB) with `RedisEnterpriseRemoteCluster (RERC)` resources to define participating clusters + +For complete YAML configuration examples, see the [YAML examples](/content/operate/kubernetes/8.2.0/reference/yaml/_index.md) section. diff --git a/content/operate/kubernetes/8.2.0/reference/api/kubernetes-api-reference-template.tmpl b/content/operate/kubernetes/8.2.0/reference/api/kubernetes-api-reference-template.tmpl new file mode 100644 index 0000000000..312622dd39 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/reference/api/kubernetes-api-reference-template.tmpl @@ -0,0 +1,105 @@ +{{- $group:= index .Groups 0 -}} +{{- $kinds:= index $group.Kinds 0 -}} +{{- $resource := $kinds.Name -}} +{{- $title:= printf "%s %s" $kinds.Name "API Reference" -}} +--- +title: {{ $title }} +alwaysopen: false +categories: +- docs +- operate +- kubernetes +linkTitle: {{ $title }} +weight: 30 +url: '/operate/kubernetes/8.2.0/reference/api/kubernetes-api-reference-template.tmpl/' +--- + +apiVersion: +{{ $group:= index .Groups 0 }} + +- [{{$group.Group}}/{{$group.Version}}](#{{ anchorize (printf "%s/%s" $group.Group $group.Version) }}) + + +{{ $group:= index .Groups 0 }} + +# {{$group.Group}}/{{$group.Version}} + +{{range $group.Kinds}} +{{- $kind := . -}} +{{range .Types}} + +{{if not .IsTopLevel}} +### {{.Name}} +{{if .ParentKey}}[↩ Parent](#{{.ParentKey}}){{end}} +{{end}} +{{.Description}} + + + + + + + + + + + + {{- if .IsTopLevel -}} + + + + + + + + + + + + + + + + + + + {{- end -}} + {{- range .Fields -}} + + + + + + + {{- end -}} + +
NameTypeDescriptionRequired
apiVersionstring{{$group.Group}}/{{$group.Version}}true
kindstring{{$kind.Name}}true
metadataobjectRefer to the Kubernetes API documentation for the fields of the `metadata` field.true
{{if .TypeKey}}{{.Name}}{{else}}{{.Name}}{{end}}{{.Type}} + {{.Description}}
+ {{- if or .Schema.XValidations .Schema.Format .Schema.Enum .Schema.Default .Schema.Minimum .Schema.Maximum }} +
+ {{- end}} + {{- if .Schema.XValidations }} + Validations: + {{- range .Schema.XValidations -}} +
  • {{ .Rule }}: {{ .Message }}
    • + {{- end -}} + {{- end }} + {{- if .Schema.Format }} + Format: {{ .Schema.Format }}
    + {{- end }} + {{- if .Schema.Enum }} + Enum: {{ .Schema.Enum | toStrings | join ", " }}
    + {{- end }} + {{- if .Schema.Default }} + Default: {{ .Schema.Default }}
    + {{- end }} + {{- if .Schema.Minimum }} + Minimum: {{ .Schema.Minimum }}
    + {{- end }} + {{- if .Schema.Maximum }} + Maximum: {{ .Schema.Maximum }}
    + {{- end }} +     		{{.Required}}
    + +{{- end}}{{/* range .Types */}} +{{- end}}{{/* range .Kinds */}} diff --git a/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_acl_api.md b/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_acl_api.md new file mode 100644 index 0000000000..5179fc7e2a --- /dev/null +++ b/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_acl_api.md @@ -0,0 +1,128 @@ +--- +title: RedisEnterpriseACL API Reference +alwaysopen: false +categories: +- docs +- operate +- kubernetes +linkTitle: REACL API +weight: 30 +url: '/operate/kubernetes/8.2.0/reference/api/redis_enterprise_acl_api/' +--- + +apiVersion: + + +- [app.redislabs.com/v1alpha1](#appredislabscomv1alpha1) + + + + +# app.redislabs.com/v1alpha1 + + + + +RedisEnterpriseACL represents an access control list definition for a Redis database. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    apiVersionstringapp.redislabs.com/v1alpha1true
    kindstringRedisEnterpriseACLtrue
    metadataobjectRefer to the Kubernetes API documentation for the fields of the `metadata` field.true
    specobject + RedisEnterpriseACLSpec defines the desired state of RedisEnterpriseACL
    +     		false
    statusobject + RedisEnterpriseACLStatus defines the observed state of RedisEnterpriseACL
    +     		false
    + + +### spec +[↩ Parent](#) + +RedisEnterpriseACLSpec defines the desired state of RedisEnterpriseACL + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    aclstring + Redis ACL rule
    +     		false
    + + +### status +[↩ Parent](#) + +RedisEnterpriseACLStatus defines the observed state of RedisEnterpriseACL + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    observedGenerationinteger + The most recent generation of this resource that was observed and acted upon by the controller.
    +
    + Format: int64
    +     		false
    uidstring + The internal UID of the Redis ACL object defined in the Redis Enterprise Cluster.
    +     		false
    diff --git a/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_active_active_database_api.md b/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_active_active_database_api.md new file mode 100644 index 0000000000..3d09e70413 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_active_active_database_api.md @@ -0,0 +1,1906 @@ +--- +title: RedisEnterpriseActiveActiveDatabase API Reference +alwaysopen: false +categories: +- docs +- operate +- kubernetes +linkTitle: REAADB API +weight: 30 +url: '/operate/kubernetes/8.2.0/reference/api/redis_enterprise_active_active_database_api/' +--- + +apiVersion: + + +- [app.redislabs.com/v1alpha1](#appredislabscomv1alpha1) + + + + +# app.redislabs.com/v1alpha1 + + + + +The schema for the redisenterpriseactiveactivedatabase API. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    apiVersionstringapp.redislabs.com/v1alpha1true
    kindstringRedisEnterpriseActiveActiveDatabasetrue
    metadataobjectRefer to the Kubernetes API documentation for the fields of the `metadata` field.true
    specobject + Defines the desired state of RedisEnterpriseActiveActiveDatabase.
    +     		false
    statusobject + Defines the observed state of RedisEnterpriseActiveActiveDatabase.
    +     		false
    + + +### spec +[↩ Parent](#) + +Defines the desired state of RedisEnterpriseActiveActiveDatabase. + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    participatingClusters[]object + List of participating cluster specifications and configurations.
    +     		true
    globalConfigurationsobject + Global configurations for the Active-Active database. Contains the global properties for each participating cluster database within the Active-Active database.
    +     		false
    redisEnterpriseClusterobject + Connection to the Redis Enterprise Cluster.
    +     		false
    + + +### spec.participatingClusters[] +[↩ Parent](#spec) + +Specifies the configuration for a participating cluster in the Active-Active database. + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    namestring + Name of the remote cluster custom resource to link.
    +     		true
    externalReplicationPortinteger + Port number for the replication endpoint. Use this field when you use LoadBalancers for synchronization between Active-Active replicas and need to specify the port number that the LoadBalancer listens on.
    +     		false
    namespacestring + Namespace where the REAADB object is deployed within the corresponding participating cluster. Ensure that the Redis Enterprise operator is configured to watch this namespace in the corresponding cluster and that the required RBAC configuration is properly set up. For more information about setting up multiple namespaces, see https://redis.io/docs/latest/operate/kubernetes/re-clusters/multi-namespace/. If you don't specify a namespace, the REAADB is deployed to the REC's namespace in the corresponding cluster.
    +     		false
    + + +### spec.globalConfigurations +[↩ Parent](#spec) + +Global configurations for the Active-Active database. Contains the global properties for each participating cluster database within the Active-Active database. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    activeActiveobject + Connection/ association to the Active-Active database.
    +     		false
    alertSettingsobject + Settings for database alerts.
    +     		false
    auditingobject + Database auditing configuration.
    +     		false
    backupobject + Target for automatic database backups.
    +     		false
    clientAuthenticationCertificates[]string + The Secrets containing TLS Client Certificate to use for Authentication
    +     		false
    connectionSettingsobject + Connection-related settings such as proxy connections and scheduling policy.
    +     		false
    dataInternodeEncryptionboolean + Internode encryption (INE) setting. An optional boolean setting, overriding a similar cluster-wide policy. If set to False, INE is guaranteed to be turned off for this DB (regardless of cluster-wide policy). If set to True, INE will be turned on, unless the capability is not supported by the DB ( in such a case we will get an error and database creation will fail). If left unspecified, will be disabled if internode encryption is not supported by the DB (regardless of cluster default). Deleting this property after explicitly setting its value shall have no effect.
    +     		false
    databasePortinteger + Database port number. TCP port on which the database is available. Will be generated automatically if omitted. can not be changed after creation
    +     		false
    databaseSecretNamestring + The name of the secret that holds the password to the database (redis databases only). If secret does not exist, it will be created. To define the password, create an opaque secret and set the name in the spec. The password will be taken from the value of the 'password' key. Use an empty string as value within the secret to disable authentication for the database. Notes - For Active-Active databases this secret will not be automatically created, and also, memcached databases must not be set with a value, and a secret/password will not be automatically created for them. Use the memcachedSaslSecretName field to set authentication parameters for memcached databases.
    +     		false
    databaseServicePortinteger + A custom port to be exposed by the database services. Can be be modified/added/removed after REDB creation. If set, it'll replace the default service port (namely, databasePort or defaultRedisPort).
    +     		false
    defaultUserboolean + Is connecting with a default user allowed? If disabled, the DatabaseSecret will not be created or updated
    +     		false
    evictionPolicystring + Database eviction policy. See more https://redis.io/docs/latest/rs/administering/database-operations/eviction-policy/
    +     		false
    isRofboolean + Whether it is an RoF database or not. Applicable only for databases of type "REDIS". Assumed to be false if left blank.
    +     		false
    memcachedSaslSecretNamestring + Credentials used for binary authentication in memcached databases. The credentials should be saved as an opaque secret and the name of that secret should be configured using this field. For username, use 'username' as the key and the actual username as the value. For password, use 'password' as the key and the actual password as the value. Note that connections are not encrypted.
    +     		false
    memorySizestring + memory size of database. use formats like 100MB, 0.1GB. minimum value in 100MB. When redis on flash (RoF) is enabled, this value refers to RAM+Flash memory, and it must not be below 1GB.
    +     		false
    modulesList[]object + List of modules associated with the database. The list of valid modules for the specific cluster can be retrieved from the status of the REC object. Use the "name" and "versions" fields for the specific module configuration. If specifying an explicit version for a module, automatic modules versions upgrade must be disabled by setting the '.upgradeSpec.upgradeModulesToLatest' field in the REC to 'false'. Note that the option to specify module versions is deprecated, and will be removed in future releases.
    +     		false
    ossClusterboolean + OSS Cluster mode option. Note that not all client libraries support OSS cluster mode.
    +     		false
    ossClusterSettingsobject + Additional OSS cluster settings that may be provided to tweak OSS cluster behavior.
    +     		false
    persistenceenum + Database on-disk persistence policy
    +
    + Enum: disabled, aofEverySecond, aofAlways, snapshotEvery1Hour, snapshotEvery6Hour, snapshotEvery12Hour
    +     		false
    proxyPolicystring + The policy used for proxy binding to the endpoint. Supported proxy policies are: single/all-master-shards/all-nodes When left blank, the default value will be chosen according to the value of ossCluster - single if disabled, all-master-shards when enabled
    +     		false
    rackAwareboolean + Whether database should be rack aware. This improves availability - more information: https://redis.io/docs/latest/rs/concepts/high-availability/rack-zone-awareness/
    +     		false
    redisEnterpriseClusterobject + Connection to Redis Enterprise Cluster
    +     		false
    redisVersionstring + Redis OSS version. Version can be specified via prefix, or via channels - for existing databases - Upgrade Redis OSS version. For new databases - the version which the database will be created with. If set to 'major' - will always upgrade to the most recent major Redis version. If set to 'latest' - will always upgrade to the most recent Redis version. Depends on 'redisUpgradePolicy' - if you want to set the value to 'latest' for some databases, you must set redisUpgradePolicy on the cluster before. Possible values are 'major' or 'latest' When using upgrade - make sure to backup the database before. This value is used only for database type 'redis'. Note - Specifying Redis version is currently not supported for Active-Active database.
    +     		false
    replicaSources[]object + What databases to replicate from
    +     		false
    replicationboolean + In-memory database replication. When enabled, database will have replica shard for every master - leading to higher availability. Defaults to false.
    +     		false
    resp3boolean + Whether this database supports RESP3 protocol. Note - Deleting this property after explicitly setting its value shall have no effect. Please view the corresponding field in RS doc for more info.
    +     		false
    rofRamRatiointeger + RAM allocation ratio for Redis Flex (v2) databases as a percentage of total data size. Valid range 0-100. When omitted, RS uses the default value of 50%. Controls how much RAM is allocated per unit of data (e.g., 30% means 3MB RAM per 10MB data). RAM grows proportionally with data until rofRamSize limit is reached (if specified). Only applicable when isRof=true and Redis version >= 8.0 (BigStore v2 - Redis Flex).
    +
    + Minimum: 0
    + Maximum: 100
    +     		false
    rofRamSizestring + The size of the RAM portion of an RoF database. Similarly to "memorySize" use formats like 100MB, 0.1GB It must be at least 10% of combined memory size (RAM+Flash), as specified by "memorySize".
    +     		false
    rolesPermissions[]object + List of Redis Enteprise ACL and Role bindings to apply
    +     		false
    searchOnBigstoreboolean + Enables search module indexing on flash storage for Redis Flex (v2) databases. Only applicable when isRof=true and Redis version >= 8.6. Defaults to false.
    +     		false
    shardCountinteger + Number of database server-side shards
    +     		false
    shardingEnabledboolean + Toggles database sharding for REAADBs (Active Active databases) and enabled by default. This field is blocked for REDB (non-Active Active databases) and sharding is toggled via the shardCount field - when shardCount is 1 this is disabled otherwise enabled.
    +     		false
    shardsPlacementstring + Control the density of shards - should they reside on as few or as many nodes as possible. Available options are "dense" or "sparse". If left unset, defaults to "dense".
    +     		false
    tagsmap[string]string + Tags to apply to the Active-Active database, as a map of key to value. The same set is merged into the BDB tags array of the default config and every participating-cluster instance. On key collision with an existing BDB tag, the value here wins. Removing a key after it was previously synced (recorded in status.managedTags) removes it from every BDB. Operator-reserved keys (managed_by, redb_name, redb_namespace, db_service_port, oss_cluster_access, redb_resource, redb_resource_name, global_configurations_spec, replication_endpoint_port, rerc_name, and the secret-tracking keys redis.io/db-sec-name, redis.io/db-sec-latest, redis.io/certs-latest, redis.io/bu-sec-name, redis.io/bu-sec-latest) are not allowed and will be rejected.
    +     		false
    tlsModeenum + Require SSL authenticated and encrypted connections to the database. enabled - all incoming connections to the Database must use SSL. disabled - no incoming connection to the Database should use SSL. replica_ssl - databases that replicate from this one need to use SSL.
    +
    + Enum: disabled, enabled, replica_ssl
    +     		false
    typeenum + The type of the database.
    +
    + Enum: redis, memcached
    +     		false
    upgradeSpecobject + Specifications for DB upgrade.
    +     		false
    + + +### spec.globalConfigurations.activeActive +[↩ Parent](#specglobalconfigurations) + +Connection/ association to the Active-Active database. + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    namestring + The the corresponding Active-Active database name, Redis Enterprise Active Active Database custom resource name, this Resource is associated with. In case this resource is created manually at the active active database creation this field must be filled via the user, otherwise, the operator will assign this field automatically. Note: this feature is currently unsupported.
    +     		true
    participatingClusterNamestring + The corresponding participating cluster name, Redis Enterprise Remote Cluster custom resource name, in the Active-Active database, In case this resource is created manually at the active active database creation this field must be filled via the user, otherwise, the operator will assign this field automatically. Note: this feature is currently unsupported.
    +     		true
    + + +### spec.globalConfigurations.alertSettings +[↩ Parent](#specglobalconfigurations) + +Settings for database alerts. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    bdb_backup_delayedobject + Periodic backup has been delayed for longer than specified threshold value [minutes]
    +     		false
    bdb_crdt_src_high_syncer_lagobject + Active-active source - sync lag is higher than specified threshold value [seconds]
    +     		false
    bdb_crdt_src_syncer_connection_errorobject + Active-active source - sync has connection error while trying to connect replica source
    +     		false
    bdb_crdt_src_syncer_general_errorobject + Active-active source - sync encountered in general error
    +     		false
    bdb_high_latencyobject + Latency is higher than specified threshold value [micro-sec]
    +     		false
    bdb_high_throughputobject + Throughput is higher than specified threshold value [requests / sec.]
    +     		false
    bdb_long_running_actionobject + An alert for state-machines that are running for too long
    +     		false
    bdb_low_throughputobject + Throughput is lower than specified threshold value [requests / sec.]
    +     		false
    bdb_proxy_cert_expiring_soonobject + Proxy certificate will expire in less than specified threshold value [days]
    +     		false
    bdb_ram_dataset_overheadobject + Dataset RAM overhead of a shard has reached the threshold value [% of its RAM limit]
    +     		false
    bdb_ram_valuesobject + Percent of values kept in a shard's RAM is lower than [% of its key count]
    +     		false
    bdb_replica_src_high_syncer_lagobject + Replica-of source - sync lag is higher than specified threshold value [seconds]
    +     		false
    bdb_replica_src_syncer_connection_errorobject + Replica-of source - sync has connection error while trying to connect replica source
    +     		false
    bdb_shard_num_ram_valuesobject + Number of values kept in a shard's RAM is lower than [values]
    +     		false
    bdb_sizeobject + Dataset size has reached the threshold value [% of the memory limit]
    +     		false
    + + +### spec.globalConfigurations.alertSettings.bdb_backup_delayed +[↩ Parent](#specglobalconfigurationsalertsettings) + +Periodic backup has been delayed for longer than specified threshold value [minutes] + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean + Alert enabled or disabled
    +     		true
    thresholdstring + Threshold for alert going on/off
    +     		true
    + + +### spec.globalConfigurations.alertSettings.bdb_crdt_src_high_syncer_lag +[↩ Parent](#specglobalconfigurationsalertsettings) + +Active-active source - sync lag is higher than specified threshold value [seconds] + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean + Alert enabled or disabled
    +     		true
    thresholdstring + Threshold for alert going on/off
    +     		true
    + + +### spec.globalConfigurations.alertSettings.bdb_crdt_src_syncer_connection_error +[↩ Parent](#specglobalconfigurationsalertsettings) + +Active-active source - sync has connection error while trying to connect replica source + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean + Alert enabled or disabled
    +     		true
    thresholdstring + Threshold for alert going on/off
    +     		true
    + + +### spec.globalConfigurations.alertSettings.bdb_crdt_src_syncer_general_error +[↩ Parent](#specglobalconfigurationsalertsettings) + +Active-active source - sync encountered in general error + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean + Alert enabled or disabled
    +     		true
    thresholdstring + Threshold for alert going on/off
    +     		true
    + + +### spec.globalConfigurations.alertSettings.bdb_high_latency +[↩ Parent](#specglobalconfigurationsalertsettings) + +Latency is higher than specified threshold value [micro-sec] + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean + Alert enabled or disabled
    +     		true
    thresholdstring + Threshold for alert going on/off
    +     		true
    + + +### spec.globalConfigurations.alertSettings.bdb_high_throughput +[↩ Parent](#specglobalconfigurationsalertsettings) + +Throughput is higher than specified threshold value [requests / sec.] + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean + Alert enabled or disabled
    +     		true
    thresholdstring + Threshold for alert going on/off
    +     		true
    + + +### spec.globalConfigurations.alertSettings.bdb_long_running_action +[↩ Parent](#specglobalconfigurationsalertsettings) + +An alert for state-machines that are running for too long + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean + Alert enabled or disabled
    +     		true
    thresholdstring + Threshold for alert going on/off
    +     		true
    + + +### spec.globalConfigurations.alertSettings.bdb_low_throughput +[↩ Parent](#specglobalconfigurationsalertsettings) + +Throughput is lower than specified threshold value [requests / sec.] + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean + Alert enabled or disabled
    +     		true
    thresholdstring + Threshold for alert going on/off
    +     		true
    + + +### spec.globalConfigurations.alertSettings.bdb_proxy_cert_expiring_soon +[↩ Parent](#specglobalconfigurationsalertsettings) + +Proxy certificate will expire in less than specified threshold value [days] + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean + Alert enabled or disabled
    +     		true
    thresholdstring + Threshold for alert going on/off
    +     		true
    + + +### spec.globalConfigurations.alertSettings.bdb_ram_dataset_overhead +[↩ Parent](#specglobalconfigurationsalertsettings) + +Dataset RAM overhead of a shard has reached the threshold value [% of its RAM limit] + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean + Alert enabled or disabled
    +     		true
    thresholdstring + Threshold for alert going on/off
    +     		true
    + + +### spec.globalConfigurations.alertSettings.bdb_ram_values +[↩ Parent](#specglobalconfigurationsalertsettings) + +Percent of values kept in a shard's RAM is lower than [% of its key count] + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean + Alert enabled or disabled
    +     		true
    thresholdstring + Threshold for alert going on/off
    +     		true
    + + +### spec.globalConfigurations.alertSettings.bdb_replica_src_high_syncer_lag +[↩ Parent](#specglobalconfigurationsalertsettings) + +Replica-of source - sync lag is higher than specified threshold value [seconds] + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean + Alert enabled or disabled
    +     		true
    thresholdstring + Threshold for alert going on/off
    +     		true
    + + +### spec.globalConfigurations.alertSettings.bdb_replica_src_syncer_connection_error +[↩ Parent](#specglobalconfigurationsalertsettings) + +Replica-of source - sync has connection error while trying to connect replica source + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean + Alert enabled or disabled
    +     		true
    thresholdstring + Threshold for alert going on/off
    +     		true
    + + +### spec.globalConfigurations.alertSettings.bdb_shard_num_ram_values +[↩ Parent](#specglobalconfigurationsalertsettings) + +Number of values kept in a shard's RAM is lower than [values] + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean + Alert enabled or disabled
    +     		true
    thresholdstring + Threshold for alert going on/off
    +     		true
    + + +### spec.globalConfigurations.alertSettings.bdb_size +[↩ Parent](#specglobalconfigurationsalertsettings) + +Dataset size has reached the threshold value [% of the memory limit] + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean + Alert enabled or disabled
    +     		true
    thresholdstring + Threshold for alert going on/off
    +     		true
    + + +### spec.globalConfigurations.auditing +[↩ Parent](#specglobalconfigurations) + +Database auditing configuration. + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    dbConnsAuditingboolean + Deprecated: use auditMode instead. +Enables auditing of database connection and authentication events. +When enabled, connection, authentication, and disconnection events are tracked and sent +to the configured audit listener (configured at the cluster level). +The cluster-level auditing configuration must be set before enabling this on a database. +Setting dbConnsAuditing to true is equivalent to auditMode "connection". +If both dbConnsAuditing and auditMode are set, auditMode takes precedence.
    +     		false
    + + +### spec.globalConfigurations.backup +[↩ Parent](#specglobalconfigurations) + +Target for automatic database backups. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    absobject +
    +     		false
    ftpobject +
    +     		false
    gcsobject + GoogleStorage
    +     		false
    intervalinteger + Backup Interval in seconds. Specifies the time interval in seconds at which periodic backup is performed.
    +     		false
    intervalOffsetinteger + Backup Interval Offset in seconds. Specifies a time offset in seconds at which the periodic backup job starts. This can only be used if the backup interval is 24 hours (86400 seconds) or 12 hours (43200 seconds). The offset is relative to 00:00 UTC for 24-hour mode, and 00:00 + 12:00 UTC for 12-hour mode. If not specified, a random starting time (offset) is automatically chosen. The offset must be less than the backup interval.
    +     		false
    mountobject + MountPointStorage
    +     		false
    s3object +
    +     		false
    sftpobject +
    +     		false
    swiftobject +
    +     		false
    + + +### spec.globalConfigurations.backup.abs +[↩ Parent](#specglobalconfigurationsbackup) + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    absSecretNamestring + The name of the secret that holds ABS credentials. The secret must contain the keys "AccountName" and "AccountKey", and these must hold the corresponding credentials
    +     		true
    containerstring + Azure Blob Storage container name.
    +     		true
    subdirstring + Optional. Azure Blob Storage subdir under container.
    +     		false
    + + +### spec.globalConfigurations.backup.ftp +[↩ Parent](#specglobalconfigurationsbackup) + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    urlstring + a URI of the "ftps://[USER[:PASSWORD]@]HOST[:PORT]/PATH[/]" format
    +     		true
    + + +### spec.globalConfigurations.backup.gcs +[↩ Parent](#specglobalconfigurationsbackup) + +GoogleStorage + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    bucketNamestring + Google Storage bucket name.
    +     		true
    gcsSecretNamestring + The name of the secret that holds the Google Cloud Storage credentials. The secret must contain the keys "CLIENT_ID", "PRIVATE_KEY", "PRIVATE_KEY_ID", "CLIENT_EMAIL" and these must hold the corresponding credentials. The keys should correspond to the values in the key JSON.
    +     		true
    subdirstring + Optional. Google Storage subdir under bucket.
    +     		false
    + + +### spec.globalConfigurations.backup.mount +[↩ Parent](#specglobalconfigurationsbackup) + +MountPointStorage + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    pathstring + Path to the local mount point. You must create the mount point on all nodes, and the redislabs:redislabs user must have read and write permissions on the local mount point.
    +     		true
    + + +### spec.globalConfigurations.backup.s3 +[↩ Parent](#specglobalconfigurationsbackup) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    awsSecretNamestring + The name of the secret that holds the AWS credentials. The secret must contain the keys "AWS_ACCESS_KEY_ID" and "AWS_SECRET_ACCESS_KEY", and these must hold the corresponding credentials.
    +     		true
    bucketNamestring + Amazon S3 bucket name.
    +     		true
    regionNamestring + Optional. Amazon S3 region name. If not specified, the region is auto-detected using a HEAD request to the bucket. For AWS GovCloud or other regions where auto-detection may not work, specify the region explicitly (e.g., "us-gov-east-1").
    +     		false
    subdirstring + Optional. Amazon S3 subdir under bucket.
    +     		false
    + + +### spec.globalConfigurations.backup.sftp +[↩ Parent](#specglobalconfigurationsbackup) + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    sftpSecretNamestring + The name of the secret that holds SFTP credentials. The secret must contain the "Key" key, which is the SSH private key for connecting to the sftp server.
    +     		true
    sftp_urlstring + SFTP url
    +     		true
    + + +### spec.globalConfigurations.backup.swift +[↩ Parent](#specglobalconfigurationsbackup) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    auth_urlstring + Swift service authentication URL.
    +     		true
    containerstring + Swift object store container for storing the backup files.
    +     		true
    swiftSecretNamestring + The name of the secret that holds Swift credentials. The secret must contain the keys "Key" and "User", and these must hold the corresponding credentials: service access key and service user name (pattern for the latter does not allow special characters &,<,>,")
    +     		true
    prefixstring + Optional. Prefix (path) of backup files in the swift container.
    +     		false
    + + +### spec.globalConfigurations.connectionSettings +[↩ Parent](#specglobalconfigurations) + +Connection-related settings such as proxy connections and scheduling policy. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    connectionLimitTypeenum + Connections limit type.
    +
    + Enum: per-thread, per-shard
    +     		false
    connectionSchedulingPolicyenum + Controls how server-side connections are used when forwarding traffic to shards. Values: cmp: Closest to max_pipelined policy. Pick the connection with the most pipelined commands that has not reached the max_pipelined limit. mru: Try to use most recently used connections. spread: Try to use all connections. mnp: Minimal pipeline policy. Pick the connection with the least pipelined commands.
    +
    + Enum: cmp, mru, spread, mnp
    +     		false
    globalMaxDedicatedConnectionsinteger + Defines the maximum number of dedicated server connections for a given database. The total number across all workers. The default is 0 for unlimited. Defaults to 0, which means unlimited.
    +     		false
    internalConnectionsinteger + The number of internal proxy connections. Defaults to 5 if unspecified.
    +     		false
    minDedicatedConnectionsinteger + Number of dedicated server connections the DMC has per worker per shard. Defaults to 2 if unspecified.
    +     		false
    + + +### spec.globalConfigurations.modulesList[] +[↩ Parent](#specglobalconfigurations) + +Redis Enterprise module (see https://redis.io/docs/latest/develop/reference/modules/) + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    namestring + The name of the module, e.g. "search" or "ReJSON". The complete list of modules available in the cluster can be retrieved from the '.status.modules' field in the REC.
    +     		true
    configstring + Module command line arguments e.g. VKEY_MAX_ENTITY_COUNT 30 30
    +     		false
    versionstring + The semantic version of the module, e.g. '1.6.12'. Optional for REDB, must be set for REAADB. Note that this field is deprecated, and will be removed in future releases.
    +     		false
    + + +### spec.globalConfigurations.ossClusterSettings +[↩ Parent](#specglobalconfigurations) + +Additional OSS cluster settings that may be provided to tweak OSS cluster behavior. + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enableExternalAccessboolean + Toggles whether this database supports external access in OSS cluster mode. When enabled, advertised database topology includes the external endpoints for the Redis Enterprise nodes hosting the database shards. The external access mechanism (e.g., LoadBalancer services) is configured via the ossClusterSettings.externalAccessType field of the RedisEnterpriseCluster. When external access is enabled, the corresponding database secret will have the list of primary shard IPs in the oss_startup_nodes field. Currently enabling OSS cluster API in conjunction with external access is not supported for active active databases.
    +     		false
    + + +### spec.globalConfigurations.redisEnterpriseCluster +[↩ Parent](#specglobalconfigurations) + +Connection to Redis Enterprise Cluster + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    namestring + The name of the Redis Enterprise Cluster where the database should be stored.
    +     		true
    + + +### spec.globalConfigurations.replicaSources[] +[↩ Parent](#specglobalconfigurations) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    replicaSourceNamestring + The name of the resource from which the source database URI is derived. The type of resource must match the type specified in the ReplicaSourceType field.
    +     		true
    replicaSourceTypestring + The type of resource from which the source database URI is derived. If set to 'SECRET', the source database URI is derived from the secret named in the ReplicaSourceName field. The secret must have a key named 'uri' that defines the URI of the source database in the form of 'redis://...'. The type of secret (kubernetes, vault, ...) is determined by the secret mechanism used by the underlying REC object. If set to 'REDB', the source database URI is derived from the RedisEnterpriseDatabase resource named in the ReplicaSourceName field.
    +     		true
    clientKeySecretstring + Secret that defines the client certificate and key used by the syncer in the target database cluster. The secret must have 2 keys in its map: "cert" which is the PEM encoded certificate, and "key" which is the PEM encoded private key.
    +     		false
    compressioninteger + GZIP compression level (0-6) to use for replication.
    +     		false
    serverCertSecretstring + Secret that defines the server certificate used by the proxy in the source database cluster. The secret must have 1 key in its map: "cert" which is the PEM encoded certificate.
    +     		false
    tlsSniNamestring + TLS SNI name to use for the replication link.
    +     		false
    + + +### spec.globalConfigurations.rolesPermissions[] +[↩ Parent](#specglobalconfigurations) + +Redis Enterprise Role and ACL Binding + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    aclstring + Acl Name of RolePermissionType (note: use exact name of the ACL from the Redis Enterprise ACL list, case sensitive)
    +     		true
    rolestring + Role Name of RolePermissionType (note: use exact name of the role from the Redis Enterprise role list, case sensitive)
    +     		true
    typeenum + Type of Redis Enterprise Database Role Permission. Currently, only "redis-enterprise" is supported, which uses roles and ACLs defined within Redis Enterprise directly.
    +
    + Enum: redis-enterprise
    + Default: redis-enterprise
    +     		true
    + + +### spec.globalConfigurations.upgradeSpec +[↩ Parent](#specglobalconfigurations) + +Specifications for DB upgrade. + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    upgradeModulesToLatestboolean + Upgrades the modules to the latest version that supports the DB version during a DB upgrade action, to upgrade the DB version view the 'redisVersion' field. Note - This field is currently not supported for Active-Active databases.
    +     		true
    + + +### spec.redisEnterpriseCluster +[↩ Parent](#spec) + +Connection to the Redis Enterprise Cluster. + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    namestring + The name of the Redis Enterprise Cluster where the database should be stored.
    +     		true
    + + +### status +[↩ Parent](#) + +Defines the observed state of RedisEnterpriseActiveActiveDatabase. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    clusterCertificatesGenerationinteger + Certificate generation number from the participating cluster's REC.Status.CertificatesStatus.Generation. The operator monitors this field to detect when proxy or syncer certificates are updated on the local participating cluster. When the operator detects a change, it automatically runs a CRDB force update (equivalent to 'crdb-cli crdb update --force'), which synchronizes the certificate changes to all participating clusters and prevents sync issues. This eliminates the manual step of running crdb-cli commands when you rotate certificates in Active-Active deployments on Kubernetes.
    +
    + Format: int64
    +     		false
    guidstring + GUID of the Active-Active database.
    +     		false
    lastTaskUidstring + UID of the last Active-Active database task.
    +     		false
    linkedRedbs[]string + List of linked REDBs.
    +     		false
    managedTags[]string + List of user-declared tag keys (from spec.globalConfigurations.tags) that were last successfully reconciled to the Active-Active database's BDBs (default config and every participating-cluster instance). Used to detect tag deletions on the next reconcile: keys present here but absent from the current spec.globalConfigurations.tags are removed from all BDB tag sets. Operator-internal keys (managed_by, redb_resource, global_configurations_spec, etc.) are not tracked here.
    +     		false
    participatingClusters[]object + List of participating cluster statuses.
    +     		false
    redisEnterpriseClusterstring + Name of the Redis Enterprise Cluster that this resource is associated with.
    +     		false
    replicationStatusenum + Overall replication status.
    +
    + Enum: up, down
    +     		false
    secretsStatus[]object + Status of the secrets.
    +     		false
    specStatusstring + Indicates whether the desired specification is valid.
    +     		false
    statusstring + Status of the Active-Active database. This status doesn't include the replication link (data path) status. To view the replication link status, see the ReplicationStatus field or status.replicationStatus on the custom resource.
    +     		false
    + + +### status.participatingClusters[] +[↩ Parent](#status) + +The status of a participating cluster. + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    namestring + Name of the linked remote cluster custom resource.
    +     		true
    idinteger + ID of the instance in the Active-Active database.
    +
    + Format: int64
    +     		false
    replicationStatusenum + Replication status of the participating cluster.
    +
    + Enum: up, down
    +     		false
    + + +### status.secretsStatus[] +[↩ Parent](#status) + +The status of a secret. + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    namestring + Name of the secret.
    +     		true
    statusenum + Status of the secret.
    +
    + Enum: Valid, Invalid
    +     		false
    diff --git a/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md b/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md new file mode 100644 index 0000000000..9c9a1ee7f9 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md @@ -0,0 +1,4993 @@ +--- +title: RedisEnterpriseCluster API Reference +alwaysopen: false +categories: +- docs +- operate +- kubernetes +linkTitle: REC API +weight: 30 +url: '/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api/' +--- + +apiVersion: + + +- [app.redislabs.com/v1](#appredislabscomv1) + + + + +# app.redislabs.com/v1 + + + + +RedisEnterpriseCluster is the Schema for the redisenterpriseclusters API + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    apiVersionstringapp.redislabs.com/v1true
    kindstringRedisEnterpriseClustertrue
    metadataobjectRefer to the Kubernetes API documentation for the fields of the `metadata` field.true
    specobject + RedisEnterpriseClusterSpec defines the desired state of RedisEnterpriseCluster
    +     		false
    statusobject +
    +     		false
    + + +### spec +[↩ Parent](#) + +RedisEnterpriseClusterSpec defines the desired state of RedisEnterpriseCluster + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    accessControlobject + Access control configuration for the cluster.
    +     		false
    activeActiveobject + Specification for ActiveActive setup. At most one of ingressOrRouteSpec or activeActive fields can be set at the same time.
    +     		false
    antiAffinityAdditionalTopologyKeys[]string + Additional antiAffinity terms in order to support installation on different zones/vcenters
    +     		false
    auditingobject + Cluster-level configuration for auditing database connection and authentication events. Includes both the audit listener connection parameters and the default policy for new databases.
    +     		false
    backupobject + Cluster-wide backup configurations
    +     		false
    bootstrapperImageSpecobject + Specification for Bootstrapper container image
    +     		false
    bootstrapperResourcesobject + Compute resource requirements for bootstrapper containers
    +     		false
    certificatesobject + RS Cluster Certificates. Used to modify the certificates used by the cluster. See the "RSClusterCertificates" struct described above to see the supported certificates.
    +     		false
    clusterCredentialSecretNamestring + Name or path of the secret containing cluster credentials. Defaults to the cluster name if left blank. For Kubernetes secrets (default): Can be customized to any valid secret name, or left blank to use the cluster name. The secret can be pre-created with 'username' and 'password' fields, or otherwise it will be automatically created with a default username and auto-generated password. On running clusters, this field can be changed to point to a different existing secret. The new secret must exist, contain valid 'username' and 'password' fields, and the credentials must work with the RS cluster. For Vault secrets: Can be customized with the path of the secret within Vault. The secret must be pre-created in Vault.
    +     		false
    clusterCredentialSecretRolestring + Used only if ClusterCredentialSecretType is vault, to define vault role to be used. If blank, defaults to "redis-enterprise-operator"
    +     		false
    clusterCredentialSecretTypeenum + Type of Secret to use for ClusterCredential, Vault, Kuberetes,... If left blank, will default ot kubernetes secrets
    +
    + Enum: vault, kubernetes
    +     		false
    clusterRecoveryboolean + ClusterRecovery initiates cluster recovery when set to true. Note that this field is cleared automatically after the cluster is recovered
    +     		false
    containerTimezoneobject + Container timezone configuration. While the default timezone on all containers is UTC, this setting can be used to set the timezone on services rigger/bootstrapper/RS containers. You can either propagate the hosts timezone to RS pods or set it manually via timezoneName.
    +     		false
    createServiceAccountboolean + Creates a service account for Redis Enterprise.
    +     		false
    dataInternodeEncryptionboolean + Internode encryption (INE) cluster wide policy. An optional boolean setting. Specifies if INE should be on/off for new created REDBs. May be overridden for specific REDB via similar setting, please view the similar setting for REDB for more info.
    +     		false
    dmcExternalCBAAuthenticationboolean + Whether Certificate-Based Authentication (CBA) for client connections to databases is delegated to an external gRPC authentication service, instead of being handled locally inside DMC (the data-path management proxy). If omitted, this field is evaluated as false. Possible values: true/false
    +     		false
    encryptPkeysboolean + Private key encryption Possible values: true/false
    +     		false
    enforceIPv4boolean + Sets ENFORCE_IPV4 environment variable
    +     		false
    extraEnvVars[]object + ADVANCED USAGE: use carefully. Add environment variables to RS StatefulSet's containers.
    +     		false
    extraLabelsmap[string]string + Labels that the user defines for their convenience
    +     		false
    hostAliases[]object + Adds hostAliases entries to the Redis Enterprise pods
    +     		false
    ingressOrRouteSpecobject + Access configurations for the Redis Enterprise cluster and databases. At most one of ingressOrRouteSpec or activeActive fields can be set at the same time.
    +     		false
    ldapobject + Cluster-level LDAP configuration, such as server addresses, protocol, authentication and query settings.
    +     		false
    licensestring + Redis Enterprise License
    +     		false
    licenseSecretNamestring + K8s secret or Vault Secret Name/Path to use for Cluster License. When left blank, the license is read from the "license" field. Note that you can't specify non-empty values in both "license" and "licenseSecretName", only one of these fields can be used to pass the license string. The license needs to be stored under the key "license".
    +     		false
    metricsConfigobject + Cluster-level configuration for metrics. Currently controls which database tag keys may be exposed as labels on Redis Software metrics.
    +     		false
    nodeSelectormap[string]string + Node selector for scheduling pods on specific nodes. This applies to all pods managed by the operator: Redis Enterprise nodes, Services Rigger, and Call Home Client.
    +     		false
    nodesinteger + Number of Redis Enterprise nodes (pods)
    +
    + Format: int32
    +     		false
    ocspConfigurationobject + An API object that represents the cluster's OCSP configuration. To enable OCSP, the cluster's proxy certificate should contain the OCSP responder URL.
    +     		false
    ossClusterSettingsobject + Cluster-level configuration for OSS cluster mode databases.
    +     		false
    persistentSpecobject + Persistent storage configuration for Redis Enterprise cluster.
    +     		false
    podAnnotationsmap[string]string + annotations for the service rigger and redis enterprise pods
    +     		false
    podAntiAffinityobject + Override for the default anti-affinity rules of the Redis Enterprise pods. More info: https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#an-example-of-a-pod-that-uses-pod-affinity
    +     		false
    podStartingPolicyobject + Mitigation setting for STS pods stuck in "ContainerCreating"
    +     		false
    podTolerations[]object + Tolerations that are added to all managed pods. More information: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/
    +     		false
    priorityClassNamestring + Adds the priority class to pods managed by the operator
    +     		false
    pullSecrets[]object + PullSecrets is an optional list of references to secrets in the same namespace to use for pulling any of the images. If specified, these secrets will be passed to individual puller implementations for them to use. More info: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
    +     		false
    rackAwarenessNodeLabelstring + Node label that specifies rack ID - if specified, will create rack aware cluster. Rack awareness requires node label must exist on all nodes. Additionally, operator needs a special cluster role with permission to list nodes.
    +     		false
    redisEnterpriseAdditionalPodSpecAttributesobject + ADVANCED USAGE USE AT YOUR OWN RISK - +specify pod attributes that are required for the statefulset - Redis Enterprise pods. +Pod attributes managed by the operator might override these settings. +Also make sure the attributes are supported by the K8s version running on the cluster - +the operator does not validate that.
    +     		false
    redisEnterpriseIPFamilyenum + Reserved, future use, only for use if instructed by Redis. IPFamily dictates what IP family to choose for pods' internal and external communication.
    +
    + Enum: IPv4, IPv6
    +     		false
    redisEnterpriseImageSpecobject + Specification for Redis Enterprise container image
    +     		false
    redisEnterpriseNodeResourcesobject + Compute resource requirements for Redis Enterprise containers
    +     		false
    redisEnterprisePodAnnotationsmap[string]string + annotations for redis enterprise pod
    +     		false
    redisEnterpriseServicesConfigurationobject + RS Cluster optional services settings
    +     		false
    redisEnterpriseServicesRiggerImageSpecobject + Specification for Services Rigger container image
    +     		false
    redisEnterpriseServicesRiggerResourcesobject + Compute resource requirements for Services Rigger pod
    +     		false
    redisEnterpriseTerminationGracePeriodSecondsinteger + The TerminationGracePeriodSeconds value for the (STS created) REC pods
    +
    + Format: int64
    +     		false
    redisEnterpriseVolumeMounts[]object + additional volume mounts within the redis enterprise containers. More info: https://kubernetes.io/docs/concepts/storage/volumes/
    +     		false
    redisOnFlashSpecobject + Auto Tiering (Redis on Flash) configuration. When provided, the cluster can create Auto Tiering databases.
    +     		false
    redisUpgradePolicyenum + Redis upgrade policy to be set on the Redis Enterprise cluster. Possible values: major/latest This value is used by the cluster to choose the Redis version of the database when an upgrade is performed. The Redis Enterprise cluster includes multiple versions of OSS Redis that can be used for databases.
    +
    + Enum: major, latest
    +     		false
    resp3Defaultboolean + Whether databases will turn on RESP3 compatibility upon database upgrade. Note - Deleting this property after explicitly setting its value shall have no effect. Please view the corresponding field in RS doc for more info.
    +     		false
    securityContextobject + The security configuration that will be applied to RS pods.
    +     		false
    serviceAccountNamestring + Name of the service account to use for Redis Enterprise.
    +     		false
    servicesobject + Customization options for operator-managed service resources created for Redis Enterprise clusters and databases
    +     		false
    servicesRiggerSpecobject + Specification for service rigger
    +     		false
    sideContainersSpec[]object +
    +     		false
    slaveHAobject + Slave high availability mechanism configuration.
    +     		false
    ssoobject + Cluster-level SSO configuration for authentication to the Cluster Manager UI.
    +     		false
    uiAnnotationsmap[string]string + Additional annotations for the Redis Enterprise UI service. These annotations override overlapping global annotations set under spec.services.servicesAnnotations. The specified annotations will not override annotations that already exist and didn't originate from the operator, except for the 'redis.io/last-keys' annotation which is reserved.
    +     		false
    uiServiceTypeenum + Service type for exposing the Redis Enterprise UI (https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types).
    +
    + Enum: ClusterIP, NodePort, LoadBalancer, ExternalName
    +     		false
    upgradeSpecobject + Redis Enterprise upgrade configuration
    +     		false
    usageMeterobject + The configuration of the usage meter.
    +     		false
    usageReportFieldsenum + Controls which fields are included in the usage (call-home) report. "all" also includes node count and hostname; "restricted" sends the reduced report. Default is "restricted". Should not be changed unless instructed by Redis support.
    +
    + Enum: all, restricted
    +     		false
    userDefinedModules[]object + List of user-defined modules to be downloaded and installed during cluster bootstrap The modules on the list will be downloaded on cluster creation, upgrade, scale-out and recovery and installed on all nodes. Note that changing this field for a running cluster will trigger a rolling update.
    +     		false
    usernamestring + Username for the admin user of Redis Enterprise
    +     		false
    vaultCASecretstring + K8s secret name containing Vault's CA cert - defaults to "vault-ca-cert"
    +     		false
    volumes[]object + additional volumes
    +     		false
    + + +### spec.accessControl +[↩ Parent](#spec) + +Access control configuration for the cluster. + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    policyobject + Policy settings for access control behavior.
    +     		false
    + + +### spec.accessControl.policy +[↩ Parent](#specaccesscontrol) + +Policy settings for access control behavior. + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    allowREDBRolesPermissionsboolean + Controls whether the deprecated REDB spec.rolesPermissions field is reconciled. When enabled (default): both spec.rolesPermissions and RedisEnterpriseRole objects are reconciled. When disabled: only RedisEnterpriseRole objects are reconciled, spec.rolesPermissions is ignored.
    +     		false
    + + +### spec.activeActive +[↩ Parent](#spec) + +Specification for ActiveActive setup. At most one of ingressOrRouteSpec or activeActive fields can be set at the same time. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    apiIngressUrlstring + RS API URL
    +     		true
    dbIngressSuffixstring + DB ENDPOINT SUFFIX - will be used to set the db host. ingress Creates a host name so it should be unique if more than one db is created on the cluster with the same name
    +     		true
    methodenum + Used to distinguish between different platforms implementation
    +
    + Enum: openShiftRoute, ingress
    +     		true
    ingressAnnotationsmap[string]string + Used for ingress controllers such as ha-proxy or nginx in GKE
    +     		false
    + + +### spec.auditing +[↩ Parent](#spec) + +Cluster-level configuration for auditing database connection and authentication events. Includes both the audit listener connection parameters and the default policy for new databases. + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    configobject + Configuration for the audit listener connection
    +     		false
    dbConnsAuditingboolean + Cluster-wide default policy for database connection auditing. When set to true, connection auditing will be enabled by default for all new databases. Existing databases are not affected and can override this setting individually.
    +     		false
    + + +### spec.auditing.config +[↩ Parent](#specauditing) + +Configuration for the audit listener connection + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    auditAddressstring + TCP/IP address or file path where audit notifications will be sent. For TCP protocol: IP address of the audit listener. For local protocol: file path for audit output (development/testing only).
    +     		true
    auditProtocolenum + Protocol used to send audit notifications. Valid values: "TCP" or "local". For production systems, use "TCP". "local" is for development/testing only.
    +
    + Enum: TCP, local
    +     		true
    auditPortinteger + Port number where audit notifications will be sent (TCP protocol only).
    +     		false
    auditReconnectIntervalinteger + Interval in seconds between attempts to reconnect to the audit listener.
    +     		false
    auditReconnectMaxAttemptsinteger + Maximum number of attempts to reconnect to the audit listener. Set to 0 for infinite attempts.
    +     		false
    + + +### spec.backup +[↩ Parent](#spec) + +Cluster-wide backup configurations + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    s3object + Configurations for backups to s3 and s3-compatible storage
    +     		false
    + + +### spec.backup.s3 +[↩ Parent](#specbackup) + +Configurations for backups to s3 and s3-compatible storage + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    caCertificateSecretNamestring + Secret name that holds the S3 CA certificate, which contains the TLS certificate mapped to the key in the secret 'cert'
    +     		false
    urlstring + Specifies the URL for S3 export and import
    +     		false
    + + +### spec.bootstrapperImageSpec +[↩ Parent](#spec) + +Specification for Bootstrapper container image + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    digestHashstring + The digest hash of the container image to pull. When specified, the container image is pulled according to the digest hash instead of the image tag. The versionTag field must also be specified with the image tag matching this digest hash. Note: This field is only supported for OLM deployments.
    +     		false
    imagePullPolicystring + The image pull policy to be applied to the container image. One of Always, Never, IfNotPresent.
    +     		false
    repositorystring + The repository (name) of the container image to be deployed.
    +     		false
    versionTagstring + The tag of the container image to be deployed.
    +     		false
    + + +### spec.bootstrapperResources +[↩ Parent](#spec) + +Compute resource requirements for bootstrapper containers + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    claims[]object +
    +     		false
    limitsmap[string]int or string + Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/
    +     		false
    requestsmap[string]int or string + Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/
    +     		false
    + + +### spec.bootstrapperResources.claims[] +[↩ Parent](#specbootstrapperresources) + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    namestring +
    +     		true
    + + +### spec.certificates +[↩ Parent](#spec) + +RS Cluster Certificates. Used to modify the certificates used by the cluster. See the "RSClusterCertificates" struct described above to see the supported certificates. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    apiCertificateSecretNamestring + Secret name to use for cluster's API certificate. The secret must have the following keys - A key named 'certificate'/'cert'/'tls.crt' with the value of the certificate in PEM format. A key named 'key'/'tls.key' with the value of the private key. Optionally, a key named 'ca.crt', containing the public certificate of the root CA. If present, the root CA certificate is appended to the certificate provided in the 'tls.crt' (or equivalent) key, to form a full certificate chain. Otherwise, the certificate in 'tls.crt' must include a full certificate chain inline. This key is typically populated by the cert-manager when it has access to the root certificate. Otherwise, it could be added manually. If left blank, a cluster-provided certificate will be used.
    +     		false
    cmCertificateSecretNamestring + Secret name to use for cluster's CM (Cluster Manager) certificate. The secret must have the following keys - A key named 'certificate'/'cert'/'tls.crt' with the value of the certificate in PEM format. A key named 'key'/'tls.key' with the value of the private key. Optionally, a key named 'ca.crt', containing the public certificate of the root CA. If present, the root CA certificate is appended to the certificate provided in the 'tls.crt' (or equivalent) key, to form a full certificate chain. Otherwise, the certificate in 'tls.crt' must include a full certificate chain inline. This key is typically populated by the cert-manager when it has access to the root certificate. Otherwise, it could be added manually. If left blank, a cluster-provided certificate will be used.
    +     		false
    cpInternodeEncryptionCertificateSecretNamestring + Secret name to use for cluster's Control Plane Internode Encryption (CPINE) certificate. The secret must have the following keys - A key named 'certificate'/'cert'/'tls.crt' with the value of the certificate in PEM format. A key named 'key'/'tls.key' with the value of the private key. Optionally, a key named 'ca.crt', containing the public certificate of the root CA. If present, the root CA certificate is appended to the certificate provided in the 'tls.crt' (or equivalent) key, to form a full certificate chain. Otherwise, the certificate in 'tls.crt' must include a full certificate chain inline. This key is typically populated by the cert-manager when it has access to the root certificate. Otherwise, it could be added manually. If left blank, a cluster-provided certificate will be used.
    +     		false
    dpInternodeEncryptionCertificateSecretNamestring + Secret name to use for cluster's Data Plane Internode Encryption (DPINE) certificate. The secret must have the following keys - A key named 'certificate'/'cert'/'tls.crt' with the value of the certificate in PEM format. A key named 'key'/'tls.key' with the value of the private key. Optionally, a key named 'ca.crt', containing the public certificate of the root CA. If present, the root CA certificate is appended to the certificate provided in the 'tls.crt' (or equivalent) key, to form a full certificate chain. Otherwise, the certificate in 'tls.crt' must include a full certificate chain inline. This key is typically populated by the cert-manager when it has access to the root certificate. Otherwise, it could be added manually. If left blank, a cluster-provided certificate will be used.
    +     		false
    ldapClientCertificateSecretNamestring + Secret name to use for cluster's LDAP client certificate. The secret must have the following keys - A key named 'certificate'/'cert'/'tls.crt' with the value of the certificate in PEM format. A key named 'key'/'tls.key' with the value of the private key. Optionally, a key named 'ca.crt', containing the public certificate of the root CA. If present, the root CA certificate is appended to the certificate provided in the 'tls.crt' (or equivalent) key, to form a full certificate chain. Otherwise, the certificate in 'tls.crt' must include a full certificate chain inline. This key is typically populated by the cert-manager when it has access to the root certificate. Otherwise, it could be added manually. If left blank, LDAP client certificate authentication will be disabled.
    +     		false
    metricsExporterCertificateSecretNamestring + Secret name to use for cluster's Metrics Exporter certificate. The secret must have the following keys - A key named 'certificate'/'cert'/'tls.crt' with the value of the certificate in PEM format. A key named 'key'/'tls.key' with the value of the private key. Optionally, a key named 'ca.crt', containing the public certificate of the root CA. If present, the root CA certificate is appended to the certificate provided in the 'tls.crt' (or equivalent) key, to form a full certificate chain. Otherwise, the certificate in 'tls.crt' must include a full certificate chain inline. This key is typically populated by the cert-manager when it has access to the root certificate. Otherwise, it could be added manually. If left blank, a cluster-provided certificate will be used.
    +     		false
    proxyCertificateSecretNamestring + Secret name to use for cluster's Proxy certificate. The secret must have the following keys - A key named 'certificate'/'cert'/'tls.crt' with the value of the certificate in PEM format. A key named 'key'/'tls.key' with the value of the private key. Optionally, a key named 'ca.crt', containing the public certificate of the root CA. If present, the root CA certificate is appended to the certificate provided in the 'tls.crt' (or equivalent) key, to form a full certificate chain. Otherwise, the certificate in 'tls.crt' must include a full certificate chain inline. This key is typically populated by the cert-manager when it has access to the root certificate. Otherwise, it could be added manually. If left blank, a cluster-provided certificate will be used. Note - For Active-Active databases (REAADB), certificate updates are automatically reconciled. When you update this secret, the operator detects the change and automatically executes a CRDB force update (equivalent to 'crdb-cli crdb update --force'), which synchronizes the certificate changes to all participating clusters, eliminating the need for manual intervention.
    +     		false
    ssoIssuerCertificateSecretNamestring + Secret name to use for the SSO Identity Provider (IdP) certificate. This is the public certificate from your SAML Identity Provider used to verify SAML assertions. The secret must contain a single field named 'certificate'/'cert'/'tls.crt' (no 'key' field needed for IdP cert). This certificate must be configured as part of the SSO setup, before SSO can be enabled for the cluster. Note - While IdP metadata XML may contain the certificate, Redis Enterprise Server does not use it from there, so the certificate must be provided separately via this secret.
    +     		false
    ssoServiceCertificateSecretNamestring + Secret name to use for the SSO Service Provider (SP) certificate. This certificate is used by the cluster to sign SAML requests and encrypt SAML responses, and it must be configured as part of the SSO setup, before SSO can be enabled for the cluster. The secret must have the following keys - A key named 'certificate'/'cert'/'tls.crt' with the value of the certificate in PEM format. A key named 'key'/'tls.key' with the value of the private key. Optionally, a key named 'ca.crt', containing the public certificate of the root CA. If present, the root CA certificate is appended to the certificate provided in the 'tls.crt' (or equivalent) key, to form a full certificate chain. Otherwise, the certificate in 'tls.crt' must include a full certificate chain inline. This key is typically populated by the cert-manager when it has access to the root certificate. Otherwise, it could be added manually. If left blank, SSO will not be configured.
    +     		false
    syncerCertificateSecretNamestring + Secret name to use for cluster's Syncer certificate. The secret must have the following keys - A key named 'certificate'/'cert'/'tls.crt' with the value of the certificate in PEM format. A key named 'key'/'tls.key' with the value of the private key. Optionally, a key named 'ca.crt', containing the public certificate of the root CA. If present, the root CA certificate is appended to the certificate provided in the 'tls.crt' (or equivalent) key, to form a full certificate chain. Otherwise, the certificate in 'tls.crt' must include a full certificate chain inline. This key is typically populated by the cert-manager when it has access to the root certificate. Otherwise, it could be added manually. If left blank, a cluster-provided certificate will be used. Note - For Active-Active databases (REAADB), certificate updates are automatically reconciled. When you update this secret, the operator detects the change and automatically executes a CRDB force update (equivalent to 'crdb-cli crdb update --force'), which synchronizes the certificate changes to all participating clusters, eliminating the need for manual intervention.
    +     		false
    + + +### spec.containerTimezone +[↩ Parent](#spec) + +Container timezone configuration. While the default timezone on all containers is UTC, this setting can be used to set the timezone on services rigger/bootstrapper/RS containers. You can either propagate the hosts timezone to RS pods or set it manually via timezoneName. + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    propagateHostobject + Identifies that container timezone should be in sync with the host, this option mounts a hostPath volume onto RS pods that could be restricted in some systems.
    +     		false
    timezoneNamestring + POSIX-style timezone name as a string to be passed as EnvVar to RE pods, e.g. "Europe/London".
    +     		false
    + + +### spec.extraEnvVars[] +[↩ Parent](#spec) + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    namestring +
    +     		true
    valuestring +
    +     		false
    valueFromobject +
    +     		false
    + + +### spec.hostAliases[] +[↩ Parent](#spec) + +HostAlias holds the mapping between IP and hostnames that will be injected as an entry in the pod's hosts file. + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    hostnames[]string + Hostnames for the above IP address.
    +     		false
    ipstring + IP address of the host file entry.
    +     		false
    + + +### spec.ingressOrRouteSpec +[↩ Parent](#spec) + +Access configurations for the Redis Enterprise cluster and databases. At most one of ingressOrRouteSpec or activeActive fields can be set at the same time. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    apiFqdnUrlstring + RS API URL
    +     		true
    dbFqdnSuffixstring + DB ENDPOINT SUFFIX - will be used to set the db host ingress . Creates a host name so it should be unique if more than one db is created on the cluster with the same name
    +     		true
    methodenum + Used to distinguish between different platforms implementation.
    +
    + Enum: openShiftRoute, ingress, istio
    +     		true
    ingressAnnotationsmap[string]string + Additional annotations to set on ingress resources created by the operator
    +     		false
    + + +### spec.ldap +[↩ Parent](#spec) + +Cluster-level LDAP configuration, such as server addresses, protocol, authentication and query settings. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    authenticationQueryobject + Configuration of authentication queries, mapping between the username, provided to the cluster for authentication, and the LDAP Distinguished Name.
    +     		true
    authorizationQueryobject + Configuration of authorization queries, mapping between a user's Distinguished Name and its group memberships.
    +     		true
    protocolenum + Specifies the LDAP protocol to use. One of: LDAP, LDAPS, STARTTLS.
    +
    + Enum: LDAP, LDAPS, STARTTLS
    +     		true
    servers[]object + One or more LDAP servers. If multiple servers are specified, they must all share an identical organization tree structure.
    +     		true
    bindCredentialsSecretNamestring + Name of a secret within the same namespace, holding the credentials used to communicate with the LDAP server for authentication queries. The secret must have a key named 'dn' with the Distinguished Name of the user to execute the query, and 'password' with its password. If left blank, credentials-based authentication is disabled.
    +     		false
    caCertificateSecretNamestring + Name of a secret within the same namespace, holding a PEM-encoded CA certificate for validating the TLS connection to the LDAP server. The secret must have a key named 'cert' with the certificate data. This field is applicable only when the protocol is LDAPS or STARTTLS.
    +     		false
    cacheTTLSecondsinteger + The maximum TTL of cached entries.
    +     		false
    cbaboolean + Whether to allow LDAP as an identity source for certificate-based authentication. Disabled by default.
    +     		false
    cbaIdentityOidstring + The certificate subject OID to use when CBA identity source is set to SubjectOID.
    +     		false
    cbaIdentitySourceenum + The certificate subject identity source to use for LDAP lookup. Applicable only when CBA is enabled. One of SubjectCN, SubjectOID.
    +
    + Enum: SubjectCN, SubjectOID
    +     		false
    directoryTimeoutSecondsinteger + The connection timeout to the LDAP server when authenticating a user, in seconds
    +     		false
    enabledForControlPlaneboolean + Whether to enable LDAP for control plane access. Disabled by default.
    +     		false
    enabledForDataPlaneboolean + Whether to enable LDAP for data plane access. Disabled by default.
    +     		false
    + + +### spec.ldap.authenticationQuery +[↩ Parent](#specldap) + +Configuration of authentication queries, mapping between the username, provided to the cluster for authentication, and the LDAP Distinguished Name. + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    queryobject + Configuration for a search query. Mutually exclusive with the 'template' field. The substring '%u' in the query filter will be replaced with the username.
    +     		false
    templatestring + Configuration for a template query. Mutually exclusive with the 'query' field. The substring '%u' will be replaced with the username, e.g., 'cn=%u,ou=dev,dc=example,dc=com'.
    +     		false
    + + +### spec.ldap.authenticationQuery.query +[↩ Parent](#specldapauthenticationquery) + +Configuration for a search query. Mutually exclusive with the 'template' field. The substring '%u' in the query filter will be replaced with the username. + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    basestring + The Distinguished Name of the entry at which to start the search, e.g., 'ou=dev,dc=example,dc=com'.
    +     		true
    filterstring + An RFC-4515 string representation of the filter to apply in the search. For an authentication query, the substring '%u' will be replaced with the username, e.g., '(cn=%u)'. For an authorization query, the substring '%D' will be replaced with the user's Distinguished Name, e.g., '(members=%D)'.
    +     		true
    scopeenum + The search scope for an LDAP query. One of: BaseObject, SingleLevel, WholeSubtree
    +
    + Enum: BaseObject, SingleLevel, WholeSubtree
    +     		true
    + + +### spec.ldap.authorizationQuery +[↩ Parent](#specldap) + +Configuration of authorization queries, mapping between a user's Distinguished Name and its group memberships. + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    attributestring + Configuration for an attribute query. Mutually exclusive with the 'query' field. Holds the name of an attribute of the LDAP user entity that contains a list of the groups that the user belongs to, e.g., 'memberOf'.
    +     		false
    queryobject + Configuration for a search query. Mutually exclusive with the 'attribute' field. The substring '%D' in the query filter will be replaced with the user's Distinguished Name.
    +     		false
    + + +### spec.ldap.authorizationQuery.query +[↩ Parent](#specldapauthorizationquery) + +Configuration for a search query. Mutually exclusive with the 'attribute' field. The substring '%D' in the query filter will be replaced with the user's Distinguished Name. + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    basestring + The Distinguished Name of the entry at which to start the search, e.g., 'ou=dev,dc=example,dc=com'.
    +     		true
    filterstring + An RFC-4515 string representation of the filter to apply in the search. For an authentication query, the substring '%u' will be replaced with the username, e.g., '(cn=%u)'. For an authorization query, the substring '%D' will be replaced with the user's Distinguished Name, e.g., '(members=%D)'.
    +     		true
    scopeenum + The search scope for an LDAP query. One of: BaseObject, SingleLevel, WholeSubtree
    +
    + Enum: BaseObject, SingleLevel, WholeSubtree
    +     		true
    + + +### spec.ldap.servers[] +[↩ Parent](#specldap) + +Address of an LDAP server. + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    hoststring + Host name of the LDAP server
    +     		true
    portinteger + Port number of the LDAP server. If unspecified, defaults to 389 for LDAP and STARTTLS protocols, and 636 for LDAPS protocol.
    +
    + Format: int32
    +     		false
    + + +### spec.metricsConfig +[↩ Parent](#spec) + +Cluster-level configuration for metrics. Currently controls which database tag keys may be exposed as labels on Redis Software metrics. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    exposeDatabaseTagsboolean + Master switch for tag-derived metrics. When true, database tag keys listed in metricsTagKeysExposed may be used as labels by metrics consumers. When false, no database tag keys are exposed as metric labels, even if listed in metricsTagKeysExposed. Defaults to false if unset.
    +     		false
    keyDistributionEnabledboolean + When true, emit Prometheus classic histogram metrics for key size/items distributions. Defaults to false if unset.
    +     		false
    keyItemsBucketsstring + Comma-separated bucket boundaries for key items distribution histograms (e.g., "1M,8M"). Defaults to empty string if unset.
    +     		false
    keySizeBucketsstring + Comma-separated bucket boundaries for key size distribution histograms (e.g., "128M,512M"). Defaults to empty string if unset.
    +     		false
    localStorageMaxSizeMbinteger + Maximum size in MB for local metrics storage. Defaults to 1024 if unset.
    +     		false
    localStorageRetentionDaysinteger + Number of days to retain local metrics data. Defaults to 8 if unset.
    +     		false
    maxRequestsInFlightinteger + Maximum number of concurrent master_aggregator scrape requests. Set to 0 to disable the limit. Defaults to 2 if unset.
    +     		false
    metricsTagKeysExposed[]string + Allowlist of database tag keys that may be exposed as metric labels when exposeDatabaseTags is true. Keys not listed here are never exposed as metric labels even if they appear on a database. Keys are lowercased on the cluster, must be 1-64 characters and not whitespace-only, and may contain ASCII letters, digits, spaces, '-', '_', '.', '+', '@', and ':'. At most 50 keys are allowed. Defaults to empty list if unset.
    +     		false
    + + +### spec.ocspConfiguration +[↩ Parent](#spec) + +An API object that represents the cluster's OCSP configuration. To enable OCSP, the cluster's proxy certificate should contain the OCSP responder URL. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    ocspFunctionalityboolean + Whether to enable/disable OCSP mechanism for the cluster.
    +     		false
    queryFrequencyinteger + Determines the interval (in seconds) in which the control plane will poll the OCSP responder for a new status for the server certificate. Minimum value is 60. Maximum value is 86400.
    +     		false
    recoveryFrequencyinteger + Determines the interval (in seconds) in which the control plane will poll the OCSP responder for a new status for the server certificate when the current staple is invalid. Minimum value is 60. Maximum value is 86400.
    +     		false
    recoveryMaxTriesinteger + Determines the maximum number for the OCSP recovery attempts. After max number of tries passed, the control plane will revert back to the regular frequency. Minimum value is 1. Maximum value is 100.
    +     		false
    responseTimeoutinteger + Determines the time interval (in seconds) for which the request waits for a response from the OCSP responder. Minimum value is 1. Maximum value is 60.
    +     		false
    + + +### spec.ossClusterSettings +[↩ Parent](#spec) + +Cluster-level configuration for OSS cluster mode databases. + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    externalAccessTypeenum + Specifies the mechanism for enabling external access to OSS cluster databases. When unset or set to "Disabled", external access is not allowed for any OSS cluster databases. When set to a specific mechanism (e.g., "LoadBalancer"), that mechanism is used to provide external access. Note: Individual databases must still enable external access via their ossClusterSettings.enableExternalAccess field.
    +
    + Enum: LoadBalancer, Disabled
    +     		false
    loadBalancerobject + Configuration for LoadBalancer services created to assign public IPs for Redis Enterprise cluster nodes.
    +     		false
    podCIDRs[]string + A list of Kubernetes pod CIDR ranges from which pod IPs are allocated. Supports both IPv4 (e.g., "10.30.0.0/16") and IPv6 addresses. This field should only be configured when OSS cluster databases need to be accessed from both internal and external clients. When configured, internal communication can reach pods directly using their pod IPs, bypassing the external access mechanism (e.g., load balancer services) for improved performance. IMPORTANT: For this feature to work correctly, the entire data path must preserve the client source IP address. This is required because the Redis server uses the client's source IP to construct the CLUSTER SHARDS/SLOTS response - returning pod IPs for internal clients (matching podCIDRs) or load balancer addresses for external clients. On cloud platforms, this typically requires configuring the load balancer to preserve source IPs.
    +     		false
    + + +### spec.ossClusterSettings.loadBalancer +[↩ Parent](#specossclustersettings) + +Configuration for LoadBalancer services created to assign public IPs for Redis Enterprise cluster nodes. + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    externalTrafficPolicyenum + ExternalTrafficPolicy specifies the externalTrafficPolicy for LoadBalancer services created for Redis Enterprise cluster nodes. Choose "Local" to configure the LoadBalancer to only route traffic to the single worker node hosting the Redis Enterprise cluster node for that service. Choose "Cluster" to route traffic to any worker node, providing a more stable behavior during failovers, but with increased overhead due to additional hop. Defaults to "Local" when podCIDRs is configured, and "Cluster" otherwise.
    +
    + Enum: Cluster, Local
    +     		false
    serviceAnnotationsmap[string]string + Additional annotations to set on LoadBalancer services created for Redis Enterprise cluster nodes. These annotations are merged with global service annotations from spec.services.servicesAnnotations.
    +     		false
    + + +### spec.persistentSpec +[↩ Parent](#spec) + +Persistent storage configuration for Redis Enterprise cluster. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enablePersistentVolumeResizeboolean + Whether to enable PersistentVolumes resize. Disabled by default. Read the instruction in pvc_expansion readme carefully before using this feature.
    +     		false
    enabledboolean + Whether to add persistent volume to Redis Enterprise pods
    +     		false
    storageClassNamestring + Storage class for persistent volume in Redis Enterprise pods. Leave empty to use the default. If using the default this way, make sure the Kubernetes Cluster has a default Storage Class configured. This can be done by running a `kubectl get storageclass` and see if one of the Storage Classes' names contains a `(default)` mark.
    +     		false
    volumeSizeint or string + By default, if you omit spec.persistentSpec.volumeSize, the operator allocates a persistent volume that is five times (5x) the Redis Enterprise node memory request defined in spec.redisEnterpriseNodeResources.requests.memory (per node). This 5x ratio is the recommended minimum capacity. To enable resizing after creating the cluster, see the Expand PVC docs: https://redis.io/docs/latest/operate/kubernetes/re-clusters/expand-pvc/
    +     		false
    + + +### spec.podAntiAffinity +[↩ Parent](#spec) + +Override for the default anti-affinity rules of the Redis Enterprise pods. More info: https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#an-example-of-a-pod-that-uses-pod-affinity + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    preferredDuringSchedulingIgnoredDuringExecution[]object +
    +     		false
    requiredDuringSchedulingIgnoredDuringExecution[]object +
    +     		false
    + + +### spec.podStartingPolicy +[↩ Parent](#spec) + +Mitigation setting for STS pods stuck in "ContainerCreating" + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean + Enables detection and mitigation of pod startup issues.
    +     		true
    startingThresholdSecondsinteger + Time in seconds to wait before taking action on a pod stuck during startup. Set to 0 to disable.
    +
    + Format: int32
    +     		true
    + + +### spec.podTolerations[] +[↩ Parent](#spec) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    effectstring +
    +     		false
    keystring +
    +     		false
    operatorstring +
    +     		false
    tolerationSecondsinteger +
    +
    + Format: int64
    +     		false
    valuestring +
    +     		false
    + + +### spec.pullSecrets[] +[↩ Parent](#spec) + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    namestring + Secret name
    +     		false
    + + +### spec.redisEnterpriseAdditionalPodSpecAttributes +[↩ Parent](#spec) + +ADVANCED USAGE USE AT YOUR OWN RISK - +specify pod attributes that are required for the statefulset - Redis Enterprise pods. +Pod attributes managed by the operator might override these settings. +Also make sure the attributes are supported by the K8s version running on the cluster - +the operator does not validate that. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    activeDeadlineSecondsinteger +
    +
    + Format: int64
    +     		false
    affinityobject +
    +     		false
    automountServiceAccountTokenboolean +
    +     		false
    dnsConfigobject +
    +     		false
    dnsPolicystring +
    +     		false
    enableServiceLinksboolean +
    +     		false
    ephemeralContainers[]object +
    +     		false
    hostAliases[]object +
    +     		false
    hostIPCboolean +
    +     		false
    hostNetworkboolean +
    +     		false
    hostPIDboolean +
    +     		false
    hostUsersboolean +
    +     		false
    hostnamestring +
    +     		false
    hostnameOverridestring +
    +     		false
    imagePullSecrets[]object +
    +     		false
    initContainers[]object +
    +     		false
    nodeNamestring +
    +     		false
    nodeSelectormap[string]string +
    +     		false
    osobject +
    +     		false
    overheadmap[string]int or string +
    +     		false
    preemptionPolicystring +
    +     		false
    priorityinteger +
    +
    + Format: int32
    +     		false
    priorityClassNamestring +
    +     		false
    readinessGates[]object +
    +     		false
    resourceClaims[]object +
    +     		false
    resourcesobject +
    +     		false
    restartPolicystring +
    +     		false
    runtimeClassNamestring +
    +     		false
    schedulerNamestring +
    +     		false
    schedulingGates[]object +
    +     		false
    securityContextobject +
    +     		false
    serviceAccountstring +
    +     		false
    serviceAccountNamestring +
    +     		false
    setHostnameAsFQDNboolean +
    +     		false
    shareProcessNamespaceboolean +
    +     		false
    subdomainstring +
    +     		false
    terminationGracePeriodSecondsinteger +
    +
    + Format: int64
    +     		false
    tolerations[]object +
    +     		false
    topologySpreadConstraints[]object +
    +     		false
    volumes[]object +
    +     		false
    workloadRefobject +
    +     		false
    + + +### spec.redisEnterpriseImageSpec +[↩ Parent](#spec) + +Specification for Redis Enterprise container image + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    digestHashstring + The digest hash of the container image to pull. When specified, the container image is pulled according to the digest hash instead of the image tag. The versionTag field must also be specified with the image tag matching this digest hash. Note: This field is only supported for OLM deployments.
    +     		false
    imagePullPolicystring + The image pull policy to be applied to the container image. One of Always, Never, IfNotPresent.
    +     		false
    repositorystring + The repository (name) of the container image to be deployed.
    +     		false
    versionTagstring + The tag of the container image to be deployed.
    +     		false
    + + +### spec.redisEnterpriseNodeResources +[↩ Parent](#spec) + +Compute resource requirements for Redis Enterprise containers + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    claims[]object +
    +     		false
    limitsmap[string]int or string + Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/
    +     		false
    requestsmap[string]int or string + Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/
    +     		false
    + + +### spec.redisEnterpriseNodeResources.claims[] +[↩ Parent](#specredisenterprisenoderesources) + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    namestring +
    +     		true
    + + +### spec.redisEnterpriseServicesConfiguration +[↩ Parent](#spec) + +RS Cluster optional services settings + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    cmServerobject +
    +     		false
    crdbCoordinatorobject +
    +     		false
    crdbWorkerobject +
    +     		false
    mdnsServerobject +
    +     		false
    pdnsServerobject +
    +     		false
    saslauthdobject +
    +     		false
    statsArchiverobject +
    +     		false
    + + +### spec.redisEnterpriseServicesConfiguration.cmServer +[↩ Parent](#specredisenterpriseservicesconfiguration) + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    operatingModeenum + Whether to enable/disable the CM server
    +
    + Enum: enabled, disabled
    +     		true
    + + +### spec.redisEnterpriseServicesConfiguration.crdbCoordinator +[↩ Parent](#specredisenterpriseservicesconfiguration) + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    operatingModeenum + Whether to enable/disable the crdb coordinator process
    +
    + Enum: enabled, disabled
    +     		true
    + + +### spec.redisEnterpriseServicesConfiguration.crdbWorker +[↩ Parent](#specredisenterpriseservicesconfiguration) + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    operatingModeenum + Whether to enable/disable the crdb worker processes
    +
    + Enum: enabled, disabled
    +     		true
    + + +### spec.redisEnterpriseServicesConfiguration.mdnsServer +[↩ Parent](#specredisenterpriseservicesconfiguration) + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    operatingModeenum + Whether to enable/disable the Multicast DNS server
    +
    + Enum: enabled, disabled
    +     		true
    + + +### spec.redisEnterpriseServicesConfiguration.pdnsServer +[↩ Parent](#specredisenterpriseservicesconfiguration) + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    operatingModeenum + Deprecated: The PDNS Server is now disabled by the operator. This field will be ignored.
    +
    + Enum: enabled, disabled
    +     		true
    + + +### spec.redisEnterpriseServicesConfiguration.saslauthd +[↩ Parent](#specredisenterpriseservicesconfiguration) + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    operatingModeenum + Whether to enable/disable the saslauthd service
    +
    + Enum: enabled, disabled
    +     		true
    + + +### spec.redisEnterpriseServicesConfiguration.statsArchiver +[↩ Parent](#specredisenterpriseservicesconfiguration) + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    operatingModeenum + Whether to enable/disable the stats archiver service
    +
    + Enum: enabled, disabled
    +     		true
    + + +### spec.redisEnterpriseServicesRiggerImageSpec +[↩ Parent](#spec) + +Specification for Services Rigger container image + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    digestHashstring + The digest hash of the container image to pull. When specified, the container image is pulled according to the digest hash instead of the image tag. The versionTag field must also be specified with the image tag matching this digest hash. Note: This field is only supported for OLM deployments.
    +     		false
    imagePullPolicystring + The image pull policy to be applied to the container image. One of Always, Never, IfNotPresent.
    +     		false
    repositorystring + The repository (name) of the container image to be deployed.
    +     		false
    versionTagstring + The tag of the container image to be deployed.
    +     		false
    + + +### spec.redisEnterpriseServicesRiggerResources +[↩ Parent](#spec) + +Compute resource requirements for Services Rigger pod + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    claims[]object +
    +     		false
    limitsmap[string]int or string + Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/
    +     		false
    requestsmap[string]int or string + Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/
    +     		false
    + + +### spec.redisEnterpriseVolumeMounts[] +[↩ Parent](#spec) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    mountPathstring +
    +     		true
    namestring +
    +     		true
    mountPropagationstring +
    +     		false
    readOnlyboolean +
    +     		false
    subPathstring +
    +     		false
    subPathExprstring +
    +     		false
    + + +### spec.redisOnFlashSpec +[↩ Parent](#spec) + +Auto Tiering (Redis on Flash) configuration. When provided, the cluster can create Auto Tiering databases. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean +
    +     		true
    storageClassNamestring +
    +     		true
    bigStoreDriverenum +
    +
    + Enum: rocksdb, speedb
    +     		false
    flashDiskSizeint or string +
    +     		false
    flashStorageEngineenum +
    +
    + Enum: rocksdb
    +     		false
    + + +### spec.securityContext +[↩ Parent](#spec) + +The security configuration that will be applied to RS pods. + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    fipsobject + FIPS-140 compliance mode settings.
    +     		false
    readOnlyRootFilesystemPolicyobject + Policy controlling whether to enable read-only root filesystem for the Redis Enterprise software containers. Note that certain filesystem paths remain writable through mounted volumes to ensure proper functionality.
    +     		false
    resourceLimitsobject + Settings pertaining to resource limits management by the Redis Enterprise node container.
    +     		false
    + + +### spec.securityContext.fips +[↩ Parent](#specsecuritycontext) + +FIPS-140 compliance mode settings. + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean + Toggles whether FIPS-140 compliance mode is enabled or not. Defaults to false.
    +     		false
    + + +### spec.securityContext.readOnlyRootFilesystemPolicy +[↩ Parent](#specsecuritycontext) + +Policy controlling whether to enable read-only root filesystem for the Redis Enterprise software containers. Note that certain filesystem paths remain writable through mounted volumes to ensure proper functionality. + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean + Whether to enable read-only root filesystem for the Redis Enterprise software containers. Default is false.
    +     		true
    + + +### spec.securityContext.resourceLimits +[↩ Parent](#specsecuritycontext) + +Settings pertaining to resource limits management by the Redis Enterprise node container. + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    allowAutoAdjustmentboolean + Allow Redis Enterprise to adjust resource limits, like max open file descriptors, of its data plane processes. When this option is enabled, the SYS_RESOURCE capability is added to the Redis Enterprise pods, and their allowPrivilegeEscalation field is set. Turned off by default.
    +     		false
    + + +### spec.services +[↩ Parent](#spec) + +Customization options for operator-managed service resources created for Redis Enterprise clusters and databases + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    apiServiceobject + Customization options for the REC API service.
    +     		false
    servicesAnnotationsmap[string]string + Global additional annotations to set on service resources created by the operator. The specified annotations will not override annotations that already exist and didn't originate from the operator.
    +     		false
    + + +### spec.services.apiService +[↩ Parent](#specservices) + +Customization options for the REC API service. + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    typeenum + Type of service to create for the REC API service. Defaults to ClusterIP service, if not specified otherwise.
    +
    + Enum: ClusterIP, NodePort, LoadBalancer
    +     		false
    + + +### spec.servicesRiggerSpec +[↩ Parent](#spec) + +Specification for service rigger + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    databaseServicePortPolicyenum + databaseServicePortPolicy instructs how to determine the service ports for REDB services. Defaults to DatabasePortForward, if not specified otherwise. Note - Regardless whether this flag is set or not, if an REDB/REAADB configured with databaseServicePort that would be the port exposed by the Service. DatabasePortForward - The service port will be the same as the database port. RedisDefaultPort - The service port will be the default Redis port (6379).
    +
    + Enum: DatabasePortForward, RedisDefaultPort
    +     		false
    databaseServiceTypestring + Service types for access to databases. should be a comma separated list. The possible values are cluster_ip, headless and load_balancer.
    +     		false
    extraEnvVars[]object +
    +     		false
    podAnnotationsmap[string]string + annotations for the service rigger pod
    +     		false
    serviceNamingenum + Used to determine how to name the services created automatically when a database is created. When bdb_name is used, the database name will be also used for the service name. When redis-port is used, the service will be named redis-.
    +
    + Enum: bdb_name, redis-port
    +     		false
    servicesRiggerAdditionalPodSpecAttributesobject + ADVANCED USAGE USE AT YOUR OWN RISK - specify pod attributes that are required for the rigger deployment pod. +Pod attributes managed by the operator might override these settings (Containers, serviceAccountName, +ImagePullSecrets, nodeSelector, PriorityClassName, PodSecurityContext). +podTolerations are merged with tolerations defined here. +Also make sure the attributes are supported by the K8s version running on the cluster - +the operator does not validate that.
    +     		false
    + + +### spec.servicesRiggerSpec.extraEnvVars[] +[↩ Parent](#specservicesriggerspec) + +EnvVar represents an environment variable present in a Container. More info: https://kubernetes.io/docs/tasks/inject-data-application/environment-variable-expose-pod-information/ + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    namestring + Name of the environment variable.
    +     		true
    valuestring +
    +     		false
    valueFromobject + Source for the environment variable's value. Cannot be used if value is not empty.
    +     		false
    + + +### spec.servicesRiggerSpec.extraEnvVars[].valueFrom +[↩ Parent](#specservicesriggerspecextraenvvars) + +Source for the environment variable's value. Cannot be used if value is not empty. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    configMapKeyRefobject + Selects a key of a ConfigMap.
    +     		false
    fieldRefobject + Selects a field of the pod
    +     		false
    resourceFieldRefobject + Selects a resource of the container: only resources limits and requests are currently supported.
    +     		false
    secretKeyRefobject + Selects a key of a secret in the pod's namespace
    +     		false
    + + +### spec.servicesRiggerSpec.extraEnvVars[].valueFrom.configMapKeyRef +[↩ Parent](#specservicesriggerspecextraenvvarsvaluefrom) + +Selects a key of a ConfigMap. + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    keystring + The key to select.
    +     		true
    namestring + Name of the referent
    +     		false
    optionalboolean + Specify whether the ConfigMap or its key must be defined
    +     		false
    + + +### spec.servicesRiggerSpec.extraEnvVars[].valueFrom.fieldRef +[↩ Parent](#specservicesriggerspecextraenvvarsvaluefrom) + +Selects a field of the pod + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    fieldPathstring + Path of the field to select in the specified API version.
    +     		true
    apiVersionstring + Version of the schema the FieldPath is written in terms of, defaults to "v1".
    +     		false
    + + +### spec.servicesRiggerSpec.extraEnvVars[].valueFrom.resourceFieldRef +[↩ Parent](#specservicesriggerspecextraenvvarsvaluefrom) + +Selects a resource of the container: only resources limits and requests are currently supported. + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    resourcestring + Required: resource to select
    +     		true
    containerNamestring + Container name: required for volumes, optional for env vars
    +     		false
    divisorint or string + Specifies the output format of the exposed resources, defaults to "1"
    +     		false
    + + +### spec.servicesRiggerSpec.extraEnvVars[].valueFrom.secretKeyRef +[↩ Parent](#specservicesriggerspecextraenvvarsvaluefrom) + +Selects a key of a secret in the pod's namespace + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    keystring + The key of the secret to select from. Must be a valid secret key.
    +     		true
    namestring + Name of the referent
    +     		false
    optionalboolean + Specify whether the Secret or its key must be defined
    +     		false
    + + +### spec.servicesRiggerSpec.servicesRiggerAdditionalPodSpecAttributes +[↩ Parent](#specservicesriggerspec) + +ADVANCED USAGE USE AT YOUR OWN RISK - specify pod attributes that are required for the rigger deployment pod. +Pod attributes managed by the operator might override these settings (Containers, serviceAccountName, +ImagePullSecrets, nodeSelector, PriorityClassName, PodSecurityContext). +podTolerations are merged with tolerations defined here. +Also make sure the attributes are supported by the K8s version running on the cluster - +the operator does not validate that. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    activeDeadlineSecondsinteger +
    +
    + Format: int64
    +     		false
    affinityobject +
    +     		false
    automountServiceAccountTokenboolean +
    +     		false
    dnsConfigobject +
    +     		false
    dnsPolicystring +
    +     		false
    enableServiceLinksboolean +
    +     		false
    ephemeralContainers[]object +
    +     		false
    hostAliases[]object +
    +     		false
    hostIPCboolean +
    +     		false
    hostNetworkboolean +
    +     		false
    hostPIDboolean +
    +     		false
    hostUsersboolean +
    +     		false
    hostnamestring +
    +     		false
    hostnameOverridestring +
    +     		false
    imagePullSecrets[]object +
    +     		false
    initContainers[]object +
    +     		false
    nodeNamestring +
    +     		false
    nodeSelectormap[string]string +
    +     		false
    osobject +
    +     		false
    overheadmap[string]int or string +
    +     		false
    preemptionPolicystring +
    +     		false
    priorityinteger +
    +
    + Format: int32
    +     		false
    priorityClassNamestring +
    +     		false
    readinessGates[]object +
    +     		false
    resourceClaims[]object +
    +     		false
    resourcesobject +
    +     		false
    restartPolicystring +
    +     		false
    runtimeClassNamestring +
    +     		false
    schedulerNamestring +
    +     		false
    schedulingGates[]object +
    +     		false
    securityContextobject +
    +     		false
    serviceAccountstring +
    +     		false
    serviceAccountNamestring +
    +     		false
    setHostnameAsFQDNboolean +
    +     		false
    shareProcessNamespaceboolean +
    +     		false
    subdomainstring +
    +     		false
    terminationGracePeriodSecondsinteger +
    +
    + Format: int64
    +     		false
    tolerations[]object +
    +     		false
    topologySpreadConstraints[]object +
    +     		false
    volumes[]object +
    +     		false
    workloadRefobject +
    +     		false
    + + +### spec.sideContainersSpec[] +[↩ Parent](#spec) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    namestring +
    +     		true
    args[]string +
    +     		false
    command[]string +
    +     		false
    env[]object +
    +     		false
    envFrom[]object +
    +     		false
    imagestring +
    +     		false
    imagePullPolicystring +
    +     		false
    lifecycleobject +
    +     		false
    livenessProbeobject +
    +     		false
    ports[]object +
    +     		false
    readinessProbeobject +
    +     		false
    resourcesobject +
    +     		false
    securityContextobject +
    +     		false
    startupProbeobject +
    +     		false
    stdinboolean +
    +     		false
    stdinOnceboolean +
    +     		false
    terminationMessagePathstring +
    +     		false
    terminationMessagePolicystring +
    +     		false
    ttyboolean +
    +     		false
    volumeDevices[]object +
    +     		false
    volumeMounts[]object +
    +     		false
    workingDirstring +
    +     		false
    + + +### spec.slaveHA +[↩ Parent](#spec) + +Slave high availability mechanism configuration. + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    slaveHAGracePeriodinteger + Grace period in seconds between node failure and when the high availability mechanism starts relocating shards. Set to 0 to not affect cluster configuration.
    +
    + Format: int32
    +     		true
    + + +### spec.sso +[↩ Parent](#spec) + +Cluster-level SSO configuration for authentication to the Cluster Manager UI. + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean + Enables SSO for Cluster Manager authentication. SSO requires the following configuration - Service Provider certificate (spec.certificates.ssoServiceCertificateSecretName), Identity Provider certificate (spec.certificates.ssoIssuerCertificateSecretName), IdP metadata or manual issuer configuration (spec.sso.saml.idpMetadataSecretName or spec.sso.saml.issuer), and Base address for Service Provider URLs (auto-determined from UI service or set via spec.sso.saml.serviceProvider.baseAddress).
    +     		true
    samlobject + SAML-based SSO configuration. Currently,SAML is the only supported SSO protocol.
    +     		true
    enforceSSOboolean + Enforces SSO-only authentication for the Cluster Manager. When true, local username/password authentication is disabled for non-admin users. When false (default), both SSO and local authentication are available.
    +     		false
    + + +### spec.sso.saml +[↩ Parent](#specsso) + +SAML-based SSO configuration. Currently,SAML is the only supported SSO protocol. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    idpMetadataSecretNamestring + Name of a secret in the same namespace that contains the Identity Provider (IdP) metadata XML. The secret must contain a key named 'idp_metadata' with the IdP metadata XML content. The XML can be plain text or base64-encoded; the operator handles encoding as needed. Obtain this metadata from your SAML Identity Provider (e.g., Okta or Azure AD). This is the recommended configuration method, as it's less error-prone. Either idpMetadataSecretName or issuer must be specified. If both are provided, idpMetadataSecretName takes precedence and issuer is ignored.
    +     		false
    issuerobject + Manual Identity Provider (IdP) configuration. Use this when IdP metadata XML is unavailable. Either idpMetadataSecretName or issuer must be specified. If both are provided, idpMetadataSecretName takes precedence and issuer is ignored.
    +     		false
    serviceProviderobject + Service Provider (SP) configuration.
    +     		false
    spMetadataSecretNamestring + Name of a secret where the operator stores the Service Provider (SP) metadata XML. The operator creates this secret with a key named 'sp_metadata' that contains the base64-encoded SP metadata XML. Upload this metadata to your Identity Provider. If not specified, defaults to "-sso-sp-metadata". If not specified, the Service Provider metadata isn't stored in a K8s secret, but can still be obtained directly from the cluster's UI and/or API. Note: This secret is only created when the cluster is configured to use Kubernetes secrets (spec.clusterCredentialSecretType is unset or set to "kubernetes"). When using Vault secrets, the operator does not create this secret. Users can obtain the SP metadata directly from the Redis Enterprise Server API endpoint: GET /v1/cluster/sso/saml/metadata/sp and store it in Vault themselves if needed.
    +     		false
    + + +### spec.sso.saml.issuer +[↩ Parent](#specssosaml) + +Manual Identity Provider (IdP) configuration. Use this when IdP metadata XML is unavailable. Either idpMetadataSecretName or issuer must be specified. If both are provided, idpMetadataSecretName takes precedence and issuer is ignored. + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    entityIDstring + Identity Provider entity ID (issuer identifier). Example: "urn:sso:example:idp" or "https://idp.example.com".
    +     		true
    loginURLstring + Identity Provider SSO login URL where SAML authentication requests are sent. Example: "https://idp.example.com/sso/saml".
    +     		true
    logoutURLstring + Identity Provider single logout URL where SAML logout requests are sent.
    +     		false
    + + +### spec.sso.saml.serviceProvider +[↩ Parent](#specssosaml) + +Service Provider (SP) configuration. + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    baseAddressstring + Base address used to construct Service Provider (SP) URLs, such as the ACS URL and SLO URL. Format: [://][:]. Examples: "https://redis-ui.example.com:9443" (recommended - explicit scheme), "redis-ui.example.com:9443" (defaults to https://), "http://redis-ui.example.com:9443" (NOT recommended for production). If the scheme is not specified, the operator automatically prepends "https://". WARNING: Using "http://" is NOT recommended for production environments as it transmits sensitive SAML assertions in plaintext. Only use "http://" for testing/development purposes. If set, this value is used to construct the SP URLs. If unset, the base address is automatically determined from the REC Cluster Manager UI service: - If the UI service type is LoadBalancer (configured via spec.uiServiceType), the load balancer address is used. - Otherwise, the cluster-internal DNS name is used (e.g., rec-ui.svc.cluster.local). - The port defaults to 8443 if not specified. Usage guidelines: - For LoadBalancer services: Leave this field blank to use the default REC UI service, or set it explicitly to the LoadBalancer address for custom services. - For Ingress: Set this to the ingress hostname and port (typically 443), e.g., "https://redis-ui.example.com:443".
    +     		false
    + + +### spec.upgradeSpec +[↩ Parent](#spec) + +Redis Enterprise upgrade configuration + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    autoUpgradeRedisEnterpriseboolean + Enables automatic Redis Enterprise upgrades when the operator is upgraded.
    +     		true
    + + +### spec.usageMeter +[↩ Parent](#spec) + +The configuration of the usage meter. + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    callHomeClientobject +
    +     		false
    + + +### spec.usageMeter.callHomeClient +[↩ Parent](#specusagemeter) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    cronExpressionstring + Cron expression for scheduling the call home CronJob (e.g., "0 */6 * * *"). If not specified, the CALL_HOME_CLIENT_CRON_SCHEDULE environment variable is used, or the default value of "0 23 * * *" (23:00 UTC daily). Changing defaults is not recommended.
    +     		false
    disabledboolean + Whether to disable the call home client. Enabled by default.
    +     		false
    imageSpecobject + Image specification
    +     		false
    intervalstring + Interval between call home reports (e.g., "1h", "30m"). Passed as --interval flag to the call home client binary. If not specified, the CALL_HOME_CLIENT_INTERVAL environment variable is used, or the default value of 24h. Changing defaults is not recommended.
    +     		false
    proxySecretNamestring + if needed, add proxy details in secret. the name of the proxy secret in the secret, can send the following keys: proxy-url, proxy-username, proxy-password (the url includes the proxy port).
    +     		false
    resourcesobject + Compute resource requirements for Call Home Client pod
    +     		false
    s3Targetobject + S3-compatible storage target for call home data upload. When enabled, call home data will be uploaded to this S3 target only. Before using this feature, please coordinate with Redis.
    +     		false
    + + +### spec.usageMeter.callHomeClient.imageSpec +[↩ Parent](#specusagemetercallhomeclient) + +Image specification + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    digestHashstring + The digest hash of the container image to pull. When specified, the container image is pulled according to the digest hash instead of the image tag. The versionTag field must also be specified with the image tag matching this digest hash. Note: This field is only supported for OLM deployments.
    +     		false
    imagePullPolicystring + The image pull policy to be applied to the container image. One of Always, Never, IfNotPresent.
    +     		false
    repositorystring + The repository (name) of the container image to be deployed.
    +     		false
    versionTagstring + The tag of the container image to be deployed.
    +     		false
    + + +### spec.usageMeter.callHomeClient.resources +[↩ Parent](#specusagemetercallhomeclient) + +Compute resource requirements for Call Home Client pod + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    claims[]object + Claims lists the names of resources, defined in spec.resourceClaims, that are used by this container. + This is an alpha field and requires enabling the DynamicResourceAllocation feature gate. + This field is immutable. It can only be set for containers.
    +     		false
    limitsmap[string]int or string + Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
    +     		false
    requestsmap[string]int or string + Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. Requests cannot exceed Limits. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
    +     		false
    + + +### spec.usageMeter.callHomeClient.resources.claims[] +[↩ Parent](#specusagemetercallhomeclientresources) + +ResourceClaim references one entry in PodSpec.ResourceClaims. + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    namestring + Name must match the name of one entry in pod.spec.resourceClaims of the Pod where this field is used. It makes that resource available inside a container.
    +     		true
    + + +### spec.usageMeter.callHomeClient.s3Target +[↩ Parent](#specusagemetercallhomeclient) + +S3-compatible storage target for call home data upload. When enabled, call home data will be uploaded to this S3 target only. Before using this feature, please coordinate with Redis. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    bucketstring + S3 bucket name. Required when S3Target is enabled.
    +     		false
    credentialsSecretNamestring + Name of the Kubernetes secret containing S3 credentials. The secret must contain keys "access-key" and "secret-key". Optional keys - "session-token" (for AWS STS), "ca-cert" (for custom CA). The credentials must have s3:PutObject permission on the target bucket.
    +     		false
    enabledboolean + Whether S3 upload is enabled. When true, call home data will be uploaded to the specified S3 target only.
    +     		false
    endpointstring + S3-compatible endpoint URL.
    +     		false
    prefixstring + S3 object key prefix/subfolder for uploaded files (e.g., "reports/2025"). If specified, files will be uploaded to s3://bucket/prefix/filename.
    +     		false
    regionstring + AWS region for the S3 bucket (e.g., "us-east-1").
    +     		false
    urlstring + Full S3 URL including bucket (e.g., "https://bucket.s3.region.amazonaws.com" or "s3://bucket/prefix").
    +     		false
    + + +### spec.userDefinedModules[] +[↩ Parent](#spec) + +UserDefinedModule represents a user-defined Redis module to be downloaded and installed during bootstrap + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    namestring + Name of the module
    +     		true
    sourceobject + Source location for downloading the module
    +     		true
    + + +### spec.userDefinedModules[].source +[↩ Parent](#specuserdefinedmodules) + +Source location for downloading the module + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    httpobject + HTTP source configuration for downloading the module via HTTP
    +     		false
    httpsobject + HTTPS source configuration for downloading the module via HTTPS
    +     		false
    + + +### spec.userDefinedModules[].source.http +[↩ Parent](#specuserdefinedmodulessource) + +HTTP source configuration for downloading the module via HTTP + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    urlstring + URL to download the module from (must use http:// scheme)
    +     		true
    credentialsSecretstring + Name of the Kubernetes secret containing credentials for downloading the module, if needed. The secret must contain 'username' and 'password' keys.
    +     		false
    + + +### spec.userDefinedModules[].source.https +[↩ Parent](#specuserdefinedmodulessource) + +HTTPS source configuration for downloading the module via HTTPS + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    urlstring + URL to download the module from (must use https:// scheme)
    +     		true
    credentialsSecretstring + Name of the Kubernetes secret containing credentials for downloading the module, if needed. The secret must contain 'username' and 'password' keys.
    +     		false
    + + +### spec.volumes[] +[↩ Parent](#spec) + +Volume represents a named volume in a pod that may be accessed by any container in the pod. More info: https://kubernetes.io/docs/concepts/storage/volumes + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    namestring +
    +     		true
    awsElasticBlockStoreobject +
    +     		false
    azureDiskobject +
    +     		false
    azureFileobject +
    +     		false
    cephfsobject +
    +     		false
    cinderobject +
    +     		false
    configMapobject +
    +     		false
    csiobject +
    +     		false
    downwardAPIobject +
    +     		false
    emptyDirobject +
    +     		false
    fcobject +
    +     		false
    flexVolumeobject +
    +     		false
    flockerobject +
    +     		false
    gcePersistentDiskobject +
    +     		false
    gitRepoobject +
    +     		false
    glusterfsobject +
    +     		false
    hostPathobject +
    +     		false
    iscsiobject +
    +     		false
    nfsobject +
    +     		false
    persistentVolumeClaimobject +
    +     		false
    photonPersistentDiskobject +
    +     		false
    portworxVolumeobject +
    +     		false
    projectedobject +
    +     		false
    quobyteobject +
    +     		false
    rbdobject +
    +     		false
    scaleIOobject +
    +     		false
    secretobject +
    +     		false
    storageosobject +
    +     		false
    vsphereVolumeobject +
    +     		false
    + + +### status +[↩ Parent](#) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    bundledDatabaseVersions[]object + Versions of open source databases bundled by Redis Enterprise Software - please note that in order to use a specific version it should be supported by the ‘upgradePolicy’ - ‘major’ or ‘latest’ according to the desired version (major/minor)
    +     		false
    certificatesStatusobject + Stores information about cluster certificates and their update process. In Active-Active databases, this is used to detect updates to the certificates, and trigger synchronization across the participating clusters.
    +     		false
    clusterCredentialSecretNamestring + The name of the secret containing cluster credentials currently in use by the cluster. This field tracks the current credential secret name and is updated when the secret name changes.
    +     		false
    ingressOrRouteMethodStatusstring + The ingressOrRouteSpec/ActiveActive spec method that exist
    +     		false
    licenseStatusobject +
    +     		false
    managedAPIsobject + Indicates cluster APIs that are being managed by the operator. This only applies to cluster APIs which are optionally-managed by the operator, such as cluster LDAP configuration. Most other APIs are automatically managed by the operator, and are not listed here.
    +     		false
    modules[]object +
    +     		false
    ocspStatusobject + An API object that represents the cluster's OCSP status
    +     		false
    persistenceStatusobject + The status of the Persistent Volume Claims that are used for Redis Enterprise cluster persistence. The status will correspond to the status of one or more of the PVCs (failed/resizing if one of them is in resize or failed to resize)
    +     		false
    redisEnterpriseIPFamilystring +
    +     		false
    specStatusstring +
    +     		false
    statestring +
    +     		false
    + + +### status.bundledDatabaseVersions[] +[↩ Parent](#status) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    dbTypestring +
    +     		true
    versionstring +
    +     		true
    featureSupportobject + Feature support flags for this database version
    +     		false
    majorboolean +
    +     		false
    + + +### status.bundledDatabaseVersions[].featureSupport +[↩ Parent](#statusbundleddatabaseversions) + +Feature support flags for this database version + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    activeActiveboolean + Indicates whether this version supports Active-Active (CRDB) databases
    +     		true
    flexboolean + Indicates whether this version supports Redis on Flash (Flex)
    +     		true
    + + +### status.certificatesStatus +[↩ Parent](#status) + +Stores information about cluster certificates and their update process. In Active-Active databases, this is used to detect updates to the certificates, and trigger synchronization across the participating clusters. + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    generationinteger + Generation stores the version of the cluster's Proxy and Syncer certificate secrets. This generation counter is automatically incremented when proxy or syncer certificates are updated. In Active-Active databases (REAADB), the operator monitors this field to detect certificate changes and automatically triggers a CRDB force update (equivalent to 'crdb-cli crdb update --force'), which synchronizes the certificate changes to all participating clusters, eliminating the need for manual intervention to maintain sync.
    +
    + Format: int64
    +     		false
    updateStatusstring + The status of the cluster's certificates update
    +     		false
    + + +### status.licenseStatus +[↩ Parent](#status) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    activationDatestring +
    +     		false
    expirationDatestring +
    +     		false
    features[]string +
    +     		false
    flashShardsinteger +
    +
    + Format: int32
    +     		false
    flashShardsLimitinteger +
    +
    + Format: int32
    +     		false
    licenseStatestring +
    +     		false
    ownerstring +
    +     		false
    ramShardsinteger +
    +
    + Format: int32
    +     		false
    ramShardsLimitinteger +
    +
    + Format: int32
    +     		false
    shardsLimitinteger +
    +     		false
    shardsUsagestring +
    +     		false
    + + +### status.managedAPIs +[↩ Parent](#status) + +Indicates cluster APIs that are being managed by the operator. This only applies to cluster APIs which are optionally-managed by the operator, such as cluster LDAP configuration. Most other APIs are automatically managed by the operator, and are not listed here. + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    ldapboolean + Indicate whether cluster LDAP configuration is managed by the operator. When this is enabled, the operator will reconcile the cluster LDAP configuration according to the '.spec.ldap' field in the RedisEnterpriseCluster resource.
    +     		false
    + + +### status.modules[] +[↩ Parent](#status) + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    displayNamestring +
    +     		false
    namestring +
    +     		false
    versions[]string +
    +     		false
    + + +### status.ocspStatus +[↩ Parent](#status) + +An API object that represents the cluster's OCSP status + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    certStatusstring + Indicates the proxy certificate status - GOOD/REVOKED/UNKNOWN.
    +     		false
    nextUpdatestring + The time at or before which newer information will be available about the status of the certificate (if available)
    +     		false
    producedAtstring + The time at which the OCSP responder signed this response.
    +     		false
    responderUrlstring + The OCSP responder url from which this status came from.
    +     		false
    revocationTimestring + The time at which the certificate was revoked or placed on hold.
    +     		false
    thisUpdatestring + The most recent time at which the status being indicated is known by the responder to have been correct.
    +     		false
    + + +### status.persistenceStatus +[↩ Parent](#status) + +The status of the Persistent Volume Claims that are used for Redis Enterprise cluster persistence. The status will correspond to the status of one or more of the PVCs (failed/resizing if one of them is in resize or failed to resize) + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    statusstring + The current status of the PVCs
    +     		false
    succeededstring + The number of PVCs that are provisioned with the expected size
    +     		false
    diff --git a/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_empty_fields b/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_empty_fields new file mode 100644 index 0000000000..bb7f604a61 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_empty_fields @@ -0,0 +1,8 @@ +spec.extraEnvVars +spec.podAntiAffinity +spec.redisEnterpriseAdditionalPodSpecAttributes +spec.redisEnterpriseServicesRiggerResources +spec.redisEnterpriseVolumeMounts +spec.servicesRiggerSpec.servicesRiggerAdditionalPodSpecAttributes +spec.sideContainersSpec +spec.volumes diff --git a/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_role_api.md b/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_role_api.md new file mode 100644 index 0000000000..781009271d --- /dev/null +++ b/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_role_api.md @@ -0,0 +1,179 @@ +--- +title: RedisEnterpriseClusterRole API Reference +alwaysopen: false +categories: +- docs +- operate +- kubernetes +linkTitle: RECROLE API +weight: 30 +url: '/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_role_api/' +--- + +apiVersion: + + +- [app.redislabs.com/v1alpha1](#appredislabscomv1alpha1) + + + + +# app.redislabs.com/v1alpha1 + + + + +RedisEnterpriseClusterRole is a cluster role definition. It grants users cluster-scoped access across the Redis Enterprise Cluster. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    apiVersionstringapp.redislabs.com/v1alpha1true
    kindstringRedisEnterpriseClusterRoletrue
    metadataobjectRefer to the Kubernetes API documentation for the fields of the `metadata` field.true
    specobject + RedisEnterpriseClusterRoleSpec defines the desired state of a RedisEnterpriseClusterRole.
    +     		false
    statusobject + RedisEnterpriseRoleStatus defines the observed state of a Redis Enterprise role object. This status type is used by both RedisEnterpriseClusterRole and RedisEnterpriseRole which share the same status structure.
    +     		false
    + + +### spec +[↩ Parent](#) + +RedisEnterpriseClusterRoleSpec defines the desired state of a RedisEnterpriseClusterRole. + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    aclobject + ACL defines the data access permissions granted by this cluster role. +Users with this role can access any database in the Redis Enterprise Cluster, +subject to the permissions granted by the referenced ACL. +References a RedisEnterpriseACL (kind RedisEnterpriseACL or empty) or an +internal Redis Enterprise ACL by name (kind 'redis_acl').
    +     		false
    managementRoleenum + ManagementRole is the named set of Redis Enterprise management permissions assigned to this cluster role. If omitted, the role uses the None management role.
    +
    + Enum: Admin, ClusterMember, ClusterViewer, DBMember, DBViewer, UserManager, None
    +     		false
    + + +### spec.acl +[↩ Parent](#spec) + +ACL defines the data access permissions granted by this cluster role. +Users with this role can access any database in the Redis Enterprise Cluster, +subject to the permissions granted by the referenced ACL. +References a RedisEnterpriseACL (kind RedisEnterpriseACL or empty) or an +internal Redis Enterprise ACL by name (kind 'redis_acl'). + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    namestring + Name of the referent.
    +     		true
    kindenum + Kind of the referent.
    +
    + Enum: , RedisEnterpriseACL, redis_acl
    +     		false
    + + +### status +[↩ Parent](#) + +RedisEnterpriseRoleStatus defines the observed state of a Redis Enterprise role object. This status type is used by both RedisEnterpriseClusterRole and RedisEnterpriseRole which share the same status structure. + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    observedGenerationinteger + The most recent generation of this resource that was observed and acted upon by the controller.
    +
    + Format: int64
    +     		false
    uidstring + The internal UID of the Role object defined in the Redis Enterprise Cluster.
    +     		false
    diff --git a/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_role_binding_api.md b/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_role_binding_api.md new file mode 100644 index 0000000000..62ac943835 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_role_binding_api.md @@ -0,0 +1,175 @@ +--- +title: RedisEnterpriseClusterRoleBinding API Reference +alwaysopen: false +categories: +- docs +- operate +- kubernetes +linkTitle: RECROLEBINDING API +weight: 30 +url: '/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_role_binding_api/' +--- + +apiVersion: + + +- [app.redislabs.com/v1alpha1](#appredislabscomv1alpha1) + + + + +# app.redislabs.com/v1alpha1 + + + + +RedisEnterpriseClusterRoleBinding binds users to a RedisEnterpriseClusterRole. It grants the users the cluster access defined by that role. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    apiVersionstringapp.redislabs.com/v1alpha1true
    kindstringRedisEnterpriseClusterRoleBindingtrue
    metadataobjectRefer to the Kubernetes API documentation for the fields of the `metadata` field.true
    specobject + RedisEnterpriseRoleBindingSpec defines the desired state of a RedisEnterpriseClusterRoleBinding or RedisEnterpriseRoleBinding.
    +     		false
    statusobject + RedisEnterpriseRoleBindingStatus defines the observed state of a RedisEnterpriseClusterRoleBinding or RedisEnterpriseRoleBinding.
    +     		false
    + + +### spec +[↩ Parent](#) + +RedisEnterpriseRoleBindingSpec defines the desired state of a RedisEnterpriseClusterRoleBinding or RedisEnterpriseRoleBinding. + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    roleRefobject + RoleRef holds a reference to the role object being bound. For RedisEnterpriseClusterRoleBinding, this references a RedisEnterpriseClusterRole in the same namespace (kind RedisEnterpriseClusterRole or empty) or an internal Redis Enterprise role by name (kind 'role'). For RedisEnterpriseRoleBinding, this references a RedisEnterpriseRole in the same namespace (kind RedisEnterpriseRole or empty) or an internal Redis Enterprise role by name (kind 'role').
    +     		true
    subjects[]object + Subjects holds references to the objects the role applies to. Each subject is a reference to a RedisEnterpriseUser object within the same namespace. The 'kind' field can be left empty, or set to RedisEnterpriseUser. A maximum of 100 subjects may be specified; to bind more, create additional bindings referencing the same role.
    +
    + Validations:
  • self.all(s1, self.exists_one(s2, + (has(s1.kind) && s1.kind != '' ? s1.kind : 'RedisEnterpriseUser') == + (has(s2.kind) && s2.kind != '' ? s2.kind : 'RedisEnterpriseUser') && + s1.name == s2.name)) +: subjects must be unique by kind and name
    • +     		false
    + + +### spec.roleRef +[↩ Parent](#spec) + +RoleRef holds a reference to the role object being bound. For RedisEnterpriseClusterRoleBinding, this references a RedisEnterpriseClusterRole in the same namespace (kind RedisEnterpriseClusterRole or empty) or an internal Redis Enterprise role by name (kind 'role'). For RedisEnterpriseRoleBinding, this references a RedisEnterpriseRole in the same namespace (kind RedisEnterpriseRole or empty) or an internal Redis Enterprise role by name (kind 'role'). + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    namestring + Name of the referent.
    +     		true
    kindenum + Kind of the referent.
    +
    + Enum: , RedisEnterpriseClusterRole, role
    +     		false
    + + +### spec.subjects[] +[↩ Parent](#spec) + +A reference to another object. + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    namestring + Name of the referent.
    +     		true
    kindenum + Kind of the referent.
    +
    + Enum: , RedisEnterpriseUser
    +     		false
    diff --git a/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md b/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md new file mode 100644 index 0000000000..df316bc55f --- /dev/null +++ b/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md @@ -0,0 +1,2118 @@ +--- +title: RedisEnterpriseDatabase API Reference +alwaysopen: false +categories: +- docs +- operate +- kubernetes +linkTitle: REDB API +weight: 30 +url: '/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api/' +--- + +apiVersion: + + +- [app.redislabs.com/v1alpha1](#appredislabscomv1alpha1) + + + + +# app.redislabs.com/v1alpha1 + + + + +RedisEnterpriseDatabase is the Schema for the redisenterprisedatabases API + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    apiVersionstringapp.redislabs.com/v1alpha1true
    kindstringRedisEnterpriseDatabasetrue
    metadataobjectRefer to the Kubernetes API documentation for the fields of the `metadata` field.true
    specobject + RedisEnterpriseDatabaseSpec defines the desired state of RedisEnterpriseDatabase
    +     		false
    statusobject + RedisEnterpriseDatabaseStatus defines the observed state of RedisEnterpriseDatabase
    +     		false
    + + +### spec +[↩ Parent](#) + +RedisEnterpriseDatabaseSpec defines the desired state of RedisEnterpriseDatabase + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    activeActiveobject + Connection/ association to the Active-Active database.
    +     		false
    alertSettingsobject + Settings for database alerts
    +     		false
    auditingobject + Database auditing configuration.
    +     		false
    backupobject + Target for automatic database backups.
    +     		false
    clientAuthenticationCertificates[]string + The Secrets containing TLS Client Certificate to use for Authentication
    +     		false
    connectionSettingsobject + Connection-related settings such as proxy connections and scheduling policy.
    +     		false
    dataInternodeEncryptionboolean + Internode encryption (INE) setting. An optional boolean setting, overriding a similar cluster-wide policy. If set to False, INE is guaranteed to be turned off for this DB (regardless of cluster-wide policy). If set to True, INE will be turned on, unless the capability is not supported by the DB ( in such a case we will get an error and database creation will fail). If left unspecified, will be disabled if internode encryption is not supported by the DB (regardless of cluster default). Deleting this property after explicitly setting its value shall have no effect.
    +     		false
    databasePortinteger + TCP port assigned to the database within the Redis Enterprise cluster. Must be unique across all databases in the Redis Enterprise cluster. Will be generated automatically if omitted. can not be changed after creation
    +     		false
    databaseSecretNamestring + Name of the secret containing the database password (Redis databases only). The secret is created automatically if it does not exist. The password is stored under the "password" key in the secret. If creating the secret manually, create an opaque secret with the password under the "password" key. To disable authentication, set the value of the "password" key in the secret to an empty string. Note: For Active-Active databases, this secret is not created automatically. For memcached databases, use memcachedSaslSecretName instead.
    +     		false
    databaseServicePortinteger + A custom port to be exposed by the database Services. Can be modified/added/removed after REDB creation. If set, it'll replace the default service port (namely, databasePort or defaultRedisPort).
    +     		false
    defaultUserboolean + Allows connections with the default user. When disabled, the DatabaseSecret is not created or updated.
    +     		false
    evictionPolicystring + Database eviction policy. See https://redis.io/docs/latest/operate/rs/databases/memory-performance/eviction-policy/
    +     		false
    isRofboolean + Enables Auto Tiering (formerly Redis on Flash) for Redis databases only. Defaults to false.
    +     		false
    memcachedSaslSecretNamestring + Name of the secret containing credentials for memcached database authentication. Store credentials in an opaque secret with "username" and "password" keys. Note: Connections are not encrypted.
    +     		false
    memorySizestring + Memory size for the database using formats like 100MB or 0.1GB. Minimum value is 100MB. For Auto Tiering (formerly Redis on Flash), this value represents RAM+Flash memory and must be at least 1GB.
    +     		false
    modulesList[]object + List of modules associated with the database. The list of valid modules for the specific cluster can be retrieved from the status of the REC object. Use the "name" and "versions" fields for the specific module configuration. If specifying an explicit version for a module, automatic modules versions upgrade must be disabled by setting the '.upgradeSpec.upgradeModulesToLatest' field in the REC to 'false'. Note that the option to specify module versions is deprecated, and will be removed in future releases. for Redis version 8 and above, bundled modules are enabled automatically, so there is no need to specify them
    +     		false
    ossClusterboolean + Enables OSS cluster mode for this database. By default, advertised database topology includes the internal endpoints (pod IPs) for the Redis Enterprise nodes hosting the database shards. To enable external access, configure ossClusterSettings.enableExternalAccess for this RedisEnterpriseDatabase as well as ossClusterSettings.externalAccessType for the RedisEnterpriseCluster. Note: Not all client libraries support OSS cluster mode.
    +     		false
    ossClusterSettingsobject + Additional OSS cluster mode settings.
    +     		false
    persistenceenum + Database persistence policy for on-disk storage.
    +
    + Enum: disabled, aofEverySecond, aofAlways, snapshotEvery1Hour, snapshotEvery6Hour, snapshotEvery12Hour
    +     		false
    proxyPolicystring + Proxy policy for the database. Supported proxy policies are: single/all-master-shards/all-nodes When left blank, the default value will be chosen according to the value of ossCluster - single if disabled, all-master-shards when enabled
    +     		false
    rackAwareboolean + Enables rack awareness for improved availability. See https://redis.io/docs/latest/operate/rs/clusters/configure/rack-zone-awareness/
    +     		false
    redisEnterpriseClusterobject + Connection to the Redis Enterprise Cluster.
    +     		false
    redisVersionstring + Redis OSS version. Version can be specified via prefix, or via channels - for existing databases - Upgrade Redis OSS version. For new databases - the version which the database will be created with. If set to 'major' - will always upgrade to the most recent major Redis version. If set to 'latest' - will always upgrade to the most recent Redis version. Depends on 'redisUpgradePolicy' - if you want to set the value to 'latest' for some databases, you must set redisUpgradePolicy on the cluster before. Possible values are 'major' or 'latest' When using upgrade - make sure to backup the database before. This value is used only for database type 'redis'
    +     		false
    replicaSources[]object + What databases to replicate from
    +     		false
    replicationboolean + Enables in-memory database replication for higher availability. Creates a replica shard for every master shard. Defaults to false.
    +     		false
    resp3boolean + Whether this database supports RESP3 protocol. Note - Deleting this property after explicitly setting its value shall have no effect. Please view the corresponding field in RS doc for more info.
    +     		false
    rofRamRatiointeger + RAM allocation ratio for Redis Flex (v2) databases as a percentage of total data size. Valid range 0-100. When omitted, RS uses the default value of 50%. Controls how much RAM is allocated per unit of data (e.g., 30% means 3MB RAM per 10MB data). RAM grows proportionally with data until rofRamSize limit is reached (if specified). Only applicable when isRof=true and Redis version >= 8.0 (BigStore v2 - Redis Flex).
    +
    + Minimum: 0
    + Maximum: 100
    +     		false
    rofRamSizestring + The size of the RAM portion of an Auto Tiering (formerly Redis on Flash) database. Similarly to "memorySize" use formats like 100MB, 0.1GB. It must be at least 10% of combined memory size (RAM and Flash), as specified by "memorySize".
    +     		false
    rolesPermissions[]object + Deprecated: List of Redis Enterprise ACL and Role bindings to apply. This field is deprecated and is ignored when the cluster's spec.accessControl.policy.allowREDBRolesPermissions is set to false.
    +     		false
    searchOnBigstoreboolean + Enables search module indexing on flash storage for Redis Flex (v2) databases. Only applicable when isRof=true and Redis version >= 8.6. Defaults to false.
    +     		false
    shardCountinteger + Number of database server-side shards.
    +     		false
    shardingEnabledboolean + Toggles database sharding for REAADBs (Active Active databases) and enabled by default. This field is blocked for REDB (non-Active Active databases) and sharding is toggled via the shardCount field - when shardCount is 1 this is disabled otherwise enabled.
    +     		false
    shardsPlacementenum + Shard placement strategy: "dense" or "sparse". dense: Shards reside on as few nodes as possible. sparse: Shards are distributed across as many nodes as possible.
    +
    + Enum: dense, sparse
    +     		false
    tagsmap[string]string + Tags to apply to the database, as a map of key to value. Tags are merged into the BDB's tags on the cluster. If a key here collides with an existing BDB tag, the value from spec.tags wins. Removing a key from spec.tags after it was previously synced (recorded in status.managedTags) will remove it from the BDB. Operator-reserved keys (managed_by, redb_name, redb_namespace, db_service_port, oss_cluster_access) are not allowed and will be rejected.
    +     		false
    tlsModeenum + Require TLS authenticated and encrypted connections to the database. enabled - all client and replication connections to the Database must use TLS. disabled - no incoming connection to the Database should use TLS. replica_ssl - databases that replicate from this one need to use TLS.
    +
    + Enum: disabled, enabled, replica_ssl
    +     		false
    typeenum + Database type: redis or memcached.
    +
    + Enum: redis, memcached
    +     		false
    upgradeSpecobject + Specifications for DB upgrade.
    +     		false
    + + +### spec.activeActive +[↩ Parent](#spec) + +Connection/ association to the Active-Active database. + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    namestring + The the corresponding Active-Active database name, Redis Enterprise Active Active Database custom resource name, this Resource is associated with. In case this resource is created manually at the active active database creation this field must be filled via the user, otherwise, the operator will assign this field automatically. Note: this feature is currently unsupported.
    +     		true
    participatingClusterNamestring + The corresponding participating cluster name, Redis Enterprise Remote Cluster custom resource name, in the Active-Active database, In case this resource is created manually at the active active database creation this field must be filled via the user, otherwise, the operator will assign this field automatically. Note: this feature is currently unsupported.
    +     		true
    + + +### spec.alertSettings +[↩ Parent](#spec) + +Settings for database alerts + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    bdb_backup_delayedobject + Periodic backup has been delayed for longer than specified threshold value [minutes]. -Note threshold is commented (allow string/int/float and support backwards compatibility) but is required
    +     		false
    bdb_crdt_src_high_syncer_lagobject + Active-active source - sync lag is higher than specified threshold value [seconds] -Note threshold is commented (allow string/int/float and support backwards compatibility) but is required
    +     		false
    bdb_crdt_src_syncer_connection_errorobject + Active-active source - sync has connection error while trying to connect replica source -Note threshold is commented (allow string/int/float and support backwards compatibility) but is required
    +     		false
    bdb_crdt_src_syncer_general_errorobject + Active-active source - sync encountered in general error -Note threshold is commented (allow string/int/float and support backwards compatibility) but is required
    +     		false
    bdb_high_latencyobject + Latency is higher than specified threshold value [micro-sec] -Note threshold is commented (allow string/int/float and support backwards compatibility) but is required
    +     		false
    bdb_high_throughputobject + Throughput is higher than specified threshold value [requests / sec.] -Note threshold is commented (allow string/int/float and support backwards compatibility) but is required
    +     		false
    bdb_long_running_actionobject + An alert for state-machines that are running for too long -Note threshold is commented (allow string/int/float and support backwards compatibility) but is required
    +     		false
    bdb_low_throughputobject + Throughput is lower than specified threshold value [requests / sec.] -Note threshold is commented (allow string/int/float and support backwards compatibility) but is required
    +     		false
    bdb_proxy_cert_expiring_soonobject + Proxy certificate will expire in less than specified threshold value [days]
    +     		false
    bdb_ram_dataset_overheadobject + Dataset RAM overhead of a shard has reached the threshold value [% of its RAM limit] -Note threshold is commented (allow string/int/float and support backwards compatibility) but is required
    +     		false
    bdb_ram_valuesobject + Percent of values kept in a shard's RAM is lower than [% of its key count] -Note threshold is commented (allow string/int/float and support backwards compatibility) but is required
    +     		false
    bdb_replica_src_high_syncer_lagobject + Replica-of source - sync lag is higher than specified threshold value [seconds] -Note threshold is commented (allow string/int/float and support backwards compatibility) but is required
    +     		false
    bdb_replica_src_syncer_connection_errorobject + Replica-of source - sync has connection error while trying to connect replica source -Note threshold is commented (allow string/int/float and support backwards compatibility) but is required
    +     		false
    bdb_shard_num_ram_valuesobject + Number of values kept in a shard's RAM is lower than [values] -Note threshold is commented (allow string/int/float and support backwards compatibility) but is required
    +     		false
    bdb_sizeobject + Dataset size has reached the threshold value [% of the memory limit] expected fields: -Note threshold is commented (allow string/int/float and support backwards compatibility) but is required
    +     		false
    + + +### spec.alertSettings.bdb_backup_delayed +[↩ Parent](#specalertsettings) + +Periodic backup has been delayed for longer than specified threshold value [minutes]. -Note threshold is commented (allow string/int/float and support backwards compatibility) but is required + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean + Alert enabled or disabled
    +     		false
    + + +### spec.alertSettings.bdb_crdt_src_high_syncer_lag +[↩ Parent](#specalertsettings) + +Active-active source - sync lag is higher than specified threshold value [seconds] -Note threshold is commented (allow string/int/float and support backwards compatibility) but is required + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean + Alert enabled or disabled
    +     		false
    + + +### spec.alertSettings.bdb_crdt_src_syncer_connection_error +[↩ Parent](#specalertsettings) + +Active-active source - sync has connection error while trying to connect replica source -Note threshold is commented (allow string/int/float and support backwards compatibility) but is required + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean + Alert enabled or disabled
    +     		false
    + + +### spec.alertSettings.bdb_crdt_src_syncer_general_error +[↩ Parent](#specalertsettings) + +Active-active source - sync encountered in general error -Note threshold is commented (allow string/int/float and support backwards compatibility) but is required + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean + Alert enabled or disabled
    +     		false
    + + +### spec.alertSettings.bdb_high_latency +[↩ Parent](#specalertsettings) + +Latency is higher than specified threshold value [micro-sec] -Note threshold is commented (allow string/int/float and support backwards compatibility) but is required + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean + Alert enabled or disabled
    +     		false
    + + +### spec.alertSettings.bdb_high_throughput +[↩ Parent](#specalertsettings) + +Throughput is higher than specified threshold value [requests / sec.] -Note threshold is commented (allow string/int/float and support backwards compatibility) but is required + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean + Alert enabled or disabled
    +     		false
    + + +### spec.alertSettings.bdb_long_running_action +[↩ Parent](#specalertsettings) + +An alert for state-machines that are running for too long -Note threshold is commented (allow string/int/float and support backwards compatibility) but is required + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean + Alert enabled or disabled
    +     		false
    + + +### spec.alertSettings.bdb_low_throughput +[↩ Parent](#specalertsettings) + +Throughput is lower than specified threshold value [requests / sec.] -Note threshold is commented (allow string/int/float and support backwards compatibility) but is required + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean + Alert enabled or disabled
    +     		false
    + + +### spec.alertSettings.bdb_proxy_cert_expiring_soon +[↩ Parent](#specalertsettings) + +Proxy certificate will expire in less than specified threshold value [days] + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean + Alert enabled or disabled
    +     		false
    + + +### spec.alertSettings.bdb_ram_dataset_overhead +[↩ Parent](#specalertsettings) + +Dataset RAM overhead of a shard has reached the threshold value [% of its RAM limit] -Note threshold is commented (allow string/int/float and support backwards compatibility) but is required + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean + Alert enabled or disabled
    +     		false
    + + +### spec.alertSettings.bdb_ram_values +[↩ Parent](#specalertsettings) + +Percent of values kept in a shard's RAM is lower than [% of its key count] -Note threshold is commented (allow string/int/float and support backwards compatibility) but is required + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean + Alert enabled or disabled
    +     		false
    + + +### spec.alertSettings.bdb_replica_src_high_syncer_lag +[↩ Parent](#specalertsettings) + +Replica-of source - sync lag is higher than specified threshold value [seconds] -Note threshold is commented (allow string/int/float and support backwards compatibility) but is required + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean + Alert enabled or disabled
    +     		false
    + + +### spec.alertSettings.bdb_replica_src_syncer_connection_error +[↩ Parent](#specalertsettings) + +Replica-of source - sync has connection error while trying to connect replica source -Note threshold is commented (allow string/int/float and support backwards compatibility) but is required + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean + Alert enabled or disabled
    +     		false
    + + +### spec.alertSettings.bdb_shard_num_ram_values +[↩ Parent](#specalertsettings) + +Number of values kept in a shard's RAM is lower than [values] -Note threshold is commented (allow string/int/float and support backwards compatibility) but is required + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean + Alert enabled or disabled
    +     		false
    + + +### spec.alertSettings.bdb_size +[↩ Parent](#specalertsettings) + +Dataset size has reached the threshold value [% of the memory limit] expected fields: -Note threshold is commented (allow string/int/float and support backwards compatibility) but is required + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean + Alert enabled or disabled
    +     		false
    + + +### spec.auditing +[↩ Parent](#spec) + +Database auditing configuration. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    auditModeenum + Scope used for database auditing. +"disabled" - auditing is disabled. +"connection" - audit connection, authentication and disconnection events only. +"connection_and_crud" - audit connection events and data operations (CRUD). +The cluster-level auditing configuration must be set before enabling auditing on a database. +Defaults to "disabled" when not set; removing the auditing configuration disables auditing.
    +
    + Enum: disabled, connection, connection_and_crud
    +     		false
    dbConnsAuditingboolean + Deprecated: use auditMode instead. +Enables auditing of database connection and authentication events. +When enabled, connection, authentication, and disconnection events are tracked and sent +to the configured audit listener (configured at the cluster level). +The cluster-level auditing configuration must be set before enabling this on a database. +Setting dbConnsAuditing to true is equivalent to auditMode "connection". +If both dbConnsAuditing and auditMode are set, auditMode takes precedence.
    +     		false
    sourceIpFilterobject + Filters data operation auditing events by client source IP address. +Only relevant when auditMode is "connection_and_crud". +Note: source IP filtering may not work as expected for Redis clients running outside +of the Kubernetes cluster. The operator does not automatically customize the +LoadBalancer service or ingress/route to preserve the client source IP, so connections +may be seen with a translated (NAT'd) address. If you choose to use this filter, it is +your responsibility to ensure source IP preservation is configured and to verify that +auditing behaves as expected in your environment.
    +     		false
    usernameFilterobject + Filters data operation auditing events by client usernames. +Only relevant when auditMode is "connection_and_crud".
    +     		false
    + + +### spec.auditing.sourceIpFilter +[↩ Parent](#specauditing) + +Filters data operation auditing events by client source IP address. +Only relevant when auditMode is "connection_and_crud". +Note: source IP filtering may not work as expected for Redis clients running outside +of the Kubernetes cluster. The operator does not automatically customize the +LoadBalancer service or ingress/route to preserve the client source IP, so connections +may be seen with a translated (NAT'd) address. If you choose to use this filter, it is +your responsibility to ensure source IP preservation is configured and to verify that +auditing behaves as expected in your environment. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    cidrRanges[]string + List of CIDR ranges (IPv4 and/or IPv6) to filter by.
    +     		false
    enabledboolean + Whether the source IP filter is active. +When enabled, at least one entry must be provided across ipAddresses and cidrRanges.
    +     		false
    filterTypeenum + Filter type to use for the source IP filter. +"inclusive" - only audit events from the listed addresses/ranges. +"exclusive" - audit events from all addresses/ranges except the listed ones. +The combined number of ipAddresses and cidrRanges may not exceed 512.
    +
    + Enum: inclusive, exclusive
    +     		false
    ipAddresses[]string + List of IP addresses (IPv4 and/or IPv6) to filter by.
    +     		false
    + + +### spec.auditing.usernameFilter +[↩ Parent](#specauditing) + +Filters data operation auditing events by client usernames. +Only relevant when auditMode is "connection_and_crud". + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean + Whether the username filter is active. +When enabled, usernames must contain at least one entry.
    +     		false
    filterTypeenum + Filter type to use for the username filter. +"inclusive" - only audit events from the listed usernames. +"exclusive" - audit events from all usernames except the listed ones.
    +
    + Enum: inclusive, exclusive
    +     		false
    usernames[]string + List of usernames to filter by. +Each username must be 1-255 printable ASCII characters and may not contain +the characters '&', '<', '>' or '"'. At most 512 usernames are allowed.
    +     		false
    + + +### spec.backup +[↩ Parent](#spec) + +Target for automatic database backups. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    absobject +
    +     		false
    ftpobject +
    +     		false
    gcsobject + GoogleStorage
    +     		false
    intervalinteger + Backup Interval in seconds. Specifies the time interval in seconds at which periodic backup is performed.
    +     		false
    intervalOffsetinteger + Backup Interval Offset in seconds. Specifies a time offset in seconds at which the periodic backup job starts. This can only be used if the backup interval is 24 hours (86400 seconds) or 12 hours (43200 seconds). The offset is relative to 00:00 UTC for 24-hour mode, and 00:00 + 12:00 UTC for 12-hour mode. If not specified, a random starting time (offset) is automatically chosen. The offset must be less than the backup interval.
    +     		false
    mountobject + MountPointStorage
    +     		false
    s3object +
    +     		false
    sftpobject +
    +     		false
    swiftobject +
    +     		false
    + + +### spec.backup.abs +[↩ Parent](#specbackup) + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    absSecretNamestring + The name of the K8s secret that holds ABS credentials. The secret must contain the keys "AccountName" and "AccountKey", and these must hold the corresponding credentials
    +     		true
    containerstring + Azure Blob Storage container name.
    +     		true
    subdirstring + Optional. Azure Blob Storage subdir under container.
    +     		false
    + + +### spec.backup.ftp +[↩ Parent](#specbackup) + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    urlstring + a URI of the ftps://[USER[:PASSWORD]@]HOST[:PORT]/PATH[/]
    +     		true
    + + +### spec.backup.gcs +[↩ Parent](#specbackup) + +GoogleStorage + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    bucketNamestring + Google Storage bucket name.
    +     		true
    gcsSecretNamestring + The name of the K8s secret that holds the Google Cloud Storage credentials. The secret must contain the keys "CLIENT_ID", "PRIVATE_KEY", "PRIVATE_KEY_ID", "CLIENT_EMAIL" and these must hold the corresponding credentials. The keys should correspond to the values in the key JSON.
    +     		true
    subdirstring + Optional. Google Storage subdir under bucket.
    +     		false
    + + +### spec.backup.mount +[↩ Parent](#specbackup) + +MountPointStorage + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    pathstring + Path to the local mount point. You must create the mount point on all nodes, and the redislabs:redislabs user must have read and write permissions on the local mount point.
    +     		true
    + + +### spec.backup.s3 +[↩ Parent](#specbackup) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    awsSecretNamestring + The name of the K8s secret that holds the AWS credentials. The secret must contain the keys "AWS_ACCESS_KEY_ID" and "AWS_SECRET_ACCESS_KEY", and these must hold the corresponding credentials.
    +     		true
    bucketNamestring + Amazon S3 bucket name.
    +     		true
    regionNamestring + Optional. Amazon S3 region name. If not specified, the region is auto-detected using a HEAD request to the bucket. For AWS GovCloud or other regions where auto-detection may not work, specify the region explicitly (e.g., "us-gov-east-1").
    +     		false
    subdirstring + Optional. Amazon S3 subdir under bucket.
    +     		false
    + + +### spec.backup.sftp +[↩ Parent](#specbackup) + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    sftpSecretNamestring + The name of the K8s secret that holds SFTP credentials. The secret must contain the "Key" key, which is the SSH private key for connecting to the sftp server.
    +     		true
    sftp_urlstring + SFTP url
    +     		true
    + + +### spec.backup.swift +[↩ Parent](#specbackup) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    auth_urlstring + Swift service authentication URL.
    +     		true
    containerstring + Swift object store container for storing the backup files.
    +     		true
    swiftSecretNamestring + The name of the K8s secret that holds Swift credentials. The secret must contain the keys "Key" and "User", and these must hold the corresponding credentials: service access key and service user name (pattern for the latter does not allow special characters &,<,>,")
    +     		true
    prefixstring + Optional. Prefix (path) of backup files in the swift container.
    +     		false
    + + +### spec.connectionSettings +[↩ Parent](#spec) + +Connection-related settings such as proxy connections and scheduling policy. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    connectionLimitTypeenum + Connections limit type.
    +
    + Enum: per-thread, per-shard
    +     		false
    connectionSchedulingPolicyenum + Controls how server-side connections are used when forwarding traffic to shards. Values: cmp: Closest to max_pipelined policy. Pick the connection with the most pipelined commands that has not reached the max_pipelined limit. mru: Try to use most recently used connections. spread: Try to use all connections. mnp: Minimal pipeline policy. Pick the connection with the least pipelined commands.
    +
    + Enum: cmp, mru, spread, mnp
    +     		false
    globalMaxDedicatedConnectionsinteger + Defines the maximum number of dedicated server connections for a given database. The total number across all workers. The default is 0 for unlimited. Defaults to 0, which means unlimited.
    +     		false
    internalConnectionsinteger + The number of internal proxy connections. Defaults to 5 if unspecified.
    +     		false
    minDedicatedConnectionsinteger + Number of dedicated server connections the DMC has per worker per shard. Defaults to 2 if unspecified.
    +     		false
    + + +### spec.modulesList[] +[↩ Parent](#spec) + +Redis Enterprise module (see https://redis.io/docs/latest/develop/reference/modules/) + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    namestring + The name of the module, e.g. "search" or "ReJSON". The complete list of modules available in the cluster can be retrieved from the '.status.modules' field in the REC.
    +     		true
    configstring + Module command line arguments e.g. VKEY_MAX_ENTITY_COUNT 30 30
    +     		false
    versionstring + The semantic version of the module, e.g. '1.6.12'. Optional for REDB, must be set for REAADB. Note that this field is deprecated, and will be removed in future releases.
    +     		false
    + + +### spec.ossClusterSettings +[↩ Parent](#spec) + +Additional OSS cluster mode settings. + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enableExternalAccessboolean + Toggles whether this database supports external access in OSS cluster mode. When enabled, advertised database topology includes the external endpoints for the Redis Enterprise nodes hosting the database shards. The external access mechanism (e.g., LoadBalancer services) is configured via the ossClusterSettings.externalAccessType field of the RedisEnterpriseCluster. When external access is enabled, the corresponding database secret will have the list of primary shard IPs in the oss_startup_nodes field. Currently enabling OSS cluster API in conjunction with external access is not supported for active active databases.
    +     		false
    + + +### spec.redisEnterpriseCluster +[↩ Parent](#spec) + +Connection to the Redis Enterprise Cluster. + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    namestring + The name of the Redis Enterprise Cluster where the database should be stored.
    +     		true
    + + +### spec.replicaSources[] +[↩ Parent](#spec) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    replicaSourceNamestring + The name of the resource from which the source database URI is derived. The type of resource must match the type specified in the ReplicaSourceType field.
    +     		true
    replicaSourceTypestring + The type of resource from which the source database URI is derived. If set to 'SECRET', the source database URI is derived from the secret named in the ReplicaSourceName field. The secret must have a key named 'uri' that defines the URI of the source database in the form of 'redis://...'. The type of secret (kubernetes, vault, ...) is determined by the secret mechanism used by the underlying REC object. If set to 'REDB', the source database URI is derived from the RedisEnterpriseDatabase resource named in the ReplicaSourceName field.
    +     		true
    clientKeySecretstring + Secret that defines the client certificate and key used by the syncer in the target database cluster. The secret must the following keys in it's data: - A key named 'cert'/'certificate'/'tls.crt' which is the PEM encoded certificate - A key named 'key'/'tls.key' which is the PEM encoded private key. - Optionally, a key named 'ca.crt', containing the public certificate of the root CA. If present, the root CA certificate is appended to the certificate provided in the 'tls.crt' (or equivalent) key, to form a full certificate chain. Otherwise, the certificate in 'cert'/'certificate'/'tls.crt' must include a full certificate chain inline. This key is typically populated by the cert-manager when it has access to the root certificate. Otherwise, it could be added manually.
    +     		false
    compressioninteger + GZIP compression level (0-6) to use for replication.
    +     		false
    serverCertSecretstring + Secret that defines the server certificate used by the proxy in the source database cluster. The secret must have 1 key in its map named 'cert'/'certificate'/'tls.crt' which is the PEM encoded certificate.
    +     		false
    tlsSniNamestring + TLS SNI name to use for the replication link.
    +     		false
    + + +### spec.rolesPermissions[] +[↩ Parent](#spec) + +Redis Enterprise Role and ACL Binding + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    aclstring + Acl Name of RolePermissionType
    +     		true
    rolestring + Role Name of RolePermissionType
    +     		true
    typeenum + Type of Redis Enterprise Database Role Permission. Currently, only "redis-enterprise" is supported, which uses roles and ACLs defined within Redis Enterprise directly.
    +
    + Enum: redis-enterprise
    + Default: redis-enterprise
    +     		true
    + + +### spec.upgradeSpec +[↩ Parent](#spec) + +Specifications for DB upgrade. + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    upgradeModulesToLatestboolean + DEPRECATED Upgrades the modules to the latest version that supports the DB version during a DB upgrade action, to upgrade the DB version view the 'redisVersion' field. Notes - All modules must be without specifying the version. in addition, This field is currently not supported for Active-Active databases. The default is true
    +     		true
    + + +### status +[↩ Parent](#) + +RedisEnterpriseDatabaseStatus defines the observed state of RedisEnterpriseDatabase + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    activeActiveobject + Connection/ association to the Active-Active database.
    +     		false
    backupInfoobject + Information on the database's periodic backup
    +     		false
    bigstoreVersioninteger + BigStore version for Redis on Flash databases (1 for Auto Tiering, 2 for Redis Flex). Read-only field populated from RS.
    +     		false
    createdTimestring + Time when the database was created
    +     		false
    databaseUIDstring + Database UID provided by redis enterprise
    +     		false
    internalEndpoints[]object + Endpoints listed internally by the Redis Enterprise Cluster. Can be used to correlate a ReplicaSourceStatus entry.
    +     		false
    lastActionStatusstring + Status of the last action done by operator on this database
    +     		false
    lastActionUidstring + UID of the last action done by operator on this database
    +     		false
    lastUpdatedstring + Time when the database was last updated
    +     		false
    managedFields[]string + List of optional spec fields the operator has taken ownership of after the user configured them at least once, so removing such a field resets the BDB to its default instead of leaving the inherited value in place.
    +     		false
    managedTags[]string + List of user-declared tag keys (from spec.tags) that were last successfully reconciled to the BDB. Used to detect tag deletions on the next reconcile: keys present here but absent from the current spec.tags are removed from the BDB. Operator-internal keys (managed_by, redb_name, etc.) are not tracked here.
    +     		false
    observedGenerationinteger + The generation (built in update counter of K8s) of the REDB resource that was fully acted upon, meaning that all changes were handled and sent as an API call to the Redis Enterprise Cluster (REC). This field value should equal the current generation when the resource changes were handled. Note: the lastActionStatus field tracks actions handled asynchronously by the Redis Enterprise Cluster.
    +
    + Format: int64
    +     		false
    redisEnterpriseClusterstring + The Redis Enterprise Cluster Object this Resource is associated with
    +     		false
    replicaSourceStatuses[]object + ReplicaSource statuses
    +     		false
    rolesPermissions[]object + The set of role-permission tuples associated with this database. +Populated from spec.rolesPermissions (deprecated), RedisEnterpriseRole objects whose +scopes reference this database, and RedisEnterpriseClusterRole objects (whose ACLs +apply to every database in the Redis Enterprise Cluster).
    +     		false
    shardStatusesmap[string]integer + Aggregated statuses of shards
    +     		false
    specStatusstring + Whether the desired specification is valid
    +     		false
    statusstring + The status of the database
    +     		false
    versionstring + Database compatibility version
    +     		false
    + + +### status.activeActive +[↩ Parent](#status) + +Connection/ association to the Active-Active database. + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    namestring + The the corresponding Active-Active database name, Redis Enterprise Active Active Database custom resource name, this Resource is associated with. In case this resource is created manually at the active active database creation this field must be filled via the user, otherwise, the operator will assign this field automatically. Note: this feature is currently unsupported.
    +     		true
    participatingClusterNamestring + The corresponding participating cluster name, Redis Enterprise Remote Cluster custom resource name, in the Active-Active database, In case this resource is created manually at the active active database creation this field must be filled via the user, otherwise, the operator will assign this field automatically. Note: this feature is currently unsupported.
    +     		true
    + + +### status.backupInfo +[↩ Parent](#status) + +Information on the database's periodic backup + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    backupFailureReasonstring + Reason of last failed backup process
    +     		false
    backupHistoryinteger + Backup history retention policy (number of days, 0 is forever)
    +     		false
    backupIntervalinteger + Interval in seconds in which automatic backup will be initiated
    +     		false
    backupIntervalOffsetinteger + Offset (in seconds) from round backup interval when automatic backup will be initiated (should be less than backup_interval)
    +     		false
    backupProgressPercentageinteger + Database scheduled periodic backup progress (percentage)
    +     		false
    backupStatusstring + Status of scheduled periodic backup process
    +     		false
    lastBackupTimestring + Time of last successful backup
    +     		false
    + + +### status.internalEndpoints[] +[↩ Parent](#status) + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    hoststring + Hostname assigned to the database
    +     		false
    portinteger + Database port name
    +     		false
    + + +### status.replicaSourceStatuses[] +[↩ Parent](#status) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    endpointHoststring + The internal host name of the replica source database. Can be used as an identifier. See the internalEndpoints list on the REDB status.
    +     		true
    laginteger + Lag in millisec between source and destination (while synced).
    +     		false
    lastErrorstring + Last error encountered when syncing from the source.
    +     		false
    lastUpdatestring + Time when we last receive an update from the source.
    +     		false
    rdbSizeinteger + The source’s RDB size to be transferred during the syncing phase.
    +     		false
    rdbTransferredinteger + Number of bytes transferred from the source’s RDB during the syncing phase.
    +     		false
    statusstring + Sync status of this source
    +     		false
    + + +### status.rolesPermissions[] +[↩ Parent](#status) + +RolePermissionStatus is an observed role-ACL binding on the database. +Role kind: RedisEnterpriseRole, RedisEnterpriseClusterRole, or role; ACL kind: +RedisEnterpriseACL or redis_acl. + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    aclobject + Reference to the ACL.
    +     		true
    roleobject + Reference to the role.
    +     		true
    + + +### status.rolesPermissions[].acl +[↩ Parent](#statusrolespermissions) + +Reference to the ACL. + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    kindstring + Kind of the referent.
    +     		false
    namestring + Name of the referent.
    +     		false
    + + +### status.rolesPermissions[].role +[↩ Parent](#statusrolespermissions) + +Reference to the role. + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    kindstring + Kind of the referent.
    +     		false
    namestring + Name of the referent.
    +     		false
    diff --git a/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_remote_cluster_api.md b/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_remote_cluster_api.md new file mode 100644 index 0000000000..e90dcabdf4 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_remote_cluster_api.md @@ -0,0 +1,182 @@ +--- +title: RedisEnterpriseRemoteCluster API Reference +alwaysopen: false +categories: +- docs +- operate +- kubernetes +linkTitle: RERC API +weight: 30 +url: '/operate/kubernetes/8.2.0/reference/api/redis_enterprise_remote_cluster_api/' +--- + +apiVersion: + + +- [app.redislabs.com/v1alpha1](#appredislabscomv1alpha1) + + + + +# app.redislabs.com/v1alpha1 + + + + +Represents a remote participating cluster. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    apiVersionstringapp.redislabs.com/v1alpha1true
    kindstringRedisEnterpriseRemoteClustertrue
    metadataobjectRefer to the Kubernetes API documentation for the fields of the `metadata` field.true
    specobject +
    +     		false
    statusobject +
    +     		false
    + + +### spec +[↩ Parent](#) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    apiFqdnUrlstring + URL of the cluster. Used for the Active-Active database URL.
    +     		true
    recNamestring + Name of the REC that this RERC points to.
    +     		true
    recNamespacestring + Namespace of the REC that this RERC points to.
    +     		true
    apiPortinteger + Port number of the cluster's URL. Used for connectivity and synchronization.
    +     		false
    dbFqdnSuffixstring + Database URL suffix. Used for the Active-Active database replication endpoint and replication endpoint SNI.
    +     		false
    secretNamestring + Name of the secret containing cluster credentials. Must use the following format: "redis-enterprise-".
    +     		false
    + + +### status +[↩ Parent](#) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    internalObservedSecretResourceVersionstring + Observed secret resource version. For internal use only.
    +     		false
    localboolean + Indicates whether this object represents a local or remote cluster.
    +     		false
    observedGenerationinteger + Most recent generation observed for this RERC. Corresponds to the RERC's generation, which is updated by the API Server.
    +     		false
    specStatusstring + Indicates whether the desired specification is valid.
    +     		false
    statusstring + Status of the remote cluster.
    +     		false
    diff --git a/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_role_api.md b/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_role_api.md new file mode 100644 index 0000000000..2a7d18b30b --- /dev/null +++ b/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_role_api.md @@ -0,0 +1,299 @@ +--- +title: RedisEnterpriseRole API Reference +alwaysopen: false +categories: +- docs +- operate +- kubernetes +linkTitle: REROLE API +weight: 30 +url: '/operate/kubernetes/8.2.0/reference/api/redis_enterprise_role_api/' +--- + +apiVersion: + + +- [app.redislabs.com/v1alpha1](#appredislabscomv1alpha1) + + + + +# app.redislabs.com/v1alpha1 + + + + +RedisEnterpriseRole is a scoped role definition. It grants users access to selected databases within a Redis Enterprise Cluster. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    apiVersionstringapp.redislabs.com/v1alpha1true
    kindstringRedisEnterpriseRoletrue
    metadataobjectRefer to the Kubernetes API documentation for the fields of the `metadata` field.true
    specobject + RedisEnterpriseRoleSpec defines the desired state of a RedisEnterpriseRole.
    +     		false
    statusobject + RedisEnterpriseRoleStatus defines the observed state of a Redis Enterprise role object. This status type is used by both RedisEnterpriseClusterRole and RedisEnterpriseRole which share the same status structure.
    +     		false
    + + +### spec +[↩ Parent](#) + +RedisEnterpriseRoleSpec defines the desired state of a RedisEnterpriseRole. + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    scopes[]object + Scopes define the databases to which this role applies. +Each scope references a RedisEnterpriseDatabase by name or selector (kind +RedisEnterpriseDatabase or empty), or an internal Redis Enterprise database +by name (kind 'bdb', name only). +For 'bdb' scopes the role grants management access only; its ACL is not bound +to that database's permissions.
    +     		true
    aclobject + ACL defines the data access permissions granted by this scoped role. +References a RedisEnterpriseACL (kind RedisEnterpriseACL or empty) or an +internal Redis Enterprise ACL by name (kind 'redis_acl').
    +     		false
    managementRoleenum + ManagementRole is the named set of Redis Enterprise management permissions assigned to this scoped role. RedisEnterpriseRole supports only database-scoped management roles. If omitted, the role uses the None management role.
    +
    + Enum: DBMember, DBViewer, None
    +     		false
    + + +### spec.scopes[] +[↩ Parent](#spec) + +A reference to another object(s), either directly by name, or via a label selector. + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    kindenum + Kind of the referent.
    +
    + Enum: , RedisEnterpriseDatabase, bdb
    +     		false
    namestring + Name of the referent.
    +     		false
    selectorobject + Selector based on label attached to the object.
    +     		false
    + + +### spec.scopes[].selector +[↩ Parent](#specscopes) + +Selector based on label attached to the object. + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    matchExpressions[]object + matchExpressions is a list of label selector requirements. The requirements are ANDed.
    +     		false
    matchLabelsmap[string]string + matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
    +     		false
    + + +### spec.scopes[].selector.matchExpressions[] +[↩ Parent](#specscopesselector) + +A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values. + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    keystring + key is the label key that the selector applies to.
    +     		true
    operatorstring + operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
    +     		true
    values[]string + values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
    +     		false
    + + +### spec.acl +[↩ Parent](#spec) + +ACL defines the data access permissions granted by this scoped role. +References a RedisEnterpriseACL (kind RedisEnterpriseACL or empty) or an +internal Redis Enterprise ACL by name (kind 'redis_acl'). + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    namestring + Name of the referent.
    +     		true
    kindenum + Kind of the referent.
    +
    + Enum: , RedisEnterpriseACL, redis_acl
    +     		false
    + + +### status +[↩ Parent](#) + +RedisEnterpriseRoleStatus defines the observed state of a Redis Enterprise role object. This status type is used by both RedisEnterpriseClusterRole and RedisEnterpriseRole which share the same status structure. + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    observedGenerationinteger + The most recent generation of this resource that was observed and acted upon by the controller.
    +
    + Format: int64
    +     		false
    uidstring + The internal UID of the Role object defined in the Redis Enterprise Cluster.
    +     		false
    diff --git a/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_role_binding_api.md b/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_role_binding_api.md new file mode 100644 index 0000000000..53a70090c0 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_role_binding_api.md @@ -0,0 +1,175 @@ +--- +title: RedisEnterpriseRoleBinding API Reference +alwaysopen: false +categories: +- docs +- operate +- kubernetes +linkTitle: REROLEBINDING API +weight: 30 +url: '/operate/kubernetes/8.2.0/reference/api/redis_enterprise_role_binding_api/' +--- + +apiVersion: + + +- [app.redislabs.com/v1alpha1](#appredislabscomv1alpha1) + + + + +# app.redislabs.com/v1alpha1 + + + + +RedisEnterpriseRoleBinding binds users to a RedisEnterpriseRole. It grants the users the scoped access defined by that role. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    apiVersionstringapp.redislabs.com/v1alpha1true
    kindstringRedisEnterpriseRoleBindingtrue
    metadataobjectRefer to the Kubernetes API documentation for the fields of the `metadata` field.true
    specobject + RedisEnterpriseRoleBindingSpec defines the desired state of a RedisEnterpriseClusterRoleBinding or RedisEnterpriseRoleBinding.
    +     		false
    statusobject + RedisEnterpriseRoleBindingStatus defines the observed state of a RedisEnterpriseClusterRoleBinding or RedisEnterpriseRoleBinding.
    +     		false
    + + +### spec +[↩ Parent](#) + +RedisEnterpriseRoleBindingSpec defines the desired state of a RedisEnterpriseClusterRoleBinding or RedisEnterpriseRoleBinding. + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    roleRefobject + RoleRef holds a reference to the role object being bound. For RedisEnterpriseClusterRoleBinding, this references a RedisEnterpriseClusterRole in the same namespace (kind RedisEnterpriseClusterRole or empty) or an internal Redis Enterprise role by name (kind 'role'). For RedisEnterpriseRoleBinding, this references a RedisEnterpriseRole in the same namespace (kind RedisEnterpriseRole or empty) or an internal Redis Enterprise role by name (kind 'role').
    +     		true
    subjects[]object + Subjects holds references to the objects the role applies to. Each subject is a reference to a RedisEnterpriseUser object within the same namespace. The 'kind' field can be left empty, or set to RedisEnterpriseUser. A maximum of 100 subjects may be specified; to bind more, create additional bindings referencing the same role.
    +
    + Validations:
  • self.all(s1, self.exists_one(s2, + (has(s1.kind) && s1.kind != '' ? s1.kind : 'RedisEnterpriseUser') == + (has(s2.kind) && s2.kind != '' ? s2.kind : 'RedisEnterpriseUser') && + s1.name == s2.name)) +: subjects must be unique by kind and name
    • +     		false
    + + +### spec.roleRef +[↩ Parent](#spec) + +RoleRef holds a reference to the role object being bound. For RedisEnterpriseClusterRoleBinding, this references a RedisEnterpriseClusterRole in the same namespace (kind RedisEnterpriseClusterRole or empty) or an internal Redis Enterprise role by name (kind 'role'). For RedisEnterpriseRoleBinding, this references a RedisEnterpriseRole in the same namespace (kind RedisEnterpriseRole or empty) or an internal Redis Enterprise role by name (kind 'role'). + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    namestring + Name of the referent.
    +     		true
    kindenum + Kind of the referent.
    +
    + Enum: , RedisEnterpriseRole, role
    +     		false
    + + +### spec.subjects[] +[↩ Parent](#spec) + +A reference to another object. + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    namestring + Name of the referent.
    +     		true
    kindenum + Kind of the referent.
    +
    + Enum: , RedisEnterpriseUser
    +     		false
    diff --git a/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_user_api.md b/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_user_api.md new file mode 100644 index 0000000000..4bcb4289c4 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_user_api.md @@ -0,0 +1,501 @@ +--- +title: RedisEnterpriseUser API Reference +alwaysopen: false +categories: +- docs +- operate +- kubernetes +linkTitle: REUSER API +weight: 30 +url: '/operate/kubernetes/8.2.0/reference/api/redis_enterprise_user_api/' +--- + +apiVersion: + + +- [app.redislabs.com/v1alpha1](#appredislabscomv1alpha1) + + + + +# app.redislabs.com/v1alpha1 + + + + +RedisEnterpriseUser represents a user definition within a Redis Enterprise Cluster. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    apiVersionstringapp.redislabs.com/v1alpha1true
    kindstringRedisEnterpriseUsertrue
    metadataobjectRefer to the Kubernetes API documentation for the fields of the `metadata` field.true
    specobject + RedisEnterpriseUserSpec defines the desired state of a RedisEnterpriseUser.
    +     		false
    statusobject + RedisEnterpriseUserStatus defines the observed state of a RedisEnterpriseUser.
    +     		false
    + + +### spec +[↩ Parent](#) + +RedisEnterpriseUserSpec defines the desired state of a RedisEnterpriseUser. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    alertsobject + Email alerts settings for this user. User alerts settings are effective only when cluster alerts settings are configured.
    +     		false
    authMethodenum + AuthMethod selects how the user authenticates to Redis Enterprise. 'Basic' (the default) authenticates with Basic Auth, using a password sourced from passwordSecrets. 'Certificate' authenticates with a client certificate whose subject matches certificateSubjectLine; such users carry no password (passwordSecrets must be empty). Certificate users additionally require certificate-based authentication to be configured at the cluster level.
    +
    + Enum: Basic, Certificate
    + Default: Basic
    +     		false
    certificateSubjectLinestring + CertificateSubjectLine is the certificate subject line (RFC2253) used to match the client certificate for certificate-auth users. Required when authMethod is 'Certificate'; must not be set otherwise.
    +     		false
    emailstring + The email address for the user. Optional (Redis Enterprise accepts an empty email). For password users, changing the email after creation is permitted only while spec.passwordSecrets contains exactly one entry: to change the email of a user with two password secrets (Rotatable rotation in progress), first reduce passwordSecrets to a single entry, change the email, then add the second secret back. Certificate-auth users carry no password, so this restriction does not apply to them.
    +     		false
    passwordModeenum + Defines how passwords are managed for this user. Single: Only a single password secret can be specified at a time. Passwords changes can be done by mutating the secret in-place, or by replacing the secret reference to point to another secret. Rotatable: Allows for one or two password secrets to be specified simultaneously. The secrets are required to be immutable, and the operator automatically sets their 'immutable: true' attribute (K8s secrets only) if not already set. Vault secrets are assumed to be used as immutable and are not made immutable automatically. Password changes are typically done by adding a new password secret, and later removing the old one when it's no longer needed. When empty, Single behavior is used. Must not be set for certificate-auth users (authMethod: Certificate), which carry no password.
    +
    + Enum: Single, Rotatable
    +     		false
    passwordSecrets[]object + List of one or more secrets holding the user's password. Each secret must have a key named 'password' with the user's desired password. If multiple secrets are specified, then the passwords from all of them can be used for authenticating as the user. When the user signin status is 'Locked', the user won't be able to authenticate with Redis Enterprise API or UI, and must be unlocked by another user with Admin-role access. To unlock the user, first set a new password via the 'passwordSecrets' field, and then follow the password reset instructions in the following link: https://redis.io/docs/latest/operate/rs/security/access-control/manage-users/login-lockout/#unlock-locked-user-accounts
    +     		false
    usernamestring + The username associated with the user. The username must be unique within the Redis Enterprise cluster, and include only ASCII characters except &,<,>,". If not specified, a default username is assigned and can be discovered via the status section of the RedisEnterpriseUser.
    +     		false
    + + +### spec.alerts +[↩ Parent](#spec) + +Email alerts settings for this user. User alerts settings are effective only when cluster alerts settings are configured. + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean + Master toggle for email alerts.
    +     		true
    clusterAlertsobject + UserClusterAlertSettings specifies cluster alerts settings for a user.
    +     		false
    databaseAlertsobject + UserDatabaseAlertSettings specifies database alerts settings for a user.
    +     		false
    + + +### spec.alerts.clusterAlerts +[↩ Parent](#specalerts) + +UserClusterAlertSettings specifies cluster alerts settings for a user. + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    enabledboolean + Determines whether cluster alerts are enabled for the user.
    +     		true
    + + +### spec.alerts.databaseAlerts +[↩ Parent](#specalerts) + +UserDatabaseAlertSettings specifies database alerts settings for a user. + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    databases[]object + The databases for which alerts are enabled for this user. If not specified, defaults to all databases.
    +     		false
    + + +### spec.alerts.databaseAlerts.databases[] +[↩ Parent](#specalertsdatabasealerts) + +A reference to another object. + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    kindstring + Kind of the referent.
    +     		false
    namestring + Name of the referent.
    +     		false
    + + +### spec.passwordSecrets[] +[↩ Parent](#spec) + +A reference to a Secret object. + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    namestring + Name of the Secret.
    +     		true
    + + +### status +[↩ Parent](#) + +RedisEnterpriseUserStatus defines the observed state of a RedisEnterpriseUser. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    conditions[]object + Conditions represent the latest available observations of the user's state.
    +     		false
    observedGenerationinteger + The most recent generation of this resource that was observed and acted upon by the controller.
    +
    + Format: int64
    +     		false
    passwordIssueDatestring + The date in which the password was set.
    +     		false
    passwordSecrets[]object + List of secrets holding the user's password, with their versions.
    +     		false
    roles[]object + List of roles assigned to the user.
    +     		false
    rolesDisplaystring + Formatted string representation of the roles assigned to the user.
    +     		false
    signinStatusstring + Sign-in status for the user. When set to 'Locked', the user won't be able to authenticate with Redis Enterprise API or UI, and must be unlocked by another user with Admin-role access. To unlock the user, set a new password via the 'passwordSecrets' field, and then follow the password reset instructions in the following link: https://redis.io/docs/latest/operate/rs/security/access-control/manage-users/login-lockout/#unlock-locked-user-accounts
    +     		false
    uidstring + The internal UID of the User object defined in the Redis Enterprise Cluster.
    +     		false
    usernamestring + The effective username assigned for the user.
    +     		false
    + + +### status.conditions[] +[↩ Parent](#status) + +Condition contains details for one aspect of the current state of this API Resource. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    lastTransitionTimestring + lastTransitionTime is the last time the condition transitioned from one status to another.
    +
    + Format: date-time
    +     		true
    statusenum + status of the condition, one of True, False, Unknown.
    +
    + Enum: True, False, Unknown
    +     		true
    typestring + type of condition in CamelCase or in foo.example.com/CamelCase.
    +     		true
    messagestring + message is a human readable message indicating details about the transition.
    +     		false
    observedGenerationinteger + observedGeneration represents the .metadata.generation that the condition was set based upon.
    +
    + Format: int64
    + Minimum: 0
    +     		false
    reasonstring + reason contains a programmatic identifier indicating the reason for the condition's last transition.
    +     		false
    + + +### status.passwordSecrets[] +[↩ Parent](#status) + +A versioned reference to a Secret object. + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    namestring + Name of the Secret.
    +     		false
    versionstring + Version of the Secret.
    +     		false
    + + +### status.roles[] +[↩ Parent](#status) + +A reference to another object. + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionRequired
    kindstring + Kind of the referent.
    +     		false
    namestring + Name of the referent.
    +     		false
    diff --git a/content/operate/kubernetes/8.2.0/reference/supported_k8s_distributions.md b/content/operate/kubernetes/8.2.0/reference/supported_k8s_distributions.md new file mode 100644 index 0000000000..ed8beee2f9 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/reference/supported_k8s_distributions.md @@ -0,0 +1,78 @@ +--- +title: Supported Kubernetes distributions +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Support matrix for the current Redis Software for Kubernetes operator +linkTitle: Supported distributions +weight: 10 +url: '/operate/kubernetes/8.2.0/reference/supported_k8s_distributions/' +--- + +## Kubernetes version support + +Redis Software for Kubernetes is compatible with [CNCF-conformant](https://www.cncf.io/training/certification/software-conformance/) Kubernetes platforms. The operator follows standard Kubernetes APIs and practices and is designed to run consistently across certified Kubernetes environments. + +Redis tests Redis Software for Kubernetes on the following platforms: + +- Red Hat OpenShift +- Google Kubernetes Engine (GKE) +- Rancher Kubernetes Engine (RKE) +- Azure Kubernetes Service (AKS) +- Amazon Elastic Kubernetes Service (EKS) + +Redis also supports other CNCF-conformant distributions that run a supported Kubernetes version, even without explicit testing. + +The tables below use the following status icons: + +✅ Supported – This Kubernetes version is supported for this operator version. + +⚠️ Deprecated – Support for this Kubernetes version will be removed in a future release. + +❌ End of life – No longer supported. + +### Latest release + +Operator release **8.2.0-12** supports the following Kubernetes and OpenShift versions. + +| OpenShift | Kubernetes | Support status | +|---|---|---| +| — | 1.36 | ✅ Supported | +| 4.22 | 1.35 | ✅ Supported | +| 4.21 | 1.34 | ✅ Supported | +| 4.20 | 1.33 | ✅ Supported | +| 4.19 | 1.32 | ⚠️ Deprecated | + +### Version history + +The following table lists the Kubernetes versions supported by every operator version. Operator versions run from newest (left) to oldest (right). + +{{}}| | Redis operator | **8.2.0-12** | **8.0.20-21** | **8.0.18-11** | **8.0.10-15** | **8.0.6-6** | **8.0.2-2** | **7.22.2-21** | **7.22.0-15** | **7.22.0-7** | **7.8.6-1** | **7.8.4-9** | **7.8.4-8** | **7.8.2-6** | **7.4.6-2** | **7.4.2-12** | **7.4.2-2** | **7.2.4-12** | **7.2.4-7** | **7.2.4-2** | **6.4.2-8** | **6.4.2-6** | **6.4.2-5** | **6.4.2-4** | **6.2.18-41** | **6.2.18-3** | **6.2.12-1** | **6.2.10-45** | **6.2.10-34** | **6.2.10-4** | **6.2.8-15** | **6.2.8-11** | **6.2.8-2** | **6.2.4-1** | +|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---| +| | | July 2026 | May 2026 | March 2026 | Jan 2026 | Dec 2025 | Oct 2025 | Oct 2025 | July 2025 | April 2025 | March 2025 | March 2025 | Feb 2025 | Nov 2024 | July 2024 | May 2024 | March 2024 | Dec 2023 | Oct 2023 | Aug 2023 | July 2023 | June 2023 | April 2023 | March 2023 | Jan 2023 | Nov 2022 | Sept 2022 | July 2022 | May 2022 | March 2022 | Jan 2022 | Jan 2022 | Nov 2021 | Sept 2021 | +| OpenShift | Kubernetes | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | +| — | 1.36 | ✅ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | +| 4.22 | 1.35 | ✅ | ✅ | ✅ | ✅ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | +| 4.21 | 1.34 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | | | | | | | | | | | | | | | | | | | | | | | +| 4.20 | 1.33 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | | | | | | | | | | | | | | | | | | | | | +| 4.19 | 1.32 | ⚠️ | ⚠️ | ⚠️ | ⚠️ | ⚠️ | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | | | | | | | | | | | | | | | | | | | +| 4.18 | 1.31 | ❌ | ⚠️ | ⚠️ | ⚠️ | ⚠️ | ⚠️ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | | | | | | | | | | | | | | | | +| 4.17 | 1.30 | | | | | | ❌ | ⚠️ | ⚠️ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | | | | | | | | | | | | | | | +| 4.16 | 1.29 | | | | | | | | | ❌ | ⚠️ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | | | | | | | | | | | | | +| 4.15 | 1.28 | | | | | | | | | ❌ | ⚠️ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | | | | | | | | | | | | +| 4.14 | 1.27 | | | | | | | | | | | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | | | | | | | | +| 4.13 | 1.26 | | | | | | | | | | | | | | ⚠️ | ⚠️ | ⚠️ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | | | | | | +| 4.12 | 1.25 | | | | | | | | | | | | | | | | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | | | | +| 4.11 | 1.24 | | | | | | | | | | | | | | | | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | | +| 4.10 | 1.23 | | | | | | | | | | | | | | | | | | | ❌ | ⚠️ | ⚠️ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | +| 4.09 | 1.22 | | | | | | | | | | | | | | | | | | | | | ❌ | ⚠️ | ⚠️ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | +| 4.08 | 1.21 | | | | | | | | | | | | | | | | | | | | | | | | | | ❌ | ⚠️ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| 4.07 | 1.20 | | | | | | | | | | | | | | | | | | | | | | | | | | | ❌ | ⚠️ | ✅ | ✅ | ✅ | ✅ | ✅ | +| 4.06 | 1.19 | | | | | | | | | | | | | | | | | | | | | | | | | | | ❌ | ⚠️ | ✅ | ✅ | ✅ | ✅ | ✅ | +| 4.05 | 1.18 | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ✅ | ✅ | ✅ | ✅ | ✅ | +| 4.04 | 1.17 | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ❌ | ⚠️ | +| 4.03 | 1.16 | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ❌ | ⚠️ | + +{{}} diff --git a/content/operate/kubernetes/8.2.0/reference/yaml/_index.md b/content/operate/kubernetes/8.2.0/reference/yaml/_index.md new file mode 100644 index 0000000000..b5880a8b1e --- /dev/null +++ b/content/operate/kubernetes/8.2.0/reference/yaml/_index.md @@ -0,0 +1,89 @@ +--- +Title: YAML examples +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Example YAML files for deploying Redis Enterprise on Kubernetes with different configurations. +hideListLinks: true +linkTitle: YAML examples +weight: 85 +url: '/operate/kubernetes/8.2.0/reference/yaml/' +--- + +This section provides complete YAML examples that cover common Redis Enterprise for Kubernetes deployment scenarios. Each example includes the necessary configuration files and step-by-step instructions for editing and applying them. + +## How to use these examples + +### Download and customize + +1. Copy the YAML content from the examples below +2. Save each YAML block to a separate file with a descriptive name +3. Edit the configuration values to match your environment +4. Apply the files in the correct order using `kubectl apply` + +### Configuration storage + +Redis Enterprise for Kubernetes stores configuration in several places: + +- Custom resources: Cluster and database specifications are stored as Kubernetes custom resources (REC, REDB, REAADB, RERC) +- Secrets: Sensitive data like passwords and certificates are stored in Kubernetes secrets +- ConfigMaps: Non-sensitive configuration data is stored in ConfigMaps +- RBAC resources: Permissions are defined through Roles, ClusterRoles, and their bindings + +### Applying YAML files + +Apply YAML files using `kubectl apply`: + +```bash +# Apply a single file +kubectl apply -f my-config.yaml + +# Apply multiple files +kubectl apply -f rbac/ -f cluster/ -f database/ + +# Validate files without applying +kubectl apply --dry-run=client -f my-config.yaml +``` + +### Monitoring deployment + +Check the status of your resources after applying: + +```bash +# Check operator deployment +kubectl get deployment redis-enterprise-operator + +# Check cluster status +kubectl get rec +kubectl describe rec + +# Check database status +kubectl get redb +kubectl describe redb + +# View events for troubleshooting +kubectl get events --sort-by=.metadata.creationTimestamp +``` + +## Example categories + +- [Basic deployment examples](/content/operate/kubernetes/8.2.0/reference/yaml/basic-deployment.md) - Service account, RBAC, cluster, and database configurations +- [Rack awareness examples](/content/operate/kubernetes/8.2.0/reference/yaml/rack-awareness.md) - Rack-aware cluster configuration and required RBAC +- [Active-Active examples](/content/operate/kubernetes/8.2.0/reference/yaml/active-active.md) - Multi-cluster Active-Active database setup +- [Multi-namespace examples](/content/operate/kubernetes/8.2.0/reference/yaml/multi-namespace.md) - Cross-namespace operator and cluster configurations +- [Log collector RBAC examples](/content/operate/kubernetes/8.2.0/reference/yaml/log-collector-rbac.md) - RBAC permissions for log collection in restricted and all modes + +## Best practices + +- Validate configuration: Use `kubectl apply --dry-run=client` to validate YAML syntax and object schemas before applying +- Version control: Store your customized YAML files in version control +- Resource naming: Use consistent, descriptive names for all resources + +## Related documentation + +- [Reference](/content/operate/kubernetes/8.2.0/reference/_index.md) - Complete API specifications for all custom resources +- [Deploy Redis Enterprise Software for Kubernetes](/content/operate/kubernetes/8.2.0/deployment/quick-start.md) - Step-by-step deployment instructions +- [Manage databases in multiple namespaces](/content/operate/kubernetes/8.2.0/re-clusters/multi-namespace.md) - Detailed multi-namespace setup instructions +- [Active-Active databases](/content/operate/kubernetes/8.2.0/active-active/_index.md) - Active-Active configuration and management diff --git a/content/operate/kubernetes/8.2.0/reference/yaml/active-active.md b/content/operate/kubernetes/8.2.0/reference/yaml/active-active.md new file mode 100644 index 0000000000..dec0335aad --- /dev/null +++ b/content/operate/kubernetes/8.2.0/reference/yaml/active-active.md @@ -0,0 +1,94 @@ +--- +Title: Active-Active examples +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: YAML examples for Active-Active Redis Enterprise databases across multiple Kubernetes clusters. +linkTitle: Active-Active +weight: 30 +url: '/operate/kubernetes/8.2.0/reference/yaml/active-active/' +--- + +This page provides YAML examples for deploying Active-Active Redis Enterprise databases across multiple Kubernetes clusters. Active-Active databases provide multi-master replication with conflict resolution, enabling global distribution and local read/write access. + +For complete deployment instructions, see [Active-Active databases](/content/operate/kubernetes/8.2.0/active-active/_index.md). + +## Applying the configuration + +To deploy Active-Active databases using these YAML files, follow [Create Active-Active database (REAADB)](/content/operate/kubernetes/8.2.0/active-active/create-reaadb.md), which provides detailed instructions for preparing clusters, creating RERC resources, and deploying REAADB configurations. + +## Namespace examples + +A namespace is an abstraction used by Kubernetes to support multiple virtual clusters on the same physical cluster. + +`ns-illinois.yaml` is used in [Create Active-Active database](/content/operate/kubernetes/8.2.0/active-active/create-reaadb.md#example-values). + +{{}} + +`ns-virginia.yaml` is used in [Create Active-Active database](/content/operate/kubernetes/8.2.0/active-active/create-reaadb.md#example-values). + +{{}} + +## REC examples + +A Redis Enterprise cluster is a collection of Redis Enterprise nodes that pools system resources across nodes and supports multi-tenant database instances. + +`rec-chicago.yaml` is used in [Create Active-Active database](/content/operate/kubernetes/8.2.0/active-active/create-reaadb.md#prerequisites) and [Create RERC](/content/operate/kubernetes/8.2.0/active-active/create-reaadb.md#create-rerc). + +{{}} + +`rec-arlington.yaml` is used in [Create Active-Active database](/content/operate/kubernetes/8.2.0/active-active/create-reaadb.md#prerequisites) and [Create RERC](/content/operate/kubernetes/8.2.0/active-active/create-reaadb.md#create-rerc). + +{{}} + +## RERC examples + +RedisEnterpriseRemoteCluster represents a remote participating cluster. + +`rerc-ohare.yaml` is used in the [Create RERC](/content/operate/kubernetes/8.2.0/active-active/create-reaadb.md#create-rerc) section. + +{{}} + +`rerc-raegan.yaml` is used in the [Create RERC](/content/operate/kubernetes/8.2.0/active-active/create-reaadb.md#create-rerc) section. + +{{}} + +### RERC configuration + +- [metadata.name](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_remote_cluster_api.md#appredislabscomv1alpha1): Unique name for this remote cluster reference +- [spec.recName](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_remote_cluster_api.md#spec): Name of the remote REC +- [spec.recNamespace](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_remote_cluster_api.md#spec): Namespace of the remote REC +- [spec.apiFqdnUrl](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_remote_cluster_api.md#spec): API endpoint URL for the remote cluster +- [spec.dbFqdnSuffix](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_remote_cluster_api.md#spec): Database hostname suffix for the remote cluster +- [spec.secretName](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_remote_cluster_api.md#spec): Secret containing authentication credentials + +Edit the values in the downloaded YAML file for your specific setup, updating the remote cluster details, API endpoints, and secret names to match your actual environment. + +## Active-Active database examples + +Active-Active databases are geo-distributed databases that span multiple Redis Enterprise clusters and use multi-primary replication and conflict-free replicated data types (CRDTs). + +`reaadb-boeing.yaml` is used in the [Create Active-Active database](/content/operate/kubernetes/8.2.0/active-active/create-reaadb.md#create-reaadb) section. + +{{}} + +### REAADB configuration + +- [metadata.name](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_active_active_database_api.md#appredislabscomv1alpha1): Active-Active database name +- [spec.participatingClusters](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_active_active_database_api.md#specparticipatingclusters): List of RERC names that participate in this database +- [spec.globalConfigurations](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_active_active_database_api.md#specglobalconfigurations): Database settings applied to all participating clusters + +Edit the downloaded YAML file to add global database settings such as memory allocation, shard count, replication settings, database secrets, Redis modules, and database-specific Redis configuration. + +## Related documentation + +- [Active-Active databases (index)](/content/operate/kubernetes/8.2.0/active-active/_index.md) +- [Prepare participating clusters](/content/operate/kubernetes/8.2.0/active-active/prepare-clusters.md) +- [Create Active-Active database (REAADB)](/content/operate/kubernetes/8.2.0/active-active/create-reaadb.md) +- [Edit global configuration](/content/operate/kubernetes/8.2.0/active-active/global-config.md) +- [Sync global database secret](/content/operate/kubernetes/8.2.0/active-active/global-db-secret.md) +- [RERC API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_remote_cluster_api.md) +- [REAADB API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_active_active_database_api.md) +- [Networking configuration](/content/operate/kubernetes/8.2.0/networking/_index.md) diff --git a/content/operate/kubernetes/8.2.0/reference/yaml/basic-deployment.md b/content/operate/kubernetes/8.2.0/reference/yaml/basic-deployment.md new file mode 100644 index 0000000000..e1355bcf39 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/reference/yaml/basic-deployment.md @@ -0,0 +1,93 @@ +--- +Title: Basic deployment examples +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: YAML examples for basic Redis Enterprise deployment including RBAC, cluster, and database configurations. +linkTitle: Basic deployment +weight: 10 +url: '/operate/kubernetes/8.2.0/reference/yaml/basic-deployment/' +--- + +This page provides complete YAML examples for a basic Redis Enterprise deployment on Kubernetes. These examples include all the essential components you need to deploy a Redis Enterprise cluster and create a database. + +For complete deployment instructions, see [Deploy on Kubernetes](/content/operate/kubernetes/8.2.0/deployment/quick-start.md). + +## Service account + +The service account provides an identity for the Redis Enterprise operator. + +{{}} + +Service account configuration: +- `name`: The service account name used by the operator +- `labels`: Standard labels for Redis Enterprise resources + +## Role + +The Role defines the permissions needed by the Redis Enterprise operator within the namespace. + +{{}} + +Role configuration: +- `name`: Must match the role name referenced in the role binding +- `rules`: Comprehensive permissions for managing Redis Enterprise resources +- `apiGroups`: Includes core Kubernetes APIs and Redis Enterprise custom resources + +Key permissions: +- `app.redislabs.com`: Full access to Redis Enterprise custom resources +- `secrets`: Manage TLS certificates and database credentials +- `services`: Create and manage service endpoints +- `pods`: Monitor and manage Redis Enterprise pods +- `persistentvolumeclaims`: Manage persistent storage + +## Role binding + +The RoleBinding connects the service account to the role, granting the necessary permissions. + +{{}} + +Role binding configuration: +- `subjects.name`: Must match the service account name +- `roleRef.name`: Must match the role name +- `namespace`: Apply in the same namespace as other resources + +## Redis Enterprise cluster + +The RedisEnterpriseCluster (REC) custom resource defines the cluster specification. + +{{}} + +Cluster configuration: +- `metadata.name`: Cluster name (cannot be changed after creation) +- `spec.nodes`: Number of Redis Enterprise nodes (minimum 3) +- `persistentSpec.volumeSize`: Storage size per node +- `redisEnterpriseNodeResources`: CPU and memory allocation per node + +Edit the values in the downloaded YAML file based on your requirements, such as increasing the number of nodes, adjusting storage size, or modifying resource allocation. + +## Redis Enterprise database + +The RedisEnterpriseDatabase (REDB) custom resource defines the database specification. + +{{}} + +Database configuration: +- `metadata.name`: Database name +- `spec.memorySize`: Memory allocation for the database +- `spec.shardCount`: Number of shards (affects performance and scalability) +- `spec.replication`: Enable/disable database replication + +Edit the values in the downloaded YAML file based on your requirements, such as increasing memory for larger datasets, adding more shards for better performance, enabling replication for high availability, or adding Redis modules. + +## Apply the configuration + +To deploy these YAML files, follow [Deploy on Kubernetes](/content/operate/kubernetes/8.2.0/deployment/quick-start.md), which provides step-by-step instructions for creating namespaces, deploying the operator, and applying these configuration files. + +## Related documentation + +- [Deploy on Kubernetes](/content/operate/kubernetes/8.2.0/deployment/quick-start.md) +- [REC API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md) +- [REDB API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md) diff --git a/content/operate/kubernetes/8.2.0/reference/yaml/log-collector-rbac.md b/content/operate/kubernetes/8.2.0/reference/yaml/log-collector-rbac.md new file mode 100644 index 0000000000..dbbe7c8b47 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/reference/yaml/log-collector-rbac.md @@ -0,0 +1,155 @@ +--- +Title: Log collector RBAC examples +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: YAML examples for configuring RBAC permissions for the Redis Enterprise log collector tool in `restricted` and `all` modes. +linkTitle: Log collector RBAC +weight: 50 +url: '/operate/kubernetes/8.2.0/reference/yaml/log-collector-rbac/' +--- + +This page provides YAML examples for configuring RBAC permissions for the Redis Enterprise log collector tool. The log collector requires different permission levels depending on the collection mode you choose. + +For complete log collection instructions, see [Collect logs](/content/operate/kubernetes/8.2.0/logs/collect-logs.md). + +## Prerequisites + +- Install the [Redis Enterprise operator](/content/operate/kubernetes/8.2.0/deployment/_index.md) +- Appropriate permissions to create RBAC resources in target namespaces +- Understanding of your deployment model (single namespace, multi-namespace, etc.) + +## Collection modes + +The log collector has two collection modes that require different RBAC permissions: + +- **`restricted` mode** (recommended): Collects only Redis Enterprise resources with minimal security exposure. Default for versions 6.2.18-3+. +- **`all` mode**: Collects comprehensive cluster information including nodes, storage classes, and operator resources. Use when specifically requested by Redis Support. + +## `restricted` mode RBAC + +The `restricted` mode configuration provides minimal permissions for collecting Redis Enterprise resources only. + +{{}} + +`restricted` mode configuration: + +- `Role`: Namespace-scoped permissions for Redis Enterprise resources +- `ClusterRole`: Cluster-wide permissions for CRDs and basic cluster resources +- `rules`: Minimal permissions for Redis Enterprise diagnostics + +Key permissions: + +- `pods, pods/log, pods/exec`: Access to pod information and logs +- `app.redislabs.com/*`: All Redis Enterprise custom resources +- `persistentvolumes`: Storage information for troubleshooting + +## `all` mode RBAC + +The `all` mode configuration provides comprehensive permissions for collecting detailed cluster information. + +{{}} + +`all` mode configuration: + +- `Role`: Extended namespace permissions including operator resources +- `ClusterRole`: Additional cluster-wide permissions for nodes and storage +- `rules`: Comprehensive permissions for full cluster diagnostics + +Additional permissions in `all` mode: + +- `nodes`: Node information and status +- `storageclasses, volumeattachments`: Storage system details +- `operators.coreos.com/*`: OpenShift operator information +- `networking.istio.io/*`: Istio service mesh resources + +## Apply the configuration + +### Namespace requirements + +Create the Role and RoleBinding in every namespace where you need to collect logs: + +- Single namespace: Apply to the namespace where Redis Enterprise runs +- Multi-namespace with single REC: Apply to the REC namespace plus each REDB namespace +- Multi-namespace with multiple RECs: Apply to each REC namespace + +The ClusterRole and ClusterRoleBinding need to be created only once per cluster. + +Edit the values in the downloaded YAML file for your specific setup, updating the namespace references and role binding subjects to match your environment. + +### Role binding configuration + +The RBAC configurations include both roles and role bindings. The role bindings must reference the user or service account that will execute the log collector: + +- User subject: If running the log collector as a specific user, update the `subjects` section in the RoleBinding and ClusterRoleBinding to reference your username +- Service account: If using a service account, create or reference the appropriate service account in the role bindings + +### Manual deployment + +To apply the RBAC configurations manually: + +```bash +# Apply restricted mode RBAC +kubectl apply -f log-collector-restricted-rbac.yaml --namespace + +# Apply all mode RBAC +kubectl apply -f log-collector-all-rbac.yaml --namespace +``` + +## Usage + +After applying the RBAC configuration, run the log collector: + +```bash +# Restricted mode (default for 6.2.18-3+) +python log_collector.py -m restricted -n + +# All mode +python log_collector.py -m all -n +``` + +## Security considerations + +### Best practices + +- Use `restricted` mode unless you specifically need additional cluster information +- Limit namespace access to only where log collection is needed +- Handle collected data according to your organization's security policies + +### Secrets permission explanation + +The RBAC configurations request read access to secrets in the collected namespaces. **Secrets are not collected or included in the log package sent to Redis Support.** This permission is required because: + +- The log collector uses Helm commands (`helm list`, `helm get all`) to gather Redis Enterprise Helm chart deployment information +- Helm stores its deployment metadata in Kubernetes secrets +- This metadata contains only deployment configuration (not sensitive data) + +If your security policies prohibit secrets access, you can remove the secrets permission from the Role, but this will limit the log collector's ability to gather Helm deployment information. + +## Troubleshooting + +### Permission errors + +- Verify that roles and bindings are applied correctly in the target namespaces +- Check that the ClusterRole is applied cluster-wide +- Ensure the service account has proper role bindings + +### Missing resources + +- Consider switching to `all` mode if additional cluster resources are needed +- Verify that custom resource definitions are installed +- Check that the operator has proper permissions + +## Next steps + +- [Collect logs guide](/content/operate/kubernetes/8.2.0/logs/collect-logs.md) +- [Basic deployment examples](/content/operate/kubernetes/8.2.0/reference/yaml/basic-deployment.md) +- [Multi-namespace deployment](/content/operate/kubernetes/8.2.0/reference/yaml/multi-namespace.md) + +## Related documentation + +- [Kubernetes RBAC documentation](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) +- [Redis Enterprise troubleshooting](/content/operate/kubernetes/8.2.0/logs/_index.md) +- [Operator deployment guide](/content/operate/kubernetes/8.2.0/deployment/_index.md) diff --git a/content/operate/kubernetes/8.2.0/reference/yaml/multi-namespace.md b/content/operate/kubernetes/8.2.0/reference/yaml/multi-namespace.md new file mode 100644 index 0000000000..54d1f15bed --- /dev/null +++ b/content/operate/kubernetes/8.2.0/reference/yaml/multi-namespace.md @@ -0,0 +1,68 @@ +--- +Title: Multi-namespace examples +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: YAML examples for deploying Redis Enterprise across multiple Kubernetes namespaces. +linkTitle: Multi-namespace +weight: 40 +url: '/operate/kubernetes/8.2.0/reference/yaml/multi-namespace/' +--- + +Multi-namespace deployment lets a single Redis Enterprise operator manage clusters and databases in different namespaces, providing better resource isolation and organization. + +Multi-namespace deployment enables: +- Namespace isolation: Separate Redis Enterprise resources by team, environment, or application +- Centralized management: Single operator manages multiple namespaces +- Resource sharing: Efficient use of cluster resources across namespaces +- Flexible RBAC: Fine-grained permissions per namespace + +This example shows: +- Operator namespace: `redis-enterprise-operator` (where the operator and REC run) +- Consumer namespaces: `app-production`, `app-staging` (where REDB resources are created) + +For complete deployment instructions, see [Manage databases in multiple namespaces](/content/operate/kubernetes/8.2.0/re-clusters/multi-namespace.md). + +## Operator service account + +Deploy these resources in the namespace where the Redis Enterprise operator runs. + +{{}} + +## Operator cluster role + +Grant the operator cluster-wide permissions to manage resources across namespaces. + +{{}} + +## Operator cluster role binding + +{{}} + +## Consumer role + +{{}} + +## Consumer role binding + +{{}} + +Consumer namespace configuration: + +- `subjects.name`: Must match the operator service account name +- `subjects.namespace`: Must be the operator namespace, not the consumer namespace +- `roleRef.name`: Must match the consumer role name + +## Next steps + +- [Configure networking across namespaces](/content/operate/kubernetes/8.2.0/networking/_index.md) +- [Set up monitoring for multi-namespace deployment](/content/operate/kubernetes/8.2.0/re-clusters/connect-prometheus-operator.md) +- [Learn about resource management](/content/operate/kubernetes/8.2.0/recommendations/_index.md) + +## Related documentation + +- [Manage databases in multiple namespaces](/content/operate/kubernetes/8.2.0/re-clusters/multi-namespace.md) +- [RBAC configuration](/content/operate/kubernetes/8.2.0/security/_index.md) +- [Kubernetes namespaces](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/) diff --git a/content/operate/kubernetes/8.2.0/reference/yaml/rack-awareness.md b/content/operate/kubernetes/8.2.0/reference/yaml/rack-awareness.md new file mode 100644 index 0000000000..fc8e023819 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/reference/yaml/rack-awareness.md @@ -0,0 +1,132 @@ +--- +Title: Rack awareness examples +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: YAML examples for rack-aware Redis Enterprise deployments that distribute nodes across availability zones. +linkTitle: Rack awareness +weight: 20 +url: '/operate/kubernetes/8.2.0/reference/yaml/rack-awareness/' +--- + +This page provides YAML examples for deploying Redis Enterprise with [rack awareness](/content/operate/kubernetes/8.2.0/recommendations/node-selection.md#using-rack-awareness). Rack awareness distributes Redis Enterprise nodes and database shards across different availability zones or failure domains to improve high availability and fault tolerance. + +## Prerequisites + +- Label [Kubernetes nodes](https://kubernetes.io/docs/concepts/architecture/nodes/) with zone information +- Typically uses the standard label `topology.kubernetes.io/zone` +- Verify node labels: `kubectl get nodes -o custom-columns="name:metadata.name","rack\\zone:metadata.labels.topology\.kubernetes\.io/zone"` +- Install the [Redis Enterprise operator](/content/operate/kubernetes/8.2.0/deployment/_index.md) + +For complete deployment instructions, see [Deploy on Kubernetes](/content/operate/kubernetes/8.2.0/deployment/_index.md). + +## Service account + +The service account for rack-aware deployments is the same as [basic deployments](/content/operate/kubernetes/8.2.0/reference/yaml/basic-deployment.md#service-account). + +{{}} + +## Cluster role + +Rack awareness requires additional permissions to read [node labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/) across the cluster. + +{{}} + +Cluster role configuration: + +- `name`: ClusterRole name for rack awareness permissions +- `rules`: Permissions to read nodes and their labels cluster-wide +- `resources`: Access to `nodes` resource for zone label discovery + +Key permissions: + +- `nodes`: Read access to discover node zone labels +- `get, list, watch`: Monitor node changes and zone assignments + +## Cluster role binding + +The [ClusterRoleBinding](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#rolebinding-and-clusterrolebinding) grants cluster-wide permissions to the service account. + +{{}} + +Cluster role binding configuration: + +- `subjects.name`: Must match the service account name +- `subjects.namespace`: Namespace where the operator is deployed +- `roleRef.name`: Must match the cluster role name + +## Rack-aware Redis Enterprise cluster + +The rack-aware [REC configuration](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md) includes the `rackAwarenessNodeLabel` field. + +{{}} + +Rack-aware cluster configuration: + +- `metadata.name`: Cluster name (cannot be changed after creation) +- `spec.rackAwarenessNodeLabel`: Node label used for zone identification +- `spec.nodes`: Minimum 3 nodes, ideally distributed across zones + +Edit the values in the downloaded YAML file based on your environment, such as increasing nodes for better zone distribution, using custom zone labels, adding resource specifications, or enabling persistent storage. + +### Common zone labels + +Different Kubernetes distributions use different zone labels: + +- `Standard`: `topology.kubernetes.io/zone` +- `Legacy`: `failure-domain.beta.kubernetes.io/zone` +- `Custom`: Your organization's specific labeling scheme + +Verify the correct label on your nodes: + +```bash +kubectl get nodes -o custom-columns="name:metadata.name","rack\\zone:metadata.labels.topology\.kubernetes\.io/zone" +``` + +## Redis Enterprise database + +Database configuration for rack-aware clusters is the same as [basic deployments](/content/operate/kubernetes/8.2.0/reference/yaml/basic-deployment.md#redis-enterprise-database). + +**Important**: For rack awareness to be effective, ensure your database has replication enabled. Rack awareness distributes primary and replica shards across zones, so databases without replication will not benefit from zone distribution. + +{{}} + +## Apply the configuration + +To deploy rack-aware Redis Enterprise clusters, follow [Deploy on Kubernetes](/content/operate/kubernetes/8.2.0/deployment/_index.md) and ensure your Kubernetes nodes have proper zone labels. For detailed rack awareness configuration, see the [node selection recommendations](/content/operate/kubernetes/8.2.0/recommendations/node-selection.md). + +## Troubleshooting + +### Nodes not distributed across zones + +- Verify node labels are correct +- Check that sufficient nodes exist in each zone +- Ensure the `rackAwarenessNodeLabel` matches actual node labels + +### Cluster role permissions denied + +- Verify the ClusterRole and ClusterRoleBinding are applied +- Check that the service account name matches in all resources + +### Database shards not distributed + +- Confirm the cluster has rack awareness enabled +- **Check that the database has replication enabled** - rack awareness distributes primary/replica pairs across zones +- Verify the database has multiple shards +- Ensure sufficient nodes exist across zones + +## Next steps + +- [Configure Active-Active databases](/content/operate/kubernetes/8.2.0/reference/yaml/active-active.md) +- [Set up multi-namespace deployment](/content/operate/kubernetes/8.2.0/reference/yaml/multi-namespace.md) +- [Learn about database replication](/content/operate/kubernetes/8.2.0/re-databases/replica-redb.md) + +## Related documentation + +- [Node selection recommendations](/content/operate/kubernetes/8.2.0/recommendations/node-selection.md) +- [REC API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md) +- [REDB API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md) +- [Kubernetes node affinity](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/) +- [Redis Enterprise cluster architecture](/content/operate/kubernetes/8.2.0/architecture/_index.md) diff --git a/content/operate/kubernetes/8.2.0/security/_index.md b/content/operate/kubernetes/8.2.0/security/_index.md new file mode 100644 index 0000000000..4ef1c77ee7 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/security/_index.md @@ -0,0 +1,39 @@ +--- +Title: Security +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Configure security settings for Redis Software clusters and databases on Kubernetes. +hideListLinks: true +linkTitle: Security +weight: 50 +url: '/operate/kubernetes/8.2.0/security/' +--- + +Configure security settings for Redis for Kubernetes. Security covers access control, cluster credentials, external identity providers, TLS certificates and encryption, and external secret management. + +## Access control + +- [Access control](/content/operate/kubernetes/8.2.0/security/access-control/_index.md) — manage Redis Software users, roles, ACLs, and role bindings as Kubernetes custom resources. + +## Authentication + +- [Authentication](/content/operate/kubernetes/8.2.0/security/authentication/_index.md) — manage cluster credentials, LDAP, SAML SSO, and configuration secrets. + +## Certificates and encryption + +- [Certificates and encryption](/content/operate/kubernetes/8.2.0/security/certificates/_index.md) — provision TLS certificates, integrate cert-manager, add client certificates, and enable internode encryption. + +## Secret management + +- [HashiCorp Vault integration](/content/operate/kubernetes/8.2.0/security/vault.md) — use HashiCorp Vault as the centralized secret store for Redis for Kubernetes. + +## Resource management + +- [Allow resource adjustment](/content/operate/kubernetes/8.2.0/security/allow-resource-adjustment.md) — enable automatic adjustment of system resources for security compliance. + +## Compliance + +- [FIPS compliance](/content/operate/kubernetes/8.2.0/security/fips.md) — run your cluster in FIPS 140-3 compliance mode. diff --git a/content/operate/kubernetes/8.2.0/security/access-control/_index.md b/content/operate/kubernetes/8.2.0/security/access-control/_index.md new file mode 100644 index 0000000000..f0b1db2938 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/security/access-control/_index.md @@ -0,0 +1,150 @@ +--- +Title: Access control +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Manage Redis Software users, roles, ACLs, and role bindings on Kubernetes with custom resources. +hideListLinks: true +linkTitle: Access control +weight: 20 +url: '/operate/kubernetes/8.2.0/security/access-control/' +--- + +Access control lets you manage Redis Software users, roles, ACLs, and role bindings as Kubernetes custom resources. The operator reconciles each resource into the corresponding Redis Software object, so you can use GitOps workflows and Kubernetes Secrets instead of working only through the Redis Software REST API or Cluster Manager UI. + +## How access control works on Redis Software for Kubernetes + +You declare these `app.redislabs.com/v1alpha1` custom resources: + +| Resource | Scope | Purpose | +| --- | --- | --- | +| `RedisEnterpriseUser` | — | A Redis Software user, with credentials in a Kubernetes Secret. | +| `RedisEnterpriseACL` | — | A Redis ACL rule, mapped to a Redis Software ACL object. | +| `RedisEnterpriseRole` | Database | A management role and/or ACL applied to one or more REDBs selected by `spec.scopes`. | +| `RedisEnterpriseRoleBinding` | Database | Assigns a `RedisEnterpriseRole` to a user. | +| `RedisEnterpriseClusterRole` | Cluster | A management role and/or ACL applied across every REDB in the cluster. | +| `RedisEnterpriseClusterRoleBinding` | Cluster | Assigns a `RedisEnterpriseClusterRole` to a user. | + +When you apply one of these resources, the operator: + +1. Validates the spec. +2. Creates or updates the matching object in Redis Software. +3. Reports the resolved Redis Software UID and other state in the resource's `status`. +4. Emits Kubernetes events on reconciliation problems. + +## Roles and bindings + +The role and binding CRDs follow the same pattern as Kubernetes' own RBAC: a `Role` paired with a `RoleBinding` for the narrower scope, and a `ClusterRole` paired with a `ClusterRoleBinding` for cluster-wide access. The narrower scope is the unqualified default — that's why `RedisEnterpriseRole` (no qualifier) is the database-scoped kind, while `RedisEnterpriseClusterRole` carries the explicit `Cluster` prefix. + +### Database scope vs. cluster scope + +| | `RedisEnterpriseRole` | `RedisEnterpriseClusterRole` | +| --- | --- | --- | +| Scope | One or more REDBs | Every REDB in the cluster | +| Selects targets via | `spec.scopes` (REDB name or label selector) — required | No selector; applies cluster-wide | +| `managementRole` values | `DBMember`, `DBViewer`, `None` | `Admin`, `ClusterMember`, `ClusterViewer`, `DBMember`, `DBViewer`, `UserManager`, `None` | +| Binding kind | `RedisEnterpriseRoleBinding` | `RedisEnterpriseClusterRoleBinding` | + +A `RedisEnterpriseClusterRole` applies to every REDB in the cluster, including REDBs represented by resources in other namespaces. The operator applies the role to each REDB individually, so it doesn't reach databases created directly through the Redis Software REST API without a matching REDB resource. + +### What a role grants + +Every role carries permissions on two independent planes. Set either, or both: + +- **`spec.managementRole`** — Redis Software API and Cluster Manager UI permissions, chosen from the built-in roles listed in the table above. Same set of roles you'd assign in Cluster Manager today. +- **`spec.acl`** — a single `RedisEnterpriseACL` reference. The ACL controls Redis data-path access (commands, key patterns, categories). To apply different data-path access to different databases, create a separate role for each. + +### How a user gets permissions + +`RedisEnterpriseUser.spec` has no role references. Permissions reach a user through a binding: + +1. Create a `RedisEnterpriseACL` if you need data-path access. +2. Create a `RedisEnterpriseRole` or `RedisEnterpriseClusterRole` that sets `managementRole`, references the ACL, or both. +3. Create a `RedisEnterpriseRoleBinding` or `RedisEnterpriseClusterRoleBinding` whose `roleRef` points at the role and whose `subjects` list includes the user. + +The user's effective roles appear in `status.roles`. A user with no binding gets the Redis Software `none` role so it's never roleless, but it has zero permissions until a binding lands. + +### End-to-end example + +End-to-end: an ACL, a database-scoped role that uses it, a binding that hands the role to a user, and the user itself. All four resources live in the operator namespace. + +```yaml +--- +apiVersion: app.redislabs.com/v1alpha1 +kind: RedisEnterpriseACL +metadata: + name: read-only +spec: + acl: "+@read ~*" +--- +apiVersion: app.redislabs.com/v1alpha1 +kind: RedisEnterpriseRole +metadata: + name: orders-viewer +spec: + managementRole: DBViewer + scopes: + - name: orders + acl: + name: read-only +--- +apiVersion: app.redislabs.com/v1alpha1 +kind: RedisEnterpriseRoleBinding +metadata: + name: alice-orders-viewer +spec: + roleRef: + name: orders-viewer + subjects: + - name: alice +--- +apiVersion: app.redislabs.com/v1alpha1 +kind: RedisEnterpriseUser +metadata: + name: alice +spec: + email: alice@example.com + username: alice + passwordSecrets: + - name: alice-password +``` + +After applying this and a Secret named `alice-password` with a `password` key, Alice can sign in to Redis Software with `DBViewer` permissions on the `orders` REDB and run read-only Redis commands on every key in that database. + +## What's the same as Redis Software + +The underlying Redis Software behavior is unchanged. For concepts and reference details, see the existing Redis Software docs: + +- [Cluster-scoped role definitions](/content/operate/rs/security/access-control/create-cluster-roles.md) — what `Admin`, `ClusterMember`, `ClusterViewer`, and `UserManager` grant. +- [Database-scoped role definitions](/content/operate/rs/security/access-control/create-db-roles.md) — what `DBMember` and `DBViewer` grant. +- [Combined cluster and database roles](/content/operate/rs/security/access-control/create-combined-roles.md) — when a role grants both planes. +- [Redis ACL syntax](/content/operate/rs/security/access-control/redis-acl-overview.md) — rule format for `RedisEnterpriseACL` resources. +- [Login lockout and unlock](/content/operate/rs/security/access-control/manage-users/login-lockout.md) — how locked users are recovered. +- [Password complexity rules](/content/operate/rs/security/access-control/manage-passwords/password-complexity-rules.md) and [password expiration](/content/operate/rs/security/access-control/manage-passwords/password-expiration.md) — applied by Redis Software regardless of how the password is delivered. +- [Default user](/content/operate/rs/security/access-control/manage-users/default-user.md) — the built-in cluster admin account. + +## What's different on Kubernetes + +- **Resources are declarative.** You define users, roles, ACLs, and bindings in YAML and let the operator apply them. The Cluster Manager UI and REST API still work but are no longer the source of truth. +- **Role assignment lives on the binding, not the user.** In Redis Software, you assign roles by editing the user. On Kubernetes, you create a separate `RedisEnterpriseRoleBinding` or `RedisEnterpriseClusterRoleBinding`. See [Roles and bindings](#roles-and-bindings). +- **Database and cluster scope are separate objects.** A `RedisEnterpriseRole` covers database scope and a `RedisEnterpriseClusterRole` covers cluster scope. In Redis Software, a single role object carries both. +- **Passwords live in Kubernetes Secrets.** Each `RedisEnterpriseUser` references one or more Secrets. A `Rotatable` mode supports two Secrets at once for zero-downtime rotation. The operator marks Kubernetes Secrets immutable to prevent in-place edits. + +## Known limitations + +Access control resources are reconciled only in the operator namespace. Password Secrets must live in the same namespace, and database scopes resolve to REDBs in that namespace. + +## In this section + +- [Manage users](/content/operate/kubernetes/8.2.0/security/access-control/manage-users.md) — create `RedisEnterpriseUser` resources, rotate passwords, recover from lockouts. +- [Manage roles](/content/operate/kubernetes/8.2.0/security/access-control/manage-roles.md) — create database and cluster roles with the right scope and management permissions. +- [Manage ACLs](/content/operate/kubernetes/8.2.0/security/access-control/manage-acls.md) — create and update `RedisEnterpriseACL` resources used by roles. +- [Manage role bindings](/content/operate/kubernetes/8.2.0/security/access-control/manage-bindings.md) — assign roles to users with `RedisEnterpriseRoleBinding` and `RedisEnterpriseClusterRoleBinding`. +- [Migrate from REDB rolesPermissions](/content/operate/kubernetes/8.2.0/security/access-control/migrate-rolespermissions.md) — move from the deprecated `RedisEnterpriseDatabase.spec.rolesPermissions` field to the new CRD model. + +## Related topics + +- [Redis Software for Kubernetes operator API reference](/content/operate/kubernetes/8.2.0/reference/api/_index.md) — field-by-field specification for every CRD in the `app.redislabs.com/v1alpha1` group. +- [Redis databases (REDB)](/content/operate/kubernetes/8.2.0/re-databases/_index.md) — the resources that role scopes resolve against. diff --git a/content/operate/kubernetes/8.2.0/security/access-control/manage-acls.md b/content/operate/kubernetes/8.2.0/security/access-control/manage-acls.md new file mode 100644 index 0000000000..8118326a28 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/security/access-control/manage-acls.md @@ -0,0 +1,121 @@ +--- +Title: Manage ACLs +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Create and update RedisEnterpriseACL resources to control Redis data-path permissions on Kubernetes. +linkTitle: Manage ACLs +weight: 30 +url: '/operate/kubernetes/8.2.0/security/access-control/manage-acls/' +--- + +A `RedisEnterpriseACL` resource holds a Redis ACL rule that controls which commands, keys, and categories a user can access at the Redis data path. The operator reconciles the resource into a Redis Software ACL object that roles can reference. + +ACLs are reusable: one `RedisEnterpriseACL` can be attached to any number of `RedisEnterpriseRole` or `RedisEnterpriseClusterRole` resources. The role decides which databases the ACL applies to; the ACL itself just defines the rule. + +To grant a user the permissions in an ACL, reference the ACL from a role and bind the role to the user. See [Manage roles](/content/operate/kubernetes/8.2.0/security/access-control/manage-roles.md) and [Manage role bindings](/content/operate/kubernetes/8.2.0/security/access-control/manage-bindings.md). + +## Before you start + +- Requires Redis Software for Kubernetes operator 8.2.0-12 or later. +- The `RedisEnterpriseACL` resource must live in the operator namespace. +- The rule string uses Redis ACL syntax — key patterns, command categories, and explicit commands. See [Redis ACL overview](/content/operate/rs/security/access-control/redis-acl-overview.md) for the full syntax. + +## Create an ACL + +`spec.acl` is a single Redis ACL rule string. + +```yaml +apiVersion: app.redislabs.com/v1alpha1 +kind: RedisEnterpriseACL +metadata: + name: read-only +spec: + acl: "+@read ~*" +``` + +Apply the resource and confirm reconciliation: + +```sh +kubectl apply -f read-only-acl.yaml +kubectl get redisenterpriseacl read-only -o yaml +``` + +`status.uid` holds the Redis Software ACL UID once the operator has created the object. + +### Common rule patterns + +| Use case | Rule | +| --- | --- | +| Read-only access to all keys | `+@read ~*` | +| Read and write access to all keys | `+@all ~*` | +| Read-only access to a key prefix | `+@read ~customer:*` | +| A specific command set | `+get +set +del ~app:*` | +| Block dangerous commands | `+@all -@dangerous ~*` | + +For category names (`@read`, `@write`, `@admin`, `@dangerous`, etc.) and the full operator precedence rules, see [Redis ACL overview](/content/operate/rs/security/access-control/redis-acl-overview.md). + +## Update an ACL + +Edit `spec.acl` and re-apply. The operator updates the Redis Software ACL object in place, so the change immediately affects every database that uses the ACL through a role. + +```sh +kubectl edit redisenterpriseacl read-only +``` + +`status.observedGeneration` reaches the resource's `metadata.generation` once the update has been applied. + +An update changes effective permissions for every connected user. Treat ACL changes the same way as role changes: validate them on a non-production cluster first. For a safer rollout, create a new ACL and swap the role reference instead of editing the existing ACL in place. + +## Inspect ACL status + +The `status` block is minimal: + +| Field | Meaning | +| --- | --- | +| `uid` | Internal Redis Software ACL UID. Present once reconciliation succeeds. | +| `observedGeneration` | The `metadata.generation` the operator last acted on. | + +To find roles that reference this ACL, scan the role resources: + +```sh +kubectl get redisenterpriserole -o yaml | \ + yq '.items[] | select(.spec.acl.name == "read-only") | .metadata.name' +kubectl get redisenterpriseclusterrole -o yaml | \ + yq '.items[] | select(.spec.acl.name == "read-only") | .metadata.name' +``` + +## Delete an ACL + +Delete or modify any roles that reference the ACL first (modify them to no longer reference it), then delete the ACL itself: + +```sh +kubectl delete redisenterpriseacl read-only +``` + +If a role still references the ACL, Redis Software may reject the deletion. The operator emits an `RSOperationFailed` event with the underlying message. Remove the reference (or delete the role) and retry. + +## Troubleshoot + +Watch reconciliation events with `kubectl describe redisenterpriseacl `. Common events: + +| Event | Meaning | +| --- | --- | +| `RSObjectNotFound` | The Redis Software ACL the resource previously resolved against no longer exists. The operator will recreate it on the next reconcile. | +| `RSOperationFailed` | A Redis Software API call failed. The message typically includes the syntax error or in-use conflict. | + +Other things to check: + +- **`status.uid` is empty** — The operator hasn't reconciled the ACL yet, or Redis Software rejected the rule. Check the events for an `RSOperationFailed` with the syntax message. +- **Rule parses but grants nothing** — A common cause is an explicit `-@all` later in the rule overriding earlier `+` clauses. Redis ACL evaluation is order-sensitive; see the [Redis ACL overview](/content/operate/rs/security/access-control/redis-acl-overview.md). +- **Delete is blocked** — A `RedisEnterpriseRole` or `RedisEnterpriseClusterRole` still references the ACL in `spec.acl`. Remove the reference or delete the role first. + +For full field details, see the [`RedisEnterpriseACL`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_acl_api.md) API reference. + +## Related topics + +- [Redis ACL overview](/content/operate/rs/security/access-control/redis-acl-overview.md) — rule syntax, categories, and evaluation order. +- [Manage roles](/content/operate/kubernetes/8.2.0/security/access-control/manage-roles.md) — attach ACLs to `RedisEnterpriseRole` and `RedisEnterpriseClusterRole` resources. +- [Manage role bindings](/content/operate/kubernetes/8.2.0/security/access-control/manage-bindings.md) — grant the role to a user. diff --git a/content/operate/kubernetes/8.2.0/security/access-control/manage-bindings.md b/content/operate/kubernetes/8.2.0/security/access-control/manage-bindings.md new file mode 100644 index 0000000000..f46c13ab1c --- /dev/null +++ b/content/operate/kubernetes/8.2.0/security/access-control/manage-bindings.md @@ -0,0 +1,174 @@ +--- +Title: Manage role bindings +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Assign roles to users with RedisEnterpriseRoleBinding and RedisEnterpriseClusterRoleBinding resources. +linkTitle: Manage role bindings +weight: 40 +url: '/operate/kubernetes/8.2.0/security/access-control/manage-bindings/' +--- + +A role binding assigns a role to one or more users. Redis Software for Kubernetes supports two binding kinds, one for database-scoped roles and one for cluster-scoped roles: + +- `RedisEnterpriseRoleBinding` — binds a `RedisEnterpriseRole` (database-scoped) to users. +- `RedisEnterpriseClusterRoleBinding` — binds a `RedisEnterpriseClusterRole` (cluster-scoped) to users. + +Both have the same two spec fields: `roleRef` points at a single role, and `subjects` lists the users who receive it. A user holds the permissions defined by every role bound to it across all bindings. + +For the conceptual model and a complete end-to-end example, see [Roles and bindings](/content/operate/kubernetes/8.2.0/security/access-control/_index.md#roles-and-bindings). + +## Common patterns + +Each binding references exactly one role through `roleRef` and lists up to 100 subjects. A user can be the subject of many bindings, so you grant a user multiple roles by creating multiple bindings. Beyond that, the CRD model doesn't enforce a granularity — pick one of these patterns and stick with it across the namespace to make audits and reviews predictable: + +- **One binding per role, many subjects** — every user with the role lives in one resource. A single apply changes access for every user at once, which can be either a feature or a hazard depending on the change. +- **One binding per user-role pair** — most verbose, but each grant is a discrete resource. Useful for attributing changes in GitOps and for scoping Kubernetes RBAC permissions on individual bindings. + +## Before you start + +- Requires Redis Software for Kubernetes operator 8.2.0-12 or later. +- The binding resource, the role it references, and any `RedisEnterpriseUser` subjects must all live in the operator namespace. +- Create the role before the binding when possible. The operator still admits a binding that references a missing role — the binding stays unresolved until the role lands. + +## Create a database role binding + +`RedisEnterpriseRoleBinding` references a `RedisEnterpriseRole` and a list of subjects: + +```yaml +apiVersion: app.redislabs.com/v1alpha1 +kind: RedisEnterpriseRoleBinding +metadata: + name: alice-orders-viewer +spec: + roleRef: + kind: RedisEnterpriseRole + name: orders-viewer + subjects: + - kind: RedisEnterpriseUser + name: alice +``` + +`kind` on `roleRef` defaults to `RedisEnterpriseRole` and can be omitted. Subjects default to `RedisEnterpriseUser` the same way. + +To grant the same role to several users, list them as separate subjects: + +```yaml +spec: + roleRef: + name: orders-viewer + subjects: + - name: alice + - name: bob + - name: carol +``` + +### Apply order doesn't matter + +The operator admits a binding even when the role it references doesn't exist yet. The binding stays unresolved until the role lands, and the user picks up the role on the next reconcile. This lets `kubectl apply -f manifests/` work regardless of file order. + +While the role is missing, the affected user's `RolesBound` condition is `False` with reason `RoleNotFound`: + +```sh +kubectl get redisenterpriseuser alice -o jsonpath='{.status.conditions[?(@.type=="RolesBound")]}' +``` + +The condition flips to `True` after the missing role exists and reconciles. + +## Create a cluster role binding + +`RedisEnterpriseClusterRoleBinding` has the same shape but references a `RedisEnterpriseClusterRole`: + +```yaml +apiVersion: app.redislabs.com/v1alpha1 +kind: RedisEnterpriseClusterRoleBinding +metadata: + name: ops-admins +spec: + roleRef: + kind: RedisEnterpriseClusterRole + name: cluster-admin + subjects: + - name: alice + - name: bob +``` + +A cluster role binding grants the role across every REDB in the cluster, including REDBs represented by resources in other namespaces. Reserve cluster role bindings for administrators who need cluster-wide access. + +## Subjects + +Each entry in `spec.subjects` references a `RedisEnterpriseUser` in the operator namespace. `spec.subjects[].kind` can be left empty or set to `RedisEnterpriseUser`, and `name` is the resource's `metadata.name`: + +```yaml +spec: + subjects: + - kind: RedisEnterpriseUser + name: alice + - name: bob # kind defaults to RedisEnterpriseUser +``` + +A binding can list up to 100 subjects. To grant the role to more users, create additional bindings that reference the same role. + +## Update a binding + +Edit `spec.subjects` to add or remove users from the role, or change `roleRef` to point at a different role. + +- **Add a user** — append a new entry to `subjects` and apply. The user picks up the role on the next reconcile. +- **Remove a user** — delete the entry from `subjects`. The user loses the role; other subjects in the binding keep their access. +- **Replace the role** — change `roleRef.name`. Subjects lose the old role and gain the new one. Consider creating a new binding instead if you need a staged rollout. + +A user's effective roles appear in `status.roles` on the `RedisEnterpriseUser`. + +## Find bindings that reference a role or user + +Before renaming or deleting a role, find every binding that points at it: + +```sh +kubectl get redisenterpriserolebinding -o json | \ + jq '.items[] | select(.spec.roleRef.name == "orders-viewer") | .metadata.name' +kubectl get redisenterpriseclusterrolebinding -o json | \ + jq '.items[] | select(.spec.roleRef.name == "orders-viewer") | .metadata.name' +``` + +To find every binding that grants a role to a specific user: + +```sh +kubectl get redisenterpriserolebinding -o json | \ + jq '.items[] | select(.spec.subjects[]?.name == "alice") | .metadata.name' +kubectl get redisenterpriseclusterrolebinding -o json | \ + jq '.items[] | select(.spec.subjects[]?.name == "alice") | .metadata.name' +``` + +## Inspect binding status + +The binding's own `status` block is empty. To see whether a binding has actually granted permissions, check the referenced user instead: + +```sh +kubectl get redisenterpriseuser alice -o jsonpath='{.status.roles}' +``` + +## Delete a binding + +```sh +kubectl delete redisenterpriserolebinding alice-orders-viewer +``` + +Removing the binding revokes the role from every subject listed in it. If you delete the only binding that granted a user any role, the operator falls back to assigning the `none` role so the user is never roleless. + +## Troubleshoot + +Watch reconciliation events with `kubectl describe redisenterpriserolebinding ` (or `redisenterpriseclusterrolebinding`). Since bindings are observed by the user controller, the most useful diagnostic signals appear on the user resource: + +- **The user's `RolesBound` condition is `False` with reason `RoleNotFound`** — A binding references this user but points at a role that doesn't exist. Either create the missing role or fix `roleRef.name`. See [Apply order doesn't matter](#apply-order-doesnt-matter). +- **User has permissions you didn't expect** — Multiple bindings may be granting the same user different roles. Use the recipes in [Find bindings that reference a role or user](#find-bindings-that-reference-a-role-or-user) to list everything that targets the user. +- **`MissingRoleUIDs` event on a user** — Redis Software has role UIDs assigned to the user that the operator can't trace back to any Kubernetes role resource. This typically means roles were granted directly through the Redis Software REST API or Cluster Manager UI, bypassing the CRD model. Recreate the assignment as a `RedisEnterpriseRoleBinding` (or `RedisEnterpriseClusterRoleBinding`) so the CRD model is the source of truth, then revoke the direct assignment. + +For full field details, see the [`RedisEnterpriseRoleBinding`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_role_binding_api.md) and [`RedisEnterpriseClusterRoleBinding`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_role_binding_api.md) API reference. + +## Related topics + +- [Roles and bindings](/content/operate/kubernetes/8.2.0/security/access-control/_index.md#roles-and-bindings) — the conceptual model and an end-to-end example. +- [Manage roles](/content/operate/kubernetes/8.2.0/security/access-control/manage-roles.md) — create the roles that a binding references. +- [Manage users](/content/operate/kubernetes/8.2.0/security/access-control/manage-users.md) — create the users that a binding lists as subjects. diff --git a/content/operate/kubernetes/8.2.0/security/access-control/manage-roles.md b/content/operate/kubernetes/8.2.0/security/access-control/manage-roles.md new file mode 100644 index 0000000000..d0d6568219 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/security/access-control/manage-roles.md @@ -0,0 +1,200 @@ +--- +Title: Manage roles +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Create RedisEnterpriseRole and RedisEnterpriseClusterRole resources to grant Redis Software permissions on Kubernetes. +linkTitle: Manage roles +weight: 20 +url: '/operate/kubernetes/8.2.0/security/access-control/manage-roles/' +--- + +A role defines a reusable set of Redis Software permissions — Cluster Manager and API access, Redis data-path access, or both — that you grant to users by creating a binding. Redis Software for Kubernetes supports two role kinds, one scoped to one or more databases, the other scoped to the entire cluster: + +- `RedisEnterpriseRole` — applies to one or more REDBs selected by `spec.scopes`. Use when you want to grant access to a specific database or set of databases. +- `RedisEnterpriseClusterRole` — applies cluster-wide, across every REDB. Use for administrative access or for permissions you want everywhere. + +For details on how roles and bindings work together, see [Roles and bindings](/content/operate/kubernetes/8.2.0/security/access-control/_index.md#roles-and-bindings). To assign a role to a user, see [Manage role bindings](/content/operate/kubernetes/8.2.0/security/access-control/manage-bindings.md). + +## Before you start + +- Requires Redis Software for Kubernetes operator 8.2.0-12 or later. +- The role resource must live in the operator namespace. Database scopes resolve to REDBs in that namespace. +- Decide whether you need [management permissions](#choose-a-management-role), [data-path permissions](#attach-an-acl), or both. +- If the role references one or more `RedisEnterpriseACL` resources, create those first. See [Manage ACLs](/content/operate/kubernetes/8.2.0/security/access-control/manage-acls.md). + +## Choose a management role + +`spec.managementRole` picks a Redis Software built-in role that controls API and Cluster Manager UI permissions. The allowed values differ by CRD: + +| CRD | Allowed `managementRole` values | +| --- | --- | +| `RedisEnterpriseRole` | `DBMember`, `DBViewer`, `None` | +| `RedisEnterpriseClusterRole` | `Admin`, `ClusterMember`, `ClusterViewer`, `DBMember`, `DBViewer`, `UserManager`, `None` | + +`None` grants no management permissions and is the default when `managementRole` is omitted. For what each Redis Software role grants, see [Cluster-scoped role definitions](/content/operate/rs/security/access-control/create-cluster-roles.md) and [Database-scoped role definitions](/content/operate/rs/security/access-control/create-db-roles.md). + +## Create a database role + +A `RedisEnterpriseRole` must reference at least one database in `spec.scopes`. Each scope picks REDBs by name or by label selector — not both. + +### Scope a role by REDB name + +```yaml +apiVersion: app.redislabs.com/v1alpha1 +kind: RedisEnterpriseRole +metadata: + name: orders-viewer +spec: + managementRole: DBViewer + scopes: + - kind: RedisEnterpriseDatabase + name: orders + acl: + kind: RedisEnterpriseACL + name: read-only +``` + +`kind` defaults to `RedisEnterpriseDatabase` and can be omitted. + +To scope a role to several databases by name, list each one as its own scope entry: + +```yaml +spec: + managementRole: DBViewer + scopes: + - name: orders + - name: customers + - name: inventory + acl: + name: read-only +``` + +The role's `spec.acl` applies to every REDB in `spec.scopes`. If a database needs a different ACL, create a separate role for it. + +### Scope a role by label selector + +Use a selector when you want the role to follow a set of REDBs that share labels, rather than naming each one: + +```yaml +apiVersion: app.redislabs.com/v1alpha1 +kind: RedisEnterpriseRole +metadata: + name: prod-db-viewer +spec: + managementRole: DBViewer + scopes: + - selector: + matchLabels: + environment: production + acl: + name: read-only +``` + +`selector.matchExpressions` is also supported. + +### Scope rules + +- At least one entry in `spec.scopes` is required. +- Each scope must set `name` or `selector`, not both. +- `scopes[].kind` must be `RedisEnterpriseDatabase` or empty. + +## Create a cluster role + +A `RedisEnterpriseClusterRole` has no `scopes` field — it applies across every REDB in the Redis Software cluster. + +```yaml +apiVersion: app.redislabs.com/v1alpha1 +kind: RedisEnterpriseClusterRole +metadata: + name: support-readonly +spec: + managementRole: ClusterViewer + acl: + name: read-only +``` + +Common patterns: + +- **Read-only operator** — `managementRole: ClusterViewer`, no ACL. +- **Cluster admin** — `managementRole: Admin`, no ACL. Use sparingly. +- **Cluster-wide data access** — `managementRole: None` (or omit) with an `acl`. The ACL applies to every REDB in the cluster. + +## Attach an ACL + +Both role kinds carry a single `RedisEnterpriseACL` reference in `spec.acl`. The ACL grants Redis data-path permissions (commands, key patterns, categories) to users who hold the role. + +```yaml +spec: + acl: + kind: RedisEnterpriseACL + name: read-only # kind defaults to RedisEnterpriseACL and can be omitted +``` + +Rules: + +- `acl.kind` must be `RedisEnterpriseACL` or empty. +- For a `RedisEnterpriseRole`, the ACL applies to every database the role's scopes select. If you need different data-path access for different databases, create a separate role for each. +- For a `RedisEnterpriseClusterRole`, the ACL applies to every REDB in the cluster. + +Set `spec.managementRole` alone, `spec.acl` alone, or both. A role with neither set effectively grants nothing. + +## Update a role + +`kubectl apply` (or `kubectl edit`) updates the underlying Redis Software role. The operator reconciles changes to: + +- `managementRole` — replaces the management permission set on the Redis Software role. +- `scopes` — re-resolves which REDBs the role attaches to. REDBs that drop out of the scope have the role's permissions removed. +- `acl` — re-applies the data-path permissions to scoped REDBs (or cluster-wide for cluster roles). + +`status.observedGeneration` reaches the resource's `metadata.generation` once the update has been applied. + +## Inspect role status + +The `status` block is intentionally minimal: + +| Field | Meaning | +| --- | --- | +| `uid` | Internal Redis Software role UID. Present once the role has reconciled successfully. A role must have a `uid` before it can contribute permissions to any database. | +| `observedGeneration` | The `metadata.generation` the operator last acted on. Compare with `metadata.generation` to confirm the latest spec has been processed. | + +To see which users currently hold the role, list bindings that reference it: + +```sh +kubectl get redisenterpriserolebinding -o yaml | \ + yq '.items[] | select(.spec.roleRef.name == "orders-viewer")' +``` + +For cluster roles, replace `redisenterpriserolebinding` with `redisenterpriseclusterrolebinding`. + +## Delete a role + +Delete any bindings that reference the role first, then delete the role. Find the bindings with the recipes in [Find bindings that reference a role or user](/content/operate/kubernetes/8.2.0/security/access-control/manage-bindings.md#find-bindings-that-reference-a-role-or-user), delete each by name, then delete the role: + +```sh +kubectl delete redisenterpriserolebinding alice-orders-viewer +kubectl delete redisenterpriserole orders-viewer +``` + +If a binding still references the role at the moment of deletion, Redis Software may reject the delete and the operator emits a `RoleDeletionBlocked` event. Resolve the blocking binding and retry. + +## Troubleshoot + +Watch reconciliation events with `kubectl describe redisenterpriserole ` (or `redisenterpriseclusterrole`). Common issues: + +- **`status.uid` is empty** — The role hasn't reconciled. Check the events. Common causes: an ACL reference points to a non-existent `RedisEnterpriseACL`, or admission rejected the spec (missing scopes, scope with both `name` and `selector`, wrong `kind`). +- **Scope selector matches nothing** — A label-selector scope is valid even if no REDB currently matches. The role contributes permissions only to REDBs that match at reconcile time. Add the labels or fix the selector. +- **Permissions don't reach the database** — Check `status.uid` on the role, the matching REDB's `status.rolesPermissions`, and confirm a binding assigns the role to the user. +- **`RoleDeletionBlocked`** — A binding still references the role in Redis Software. Delete the binding first. + +For full field details, see the [`RedisEnterpriseRole`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_role_api.md) and [`RedisEnterpriseClusterRole`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_role_api.md) API reference. + +## Related topics + +- [Roles and bindings](/content/operate/kubernetes/8.2.0/security/access-control/_index.md#roles-and-bindings) — the conceptual model. +- [Manage role bindings](/content/operate/kubernetes/8.2.0/security/access-control/manage-bindings.md) — assign a role to a user. +- [Manage ACLs](/content/operate/kubernetes/8.2.0/security/access-control/manage-acls.md) — define the data-path permissions a role references. +- [Manage users](/content/operate/kubernetes/8.2.0/security/access-control/manage-users.md) — create the users that bindings target. +- [Migrate from REDB rolesPermissions](/content/operate/kubernetes/8.2.0/security/access-control/migrate-rolespermissions.md) — move from the deprecated `RedisEnterpriseDatabase.spec.rolesPermissions` field to the new CRD model. diff --git a/content/operate/kubernetes/8.2.0/security/access-control/manage-users.md b/content/operate/kubernetes/8.2.0/security/access-control/manage-users.md new file mode 100644 index 0000000000..34f94e4cd0 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/security/access-control/manage-users.md @@ -0,0 +1,199 @@ +--- +Title: Manage users +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Create and update Redis Software users on Kubernetes with the RedisEnterpriseUser custom resource. +linkTitle: Manage users +weight: 10 +url: '/operate/kubernetes/8.2.0/security/access-control/manage-users/' +--- + +A `RedisEnterpriseUser` resource defines a Redis Software user. The operator creates the user in Redis Software and keeps it in sync with the resource. Passwords live in Kubernetes Secrets that the resource references by name. + +This page covers creating users, changing passwords, and recovering locked accounts. To grant a user permissions, see [Manage role bindings](/content/operate/kubernetes/8.2.0/security/access-control/manage-bindings.md). + +## Before you start + +- Requires Redis Software for Kubernetes operator 8.2.0-12 or later. +- The `RedisEnterpriseUser` resource and every referenced password Secret must live in the operator namespace. +- Passwords must satisfy the cluster's [password complexity rules](/content/operate/rs/security/access-control/manage-passwords/password-complexity-rules.md). + +## Create a user + +1. Create a Secret with the password under the key `password`: + + ```sh + kubectl create secret generic alice-password \ + --from-literal=password='S0me-Str0ng-Passw0rd!' + ``` + +2. Create the `RedisEnterpriseUser` resource: + + ```yaml + apiVersion: app.redislabs.com/v1alpha1 + kind: RedisEnterpriseUser + metadata: + name: alice + spec: + email: alice@example.com + username: alice + passwordSecrets: + - name: alice-password + ``` + +3. Apply the resource and confirm the operator created the user: + + ```sh + kubectl apply -f alice.yaml + kubectl get redisenterpriseuser alice -o yaml + ``` + + `status.uid` holds the Redis Software user ID once reconciliation succeeds. `status.signinStatus` reports the user's current sign-in state. + +The new user has no permissions until you create a role binding. The operator assigns the Redis Software `none` role so the user is never roleless. + +### Required and optional fields + +| Field | Required | Notes | +| --- | --- | --- | +| `spec.email` | Yes | Must be unique in the cluster. | +| `spec.username` | No | Defaults to a generated value. ASCII only, excluding `&`, `<`, `>`, `"`. The effective value appears in `status.username`. | +| `spec.passwordSecrets` | Yes | At least one Secret. Each Secret must have a `password` key. | +| `spec.passwordMode` | No | `Single` (default) or `Rotatable`. See [Choose a password mode](#choose-a-password-mode). | +| `spec.alerts` | No | Email alert settings. Effective only when [cluster alerts](/content/operate/rs/clusters/configure/cluster-settings.md#alert-settings) are configured. | + +For the full schema, see [`RedisEnterpriseUser`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_user_api.md). + +### Use a generated username + +If you omit `spec.username`, the operator assigns one and reports it in `status.username`. Read it with: + +```sh +kubectl get redisenterpriseuser alice -o jsonpath='{.status.username}' +``` + +Use that value when you sign in to Redis Software. To reference the user from a binding's `subjects` list, use the resource name (`metadata.name`) instead — not the generated username. + +## Choose a password mode + +`spec.passwordMode` controls how passwords change. + +- **`Single`** (default) — Exactly one entry in `passwordSecrets`. To change the password, update the value in the Secret or point the resource at a different Secret. Suitable for interactive users. +- **`Rotatable`** — One or two entries in `passwordSecrets`. The operator marks each referenced Kubernetes Secret immutable so the password can't be edited in place. Suitable for service accounts that need zero-downtime credential rotation. Vault-backed secrets are treated as immutable but aren't modified by the operator. + +You can't switch modes while the user has two password secrets. Reduce to one secret first. + +## Change a password + +### Single mode + +Edit the Secret value, then re-apply it. The operator detects the new version and updates the user's password. + +```sh +kubectl create secret generic alice-password \ + --from-literal=password='N3w-Str0ng-Passw0rd!' \ + --dry-run=client -o yaml | kubectl apply -f - +``` + +Alternatively, create a new Secret and update `spec.passwordSecrets[0].name` to point at it. + +### Rotatable mode (zero-downtime rotation) + +In Rotatable mode the operator makes referenced Kubernetes Secrets immutable, so you rotate by adding a new Secret alongside the old one. + +1. Create a new Secret with the next password. +2. Add it to `spec.passwordSecrets`. Both passwords now authenticate. +3. Update clients to use the new password. +4. Remove the old Secret from `spec.passwordSecrets`. Only the new password authenticates. +5. Delete the old Secret when nothing else references it. + +`status.passwordSecrets` lists each active Secret with its resolved version. + +## Update the email address + +You can change `spec.email` only while `passwordSecrets` contains exactly one entry. If a Rotatable rotation is in progress, reduce to a single secret first, change the email, then add the second secret back. + +## Configure email alerts + +Email alerts deliver only when the cluster has alert email settings configured. The user's alerts have two layers: a master `enabled` toggle for the whole user, and a per-category toggle for cluster alerts. Database alerts are configured by listing the databases the user should receive alerts for: + +```yaml +spec: + alerts: + enabled: true # master toggle; required for any alert to deliver + clusterAlerts: + enabled: true # opt in to cluster-level alerts + databaseAlerts: + databases: # list specific databases, or omit to receive all + - name: my-database +``` + +Omit `databaseAlerts.databases` to receive alerts for every database. + +## Inspect user status + +The `status` block reports observed state from Redis Software: + +| Field | Meaning | +| --- | --- | +| `uid` | Internal Redis Software user ID. Appears once the user is reconciled. | +| `username` | Effective username, including any default the operator assigned. | +| `roles` / `rolesDisplay` | Roles currently bound to the user. | +| `signinStatus` | `Unknown`, `Active`, `Locked`, or `PasswordExpired`. | +| `passwordIssueDate` | When Redis Software last accepted the user's password. | +| `passwordSecrets` | Each referenced Secret with the resolved version the operator reconciled. | +| `observedGeneration` | The `metadata.generation` the operator last acted on. Compare with `metadata.generation` to confirm reconciliation has caught up. | +| `conditions` | The `RolesBound` condition reports whether every bound role resolves. | + +## Recover a locked user + +`status.signinStatus: Locked` means the user failed too many sign-in attempts. The operator skips password changes while the user is locked, so you must update the resource before unlocking — otherwise the operator can later reconcile the old desired password back onto the user. + +1. Update the password in the `RedisEnterpriseUser` source of truth: change the referenced Secret value (Single mode) or add a new Secret reference (Rotatable mode). +2. Follow the [Redis Software unlock procedure](/content/operate/rs/security/access-control/manage-users/login-lockout.md#unlock-locked-user-accounts) to reset and unlock the account in the cluster. + +`status.signinStatus: PasswordExpired` clears once you set a new password through the resource. + +## Delete a user + +Delete every binding that references the user before deleting the user itself. Use the recipes in [Find bindings that reference a role or user](/content/operate/kubernetes/8.2.0/security/access-control/manage-bindings.md#find-bindings-that-reference-a-role-or-user) to list them, delete each by name, then delete the user: + +```sh +kubectl delete redisenterpriserolebinding alice-orders-viewer +kubectl delete redisenterpriseuser alice +``` + +The operator removes the user from Redis Software. A finalizer keeps the Kubernetes resource until the Redis Software user and any related Secret finalizers are cleaned up; deletion may take longer or stall if the Redis Software API is unavailable. + +Password Secrets aren't deleted — remove them separately when nothing else references them. + +## Troubleshoot + +Watch reconciliation events with `kubectl describe redisenterpriseuser `. Common events: + +| Event | Meaning | +| --- | --- | +| `PasswordSecretMissing` | A name in `passwordSecrets` doesn't exist in the operator namespace. | +| `PasswordSecretInvalid` | The Secret exists but has no `password` key, or the value is empty. | +| `UserPasswordAdded` / `UserPasswordReplaced` / `UserPasswordDeleted` | Normal reconciliation actions. Useful for confirming a rotation step. | +| `UserLocked` | Password operations are skipped because the user is locked. See [Recover a locked user](#recover-a-locked-user). | +| `MissingRoleUIDs` | The user has Redis Software role UIDs that no longer map back to a Kubernetes role resource. | +| `RSObjectNotFound` | A Redis Software object the user previously resolved against is gone. | +| `RSOperationFailed` | A Redis Software API call failed; check the message for details. | + +Other things to check: + +- **`status.signinStatus: Unknown`** — The operator hasn't reconciled the user yet, or it can't read the referenced Secret. Check `PasswordSecretMissing` and `PasswordSecretInvalid` events. +- **`RolesBound` condition is `False` with reason `RoleNotFound`** — A binding references this user but points at a role that doesn't exist. Create the role or fix the binding. +- **Secret edit rejected** — In Rotatable mode the operator sets `immutable: true` on the Secret. Create a new Secret instead of editing an existing one. +- **Cluster Manager UI shows a different role than expected** — Roles come from `RedisEnterpriseRoleBinding` and `RedisEnterpriseClusterRoleBinding` resources, not from the user spec. Check the bindings, not the user. + +## Related topics + +- [Manage role bindings](/content/operate/kubernetes/8.2.0/security/access-control/manage-bindings.md) — assign roles to this user. +- [Default user](/content/operate/rs/security/access-control/manage-users/default-user.md) — the built-in cluster admin account, managed outside the CRD model. +- [Password complexity rules](/content/operate/rs/security/access-control/manage-passwords/password-complexity-rules.md) and [password expiration](/content/operate/rs/security/access-control/manage-passwords/password-expiration.md). +- [`RedisEnterpriseUser` API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_user_api.md). diff --git a/content/operate/kubernetes/8.2.0/security/access-control/migrate-rolespermissions.md b/content/operate/kubernetes/8.2.0/security/access-control/migrate-rolespermissions.md new file mode 100644 index 0000000000..3bf108a805 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/security/access-control/migrate-rolespermissions.md @@ -0,0 +1,194 @@ +--- +Title: Migrate from REDB rolesPermissions +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Move database role assignments from the deprecated RedisEnterpriseDatabase.spec.rolesPermissions field to the RBAC CRD model. +linkTitle: Migrate from REDB rolesPermissions +weight: 50 +url: '/operate/kubernetes/8.2.0/security/access-control/migrate-rolespermissions/' +--- + +`RedisEnterpriseDatabase.spec.rolesPermissions` assigns Redis Software role and ACL names directly to a database. The field is deprecated, replaced by the RBAC CRDs (`RedisEnterpriseRole`, `RedisEnterpriseACL`, and bindings), which let you manage roles, ACLs, and assignments as Kubernetes resources. + +This page covers moving an existing database from `rolesPermissions` to the CRD model, then disabling the deprecated field at the cluster level. + +## Before you start + +- Requires Redis Software for Kubernetes operator 8.2.0-12 or later. +- Every user who currently holds access through `rolesPermissions` must exist as a `RedisEnterpriseUser` resource before you can bind a CRD role to them. If you created users through the Redis Software REST API or Cluster Manager UI, migrate them first. See [Manage users](/content/operate/kubernetes/8.2.0/security/access-control/manage-users.md). +- List which Redis Software users hold each role. The CRD inventory only captures the database side of the assignment; the user-to-role mapping lives in Redis Software. Pull it from the Cluster Manager UI or the [Redis Software users REST API](/content/operate/rs/references/rest-api/objects/user.md). + +## How the two sources interact + +By default, the operator reconciles both sources of database role permissions: + +- The deprecated `RedisEnterpriseDatabase.spec.rolesPermissions` list on each REDB. +- Permissions derived from `RedisEnterpriseRole` resources that scope to the REDB. + +The operator merges and de-duplicates the two sets before sending them to Redis Software. `RedisEnterpriseDatabase.status.rolesPermissions` shows the merged result: + +- Entries from the deprecated field use Redis Software's internal role and ACL names. +- Entries from CRDs reference `RedisEnterpriseRole` and `RedisEnterpriseACL` resources. + +You can run both sources in parallel during migration. Once every database is completely migrated, disable the deprecated field cluster-wide (see [Disable the deprecated field](#disable-the-deprecated-field)). + +## Reference existing Redis Software objects + +You don't have to recreate every database, ACL, and role as a custom resource. When an object was created directly through the Redis Software REST API or Cluster Manager UI, a CRD reference can point at it by name using an internal kind, so you can bring it under CRD-managed bindings without redefining it: + +| Field | Internal kind | References | +| --- | --- | --- | +| `scopes[].kind` | `bdb` | An internal Redis Software database by name. Name only — no selector. The role grants management access only; its ACL isn't bound to that database. | +| `acl.kind` | `redis_acl` | An internal Redis Software ACL by name. | +| `roleRef.kind` | `role` | An internal Redis Software role by name, from a binding. | + +The reference resolves by name against the existing Redis Software object. Use internal references to migrate assignments incrementally, then replace them with full `RedisEnterpriseDatabase`, `RedisEnterpriseACL`, and `RedisEnterpriseRole` resources as you bring each object under the CRD model. + +## Migrate a database + +For each database, capture what `rolesPermissions` currently grants, recreate it as CRDs, verify the result, then remove the deprecated field. + +### 1. Inventory the existing entries + +List the `rolesPermissions` entries on every REDB in the operator namespace: + +```sh +kubectl get redisenterprisedatabase -o yaml | \ + yq '.items[] | {name: .metadata.name, rolesPermissions: .spec.rolesPermissions}' +``` + +Each entry has a `role` (Redis Software role name) and an `acl` (Redis Software ACL name). The `type` is always `redis-enterprise`. + +### 2. Create RedisEnterpriseACL resources + +For every distinct ACL name in the inventory, create a `RedisEnterpriseACL` resource that holds the same rule string. See [Manage ACLs](/content/operate/kubernetes/8.2.0/security/access-control/manage-acls.md). + +```yaml +apiVersion: app.redislabs.com/v1alpha1 +kind: RedisEnterpriseACL +metadata: + name: read-only +spec: + acl: "+@read ~*" +``` + +The resource `metadata.name` doesn't have to match the original Redis Software ACL name, but matching makes the merged `status.rolesPermissions` easier to read. + +### 3. Create RedisEnterpriseRole resources + +For each role-ACL-REDB combination in the inventory, create a `RedisEnterpriseRole` scoped to the target REDB. See [Manage roles](/content/operate/kubernetes/8.2.0/security/access-control/manage-roles.md). + +```yaml +apiVersion: app.redislabs.com/v1alpha1 +kind: RedisEnterpriseRole +metadata: + name: orders-viewer +spec: + managementRole: DBViewer + scopes: + - name: orders + acl: + name: read-only +``` + +Set `managementRole` to match the Redis Software role's management level. If the original entry only granted data-path access, omit `managementRole` — it defaults to `None`. + +If the same role-ACL pair applies to more than one REDB, collapse them into a single `RedisEnterpriseRole` with multiple `scopes` entries instead of creating one role per REDB: + +```yaml +spec: + managementRole: DBViewer + scopes: + - name: orders + - name: customers + - name: inventory + acl: + name: read-only +``` + +### 4. Create bindings for affected users + +For each user that previously gained access through `rolesPermissions`, create a `RedisEnterpriseRoleBinding` that references the new role. See [Manage role bindings](/content/operate/kubernetes/8.2.0/security/access-control/manage-bindings.md). + +```yaml +apiVersion: app.redislabs.com/v1alpha1 +kind: RedisEnterpriseRoleBinding +metadata: + name: alice-orders-viewer +spec: + roleRef: + name: orders-viewer + subjects: + - name: alice +``` + +### 5. Verify + +Confirm Redis Software still grants the expected access: + +```sh +kubectl get redisenterprisedatabase orders -o yaml | \ + yq '.status.rolesPermissions' +``` + +The merged `status.rolesPermissions` shows entries from both sources. Before you remove the deprecated entries, authenticate as one of the affected users and run a Redis command the role allows: + +```sh +redis-cli -h -p --user alice --pass GET some:key +``` + +Replace the endpoint and port with the values from the REDB's `status.publicService` (or `status.internalService`). + +### 6. Remove the deprecated entries + +Once verification passes, remove the migrated entries from `spec.rolesPermissions`: + +```sh +kubectl edit redisenterprisedatabase orders +``` + +After the next reconcile, `status.rolesPermissions` should contain only CRD-derived entries. Repeat for every REDB. + +## Disable the deprecated field + +After every database is completely migrated, switch the cluster policy to ignore `spec.rolesPermissions` so no one can re-introduce it: + +```yaml +apiVersion: app.redislabs.com/v1 +kind: RedisEnterpriseCluster +metadata: + name: rec +spec: + accessControl: + policy: + allowREDBRolesPermissions: false +``` + +With `allowREDBRolesPermissions: false`: + +- The operator stops reconciling `spec.rolesPermissions` on every REDB. +- The operator removes from Redis Software any permissions that came only from `spec.rolesPermissions`. +- The operator emits a `RolesPermissionsIgnored` event on any REDB that still has the deprecated field set. + +The default is `true` (both sources reconciled). Set the flag back to `true` only as a rollback. + +## Roll back + +To revert to the deprecated field, set `allowREDBRolesPermissions: true` (or omit it) on the `RedisEnterpriseCluster` and re-add the previous entries to each REDB's `spec.rolesPermissions`. CRD-managed permissions continue to work in parallel. + +## Troubleshoot + +- **A user is missing access after you remove the deprecated entries** — A `RedisEnterpriseRole` or binding is missing for that user. Check `RedisEnterpriseUser.status.roles` against the original inventory and add the missing CRDs. +- **`status.rolesPermissions` is empty after migration** — The new roles haven't reconciled. Each `RedisEnterpriseRole` needs `status.uid` populated before it contributes to a REDB. Check the role's events. +- **`RolesPermissionsIgnored` event on a REDB after disabling the field** — A `spec.rolesPermissions` entry remains on the REDB. Edit the REDB and remove the field entirely. + +## Related topics + +- [Manage roles](/content/operate/kubernetes/8.2.0/security/access-control/manage-roles.md) — full details on `RedisEnterpriseRole` and `RedisEnterpriseClusterRole`. +- [Manage ACLs](/content/operate/kubernetes/8.2.0/security/access-control/manage-acls.md) — define the data-path permissions roles reference. +- [Manage role bindings](/content/operate/kubernetes/8.2.0/security/access-control/manage-bindings.md) — assign the new roles to users. +- [`RedisEnterpriseDatabase` API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md) — the source schema, including the deprecated `rolesPermissions` field. +- [`RedisEnterpriseCluster` API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md) — the `accessControl.policy.allowREDBRolesPermissions` flag. diff --git a/content/operate/kubernetes/8.2.0/security/allow-resource-adjustment.md b/content/operate/kubernetes/8.2.0/security/allow-resource-adjustment.md new file mode 100644 index 0000000000..9ac69dee33 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/security/allow-resource-adjustment.md @@ -0,0 +1,128 @@ +--- +categories: +- docs +- operate +- kubernetes +description: Enable automatic system resource adjustments for Redis Enterprise to increase file descriptor limits. +linkTitle: Auto resource adjustment +title: Allow automatic resource adjustment +weight: 50 +url: '/operate/kubernetes/8.2.0/security/allow-resource-adjustment/' +--- + +Redis Enterprise for Kubernetes 7.22.0-6 introduces the ability to run with automatic resource adjustment disabled, which drops all capabilities from the Redis Enterprise container and sets `allowPrivilegeEscalation` to `false`. All other security-related settings remain the same as in automatic resource adjustment enabled. Automatic resource adjustment disabled is the default for installations and upgrades of the Redis Enterprise operator for versions 7.22.0-6 and later. + +## Default behavior + +Automatic resource adjustment is disabled by default for installations and upgrades of the Redis Enterprise operator for versions 7.22.0-6 and later. This default behavior is in effect if REC spec has `allowAutoAdjustment` set to `false` or removed. + +If automatic resource adjustment is disabled, the REC security context looks like this: + +```yaml +securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: + - ALL + privileged: false + readOnlyRootFilesystem: false +``` + +## Enable automatic resource adjustment + +To allow the Redis Enterprise container to adjust system resource limits automatically, set `allowAutoAdjustment` to `true`. This will grant the container elevated capabilities such as `SYS_RESOURCE`. Note that changing this value on a running cluster will trigger a rolling update. + +```yaml +spec: + securityContext: + resourceLimits: + allowAutoAdjustment: true +``` + +Enabling automatic resource adjustment results in the following security context: + +**Note:** Enabling `allowAutoAdjustment` grants the container the `SYS_RESOURCE` capability and permits privilege escalation. + +```yaml +securityContext: + allowPrivilegeEscalation: true + capabilities: + add: + - SYS_RESOURCE + drop: + - ALL + privileged: false + readOnlyRootFilesystem: false +``` + +## OpenShift upgrades + +If you're upgrading OpenShift to 7.22.0-6, update your existing SCC (security context constraint). + +If running with automatic resource adjustment disabled, remove the custom `redis-enterprise-scc-v2` SCC and unbind it from the REC service account after you complete the upgrade. + +```sh +oc delete scc/redis-enterprise-scc-v2 +``` + +```sh +oc adm policy remove-scc-from-user redis-enterprise-scc-v2 -z +``` + +If running with automatic resource adjustment enabled, manually reapply the [security context constraints (SCC)](https://docs.openshift.com/container-platform/4.8/authentication/managing-security-context-constraints.html) file ([`scc.yaml`](/content/operate/kubernetes/8.2.0/deployment/openshift/openshift-cli.md#deploy-the-operator)). + +```sh +oc apply -f openshift/scc.yaml +``` + +```sh +oc adm policy add-scc-to-user redis-enterprise-scc-v2 \ + system:serviceaccount:: +``` + +## New OpenShift installations + +New installations of Redis Enterprise for Kubernetes 7.22.0-6 and later automatically run with automatic resource adjustment disabled, using a built-in `nonroot-v2` which is more secure and less permissive. + +To enable automatic resource adjustment after installation, apply and grant permissions to the `redis-enterprise-scc-v2` SCC. + +1. Apply the `scc.yaml` file. + + > [!WARNING] + > Do not edit this file. + + ```sh + oc apply -f openshift/scc.yaml + ``` + + You should see the following output: + + ```sh + securitycontextconstraints.security.openshift.io "redis-enterprise-scc-v2" configured + ``` + +1. Provide the operator permissions for the pods. + + ```sh + oc adm policy add-scc-to-user redis-enterprise-scc-v2 \ + system:serviceaccount:: + ``` + +## SYS_RESOURCE + +Some Redis Enterprise processes may require the `SYS_RESOURCE` capability to raise resource limits, such as the maximum number of open file descriptors. + +Some Redis Enterprise processes require the ability to open at least 100,000 file descriptors. If the default is lower and `SYS_RESOURCE` is not enabled, these processes may fail. + +## Choose whether to enable automatic resource adjustment + +Use the following guidance to decide whether to enable automatic resource adjustment: + +- If you're running on a major cloud provider such as AWS, GKE, or AKS, automatic resource adjustment disabled is likely sufficient. +- If you're running on-prem or using Kubespray, verify your file descriptor limits. You can: + - Configure limits manually and use automatic resource adjustment disabled. + - Enable automatic resource adjustment to allow Redis Enterprise to increase limits, which requires privilege escalation. + +If you are already running a Redis Enterprise cluster on Kubernetes, your worker nodes are likely configured correctly. In this case, it is safe to upgrade the operator and use automatic resource adjustment disabled. + +Based on our testing, all major cloud providers configure Kubernetes worker nodes with file descriptor limits well above the required minimum. These environments typically work without enabling automatic resource adjustment. The only known exception is clusters created with [Kubespray](https://kubespray.io/#/), which sets default file descriptor limits below the required 100,000. If you use Kubespray with default settings, you must run the operator with automatic resource adjustment enabled. diff --git a/content/operate/kubernetes/8.2.0/security/authentication/_index.md b/content/operate/kubernetes/8.2.0/security/authentication/_index.md new file mode 100644 index 0000000000..30319adc9b --- /dev/null +++ b/content/operate/kubernetes/8.2.0/security/authentication/_index.md @@ -0,0 +1,47 @@ +--- +Title: Authentication +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Manage Redis Software cluster credentials, LDAP, SSO, and configuration secrets on Kubernetes. +hideListLinks: true +linkTitle: Authentication +weight: 10 +url: '/operate/kubernetes/8.2.0/security/authentication/' +--- + +Authentication covers cluster credentials, external identity providers (LDAP and SAML SSO), and configuration secrets. The operator generates the initial cluster admin credentials, applies LDAP and SSO settings from the `RedisEnterpriseCluster` spec, and reads configuration values from Kubernetes Secrets you can update without a cluster restart. + +## How authentication works on Redis for Kubernetes + +- **Cluster credentials** are auto-generated at install and stored in a Kubernetes Secret named after the REC resource. Retrieve and update them with `kubectl`. +- **LDAP** is configured on the `RedisEnterpriseCluster` spec. The operator applies the configuration through the Redis Software REST API. +- **SAML SSO** is enabled on the REC spec. The operator configures the identity provider connection in Redis Software. +- **Configuration secrets** let you store sensitive configuration items in Kubernetes Secrets that the operator references. Updates to the Secret reconcile automatically. + +## What's the same as Redis Software + +The underlying Redis Software behavior is unchanged. For concepts and reference details, see the existing Redis Software docs: + +- [LDAP authentication overview](/content/operate/rs/security/access-control/ldap/_index.md) — server requirements, supported attributes, and the LDAP model. +- [Enable role-based LDAP](/content/operate/rs/security/access-control/ldap/enable-role-based-ldap.md) — concepts behind role-based LDAP. +- [Map LDAP groups to roles](/content/operate/rs/security/access-control/ldap/map-ldap-groups-to-roles.md) — group-to-role mapping rules. +- [SAML single sign-on](/content/operate/rs/security/access-control/saml-sso.md) — identity provider requirements and SAML attribute mappings. +- [Default user](/content/operate/rs/security/access-control/manage-users/default-user.md) — what the bootstrap admin account is for. + +## What's different on Kubernetes + +- **Initial credentials are auto-generated by default.** If you don't provide them, the operator generates them and stores them in the credentials Secret, which you retrieve after the REC is up. You can instead supply your own Secret name and credentials at cluster creation time. +- **Change the cluster admin credentials in Redis Software, then mirror them to the Secret.** Update the credentials directly in Redis Software and reflect the new values in the credentials Secret — see [Manage REC credentials](/content/operate/kubernetes/8.2.0/security/authentication/manage-rec-credentials.md). Credentials for other users are managed through their `RedisEnterpriseUser` resources and Secrets. +- **LDAP and SSO configuration is part of the REC spec.** The operator applies it through the Redis Software REST API, so the configuration is source-controlled. +- **Sensitive values live in Kubernetes Secrets** (or HashiCorp Vault) instead of in Redis Software configuration files. + +## In this section + +- [Manage REC credentials](/content/operate/kubernetes/8.2.0/security/authentication/manage-rec-credentials.md) — retrieve and update the cluster admin credentials Secret. +- [Manage REDB passwords](/content/operate/kubernetes/8.2.0/security/authentication/manage-redb-credentials.md) — retrieve and rotate database passwords through the database Secret. +- [Configuration secrets](/content/operate/kubernetes/8.2.0/security/authentication/configuration-secrets.md) — store config items in Kubernetes Secrets and reconcile updates automatically. +- [LDAP authentication](/content/operate/kubernetes/8.2.0/security/authentication/ldap.md) — configure LDAP for Cluster Manager and database access. +- [SSO authentication](/content/operate/kubernetes/8.2.0/security/authentication/sso.md) — configure SAML single sign-on for the Cluster Manager UI. diff --git a/content/operate/kubernetes/8.2.0/security/authentication/configuration-secrets.md b/content/operate/kubernetes/8.2.0/security/authentication/configuration-secrets.md new file mode 100644 index 0000000000..5f089e3412 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/security/authentication/configuration-secrets.md @@ -0,0 +1,156 @@ +--- +categories: +- docs +- operate +- kubernetes +aliases: [/operate/kubernetes/security/configuration-secrets/] +description: Store Redis Enterprise configuration items in Kubernetes Secrets for automatic updates and secure management. +linkTitle: Configuration secrets +title: Store configuration in Kubernetes Secrets +weight: 20 +url: '/operate/kubernetes/8.2.0/security/authentication/configuration-secrets/' +--- + +You can store Redis Enterprise configuration items in Kubernetes Secrets for automatic updates and secure management. When you update these Secrets, the operator immediately reads the changes and propagates them to the Redis Enterprise Cluster (REC). + +## License configuration + +Redis Enterprise clusters require a valid license. You can apply licenses using Kubernetes Secrets (recommended) or embed them directly in the cluster specification. + +### Determine your cluster FQDN + +To configure licensing, you need your Redis Enterprise cluster's fully qualified domain name (FQDN). Use this format: `..svc.cluster.local` + +For example: `my-rec.my-ns.svc.cluster.local` + +### Use a Kubernetes Secret (recommended) + +1. Create a secret from your license file: + + ```sh + kubectl -n create secret generic rec-license --from-file=license=./license.txt + ``` + +2. Add the secret reference to your REC specification: + + ```yaml + spec: + licenseSecretName: rec-license + ``` + +### Embed license directly in REC specification + +Alternatively, you can embed the license directly in the REC YAML: + +```yaml +spec: + nodes: 3 + license: | + ----- LICENSE START ----- + eai14c/y6XNVykffDQSPUsHKcmpgOFUlmyTBDUEZEz+GLbXAgQFOmxcdbR9J + ...remaining license key content... + ----- LICENSE END ----- +``` + +> [!NOTE] +> You must include the pipe symbol (`|`) after `license:` and maintain proper indentation. + +## Cluster credential configuration + +Redis Enterprise clusters require admin credentials for the operator to manage the cluster. You can configure these credentials using Kubernetes Secrets. + +### Default behavior + +By default, the operator automatically creates a secret with a random username and auto-generated password during cluster creation. The secret name defaults to the cluster name. + +### Customize the credential secret + +You can customize the credential secret name during cluster creation using the `clusterCredentialSecretName` field in your REC specification. The secret must contain `username` and `password` fields. + +> [!NOTE] +> The `clusterCredentialSecretName` field cannot be changed after cluster creation. + +For detailed instructions, see [Customize the credential secret name](/content/operate/kubernetes/8.2.0/security/authentication/manage-rec-credentials.md#customize-the-credential-secret-name). + +## TLS certificate configuration + +You can store TLS certificates in Kubernetes Secrets to secure communication between clients and Redis Enterprise databases. + +### Client certificates for mTLS + +1. Create a secret with your client certificate: + + ```sh + kubectl -n create secret generic client-cert-secret --from-file=cert= + ``` + +2. Add the secret to your REDB using the `clientAuthenticationCertificates` property. See [Add client certificates](/content/operate/kubernetes/8.2.0/security/certificates/add-client-certificates.md) for details. + +### Service certificates + +To configure certificates for proxy, API, or other services, create secrets with certificate and key files: + +```sh +kubectl create secret generic \ + --from-file=certificate= \ + --from-file=key= \ + --from-literal=name= +``` + +### Internode encryption certificates + +You can provide custom certificates for control plane and data plane internode encryption. Create separate secrets for each encryption type: + +```sh +kubectl create secret generic cp-internode-cert \ + --from-file=certificate= \ + --from-file=key= \ + --from-literal=name=cp_internode_encryption +``` + +```sh +kubectl create secret generic dp-internode-cert \ + --from-file=certificate= \ + --from-file=key= \ + --from-literal=name=dp_internode_encryption +``` + +Reference these secrets in your REC specification under `spec.certificates`. See [Internode encryption](/content/operate/kubernetes/8.2.0/security/certificates/internode-encryption.md) for complete configuration details. + +## Secrets and PEM files in Redis Enterprise pods + +Redis Enterprise pods use Kubernetes Secrets and PEM-encoded certificates and keys for cluster formation, node identity, encrypted communication, and automated recovery. Their presence is expected — not a sign of compromise. + +You create the Secrets. The operator references them and manages their lifecycle (for example, when you rename the credential Secret). TLS, license, and client authentication Secrets are always user-supplied. + +### What you'll see in the pod + +- **Mounted Secret volumes** at operator-managed paths such as: + - `/opt/redislabs/credentials` — cluster admin credential Secret. + - `/opt/redislabs/proxy` — call-home proxy credentials, when configured. +- **PEM-encoded certificates and keys** for TLS, internode encryption, and proxy or database endpoints. The operator applies these certificates through the cluster's REST API rather than as Secret volume mounts, and Redis Enterprise writes them to the pod filesystem. Exact paths vary by version and component. + +### What the Secrets contain + +Field names vary by deployment. + +- **Cluster admin credentials** — `username` and `password` in the Secret named by `clusterCredentialSecretName`. +- **License** — `license` field in the Secret named by `licenseSecretName`. +- **Cluster Certificate Authority (CA)** — `ca.crt` or `ca.pem`. Validates peer certificates for mutual TLS. Optional. +- **Service TLS certificates and keys** for API, Cluster Manager (CM), metrics exporter, proxy, syncer, and LDAP. Fields: `certificate`, `cert`, or `tls.crt`, plus `key` or `tls.key`. See [Service certificates](#service-certificates). +- **Client authentication certificates** for databases. Set in the Redis Enterprise database (REDB) `clientAuthenticationCertificates` field. + +## Best practices + +- Store sensitive configuration in Secrets rather than directly in YAML files. +- Use `--from-file` to avoid manual base64 encoding. +- Create secrets in the same namespace as your REC or REDB resources. +- Use descriptive secret names for easy identification. +- Regularly rotate certificates and update secrets. + +## See also + +- [Manage REC credentials](/content/operate/kubernetes/8.2.0/security/authentication/manage-rec-credentials.md) +- [Manage REC certificates](/content/operate/kubernetes/8.2.0/security/certificates/manage-rec-certificates.md) +- [Add client certificates](/content/operate/kubernetes/8.2.0/security/certificates/add-client-certificates.md) +- [Redis Enterprise Cluster API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md) diff --git a/content/operate/kubernetes/8.2.0/security/authentication/ldap.md b/content/operate/kubernetes/8.2.0/security/authentication/ldap.md new file mode 100644 index 0000000000..29bff6e4f6 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/security/authentication/ldap.md @@ -0,0 +1,169 @@ +--- +Title: Enable LDAP authentication +alwaysopen: false +categories: +- docs +- operate +- kubernetes +aliases: [/operate/kubernetes/security/ldap/] +description: Enable LDAP authentication for Redis Enterprise for Kubernetes. +linkTitle: Enable LDAP +weight: 30 +url: '/operate/kubernetes/8.2.0/security/authentication/ldap/' +--- + +## LDAP support for Redis Enterprise Software + +Redis Enterprise Software supports LDAP authentication and authorization through [role-based access controls](/content/operate/rs/security/access-control/_index.md) (RBAC). You can map LDAP groups to [Redis Enterprise roles](/content/operate/rs/security/access-control/_index.md) to control access to your database and the Cluster Manager UI. For more details on how LDAP works with Redis Enterprise, see [LDAP authentication](/content/operate/rs/security/access-control/ldap/_index.md). + +Redis Enterprise for Kubernetes supports enabling and configuring LDAP authentication using the `RedisEnterpriseCluster` (REC) custom resource. Currently, the Redis Enterprise cluster (REC) only supports configuration related to the LDAP server, such as server addresses, connection details, credentials, and query configuration. + +To [map LDAP groups to Redis Enterprise access control roles](/content/operate/rs/security/access-control/ldap/enable-role-based-ldap.md), you'll need to use the Redis Enterprise [API](/content/operate/rs/references/rest-api/requests/ldap_mappings/_index.md) or [admin console](/content/operate/rs/security/access-control/ldap/enable-role-based-ldap.md). + +## Enable LDAP + +To enable LDAP for your REC, use the `.spec.ldap` field in the `RedisEnterpriseCluster` custom resource. + +The following `RedisEnterpriseCluster` example resource enables a basic LDAP configuration: + +```yaml +apiVersion: app.redislabs.com/v1 +kind: RedisEnterpriseCluster +metadata: + name: rec +spec: + nodes: 3 + ldap: + protocol: LDAP + servers: + - host: openldap.openldap.svc + port: 389 + bindCredentialsSecretName: ldap-bind-credentials + cacheTTLSeconds: 600 + enabledForControlPlane: true + enabledForDataPlane: true + authenticationQuery: + template: cn=%u,ou=default,dc=example,dc=org + authorizationQuery: + attribute: memberOf +``` + +Refer to the `RedisEnterpriseCluster` [API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#specldap) for full details on the available fields. + +### Bind credentials + +For LDAP servers that require authentication for client queries, store the bind credentials in a secret and reference them in the `RedisEnterpriseCluster` custom resource. + +1. Create a secret to store the bind credentials. + + ```sh + kubectl -n create secret generic \ + --from-literal=dn='' \ + --from-literal=password= + ``` + The secret must: + - Reside within the same namespace as the `RedisEnterpriseCluster` custom resource. + - Include a `dn` key with the distinguished name for the user performing the query (such as `cn=admin,dc=example,dc=org`). + - Include a `password` key with the bind password. + + Replace the `` in the command above with your own values. + +1. Reference the secret name in the `.spec.ldap.bindCredentialsSecretName` field of the `RedisEnterpriseCluster` custom resource. + + ```yaml + spec: + ldap: + bindCredentialsSecretName: + ``` + +### LDAPS or STARTTLS protocols + +In addition to plain LDAP protocol, Redis Enterprise Software also supports LDAPS and STARTTLS protocols for secure communication with the LDAP server. + +To enable one of these protocols, edit the `spec.ldap.protocol` field in the `RedisEnterpriseCluster` custom resource: + +#### Enable `LDAPS` + + ```yaml + spec: + ldap: + protocol: LDAPS + ``` + + Default port: 636 + +#### Enable `STARTTLS` + + ```yaml + spec: + ldap: + protocol: STARTTLS + ``` + + Default port: 389 + +### CA certificate + +To use a custom CA certificate for validating the LDAP server certificate, store the CA certificate in a secret and reference the secret in the `RedisEnterpriseCluster` custom resource. + +1. Create a secret to hold the CA certificate. + + ```sh + kubectl -n create secret generic \ + --from-file=cert=.pem + ``` + + The secret must: + - Reside within the same namespace as the `RedisEnterpriseCluster` custom resource. + - Include a `cert` key with a PEM-encoded CA certificate (such as `cacert.pem`). + + Replace the `` in the command above with your own values. + +1. Reference the secret name in the `spec.ldap.caCertificateSecretName` field of the `RedisEnterpriseCluster` custom resource. + + ```yaml + spec: + ldap: + caCertificateSecretName: + ``` + +### Client certificates + +To use an LDAP client certificate, store the certificate in a secret and reference the secret in the `RedisEnterpriseCluster` custom resource. + +1. Create a secret to hold the client certificate. + + ```sh + kubectl -n create secret generic \ + --from-literal=name=ldap_client \ + --from-file=certificate= \ + --from-file=key= + ``` + + The secret must: + - Reside within the same namespace as the `RedisEnterpriseCluster` custom resource. + - Include a `name` key explicitly set to `ldap_client`. + - Include a `certificate` key for the public key (such as `cert.pem`). + - Include a `key` key for the private key (such as `key.pem`). + + + Replace the `` in the command above with your own values. + +1. Reference the secret name in the `.spec.certificates.ldapClientCertificateSecretName` field of the `RedisEnterpriseCluster` custom resource, substituting your own values for ``. + + ```yaml + spec: + certificates: + ldapClientCertificateSecretName: + ``` + +## Known limitations + +Redis Enterprise Software can't resolve DNS names with a `.local` suffix. + If your LDAP server is in the same Kubernetes cluster and exposed via a Service object, *avoid* addresses such as `openldap.openldap.svc.cluster.local`. Instead, *use short-form addresses* such as `openldap.openldap.svc`. + +## Next steps + +To [map LDAP groups to Redis Enterprise access control roles](/content/operate/rs/security/access-control/ldap/enable-role-based-ldap.md), you'll need to use the Redis Enterprise [API](/content/operate/rs/references/rest-api/requests/ldap_mappings/_index.md) or [admin console](/content/operate/rs/security/access-control/ldap/enable-role-based-ldap.md). + +For more details on how LDAP works with Redis Enterprise, see [LDAP authentication](/content/operate/rs/security/access-control/ldap/_index.md). diff --git a/content/operate/kubernetes/8.2.0/security/authentication/manage-rec-credentials.md b/content/operate/kubernetes/8.2.0/security/authentication/manage-rec-credentials.md new file mode 100644 index 0000000000..ed9ae9f8a0 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/security/authentication/manage-rec-credentials.md @@ -0,0 +1,134 @@ +--- +Title: Manage Redis Enterprise cluster (REC) credentials +alwaysopen: false +categories: +- docs +- operate +- kubernetes +aliases: [/operate/kubernetes/security/manage-rec-credentials/] +linkTitle: Manage REC credentials +weight: 10 +url: '/operate/kubernetes/8.2.0/security/authentication/manage-rec-credentials/' +--- +Redis Enterprise for Kubernetes uses a custom resource called [`RedisEnterpriseCluster`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md) to create a Redis Enterprise cluster (REC). During creation, it generates random credentials for the operator to use. The credentials are saved in a Kubernetes (K8s) [secret](https://kubernetes.io/docs/concepts/configuration/secret/). The secret name defaults to the cluster name and is specified by the `clusterCredentialSecretName` field in the REC specification. + +## Retrieve the current username and password + +The credentials can be used to access the Redis Enterprise admin console or the API. Connectivity must be configured to the REC [pods](https://kubernetes.io/docs/concepts/workloads/pods/) using an appropriate service (or port forwarding). + +1. Inspect the random username and password created by the operator during creation with the `kubectl get secret` command. + + ```sh + kubectl get secret rec -o jsonpath='{.data}' + ``` + + The command outputs the encoded password and username, similar to the example below. + + ```sh + map[password:MTIzNDU2NzgK username:ZGVtb0BleGFtcGxlLmNvbQo=] + ``` + +1. Decode each value with the `echo` and `base64` commands, using the encoded strings from the previous step: + + ```bash + echo MTIzNDU2NzgK | base64 --decode + echo ZGVtb0BleGFtcGxlLmNvbQo= | base64 --decode + ``` + + In this example, the plain text password is `12345678` and the username is `demo@example.com`. + +## Change the Redis Enterprise cluster (REC) credentials + +### Change the REC password for the current username + +1. Access a [pod](https://kubernetes.io/docs/concepts/workloads/pods/) running a Redis Enterprise cluster: + + ```sh + kubectl exec -it -0 -c redis-enterprise-node -- /bin/bash + ``` + +1. Add a new password for the existing user: + + ```bash + REC_USER="`cat /opt/redislabs/credentials/username`" \ + REC_PASSWORD="`cat /opt/redislabs/credentials/password`" \ + curl -k --request POST \ + --url https://localhost:9443/v1/users/password \ + -u "$REC_USER:$REC_PASSWORD" \ + --header 'Content-Type: application/json' \ + --data "{\"username\":\"$REC_USER\", \ + \"new_password\":\"\"}" + ``` + +1. From outside the pod, update the REC credential secret: + + ```sh + kubectl create secret generic \ + --save-config \ + --dry-run=client \ + --from-literal=username= \ + --from-literal=password= \ + -o yaml | \ + kubectl apply -f - + ``` + +1. Wait five minutes for all the components to read the new password from the updated secret. If you proceed to the next step too soon, the account could get locked. + +1. Access a pod running a Redis Enterprise cluster again: + + ```sh + kubectl exec -it -0 -c redis-enterprise-node -- /bin/bash + ``` + +1. Remove the previous password to ensure only the new one applies: + + ```sh + REC_USER="`cat /opt/redislabs/credentials/username`"; \ + REC_PASSWORD="`cat /opt/redislabs/credentials/password`"; \ + curl -k --request DELETE \ + --url https://localhost:9443/v1/users/password \ + -u "$REC_USER:$REC_PASSWORD" \ + --header 'Content-Type: application/json' \ + --data "{\"username\":\"$REC_USER\", \ + \"old_password\":\"\"}" + ``` + +> [!NOTE] +> The username for the K8s secret is the email displayed on the Redis Enterprise admin console. + +### Change both the REC username and password + +1. [Connect to the admin console](/content/operate/kubernetes/8.2.0/re-clusters/connect-to-admin-console.md). + +1. [Add another admin user](/content/operate/rs/security/access-control/create-users.md) and choose a new password. + +1. Specify the new username in the `username` field of your REC custom resource spec. + +1. Update the REC credential secret: + + ```sh + kubectl create secret generic \ + --save-config \ + --dry-run=client \ + --from-literal=username= \ + --from-literal=password= \ + -o yaml | \ + kubectl apply -f - + ``` + +1. Wait five minutes for all the components to read the new password from the updated secret. If you proceed to the next step too soon, the account could get locked. + +1. Delete the previous admin user from the cluster. + +> [!NOTE] +> The operator may log errors in the time between updating the username in the REC spec and the secret update. + +### Update the credentials secret in Vault + +If you store your secrets with HashiCorp Vault, update the secret for the REC credentials with the following key-value pairs: + +```sh +username:, password: +``` + +For more information about Vault integration with the Redis Enterprise Cluster see [Integrating Redis Enterprise for Kubernetes with HashiCorp Vault](https://github.com/RedisLabs/redis-enterprise-k8s-docs/blob/master/vault/README.md). diff --git a/content/operate/kubernetes/8.2.0/security/authentication/manage-redb-credentials.md b/content/operate/kubernetes/8.2.0/security/authentication/manage-redb-credentials.md new file mode 100644 index 0000000000..8bd42335ef --- /dev/null +++ b/content/operate/kubernetes/8.2.0/security/authentication/manage-redb-credentials.md @@ -0,0 +1,70 @@ +--- +Title: Manage Redis Enterprise database (REDB) passwords +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Retrieve and rotate Redis Enterprise database (REDB) passwords on Kubernetes. +linkTitle: Manage REDB passwords +weight: 15 +url: '/operate/kubernetes/8.2.0/security/authentication/manage-redb-credentials/' +--- + +Each [`RedisEnterpriseDatabase`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md) resource has a password stored under the `password` key of the secret named by `spec.databaseSecretName`. If you don't set `databaseSecretName`, the operator creates a secret named `redb-` with a random password and updates the REDB spec to reference it. + +The operator reads the `password` key on every reconciliation and applies it to the database, so you rotate the password by updating the secret. + +## Retrieve the REDB password + +1. Find the secret name for the database: + + ```sh + kubectl get redb -o jsonpath="{.spec.databaseSecretName}" + ``` + +2. Decode the password: + + ```sh + kubectl get secret -o jsonpath="{.data.password}" | base64 --decode + ``` + +## Change the REDB password + +> [!NOTE] +> If the REDB spec sets `defaultUser: false`, the operator does not create or update the database secret. Rotating the secret has no effect in that mode — manage credentials through [access control](/content/operate/kubernetes/8.2.0/security/access-control/_index.md) instead, using [`RedisEnterpriseUser`](/content/operate/kubernetes/8.2.0/security/access-control/manage-users.md) resources. + +1. Base64-encode the new password. Use `echo -n` to avoid encoding a trailing newline: + + ```sh + echo -n '' | base64 + ``` + +2. Patch the secret with the encoded value: + + ```sh + kubectl patch secret -p='{"data":{"password":""}}' + ``` + + To edit the secret interactively, use `kubectl edit secret ` and replace the `password` value. + +3. Verify that the operator applied the change. The REDB status moves to `active-change-pending` while the update is in flight and returns to `active` when complete: + + ```sh + kubectl get redb -o jsonpath='{.status.status}' + ``` + + Then test the new password with a Redis client: + + ```sh + redis-cli -h -p -a '' PING + ``` + +To disable authentication for the default user, set the `password` value to an empty string. + +### Impact on existing client connections + +Existing client connections authenticated with the old password remain open — Redis Enterprise does not drop sessions when the password changes. New connections, and any `AUTH` commands issued on existing connections, must use the new password. Coordinate the secret update with your client configuration to avoid authentication errors. + +> [!NOTE] +> For Active-Active databases, the database secret is not created automatically. See [Create a global database secret](/content/operate/kubernetes/8.2.0/active-active/global-db-secret.md). diff --git a/content/operate/kubernetes/8.2.0/security/authentication/sso.md b/content/operate/kubernetes/8.2.0/security/authentication/sso.md new file mode 100644 index 0000000000..845bcd373d --- /dev/null +++ b/content/operate/kubernetes/8.2.0/security/authentication/sso.md @@ -0,0 +1,370 @@ +--- +Title: Enable SSO authentication +alwaysopen: false +categories: +- docs +- operate +- kubernetes +aliases: [/operate/kubernetes/security/sso/] +description: Enable SAML-based SSO authentication for Redis Enterprise for Kubernetes. +linkTitle: Enable SSO +weight: 40 +url: '/operate/kubernetes/8.2.0/security/authentication/sso/' +--- + + +Redis Enterprise Software supports SAML 2.0 single sign-on (SSO) for the Cluster Manager UI with both IdP-initiated and SP-initiated authentication. User accounts are automatically created on first sign-in using just-in-time (JIT) provisioning. + + +## IdP requirements + +Your identity provider must support: + +- SAML 2.0 protocol +- Signed SAML responses +- HTTP-Redirect binding for SP-initiated SSO +- HTTP-POST binding for SAML assertions + +## Enable SSO + +To enable SSO for your Redis Enterprise cluster (REC), follow these steps to configure SAML authentication. + +### Prerequisites + +Before enabling SSO, ensure you have: + + +1. An existing Redis Enterprise cluster (REC) deployed in Kubernetes + +2. **External access to the Cluster Manager UI** - The Cluster Manager UI must be accessible externally via a LoadBalancer service or Ingress so users can access it from their browser and the identity provider can redirect back after authentication. See [Connect to the admin console](/content/operate/kubernetes/8.2.0/re-clusters/connect-to-admin-console.md) for configuration options. + +3. A SAML 2.0-compatible identity provider (such as Okta, Azure AD, or similar) + +4. Admin access to your identity provider + +5. A TLS certificate and private key for the service provider (SP) + +> [!WARNING] +> SSO requires external access to the Cluster Manager UI. Port forwarding is not sufficient for SSO authentication because the identity provider needs to redirect users back to the UI after authentication. You must configure either a LoadBalancer service (via `spec.uiServiceType: LoadBalancer`) or an Ingress controller. + +### Step 1: Upload Service Provider certificate and private key + +The Service Provider certificate is used by the cluster to sign SAML requests and encrypt SAML responses. + +1. Create a secret with your service provider certificate and private key: + + ```sh + kubectl -n create secret generic sso-service-cert \ + --from-literal=name=sso_service \ + --from-file=certificate= \ + --from-file=key= + ``` + + The secret must: + - Reside within the same namespace as the `RedisEnterpriseCluster` custom resource. + - Include a `name` key explicitly set to `sso_service`. + - Include a `certificate` key with the public certificate in PEM format. + - Include a `key` key with the private key in PEM format. + + Replace the `` in the command above with your own values. + +2. Configure the service provider certificate in the `RedisEnterpriseCluster` custom resource: + + ```yaml + apiVersion: app.redislabs.com/v1 + kind: RedisEnterpriseCluster + metadata: + name: rec + spec: + nodes: 3 + certificates: + ssoServiceCertificateSecretName: sso-service-cert + sso: + enabled: false # SSO not yet active - set to true in Step 7 after completing configuration + enforceSSO: false # Optional: set to true to disable local authentication for non-admin users + saml: + spMetadataSecretName: sp-metadata # Optional: store SP metadata in a secret + serviceProvider: + baseAddress: "https://redis-ui.example.com:443" # Optional: customize base address + ``` + +3. Apply the configuration: + + ```sh + kubectl apply -f .yaml + ``` + +#### Configure Service Provider base address (optional) + +The base address is used to construct service provider URLs, such as the Assertion Consumer Service (ACS) URL and Single Logout (SLO) URL. + +If not specified, the base address is automatically determined from the REC Cluster Manager UI service: +- If the UI service type is `LoadBalancer` (configured via `spec.uiServiceType`), the load balancer address is used. +- Otherwise, the cluster-internal DNS name is used (for example, `rec-ui.svc.cluster.local`). +- The port defaults to 8443 if not specified. + +To explicitly set the base address, add it to the `serviceProvider` section: + +```yaml +spec: + sso: + saml: + serviceProvider: + baseAddress: "https://redis-ui.example.com:443" +``` + +Format: `[://][:]` + +Examples: +- `"https://redis-ui.example.com:443"` (recommended - explicit scheme) +- `"redis-ui.example.com:443"` (defaults to https://) +- `"http://redis-ui.example.com:9443"` (NOT recommended for production) + +> [!WARNING] +> Using `http://` is NOT recommended for production environments as it transmits sensitive SAML assertions in plaintext. Only use `http://` for testing or development purposes. + +**Usage guidelines:** +- **For LoadBalancer services:** Leave this field blank to use the default REC UI service, or set it explicitly to the LoadBalancer address for custom services. +- **For Ingress:** Set this to the ingress hostname and port (typically 443), for example, `"https://redis-ui.example.com:443"`. + +### Step 2: Download Service Provider metadata + +After applying the configuration, retrieve the service provider metadata to use when configuring your identity provider. + +#### Option A: Retrieve from Kubernetes secret + +If you configured `spMetadataSecretName` in Step 1, the operator creates a secret with the SP metadata: + +```sh +kubectl -n get secret sp-metadata -o jsonpath='{.data.sp_metadata}' | base64 -d > sp-metadata.xml +``` + +> [!NOTE] +> This secret is only created when the cluster is configured to use Kubernetes secrets (`spec.clusterCredentialSecretType` is unset or set to `"kubernetes"`). When using Vault secrets, use Option B instead. + +#### Option B: Retrieve from the API + +You can obtain the SP metadata directly from the Redis Enterprise Server API: + +```sh +kubectl -n exec -it -c redis-enterprise-node -- \ + curl -k -u ":" \ + https://localhost:9443/v1/cluster/sso/saml/metadata/sp > sp-metadata.xml +``` + +Replace ``, ``, and `` with your cluster details. + +### Step 3: Set up SAML app in your identity provider + +Use the Service Provider metadata from Step 2 to configure a SAML application in your identity provider. + +1. Sign in to your identity provider's admin console (for example, Okta, Azure AD, Google Workspace). + +2. Create a new SAML 2.0 application or integration. + +3. Upload the `sp-metadata.xml` file or manually configure the SAML settings using values from the metadata: + - **Entity ID (SP)**: Found in the `entityID` attribute of the metadata + - **Assertion Consumer Service (ACS) URL**: Found in the `AssertionConsumerService` element's `Location` attribute + - **Single Logout (SLO) URL**: Found in the `SingleLogoutService` element's `Location` attribute (if present) + +4. Configure the SAML assertion to include the following attributes: + - `email` - User's email address (required) + - `firstName` - User's first name (optional) + - `lastName` - User's last name (optional) + - `redisRoleMapping` - Role mapping for JIT user provisioning (required for new users) + +Refer to your identity provider's documentation for specific configuration steps. + +### Step 4: Download identity provider metadata + +After configuring the SAML app in your identity provider, download the identity provider metadata and certificate. + +1. In your identity provider's admin console, locate the SAML app you created in Step 3. + +2. Download the following: + - **IdP metadata XML**: Contains the IdP configuration (entity ID, SSO URL, SLO URL) + - **IdP certificate**: Public certificate used to verify SAML assertions + +Save these files for use in Step 5. + +### Step 5: Configure SAML identity provider in Redis Enterprise + +Now configure the identity provider details in your Redis Enterprise cluster. + +1. Create a secret with the Identity Provider certificate: + + ```sh + kubectl -n create secret generic sso-issuer-cert \ + --from-literal=name=sso_issuer \ + --from-file=certificate= + ``` + + The secret must: + - Reside within the same namespace as the `RedisEnterpriseCluster` custom resource. + - Include a `name` key explicitly set to `sso_issuer`. + - Include a `certificate` key with the IdP public certificate in PEM format. + - **Not** include a `key` field (only the public certificate is needed). + + Replace `` with the path to your IdP certificate file. + + > [!NOTE] + > While IdP metadata XML may contain the certificate, Redis Enterprise Server does not use it from there, so the certificate must be provided separately via this secret. + +2. Configure the IdP using one of the following options: + +#### Option A: Use IdP metadata XML (recommended) + +Using IdP metadata XML is the recommended approach as it's less error-prone. + +1. Create a secret with the IdP metadata: + + ```sh + kubectl -n create secret generic idp-metadata \ + --from-file=idp_metadata=.xml + ``` + + The secret must: + - Reside within the same namespace as the `RedisEnterpriseCluster` custom resource. + - Include an `idp_metadata` key with the IdP metadata XML content. + - The XML can be plain text or base64-encoded; the operator handles encoding as needed. + + Replace `` with the path to your IdP metadata XML file. + +2. Update the `RedisEnterpriseCluster` custom resource to add the IdP configuration: + + ```yaml + spec: + certificates: + ssoServiceCertificateSecretName: sso-service-cert + ssoIssuerCertificateSecretName: sso-issuer-cert + sso: + enabled: false # SSO not yet active - set to true in Step 7 after completing configuration + enforceSSO: false # Optional: set to true to disable local authentication for non-admin users + saml: + idpMetadataSecretName: idp-metadata + spMetadataSecretName: sp-metadata + serviceProvider: + baseAddress: "https://redis-ui.example.com:443" + ``` + +3. Apply the updated configuration: + + ```sh + kubectl apply -f .yaml + ``` + +#### Option B: Manual issuer configuration + +If IdP metadata XML is unavailable, you can manually configure the issuer settings. + +1. Update the `RedisEnterpriseCluster` custom resource with the IdP details: + + ```yaml + spec: + certificates: + ssoServiceCertificateSecretName: sso-service-cert + ssoIssuerCertificateSecretName: sso-issuer-cert + sso: + enabled: false # SSO not yet active - set to true in Step 7 after completing configuration + enforceSSO: false # Optional: set to true to disable local authentication for non-admin users + saml: + issuer: + entityID: "urn:sso:example:idp" + loginURL: "https://idp.example.com/sso/saml" + logoutURL: "https://idp.example.com/slo/saml" # optional + spMetadataSecretName: sp-metadata + serviceProvider: + baseAddress: "https://redis-ui.example.com:443" + ``` + + Replace the values with your identity provider's configuration: + + - `entityID`: Identity Provider entity ID (issuer identifier) + - `loginURL`: Identity Provider SSO login URL where SAML authentication requests are sent + - `logoutURL`: Identity Provider single logout URL (optional) + +2. Apply the updated configuration: + + ```sh + kubectl apply -f .yaml + ``` + +> [!NOTE] +> If both `idpMetadataSecretName` and `issuer` are provided, `idpMetadataSecretName` takes precedence and `issuer` is ignored. + +### Step 6: Assign SAML app to users + +In your identity provider's admin console, assign users to the SAML application you created in Step 3. + +1. Navigate to the SAML app in your identity provider. + +2. Assign existing users or groups to the application. + +3. For new users who will use just-in-time (JIT) provisioning, ensure the `redisRoleMapping` attribute is configured with appropriate Redis Enterprise roles. + +Refer to your identity provider's documentation for specific steps on assigning users to applications. + +### Step 7: Activate SSO + +Finally, activate SSO by enabling it in the `RedisEnterpriseCluster` custom resource. + +1. Update the `RedisEnterpriseCluster` custom resource to enable SSO: + + ```yaml + spec: + certificates: + ssoServiceCertificateSecretName: sso-service-cert + ssoIssuerCertificateSecretName: sso-issuer-cert + sso: + enabled: true + enforceSSO: false # Set to true to disable local authentication for non-admin users + saml: + idpMetadataSecretName: idp-metadata + spMetadataSecretName: sp-metadata + serviceProvider: + baseAddress: "https://redis-ui.example.com:443" + ``` + +2. Apply the configuration: + + ```sh + kubectl apply -f .yaml + ``` + +3. Test SSO by accessing the Cluster Manager UI and clicking **Sign in with SSO**. + +## Complete example + +Here's a complete example of a `RedisEnterpriseCluster` resource with SSO enabled: + +```yaml +apiVersion: app.redislabs.com/v1 +kind: RedisEnterpriseCluster +metadata: + name: rec +spec: + nodes: 3 + certificates: + ssoServiceCertificateSecretName: sso-service-cert + ssoIssuerCertificateSecretName: sso-issuer-cert + sso: + enabled: true + enforceSSO: false + saml: + idpMetadataSecretName: idp-metadata + spMetadataSecretName: sp-metadata + serviceProvider: + baseAddress: "https://redis-ui.example.com:443" +``` + +Refer to the `RedisEnterpriseCluster` [API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#specsso) for full details on the available fields. + +## Next steps + +After enabling SSO: + +1. Configure users in your identity provider with matching email addresses +2. Set up the `redisRoleMapping` attribute in your identity provider to assign appropriate roles for new users +3. Test both IdP-initiated and SP-initiated SSO flows + +For more information about Redis Enterprise Software security, see [Access control](/content/operate/rs/security/access-control/_index.md). diff --git a/content/operate/kubernetes/8.2.0/security/certificates/_index.md b/content/operate/kubernetes/8.2.0/security/certificates/_index.md new file mode 100644 index 0000000000..25971592a5 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/security/certificates/_index.md @@ -0,0 +1,48 @@ +--- +Title: Certificates and encryption +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Manage TLS certificates, client certificates, and internode encryption for Redis Software on Kubernetes. +hideListLinks: true +linkTitle: Certificates and encryption +weight: 30 +url: '/operate/kubernetes/8.2.0/security/certificates/' +--- + +Certificates and encryption use Kubernetes Secrets and cert-manager integration to provision, distribute, and rotate the TLS certificates that Redis Software relies on. The operator distributes referenced certificates across every cluster node. + +## How certificates work on Redis for Kubernetes + +- **Cluster certificates** live in Kubernetes Secrets that the `RedisEnterpriseCluster` spec references. The operator distributes them to every cluster node. +- **cert-manager** can issue and rotate certificates automatically. +- **Client certificates** live in a Secret that the database references for mutual TLS authentication. +- **Internode encryption** is configured on the REC spec. The operator places the certificates on each node. + +## What's the same as Redis Software + +The underlying certificate roles, requirements, and TLS behavior are unchanged. For concepts and reference details, see the existing Redis Software docs: + +- [Certificate roles and types](/content/operate/rs/security/certificates/_index.md) — which certificate is used for what. +- [Create certificates](/content/operate/rs/security/certificates/create-certificates.md) — certificate requirements (SAN, CN, validity). +- [Update certificates](/content/operate/rs/security/certificates/updating-certificates.md) — rotation considerations on Redis Software. +- [Monitor certificates](/content/operate/rs/security/certificates/monitor-certificates.md) — certificate expiration alerts. +- [Client certificate authentication](/content/operate/rs/security/certificates/certificate-based-authentication.md) — how the cluster validates client certificates. +- [TLS protocols](/content/operate/rs/security/encryption/tls/tls-protocols.md) and [ciphers](/content/operate/rs/security/encryption/tls/ciphers.md) — protocol and cipher selection. +- [Enable TLS](/content/operate/rs/security/encryption/tls/enable-tls.md) — TLS for management, replication, and client connections. +- [Internode encryption](/content/operate/rs/security/encryption/internode-encryption.md) — purpose and scope. +- [PEM encryption](/content/operate/rs/security/encryption/pem-encryption.md) — encrypted private keys. + +## What's different on Kubernetes + +- **You capture certificates in Kubernetes Secrets and reference them declaratively in the REC spec.** The operator applies them to the cluster through the Redis Software REST API — the same way certificates are applied on Redis Software, cluster-wide rather than file-by-file on each node. +- **cert-manager can issue and rotate certificates automatically**, replacing manual rotation steps. + +## In this section + +- [Manage REC certificates](/content/operate/kubernetes/8.2.0/security/certificates/manage-rec-certificates.md) — configure cluster TLS certificates. +- [cert-manager integration](/content/operate/kubernetes/8.2.0/security/certificates/cert-manager.md) — automate certificate issuance and rotation with cert-manager. +- [Add client certificates](/content/operate/kubernetes/8.2.0/security/certificates/add-client-certificates.md) — enable client certificate authentication for databases. +- [Internode encryption](/content/operate/kubernetes/8.2.0/security/certificates/internode-encryption.md) — enable encryption between cluster nodes. diff --git a/content/operate/kubernetes/8.2.0/security/certificates/add-client-certificates.md b/content/operate/kubernetes/8.2.0/security/certificates/add-client-certificates.md new file mode 100644 index 0000000000..14fc2d5163 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/security/certificates/add-client-certificates.md @@ -0,0 +1,45 @@ +--- +categories: +- docs +- operate +- kubernetes +aliases: [/operate/kubernetes/security/add-client-certificates/] +description: Add client certificates to your REDB custom resource. +linkTitle: Add client certificates +title: Add client certificates +weight: 30 +url: '/operate/kubernetes/8.2.0/security/certificates/add-client-certificates/' +--- + +For each client certificate you want to use with your database, you need to create a Kubernetes secret to hold it. You can then reference that secret in your Redis Enterprise database (REDB) custom resource spec. + +## Create a secret to hold the new certificate + +1. [Create the secret config file](https://kubernetes.io/docs/tasks/configmap-secret/managing-secret-using-config-file/) with the required fields shown below. + + ```yaml + apiVersion: v1 + kind: Secret + type: Opaque + metadata: + name: + namespace: + data: + cert: + ``` + +1. Apply the file to create the secret resource. + + ```bash + kubectl apply -f .yaml + ``` + +## Edit the REDB resource + +1. Add the secret name to the REDB custom resource (`redb.yaml`) with the `clientAuthenticationCertificates` property in the `spec` section. + + ```yaml + spec: + clientAuthenticationCertificates: + - + ``` \ No newline at end of file diff --git a/content/operate/kubernetes/8.2.0/security/certificates/cert-manager.md b/content/operate/kubernetes/8.2.0/security/certificates/cert-manager.md new file mode 100644 index 0000000000..2d90ad4b74 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/security/certificates/cert-manager.md @@ -0,0 +1,363 @@ +--- +Title: Integrate cert-manager with Redis for Kubernetes +alwaysopen: false +categories: +- docs +- operate +- kubernetes +aliases: [/operate/kubernetes/security/cert-manager/] +description: Automate TLS certificate management for Redis for Kubernetes using cert-manager. +linkTitle: cert-manager +weight: 20 +url: '/operate/kubernetes/8.2.0/security/certificates/cert-manager/' +--- + +[cert-manager](https://cert-manager.io/) is a Kubernetes add-on that automates the management and issuance of TLS certificates. The Redis operator integrates with cert-manager, so you can use automatically managed certificates for: + +- Redis Enterprise cluster (REC) components (API, CM, proxy, syncer, and others) +- Database replication with TLS +- LDAP client authentication +- SSO/SAML certificates + +Benefits of using cert-manager include: + +- **Automatic certificate renewal**: cert-manager handles certificate rotation before expiration. +- **Standardized management**: Use the same certificate management approach across your Kubernetes infrastructure. +- **Multiple certificate authorities**: Support for Let's Encrypt, private CAs, Vault, and more. +- **Automatic propagation**: For Active-Active databases, certificate changes automatically sync across all participating clusters. + +> [!WARNING] +> The cert-manager integration uses Kubernetes secrets. It is not compatible with Vault-based secret management (when `clusterCredentialSecretType: vault`). See [HashiCorp Vault integration](/content/operate/kubernetes/8.2.0/security/vault.md) for details. + +## Prerequisites + +- Kubernetes cluster with Redis Software for Kubernetes 8.0.18 or later +- cert-manager v1.19.0 or later installed + +If cert-manager is not already installed, see the [cert-manager installation documentation](https://cert-manager.io/docs/installation/). + +## How it works + +cert-manager creates standard Kubernetes TLS secrets with the following fields: + +- `tls.crt`: The certificate in PEM format +- `tls.key`: The private key in PEM format +- `ca.crt`: The root CA certificate + +The Redis Enterprise operator automatically recognizes these secrets and can use them interchangeably with manually created secrets. + +> [!NOTE] +> If you currently use opaque secrets for your certificates, you can switch to cert-manager's TLS secrets without any additional configuration changes to your Redis resources. + +### Supported secret formats + +The operator supports multiple field names for backward compatibility: + +| Purpose | Accepted field names | +|---------|---------------------| +| Certificate | `tls.crt`, `cert`, `certificate` | +| Private key | `tls.key`, `key` | +| CA certificate | `ca.crt` | + +> [!NOTE] +> Support for the `tls.crt` and `tls.key` field names requires Redis Software for Kubernetes 8.0.18 or later. For secret requirements on earlier versions, see [Manage REC certificates](/content/operate/kubernetes/8.2.0/security/certificates/manage-rec-certificates.md). + +> [!NOTE] +> The `ca.crt` field is automatically appended to the certificate chain when present. cert-manager typically populates this field when it has access to the root certificate. + +## Quick start + +After you install cert-manager and configure an [`Issuer` or `ClusterIssuer`](https://cert-manager.io/docs/concepts/issuer/), create a `Certificate` resource to generate TLS secrets, then reference the secret name in your Redis custom resources. + +### Create a certificate and configure your REC + +The following example creates a certificate and references the generated secret in a `RedisEnterpriseCluster` resource: + +```yaml +apiVersion: cert-manager.io/v1 +kind: Certificate +metadata: + name: redis-api-cert + namespace: redis-namespace +spec: + secretName: redis-api-tls + duration: 2160h # 90 days + renewBefore: 360h # 15 days before expiration + issuerRef: + name: my-issuer + kind: ClusterIssuer +--- +apiVersion: app.redislabs.com/v1 +kind: RedisEnterpriseCluster +metadata: + name: rec + namespace: redis-namespace +spec: + nodes: 3 + certificates: + apiCertificateSecretName: redis-api-tls +``` + +The operator automatically reads the certificate from the `redis-api-tls` secret created by cert-manager. No additional configuration is needed, even if you are migrating from manually created opaque secrets. + +## Secure all cluster components + +Request certificates for each component and reference them in the REC: + +```yaml +apiVersion: app.redislabs.com/v1 +kind: RedisEnterpriseCluster +metadata: + name: rec +spec: + nodes: 3 + certificates: + apiCertificateSecretName: api-tls + cmCertificateSecretName: cm-tls + proxyCertificateSecretName: proxy-tls + syncerCertificateSecretName: syncer-tls + metricsExporterCertificateSecretName: metrics-tls +``` + +Each secret name corresponds to a `Certificate` resource managed by cert-manager. For details on these fields, see the [RedisEnterpriseCluster API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md). + +## Database replication with TLS + +For database replication with client certificates: + +```yaml +apiVersion: cert-manager.io/v1 +kind: Certificate +metadata: + name: replication-client-cert + namespace: redis-namespace +spec: + secretName: replication-client-tls + issuerRef: + name: my-ca-issuer + kind: ClusterIssuer + commonName: replication-client +--- +apiVersion: app.redislabs.com/v1alpha1 +kind: RedisEnterpriseDatabase +metadata: + name: target-db +spec: + replicaSources: + - replicaSourceType: SECRET + replicaSourceName: source-uri-secret + clientKeySecret: replication-client-tls +``` + +## LDAP client authentication + +When using LDAP with client certificate authentication: + +```yaml +apiVersion: cert-manager.io/v1 +kind: Certificate +metadata: + name: ldap-client-cert + namespace: redis-namespace +spec: + secretName: ldap-client-tls + issuerRef: + name: ldap-ca-issuer + kind: ClusterIssuer + commonName: redis-ldap-client +--- +apiVersion: app.redislabs.com/v1 +kind: RedisEnterpriseCluster +metadata: + name: rec +spec: + nodes: 3 + certificates: + ldapClientCertificateSecretName: ldap-client-tls + ldap: + protocol: LDAPS + servers: + - host: ldap.example.com + port: 636 +``` + +For more details on LDAP configuration, see [Enable LDAP authentication](/content/operate/kubernetes/8.2.0/security/authentication/ldap.md). + +## Active-Active databases with automatic certificate sync + +For Active-Active databases, certificate updates to proxy or syncer certificates automatically trigger synchronization across all participating clusters: + +```yaml +apiVersion: app.redislabs.com/v1 +kind: RedisEnterpriseCluster +metadata: + name: rec +spec: + nodes: 3 + certificates: + proxyCertificateSecretName: proxy-tls + syncerCertificateSecretName: syncer-tls +``` + +When cert-manager renews these certificates, the operator: + +1. Detects the secret change. +1. Updates the certificate generation counter. +1. Triggers a CRDB force update automatically. +1. Synchronizes the new certificates to all participating clusters. + +No manual intervention is required. + +## Use production certificate authorities + +### Let's Encrypt and ACME issuers + +You can use Let's Encrypt and other ACME issuers, but not as an out-of-the-box integration. ACME issuers don't populate the `ca.crt` field of the generated TLS secret with the root certificate, so the secret that cert-manager creates doesn't include the full certificate chain. + +Redis Software needs the full chain, including the root CA, to trust the certificate. Whenever cert-manager doesn't populate `ca.crt` with the root certificate, supply the full chain yourself: + +1. Assemble the full certificate chain, including the root CA certificate. +1. Create a Kubernetes secret that contains the full chain. +1. Reference that secret in your Redis custom resource instead of the secret that cert-manager generates. + +> [!NOTE] +> This applies to any issuer that doesn't populate `ca.crt` with the root certificate, not only Let's Encrypt. + +For production environments where the issuer supplies the full chain automatically, such as a private CA or HashiCorp Vault, no extra steps are required. + +### Private CA + +For internal deployments with a private certificate authority: + +```yaml +apiVersion: cert-manager.io/v1 +kind: ClusterIssuer +metadata: + name: private-ca-issuer +spec: + ca: + secretName: ca-key-pair +``` + +The `ca-key-pair` secret must contain your CA's certificate and private key. + +## Migrate from manual certificate management + +If you currently use manually created secrets, you can migrate to cert-manager without downtime. + +1. Install cert-manager and create issuers (as shown in previous examples). + +1. Create `Certificate` resources with **different secret names** than your current secrets: + + ```yaml + apiVersion: cert-manager.io/v1 + kind: Certificate + metadata: + name: new-api-cert + spec: + secretName: api-tls-new + issuerRef: + name: my-issuer + kind: ClusterIssuer + ``` + +1. Update your REC to reference the new secrets: + + ```yaml + apiVersion: app.redislabs.com/v1 + kind: RedisEnterpriseCluster + metadata: + name: rec + spec: + certificates: + apiCertificateSecretName: api-tls-new + ``` + + The operator updates the cluster with the new certificates. For Active-Active databases, the changes automatically sync to all clusters. + +1. After verifying the new certificates work correctly, delete the old manually created secrets: + + ```bash + kubectl delete secret old-api-tls -n redis-namespace + ``` + +## Certificate renewal and monitoring + +### Automatic renewal + +cert-manager automatically renews certificates before they expire based on the `renewBefore` setting in the `Certificate` spec: + +```yaml +spec: + duration: 2160h # 90 days + renewBefore: 360h # Renew 15 days before expiration +``` + +### Monitor certificate status + +Check certificate status: + +```bash +kubectl get certificate -n redis-namespace +``` + +View detailed certificate information: + +```bash +kubectl describe certificate redis-api-cert -n redis-namespace +``` + +## Troubleshooting + +### Certificate not issued + +Check the certificate status and cert-manager logs: + +```bash +kubectl describe certificate -n +kubectl logs -n cert-manager deployment/cert-manager +``` + +Common issues: + +- **Issuer not ready**: Verify your `Issuer` or `ClusterIssuer` is configured correctly. +- **DNS validation failure**: For ACME issuers, ensure DNS records are correctly configured. +- **Rate limits**: Let's Encrypt has rate limits. Use the staging environment for testing. + +### Operator not detecting certificate changes + +Verify the secret exists and has the correct format: + +```bash +kubectl get secret -n -o yaml +``` + +Confirm the secret contains `tls.crt` and `tls.key` fields. Check operator logs: + +```bash +kubectl logs -n deployment/redis-enterprise-operator +``` + +### Certificate chain issues + +If you encounter certificate chain validation errors: + +1. Verify the `ca.crt` field is present in the secret (cert-manager populates this automatically when it has access to the root CA). +1. If `ca.crt` is not present, ensure the certificate in `tls.crt` includes the full chain inline. +1. Confirm the certificate chain is in the correct order: leaf certificate first, then intermediates, then root. + +## Best practices + +- **Use appropriate certificate lifetimes**: 90 days with a 15-day renewal window for production. Use shorter lifetimes in development to test renewal. +- **Configure proper DNS names**: Include all necessary DNS names and SANs in your `Certificate` spec. +- **Monitor certificate expiration**: Set up alerts for certificate expiration, even with automatic renewal. +- **Use `ClusterIssuers` for shared issuers**: If multiple namespaces need certificates from the same CA, use `ClusterIssuers` instead of namespace-scoped `Issuers`. +- **Back up CA private keys**: If using a private CA, ensure the CA private key secret is backed up. + +## See also + +- [cert-manager documentation](https://cert-manager.io/docs/) +- [Manage REC certificates](/content/operate/kubernetes/8.2.0/security/certificates/manage-rec-certificates.md) +- [RedisEnterpriseCluster API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md) +- [RedisEnterpriseDatabase API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md) +- [HashiCorp Vault integration](/content/operate/kubernetes/8.2.0/security/vault.md) diff --git a/content/operate/kubernetes/8.2.0/security/certificates/internode-encryption.md b/content/operate/kubernetes/8.2.0/security/certificates/internode-encryption.md new file mode 100644 index 0000000000..dcff68a467 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/security/certificates/internode-encryption.md @@ -0,0 +1,137 @@ +--- +Title: Enable internode encryption +categories: +- docs +- operate +- kubernetes +aliases: [/operate/kubernetes/security/internode-encryption/] +description: Enable encryption for communication between REC nodes and configure custom certificates. +linkTitle: Internode encryption +weight: 40 +url: '/operate/kubernetes/8.2.0/security/certificates/internode-encryption/' +--- + +Internode encryption provides added security by encrypting communication between nodes in your Redis Enterprise cluster (REC). + +## Enable internode encryption + +Enable internode encryption in the `spec` section of your REC custom resource file. + +```yaml +spec: + dataInternodeEncryption: true +``` + +This change will apply to all databases created in the REC. You can override the cluster-wide setting for individual databases. + +Edit your Redis Enterprise database (REDB) custom resource file to disable internode encryption for only that database. + +```yaml +spec: + dataInternodeEncryption: false +``` + +To learn more about internode encryption, see [Internode encryption for Redis Enterprise Software](/content/operate/rs/security/encryption/internode-encryption.md). + +## Use custom certificates for internode encryption + +By default, Redis Enterprise uses self-signed certificates for internode encryption. You can provide your own certificates for both control plane and data plane internode encryption by storing them in Kubernetes secrets and referencing them in your REC specification. + +### Prerequisites + +- Certificates must be in PEM format +- You must create the Kubernetes secrets before referencing them in the REC spec +- Certificates should include the full certificate chain if using a certificate authority + +### Create secrets for internode encryption certificates + +Create Kubernetes secrets to store your internode encryption certificates. You need separate secrets for control plane and data plane encryption. + +1. Create a secret for control plane internode encryption: + + ```sh + kubectl create secret generic cp-internode-cert \ + --from-file=certificate= \ + --from-file=key= \ + --from-literal=name=cp_internode_encryption + ``` + +2. Create a secret for data plane internode encryption: + + ```sh + kubectl create secret generic dp-internode-cert \ + --from-file=certificate= \ + --from-file=key= \ + --from-literal=name=dp_internode_encryption + ``` + +### Configure certificates in REC spec + +Add the certificate secret names to the `certificates` section of your REC specification: + +```yaml +spec: + dataInternodeEncryption: true + certificates: + cpInternodeEncryptionCertificateSecretName: cp-internode-cert + dpInternodeEncryptionCertificateSecretName: dp-internode-cert +``` + +You can configure one or both certificate types. If you don't specify a certificate secret name, the cluster uses a self-signed certificate for that encryption type. + +Apply the updated REC specification: + +```sh +kubectl apply -f .yaml +``` + +### Certificate rotation + +You can rotate internode encryption certificates using either of these methods: + +#### Method 1: Update the existing secret + +Edit the certificate data in the existing Kubernetes secret. The operator automatically detects the change and applies the new certificate. + +```sh +kubectl create secret generic cp-internode-cert \ + --from-file=certificate= \ + --from-file=key= \ + --from-literal=name=cp_internode_encryption \ + --dry-run=client -o yaml | kubectl apply -f - +``` + +#### Method 2: Create a new secret and update the REC spec + +1. Create a new secret with the updated certificate: + + ```sh + kubectl create secret generic cp-internode-cert-new \ + --from-file=certificate= \ + --from-file=key= \ + --from-literal=name=cp_internode_encryption + ``` + +2. Update the REC specification to reference the new secret: + + ```yaml + spec: + certificates: + cpInternodeEncryptionCertificateSecretName: cp-internode-cert-new + ``` + +3. Apply the updated REC specification: + + ```sh + kubectl apply -f .yaml + ``` + +### Certificate lifecycle + +When you remove a certificate secret reference from the REC specification, the operator does not delete the certificate from the Redis Enterprise cluster. The cluster continues to use the previously configured certificate until you explicitly replace it or the cluster reverts to using a self-signed certificate. + +## More info + +- [Manage REC certificates](/content/operate/kubernetes/8.2.0/security/certificates/manage-rec-certificates.md) - General certificate management for Redis Enterprise clusters +- [Configuration secrets](/content/operate/kubernetes/8.2.0/security/authentication/configuration-secrets.md) - Best practices for storing configuration in Kubernetes secrets +- [Internode encryption for Redis Enterprise Software](/content/operate/rs/security/encryption/internode-encryption.md) - Detailed information about how internode encryption works diff --git a/content/operate/kubernetes/8.2.0/security/certificates/manage-rec-certificates.md b/content/operate/kubernetes/8.2.0/security/certificates/manage-rec-certificates.md new file mode 100644 index 0000000000..a3ff90be5c --- /dev/null +++ b/content/operate/kubernetes/8.2.0/security/certificates/manage-rec-certificates.md @@ -0,0 +1,130 @@ +--- +Title: Manage Redis Enterprise cluster (REC) certificates +alwaysopen: false +categories: +- docs +- operate +- kubernetes +aliases: [/operate/kubernetes/security/manage-rec-certificates/] +description: Install your own certificates to replace the self-signed certificates used by a Redis Enterprise cluster on Kubernetes. +linkTitle: Manage REC certificates +weight: 10 +url: '/operate/kubernetes/8.2.0/security/certificates/manage-rec-certificates/' +--- + +Redis Software for Kubernetes generates self-signed TLS certificates for each new cluster. You can replace any of those certificates with your own. + +You can manage REC certificates in two ways: + +- **[Method 1: Manage certificates with the REC custom resource](#method-1-manage-certificates-with-the-rec-custom-resource)** (recommended). Store each certificate in a Kubernetes secret and reference the secret from the REC custom resource. The operator applies the certificate and keeps the cluster in sync with the secret. Use this method whenever the certificate type is exposed in `spec.certificates`. +- **[Method 2: Manage certificates with the Redis Software REST API](#method-2-manage-certificates-with-the-redis-software-rest-api)**. Call the cluster's REST API directly, bypassing the operator. Use this method only when you need to follow the Redis Software procedure for a cluster that does not define the certificate in `spec.certificates`. The operator overwrites changes made this way if the same certificate is also defined in the REC custom resource. + +For the list of certificates and what each one encrypts, see the [certificates table](/content/operate/rs/security/certificates/_index.md). + +## Method 1: Manage certificates with the REC custom resource + +This is the Kubernetes-native method. The operator detects changes to a referenced secret and rotates the certificate without manual intervention. You can create the secret manually, or have [cert-manager](/content/operate/kubernetes/8.2.0/security/certificates/cert-manager.md) issue and renew it automatically. + +### Supported certificates + +The REC custom resource lets you replace these certificates through `spec.certificates`: + +| Certificate | REC custom resource field | Certificate name in Redis Software | +| ---------------------------------- | --------------------------------------------- | ---------------------------------- | +| API | `apiCertificateSecretName` | `api` | +| Cluster Manager UI | `cmCertificateSecretName` | `cm` | +| Control plane internode encryption | `cpInternodeEncryptionCertificateSecretName` | `cp_internode_encryption` | +| Data plane internode encryption | `dpInternodeEncryptionCertificateSecretName` | `dp_internode_encryption` | +| LDAP client | `ldapClientCertificateSecretName` | `ldap_client` | +| Metrics exporter | `metricsExporterCertificateSecretName` | `metrics_exporter` | +| Proxy | `proxyCertificateSecretName` | `proxy` | +| SSO issuer (SAML IdP) | `ssoIssuerCertificateSecretName` | `sso_issuer` | +| SSO service (SAML SP) | `ssoServiceCertificateSecretName` | `sso_service` | +| Syncer | `syncerCertificateSecretName` | `syncer` | + +Rotating any of these certificates does not restart REC pods. + +### Step 1: Create a secret for the certificate + +Create a Kubernetes [secret](https://kubernetes.io/docs/concepts/configuration/secret/) that holds the PEM-encoded certificate and key: + +```sh +kubectl create secret generic \ + --from-file=certificate= \ + --from-file=key= +``` + +Choose any value for ``; you'll reference it from `spec.certificates` in [Step 2](#step-2-reference-the-secret-in-the-rec-custom-resource). The operator resolves which Redis Software certificate to replace from the REC field name, not from the secret. + +#### Supported secret keys + +The operator accepts several key names for the certificate and private key, so you can use either an opaque secret or a [kubernetes.io/tls](https://kubernetes.io/docs/concepts/configuration/secret/#tls-secrets) secret (the format that cert-manager produces): + +| Field | Accepted secret keys | +| ----------- | ------------------------------------- | +| Certificate | `cert`, `certificate`, or `tls.crt` | +| Private key | `key` or `tls.key` | + +> [!NOTE] +> On Redis Software for Kubernetes versions older than 8.0.18, also include `--from-literal=name=` in the `kubectl create secret` command, where `` is the value from the **Certificate name in Redis Software** column in the [supported certificates](#supported-certificates) table. + +For internode encryption certificates, see [Internode encryption](/content/operate/kubernetes/8.2.0/security/certificates/internode-encryption.md) for the full setup, which covers enabling internode encryption alongside the certificate configuration. + +### Step 2: Reference the secret in the REC custom resource + +Edit the REC custom resource and add a `certificates` section under `spec`. Include only the fields for the certificates you are replacing; omit the rest. + +```yaml +spec: + certificates: + apiCertificateSecretName: + cmCertificateSecretName: + cpInternodeEncryptionCertificateSecretName: + dpInternodeEncryptionCertificateSecretName: + ldapClientCertificateSecretName: + metricsExporterCertificateSecretName: + proxyCertificateSecretName: + ssoIssuerCertificateSecretName: + ssoServiceCertificateSecretName: + syncerCertificateSecretName: +``` + +Apply the updated REC custom resource: + +```sh +kubectl apply -f .yaml +``` + +The operator detects the change and rotates the certificate on the cluster. New client connections use the new certificate; existing connections continue with the old one until they reconnect. + +### Step 3: Verify the rotation (optional) + +To confirm that the new certificate is in place, call the Redis Software REST API and list the active cluster certificates: + +```http +GET /v1/cluster/certificates +``` + +## Method 2: Manage certificates with the Redis Software REST API + +Use the Redis Software REST API or `rladmin` directly against the cluster, bypassing the operator. + +> [!WARNING] +> If `spec.certificates` in the REC custom resource defines the same certificate, the operator overwrites your API change. Before you update a certificate through the REST API, remove the corresponding field from `spec.certificates`, or apply the same change in both places. + +For the procedure, including the `rladmin` and REST API examples, see [Update certificates](/content/operate/rs/security/certificates/updating-certificates.md). + +After the update, verify the rotation as described in [Step 3](#step-3-verify-the-rotation-optional). + +## Active-Active database certificate updates + +The operator automates certificate updates for [Active-Active](/content/operate/kubernetes/8.2.0/active-active/_index.md) databases. When you update the proxy or syncer certificate secret referenced by the REC, the operator detects the change and propagates the new certificate to all participating clusters. + +This automation applies whether you manage the secret directly or with [cert-manager](/content/operate/kubernetes/8.2.0/security/certificates/cert-manager.md#active-active-databases-with-automatic-certificate-sync). + +## More info + +- [Update certificates](/content/operate/rs/security/certificates/updating-certificates.md) +- [Install your own certificates](/content/operate/rs/security/certificates/create-certificates.md) +- [Certificates table](/content/operate/rs/security/certificates/_index.md) +- [Glossary/Transport Layer Security (TLS)](/content/glossary/_index.md#letter-t) diff --git a/content/operate/kubernetes/8.2.0/security/fips.md b/content/operate/kubernetes/8.2.0/security/fips.md new file mode 100644 index 0000000000..604a870ee9 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/security/fips.md @@ -0,0 +1,44 @@ +--- +Title: Enable FIPS 140-3 compliance mode +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Run Redis Software for Kubernetes in FIPS 140-3 compliance mode. +linkTitle: FIPS compliance +weight: 60 +url: '/operate/kubernetes/8.2.0/security/fips/' +--- + +You can run your Redis Software for Kubernetes cluster in FIPS 140-3 compliance mode, which uses cryptographic libraries based on FIPS 140-3 validated modules and enables the use of FIPS-approved cryptographic algorithms. FIPS mode is available starting with operator version 8.2.0. + +When you enable FIPS mode, the operator deploys a FIPS-compatible image in place of the standard one. + +## Prerequisites + +FIPS compliance depends on the underlying platform, not only on Redis: + +- **FIPS-configured nodes.** A container runs in FIPS mode only if the host node is configured for FIPS. On OpenShift, the cluster must be installed in FIPS mode. Redis does not configure the host; this is the responsibility of your platform. +- **amd64 architecture.** The FIPS-compatible Redis Software image is built for `amd64` only. There is no arm64 FIPS image. + +## Enable FIPS mode + +Set `spec.securityContext.fips.enabled` to `true` on the `RedisEnterpriseCluster` (REC): + +```yaml +apiVersion: app.redislabs.com/v1 +kind: RedisEnterpriseCluster +metadata: + name: rec +spec: + securityContext: + fips: + enabled: true +``` + +When enabled, the operator adds a `.fips` suffix to the image version tag and pulls the matching FIPS-compatible image (for example, `redislabs/redis:.fips`). The field defaults to `false`. + +## Pull the image by digest + +If you pin the image by digest (`spec.redisEnterpriseImageSpec`) instead of by tag, the operator can't add the `.fips` suffix. In that case, supply a digest that points to a FIPS-compatible image yourself. diff --git a/content/operate/kubernetes/8.2.0/security/vault.md b/content/operate/kubernetes/8.2.0/security/vault.md new file mode 100644 index 0000000000..af5a3d142f --- /dev/null +++ b/content/operate/kubernetes/8.2.0/security/vault.md @@ -0,0 +1,558 @@ +--- +Title: Integrate HashiCorp Vault with Redis Enterprise for Kubernetes +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Configure HashiCorp Vault as the centralized secret management system for Redis Enterprise for Kubernetes. +linkTitle: HashiCorp Vault integration +weight: 40 +url: '/operate/kubernetes/8.2.0/security/vault/' +--- + +You can configure HashiCorp Vault as the centralized secret management system for the Redis Enterprise Kubernetes operator, replacing the default Kubernetes secrets. This integration provides enhanced security, centralized secret management, and advanced features like secret rotation and audit logging. + +## What secrets are managed by Vault? + +When Vault integration is enabled, all secrets referenced in Redis Enterprise custom resources are retrieved from Vault instead of Kubernetes secrets, including: + +{{}} +| **Category** | **Secret Type** | **API Field** | **Description** | +|---|---|---|---| +| **Cluster secrets** | | | | +| | [Cluster credentials](/content/operate/kubernetes/8.2.0/deployment/quick-start.md) | [`clusterCredentialSecretName`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#spec) | Authentication credentials for cluster access | +| | [License](/content/operate/kubernetes/8.2.0/deployment/quick-start.md#install-the-license) | [`licenseSecretName`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#spec) | Redis Enterprise license key | +| | [API certificate](/content/operate/kubernetes/8.2.0/security/certificates/manage-rec-certificates.md) | [`apiCertificateSecretName`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#speccertificates) | TLS certificate for API server | +| | [Cluster manager certificate](/content/operate/kubernetes/8.2.0/security/certificates/manage-rec-certificates.md) | [`cmCertificateSecretName`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#speccertificates) | TLS certificate for cluster manager | +| | [Metrics exporter certificate](/content/operate/kubernetes/8.2.0/re-clusters/connect-prometheus-operator.md) | [`metricsExporterCertificateSecretName`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#speccertificates) | TLS certificate for metrics exporter | +| | [Proxy certificate](/content/operate/kubernetes/8.2.0/security/certificates/manage-rec-certificates.md) | [`proxyCertificateSecretName`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#speccertificates) | TLS certificate for proxy | +| | [Syncer certificate](/content/operate/kubernetes/8.2.0/active-active/_index.md) | [`syncerCertificateSecretName`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#speccertificates) | TLS certificate for Active-Active syncer | +| | [LDAP client certificate](/content/operate/kubernetes/8.2.0/security/authentication/ldap.md) | [`ldapClientCertificateSecretName`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#speccertificates) | TLS certificate for LDAP client authentication | +| | [LDAP bind credentials](/content/operate/kubernetes/8.2.0/security/authentication/ldap.md) | [`bindCredentialsSecretName`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#specldap) | Credentials for authenticating to the LDAP server | +| | [CPINE certificate](/content/operate/kubernetes/8.2.0/security/certificates/manage-rec-certificates.md) | [`cpInternodeEncryptionCertificateSecretName`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#speccertificates) | TLS certificate for Control Plane Internode Encryption (CPINE) | +| | [DPINE certificate](/content/operate/kubernetes/8.2.0/security/certificates/manage-rec-certificates.md) | [`dpInternodeEncryptionCertificateSecretName`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#speccertificates) | TLS certificate for Data Plane Internode Encryption (DPINE) | +| | [SSO service certificate](/content/operate/kubernetes/8.2.0/security/authentication/sso.md) | [`ssoServiceCertificateSecretName`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#speccertificates) | Service Provider (SP) certificate for SAML SSO | +| | [SSO issuer certificate](/content/operate/kubernetes/8.2.0/security/authentication/sso.md) | [`ssoIssuerCertificateSecretName`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#speccertificates) | Identity Provider (IdP) public certificate for SAML SSO | +| | [SSO IdP metadata](/content/operate/kubernetes/8.2.0/security/authentication/sso.md) | [`idpMetadataSecretName`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#specssosaml) | SAML Identity Provider metadata XML | +| | [User-defined module credentials](/content/operate/kubernetes/8.2.0/re-databases/modules.md) | [`credentialsSecret`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#specuserdefinedmodulessourcehttps) | Credentials for downloading user-defined modules from authenticated repositories | +| **Database secrets** | | | | +| | [Database passwords](/content/operate/kubernetes/8.2.0/networking/database-connectivity.md#credentials-and-secrets-management) | Various | Passwords for Redis databases | +| | [Replica source client TLS key](/content/operate/kubernetes/8.2.0/re-databases/replica-redb.md) | [`clientKeySecret`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md#specreplicasources) | Client TLS key for cross-cluster replication | +| | [Replica source server certificate](/content/operate/kubernetes/8.2.0/re-databases/replica-redb.md) | [`serverCertSecret`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md#specreplicasources) | Server certificate for cross-cluster replication | +| | [S3 backup credentials](/content/operate/kubernetes/8.2.0/re-databases/_index.md) | [`awsSecretName`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md#specbackups3) | AWS S3 storage credentials for database backups | +| | [SFTP backup credentials](/content/operate/kubernetes/8.2.0/re-databases/_index.md) | [`sftpSecretName`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md#specbackupsftp) | SFTP storage credentials for database backups | +| | [Swift backup credentials](/content/operate/kubernetes/8.2.0/re-databases/_index.md) | [`swiftSecretName`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md#specbackupswift) | Swift storage credentials for database backups | +| | [Azure Blob backup credentials](/content/operate/kubernetes/8.2.0/re-databases/_index.md) | [`absSecretName`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md#specbackupabs) | Azure Blob storage credentials for database backups | +| | [Google Cloud backup credentials](/content/operate/kubernetes/8.2.0/re-databases/_index.md) | [`gcsSecretName`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md#specbackupgcs) | Google Cloud storage credentials for database backups | +| | [Client authentication certificates](/content/operate/kubernetes/8.2.0/security/certificates/add-client-certificates.md) | Various | TLS client certificates for authentication | +| **Other secrets** | | | | +| | [Remote cluster secrets](/content/operate/kubernetes/8.2.0/active-active/_index.md) | [`secretName`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_remote_cluster_api.md#spec) | Credentials for Redis Enterprise Remote Cluster (RERC) configurations | +| | [Active-Active database secrets](/content/operate/kubernetes/8.2.0/active-active/_index.md) | [`globalConfigurations`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_active_active_database_api.md#specglobalconfigurations) | All secret names specified in REAADB global configurations | +{{}} + +> [!NOTE] +> The SSO Service Provider (SP) metadata secret (`spMetadataSecretName`) is **not** managed by Vault. This secret is operator-generated and is not written to Vault. To retrieve SP metadata when using Vault, fetch it directly from the Redis Enterprise Server API (`GET /v1/cluster/sso/saml/metadata/sp`). + +For complete details on supported secrets, see the [`RedisEnterpriseCluster` API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md) and [`RedisEnterpriseDatabase` API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md). + +### Secret path structure + +Vault secrets follow a hierarchical path structure: +``` +// +``` + +Default example: +``` +secret/data/redisenterprise-redis-ns/my-cluster +secret/data/redisenterprise-redis-ns/my-database-password +secret/data/redisenterprise-redis-ns/tls-certificates +``` + +> [!NOTE] +> When using OpenShift, replace `kubectl` commands with `oc` throughout this guide. + +## Prerequisites + +Before integrating Redis Enterprise operator with HashiCorp Vault, ensure you have the following components properly configured: + +HashiCorp Vault Requirements: + +- Vault instance: HashiCorp Vault v1.15.2+ with TLS and network connectivity to your Kubernetes cluster +- Authentication method: Configure Kubernetes authentication method in Vault (see [HashiCorp's Kubernetes Auth documentation](https://developer.hashicorp.com/vault/docs/auth/kubernetes)) +- Secret engine: Enable and configure KV version 2 secret engine + - Default mount path: `secret/` (configurable) + - Used to store all Redis Enterprise secrets + - Supports versioning and metadata + +Kubernetes Requirements: + +- Vault Agent Injector: Deploy the HashiCorp Vault Agent Injector + - Enables automatic secret injection into pods + - See [Vault Agent Injector tutorial](https://developer.hashicorp.com/vault/tutorials/kubernetes/kubernetes-sidecar) +- Network access: Ensure Kubernetes cluster can reach Vault + - Configure appropriate network policies and firewall rules + - Vault typically runs on port 8200 (HTTPS) +- Service accounts: Proper RBAC configuration for operator service accounts + +Vault editions: + +This guide supports both Vault Community and Enterprise editions: + +- Vault Community: Use all commands without `-namespace` flags or `VAULT_NAMESPACE` parameters +- Vault Enterprise: Supports namespaces for logical isolation and multi-tenancy (separate from Kubernetes namespaces) + +Minimum token TTL: + +Configure Vault token policies with minimum TTL of 1 hour: +- Prevents frequent token renewal overhead +- Ensures stable operation during maintenance windows +- See [Vault token management](https://developer.hashicorp.com/vault/tutorials/tokens/token-management) + +### Deployment scenarios + +This guide covers the most common deployment scenario with the following assumptions: + +- Vault Enterprise with namespace support (adapt for Community Edition by removing namespace parameters) +- Multiple Redis Enterprise clusters in the same Kubernetes cluster +- Namespace isolation using Kubernetes namespace suffixes for Vault configurations +- Production security with proper RBAC and network policies + +> [!NOTE] +> Multi-cluster considerations: When deploying across multiple Kubernetes clusters with identical namespace names, additional prefixing may be required to avoid Vault path conflicts. + +## Configure the operator + +1. Configure Vault policies and roles + + Create a policy that grants the Redis Enterprise operator read access to secrets: + + ```bash + vault policy write -namespace= redisenterprise- - </*" { + capabilities = ["create", "read", "update", "delete", "list"] + } + path "secret/metadata/redisenterprise-/*" { + capabilities = ["list"] + } + EOF + ``` + + Parameter explanation: + - ``: Your Vault Enterprise namespace (omit for Community Edition) + - ``: Kubernetes namespace where Redis Enterprise operator is deployed + + Configure a Vault role that binds the operator's service account to the policy: + + ```bash + vault write -namespace= auth//role/redis-enterprise-operator- \ + bound_service_account_names="redis-enterprise-operator" \ + bound_service_account_namespaces= \ + policies=redisenterprise- + ``` + + Parameter explanation: + - ``: Kubernetes auth method path in Vault (default: `kubernetes`) + - Role name includes namespace for multi-tenant isolation + +2. Configure operator environment + + Create a ConfigMap with Vault configuration for the Redis Enterprise operator: + + ```yaml + # operator-environment-config.yaml + apiVersion: v1 + kind: ConfigMap + metadata: + name: operator-environment-config + namespace: + data: + CREDENTIAL_TYPE: "vault" + VAULT_SERVER_FQDN: "" + VAULT_SERVICE_PORT_HTTPS: "8200" + VAULT_SECRET_ROOT: "secret" + VAULT_SECRET_PREFIX: "redisenterprise-" + VAULT_ROLE: "redis-enterprise-operator-" + VAULT_AUTH_PATH: "" + VAULT_NAMESPACE: "" + VAULT_CACHE_SECRET_EXPIRATION_SECONDS: "120" + ``` + + Apply the configuration: + + ```bash + kubectl apply -f operator-environment-config.yaml + ``` + + Configuration parameters: + + {{}} + | Parameter | Description | Default | Required | + |-----------|-------------|---------|----------| + | `CREDENTIAL_TYPE` | Must be set to `"vault"` to enable Vault integration | - | Yes | + | `VAULT_SERVER_FQDN` | Vault server hostname (e.g., `vault.vault-ns.svc.cluster.local`) | - | Yes | + | `VAULT_SERVICE_PORT_HTTPS` | Vault HTTPS port | `8200` | Yes | + | `VAULT_SECRET_ROOT` | KV-v2 secret engine mount path | `secret` | Yes | + | `VAULT_SECRET_PREFIX` | Prefix for all Redis Enterprise secrets | `redisenterprise` | Yes | + | `VAULT_ROLE` | Vault role for operator authentication | `redis-enterprise-operator` | Yes | + | `VAULT_AUTH_PATH` | Kubernetes auth method path | `kubernetes` | Yes | + | `VAULT_NAMESPACE` | Vault Enterprise namespace | - | Enterprise only | + | `VAULT_CACHE_SECRET_EXPIRATION_SECONDS` | Secret cache duration | `120` | No | + {{}} + + Secret path construction: Secrets are stored at `/data//` + +
    + +3. Deploy the operator + + Deploy the Redis Enterprise operator following the [standard installation guide](/content/operate/kubernetes/8.2.0/deployment/_index.md). + + > [!WARNING] + > The operator pod will not be ready until the admission controller secret is stored in Vault (covered in the next step). + +
    + +4. Configure admission controller secret + + Generate and store the admission controller TLS certificate in Vault: + + ```bash + kubectl exec -it $(kubectl get pod -l name=redis-enterprise-operator -o jsonpath='{.items[0].metadata.name}') \ + -c redis-enterprise-operator -- /usr/local/bin/generate-tls -infer | tail -4 > output.json + ``` + + Copy the certificate file to Vault (if Vault is running in Kubernetes): + + ```bash + kubectl cp output.json vault-0:/tmp -n vault + ``` + + Store the certificate in Vault: + + ```bash + vault kv put -namespace= /redisenterprise-/admission-tls @output.json + ``` + + > [!NOTE] + > Once the operator is running with Vault integration, proceed to create Redis Enterprise clusters. Do not create clusters before completing this setup. + +
    + +5. Create Vault CA certificate secret + + Create a Kubernetes secret containing the Certificate Authority certificate used by your Vault instance: + + ```bash + kubectl create secret generic vault-ca-cert \ + --namespace \ + --from-file=vault.ca= + ``` + + > [!WARNING] + > The Vault server certificate must be signed by the Certificate Authority provided in this secret. + +## Create Redis Enterprise clusters + +1. Generate cluster credentials + + Unlike standard deployments, Vault integration requires manually creating cluster credentials: + + ```bash + # Generate a secure random password + openssl rand -base64 32 + ``` + + Store credentials in Vault: + + ```bash + vault kv put -namespace= \ + /redisenterprise-/ \ + username= \ + password= + ``` + + > [!NOTE] Important notes + > - The username field in the REC spec is ignored when using Vault + > - The username from the Vault secret takes precedence + > - Use strong, unique passwords for each cluster + +
    + +2. Create cluster service account role + + Configure a Vault role for the Redis Enterprise cluster's service account: + + ```bash + vault write -namespace= \ + auth//role/redis-enterprise-rec- \ + bound_service_account_names= \ + bound_service_account_namespaces= \ + policies=redisenterprise- + ``` + +
    + +3. Deploy Redis Enterprise cluster + + Create the `RedisEnterpriseCluster` resource with Vault configuration: + + ```yaml + apiVersion: app.redislabs.com/v1 + kind: RedisEnterpriseCluster + metadata: + name: rec + namespace: + labels: + app: redis-enterprise + spec: + nodes: 3 + clusterCredentialSecretName: rec + clusterCredentialSecretType: vault + clusterCredentialSecretRole: redis-enterprise-rec- + vaultCASecret: vault-ca-cert + podAnnotations: + vault.hashicorp.com/auth-path: auth/ + vault.hashicorp.com/namespace: + ``` + + Apply the configuration: + + ```bash + kubectl apply -f redis-enterprise-cluster.yaml + ``` + + Key configuration fields: + + {{}} + | Field | Description | Example | + |-------|-------------|---------| + | `clusterCredentialSecretName` | Path of the secret in Vault containing cluster credentials. Can be customized during cluster creation but cannot be changed afterward. The secret must be pre-created in Vault. | `rec` | + | `clusterCredentialSecretType` | Must be set to `vault` | `vault` | + | `clusterCredentialSecretRole` | Vault role for cluster authentication | `redis-enterprise-rec-` | + | `vaultCASecret` | Kubernetes secret containing Vault's CA certificate | `vault-ca-cert` | + | `podAnnotations` | Vault agent annotations for pod-level configuration | See example above | + {{}} + +## Create Redis Enterprise databases + +To create a Redis Enterprise database (REDB) with Vault integration: + +1. Create database password in Vault: + ```bash + vault kv put -namespace= \ + /redisenterprise-/redb- \ + password= + ``` + +
    + +2. Create the REDB custom resource: + Follow the standard [database creation process](/content/operate/kubernetes/8.2.0/re-databases/_index.md). The REC configuration automatically enables Vault integration for all databases. + +
    + +3. Configure additional secrets (optional): + Store additional REDB secrets in the path `redisenterprise-/`. Secrets must comply with the [REDB secrets schema](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md). + +> [!NOTE] +> When using the Redis Enterprise Vault plugin, set `defaultUser: false` and associate users through ACL bindings to the REDB. + +For complete field documentation, see the [Redis Enterprise database API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md). + +### Redis Enterprise Remote Cluster secrets + +The `secretName` field is supported and should be stored in HashiCorp Vault when the Redis Enterprise cluster uses Vault as a secret source. + +### Redis Enterprise Active-Active database secrets + +REAADB resources include REDB specifications in the `globalConfigurations` field. All secret names specified in these configurations are supported and should be stored in HashiCorp Vault when the Redis Enterprise cluster uses Vault as a secret source. + +## Manage secrets + +> [!NOTE] +> Complete field documentation is available in the [`RedisEnterpriseCluster` API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md) and [`RedisEnterpriseDatabase` API reference](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md). + +### Redis Enterprise cluster secrets + +#### Example REC configuration with all certificates + +```yaml +apiVersion: app.redislabs.com/v1 +kind: RedisEnterpriseCluster +metadata: + name: rec + labels: + app: redis-enterprise +spec: + nodes: 3 + licenseSecretName: + clusterCredentialSecretName: + certificates: + apiCertificateSecretName: + cmCertificateSecretName: + metricsExporterCertificateSecretName: + proxyCertificateSecretName: + syncerCertificateSecretName: + ldapClientCertificateSecretName: + cpInternodeEncryptionCertificateSecretName: + dpInternodeEncryptionCertificateSecretName: + ssoServiceCertificateSecretName: + ssoIssuerCertificateSecretName: + ldap: + bindCredentialsSecretName: + sso: + saml: + idpMetadataSecretName: + # Vault configuration + clusterCredentialSecretType: vault + clusterCredentialSecretRole: redis-enterprise-rec- + vaultCASecret: vault-ca-cert + podAnnotations: + vault.hashicorp.com/auth-path: auth/ + vault.hashicorp.com/namespace: +``` + +You can also update certificates using `kubectl patch`: + +```bash +kubectl patch rec rec --type merge --patch '{"spec": {"certificates": {"apiCertificateSecretName": ""}}}' +``` + +### Database secrets + +#### Database passwords +Store database passwords in Vault using the database name as the secret key: + +```bash +vault kv put -namespace= \ + /redisenterprise-/ \ + password= +``` + +#### Backup storage credentials +Store backup storage credentials for Redis Enterprise databases: + +```bash +vault kv put -namespace= \ + /redisenterprise-/ \ + AWS_ACCESS_KEY_ID= \ + AWS_SECRET_ACCESS_KEY= +``` + +#### TLS certificates +Store TLS certificates for database connections: + +```bash +vault kv put -namespace= \ + /redisenterprise-/ \ + tls.crt= \ + tls.key= +``` + +### User-defined module credentials + +Store credentials for downloading user-defined modules from authenticated repositories: + +```bash +vault kv put -namespace= \ + /redisenterprise-/ \ + username= \ + password= +``` + +Reference this secret in your REC specification's `userDefinedModules` section. See [Configure modules](/content/operate/kubernetes/8.2.0/re-databases/modules.md) for details. + +## Troubleshooting + +### Common Issues and Solutions + +#### Operator pod not ready + +Symptoms: Operator pod remains in `Pending` or `CrashLoopBackOff` state + +Causes and solutions: + +1. Missing admission controller secret: + ```bash + # Check if admission-tls secret exists in Vault + vault kv get -namespace= /redisenterprise-/admission-tls + ``` + +2. Vault CA certificate issues: + ```bash + # Verify vault-ca-cert secret exists + kubectl get secret vault-ca-cert -n + + # Check certificate content + kubectl get secret vault-ca-cert -n -o jsonpath='{.data.vault\.ca}' | base64 -d + ``` + +3. Network connectivity: + ```bash + # Test Vault connectivity from operator pod + kubectl exec -it -c redis-enterprise-operator -- \ + curl -k https://:8200/v1/sys/health + ``` + +#### Authentication failures + +Symptoms: `Failed to authenticate with Vault` errors in operator logs + +Solutions: + +1. Verify Vault role configuration: + ```bash + vault read -namespace= auth//role/redis-enterprise-operator- + ``` + +2. Check service account token: + ```bash + # Verify service account exists + kubectl get serviceaccount redis-enterprise-operator -n + + # Check token mount + kubectl describe pod -n | grep -A5 "Mounts:" + ``` + +#### Secret retrieval failures + +Symptoms: `Failed to read Vault secret` errors + +Solutions: + +1. Verify secret exists: + ```bash + vault kv get -namespace= /redisenterprise-/ + ``` + +2. Check policy permissions: + ```bash + vault policy read -namespace= redisenterprise- + ``` + +3. Validate secret format: + ```bash + # Cluster credentials must have 'username' and 'password' keys + vault kv get -format=json -namespace= /redisenterprise-/ + ``` + +### Debugging commands + +Check operator logs: +```bash +kubectl logs -f deployment/redis-enterprise-operator -n -c redis-enterprise-operator +``` + +Verify Vault configuration: +```bash +kubectl get configmap operator-environment-config -n -o yaml +``` + +Test Vault authentication: +```bash +# From within operator pod +kubectl exec -it -n -c redis-enterprise-operator -- \ + cat /var/run/secrets/kubernetes.io/serviceaccount/token +``` diff --git a/content/operate/kubernetes/8.2.0/upgrade/_index.md b/content/operate/kubernetes/8.2.0/upgrade/_index.md new file mode 100644 index 0000000000..33540406c9 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/upgrade/_index.md @@ -0,0 +1,53 @@ +--- +Title: Upgrade Redis Enterprise for Kubernetes +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Upgrade Redis Enterprise operator, clusters, and databases on Kubernetes. +hideListLinks: false +linkTitle: Upgrade +weight: 15 +url: '/operate/kubernetes/8.2.0/upgrade/' +--- + +Keep your Redis Enterprise deployment up to date with the latest features, security patches, and bug fixes. The upgrade process involves updating three main components in sequence: the Redis Enterprise operator, Redis Enterprise clusters (REC), and Redis Enterprise databases (REDB). + +## Upgrade methods + +Choose the appropriate upgrade method for your deployment: + +- [Upgrade Redis Enterprise for Kubernetes](/content/operate/kubernetes/8.2.0/upgrade/upgrade-redis-cluster.md) - Standard upgrade process for most Kubernetes distributions +- [Upgrade with OpenShift CLI](/content/operate/kubernetes/8.2.0/upgrade/openshift-cli.md) - OpenShift-specific upgrade using CLI tools +- [Upgrade with OpenShift OperatorHub](/content/operate/kubernetes/8.2.0/upgrade/upgrade-olm.md) - Upgrade using OpenShift OperatorHub and OLM +- [Upgrade with Helm](/content/operate/kubernetes/8.2.0/deployment/helm.md#upgrade-the-chart) - Helm-specific upgrade instructions for chart-based deployments + +## Upgrade process + +The upgrade process includes updating three components in order: + +1. **Upgrade the Redis Enterprise operator** - Update the operator to the latest version +2. **Upgrade the Redis Enterprise cluster (REC)** - Update cluster nodes and infrastructure +3. **Upgrade Redis Enterprise databases (REDB)** - Update database versions and configurations + +## Upgrade compatibility + +When upgrading, both your Kubernetes version and Redis operator version need to be supported at all times. + +> [!WARNING] +> If your current Kubernetes distribution is not [supported](/content/operate/kubernetes/8.2.0/reference/supported_k8s_distributions.md), upgrade to a supported distribution before upgrading. + +## RHEL9-based image + +As of version 7.8.2-6, Redis Enterprise images are based on Red Hat Enterprise Linux 9 (RHEL9). This means upgrades require: + +- [Cluster version of 7.4.2-2 or later](https://redis.io/docs/latest/operate/kubernetes/7.4.6/upgrade/). +- Database version 7.2 or later. +- RHEL9 compatible binaries for any modules you need. + +For detailed steps, see the relevant upgrade page: + +- [OpenShift CLI](/content/operate/kubernetes/8.2.0/upgrade/openshift-cli.md) +- [OpenShift OperatorHub](/content/operate/kubernetes/8.2.0/upgrade/upgrade-olm.md) +- [Kubernetes](/content/operate/kubernetes/8.2.0/upgrade/upgrade-redis-cluster.md) \ No newline at end of file diff --git a/content/operate/kubernetes/8.2.0/upgrade/openshift-cli.md b/content/operate/kubernetes/8.2.0/upgrade/openshift-cli.md new file mode 100644 index 0000000000..7984cdb387 --- /dev/null +++ b/content/operate/kubernetes/8.2.0/upgrade/openshift-cli.md @@ -0,0 +1,285 @@ +--- +Title: Upgrade Redis Enterprise with OpenShift CLI +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: This task describes how to upgrade a Redis Enterprise cluster via OpenShift + CLI. +linkTitle: OpenShift CLI +weight: 20 +url: '/operate/kubernetes/8.2.0/upgrade/openshift-cli/' +--- + +Redis implements rolling updates for software upgrades in Kubernetes deployments. The upgrade process includes updating three components: + + 1. [Upgrade the Redis Enterprise operator](#upgrade-the-operator) + 1. [Upgrade the Redis Enterprise cluster (REC)](#upgrade-the-redis-enterprise-cluster) + 1. [Upgrade Redis Enterprise databases (REDB)](#upgrade-databases) + +## Prerequisites + +The following steps ensure you have the minimum versions of all components necessary to upgrade to 7.8.2-6 or later. **Without these minimum versions, the upgrade will freeze and require manual recovery.** + +See the [troubleshooting](#troubleshooting) section for details on recovering a failed upgrade. + +#### Kubernetes version + +Check [Supported Kubernetes distributions](/content/operate/kubernetes/8.2.0/reference/supported_k8s_distributions.md) to make sure your Kubernetes distribution is supported. If not, upgrade your Kubernetes distribution before upgrading the Redis operator. + +#### Redis operator version + +Your Redis Enterprise clusters must be running version 7.4.2-2 or later before upgrading to 7.8.2-6 or later. See the [7.4 upgrade](https://redis.io/docs/latest/operate/kubernetes/7.4.6/upgrade/openshift-cli/) for detailed steps. + +#### Redis database version + +Your Redis databases must be running version 7.2 or later before upgrading your cluster version. See [upgrade databases](#upgrade-databases) for detailed steps. You can find your database version in the [REDB `spec.redisVersion` field](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md#spec). + +#### User-defined modules + +If your databases use user-defined modules (custom non-bundled modules): + +- Set `spec.upgradeSpec.autoUpgradeRedisEnterprise: false` in the REC custom resource before upgrading the operator. +- Define the user-defined modules in the REC custom resource before upgrading the database. +- See [Edit `redisEnterpriseImageSpec`](#edit-redisenterpriseimagespec) for more details. + +For more information about user-defined modules, see [User-defined modules](/content/operate/kubernetes/8.2.0/re-databases/modules.md#user-defined-modules). + +#### RHEL9-compatible modules + +Upgrading to Redis operator versions 7.8.2-6 and later involves migrating your Redis Enterprise nodes to RHEL9 from either Ubuntu 18 or RHEL8. If your databases use modules, you need to manually install modules compatible with RHEL9. + +To see which modules you have installed, run: + +```sh +curl -k -u : -X GET https://localhost:9443/v1/modules | jq -r 'map([.module_name, .semantic_version, (.platforms | keys)]) | .[] | .[0] as $name | .[1] as $version | .[2][] | $name + "-" + $version + "-" + .' | sort +``` + +To see which modules are currently in use, run: + +```sh +curl -k -u : -X GET https://localhost:9443/v1/bdbs | jq -r '.[].module_list | map(.module_name + "-" + .semantic_version) | .[]' +``` + +See [Upgrade modules](/content/operate/rs/installing-upgrading/modules/upgrade-module.md) for details on how to upgrade modules with the `rladmin` tool. + +#### Valid license + +Use `kubectl get rec` and verify the `LICENSE STATE` is valid on your REC before you start the upgrade process. + +## Upgrade with Helm charts + +If you installed the Redis Enterprise operator using Helm charts on OpenShift, you can upgrade using Helm commands. This method automatically handles the operator upgrade and Custom Resource Definition (CRD) updates. + +To upgrade using Helm on OpenShift: + +```sh +helm upgrade redis/redis-enterprise-operator --version \ + --set openshift.mode=true +``` + +For example: + +```sh +helm upgrade my-redis-enterprise redis/redis-enterprise-operator --version 7.8.2-2 \ + --set openshift.mode=true +``` + +After the Helm upgrade completes, continue with [upgrading the Redis Enterprise cluster](#upgrade-the-redis-enterprise-cluster) and [upgrading databases](#upgrade-databases). + +For detailed Helm upgrade instructions, see [Upgrade the chart](/content/operate/kubernetes/8.2.0/deployment/helm.md#upgrade-the-chart). + +## Upgrade the operator + +> [!WARNING] +> If your databases use user-defined modules, set `spec.upgradeSpec.autoUpgradeRedisEnterprise: false` in the REC custom resource before upgrading the operator. + +### Download the bundle + +Make sure you pull the correct version of the bundle. You can find the version tags +by checking the [operator releases on GitHub](https://github.com/RedisLabs/redis-enterprise-k8s-docs/releases) +or by [using the GitHub API](https://docs.github.com/en/rest/reference/repos#releases). + +For OpenShift environments, the name of the bundle is `openshift.bundle.yaml`, and so the `curl` command to run is: + +```sh +curl --silent -O https://raw.githubusercontent.com/RedisLabs/redis-enterprise-k8s-docs/$VERSION/openshift.bundle.yaml +``` + +If you need a different release, replace `VERSION` in the above with a specific release tag. + +### Apply the bundle + +Apply the bundle to deploy the new operator binary. This will also apply any changes in the new release to custom resource definitions, roles, role binding, or operator service accounts. + +> [!NOTE] +> If you are not pulling images from Docker Hub, update the operator image spec to point to your private repository. +> If you have made changes to the role, role binding, RBAC, or custom resource definition (CRD) in the previous version, merge them with the updated declarations in the new version files. + +If you are using OpenShift, run this instead: + +```sh +oc apply -f openshift.bundle.yaml +``` + +After running this command, you should see a result similar to this: + +```sh +role.rbac.authorization.k8s.io/redis-enterprise-operator configured +serviceaccount/redis-enterprise-operator configured +rolebinding.rbac.authorization.k8s.io/redis-enterprise-operator configured +customresourcedefinition.apiextensions.k8s.io/redisenterpriseclusters.app.redislabs.com configured +customresourcedefinition.apiextensions.k8s.io/redisenterprisedatabases.app.redislabs.com configured +deployment.apps/redis-enterprise-operator configured +``` + +### Reapply the admission controller webhook {#reapply-webhook} + +If you have the admission controller enabled, you need to manually reapply the `ValidatingWebhookConfiguration`. + +> [!NOTE] +> {{< embed-md "k8s-642-redb-admission-webhook-name-change.md" >}} + +{{< embed-md "k8s-admission-webhook-cert.md" >}} + +### Verify the operator is running + +You can check your deployment to verify the operator is running in your namespace. + +```sh +oc get deployment/redis-enterprise-operator +``` + +You should see a result similar to this: + +```sh +NAME READY UP-TO-DATE AVAILABLE AGE +redis-enterprise-operator 1/1 1 1 0m36s +``` + +> [!WARNING] +> We recommend upgrading the REC as soon as possible after updating the operator. After the operator upgrade completes, the operator suspends the management of the REC and its associated REDBs, until the REC upgrade completes. + +## Security context constraints + +Upgrades to versions 7.22.0-6 and later run in **unprivileged mode** without any additional permissions or capabilities. If you don't specifally require additional capabilities, we recommend you maintain the default unprivileged mode, as its more secure. After upgrading, remove the existing `redis-enterprise-scc-v2` SCC and unbind it from the REC service account. + +To enable automatic resource adjustment, see [Allow automatic resource adjustment > OpenShift upgrades](/content/operate/kubernetes/8.2.0/security/allow-resource-adjustment.md#openshift-upgrades). + +## Upgrade the Redis Enterprise cluster + +> [!WARNING] +> Verify your license is valid before upgrading. Invalid licenses will cause the upgrade to fail. +> +> Use `oc get rec` and verify the `LICENSE STATE` is valid on your REC before you start the upgrade process. + +The Redis Enterprise cluster (REC) can be updated automatically or manually. To trigger automatic upgrade of the REC after the operator upgrade completes, set `spec.upgradeSpec.autoUpgradeRedisEnterprise: true` in your REC. If you don't have automatic upgrade enabled, follow the below steps for the manual upgrade. + +Before beginning the upgrade of the Redis Enterprise cluster, check the K8s operator [release notes](/content/operate/kubernetes/release-notes/_index.md) to find the Redis Enterprise image tag. + +After the operator upgrade is complete, you can upgrade Redis Enterprise cluster (REC). + +### Edit `redisEnterpriseImageSpec` + +1. Edit the REC custom resource YAML file. + + ```sh + oc edit rec + ``` + +1. Replace the `versionTag:` declaration under `redisEnterpriseImageSpec` with the new version tag. + + ```YAML + spec: + redisEnterpriseImageSpec: + imagePullPolicy: IfNotPresent + repository: redislabs/redis + versionTag: + ``` + +1. Define any user-defined modules used by databases in the cluster. + + ```YAML + spec: + userDefinedModules: + - name: "custom-module" + source: + https: + url: "https://modules.company.com/search-v2.1.zip" + credentialsSecret: "module-repo-creds" + ``` + + The `name` field must match the `display_name` or `module_name` that appears in the module manifest (for example, "redisgears"). This enables the operator to run validation on the user-defined module. If these names don't match, the operator can't run validation on the user-defined module and preventable errors may occur. + + > [!NOTE] + > Adding or modifying the `userDefinedModules` list triggers a rolling restart of the Redis Enterprise cluster pods in addition to the rolling upgrade for the version change. + > + +1. Save the changes to apply. + +### Reapply roles and role bindings + +If your operator is monitoring multiple namespaces, you'll need to [reapply your role and role bindings](/content/operate/kubernetes/8.2.0/re-clusters/multi-namespace.md#create-role-and-role-binding-for-managed-namespaces) for each managed namespace. See [Manage databases in multiple namespaces](/content/operate/kubernetes/8.2.0/re-clusters/multi-namespace.md) for more details. + +### Monitor the upgrade + +You can view the state of the REC with `oc get rec`. + + During the upgrade, the state should be `Upgrade`. + When the upgrade is complete and the cluster is ready to use, the state will change to `Running`. + If the state is `InvalidUpgrade`, there is an error (usually relating to configuration) in the upgrade. + +```sh +$ oc get rec +NAME NODES VERSION STATE SPEC STATUS LICENSE STATE SHARDS LIMIT LICENSE EXPIRATION DATE AGE +rec 3 6.2.10-107 Upgrade Valid Valid 4 2022-07-16T13:59:00Z 92m +``` + +To see the status of the current rolling upgrade, run: + +```sh +oc rollout status sts +``` + +### Upgrade databases + +After the cluster is upgraded, you can upgrade your databases. + +To upgrade your REDB, specify your new database version in the `spec.redisVersion` field in the REDB or REAADB custom resources. Supported database versions for operator versions include "7.2", "7.4", "8.0", and "8.2" (note this value is a string). + +For Active-Active databases, the `redis.Version` change only needs to be applied on one participating cluster and will automatically propagate to all other participating clusters. All participating clusters must be running operator version 8.0.2-2 or later. + +If your REAADB uses supported modules, keep the existing `moduleList` version numbers unchanged when upgrading `redisVersion`. The database will automatically use the module versions that are bundled with the new Redis version, regardless of what versions are specified in `moduleList`. After the upgrade is complete, you can optionally change the old version numbers from `moduleList`, but this change has no functional impact. + +#### Upgrade with user-defined modules + +If a user-defined module is used by any database in the cluster, the module must be defined in the REC custom resource before upgrading the database. See [Add user-defined modules to the REC](/content/operate/kubernetes/8.2.0/re-databases/modules.md#add-user-defined-modules-to-the-rec) for detailed instructions. + +#### General upgrade notes + +Note that if your cluster [`redisUpgradePolicy`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#spec) or your database [`redisVersion`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md#spec) are set to `major`, you won't be able to upgrade those databases to minor versions. See [Redis upgrade policy](/content/operate/rs/installing-upgrading/upgrading/_index.md#redis-upgrade-policy) for more details. + +## Troubleshooting + +If you start an upgrade without meeting the [prerequisites](#prerequisites), the operator will freeze the upgrade. Check the operator logs for the source of the error. The REDB reconsilliation doesn't work during an upgrade, so you need to apply a manual fix with the Redis Software API (examples below). The updates will also need to be added to the REDB custom resource. + +### Invalid module version + +If the operator logs show an event related to an unsupported module, download the updated module locally, and install it using the `v2/modules` API endpoint. + +```sh +curl -sfk -u : -X POST -F 'module=@' https://localhost:9443/v2/modules +``` + +After updating the modules with the Redis Software API, update the REDB custom resource to reflect the change. + +### Invalid database version + +If the operator logs show an event related to an incompatible database version, upgrade the database using the Redis Software API. + +```sh +curl -sfk -u : -X POST -H "Content-Type: application/json" -d '{"redis_version": }' https://localhost:9443/v1/bdbs//upgrade +``` + +After updating the database with the Redis Software API, update the REDB custom resource to reflect the change. diff --git a/content/operate/kubernetes/8.2.0/upgrade/upgrade-olm.md b/content/operate/kubernetes/8.2.0/upgrade/upgrade-olm.md new file mode 100644 index 0000000000..8151bc469a --- /dev/null +++ b/content/operate/kubernetes/8.2.0/upgrade/upgrade-olm.md @@ -0,0 +1,188 @@ +--- +Title: Upgrade Redis Enterprise with OpenShift OperatorHub +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: This task describes how to upgrade a Redis Enterprise cluster via OpenShift + OperatorHub. +linkTitle: OpenShift OperatorHub +weight: 40 +url: '/operate/kubernetes/8.2.0/upgrade/upgrade-olm/' +--- + +Redis implements rolling updates for software upgrades in Kubernetes deployments. The upgrade process includes updating three components: + + 1. [Upgrade the Redis Enterprise operator](#upgrade-the-redis-enterprise-operator) + 1. [Upgrade the Redis Enterprise cluster (REC)](#upgrade-the-redis-enterprise-cluster) + 1. [Upgrade Redis Enterprise databases (REDB)](#upgrade-databases) + +## Prerequisites + +The following steps ensure you have the minimum versions of all components necessary to upgrade. **Without these minimum versions, the upgrade will freeze and require manual recovery.** + +See the [troubleshooting](#troubleshooting) section for details on recovering a failed upgrade. + +#### Kubernetes version + +Check [Supported Kubernetes distributions](/content/operate/kubernetes/8.2.0/reference/supported_k8s_distributions.md) to make sure your Kubernetes distribution is supported. If not, upgrade your Kubernetes distribution before upgrading the Redis operator. + +#### Redis operator version + +Your Redis Enterprise clusters must be running version 7.4.2-2 or later before upgrading to 7.8.2-6. See the [7.4 upgrade](https://redis.io/docs/latest/operate/kubernetes/7.4.6/upgrade/upgrade-olm/) for detailed steps. + +#### Redis database version + +Your Redis databases must be running version 7.2 or later before upgrading your cluster version. See [upgrade databases](#upgrade-databases) for detailed steps. You can find your database version in the [REDB `spec.redisVersion` field](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md#spec). + +### User-defined modules + +If your databases use user-defined modules (custom non-bundled modules): + +- Set `spec.upgradeSpec.autoUpgradeRedisEnterprise: false` in the REC custom resource before upgrading the operator. +- Define the user-defined modules in the REC custom resource before upgrading the database. +- See [Edit `redisEnterpriseImageSpec`](#edit-redisenterpriseimagespec) for more details. + +For more information about user-defined modules, see [User-defined modules](/content/operate/kubernetes/8.2.0/re-databases/modules.md#user-defined-modules). + +### Valid license + +Use `kubectl get rec` and verify the `LICENSE STATE` is valid on your REC before you start the upgrade process. + +## Upgrade the Redis Enterprise operator + +1. Select the **Redis Enterprise Operator** from the **Operators**>**Installed Operators** page. + +2. Select the **Subscription** tab. + +3. Verify your **Update approval** is set to "Manual". + +4. To upgrade to the latest version, set your **Update Channel** to ‘production’. To upgrade to an older version, specify the desired version for your **Update Channel**. + +5. Select **Upgrade available** shown under **Upgrade status**. + +6. When the "Review manual InstallPlan" section appears, select **Preview installPlan** and then **Approve** after reviewing the details. This will start the operator upgrade. + +You can monitor the upgrade from the **Installed Operators** page. A new Redis Enterprise Operator will appear in the list, with the status "Installing". OpenShift will delete the old operator, showing the "Cannot update" status during deletion. + +## Security context constraints + +Upgrades to versions 7.22.0-6 and later run in **unprivileged mode** without any additional permissions or capabilities. If you don't specifally require additional capabilities, we recommend you maintain the default unprivileged mode, as its more secure. After upgrading, remove the existing `redis-enterprise-scc-v2` SCC and unbind it from the REC service account. + +To enable automatic resource adjustment, see [Allow automatic resource adjustment > OpenShift upgrades](/content/operate/kubernetes/8.2.0/security/allow-resource-adjustment.md#openshift-upgrades). + +## Upgrade the Redis Enterprise cluster + +> [!WARNING] +> Verify your license is valid before upgrading. Invalid licenses will cause the upgrade to fail. +> +> Use `oc get rec` and verify the `LICENSE STATE` is valid on your REC before you start the upgrade process. + +The Redis Enterprise cluster (REC) can be updated automatically or manually. To trigger automatic upgrade of the REC after the operator upgrade completes, set `spec.upgradeSpec.autoUpgradeRedisEnterprise: true` in your REC. If you don't have automatic upgrade enabled, follow the below steps for the manual upgrade. + +Before beginning the upgrade of the Redis Enterprise cluster, check the K8s operator [release notes](/content/operate/kubernetes/release-notes/_index.md) to find the Redis Enterprise image tag. + +After the operator upgrade is complete, you can upgrade Redis Enterprise cluster (REC). + +### Edit `redisEnterpriseImageSpec` + +1. Edit the REC custom resource YAML file. + + ```sh + oc edit rec + ``` + +1. Replace the `versionTag:` declaration under `redisEnterpriseImageSpec` with the new version tag. + + ```YAML + spec: + redisEnterpriseImageSpec: + imagePullPolicy: IfNotPresent + repository: redislabs/redis + versionTag: + ``` + +1. Define any user-defined modules used by databases in the cluster. + + ```YAML + spec: + userDefinedModules: + - name: "custom-module" + source: + https: + url: "https://modules.company.com/search-v2.1.zip" + credentialsSecret: "module-repo-creds" + ``` + + The `name` field must match the `display_name` or `module_name` that appears in the module manifest (for example, "redisgears"). This enables the operator to run validation on the user-defined module. If these names don't match, the operator can't run validation on the user-defined module and preventable errors may occur. + + > [!NOTE] + > Adding or modifying the `userDefinedModules` list triggers a rolling restart of the Redis Enterprise cluster pods in addition to the rolling upgrade for the version change. + > + +1. Save the changes to apply. + +### Reapply roles and role bindings + +If your operator is monitoring multiple namespaces, you'll need to [reapply your role and role bindings](/content/operate/kubernetes/8.2.0/re-clusters/multi-namespace.md#create-role-and-role-binding-for-managed-namespaces) for each managed namespace. See [Manage databases in multiple namespaces](/content/operate/kubernetes/8.2.0/re-clusters/multi-namespace.md) for more details. + +### Monitor the upgrade + +You can view the state of the REC with `oc get rec`. + + During the upgrade, the state should be `Upgrade`. + When the upgrade is complete and the cluster is ready to use, the state will change to `Running`. + If the state is `InvalidUpgrade`, there is an error (usually relating to configuration) in the upgrade. + +```sh +$ oc get rec +NAME NODES VERSION STATE SPEC STATUS LICENSE STATE SHARDS LIMIT LICENSE EXPIRATION DATE AGE +rec 3 6.2.10-107 Upgrade Valid Valid 4 2022-07-16T13:59:00Z 92m +``` + +To see the status of the current rolling upgrade, run: + +```sh +oc rollout status sts +``` + +### Upgrade databases + +After the cluster is upgraded, you can upgrade your databases. + +To upgrade your REDB, specify your new database version in the `spec.redisVersion` field in the REDB or REAADB custom resources. Supported database versions for operator versions include "7.2", "7.4", "8.0", and "8.2" (note this value is a string). + +For Active-Active databases, the `redis.Version` change only needs to be applied on one participating cluster and will automatically propagate to all other participating clusters. All participating clusters must be running operator version 8.0.2-2 or later. + +If your REAADB uses supported modules, keep the existing `moduleList` version numbers unchanged when upgrading `redisVersion`. The database will automatically use the module versions that are bundled with the new Redis version, regardless of what versions are specified in `moduleList`. After the upgrade is complete, you can optionally change the old version numbers from `moduleList`, but this change has no functional impact. + +### General upgrade notes + +Note that if your cluster [`redisUpgradePolicy`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#spec) or your database [`redisVersion`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md#spec) are set to `major`, you won't be able to upgrade those databases to minor versions. See [Redis upgrade policy](/content/operate/rs/installing-upgrading/upgrading/_index.md#redis-upgrade-policy) for more details. + +The Redis Enterprise cluster (REC) can be updated automatically or manually. To trigger automatic upgrade of the REC after the operator upgrade completes, set `spec.upgradeSpec.autoUpgradeRedisEnterprise: true` in your REC. If you don't have automatic upgrade enabled, follow the below steps for the manual upgrade. + +## Troubleshooting + +If you start an upgrade without meeting the [prerequisites](#prerequisites), the operator will freeze the upgrade. Check the operator logs for the source of the error. The REDB reconsilliation doesn't work during an upgrade, so you need to apply a manual fix with the Redis Software API (examples below). The updates will also need to be added to the REDB custom resource. + +### Invalid module version + +If the operator logs show an event related to an unsupported module, download the updated module locally, and install it using the `v2/modules` API endpoint. + +```sh +curl -sfk -u : -X POST -F 'module=@' https://localhost:9443/v2/modules +``` + +After updating the modules with the Redis Software API, update the REDB custom resource to reflect the change. + +### Invalid database version + +If the operator logs show an event related to an incompatible database version, upgrade the database using the Redis Software API. + +```sh +curl -sfk -u : -X POST -H "Content-Type: application/json" -d '{"redis_version": }' https://localhost:9443/v1/bdbs//upgrade +``` + +After updating the database with the Redis Software API, update the REDB custom resource to reflect the change. diff --git a/content/operate/kubernetes/8.2.0/upgrade/upgrade-redis-cluster.md b/content/operate/kubernetes/8.2.0/upgrade/upgrade-redis-cluster.md new file mode 100644 index 0000000000..a91dd3ab3a --- /dev/null +++ b/content/operate/kubernetes/8.2.0/upgrade/upgrade-redis-cluster.md @@ -0,0 +1,310 @@ +--- +Title: Upgrade Redis Enterprise for Kubernetes +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: This task describes how to upgrade a Redis Enterprise cluster via the + operator. +linkTitle: Kubernetes +weight: 10 +url: '/operate/kubernetes/8.2.0/upgrade/upgrade-redis-cluster/' +--- + +Redis implements rolling updates for software upgrades in Kubernetes deployments. The upgrade process includes updating three components: + + 1. [Upgrade the Redis Enterprise operator](#upgrade-the-operator) + 1. [Upgrade the Redis Enterprise cluster (REC)](#upgrade-the-redis-enterprise-cluster-rec) + 1. [Upgrade Redis Enterprise databases (REDB)](#upgrade-databases) + +## Prerequisites + +Before upgrading, ensure you have the minimum versions of all components necessary for your target version. **Without these minimum versions, the upgrade may freeze and require manual recovery.** + +See the [troubleshooting](#troubleshooting) section for details on recovering a failed upgrade. + +### Kubernetes version + +Check [Supported Kubernetes distributions](/content/operate/kubernetes/8.2.0/reference/supported_k8s_distributions.md) to make sure your Kubernetes distribution is supported by your target Redis Enterprise operator version. If not, upgrade your Kubernetes distribution before upgrading the Redis operator. + +### Redis operator version + +Check the [Redis Enterprise for Kubernetes release notes](/content/operate/kubernetes/release-notes/_index.md) for the minimum operator version required for your target upgrade. Some versions may require intermediate upgrades. + +### Redis database version + +Check the release notes for your target version to determine the minimum Redis database version required. See [upgrade databases](#upgrade-databases) for detailed steps. You can find your database version in the [REDB `spec.redisVersion` field](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md#spec). + +### User-defined modules + +If your databases use user-defined modules (custom non-bundled modules): + +- Set `spec.upgradeSpec.autoUpgradeRedisEnterprise: false` in the REC custom resource before upgrading the operator. +- Define the user-defined modules in the REC custom resource before upgrading the database. +- See [Edit `redisEnterpriseImageSpec`](#edit-redisenterpriseimagespec-in-the-rec-spec) for more details. + +For more information about user-defined modules, see [User-defined modules](/content/operate/kubernetes/8.2.0/re-databases/modules.md#user-defined-modules). + +### Valid license + +Use `kubectl get rec` and verify the `LICENSE STATE` is valid on your REC before you start the upgrade process. + +## Upgrade with Helm charts + +If you installed the Redis Enterprise operator using Helm charts, you can upgrade using Helm commands. This method automatically handles the operator upgrade and Custom Resource Definition (CRD) updates. + +To upgrade using Helm: + +```sh +helm upgrade redis/redis-enterprise-operator --version +``` + +For example: + +```sh +helm upgrade my-redis-enterprise redis/redis-enterprise-operator --version +``` + +For OpenShift installations, include the `openshift.mode=true` parameter: + +```sh +helm upgrade redis/redis-enterprise-operator --version \ + --set openshift.mode=true +``` + +After the Helm upgrade completes, continue with [upgrading the Redis Enterprise cluster (REC)](#upgrade-the-redis-enterprise-cluster-rec) and [upgrading databases](#upgrade-databases). + +For detailed Helm upgrade instructions, see [Upgrade the chart](/content/operate/kubernetes/8.2.0/deployment/helm.md#upgrade-the-chart). + +## Upgrade the operator + +> [!WARNING] +> If your databases use user-defined modules, set `spec.upgradeSpec.autoUpgradeRedisEnterprise: false` in the REC custom resource before upgrading the operator. + +### Download the bundle + +Make sure you pull the correct version of the bundle. You can find the version tags +by checking the [operator releases on GitHub](https://github.com/RedisLabs/redis-enterprise-k8s-docs/releases) +or by [using the GitHub API](https://docs.github.com/en/rest/reference/repos#releases). + +You can download the bundle for the latest release with the following `curl` command: + +```sh +VERSION=`curl --silent https://api.github.com/repos/RedisLabs/redis-enterprise-k8s-docs/releases/latest | grep tag_name | awk -F'"' '{print $4}'` +curl --silent -O https://raw.githubusercontent.com/RedisLabs/redis-enterprise-k8s-docs/$VERSION/bundle.yaml +``` + +If you need a different release, replace `VERSION` in the above with a specific release tag. + +### Apply the bundle + +Apply the bundle to deploy the new operator binary. This will also apply any changes in the new release to custom resource definitions, roles, role binding, or operator service accounts. + +> [!NOTE] +> If you are not pulling images from Docker Hub, update the operator image spec to point to your private repository. +> If you have made changes to the role, role binding, RBAC, or custom resource definition (CRD) in the previous version, merge them with the updated declarations in the new version files. + +Upgrade the bundle and operator with a single command, passing in the bundle YAML file: + +```sh +kubectl apply -f bundle.yaml +``` + +After running this command, you should see a result similar to this: + +```sh +role.rbac.authorization.k8s.io/redis-enterprise-operator configured +serviceaccount/redis-enterprise-operator configured +rolebinding.rbac.authorization.k8s.io/redis-enterprise-operator configured +customresourcedefinition.apiextensions.k8s.io/redisenterpriseclusters.app.redislabs.com configured +customresourcedefinition.apiextensions.k8s.io/redisenterprisedatabases.app.redislabs.com configured +deployment.apps/redis-enterprise-operator configured +``` + +### Reapply the admission controller webhook {#reapply-webhook} + +If you have the admission controller enabled, you need to manually reapply the `ValidatingWebhookConfiguration`. + +> [!NOTE] +> {{< embed-md "k8s-642-redb-admission-webhook-name-change.md" >}} + +{{< embed-md "k8s-admission-webhook-cert.md" >}} + +### Verify the operator is running + +You can check your deployment to verify the operator is running in your namespace. + +```sh +kubectl get deployment/redis-enterprise-operator +``` + +You should see a result similar to this: + +```sh +NAME READY UP-TO-DATE AVAILABLE AGE +redis-enterprise-operator 1/1 1 1 0m36s +``` + +> [!WARNING] +> We recommend upgrading the REC as soon as possible after updating the operator. After the operator upgrade completes, the operator suspends the management of the REC and its associated REDBs, until the REC upgrade completes. + +## Upgrade the Redis Enterprise cluster (REC) + +The Redis Enterprise cluster (REC) can be updated automatically or manually. To trigger automatic upgrade of the REC after the operator upgrade completes, set `spec.upgradeSpec.autoUpgradeRedisEnterprise: true` in your REC. If you don't have automatic upgrade enabled, follow the below steps for the manual upgrade. + +> [!WARNING] +> Automatic upgrade is incompatible with image digest hashes. If you set a `digestHash` in any image spec (`redisEnterpriseImageSpec`, `redisEnterpriseServicesRiggerImageSpec`, `bootstrapperImageSpec`, or `usageMeter.callHomeClient.imageSpec`) while `autoUpgradeRedisEnterprise` is `true`, the operator invalidates the REC spec and publishes a Kubernetes warning event. Remove all `digestHash` values before you enable automatic upgrade, or upgrade the cluster [manually](#edit-redisenterpriseimagespec-in-the-rec-spec) instead. + +Before beginning the upgrade of the Redis Enterprise cluster, check the [Redis Enterprise for Kubernetes release notes](/content/operate/kubernetes/release-notes/_index.md) to find the Redis Enterprise image tag for your target version. + +After the operator upgrade is complete, you can upgrade Redis Enterprise cluster (REC). + +> [!NOTE] +> For Active-Active databases, we recommend upgrading all participating clusters to the same operator version. + +### Edit `redisEnterpriseImageSpec` in the REC spec + +1. Edit the REC custom resource YAML file. + + ```sh + kubectl edit rec + ``` + +1. Replace the `versionTag:` declaration under `redisEnterpriseImageSpec` with the new version tag. + + ```YAML + spec: + redisEnterpriseImageSpec: + imagePullPolicy: IfNotPresent + repository: redislabs/redis + versionTag: + ``` + +1. Define any user-defined modules used by databases in the cluster. + + ```YAML + spec: + userDefinedModules: + - name: "custom-module" + source: + https: + url: "https://modules.company.com/search-v2.1.zip" + credentialsSecret: "module-repo-creds" + ``` + + The `name` field must match the `display_name` or `module_name` that appears in the module manifest (for example, "redisgears"). This enables the operator to run validation on the user-defined module. If these names don't match, the operator can't run validation on the user-defined module and preventable errors may occur. + + > [!NOTE] + > Adding or modifying the `userDefinedModules` list triggers a rolling restart of the Redis Enterprise cluster pods in addition to the rolling upgrade for the version change. + > + +1. Save the changes to apply. + +### Reapply roles and role bindings + +If your operator is monitoring multiple namespaces, you'll need to [reapply your role and role bindings](/content/operate/kubernetes/8.2.0/re-clusters/multi-namespace.md#create-role-and-role-binding-for-managed-namespaces) for each managed namespace. See [Manage databases in multiple namespaces](/content/operate/kubernetes/8.2.0/re-clusters/multi-namespace.md) for more details. + +### Monitor the upgrade + +You can view the state of the REC with `kubectl get rec`. + + During the upgrade, the state should be `Upgrade`. + When the upgrade is complete and the cluster is ready to use, the state will change to `Running`. + If the state is `InvalidUpgrade`, there is an error (usually relating to configuration) in the upgrade. + +```sh +$ kubectl get rec +NAME NODES VERSION STATE SPEC STATUS LICENSE STATE SHARDS LIMIT LICENSE EXPIRATION DATE AGE +rec 3 8.0.2-17 Upgrade Valid Valid 4 2022-07-16T13:59:00Z 92m +``` + +To see the status of the current rolling upgrade, run: + +```sh +kubectl rollout status sts +``` + +## Upgrade databases + +After the cluster is upgraded, you can upgrade your databases. + +### Upgrade REDB + +To upgrade your REDB, specify your new database version in the `spec.redisVersion` field in the REDB or REAADB custom resources. Supported database versions for operator versions include "7.2", "7.4", "8.0", and "8.2" (note this value is a string). + +### Upgrade REAADB + +For Active-Active databases, the `redis.Version` change only needs to be applied on one participating cluster and will automatically propagate to all other participating clusters. All participating clusters must be running operator version 8.0.2-2 or later. + +If your REAADB uses supported modules, keep the existing `moduleList` version numbers unchanged when upgrading `redisVersion`. The database will automatically use the module versions that are bundled with the new Redis version, regardless of what versions are specified in `moduleList`. After the upgrade is complete, you can optionally change the old version numbers from `moduleList`, but this change has no functional impact. + +### Upgrade with user-defined modules + +If a user-defined module is used by any database in the cluster, the module must be defined in the REC custom resource before upgrading the database. See [Add user-defined modules to the REC](/content/operate/kubernetes/8.2.0/re-databases/modules.md#add-user-defined-modules-to-the-rec) for detailed instructions. + +### Upgrade policy + +Note that if your cluster [`redisUpgradePolicy`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_cluster_api.md#spec) or your database [`redisVersion`](/content/operate/kubernetes/8.2.0/reference/api/redis_enterprise_database_api.md#spec) are set to `major`, you won't be able to upgrade those databases to minor versions. See [Redis upgrade policy](/content/operate/rs/installing-upgrading/upgrading/_index.md#redis-upgrade-policy) for more details. + +## Troubleshooting + +If you start an upgrade without meeting the [prerequisites](#prerequisites), the operator will freeze the upgrade. Check the operator logs for the source of the error. The REDB reconsilliation doesn't work during an upgrade, so you need to apply a manual fix with the Redis Software API (examples below). The updates will also need to be added to the REDB custom resource. + +### User-defined modules + +If your databases use user-defined modules and you encounter upgrade issues: + +1. **Verify `autoUpgradeRedisEnterprise` is set to `false`**: Check the REC spec to ensure automatic upgrades are disabled. + + ```sh + kubectl get rec -o jsonpath='{.spec.upgradeSpec.autoUpgradeRedisEnterprise}' + ``` + +1. **Verify modules are defined in the REC**: Ensure all user-defined modules are listed in the REC `userDefinedModules` section before upgrading. + + ```sh + kubectl get rec -o jsonpath='{.spec.userDefinedModules}' | jq + ``` + +1. **Check module validation errors**: Review the REC status for module validation errors. + + ```sh + kubectl describe rec + ``` + +For more information about user-defined modules, see [User-defined modules](/content/operate/kubernetes/8.2.0/re-databases/modules.md#user-defined-modules). + +### Invalid module version + +If the operator logs show an event related to an unsupported module, download the updated module locally, and install it using the `v2/modules` API endpoint. + +```sh +curl -sfk -u : -X POST -F 'module=@' https://localhost:9443/v2/modules +``` + +After updating the database with the Redis Software API, update the REDB custom resource to reflect the change. + +### Invalid database version + +If the operator logs show an event related to an incompatible database version, upgrade the database using the Redis Software API. + +```sh +curl -sfk -u : -X POST -H "Content-Type: application/json" -d '{"redis_version": }' https://localhost:9443/v1/bdbs//upgrade +``` + +After updating the modules with the Redis Software API, update the REDB custom resource to reflect the change. + +### Auto-upgrade blocked by image digest hashes + +If `autoUpgradeRedisEnterprise` is `true` and any image spec sets a `digestHash`, the operator invalidates the REC spec and blocks the upgrade until you remove the digest hashes. The REC enters the `InvalidUpgrade` state and the operator publishes a Kubernetes warning event describing the problem. + +1. **Check the REC events**: Review the event published on the REC. + + ```sh + kubectl describe rec + ``` + +1. **Remove the digest hashes**: Delete the `digestHash` field from every image spec that sets it (`redisEnterpriseImageSpec`, `redisEnterpriseServicesRiggerImageSpec`, `bootstrapperImageSpec`, and `usageMeter.callHomeClient.imageSpec`). Once the spec no longer uses digest hashes, the operator revalidates it and the upgrade proceeds. + +Alternatively, set `spec.upgradeSpec.autoUpgradeRedisEnterprise: false` and [upgrade the cluster manually](#edit-redisenterpriseimagespec-in-the-rec-spec), which supports digest hashes. \ No newline at end of file diff --git a/content/operate/kubernetes/kubernetes-archive.md b/content/operate/kubernetes/kubernetes-archive.md index 31f58ab006..ec1018a182 100644 --- a/content/operate/kubernetes/kubernetes-archive.md +++ b/content/operate/kubernetes/kubernetes-archive.md @@ -13,6 +13,7 @@ weight: 99999999999 The Redis Enterprise for Kubernetes documentation is versioned. You are looking at the documentation for the latest release. +- View the [8.2.0 docs](https://redis.io/docs/latest/operate/kubernetes/8.2.0/) for Redis Software for Kubernetes versions 8.2.0-12 through 8.2.0-18. - View the [8.0.18 docs](https://redis.io/docs/latest/operate/kubernetes/8.0.18/) for Redis Enterprise for Kubernetes versions 8.0.18-11 and 8.0.20-21. - View the [8.0 docs](https://redis.io/docs/latest/operate/kubernetes/8.0/) for Redis Enterprise for Kubernetes versions 8.0.2-2 through 8.0.10-23. - View the [7.22 docs](https://redis.io/docs/latest/operate/kubernetes/7.22/) for Redis Enterprise for Kubernetes versions 7.22.0-7 through 7.22.2-40. From f0c6168a0f4246219da8efe5d5d2935e8ce0006e Mon Sep 17 00:00:00 2001 From: Kaitlyn Michael <76962844+kaitlynmichael@users.noreply.github.com> Date: Mon, 19 Oct 2026 14:33:06 -0500 Subject: [PATCH 44/76] K8s: 7.22.2-47 yellow submarine maint 14 (#4301) * K8s: 7.22.2-47 yellow submarine maint 14 * K8s: update 7.22.2 latest release to 7.22.2-47 Co-Authored-By: Claude Opus 5.5 (1M context) --------- Co-authored-by: Claude Opus 5.5 (1M context) --- .../7-22-2-releases/7-22-2-47-october2026.md | 35 +++++++++++++++++++ .../release-notes/7-22-2-releases/_index.md | 2 +- 2 files changed, 36 insertions(+), 1 deletion(-) create mode 100644 content/operate/kubernetes/release-notes/7-22-2-releases/7-22-2-47-october2026.md diff --git a/content/operate/kubernetes/release-notes/7-22-2-releases/7-22-2-47-october2026.md b/content/operate/kubernetes/release-notes/7-22-2-releases/7-22-2-47-october2026.md new file mode 100644 index 0000000000..3ffb55e6a5 --- /dev/null +++ b/content/operate/kubernetes/release-notes/7-22-2-releases/7-22-2-47-october2026.md @@ -0,0 +1,35 @@ +--- +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: Maintenance release to support Redis Software version 7.22.2-204. +hideListLinks: true +linkTitle: 7.22.2-47 (October 2026) +title: Redis Software for Kubernetes 7.22.2-47 (October 2026) release notes +weight: 13 +--- + +## Highlights + +This is a maintenance release of Redis Software for Kubernetes to support Redis Software version 7.22.2-204. For supported distributions and known limitations, see the [7.22.2 releases](/content/operate/kubernetes/release-notes/7-22-2-releases/_index.md). + +## Downloads + +- **Redis Software**: `redislabs/redis:7.22.2-204` +- **Operator**: `redislabs/operator:7.22.2-47` +- **Services Rigger**: `redislabs/k8s-controller:7.22.2-47` +- **Call Home Client**: `redislabs/re-call-home-client:7.22.2-47` + +### OpenShift downloads + +- **OLM operator bundle**: `v7.22.2-47.1` +- **Redis Software**: `registry.connect.redhat.com/redislabs/redis-enterprise:7.22.2-204` +- **Operator**: `registry.connect.redhat.com/redislabs/redis-enterprise-operator:7.22.2-47` +- **Services Rigger**: `registry.connect.redhat.com/redislabs/services-manager:7.22.2-47` +- **Call Home Client**: `registry.connect.redhat.com/redislabs/call-home-client:7.22.2-47` + +## Known limitations + +See [7.22.2 releases](/content/operate/kubernetes/release-notes/7-22-2-releases/_index.md) for information on known limitations. diff --git a/content/operate/kubernetes/release-notes/7-22-2-releases/_index.md b/content/operate/kubernetes/release-notes/7-22-2-releases/_index.md index e9064e6eb9..63d0a45e94 100644 --- a/content/operate/kubernetes/release-notes/7-22-2-releases/_index.md +++ b/content/operate/kubernetes/release-notes/7-22-2-releases/_index.md @@ -11,7 +11,7 @@ title: Redis Enterprise for Kubernetes 7.22.2 release notes weight: 90 --- -Redis Enterprise for Kubernetes 7.22.2 includes bug fixes, enhancements, and support for Redis Enterprise Software. The latest release is 7.22.2-42 with support for Redis Enterprise Software version 7.22.2-170. +Redis Enterprise for Kubernetes 7.22.2 includes bug fixes, enhancements, and support for Redis Enterprise Software. The latest release is 7.22.2-47 with support for Redis Enterprise Software version 7.22.2-204. ## Detailed release notes From ccca67ec98d9429698ef2e09d9a9419c4c6c0246 Mon Sep 17 00:00:00 2001 From: Kaitlyn Michael <76962844+kaitlynmichael@users.noreply.github.com> Date: Mon, 19 Oct 2026 14:33:15 -0500 Subject: [PATCH 45/76] RS: 7.22.2-204 (#4299) * RS: 7.22.2-204 * RS: add RS216045 to 7.22.2-204 release notes The PUBSUB SHARDNUMSUB proxy crash fix (RED-217044, a clone of RS216045) is in the 7.22.2-204 release report. Wording matches the 7.8.6-331 note. Co-Authored-By: Claude Opus 5.5 (1M context) --------- Co-authored-by: Claude Opus 5.5 (1M context) --- .../rs-7-22-releases/rs-7-22-2-204.md | 436 ++++++++++++++++++ 1 file changed, 436 insertions(+) create mode 100644 content/operate/rs/release-notes/rs-7-22-releases/rs-7-22-2-204.md diff --git a/content/operate/rs/release-notes/rs-7-22-releases/rs-7-22-2-204.md b/content/operate/rs/release-notes/rs-7-22-releases/rs-7-22-2-204.md new file mode 100644 index 0000000000..63a5008159 --- /dev/null +++ b/content/operate/rs/release-notes/rs-7-22-releases/rs-7-22-2-204.md @@ -0,0 +1,436 @@ +--- +Title: Redis Software release notes 7.22.2-204 (October 2026) +alwaysopen: false +categories: +- docs +- operate +- rs +compatibleOSSVersion: Redis 7.4.3, 7.2.7, 6.2.13 +description: Bug fixes for DMC proxy stability. Internal fixes and improvements. +linkTitle: 7.22.2-204 (October 2026) +weight: 75 +--- + +This is a maintenance release for ​​Redis Software version 7.22.2. + +## Highlights + +This version offers: + +- Bug fixes for DMC proxy stability + +- Internal fixes and improvements + +## New in this release + +### Redis database versions and feature sets + +Redis Software version 7.22.2 includes three Redis database versions: 7.4.3, 7.2.7, and 6.2.13. + +The [default Redis database version](/content/operate/rs/databases/configure/db-defaults.md#database-version) is 7.4. + +Redis Software comes packaged with several modules. As of version 7.22.0, Redis Software includes three feature sets, compatible with different Redis database versions. + +The following table shows which Redis modules are compatible with each Redis database version included in this release. + +{{< collapsible "Show feature set details" >}} + +| Redis database version | Compatible Redis modules | +|------------------------|--------------------------| +| 7.4 | [RediSearch 2.10](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisearch/redisearch-2.10-release-notes.md)
    [RedisJSON 2.8](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisjson/redisjson-2.8-release-notes.md)
    [RedisTimeSeries 1.12](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redistimeseries/redistimeseries-1.12-release-notes.md)
    [RedisBloom 2.8](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisbloom/redisbloom-2.8-release-notes.md) | +| 7.2 | [RediSearch 2.8](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisearch/redisearch-2.8-release-notes.md)
    [RedisJSON 2.6](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisjson/redisjson-2.6-release-notes.md)
    [RedisTimeSeries 1.10](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redistimeseries/redistimeseries-1.10-release-notes.md)
    [RedisBloom 2.6](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisbloom/redisbloom-2.6-release-notes.md) | +| 6.2 | [RediSearch 2.6](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisearch/redisearch-2.6-release-notes.md)
    [RedisJSON 2.4](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisjson/redisjson-2.4-release-notes.md)
    [RedisTimeSeries 1.8](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redistimeseries/redistimeseries-1.8-release-notes.md)
    [RedisBloom 2.4](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisbloom/redisbloom-2.4-release-notes.md) | + +{{< /collapsible >}} + +### Resolved issues + +- RS216045: Fixed an issue where the DMC proxy could crash repeatedly when a client sent a command without key arguments, such as `PUBSUB SHARDNUMSUB` with no channels, in a pipeline with another command. + +- RS215608: Fixed an issue where running the `dmc-cli` `list bdb` or `list server` commands could cause the proxy to crash. + +## Version changes + +### Supported platforms + +The following table provides a snapshot of supported platforms as of this Redis Software release. See the [supported platforms reference](/content/operate/rs/references/supported-platforms.md) for more details about operating system compatibility. + +{{< collapsible "Show supported platforms" >}} + +✅ Supported – The platform is supported for this version of Redis Software and Redis Stack modules. + +:warning: Deprecation warning – The platform is still supported for this version of Redis Software, but support will be removed in a future release. + +| Redis Software
    major versions | 8.0 | 7.22 | 7.8 | 7.4 | 7.2 | 6.4 | 6.2 | +|---------------------------------|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:| +| **Release date** | Oct 2025 | May 2025 | Nov 2024 | Feb 2024 | Aug 2023 | Feb 2023 | Aug 2021 | +| [**End-of-life date**](/content/operate/rs/installing-upgrading/product-lifecycle.md#endoflife-schedule) | Determined after
    next major release | Oct 2027 | May 2027 | Nov 2026 | Feb 2026 | Aug 2025 | Feb 2025 | +| **Platforms** | | | | | | | | +| RHEL 9 &
    compatible distros[1](#table-note-1) | ✅ | ✅ | ✅ | ✅ | – | – | – | +| RHEL 9
    FIPS mode[5](#table-note-5) | ✅ | ✅ | ✅ | – | – | – | – | +| RHEL 8 &
    compatible distros[1](#table-note-1) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| RHEL 7 &
    compatible distros[1](#table-note-1) | – | – | – | – | :warning: | :warning: | :warning: | +| Ubuntu 22.04[2](#table-note-2) | ✅ | ✅ | ✅ | – | – | – | – | +| Ubuntu 20.04[2](#table-note-2) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | – | +| Ubuntu 18.04[2](#table-note-2) | – | – | – | :warning: | :warning: | :warning: | :warning: | +| Ubuntu 16.04[2](#table-note-2) | – | – | – | – | :warning: | :warning: | :warning: | +| Amazon Linux 2 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | – | +| Amazon Linux 1 | – | – | – | – | :warning: | :warning: | :warning: | +| Kubernetes[3](#table-note-3) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| Docker[4](#table-note-4) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | + +1. The RHEL-compatible distributions CentOS, CentOS Stream, Alma, and Rocky are supported if they have full RHEL compatibility. Oracle Linux running the Red Hat Compatible Kernel (RHCK) is supported, but the Unbreakable Enterprise Kernel (UEK) is not supported. + +2. The server version of Ubuntu is recommended for production installations. The desktop version is only recommended for development deployments. + +3. See the [Redis Enterprise for Kubernetes documentation](/content/operate/kubernetes/reference/supported_k8s_distributions.md) for details about support per version and Kubernetes distribution. + +4. [Docker images](/content/operate/rs/installing-upgrading/quickstarts/docker-quickstart.md) of Redis Software are certified for development and testing only. + +5. Supported only if [FIPS was enabled during RHEL installation](https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/9/html/security_hardening/switching-rhel-to-fips-mode_security-hardening#proc_installing-the-system-with-fips-mode-enabled_switching-rhel-to-fips-mode) to ensure FIPS compliance. + +{{< /collapsible >}} + +## Downloads + +The following table shows the SHA256 checksums for the available packages: + +| Package | SHA256 checksum (7.22.2-204 October release) | +|---------|---------------------------------------| +| Ubuntu 20 | 5fa0b0cadf37596b93e9bbea24127eb7096073cccc6895e14c5cee3ea585f475 | +| Ubuntu 22 | 8893a1aea58a7fbee4bacb940ce9e1c652f05e6f3b08fa20251a01fe50324fc3 | +| Red Hat Enterprise Linux (RHEL) 8 | 81e35b0034daaeba41cbab05485e4db6e11b7b409522cba1597cc26553e63407 | +| Red Hat Enterprise Linux (RHEL) 9 | fb086b134e9b09bc08b840f051031310c7828a65ae1e20467ad1093cf2375e75 | +| Amazon Linux 2 | 80b4cef04796c5415f82532b409811e7becb7b66e79580b33a9b27b4293060b7 | + +## Known issues + +- RS131972: Creating an ACL that contains a line break in the Cluster Manager UI can cause shard migration to fail due to ACL errors. + +- RS155734: Endpoint availability metrics do not work as expected due to a calculation error. + +## Known limitations + +#### Upload modules before OS upgrade + +If the cluster contains any databases that use modules, you must upload module packages for the target OS version to a node in the existing cluster before you upgrade the cluster's operating system. + +See [Upgrade a cluster's operating system](/content/operate/rs/installing-upgrading/upgrading/upgrade-os.md) for detailed upgrade instructions. + +#### New Cluster Manager UI limitations + +The following legacy UI features are not yet available in the new Cluster Manager UI: + +- Purge an Active-Active instance. + + Use [`crdb-cli crdb purge-instance`](/content/operate/rs/references/cli-utilities/crdb-cli/crdb/purge-instance.md) instead. + +- Search and export the log. + +#### RedisGraph prevents upgrade to RHEL 9 + +You cannot upgrade from a prior RHEL version to RHEL 9 if the Redis Software cluster contains a RedisGraph module, even if unused by any database. The [RedisGraph module has reached End-of-Life](https://redis.com/blog/redisgraph-eol/) and is completely unavailable in RHEL 9. + +#### Query results might include hash keys with lazily expired fields + +If one or more fields of a hash key expire after an `FT.SEARCH` or `FT.AGGREGATE` query begins, Redis does not account for these lazily expired fields. As a result, keys with expired fields might still be included in the query results, leading to potentially incorrect or inconsistent results. + +#### Active defragmentation does not stop mid-key for JSON + +Active defragmentation does not stop mid-key for JSON data. Large keys are defragmented in full, which might cause latency spikes. + +## Security + +#### Open source Redis security fixes compatibility + +As part of Redis's commitment to security, Redis Software implements the latest [security fixes](https://github.com/redis/redis/releases) available with [open source Redis](https://github.com/redis/redis). Redis Software has already included the fixes for the relevant CVEs. + +Some CVEs announced for open source Redis do not affect Redis Software due to different or additional functionality available in Redis Software that is not available in open source Redis. + +Redis Software 7.22.2-204 supports open source Redis 7.4, 7.2, and 6.2. Below is the list of open source Redis CVEs and other security vulnerabilities fixed by version. + +{{< collapsible "Redis 7.4.x" >}} + +- Memory corruption in `SLOT_INFO` handling may allow an unauthenticated user to potentially run their own code. + +- A use-after-free in TLS pending-data processing may allow an authenticated user to cause a denial-of-service. + +- A vulnerability in the `RESTORE` command and in RDB loading allows an authenticated user to trigger a use-after-free in stream consumer groups via a specially crafted serialized payload, potentially resulting in remote code execution. + +- An ACL key-pattern bypass in `GEORADIUS` and `GEORADIUSBYMEMBER` may allow an authenticated user to write to keys outside their permitted key patterns. + +- An out-of-bounds read during command key extraction may cause the server to crash. + +- (CVE-2026-62356) An authenticated user may trigger a heap out-of-bounds write in RedisBloom by loading a specially crafted Count-Min Sketch (`CMSketch`) object, for example via the `RESTORE` command or a crafted RDB file. + +- RedisBloom: Out-of-bounds access in the `TopK` heap cleanup path. + +- (CVE-2026-23479) When a blocked client is evicted while re-executing a blocked command, an authenticated user may trigger a use-after-free and potentially lead to remote code execution. + +- (CVE‑2026‑25243) A vulnerability in the Redis `RESTORE` command allows an authenticated user to trigger an invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE-2026-25588) A vulnerability in the `RESTORE` command, when used with the RedisTimeSeries module, allows an authenticated attacker to trigger invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE‑2026‑25589) A vulnerability in the `RESTORE` command, when used with the RedisBloom module, allows an authenticated attacker to trigger invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE-2026-23631) An authenticated user may exploit the synchronization mechanism of the master-replica and trigger a use-after-free vulnerability, potentially leading to remote code execution. The bug affects only replicas that are configured, or may be configured with `replica-read-only` disabled, and exists in all versions of Redis with Lua scripting. + +- Insufficient sanitization of Lua script error messages can allow an authenticated user with `EVAL` permission to inject arbitrary RESP protocol responses, leading to denial of service, data manipulation, and connection pool poisoning. + +- RedisBloom: Integer overflow in `TopK` RDB loading may lead to heap out-of-bounds access. + +- RedisBloom: Missing validation in `td_histogram_t` RDB loading may lead to heap buffer overflow. + +- RedisBloom: Missing validation in `CuckooFilter` RDB loading may lead to heap out-of-bounds access. + +- RedisBloom: Missing validation in `SBChain` and `CMSketch` RDB loading may lead to heap out-of-bounds write. + +- Added validation of limits during RDB load and data creation to avoid resource exhaustion and Denial-of-Service (DoS) vulnerabilities. + +- Integer overflow in `hllPatLen`. + +- RedisBloom: Restore invalid filter. + +- RedisBloom: Cuckoo filter counter overflow. + +- RedisBloom: Invalid Bloom filters can cause arbitrary memory reads and writes. + +- RedisBloom: Reachable assert in `TopK_Create` + +- RedisBloom: Out-of-bounds access with empty Bloom chains. + +- RedisBloom: Division by zero in Cuckoo filter insertion. + +- (CVE-2025-46818) An authenticated user may use a specially crafted Lua script to manipulate different LUA objects and potentially run their own code in the context of another user. + +- (CVE-2025-46819) An authenticated user may use a specially crafted LUA script to read out-of-bound data or crash the server and lead to subsequent denial of service. + +- (CVE-2025-46817) An authenticated user may use a specially crafted Lua script to cause an integer overflow and potentially lead to remote code execution. + +- (CVE-2025-49844) An authenticated user may use a specially crafted Lua script to manipulate the garbage collector, trigger a use-after-free, and potentially lead to remote code execution. + +- (CVE-2025-32023) An authenticated user can use a specially crafted string to trigger a stack/heap out-of-bounds write on HyperLogLog operations, which can lead to remote code execution. + +- (CVE-2025-21605) An unauthenticated client can cause unlimited growth of output buffers until the server runs out of memory or is terminated, which can lead to denial-of-service. + +{{< /collapsible >}} + +{{< collapsible "Redis 7.2.x" >}} + +- Memory corruption in `SLOT_INFO` handling may allow an unauthenticated user to potentially run their own code. + +- A use-after-free in TLS pending-data processing may allow an authenticated user to cause a denial-of-service. + +- A vulnerability in the `RESTORE` command and in RDB loading allows an authenticated user to trigger a use-after-free in stream consumer groups via a specially crafted serialized payload, potentially resulting in remote code execution. + +- An ACL key-pattern bypass in `GEORADIUS` and `GEORADIUSBYMEMBER` may allow an authenticated user to write to keys outside their permitted key patterns. + +- An out-of-bounds read during command key extraction may cause the server to crash. + +- (CVE-2026-62356) An authenticated user may trigger a heap out-of-bounds write in RedisBloom by loading a specially crafted Count-Min Sketch (`CMSketch`) object, for example via the `RESTORE` command or a crafted RDB file. + +- RedisBloom: Out-of-bounds access in the `TopK` heap cleanup path. + +- (CVE-2026-23479) When a blocked client is evicted while re-executing a blocked command, an authenticated user may trigger a use-after-free and potentially lead to remote code execution. + +- (CVE‑2026‑25243) A vulnerability in the Redis `RESTORE` command allows an authenticated user to trigger an invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE-2026-25588) A vulnerability in the `RESTORE` command, when used with the RedisTimeSeries module, allows an authenticated attacker to trigger invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE‑2026‑25589) A vulnerability in the `RESTORE` command, when used with the RedisBloom module, allows an authenticated attacker to trigger invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE-2026-23631) An authenticated user may exploit the synchronization mechanism of the master-replica and trigger a use-after-free vulnerability, potentially leading to remote code execution. The bug affects only replicas that are configured, or may be configured with `replica-read-only` disabled, and exists in all versions of Redis with Lua scripting. + +- Insufficient sanitization of Lua script error messages can allow an authenticated user with `EVAL` permission to inject arbitrary RESP protocol responses, leading to denial of service, data manipulation, and connection pool poisoning. + +- RedisBloom: Integer overflow in `TopK` RDB loading may lead to heap out-of-bounds access. + +- RedisBloom: Missing validation in `td_histogram_t` RDB loading may lead to heap buffer overflow. + +- RedisBloom: Missing validation in `CuckooFilter` RDB loading may lead to heap out-of-bounds access. + +- RedisBloom: Missing validation in `SBChain` and `CMSketch` RDB loading may lead to heap out-of-bounds write. + +- Added validation of limits during RDB load and data creation to avoid resource exhaustion and Denial-of-Service (DoS) vulnerabilities. + +- Integer overflow in `hllPatLen`. + +- RedisBloom: Restore invalid filter. + +- RedisBloom: Cuckoo filter counter overflow. + +- RedisBloom: Invalid Bloom filters can cause arbitrary memory reads and writes. + +- RedisBloom: Reachable assert in `TopK_Create` + +- RedisBloom: Out-of-bounds access with empty Bloom chains. + +- RedisBloom: Division by zero in Cuckoo filter insertion. + +- (CVE-2025-46818) An authenticated user may use a specially crafted Lua script to manipulate different LUA objects and potentially run their own code in the context of another user. + +- (CVE-2025-46819) An authenticated user may use a specially crafted LUA script to read out-of-bound data or crash the server and lead to subsequent denial of service. + +- (CVE-2025-46817) An authenticated user may use a specially crafted Lua script to cause an integer overflow and potentially lead to remote code execution. + +- (CVE-2025-49844) An authenticated user may use a specially crafted Lua script to manipulate the garbage collector, trigger a use-after-free, and potentially lead to remote code execution. + +- (CVE-2025-32023) An authenticated user can use a specially crafted string to trigger a stack/heap out-of-bounds write on HyperLogLog operations, which can lead to remote code execution. + +- (CVE-2025-21605) An unauthenticated client can cause unlimited growth of output buffers until the server runs out of memory or is terminated, which can lead to denial-of-service. + +- (CVE-2024-31449) An authenticated user may use a specially crafted Lua script to trigger a stack buffer overflow in the bit library, which may potentially lead to remote code execution. + +- (CVE-2024-31228) An authenticated user can trigger a denial-of-service by using specially crafted, long string match patterns on supported commands such as `KEYS`, `SCAN`, `PSUBSCRIBE`, `FUNCTION LIST`, `COMMAND LIST`, and ACL definitions. Matching of extremely long patterns may result in unbounded recursion, leading to stack overflow and process crashes. + +- (CVE-2023-41056) In some cases, Redis may incorrectly handle resizing of memory buffers, which can result in incorrect accounting of buffer sizes and lead to heap overflow and potential remote code execution. + +- (CVE-2023-41053) Redis does not correctly identify keys accessed by `SORT_RO` and, as a result, may grant users executing this command access to keys that are not explicitly authorized by the ACL configuration. (Redis 7.2.1) + +{{< /collapsible >}} + +{{< collapsible "Redis 7.0.x" >}} + +- (CVE-2024-31449) An authenticated user may use a specially crafted Lua script to trigger a stack buffer overflow in the bit library, which may potentially lead to remote code execution. + +- (CVE-2024-31228) An authenticated user can trigger a denial-of-service by using specially crafted, long string match patterns on supported commands such as `KEYS`, `SCAN`, `PSUBSCRIBE`, `FUNCTION LIST`, `COMMAND LIST`, and ACL definitions. Matching of extremely long patterns may result in unbounded recursion, leading to stack overflow and process crashes. + +- (CVE-2023-41056) In some cases, Redis may incorrectly handle resizing of memory buffers, which can result in incorrect accounting of buffer sizes and lead to heap overflow and potential remote code execution. + +- (CVE-2023-41053) Redis does not correctly identify keys accessed by `SORT_RO` and, as a result, may grant users executing this command access to keys that are not explicitly authorized by the ACL configuration. (Redis 7.0.13) + +- (CVE-2023-36824) Extracting key names from a command and a list of arguments may, in some cases, trigger a heap overflow and result in reading random heap memory, heap corruption, and potentially remote code execution. Specifically: using `COMMAND GETKEYS*` and validation of key names in ACL rules. (Redis 7.0.12) + +- (CVE-2023-28856) Authenticated users can use the `HINCRBYFLOAT` command to create an invalid hash field that will crash Redis on access. (Redis 7.0.11) + +- (CVE-2023-28425) Specially crafted `MSETNX` commands can lead to assertion and denial-of-service. (Redis 7.0.10) + +- (CVE-2023-25155) Specially crafted `SRANDMEMBER`, `ZRANDMEMBER`, and `HRANDFIELD` commands can trigger an integer overflow, resulting in a runtime assertion and termination of the Redis server process. (Redis 7.0.9) + +- (CVE-2023-22458) Integer overflow in the Redis `HRANDFIELD` and `ZRANDMEMBER` commands can lead to denial-of-service. (Redis 7.0.8) + +- (CVE-2022-36021) String matching commands (like `SCAN` or `KEYS`) with a specially crafted pattern to trigger a denial-of-service attack on Redis can cause it to hang and consume 100% CPU time. (Redis 7.0.9) + +- (CVE-2022-35977) Integer overflow in the Redis `SETRANGE` and `SORT`/`SORT_RO` commands can drive Redis to OOM panic. (Redis 7.0.8) + +- (CVE-2022-35951) Executing an `XAUTOCLAIM` command on a stream key in a specific state, with a specially crafted `COUNT` argument, may cause an integer overflow, a subsequent heap overflow, and potentially lead to remote code execution. The problem affects Redis versions 7.0.0 or newer. (Redis 7.0.5) + +- (CVE-2022-31144) A specially crafted `XAUTOCLAIM` command on a stream key in a specific state may result in heap overflow and potentially remote code execution. The problem affects Redis versions 7.0.0 or newer. (Redis 7.0.4) + +- (CVE-2022-24834) A specially crafted Lua script executing in Redis can trigger a heap overflow in the cjson and cmsgpack libraries, and result in heap corruption and potentially remote code execution. The problem exists in all versions of Redis with Lua scripting support, starting from 2.6, and affects only authenticated and authorized users. (Redis 7.0.12) + +- (CVE-2022-24736) An attacker attempting to load a specially crafted Lua script can cause NULL pointer dereference which will result in a crash of the `redis-server` process. This issue affects all versions of Redis. (Redis 7.0.0) + +- (CVE-2022-24735) By exploiting weaknesses in the Lua script execution environment, an attacker with access to Redis can inject Lua code that will execute with the (potentially higher) privileges of another Redis user. (Redis 7.0.0) + +{{< /collapsible >}} + +{{< collapsible "Redis 6.2.x" >}} + +- Memory corruption in `SLOT_INFO` handling may allow an unauthenticated user to potentially run their own code. + +- A use-after-free in TLS pending-data processing may allow an authenticated user to cause a denial-of-service. + +- A vulnerability in the `RESTORE` command and in RDB loading allows an authenticated user to trigger a use-after-free in stream consumer groups via a specially crafted serialized payload, potentially resulting in remote code execution. + +- An ACL key-pattern bypass in `GEORADIUS` and `GEORADIUSBYMEMBER` may allow an authenticated user to write to keys outside their permitted key patterns. + +- An out-of-bounds read during command key extraction may cause the server to crash. + +- (CVE-2026-62356) An authenticated user may trigger a heap out-of-bounds write in RedisBloom by loading a specially crafted Count-Min Sketch (`CMSketch`) object, for example via the `RESTORE` command or a crafted RDB file. + +- RedisBloom: Out-of-bounds access in the `TopK` heap cleanup path. + +- (CVE-2026-23479) When a blocked client is evicted while re-executing a blocked command, an authenticated user may trigger a use-after-free and potentially lead to remote code execution. + +- (CVE‑2026‑25243) A vulnerability in the Redis `RESTORE` command allows an authenticated user to trigger an invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE-2026-25588) A vulnerability in the `RESTORE` command, when used with the RedisTimeSeries module, allows an authenticated attacker to trigger invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE‑2026‑25589) A vulnerability in the `RESTORE` command, when used with the RedisBloom module, allows an authenticated attacker to trigger invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE-2026-23631) An authenticated user may exploit the synchronization mechanism of the master-replica and trigger a use-after-free vulnerability, potentially leading to remote code execution. The bug affects only replicas that are configured, or may be configured with `replica-read-only` disabled, and exists in all versions of Redis with Lua scripting. + +- Insufficient sanitization of Lua script error messages can allow an authenticated user with `EVAL` permission to inject arbitrary RESP protocol responses, leading to denial of service, data manipulation, and connection pool poisoning. + +- RedisBloom: Integer overflow in `TopK` RDB loading may lead to heap out-of-bounds access. + +- RedisBloom: Missing validation in `td_histogram_t` RDB loading may lead to heap buffer overflow. + +- RedisBloom: Missing validation in `CuckooFilter` RDB loading may lead to heap out-of-bounds access. + +- RedisBloom: Missing validation in `SBChain` and `CMSketch` RDB loading may lead to heap out-of-bounds write. + +- Added validation of limits during RDB load and data creation to avoid resource exhaustion and Denial-of-Service (DoS) vulnerabilities. + +- Integer overflow in `hllPatLen`. + +- RedisBloom: Restore invalid filter. + +- RedisBloom: Cuckoo filter counter overflow. + +- RedisBloom: Invalid Bloom filters can cause arbitrary memory reads and writes. + +- RedisBloom: Reachable assert in `TopK_Create` + +- RedisBloom: Out-of-bounds access with empty Bloom chains. + +- RedisBloom: Division by zero in Cuckoo filter insertion. + +- (CVE-2025-46818) An authenticated user may use a specially crafted Lua script to manipulate different LUA objects and potentially run their own code in the context of another user. + +- (CVE-2025-46819) An authenticated user may use a specially crafted LUA script to read out-of-bound data or crash the server and lead to subsequent denial of service. + +- (CVE-2025-46817) An authenticated user may use a specially crafted Lua script to cause an integer overflow and potentially lead to remote code execution. + +- (CVE-2025-49844) An authenticated user may use a specially crafted Lua script to manipulate the garbage collector, trigger a use-after-free, and potentially lead to remote code execution. + +- (CVE-2025-32023) An authenticated user can use a specially crafted string to trigger a stack/heap out-of-bounds write on HyperLogLog operations, which can lead to remote code execution. + +- (CVE-2025-21605) An unauthenticated client can cause unlimited growth of output buffers until the server runs out of memory or is terminated, which can lead to denial-of-service. + +- (CVE-2024-31449) An authenticated user may use a specially crafted Lua script to trigger a stack buffer overflow in the bit library, which may potentially lead to remote code execution. + +- (CVE-2024-31228) An authenticated user can trigger a denial-of-service by using specially crafted, long string match patterns on supported commands such as `KEYS`, `SCAN`, `PSUBSCRIBE`, `FUNCTION LIST`, `COMMAND LIST`, and ACL definitions. Matching of extremely long patterns may result in unbounded recursion, leading to stack overflow and process crashes. + +- (CVE-2023-28856) Authenticated users can use the `HINCRBYFLOAT` command to create an invalid hash field that will crash Redis on access. (Redis 6.2.12) + +- (CVE-2023-25155) Specially crafted `SRANDMEMBER`, `ZRANDMEMBER`, and `HRANDFIELD` commands can trigger an integer overflow, resulting in a runtime assertion and termination of the Redis server process. (Redis 6.2.11) + +- (CVE-2023-22458) Integer overflow in the Redis `HRANDFIELD` and `ZRANDMEMBER` commands can lead to denial-of-service. (Redis 6.2.9) + +- (CVE-2022-36021) String matching commands (like `SCAN` or `KEYS`) with a specially crafted pattern to trigger a denial-of-service attack on Redis can cause it to hang and consume 100% CPU time. (Redis 6.2.11) + +- (CVE-2022-35977) Integer overflow in the Redis `SETRANGE` and `SORT`/`SORT_RO` commands can drive Redis to OOM panic. (Redis 6.2.9) + +- (CVE-2022-24834) A specially crafted Lua script executing in Redis can trigger a heap overflow in the cjson and cmsgpack libraries, and result in heap corruption and potentially remote code execution. The problem exists in all versions of Redis with Lua scripting support, starting from 2.6, and affects only authenticated and authorized users. (Redis 6.2.13) + +- (CVE-2022-24736) An attacker attempting to load a specially crafted Lua script can cause NULL pointer dereference which will result in a crash of the `redis-server` process. This issue affects all versions of Redis. (Redis 6.2.7) + +- (CVE-2022-24735) By exploiting weaknesses in the Lua script execution environment, an attacker with access to Redis can inject Lua code that will execute with the (potentially higher) privileges of another Redis user. (Redis 6.2.7) + +- (CVE-2021-41099) Integer to heap buffer overflow handling certain string commands and network payloads, when `proto-max-bulk-len` is manually configured to a non-default, very large value. (Redis 6.2.6) + +- (CVE-2021-32762) Integer to heap buffer overflow issue in `redis-cli` and `redis-sentinel` parsing large multi-bulk replies on some older and less common platforms. (Redis 6.2.6) + +- (CVE-2021-32761) An integer overflow bug in Redis version 2.2 or newer can be exploited using the `BITFIELD` command to corrupt the heap and potentially result with remote code execution. (Redis 6.2.5) + +- (CVE-2021-32687) Integer to heap buffer overflow with intsets, when `set-max-intset-entries` is manually configured to a non-default, very large value. (Redis 6.2.6) + +- (CVE-2021-32675) Denial Of Service when processing RESP request payloads with a large number of elements on many connections. (Redis 6.2.6) + +- (CVE-2021-32672) Random heap reading issue with Lua Debugger. (Redis 6.2.6) + +- (CVE-2021-32628) Integer to heap buffer overflow handling ziplist-encoded data types, when configuring a large, non-default value for `hash-max-ziplist-entries`, `hash-max-ziplist-value`, `zset-max-ziplist-entries` or `zset-max-ziplist-value`. (Redis 6.2.6) + +- (CVE-2021-32627) Integer to heap buffer overflow issue with streams, when configuring a non-default, large value for `proto-max-bulk-len` and `client-query-buffer-limit`. (Redis 6.2.6) + +- (CVE-2021-32626) Specially crafted Lua scripts may result with Heap buffer overflow. (Redis 6.2.6) + +- (CVE-2021-32625) An integer overflow bug in Redis version 6.0 or newer can be exploited using the STRALGO LCS command to corrupt the heap and potentially result with remote code execution. This is a result of an incomplete fix by CVE-2021-29477. (Redis 6.2.4) + +- (CVE-2021-29478) An integer overflow bug in Redis 6.2 could be exploited to corrupt the heap and potentially result with remote code execution. The vulnerability involves changing the default set-max-intset-entries configuration value, creating a large set key that consists of integer values and using the COPY command to duplicate it. The integer overflow bug exists in all versions of Redis starting with 2.6, where it could result with a corrupted RDB or DUMP payload, but not exploited through COPY (which did not exist before 6.2). (Redis 6.2.3) + +- (CVE-2021-29477) An integer overflow bug in Redis version 6.0 or newer could be exploited using the STRALGO LCS command to corrupt the heap and potentially result in remote code execution. The integer overflow bug exists in all versions of Redis starting with 6.0. (Redis 6.2.3) + +{{< /collapsible >}} From 862bed74cdd41918302f1c4a536cc7983ac69fbb Mon Sep 17 00:00:00 2001 From: paoloredis <158162796+paoloredis@users.noreply.github.com> Date: Tue, 20 Oct 2026 10:31:43 +0200 Subject: [PATCH 46/76] DOC-7201: Add a Previous button to the scripted demos (#4296) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * DOC-7201: Add a Previous button to the scripted demos Every step after the first now has a "← Previous: " button that goes back one step, in the seven tabbed demos: Context Retriever, Agent Memory, LangCache, RDI, RedisVL, Search indexing, and Search query. The Iris tour already had Back, so it's unchanged. In Context Retriever, RedisVL, and the two Search demos, Previous sits on the left of the step's row and Next on the right. In Agent Memory, LangCache, and RDI, it sits next to Restart and, like Next, is disabled while a step's animation runs. The wording follows each demo's own Next button. Co-Authored-By: Claude Opus 5.5 * DOC-7201: Move Restart to an icon in the demo window's corner In Agent Memory, LangCache, and RDI, Restart is now an icon button in the top-right corner of the demo window, on every step, instead of a button next to Previous. It's off until there's something to restart, and it takes the demo back to step 1. The bottom row of each step now has Previous on the left and Next, or the step's action, on the right. Co-Authored-By: Claude Opus 5.5 --------- Co-authored-by: Claude Opus 5.5 --- static/css/agent-memory-demo.css | 2 +- static/css/context-retriever-demo.css | 14 +++++++++++- static/css/langcache-demo.css | 2 +- static/css/rdi-demo.css | 4 ++-- static/js/agent-memory-demo.js | 33 ++++++++++++++++++++++----- static/js/context-retriever-demo.js | 25 +++++++++++++++----- static/js/langcache-demo.js | 33 ++++++++++++++++++++++----- static/js/rdi-demo.js | 33 +++++++++++++++++++++------ static/js/redisvl-demo.js | 21 ++++++++++++----- static/js/search-demo.js | 21 ++++++++++++----- static/js/search-index-demo.js | 21 ++++++++++++----- 11 files changed, 161 insertions(+), 48 deletions(-) diff --git a/static/css/agent-memory-demo.css b/static/css/agent-memory-demo.css index fe7d8564b4..2a67bc1bb7 100644 --- a/static/css/agent-memory-demo.css +++ b/static/css/agent-memory-demo.css @@ -120,7 +120,7 @@ padding: 8px 12px; margin-top: 12px; font-size: 13.5px; } .ram-callout p + p { margin-top: 6px !important; } -.ram-controls { display: flex; justify-content: space-between; align-items: center; gap: 8px; margin-top: 14px; } +.ram-controls { display: flex; flex-wrap: wrap; justify-content: space-between; align-items: center; gap: 8px; margin-top: 14px; } .ram .rcr-btn:disabled { opacity: .5; cursor: default; } /* Status codes line up in a column: Geist's default digits have different widths ("1" is 384 units, "0" is 671), so "201" and "200" would end at diff --git a/static/css/context-retriever-demo.css b/static/css/context-retriever-demo.css index 7a1bdc1302..a7a81c4014 100644 --- a/static/css/context-retriever-demo.css +++ b/static/css/context-retriever-demo.css @@ -69,7 +69,19 @@ .rcr-card { border: 1px solid var(--rcr-line); border-radius: 10px; background: var(--rcr-surface); padding: 12px; min-width: 0; } .rcr-cardhead { display: flex; align-items: center; justify-content: space-between; gap: 8px; font-weight: 600; font-size: 13.5px; margin-bottom: 8px; } .rcr-count { background: var(--rcr-surface-2); border: 1px solid var(--rcr-line); border-radius: 999px; padding: 0 8px; font-size: 11.5px; color: var(--rcr-muted); font-weight: 600; } -.rcr-next { display: flex; justify-content: flex-end; margin-top: 14px; } +.rcr-next { display: flex; flex-wrap: wrap; justify-content: flex-end; gap: 8px; margin-top: 14px; } +.rcr-next .rcr-prev { margin-right: auto; } /* Previous on the left, Next on the right */ +.rcr-navleft { display: flex; flex-wrap: wrap; align-items: center; gap: 8px; } /* Previous, on the left */ +.rcr-navright { display: flex; flex-wrap: wrap; align-items: center; justify-content: flex-end; gap: 8px; margin-left: auto; } /* the step's action or Next */ +/* Restart, as an icon in the demo window's top-right corner. Whatever comes first in the window, the tabs or + Agent Memory's service settings, leaves room for it. */ +.rcr.has-restart { position: relative; } +.rcr.has-restart > :first-child { margin-right: 44px; } +.rcr .rcr-restart { position: absolute; top: 18px; right: 16px; display: grid; place-items: center; width: 32px; height: 32px; padding: 0; + border: 1px solid var(--rcr-line-strong); border-radius: 8px; background: var(--rcr-surface); color: var(--rcr-muted); cursor: pointer; } +.rcr .rcr-restart:hover:not(:disabled) { border-color: var(--rcr-muted); color: var(--rcr-ink); } +.rcr .rcr-restart:disabled { opacity: .4; cursor: default; } +.rcr .rcr-restart:focus-visible { outline: none; box-shadow: 0 0 0 3px var(--rcr-focus); } .rcr-btn { cursor: pointer; border-radius: 8px; padding: 7px 14px; border: 1px solid var(--rcr-line-strong); background: #fff; font-size: 13.5px; } .rcr-btn-primary { background: var(--rcr-accent); border-color: var(--rcr-accent); color: #fff; font-weight: 600; } diff --git a/static/css/langcache-demo.css b/static/css/langcache-demo.css index c19ef332f9..583214543a 100644 --- a/static/css/langcache-demo.css +++ b/static/css/langcache-demo.css @@ -161,7 +161,7 @@ padding: 8px 12px; margin-top: 12px; font-size: 13.5px; } .rlc-callout p + p { margin-top: 6px !important; } -.rlc-controls { display: flex; justify-content: space-between; align-items: center; gap: 8px; margin-top: 14px; } +.rlc-controls { display: flex; flex-wrap: wrap; justify-content: space-between; align-items: center; gap: 8px; margin-top: 14px; } @media (prefers-reduced-motion: reduce) { .rlc-fresh { animation: none; } diff --git a/static/css/rdi-demo.css b/static/css/rdi-demo.css index de180693de..352e399c7f 100644 --- a/static/css/rdi-demo.css +++ b/static/css/rdi-demo.css @@ -189,8 +189,8 @@ padding: 8px 12px; margin-top: 12px; font-size: 13.5px; } .rdi-callout p + p { margin-top: 6px !important; } -.rdi-controls { display: flex; justify-content: space-between; align-items: center; gap: 8px; margin-top: 14px; } -.rdi-next { display: flex; flex-wrap: wrap; justify-content: flex-end; align-items: center; gap: 4px 10px; text-align: right; } +.rdi-controls { display: flex; flex-wrap: wrap; justify-content: space-between; align-items: center; gap: 8px; margin-top: 14px; } +.rdi-next { display: flex; flex-wrap: wrap; justify-content: flex-end; align-items: center; gap: 4px 10px; margin-left: auto; text-align: right; } /* stays on the right when the row wraps */ @media (prefers-reduced-motion: reduce) { .rdi-fresh { animation: none; } diff --git a/static/js/agent-memory-demo.js b/static/js/agent-memory-demo.js index 1d3d46dd09..7c655f09da 100644 --- a/static/js/agent-memory-demo.js +++ b/static/js/agent-memory-demo.js @@ -87,6 +87,9 @@ var NO_FACTS = 2; // the first-turn option with nothing lasting in it /* How the order comes out of the first two answers. */ + /* Restart's icon: a counterclockwise arrow. */ + var RESTART_ICON = ''; + function recipe(choices) { return { spice: choices[0] === 0 ? "extra spicy" : choices[0] === 1 ? "mild" : "", @@ -515,6 +518,15 @@ if (i) panels[t[0]].hidden = true; }); root.appendChild(tabs); + /* Restart, as an icon in the window's top-right corner, on every step. It's off until there's something to restart. */ + var restartBtn = el("button", "rcr-restart", RESTART_ICON); + restartBtn.type = "button"; + restartBtn.setAttribute("aria-label", "Restart the demo"); + restartBtn.title = "Restart the demo"; + restartBtn.addEventListener("click", function () { restart(); }); + root.classList.add("has-restart"); + root.appendChild(restartBtn); + function syncRestart() { restartBtn.disabled = st.turn === 0 && !st.extracted && st.lunch == null && st.tab === "chat"; } Object.keys(panels).forEach(function (k) { root.appendChild(panels[k]); }); /* Animations run as timed steps; the last step renders the final state. @@ -544,6 +556,7 @@ } function syncTabs() { root.setAttribute("data-busy", st.busy ? "true" : "false"); + syncRestart(); Array.prototype.forEach.call(tabs.children, function (b) { var why = lockReason(b.dataset.tab); b.classList.toggle("is-locked", !!why); @@ -657,18 +670,26 @@ } function controlsRow(primary) { var row = el("div", "ram-controls"); - var r = el("button", "rcr-btn", "Restart"); - r.type = "button"; - r.disabled = st.turn === 0 && !st.extracted && st.lunch == null && st.tab === "chat"; - r.addEventListener("click", restart); - row.appendChild(r); + syncRestart(); + var left = el("div", "rcr-navleft"), at = TABS.map(function (t) { return t[0]; }).indexOf(st.tab); + if (at > 0) { // back to the step before, like its tab; disabled while a step animates, like Next + var pv = el("button", "rcr-btn rcr-prev", "← Previous: " + esc(TABS[at - 1][2].charAt(0).toLowerCase() + TABS[at - 1][2].slice(1))); + pv.type = "button"; + pv.disabled = st.busy; + pv.addEventListener("click", function () { advance(TABS[at - 1][0]); }); + left.appendChild(pv); + } + row.appendChild(left); // kept even when empty, so the right-hand buttons stay on the right + /* On the right: the step's action or Next. */ + var right = el("div", "rcr-navright"); if (primary) { var b = el("button", "rcr-btn rcr-btn-primary", esc(st.busy && primary.busy ? primary.busy : primary.label)); b.type = "button"; b.disabled = st.busy; b.addEventListener("click", primary.fn); - row.appendChild(b); + right.appendChild(b); } + if (right.children.length) row.appendChild(right); return row; } function pre(obj) { return el("pre", "rcr-code rcr-code-sm", jsonHtml(obj)); } diff --git a/static/js/context-retriever-demo.js b/static/js/context-retriever-demo.js index f1e0c749c2..3e94e9fb36 100644 --- a/static/js/context-retriever-demo.js +++ b/static/js/context-retriever-demo.js @@ -561,11 +561,23 @@ if (name === "ask") renderAsk(); } - function next(label, target) { - var b = el("button", "rcr-btn rcr-btn-primary", esc(label)); - b.type = "button"; - b.addEventListener("click", function () { show(target); }); - var w = el("div", "rcr-next"); w.appendChild(b); return w; + /* The row at the end of a step: back to the step before, if there is one, and on to `target`. */ + function next(label, target, back) { + var w = el("div", "rcr-next"); + if (back) { + var t = TABS.filter(function (x) { return x[0] === back; })[0]; + var pb = el("button", "rcr-btn rcr-prev", "← Previous: " + esc(t[2].charAt(0).toLowerCase() + t[2].slice(1))); + pb.type = "button"; + pb.addEventListener("click", function () { show(back); }); + w.appendChild(pb); + } + if (label) { + var b = el("button", "rcr-btn rcr-btn-primary", esc(label)); + b.type = "button"; + b.addEventListener("click", function () { show(target); }); + w.appendChild(b); + } + return w; } /* ---- step 1: keys ---- */ @@ -633,7 +645,7 @@ idx.appendChild(el("p", "rcr-note", "Tag fields use the INDEXMISSING option, so the database must run Redis 7.4 or later. " + "Tag fields used for access control, which this demo doesn't include, also get INDEXEMPTY. The index names are illustrative.")); p.appendChild(idx); - p.appendChild(next("Next: ask questions over MCP", "ask")); + p.appendChild(next("Next: ask questions over MCP", "ask", "keys")); } /* Point an entity at the documents a template matches. When they're @@ -799,6 +811,7 @@ p.appendChild(chips); var chat = el("div", "rcr-chat"); chat.setAttribute("aria-live", "polite"); p.appendChild(chat); + p.appendChild(next(null, null, "model")); handshake(log); } diff --git a/static/js/langcache-demo.js b/static/js/langcache-demo.js index f9a8414887..06aa15e983 100644 --- a/static/js/langcache-demo.js +++ b/static/js/langcache-demo.js @@ -66,6 +66,9 @@ questions, which score just under the default threshold on purpose. */ var SAME_INTENT = { cancel: 0.91, fee: 0.94, late: 0.82 }; var CROSS_INTENT = { "cancel|fee": 0.44, "cancel|late": 0.37, "fee|late": 0.41 }; + /* Restart's icon: a counterclockwise arrow. */ + var RESTART_ICON = ''; + function pairKey(a, b) { return [a, b].sort().join("|"); } function jitter(a, b) { return (parseInt(hash8(pairKey(a, b)), 16) % 5 - 2) / 100; } function questionSim(a, b) { @@ -375,6 +378,15 @@ if (i) panels[t[0]].hidden = true; }); root.appendChild(tabs); + /* Restart, as an icon in the window's top-right corner, on every step. It's off until there's something to restart. */ + var restartBtn = el("button", "rcr-restart", RESTART_ICON); + restartBtn.type = "button"; + restartBtn.setAttribute("aria-label", "Restart the demo"); + restartBtn.title = "Restart the demo"; + restartBtn.addEventListener("click", function () { restart(); }); + root.classList.add("has-restart"); + root.appendChild(restartBtn); + function syncRestart() { restartBtn.disabled = !st.asked.length && st.tab === "questions"; } Object.keys(panels).forEach(function (k) { root.appendChild(panels[k]); }); /* Animations run as timed steps; the last step renders the final state. @@ -403,6 +415,7 @@ } function syncTabs() { root.setAttribute("data-busy", st.busy ? "true" : "false"); + syncRestart(); Array.prototype.forEach.call(tabs.children, function (b) { var why = lockReason(b.dataset.tab); b.classList.toggle("is-locked", !!why); @@ -475,18 +488,26 @@ } function controlsRow(primary) { var row = el("div", "rlc-controls"); - var r = el("button", "rcr-btn", "Restart"); - r.type = "button"; - r.disabled = !st.asked.length && st.tab === "questions"; - r.addEventListener("click", restart); - row.appendChild(r); + syncRestart(); + var left = el("div", "rcr-navleft"), at = TABS.map(function (t) { return t[0]; }).indexOf(st.tab); + if (at > 0) { // back to the step before, like its tab; disabled while a step animates, like Next + var pv = el("button", "rcr-btn rcr-prev", "← Previous: " + esc(TABS[at - 1][2].charAt(0).toLowerCase() + TABS[at - 1][2].slice(1))); + pv.type = "button"; + pv.disabled = st.busy; + pv.addEventListener("click", function () { advance(TABS[at - 1][0]); }); + left.appendChild(pv); + } + row.appendChild(left); // kept even when empty, so the right-hand buttons stay on the right + /* On the right: the step's action or Next. */ + var right = el("div", "rcr-navright"); if (primary) { var b = el("button", "rcr-btn rcr-btn-primary", esc(primary.label)); b.type = "button"; b.disabled = st.busy; b.addEventListener("click", primary.fn); - row.appendChild(b); + right.appendChild(b); } + if (right.children.length) row.appendChild(right); return row; } function pre(obj) { return el("pre", "rcr-code rcr-code-sm", jsonHtml(obj)); } diff --git a/static/js/rdi-demo.js b/static/js/rdi-demo.js index 4232406e9f..584d48f1d1 100644 --- a/static/js/rdi-demo.js +++ b/static/js/rdi-demo.js @@ -34,6 +34,9 @@ var SOURCE = "postgresql", DB = "fooddelivery", SCHEMA = "public", PIPELINE = "default"; var T0 = Date.UTC(2026, 8, 30, 9, 0, 0); // the pipeline is deployed at 09:00 UTC + /* Restart's icon: a counterclockwise arrow. */ + var RESTART_ICON = ''; + function ts(y, m, d, h) { return Math.floor(Date.UTC(y, m - 1, d, h == null ? 12 : h) / 1000); } function rowsOf(columns, list) { return list.map(function (r) { var o = {}; columns.forEach(function (c, i) { o[c[0]] = r[i]; }); return o; }); @@ -446,6 +449,15 @@ if (i) panels[t[0]].hidden = true; }); root.appendChild(tabs); + /* Restart, as an icon in the window's top-right corner, on every step. It's off until there's something to restart. */ + var restartBtn = el("button", "rcr-restart", RESTART_ICON); + restartBtn.type = "button"; + restartBtn.setAttribute("aria-label", "Restart the demo"); + restartBtn.title = "Restart the demo"; + restartBtn.addEventListener("click", function () { restart(); }); + root.classList.add("has-restart"); + root.appendChild(restartBtn); + function syncRestart() { restartBtn.disabled = !st.deployed && st.tab === "sync"; } Object.keys(panels).forEach(function (k) { root.appendChild(panels[k]); }); /* Animations replay records one at a time; the last step renders the @@ -502,6 +514,7 @@ } function syncTabs() { root.setAttribute("data-busy", st.busy ? "true" : "false"); + syncRestart(); Array.prototype.forEach.call(tabs.children, function (b) { var why = lockReason(b.dataset.tab); b.classList.toggle("is-locked", !!why); @@ -576,22 +589,28 @@ } function controlsRow(primary) { var row = el("div", "rdi-controls"); - var r = el("button", "rcr-btn", "Restart"); - r.type = "button"; - r.disabled = !st.deployed && st.tab === "sync"; - r.addEventListener("click", restart); - row.appendChild(r); + syncRestart(); + var left = el("div", "rcr-navleft"), at = TABS.map(function (t) { return t[0]; }).indexOf(st.tab); + if (at > 0) { // back to the step before, like its tab; disabled while a step animates, like Next + var pv = el("button", "rcr-btn rcr-prev", "← Previous: " + esc(TABS[at - 1][2].charAt(0).toLowerCase() + TABS[at - 1][2].slice(1))); + pv.type = "button"; + pv.disabled = st.busy; + pv.addEventListener("click", function () { advance(TABS[at - 1][0]); }); + left.appendChild(pv); + } + row.appendChild(left); // kept even when empty, so the right-hand buttons stay on the right + /* On the right: the step's action or Next. */ + var end = el("div", "rdi-next"); if (primary) { /* `wait` keeps the button in view but disabled, with a hint saying why. */ - var end = el("div", "rdi-next"); if (primary.wait) end.appendChild(el("span", "rcr-faint", esc(primary.wait))); var b = el("button", "rcr-btn rcr-btn-primary", esc(primary.label)); b.type = "button"; b.disabled = st.busy || !!primary.wait; b.addEventListener("click", primary.fn); end.appendChild(b); - row.appendChild(end); } + if (end.children.length) row.appendChild(end); return row; } function remember(d, key, dflt) { diff --git a/static/js/redisvl-demo.js b/static/js/redisvl-demo.js index 7bc2f00fff..abe493df5e 100644 --- a/static/js/redisvl-demo.js +++ b/static/js/redisvl-demo.js @@ -223,12 +223,21 @@ panel.appendChild(controls(q)); panel.appendChild(resultsCard(q, built)); panel.appendChild(codeCard(q, built)); - var i = STEPS.indexOf(step), nxt = STEPS[i + 1]; - if (nxt) { - var row = el("div", "rcr-next"), b = el("button", "rcr-btn rcr-btn-primary", "Next: " + esc(nxt.label) + " →"); - b.type = "button"; - b.addEventListener("click", function () { st.tab = nxt.key; render(); root.scrollIntoView({ block: "start", behavior: "smooth" }); }); - row.appendChild(b); + var i = STEPS.indexOf(step), nxt = STEPS[i + 1], prv = STEPS[i - 1]; + if (nxt || prv) { + var row = el("div", "rcr-next"), go = function (to) { st.tab = to.key; render(); root.scrollIntoView({ block: "start", behavior: "smooth" }); }; + if (prv) { + var pb = el("button", "rcr-btn rcr-prev", "← Previous: " + esc(prv.label)); + pb.type = "button"; + pb.addEventListener("click", function () { go(prv); }); + row.appendChild(pb); + } + if (nxt) { + var b = el("button", "rcr-btn rcr-btn-primary", "Next: " + esc(nxt.label) + " →"); + b.type = "button"; + b.addEventListener("click", function () { go(nxt); }); + row.appendChild(b); + } panel.appendChild(row); } } diff --git a/static/js/search-demo.js b/static/js/search-demo.js index 940a03b36e..f0539f0316 100644 --- a/static/js/search-demo.js +++ b/static/js/search-demo.js @@ -239,12 +239,21 @@ panel.appendChild(dataCard(cur)); if (st.tab === "agg") panel.appendChild(groupsBlock(cur)); // the groups, as the reply lists them if (cur) panel.appendChild(outputBlock(cur)); // the query's reply, exactly as redis-cli prints it - var i = STEPS.indexOf(step), nxt = STEPS[i + 1]; - if (nxt) { - var row = el("div", "rcr-next"), b = el("button", "rcr-btn rcr-btn-primary", "Next: " + esc(nxt.label) + " →"); - b.type = "button"; - b.addEventListener("click", function () { st.tab = nxt.key; st.dataKey = null; render(); root.scrollIntoView({ block: "start", behavior: "smooth" }); }); - row.appendChild(b); + var i = STEPS.indexOf(step), nxt = STEPS[i + 1], prv = STEPS[i - 1]; + if (nxt || prv) { + var row = el("div", "rcr-next"), go = function (to) { st.tab = to.key; st.dataKey = null; render(); root.scrollIntoView({ block: "start", behavior: "smooth" }); }; + if (prv) { + var pb = el("button", "rcr-btn rcr-prev", "← Previous: " + esc(prv.label)); + pb.type = "button"; + pb.addEventListener("click", function () { go(prv); }); + row.appendChild(pb); + } + if (nxt) { + var b = el("button", "rcr-btn rcr-btn-primary", "Next: " + esc(nxt.label) + " →"); + b.type = "button"; + b.addEventListener("click", function () { go(nxt); }); + row.appendChild(b); + } panel.appendChild(row); } } diff --git a/static/js/search-index-demo.js b/static/js/search-index-demo.js index 132524bfa5..ef66dd3cd3 100644 --- a/static/js/search-index-demo.js +++ b/static/js/search-index-demo.js @@ -284,12 +284,21 @@ if (cur) panel.appendChild(queryBlock(cur)); // the query, before the data it finds panel.appendChild(st.tab === "data" ? introCard() : dataCard(cur)); if (cur) panel.appendChild(outputBlock(cur)); // the query's reply, exactly as redis-cli prints it - var i = STEPS.indexOf(step), nxt = STEPS[i + 1]; - if (nxt) { - var row = el("div", "rcr-next"), b = el("button", "rcr-btn rcr-btn-primary", "Next: " + esc(nxt.label) + " →"); - b.type = "button"; - b.addEventListener("click", function () { st.tab = nxt.key; st.dataKey = null; render(); root.scrollIntoView({ block: "start", behavior: "smooth" }); }); - row.appendChild(b); + var i = STEPS.indexOf(step), nxt = STEPS[i + 1], prv = STEPS[i - 1]; + if (nxt || prv) { + var row = el("div", "rcr-next"), go = function (to) { st.tab = to.key; st.dataKey = null; render(); root.scrollIntoView({ block: "start", behavior: "smooth" }); }; + if (prv) { + var pb = el("button", "rcr-btn rcr-prev", "← Previous: " + esc(prv.label)); + pb.type = "button"; + pb.addEventListener("click", function () { go(prv); }); + row.appendChild(pb); + } + if (nxt) { + var b = el("button", "rcr-btn rcr-btn-primary", "Next: " + esc(nxt.label) + " →"); + b.type = "button"; + b.addEventListener("click", function () { go(nxt); }); + row.appendChild(b); + } panel.appendChild(row); } } From fb9d04e5e03911c5bba86c823308ea729bc33622 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Tue, 20 Oct 2026 06:23:58 -0700 Subject: [PATCH 47/76] Update content/operate/rc/api/api-reference/openapi.json (#4302) Co-authored-by: redisdocsapp[bot] <177626021+redisdocsapp[bot]@users.noreply.github.com> --- .../operate/rc/api/api-reference/openapi.json | 50 +++++++++---------- 1 file changed, 25 insertions(+), 25 deletions(-) diff --git a/content/operate/rc/api/api-reference/openapi.json b/content/operate/rc/api/api-reference/openapi.json index 9120d4ffe6..50d388708a 100644 --- a/content/operate/rc/api/api-reference/openapi.json +++ b/content/operate/rc/api/api-reference/openapi.json @@ -96,7 +96,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_3", + "operationId": "proxyDataIntegrationWorkspace", "parameters": [ { "name": "subscriptionId", @@ -166,7 +166,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace", + "operationId": "proxyDataIntegrationWorkspace_2", "parameters": [ { "name": "subscriptionId", @@ -308,7 +308,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_4", + "operationId": "proxyDataIntegrationWorkspace_3", "parameters": [ { "name": "subscriptionId", @@ -379,7 +379,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_2", + "operationId": "proxyDataIntegrationWorkspace_4", "parameters": [ { "name": "subscriptionId", @@ -452,7 +452,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_8", + "operationId": "proxyDataIntegrationWorkspace_5", "parameters": [ { "name": "subscriptionId", @@ -522,7 +522,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_5", + "operationId": "proxyDataIntegrationWorkspace_7", "parameters": [ { "name": "subscriptionId", @@ -664,7 +664,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_9", + "operationId": "proxyDataIntegrationWorkspace_8", "parameters": [ { "name": "subscriptionId", @@ -735,7 +735,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_7", + "operationId": "proxyDataIntegrationWorkspace_9", "parameters": [ { "name": "subscriptionId", @@ -8353,9 +8353,6 @@ "401": { "description": "Unauthorized - Authentication failed for requested resource" }, - "404": { - "description": "Not Found - The resource you were trying to reach was not found or does not exist" - }, "403": { "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" }, @@ -8369,6 +8366,9 @@ } } }, + "404": { + "description": "Not Found - The resource you were trying to reach was not found or does not exist" + }, "200": { "description": "Memory stores returned.", "content": { @@ -8493,9 +8493,6 @@ "401": { "description": "Unauthorized - Authentication failed for requested resource" }, - "404": { - "description": "Not Found - The resource you were trying to reach was not found or does not exist" - }, "403": { "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" }, @@ -8509,6 +8506,9 @@ } } }, + "404": { + "description": "Not Found - The resource you were trying to reach was not found or does not exist" + }, "408": { "description": "Request Timeout - The server didn't receive a complete request message within the server's allotted timeout period" }, @@ -8568,8 +8568,8 @@ "401": { "description": "Unauthorized - Authentication failed for requested resource" }, - "404": { - "description": "Not Found - The resource you were trying to reach was not found or does not exist" + "403": { + "description": "Forbidden - Not allowed to create requested resource (primarily due to not active resource)" }, "201": { "description": "API key created. The generated key value is returned once in the response body.", @@ -8581,9 +8581,6 @@ } } }, - "403": { - "description": "Forbidden - Not allowed to create requested resource (primarily due to not active resource)" - }, "424": { "description": "Failed Dependency - The memory store's database is unavailable or out of memory, e.g. /errors/resource-unavailable or /errors/database-out-of-memory. When the error originates from the Agent Memory service the body is an application/problem+json document (MemoryProblemDetails) carrying the service's error type and detail", "content": { @@ -8594,6 +8591,9 @@ } } }, + "404": { + "description": "Not Found - The resource you were trying to reach was not found or does not exist" + }, "408": { "description": "Request Timeout - The server didn't receive a complete request message within the server's allotted timeout period" }, @@ -10082,9 +10082,6 @@ "401": { "description": "Unauthorized - Authentication failed for requested resource" }, - "404": { - "description": "Not Found - The resource you were trying to reach was not found or does not exist" - }, "403": { "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" }, @@ -10108,6 +10105,9 @@ } } }, + "404": { + "description": "Not Found - The resource you were trying to reach was not found or does not exist" + }, "429": { "description": "Too Many Requests - Too many resources are concurrently created / updated / deleted in the account. Please re-submit API requests after resource changes are completed)" }, @@ -12810,9 +12810,6 @@ "401": { "description": "Unauthorized - Authentication failed for requested resource" }, - "404": { - "description": "Not Found - The resource you were trying to reach was not found or does not exist" - }, "403": { "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" }, @@ -12826,6 +12823,9 @@ } } }, + "404": { + "description": "Not Found - The resource you were trying to reach was not found or does not exist" + }, "408": { "description": "Request Timeout - The server didn't receive a complete request message within the server's allotted timeout period" }, From 84a13bc2523e02769da18a9743053108ea78a8e1 Mon Sep 17 00:00:00 2001 From: Orhan Abdula Date: Tue, 20 Oct 2026 17:26:15 +0300 Subject: [PATCH 48/76] Expand Cloud RDI database throughput guidance with memory and shard sizing --- content/operate/rc/rdi/scale-pipeline.md | 54 +++++++++++++++++++++--- 1 file changed, 47 insertions(+), 7 deletions(-) diff --git a/content/operate/rc/rdi/scale-pipeline.md b/content/operate/rc/rdi/scale-pipeline.md index 249edefeac..0974e2d349 100644 --- a/content/operate/rc/rdi/scale-pipeline.md +++ b/content/operate/rc/rdi/scale-pipeline.md @@ -132,18 +132,58 @@ records, and processor load. ## Increase RDI database throughput -Each pipeline has an RDI database. When database throughput limits ingestion -or processing, choose a higher throughput value from its **Performance** +Each pipeline has an RDI database. When the RDI database limits ingestion or +processing, increase its memory limit or throughput in its **Performance** settings. +### Relevant metrics + +To view the RDI database metrics, open the RDI database in the Cloud console +and select the **Metrics** tab. Check these metrics: + +- **Used memory**: The RDI database holds records until the processor writes + them to the target database. Large rows and high data volumes can fill the + RDI database faster than the processor drains it. When the RDI database + reaches its memory limit, it rejects writes with OOM errors and the + collectors wait. +- **Ops/sec**: If **Ops/sec** stays close to the RDI database's throughput + value, or **Latency** keeps rising, the RDI database cannot handle more + writes. + +### Single pipeline + +For a single pipeline, check **Used memory** and **Ops/sec**: + +- If **Used memory** reaches the memory limit, increase **Memory limit**. +- If **Ops/sec** stays close to the throughput value, increase **Throughput**. + +### Multiple pipelines + +For multiple pipelines, check **Used memory** and **Ops/sec** the same way as +for a single pipeline. + +When you increase **Throughput**, choose a value with as many shards as you +have pipelines. Each pipeline writes to one shard, even if it has several +sources. When the number of shards matches the number of pipelines, each +pipeline writes to its own shard. + +| Throughput | Nodes | Shards | +| --- | ---: | ---: | +| 25,000 ops/sec (default) | 1 | 1 | +| 50,000 ops/sec | 2 | 2 | +| 75,000 ops/sec | 3 | 3 | + +### Edit the performance settings + 1. In the Cloud console, open the RDI database for the pipeline. 1. Edit the database **Performance** settings. -1. Increase **Throughput** to the next available value. -1. Check the pipeline Dashboard before making another increase. +1. Increase **Memory limit** or **Throughput** to the next available value. +1. Check the RDI database **Metrics** tab and the pipeline Dashboard before + you make another increase. -Stop increasing throughput when the pipeline no longer improves. Check -processor and target performance too. More database throughput does not -resolve a bottleneck in another component. +Stop increasing the values when the pipeline no longer improves. More RDI +database throughput does not resolve a bottleneck in another component, so +check processor and target database performance too. ## Increase processor capacity From e06333bb1093785679eaa5d7a80296031e83a5ce Mon Sep 17 00:00:00 2001 From: Kaitlyn Michael <76962844+kaitlynmichael@users.noreply.github.com> Date: Tue, 20 Oct 2026 11:01:25 -0500 Subject: [PATCH 49/76] RS: 8.2.0-125 RN (#4310) * RS: 8.2.0 patch 6 RN * delete todo * RS: 8.2.0-125 maintenance release notes --- .../rs-8-2-releases/rs-8-2-0-125.md | 779 ++++++++++++++++++ 1 file changed, 779 insertions(+) create mode 100644 content/operate/rs/release-notes/rs-8-2-releases/rs-8-2-0-125.md diff --git a/content/operate/rs/release-notes/rs-8-2-releases/rs-8-2-0-125.md b/content/operate/rs/release-notes/rs-8-2-releases/rs-8-2-0-125.md new file mode 100644 index 0000000000..caf5ca023f --- /dev/null +++ b/content/operate/rs/release-notes/rs-8-2-releases/rs-8-2-0-125.md @@ -0,0 +1,779 @@ +--- +Title: Redis Software release notes 8.2.0-125 (October 2026) +alwaysopen: false +categories: +- docs +- operate +- rs +compatibleOSSVersion: Redis 8.6.2, 8.4.0, 8.2.1, 8.0.2, 7.4.3, 7.2.7, 6.2.13 +description: Bug fixes for upgrades from Redis Software 7.2.x and 7.4.x, cluster management after a primary node change, node removal during upgrades, Active-Active proxy recovery, DMC proxy stability, scaling and resharding, ACL validation, cluster recovery, Google Cloud Storage backup configuration, and signing in with passwords that contain a colon. Internal fixes and improvements. +linkTitle: 8.2.0-125 (October 2026) +weight: 77 +--- + +This is a maintenance release for ​[​Redis Software version 8.2.0](https://redis.io/downloads/#Redis_Software). + +## Highlights + +This version offers: + +- Bug fix for upgrades from Redis Software 7.2.x and 7.4.x + +- Bug fixes for cluster management after a primary node change + +- Bug fix for node removal during rolling upgrades + +- Bug fixes for DMC proxy stability and Active-Active proxy recovery + +- Bug fixes for scaling, resharding, and cluster recovery + +- Bug fix for ACL rule validation + +- Bug fix for Google Cloud Storage backup configuration + +- Bug fix for signing in with passwords that contain a colon + +- Internal fixes and improvements + +## New in this release + +### Redis database versions and feature sets + +Redis Software version 8.2.0 includes the following Redis database versions: 8.6.2, 8.4.0, 8.2.1, 8.0.2, 7.4.3, 7.2.7, and 6.2.13. + +The [default Redis database version](/content/operate/rs/databases/configure/db-defaults.md#database-version) is 8.6. + +Redis Software includes multiple feature sets, compatible with different Redis database versions. + +{{< collapsible "Show feature set details" >}} + +The following table shows which Redis modules are compatible with each Redis database version included in this release. + +| Redis database version | Compatible Redis modules | +|------------------------|--------------------------| +| 8.6 | RediSearch 8.6
    RedisJSON 8.6
    RedisTimeSeries 8.6
    RedisBloom 8.6
    See [What's new in Redis 8.6](/content/develop/whats-new/8-6.md) and [Redis Open Source 8.6 release notes](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisce/redisos-8.6-release-notes.md) | +| 8.4 | RediSearch 8.4
    RedisJSON 8.4
    RedisTimeSeries 8.4
    RedisBloom 8.4
    See [What's new in Redis 8.4](/content/develop/whats-new/8-4.md) and [Redis Open Source 8.4 release notes](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisce/redisos-8.4-release-notes.md) | +| 8.2 | RediSearch 8.2
    RedisJSON 8.2
    RedisTimeSeries 8.2
    RedisBloom 8.2
    See [What's new in Redis 8.2](/content/develop/whats-new/8-2.md) and [Redis Open Source 8.2 release notes](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisce/redisos-8.2-release-notes.md) | +| 8.0 | RediSearch 8.0
    RedisJSON 8.0
    RedisTimeSeries 8.0
    RedisBloom 8.0
    See [What's new in Redis 8.0](/content/develop/whats-new/8-0.md) and [Redis Open Source 8.0 release notes](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisce/redisos-8.0-release-notes.md) | +| 7.4 | [RediSearch 2.10](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisearch/redisearch-2.10-release-notes.md)
    [RedisJSON 2.8](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisjson/redisjson-2.8-release-notes.md)
    [RedisTimeSeries 1.12](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redistimeseries/redistimeseries-1.12-release-notes.md)
    [RedisBloom 2.8](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisbloom/redisbloom-2.8-release-notes.md) | +| 7.2 | [RediSearch 2.8](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisearch/redisearch-2.8-release-notes.md)
    [RedisJSON 2.6](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisjson/redisjson-2.6-release-notes.md)
    [RedisTimeSeries 1.10](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redistimeseries/redistimeseries-1.10-release-notes.md)
    [RedisBloom 2.6](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisbloom/redisbloom-2.6-release-notes.md) | +| 6.2 | [RediSearch 2.6](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisearch/redisearch-2.6-release-notes.md)
    [RedisJSON 2.4](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisjson/redisjson-2.4-release-notes.md)
    [RedisTimeSeries 1.8](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redistimeseries/redistimeseries-1.8-release-notes.md)
    [RedisBloom 2.4](/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisbloom/redisbloom-2.4-release-notes.md) | + +{{< /collapsible >}} + +### Resolved issues + +- RS196137: Fixed an issue where the Cluster Manager did not send the required `private_key` field when configuring Google Cloud Storage (GCS) backups. + +- RS201541: Fixed an issue where removing a node could get stuck in a decommissioning state during a rolling upgrade of a cluster with AOF persistence on RHEL and Amazon Linux. + +- RS202231: Fixed an issue where the DMC proxy could crash while resharding a database during a scale-out. + +- RS202483: Fixed an issue where ACL rule validation did not apply the cluster's `acl_pubsub_default` setting, which caused validation to reject valid ACL rules or accept invalid rules that then left the database change pending. + +- RS209436: Fixed an issue where a proxy that started during a database failover did not resume forwarding traffic and closed client connections while the database appeared active. + +- RS210971: Fixed an issue where endpoint request metrics did not count requests to Memcached databases. + +- RS211091: Fixed an issue where the cluster could become unresponsive after a newly joined node became the primary node while scaling the cluster. + +- RS212081: Fixed an issue where changing the primary node could time out and leave the cluster without a working primary node. + +- RS212205: Fixed an issue where users with a password that contains a colon (`:`) could not sign in to the Cluster Manager UI or authenticate REST API requests. + +- RS212633: Fixed an issue where recovering a cluster after a full cluster restart could use outdated persistence files, which prevented the database from being recovered. + +- RS213016: Fixed an issue where a node that was frozen and then resumed could stop responding to cluster management requests, which caused cluster health checks to fail and left the database in an `active-change-pending` state. + +- RS213801: Fixed an issue where a shard could remain in the wrong role after resharding or scaling. + +- RS215390: Fixed an issue where a database with the `all-nodes` proxy policy could stay in an `active-change-pending` state after a node was added to the cluster. + +- RS215608: Fixed an issue where running `dmc-cli list bdb` or `dmc-cli list server` could corrupt memory and cause the DMC proxy to crash. + +- RS215676: Fixed an issue where adding a participating cluster to an Active-Active database could cause the new participating cluster's shards to crash repeatedly during the full sync, which also disrupted synchronization between the other participating clusters. + +- RS216045: Fixed an issue where the DMC proxy could crash repeatedly when a client sent a command without key arguments, such as `PUBSUB SHARDNUMSUB` with no channels, in a pipeline with another command. + +- RS219531: Fixed an issue where upgrading a cluster from Redis Software version 7.2.x or 7.4.x to version 8.2.0 could fail because a node that still ran the earlier version lost its replication link to the cluster configuration store. + +## Version changes + +### JWT token invalidation on upgrade + +To improve system security, the JWT signing secret has changed in version 8.0.22 and later. As a result, upgrading invalidates all previously issued JWT tokens, which must be regenerated. Tokens used programmatically within the cluster are regenerated automatically. Cluster Manager users must sign out and sign in again, or wait for their session or token to expire. + +### Reserved ports + +Make sure the following ports are open before upgrading Redis Software. + +The following port was added as a reserved port in Redis Software version 8.0.18; however, it is optional instead of reserved in Redis Software version 8.0.20-44: + +| Port | Process name | Usage | +|------|--------------|-------| +| 3357 | reconciliation_tree_grpc | Internal communication | + +Ports reserved as of Redis Software version 7.22.0: + +| Port | Process name | Usage | +|------|--------------|-------| +| 3346 | cluster_api_internal | Cluster API internal port | +| 3351 | cluster_watchdog_grpc_api | Cluster watchdog now supports gRPC | +| 3352 | grpc_service_mesh | gRPC communication between nodes | +| 3353 | local_grpc_service_mesh | Local gRPC services | +| 3354 | grpc_gossip_envoy | gRPC gossip protocol communication between nodes | +| 3355 | authentication_service | Authentication service internal port | + +Ports reserved as of Redis Software version 7.8.2: + +| Port | Process name | Usage | +|------|--------------|-------| +| 3347 | cert_exporter | Reports cluster certificate metrics | +| 3348 | process_exporter | Reports process metrics for DMC and Redis processes | +| 3349 | cluster_wd_exporter | Reports cluster watchdog metrics | +| 3350 | db_controller | Internode communication | +| 9091 | node_exporter | Reports host node metrics related to CPU, memory, disk, and more | +| 9125 | statsd_exporter | Reports push metrics related to the DMC and syncer, and some cluster and node metrics | + +See [Ports and port ranges used by Redis Software](/content/operate/rs/networking/port-configurations.md#ports-and-port-ranges-used-by-redis-software) for a complete list. + +### Supported platforms + +The following table provides a snapshot of supported platforms as of this Redis Software release. See the [supported platforms reference](/content/operate/rs/references/supported-platforms.md) for more details about operating system compatibility. + +**Amazon Linux 2 is no longer supported.** Starting with Redis Software version 8.0.22, Redis Software no longer builds or publishes Amazon Linux 2 (`amzn2`) packages. Use Amazon Linux 2023 instead. + +{{< collapsible "Show supported platforms" >}} + +✅ Supported – The platform is supported for this version of Redis Software and Redis Stack modules. + +:warning: Deprecation warning – The platform is still supported for this version of Redis Software, but support will be removed in a future release. + +| Redis Software
    major versions | 8.2 | 8.0 | 7.22 | 7.8 | 7.4 | 7.2 | 6.4 | 6.2 | +|---------------------------------|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:| +| **Release date** | Jul 2026 | Oct 2025 | May 2025 | Nov 2024 | Feb 2024 | Aug 2023 | Feb 2023 | Aug 2021 | +| [**End-of-life date**](/content/operate/rs/installing-upgrading/product-lifecycle.md#endoflife-schedule) | Determined after
    next major release | Jul 2028 | Oct 2027 | May 2027 | Nov 2026 | Feb 2026 | Aug 2025 | Feb 2025 | +| **Platforms** | | | | | | | | | +| RHEL 9 &
    compatible distros[1](#table-note-1) | ✅ | ✅ | ✅ | ✅ | ✅ | – | – | – | +| RHEL 9
    FIPS mode[5](#table-note-5) | ✅ | ✅ | ✅ | ✅ | – | – | – | – | +| RHEL 8 &
    compatible distros[1](#table-note-1) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| RHEL 7 &
    compatible distros[1](#table-note-1) | – | – | – | – | – | :warning: | :warning: | :warning: | +| Ubuntu 22.04[2](#table-note-2) | ✅ | ✅ | ✅ | ✅ | – | – | – | – | +| Ubuntu 20.04[2](#table-note-2) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | – | +| Ubuntu 18.04[2](#table-note-2) | – | – | – | – | :warning: | :warning: | :warning: | :warning: | +| Ubuntu 16.04[2](#table-note-2) | – | – | – | – | – | :warning: | :warning: | :warning: | +| Amazon Linux 2023[6](#table-note-6) | ✅ | ✅ | – | – | – | – | – | – | +| Amazon Linux 2 | – | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | – | +| Amazon Linux 1 | – | – | – | – | – | :warning: | :warning: | :warning: | +| Kubernetes[3](#table-note-3) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| Docker[4](#table-note-4) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | + +1. The RHEL-compatible distributions CentOS, CentOS Stream, Alma, and Rocky are supported if they have full RHEL compatibility. Oracle Linux running the Red Hat Compatible Kernel (RHCK) is supported, but the Unbreakable Enterprise Kernel (UEK) is not supported. + +2. The server version of Ubuntu is recommended for production installations. The desktop version is only recommended for development deployments. + +3. See the [Redis Enterprise for Kubernetes documentation](/content/operate/kubernetes/reference/supported_k8s_distributions.md) for details about support per version and Kubernetes distribution. + +4. [Docker images](/content/operate/rs/installing-upgrading/quickstarts/docker-quickstart.md) of Redis Software are certified for development and testing only. + +5. Supported only if [FIPS was enabled during RHEL installation](https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/9/html/security_hardening/switching-rhel-to-fips-mode_security-hardening#proc_installing-the-system-with-fips-mode-enabled_switching-rhel-to-fips-mode) to ensure FIPS compliance. + +6. Amazon Linux 2023 support was added in Redis Software version 8.0.20. + +{{< /collapsible >}} + +## Downloads + +The following table shows the SHA256 checksums for the available packages: + +| Package | SHA256 checksum (8.2.0-125 October release) | +|---------|---------------------------------------| +| Ubuntu 20 | edeb44f10ac5db8648c3a1ecd4266ec4a7e73035ea2e6415468ee6596c22da00 | +| Ubuntu 22 (amd64) | 841feb35520a76b3ee51d7635f8fd2ed511b45c43a72d212c57af10394241527 | +| Ubuntu 22 (arm64) | 9c527e55411fafca18353db49613381c669a1189e784e87010d92b8d2556bf17 | +| Red Hat Enterprise Linux (RHEL) 8 | c23106b4801c07e18adc660317b6b93aa7161f5ffb4b15f74893db91c761ffe6 | +| Red Hat Enterprise Linux (RHEL) 9 (amd64) | 8bbd7fd39ac073a7b52e1545800d66c3995719b4766a58f762b5bc66c59ab5f0 | +| Red Hat Enterprise Linux (RHEL) 9 (arm64) | d908c2f20579e4e6c8e1d41c91cfb5bc48403306f4741d46f81b5afa784dcb6e | +| Red Hat Enterprise Linux (RHEL) 9 FIPS (amd64) | 896bb24fd57bfd3aaa269ebb9a31e42337c57f3afb2cae9ee45339e21b5ecfb2 | +| Amazon Linux 2023 (amd64) | 06ac73b6788a38f1144288db2bccab9656deaad9523de1e0b508eb26c1129e21 | +| Amazon Linux 2023 (arm64) | b801de92c1c1633bacd2c2ac06ffbc8586de24fe689486b8c2a04a0084e6a362 | + +## Known issues + +- RS155734: Endpoint availability metrics do not work as expected due to a calculation error. + +- For databases that use RedisTimeSeries with Redis database version 8.2, 8.4, or 8.6, after a shard restarts and recovers from its append-only file (AOF), requests routed to that shard can receive replies that belong to other commands or no reply at all. The issue does not resolve on its own. + +## Known limitations + +#### Increased memory usage in FIPS mode + +FIPS-enabled builds use more memory than non-FIPS builds. Each Go-based control-plane service uses roughly 34 MB more when running in FIPS mode, which adds up to approximately 210 MB of additional memory per node. Account for this overhead when sizing FIPS-enabled deployments. This limitation affects FIPS-enabled builds only. + +#### SFTP storage not supported in FIPS mode + +FIPS-enabled builds do not support SFTP as a storage location for backups, imports, or exports, because the SFTP library is not FIPS-compliant. In FIPS mode, use a different storage location. This limitation affects FIPS-enabled builds only. + +#### Redis Search query failures during rolling upgrades across the 8.4 version boundary + +Redis Search queries can fail during a rolling upgrade when a cluster contains shards running Redis versions earlier than 8.4 and shards running Redis version 8.4 or later, due to an internal protocol change introduced in version 8.4. This issue affects only clusters where `parallel_shards_upgrade` has been changed from its default value of `0`. If both conditions apply, expect Redis Search downtime until all nodes are upgraded. + +#### Trim ACKED not supported for Active-Active 8.4 databases + +For Active-Active databases running Redis database version 8.4, the `ACKED` option is not supported for trimming commands. + +#### Rolling upgrade limitation for clusters with custom or deprecated modules + +Due to module handling changes introduced in Redis Software version 8.0, upgrading a cluster that contains custom or deprecated modules, such as RedisGraph and RedisGears v2, can become stuck when adding a new node to the cluster during a rolling upgrade. + +#### Cluster Manager sign-outs and false "down" status during an Active-Active upgrade + +While an Active-Active database has participating clusters running different Redis Software versions, the Cluster Manager UI on an upgraded cluster can sign users out intermittently and show a participating cluster as down, even though replication is active and the syncers are connected. + +This affects the Cluster Manager UI only. It's caused by a JWT signing-key mismatch between nodes running different versions during the upgrade window (see [JWT token invalidation on upgrade](#jwt-token-invalidation-on-upgrade)). The data path and replication are not affected. + +To resolve it, upgrade the remaining participating clusters. The issue clears after every participating cluster runs the same version. If you're signed out, sign in again. + +#### Module commands limitation during Active-Active database upgrades to Redis 8.0 + +When upgrading an Active-Active database to Redis version 8.0, you cannot use module commands until all Active-Active database instances have been upgraded. Currently, these commands are not blocked automatically. + +#### Redis 8.0 database cannot be created with flash + +You cannot create a Redis 8.0 database with flash storage enabled. Create a Redis 8.0 database with RAM-only storage instead, or use Redis 8.2 for flash-enabled (Redis Flex) databases. + +#### Cluster Manager UI limitations + +The following legacy UI features are not yet available in the new Cluster Manager UI: + +- Purge an Active-Active instance. + + Use [`crdb-cli crdb purge-instance`](/content/operate/rs/references/cli-utilities/crdb-cli/crdb/purge-instance.md) instead. + +## Security + +#### Redis Open Source security fixes compatibility + +As part of Redis's commitment to security, Redis Software implements the latest [security fixes](https://github.com/redis/redis/releases) available with [Redis Open Source](https://github.com/redis/redis). Redis Software has already included the fixes for the relevant CVEs. + +Some CVEs announced for Redis Open Source do not affect Redis Software due to different or additional functionality available in Redis Software that is not available in Redis Open Source. + +Redis Software 8.2.0-125 supports Redis Open Source 8.6, 8.4, 8.2, 8.0, 7.4, 7.2, and 6.2. Below is the list of Redis Open Source CVEs and other security vulnerabilities fixed by version. + +{{< collapsible "Redis 8.6.x" >}} + +- Memory corruption in `SLOT_INFO` handling may allow an unauthenticated user to potentially run their own code. + +- A use-after-free in TLS pending-data processing may allow an authenticated user to cause a denial-of-service. + +- A vulnerability in the `RESTORE` command and in RDB loading allows an authenticated user to trigger a use-after-free in stream consumer groups via a specially crafted serialized payload, potentially resulting in remote code execution. + +- An ACL key-pattern bypass in `GEORADIUS` and `GEORADIUSBYMEMBER` may allow an authenticated user to write to keys outside their permitted key patterns. + +- An out-of-bounds read during command key extraction may cause the server to crash. + +- VSET: Missing `level` cap when loading a serialized HNSW graph may lead to resource exhaustion. + +- VSET: Missing validation of the parameter count when loading a serialized vector set may lead to oversized allocation and resource exhaustion. + +- Certificate common-name handling can allow an authentication bypass when `tls-auth-clients-user` is set to `CN`. + +- (CVE-2026-62356) An authenticated user may trigger a heap out-of-bounds write in RedisBloom by loading a specially crafted Count-Min Sketch (`CMSketch`) object, for example via the `RESTORE` command or a crafted RDB file. + +- RedisBloom: Out-of-bounds access in the `TopK` heap cleanup path. + +- VSET: Large `VADD REDUCE` values can lead to out-of-memory crashes or buffer overflow. + +- (CVE-2026-23479) When a blocked client is evicted while re-executing a blocked command, an authenticated user may trigger a use-after-free and potentially lead to remote code execution. + +- (CVE‑2026‑25243) A vulnerability in the Redis `RESTORE` command allows an authenticated user to trigger an invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE-2026-25588) A vulnerability in the `RESTORE` command, when used with the RedisTimeSeries module, allows an authenticated attacker to trigger invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE‑2026‑25589) A vulnerability in the `RESTORE` command, when used with the RedisBloom module, allows an authenticated attacker to trigger invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE-2026-23631) An authenticated user may exploit the synchronization mechanism of the master-replica and trigger a use-after-free vulnerability, potentially leading to remote code execution. The bug affects only replicas that are configured, or may be configured with `replica-read-only` disabled, and exists in all versions of Redis with Lua scripting. + +- Insufficient sanitization of Lua script error messages can allow an authenticated user with `EVAL` permission to inject arbitrary RESP protocol responses, leading to denial of service, data manipulation, and connection pool poisoning. + +- RedisBloom: Integer overflow in `TopK` RDB loading may lead to heap out-of-bounds access. + +- RedisBloom: Missing validation in `td_histogram_t` RDB loading may lead to heap buffer overflow. + +- RedisBloom: Missing validation in `CuckooFilter` RDB loading may lead to heap out-of-bounds access. + +- RedisBloom: Missing validation in `SBChain` and `CMSketch` RDB loading may lead to heap out-of-bounds write. + +- Added validation of limits during RDB load and data creation to avoid resource exhaustion and Denial-of-Service (DoS) vulnerabilities. + +{{< /collapsible >}} + +{{< collapsible "Redis 8.4.x" >}} + +- Memory corruption in `SLOT_INFO` handling may allow an unauthenticated user to potentially run their own code. + +- A use-after-free in TLS pending-data processing may allow an authenticated user to cause a denial-of-service. + +- A vulnerability in the `RESTORE` command and in RDB loading allows an authenticated user to trigger a use-after-free in stream consumer groups via a specially crafted serialized payload, potentially resulting in remote code execution. + +- An ACL key-pattern bypass in `GEORADIUS` and `GEORADIUSBYMEMBER` may allow an authenticated user to write to keys outside their permitted key patterns. + +- An out-of-bounds read during command key extraction may cause the server to crash. + +- VSET: Missing `level` cap when loading a serialized HNSW graph may lead to resource exhaustion. + +- VSET: Missing validation of the parameter count when loading a serialized vector set may lead to oversized allocation and resource exhaustion. + +- (CVE-2026-62356) An authenticated user may trigger a heap out-of-bounds write in RedisBloom by loading a specially crafted Count-Min Sketch (`CMSketch`) object, for example via the `RESTORE` command or a crafted RDB file. + +- RedisBloom: Out-of-bounds access in the `TopK` heap cleanup path. + +- VSET: Large `VADD REDUCE` values can lead to out-of-memory crashes or buffer overflow. + +- (CVE-2026-23479) When a blocked client is evicted while re-executing a blocked command, an authenticated user may trigger a use-after-free and potentially lead to remote code execution. + +- (CVE‑2026‑25243) A vulnerability in the Redis `RESTORE` command allows an authenticated user to trigger an invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE-2026-25588) A vulnerability in the `RESTORE` command, when used with the RedisTimeSeries module, allows an authenticated attacker to trigger invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE‑2026‑25589) A vulnerability in the `RESTORE` command, when used with the RedisBloom module, allows an authenticated attacker to trigger invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE-2026-23631) An authenticated user may exploit the synchronization mechanism of the master-replica and trigger a use-after-free vulnerability, potentially leading to remote code execution. The bug affects only replicas that are configured, or may be configured with `replica-read-only` disabled, and exists in all versions of Redis with Lua scripting. + +- Insufficient sanitization of Lua script error messages can allow an authenticated user with `EVAL` permission to inject arbitrary RESP protocol responses, leading to denial of service, data manipulation, and connection pool poisoning. + +- RedisBloom: Integer overflow in `TopK` RDB loading may lead to heap out-of-bounds access. + +- RedisBloom: Missing validation in `td_histogram_t` RDB loading may lead to heap buffer overflow. + +- RedisBloom: Missing validation in `CuckooFilter` RDB loading may lead to heap out-of-bounds access. + +- RedisBloom: Missing validation in `SBChain` and `CMSketch` RDB loading may lead to heap out-of-bounds write. + +- Added validation of limits during RDB load and data creation to avoid resource exhaustion and Denial-of-Service (DoS) vulnerabilities. + +{{< /collapsible >}} + +{{< collapsible "Redis 8.2.x" >}} + +- Memory corruption in `SLOT_INFO` handling may allow an unauthenticated user to potentially run their own code. + +- A use-after-free in TLS pending-data processing may allow an authenticated user to cause a denial-of-service. + +- A vulnerability in the `RESTORE` command and in RDB loading allows an authenticated user to trigger a use-after-free in stream consumer groups via a specially crafted serialized payload, potentially resulting in remote code execution. + +- An ACL key-pattern bypass in `GEORADIUS` and `GEORADIUSBYMEMBER` may allow an authenticated user to write to keys outside their permitted key patterns. + +- An out-of-bounds read during command key extraction may cause the server to crash. + +- VSET: Missing `level` cap when loading a serialized HNSW graph may lead to resource exhaustion. + +- VSET: Missing validation of the parameter count when loading a serialized vector set may lead to oversized allocation and resource exhaustion. + +- (CVE-2026-62356) An authenticated user may trigger a heap out-of-bounds write in RedisBloom by loading a specially crafted Count-Min Sketch (`CMSketch`) object, for example via the `RESTORE` command or a crafted RDB file. + +- RedisBloom: Out-of-bounds access in the `TopK` heap cleanup path. + +- VSET: Large `VADD REDUCE` values can lead to out-of-memory crashes or buffer overflow. + +- (CVE-2026-23479) When a blocked client is evicted while re-executing a blocked command, an authenticated user may trigger a use-after-free and potentially lead to remote code execution. + +- (CVE‑2026‑25243) A vulnerability in the Redis `RESTORE` command allows an authenticated user to trigger an invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE-2026-25588) A vulnerability in the `RESTORE` command, when used with the RedisTimeSeries module, allows an authenticated attacker to trigger invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE‑2026‑25589) A vulnerability in the `RESTORE` command, when used with the RedisBloom module, allows an authenticated attacker to trigger invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE-2026-23631) An authenticated user may exploit the synchronization mechanism of the master-replica and trigger a use-after-free vulnerability, potentially leading to remote code execution. The bug affects only replicas that are configured, or may be configured with `replica-read-only` disabled, and exists in all versions of Redis with Lua scripting. + +- Insufficient sanitization of Lua script error messages can allow an authenticated user with `EVAL` permission to inject arbitrary RESP protocol responses, leading to denial of service, data manipulation, and connection pool poisoning. + +- RedisBloom: Integer overflow in `TopK` RDB loading may lead to heap out-of-bounds access. + +- RedisBloom: Missing validation in `td_histogram_t` RDB loading may lead to heap buffer overflow. + +- RedisBloom: Missing validation in `CuckooFilter` RDB loading may lead to heap out-of-bounds access. + +- RedisBloom: Missing validation in `SBChain` and `CMSketch` RDB loading may lead to heap out-of-bounds write. + +- Added validation of limits during RDB load and data creation to avoid resource exhaustion and Denial-of-Service (DoS) vulnerabilities. + +- RedisBloom: Restore invalid filter. + +- (CVE-2025-62507) A user can run the `XACKDEL` command with multiple IDs and trigger a stack buffer overflow, which can potentially lead to remote code execution. + +- The `HGETEX` command can lead to a buffer overflow. + +- Integer overflow in `hllPatLen`. + +- RedisBloom: Cuckoo filter counter overflow. + +- RedisBloom: Invalid Bloom filters can cause arbitrary memory reads and writes. + +- RedisBloom: Reachable assert in `TopK_Create` + +- RedisBloom: Out-of-bounds access with empty Bloom chains. + +- RedisBloom: Division by zero in Cuckoo filter insertion. + +- (CVE-2025-46818) An authenticated user may use a specially crafted Lua script to manipulate different LUA objects and potentially run their own code in the context of another user. + +- (CVE-2025-46819) An authenticated user may use a specially crafted LUA script to read out-of-bound data or crash the server and lead to subsequent denial of service. + +- (CVE-2025-46817) An authenticated user may use a specially crafted Lua script to cause an integer overflow and potentially lead to remote code execution. + +- (CVE-2025-49844) An authenticated user may use a specially crafted Lua script to manipulate the garbage collector, trigger a use-after-free, and potentially lead to remote code execution. + +{{< /collapsible >}} + +{{< collapsible "Redis 8.0.x" >}} + +- Memory corruption in `SLOT_INFO` handling may allow an unauthenticated user to potentially run their own code. + +- A use-after-free in TLS pending-data processing may allow an authenticated user to cause a denial-of-service. + +- A vulnerability in the `RESTORE` command and in RDB loading allows an authenticated user to trigger a use-after-free in stream consumer groups via a specially crafted serialized payload, potentially resulting in remote code execution. + +- An ACL key-pattern bypass in `GEORADIUS` and `GEORADIUSBYMEMBER` may allow an authenticated user to write to keys outside their permitted key patterns. + +- An out-of-bounds read during command key extraction may cause the server to crash. + +- (CVE-2026-62356) An authenticated user may trigger a heap out-of-bounds write in RedisBloom by loading a specially crafted Count-Min Sketch (`CMSketch`) object, for example via the `RESTORE` command or a crafted RDB file. + +- RedisBloom: Out-of-bounds access in the `TopK` heap cleanup path. + +- VSET: Large `VADD REDUCE` values can lead to out-of-memory crashes or buffer overflow. + +- (CVE-2026-23479) When a blocked client is evicted while re-executing a blocked command, an authenticated user may trigger a use-after-free and potentially lead to remote code execution. + +- (CVE‑2026‑25243) A vulnerability in the Redis `RESTORE` command allows an authenticated user to trigger an invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE-2026-25588) A vulnerability in the `RESTORE` command, when used with the RedisTimeSeries module, allows an authenticated attacker to trigger invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE‑2026‑25589) A vulnerability in the `RESTORE` command, when used with the RedisBloom module, allows an authenticated attacker to trigger invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE-2026-23631) An authenticated user may exploit the synchronization mechanism of the master-replica and trigger a use-after-free vulnerability, potentially leading to remote code execution. The bug affects only replicas that are configured, or may be configured with `replica-read-only` disabled, and exists in all versions of Redis with Lua scripting. + +- Insufficient sanitization of Lua script error messages can allow an authenticated user with `EVAL` permission to inject arbitrary RESP protocol responses, leading to denial of service, data manipulation, and connection pool poisoning. + +- RedisBloom: Integer overflow in `TopK` RDB loading may lead to heap out-of-bounds access. + +- RedisBloom: Missing validation in `td_histogram_t` RDB loading may lead to heap buffer overflow. + +- RedisBloom: Missing validation in `CuckooFilter` RDB loading may lead to heap out-of-bounds access. + +- RedisBloom: Missing validation in `SBChain` and `CMSketch` RDB loading may lead to heap out-of-bounds write. + +- Added validation of limits during RDB load and data creation to avoid resource exhaustion and Denial-of-Service (DoS) vulnerabilities. + +- RedisBloom: Restore invalid filter. + +- The `HGETEX` command can lead to a buffer overflow. + +- Integer overflow in `hllPatLen`. + +- RedisBloom: Cuckoo filter counter overflow. + +- RedisBloom: Invalid Bloom filters can cause arbitrary memory reads and writes. + +- RedisBloom: Reachable assert in `TopK_Create` + +- RedisBloom: Out-of-bounds access with empty Bloom chains. + +- RedisBloom: Division by zero in Cuckoo filter insertion. + +- (CVE-2025-46818) An authenticated user may use a specially crafted Lua script to manipulate different LUA objects and potentially run their own code in the context of another user. + +- (CVE-2025-46819) An authenticated user may use a specially crafted LUA script to read out-of-bound data or crash the server and lead to subsequent denial of service. + +- (CVE-2025-46817) An authenticated user may use a specially crafted Lua script to cause an integer overflow and potentially lead to remote code execution. + +- (CVE-2025-49844) An authenticated user may use a specially crafted Lua script to manipulate the garbage collector, trigger a use-after-free, and potentially lead to remote code execution. + +{{< /collapsible >}} + +{{< collapsible "Redis 7.4.x" >}} + +- Memory corruption in `SLOT_INFO` handling may allow an unauthenticated user to potentially run their own code. + +- A use-after-free in TLS pending-data processing may allow an authenticated user to cause a denial-of-service. + +- A vulnerability in the `RESTORE` command and in RDB loading allows an authenticated user to trigger a use-after-free in stream consumer groups via a specially crafted serialized payload, potentially resulting in remote code execution. + +- An ACL key-pattern bypass in `GEORADIUS` and `GEORADIUSBYMEMBER` may allow an authenticated user to write to keys outside their permitted key patterns. + +- An out-of-bounds read during command key extraction may cause the server to crash. + +- (CVE-2026-62356) An authenticated user may trigger a heap out-of-bounds write in RedisBloom by loading a specially crafted Count-Min Sketch (`CMSketch`) object, for example via the `RESTORE` command or a crafted RDB file. + +- RedisBloom: Out-of-bounds access in the `TopK` heap cleanup path. + +- (CVE-2026-23479) When a blocked client is evicted while re-executing a blocked command, an authenticated user may trigger a use-after-free and potentially lead to remote code execution. + +- (CVE‑2026‑25243) A vulnerability in the Redis `RESTORE` command allows an authenticated user to trigger an invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE-2026-25588) A vulnerability in the `RESTORE` command, when used with the RedisTimeSeries module, allows an authenticated attacker to trigger invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE‑2026‑25589) A vulnerability in the `RESTORE` command, when used with the RedisBloom module, allows an authenticated attacker to trigger invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE-2026-23631) An authenticated user may exploit the synchronization mechanism of the master-replica and trigger a use-after-free vulnerability, potentially leading to remote code execution. The bug affects only replicas that are configured, or may be configured with `replica-read-only` disabled, and exists in all versions of Redis with Lua scripting. + +- Insufficient sanitization of Lua script error messages can allow an authenticated user with `EVAL` permission to inject arbitrary RESP protocol responses, leading to denial of service, data manipulation, and connection pool poisoning. + +- RedisBloom: Integer overflow in `TopK` RDB loading may lead to heap out-of-bounds access. + +- RedisBloom: Missing validation in `td_histogram_t` RDB loading may lead to heap buffer overflow. + +- RedisBloom: Missing validation in `CuckooFilter` RDB loading may lead to heap out-of-bounds access. + +- RedisBloom: Missing validation in `SBChain` and `CMSketch` RDB loading may lead to heap out-of-bounds write. + +- Added validation of limits during RDB load and data creation to avoid resource exhaustion and Denial-of-Service (DoS) vulnerabilities. + +- RedisBloom: Restore invalid filter. + +- Integer overflow in `hllPatLen`. + +- RedisBloom: Cuckoo filter counter overflow. + +- RedisBloom: Invalid Bloom filters can cause arbitrary memory reads and writes. + +- RedisBloom: Reachable assert in `TopK_Create` + +- RedisBloom: Out-of-bounds access with empty Bloom chains. + +- RedisBloom: Division by zero in Cuckoo filter insertion. + +- (CVE-2025-46818) An authenticated user may use a specially crafted Lua script to manipulate different LUA objects and potentially run their own code in the context of another user. + +- (CVE-2025-46819) An authenticated user may use a specially crafted LUA script to read out-of-bound data or crash the server and lead to subsequent denial of service. + +- (CVE-2025-46817) An authenticated user may use a specially crafted Lua script to cause an integer overflow and potentially lead to remote code execution. + +- (CVE-2025-49844) An authenticated user may use a specially crafted Lua script to manipulate the garbage collector, trigger a use-after-free, and potentially lead to remote code execution. + +- (CVE-2025-32023) An authenticated user can use a specially crafted string to trigger a stack/heap out-of-bounds write on HyperLogLog operations, which can lead to remote code execution. + +- (CVE-2025-21605) An unauthenticated client can cause unlimited growth of output buffers until the server runs out of memory or is terminated, which can lead to denial-of-service. + +{{< /collapsible >}} + +{{< collapsible "Redis 7.2.x" >}} + +- Memory corruption in `SLOT_INFO` handling may allow an unauthenticated user to potentially run their own code. + +- A use-after-free in TLS pending-data processing may allow an authenticated user to cause a denial-of-service. + +- A vulnerability in the `RESTORE` command and in RDB loading allows an authenticated user to trigger a use-after-free in stream consumer groups via a specially crafted serialized payload, potentially resulting in remote code execution. + +- An ACL key-pattern bypass in `GEORADIUS` and `GEORADIUSBYMEMBER` may allow an authenticated user to write to keys outside their permitted key patterns. + +- An out-of-bounds read during command key extraction may cause the server to crash. + +- (CVE-2026-62356) An authenticated user may trigger a heap out-of-bounds write in RedisBloom by loading a specially crafted Count-Min Sketch (`CMSketch`) object, for example via the `RESTORE` command or a crafted RDB file. + +- RedisBloom: Out-of-bounds access in the `TopK` heap cleanup path. + +- (CVE-2026-23479) When a blocked client is evicted while re-executing a blocked command, an authenticated user may trigger a use-after-free and potentially lead to remote code execution. + +- (CVE‑2026‑25243) A vulnerability in the Redis `RESTORE` command allows an authenticated user to trigger an invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE-2026-25588) A vulnerability in the `RESTORE` command, when used with the RedisTimeSeries module, allows an authenticated attacker to trigger invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE‑2026‑25589) A vulnerability in the `RESTORE` command, when used with the RedisBloom module, allows an authenticated attacker to trigger invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE-2026-23631) An authenticated user may exploit the synchronization mechanism of the master-replica and trigger a use-after-free vulnerability, potentially leading to remote code execution. The bug affects only replicas that are configured, or may be configured with `replica-read-only` disabled, and exists in all versions of Redis with Lua scripting. + +- Insufficient sanitization of Lua script error messages can allow an authenticated user with `EVAL` permission to inject arbitrary RESP protocol responses, leading to denial of service, data manipulation, and connection pool poisoning. + +- RedisBloom: Integer overflow in `TopK` RDB loading may lead to heap out-of-bounds access. + +- RedisBloom: Missing validation in `td_histogram_t` RDB loading may lead to heap buffer overflow. + +- RedisBloom: Missing validation in `CuckooFilter` RDB loading may lead to heap out-of-bounds access. + +- RedisBloom: Missing validation in `SBChain` and `CMSketch` RDB loading may lead to heap out-of-bounds write. + +- Added validation of limits during RDB load and data creation to avoid resource exhaustion and Denial-of-Service (DoS) vulnerabilities. + +- RedisBloom: Restore invalid filter. + +- Integer overflow in `hllPatLen`. + +- RedisBloom: Cuckoo filter counter overflow. + +- RedisBloom: Invalid Bloom filters can cause arbitrary memory reads and writes. + +- RedisBloom: Reachable assert in `TopK_Create` + +- RedisBloom: Out-of-bounds access with empty Bloom chains. + +- RedisBloom: Division by zero in Cuckoo filter insertion. + +- (CVE-2025-46818) An authenticated user may use a specially crafted Lua script to manipulate different LUA objects and potentially run their own code in the context of another user. + +- (CVE-2025-46819) An authenticated user may use a specially crafted LUA script to read out-of-bound data or crash the server and lead to subsequent denial of service. + +- (CVE-2025-46817) An authenticated user may use a specially crafted Lua script to cause an integer overflow and potentially lead to remote code execution. + +- (CVE-2025-49844) An authenticated user may use a specially crafted Lua script to manipulate the garbage collector, trigger a use-after-free, and potentially lead to remote code execution. + +- (CVE-2025-32023) An authenticated user can use a specially crafted string to trigger a stack/heap out-of-bounds write on HyperLogLog operations, which can lead to remote code execution. + +- (CVE-2025-21605) An unauthenticated client can cause unlimited growth of output buffers until the server runs out of memory or is terminated, which can lead to denial-of-service. + +- (CVE-2024-31449) An authenticated user may use a specially crafted Lua script to trigger a stack buffer overflow in the bit library, which may potentially lead to remote code execution. + +- (CVE-2024-31228) An authenticated user can trigger a denial-of-service by using specially crafted, long string match patterns on supported commands such as `KEYS`, `SCAN`, `PSUBSCRIBE`, `FUNCTION LIST`, `COMMAND LIST`, and ACL definitions. Matching of extremely long patterns may result in unbounded recursion, leading to stack overflow and process crashes. + +- (CVE-2023-41056) In some cases, Redis may incorrectly handle resizing of memory buffers, which can result in incorrect accounting of buffer sizes and lead to heap overflow and potential remote code execution. + +- (CVE-2023-41053) Redis does not correctly identify keys accessed by `SORT_RO` and, as a result, may grant users executing this command access to keys that are not explicitly authorized by the ACL configuration. (Redis 7.2.1) + +{{< /collapsible >}} + +{{< collapsible "Redis 7.0.x" >}} + +- (CVE-2024-31449) An authenticated user may use a specially crafted Lua script to trigger a stack buffer overflow in the bit library, which may potentially lead to remote code execution. + +- (CVE-2024-31228) An authenticated user can trigger a denial-of-service by using specially crafted, long string match patterns on supported commands such as `KEYS`, `SCAN`, `PSUBSCRIBE`, `FUNCTION LIST`, `COMMAND LIST`, and ACL definitions. Matching of extremely long patterns may result in unbounded recursion, leading to stack overflow and process crashes. + +- (CVE-2023-41056) In some cases, Redis may incorrectly handle resizing of memory buffers, which can result in incorrect accounting of buffer sizes and lead to heap overflow and potential remote code execution. + +- (CVE-2023-41053) Redis does not correctly identify keys accessed by `SORT_RO` and, as a result, may grant users executing this command access to keys that are not explicitly authorized by the ACL configuration. (Redis 7.0.13) + +- (CVE-2023-36824) Extracting key names from a command and a list of arguments may, in some cases, trigger a heap overflow and result in reading random heap memory, heap corruption, and potentially remote code execution. Specifically: using `COMMAND GETKEYS*` and validation of key names in ACL rules. (Redis 7.0.12) + +- (CVE-2023-28856) Authenticated users can use the `HINCRBYFLOAT` command to create an invalid hash field that will crash Redis on access. (Redis 7.0.11) + +- (CVE-2023-28425) Specially crafted `MSETNX` commands can lead to assertion and denial-of-service. (Redis 7.0.10) + +- (CVE-2023-25155) Specially crafted `SRANDMEMBER`, `ZRANDMEMBER`, and `HRANDFIELD` commands can trigger an integer overflow, resulting in a runtime assertion and termination of the Redis server process. (Redis 7.0.9) + +- (CVE-2023-22458) Integer overflow in the Redis `HRANDFIELD` and `ZRANDMEMBER` commands can lead to denial-of-service. (Redis 7.0.8) + +- (CVE-2022-36021) String matching commands (like `SCAN` or `KEYS`) with a specially crafted pattern to trigger a denial-of-service attack on Redis can cause it to hang and consume 100% CPU time. (Redis 7.0.9) + +- (CVE-2022-35977) Integer overflow in the Redis `SETRANGE` and `SORT`/`SORT_RO` commands can drive Redis to OOM panic. (Redis 7.0.8) + +- (CVE-2022-35951) Executing an `XAUTOCLAIM` command on a stream key in a specific state, with a specially crafted `COUNT` argument, may cause an integer overflow, a subsequent heap overflow, and potentially lead to remote code execution. The problem affects Redis versions 7.0.0 or newer. (Redis 7.0.5) + +- (CVE-2022-31144) A specially crafted `XAUTOCLAIM` command on a stream key in a specific state may result in heap overflow and potentially remote code execution. The problem affects Redis versions 7.0.0 or newer. (Redis 7.0.4) + +- (CVE-2022-24834) A specially crafted Lua script executing in Redis can trigger a heap overflow in the cjson and cmsgpack libraries, and result in heap corruption and potentially remote code execution. The problem exists in all versions of Redis with Lua scripting support, starting from 2.6, and affects only authenticated and authorized users. (Redis 7.0.12) + +- (CVE-2022-24736) An attacker attempting to load a specially crafted Lua script can cause NULL pointer dereference which will result in a crash of the `redis-server` process. This issue affects all versions of Redis. (Redis 7.0.0) + +- (CVE-2022-24735) By exploiting weaknesses in the Lua script execution environment, an attacker with access to Redis can inject Lua code that will execute with the (potentially higher) privileges of another Redis user. (Redis 7.0.0) + +{{< /collapsible >}} + +{{< collapsible "Redis 6.2.x" >}} + +- Memory corruption in `SLOT_INFO` handling may allow an unauthenticated user to potentially run their own code. + +- A use-after-free in TLS pending-data processing may allow an authenticated user to cause a denial-of-service. + +- A vulnerability in the `RESTORE` command and in RDB loading allows an authenticated user to trigger a use-after-free in stream consumer groups via a specially crafted serialized payload, potentially resulting in remote code execution. + +- An ACL key-pattern bypass in `GEORADIUS` and `GEORADIUSBYMEMBER` may allow an authenticated user to write to keys outside their permitted key patterns. + +- An out-of-bounds read during command key extraction may cause the server to crash. + +- (CVE-2026-62356) An authenticated user may trigger a heap out-of-bounds write in RedisBloom by loading a specially crafted Count-Min Sketch (`CMSketch`) object, for example via the `RESTORE` command or a crafted RDB file. + +- RedisBloom: Out-of-bounds access in the `TopK` heap cleanup path. + +- (CVE-2026-23479) When a blocked client is evicted while re-executing a blocked command, an authenticated user may trigger a use-after-free and potentially lead to remote code execution. + +- (CVE‑2026‑25243) A vulnerability in the Redis `RESTORE` command allows an authenticated user to trigger an invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE-2026-25588) A vulnerability in the `RESTORE` command, when used with the RedisTimeSeries module, allows an authenticated attacker to trigger invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE‑2026‑25589) A vulnerability in the `RESTORE` command, when used with the RedisBloom module, allows an authenticated attacker to trigger invalid memory access via a specially crafted serialized payload, potentially resulting in remote code execution. + +- (CVE-2026-23631) An authenticated user may exploit the synchronization mechanism of the master-replica and trigger a use-after-free vulnerability, potentially leading to remote code execution. The bug affects only replicas that are configured, or may be configured with `replica-read-only` disabled, and exists in all versions of Redis with Lua scripting. + +- Insufficient sanitization of Lua script error messages can allow an authenticated user with `EVAL` permission to inject arbitrary RESP protocol responses, leading to denial of service, data manipulation, and connection pool poisoning. + +- RedisBloom: Integer overflow in `TopK` RDB loading may lead to heap out-of-bounds access. + +- RedisBloom: Missing validation in `td_histogram_t` RDB loading may lead to heap buffer overflow. + +- RedisBloom: Missing validation in `CuckooFilter` RDB loading may lead to heap out-of-bounds access. + +- RedisBloom: Missing validation in `SBChain` and `CMSketch` RDB loading may lead to heap out-of-bounds write. + +- Added validation of limits during RDB load and data creation to avoid resource exhaustion and Denial-of-Service (DoS) vulnerabilities. + +- RedisBloom: Restore invalid filter. + +- Integer overflow in `hllPatLen`. + +- RedisBloom: Cuckoo filter counter overflow. + +- RedisBloom: Invalid Bloom filters can cause arbitrary memory reads and writes. + +- RedisBloom: Reachable assert in `TopK_Create` + +- RedisBloom: Out-of-bounds access with empty Bloom chains. + +- RedisBloom: Division by zero in Cuckoo filter insertion. + +- (CVE-2025-46818) An authenticated user may use a specially crafted Lua script to manipulate different LUA objects and potentially run their own code in the context of another user. + +- (CVE-2025-46819) An authenticated user may use a specially crafted LUA script to read out-of-bound data or crash the server and lead to subsequent denial of service. + +- (CVE-2025-46817) An authenticated user may use a specially crafted Lua script to cause an integer overflow and potentially lead to remote code execution. + +- (CVE-2025-49844) An authenticated user may use a specially crafted Lua script to manipulate the garbage collector, trigger a use-after-free, and potentially lead to remote code execution. + +- (CVE-2025-32023) An authenticated user can use a specially crafted string to trigger a stack/heap out-of-bounds write on HyperLogLog operations, which can lead to remote code execution. + +- (CVE-2025-21605) An unauthenticated client can cause unlimited growth of output buffers until the server runs out of memory or is terminated, which can lead to denial-of-service. + +- (CVE-2024-31449) An authenticated user may use a specially crafted Lua script to trigger a stack buffer overflow in the bit library, which may potentially lead to remote code execution. + +- (CVE-2024-31228) An authenticated user can trigger a denial-of-service by using specially crafted, long string match patterns on supported commands such as `KEYS`, `SCAN`, `PSUBSCRIBE`, `FUNCTION LIST`, `COMMAND LIST`, and ACL definitions. Matching of extremely long patterns may result in unbounded recursion, leading to stack overflow and process crashes. + +- (CVE-2023-28856) Authenticated users can use the `HINCRBYFLOAT` command to create an invalid hash field that will crash Redis on access. (Redis 6.2.12) + +- (CVE-2023-25155) Specially crafted `SRANDMEMBER`, `ZRANDMEMBER`, and `HRANDFIELD` commands can trigger an integer overflow, resulting in a runtime assertion and termination of the Redis server process. (Redis 6.2.11) + +- (CVE-2023-22458) Integer overflow in the Redis `HRANDFIELD` and `ZRANDMEMBER` commands can lead to denial-of-service. (Redis 6.2.9) + +- (CVE-2022-36021) String matching commands (like `SCAN` or `KEYS`) with a specially crafted pattern to trigger a denial-of-service attack on Redis can cause it to hang and consume 100% CPU time. (Redis 6.2.11) + +- (CVE-2022-35977) Integer overflow in the Redis `SETRANGE` and `SORT`/`SORT_RO` commands can drive Redis to OOM panic. (Redis 6.2.9) + +- (CVE-2022-24834) A specially crafted Lua script executing in Redis can trigger a heap overflow in the cjson and cmsgpack libraries, and result in heap corruption and potentially remote code execution. The problem exists in all versions of Redis with Lua scripting support, starting from 2.6, and affects only authenticated and authorized users. (Redis 6.2.13) + +- (CVE-2022-24736) An attacker attempting to load a specially crafted Lua script can cause NULL pointer dereference which will result in a crash of the `redis-server` process. This issue affects all versions of Redis. (Redis 6.2.7) + +- (CVE-2022-24735) By exploiting weaknesses in the Lua script execution environment, an attacker with access to Redis can inject Lua code that will execute with the (potentially higher) privileges of another Redis user. (Redis 6.2.7) + +- (CVE-2021-41099) Integer to heap buffer overflow handling certain string commands and network payloads, when `proto-max-bulk-len` is manually configured to a non-default, very large value. (Redis 6.2.6) + +- (CVE-2021-32762) Integer to heap buffer overflow issue in `redis-cli` and `redis-sentinel` parsing large multi-bulk replies on some older and less common platforms. (Redis 6.2.6) + +- (CVE-2021-32761) An integer overflow bug in Redis version 2.2 or newer can be exploited using the `BITFIELD` command to corrupt the heap and potentially result with remote code execution. (Redis 6.2.5) + +- (CVE-2021-32687) Integer to heap buffer overflow with intsets, when `set-max-intset-entries` is manually configured to a non-default, very large value. (Redis 6.2.6) + +- (CVE-2021-32675) Denial Of Service when processing RESP request payloads with a large number of elements on many connections. (Redis 6.2.6) + +- (CVE-2021-32672) Random heap reading issue with Lua Debugger. (Redis 6.2.6) + +- (CVE-2021-32628) Integer to heap buffer overflow handling ziplist-encoded data types, when configuring a large, non-default value for `hash-max-ziplist-entries`, `hash-max-ziplist-value`, `zset-max-ziplist-entries` or `zset-max-ziplist-value`. (Redis 6.2.6) + +- (CVE-2021-32627) Integer to heap buffer overflow issue with streams, when configuring a non-default, large value for `proto-max-bulk-len` and `client-query-buffer-limit`. (Redis 6.2.6) + +- (CVE-2021-32626) Specially crafted Lua scripts may result with Heap buffer overflow. (Redis 6.2.6) + +- (CVE-2021-32625) An integer overflow bug in Redis version 6.0 or newer can be exploited using the STRALGO LCS command to corrupt the heap and potentially result with remote code execution. This is a result of an incomplete fix by CVE-2021-29477. (Redis 6.2.4) + +- (CVE-2021-29478) An integer overflow bug in Redis 6.2 could be exploited to corrupt the heap and potentially result with remote code execution. The vulnerability involves changing the default set-max-intset-entries configuration value, creating a large set key that consists of integer values and using the COPY command to duplicate it. The integer overflow bug exists in all versions of Redis starting with 2.6, where it could result with a corrupted RDB or DUMP payload, but not exploited through COPY (which did not exist before 6.2). (Redis 6.2.3) + +- (CVE-2021-29477) An integer overflow bug in Redis version 6.0 or newer could be exploited using the STRALGO LCS command to corrupt the heap and potentially result in remote code execution. The integer overflow bug exists in all versions of Redis starting with 6.0. (Redis 6.2.3) + +{{< /collapsible >}} From dbe7d5fd593296b85f5b9d97b31372b540390024 Mon Sep 17 00:00:00 2001 From: Kaitlyn Michael <76962844+kaitlynmichael@users.noreply.github.com> Date: Tue, 20 Oct 2026 11:01:39 -0500 Subject: [PATCH 50/76] K8s: duckberg maint 8.2.0-19 (#4311) --- .../8-2-0-releases/8-2-0-19-october2026.md | 35 +++++++++++++++++++ .../release-notes/8-2-0-releases/_index.md | 2 +- 2 files changed, 36 insertions(+), 1 deletion(-) create mode 100644 content/operate/kubernetes/release-notes/8-2-0-releases/8-2-0-19-october2026.md diff --git a/content/operate/kubernetes/release-notes/8-2-0-releases/8-2-0-19-october2026.md b/content/operate/kubernetes/release-notes/8-2-0-releases/8-2-0-19-october2026.md new file mode 100644 index 0000000000..db8169f55e --- /dev/null +++ b/content/operate/kubernetes/release-notes/8-2-0-releases/8-2-0-19-october2026.md @@ -0,0 +1,35 @@ +--- +alwaysopen: false +categories: +- docs +- operate +- kubernetes +description: This is a maintenance release to support Redis Software version 8.2.0-125. +hideListLinks: true +linkTitle: 8.2.0-19 (October 2026) +title: Redis Software for Kubernetes 8.2.0-19 (October 2026) release notes +weight: 23 +--- + +## Highlights + +This is a maintenance release to support [Redis Software 8.2.0](/content/operate/rs/release-notes/rs-8-2-releases/_index.md) version 8.2.0-125. For version changes, supported distributions, and known limitations, see the [release notes for 8.2.0-12 (July 2026)](/content/operate/kubernetes/release-notes/8-2-0-releases/8-2-0-12-july2026.md). + +## Downloads + +- **Redis Enterprise**: `redislabs/redis:8.2.0-125.19` (`redislabs/redis:8.2.0-125.19.minimal` for minimal base image) +- **Operator**: `redislabs/operator:8.2.0-19` +- **Services Rigger**: `redislabs/k8s-controller:8.2.0-19` +- **Call Home Client**: `redislabs/re-call-home-client:8.2.0-19` + +### OpenShift downloads + +- **OLM operator bundle**: `8.2.0-19.1` +- **Redis Enterprise**: `registry.connect.redhat.com/redislabs/redis-enterprise:8.2.0-125.19` (`registry.connect.redhat.com/redislabs/redis-enterprise:8.2.0-125.19.minimal` for minimal base image) +- **Operator**: `registry.connect.redhat.com/redislabs/redis-enterprise-operator:8.2.0-19` +- **Services Rigger**: `registry.connect.redhat.com/redislabs/services-manager:8.2.0-19` +- **Call Home Client**: `registry.connect.redhat.com/redislabs/call-home-client:8.2.0-19` + +## Known limitations + +See [8.2.0 releases](/content/operate/kubernetes/release-notes/8-2-0-releases/_index.md) for information on known limitations. diff --git a/content/operate/kubernetes/release-notes/8-2-0-releases/_index.md b/content/operate/kubernetes/release-notes/8-2-0-releases/_index.md index 22e57ce95e..70c08768ef 100644 --- a/content/operate/kubernetes/release-notes/8-2-0-releases/_index.md +++ b/content/operate/kubernetes/release-notes/8-2-0-releases/_index.md @@ -11,7 +11,7 @@ title: Redis Software for Kubernetes 8.2.0 release notes weight: 83 --- -Redis Software for Kubernetes 8.2.0 includes new features, bug fixes, enhancements, and support for Redis Software 8.2.0. The latest release is 8.2.0-18 with support for Redis Software version 8.2.0-78. +Redis Software for Kubernetes 8.2.0 includes new features, bug fixes, enhancements, and support for Redis Software 8.2.0. The latest release is 8.2.0-19 with support for Redis Software version 8.2.0-125. ## Detailed release notes From 37be9d84bd7b3d7d91bdb68eff9f77b6959824be Mon Sep 17 00:00:00 2001 From: Orhan Abdula Date: Wed, 21 Oct 2026 08:57:20 +0300 Subject: [PATCH 51/76] Correct RDI pipeline hash slot placement and link to RDI database throughput guidance --- content/operate/rc/rdi/faq.md | 3 ++- content/operate/rc/rdi/scale-pipeline.md | 10 ++++++---- 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/content/operate/rc/rdi/faq.md b/content/operate/rc/rdi/faq.md index 476c6f1f26..7b7d91a053 100644 --- a/content/operate/rc/rdi/faq.md +++ b/content/operate/rc/rdi/faq.md @@ -146,7 +146,8 @@ No. Set the desired processor replica count with ### How do I increase processing capacity for a pipeline? First [identify the bottleneck](/content/operate/rc/rdi/scale-pipeline.md#identify-the-bottleneck). -Then tune the collector, RDI database, or processor. Check shared +Then tune the collector, [RDI database](/content/operate/rc/rdi/scale-pipeline.md#increase-rdi-database-throughput), +or processor. Check shared workspace network capacity before adding sources or processor replicas. ### Can I see that the processor has scaled in the console? diff --git a/content/operate/rc/rdi/scale-pipeline.md b/content/operate/rc/rdi/scale-pipeline.md index 0974e2d349..b1ed819884 100644 --- a/content/operate/rc/rdi/scale-pipeline.md +++ b/content/operate/rc/rdi/scale-pipeline.md @@ -90,7 +90,8 @@ and **RDI database load** before you change capacity: connectivity before adding processor replicas. - If the RDI database stays full or reports out-of-memory (OOM) errors, check ingestion and processing rates, database memory, and throughput. OOM alone - does not identify which capacity setting to change. + does not identify which capacity setting to change. See + [Increase RDI database throughput](#increase-rdi-database-throughput). Change one setting at a time and compare the same representative workload. Confirm that the change improves throughput or reduces pending records before @@ -163,9 +164,10 @@ For multiple pipelines, check **Used memory** and **Ops/sec** the same way as for a single pipeline. When you increase **Throughput**, choose a value with as many shards as you -have pipelines. Each pipeline writes to one shard, even if it has several -sources. When the number of shards matches the number of pipelines, each -pipeline writes to its own shard. +have pipelines. Each pipeline writes to its own hash slot, and all sources in +a pipeline share that slot. RDI spreads the pipelines' hash slots across the +slot range, so when the number of shards matches the number of pipelines, +each pipeline's slot is on a different shard. | Throughput | Nodes | Shards | | --- | ---: | ---: | From 6f63ac4f9148ef1fc75866cbe54bf502568ae9d0 Mon Sep 17 00:00:00 2001 From: Ivo Bardarov Date: Thu, 15 Oct 2026 12:03:45 +0300 Subject: [PATCH 52/76] docs(iris): MOD-19813 Agent Memory self-managed docs for the iris-identity-service subchart --- .../iris/agent-memory/self-managed/_index.md | 8 ++- .../self-managed/authentication.md | 4 +- .../self-managed/configuration.md | 10 +-- .../iris/agent-memory/self-managed/deploy.md | 24 ++++--- .../agent-memory/self-managed/operations.md | 49 +++++++++++--- .../self-managed/prerequisites.md | 67 +++++++++++-------- .../agent-memory/self-managed/reference.md | 11 ++- 7 files changed, 117 insertions(+), 56 deletions(-) diff --git a/content/operate/iris/agent-memory/self-managed/_index.md b/content/operate/iris/agent-memory/self-managed/_index.md index 1801e918eb..d7dac7eab8 100644 --- a/content/operate/iris/agent-memory/self-managed/_index.md +++ b/content/operate/iris/agent-memory/self-managed/_index.md @@ -31,15 +31,17 @@ documents the self-managed admin endpoints for stores. ## What you are deploying -The chart always deploys the Data Plane, the workers, the Control Plane, and the Identity Service. -You provide the Redis databases. +The chart deploys the Data Plane, the workers, and the Control Plane, and bundles the Identity +Service by default. You provide the Redis databases. To point the Data Plane at an Identity +Service the chart does not install, see the `identityService.enabled=false` entries in +[Helm values to review](/content/operate/iris/agent-memory/self-managed/prerequisites.md#helm-values-to-review). | Component | Purpose | Default service | | --- | --- | --- | | Redis Agent Memory Data Plane | Store-scoped runtime memory API. | `redis-agent-memory:9000` | | Redis Agent Memory worker | Background promotion, summarization, and forgetting work. | No public service | | Redis Agent Memory Control Plane | Admin API for creating and managing stores. | `redis-agent-memory-controlplane:9100` | -| Identity Service | Mints, rotates, revokes, and checks agent keys. | `redis-agent-memory-identity-service:9200` | +| Identity Service (bundled) | Mints, rotates, revokes, and checks agent keys. | `redis-agent-memory-iris-identity-service:9200` | | Store Redis | Holds session memory, long-term memory, indexes, and TTL data. | Customer-provided | | Job Redis | Holds background work for Redis Agent Memory workers. | Customer-provided | | Metadata Redis | Holds Control Plane store records. The Identity Service keeps agent-key records in the Metadata Redis named in its own metadata Secret. | Customer-provided | diff --git a/content/operate/iris/agent-memory/self-managed/authentication.md b/content/operate/iris/agent-memory/self-managed/authentication.md index a9431450c0..72a67e0365 100644 --- a/content/operate/iris/agent-memory/self-managed/authentication.md +++ b/content/operate/iris/agent-memory/self-managed/authentication.md @@ -26,8 +26,8 @@ grants. | --- | --- | --- | --- | --- | | Control Plane admin token | Operator → Control Plane (`:9100`) | `redis-agent-memory-controlplane-admin-token` / `token` | Yes (`controlplane.adminToken.autoGenerate`) | Store management: `/v1/stores`, `/v1/detectors` | | Control Plane internal token | Identity Service → Control Plane | `redis-agent-memory-controlplane-internal-token` / `token` | Yes (`controlplane.internalToken.autoGenerate`) | Checking that the store in a grant exists. Chart-managed; you handle it only with bring-your-own (BYO) configuration | -| Identity Service control token | Operator → Identity Service (`:9200`) | `redis-agent-memory-identity-service-control-token` / `token` | Yes (`identityService.controlToken.autoGenerate`) | Creating, listing, updating, rotating and revoking agent keys | -| Data Plane service credential | Data Plane → Identity Service | `redis-agent-memory-identity-service-runtime-memory-dp` / `token` | Yes (`identityService.runtime.serviceCredentials`) | Checking agent keys for `memory`. Chart-managed; you handle it only with bring-your-own (BYO) configuration | +| Identity Service control token | Operator → Identity Service (`:9200`) | `redis-agent-memory-iris-identity-service-control-token` / `token` | Yes (`identityService.control.autoGenerate`) | Creating, listing, updating, rotating and revoking agent keys | +| Data Plane service credential | Data Plane → Identity Service | `redis-agent-memory-iris-identity-service-runtime-memory-dp` / `token` | Yes (`identityService.runtime.serviceCredentials`) | Checking agent keys for `memory`. Chart-managed; you handle it only with bring-your-own (BYO) configuration | | Agent key | Application → Data Plane (`:9000`) | None; returned once when minted or rotated | No | The Data Plane operations its grants allow | Each generated Secret keeps its value across upgrades (`helm.sh/resource-policy: keep`). To bring diff --git a/content/operate/iris/agent-memory/self-managed/configuration.md b/content/operate/iris/agent-memory/self-managed/configuration.md index 4daff23f61..cdebd5f8b2 100644 --- a/content/operate/iris/agent-memory/self-managed/configuration.md +++ b/content/operate/iris/agent-memory/self-managed/configuration.md @@ -28,7 +28,7 @@ To supply complete configuration files in your own Secrets instead, see | `shared` | Settings common to the Data Plane and the Control Plane, such as `databases`. The chart merges them under `memory` and `controlplane.configData`. | | `memory` | The Data Plane and worker configuration file. | | `controlplane.configData` | The Control Plane configuration file. | -| `identityService.*` | The Identity Service. `identityService.metadata.existingSecret` names the Secret with its Metadata Redis URL. | +| `identityService.*` | The bundled Identity Service, from the published `iris-identity-service` chart; every key is that chart's values contract. `identityService.metadata.existingSecret` names the Secret with its Metadata Redis URL, and extra service configuration goes in `identityService.configData`. | | Overlay Secret (`secrets.secretName`, key `overlay.yaml`) | `metadata.urls`, `metadata.namespace`, `databases."1".urls`, `background_jobs.redis.urls`, and provider `api_key` values. The Data Plane, the workers, and the Control Plane all read it. | ## Data Plane settings @@ -53,8 +53,10 @@ Data Plane settings go under `memory`, or at the top level of a BYO Data Plane c An `llm` block names its provider with `llm.provider`, which must match an `inference_providers` entry. The endpoint goes in `inference_providers..endpoint`, not in the `llm` block. -In the chart-rendered configuration, the chart writes the agent-key introspection settings for you. -Leave `auth.method` unset to keep the `agent_key` default. +In the chart-rendered configuration, the chart writes the agent-key introspection settings for you: +it owns `auth.agent_keys.introspection.base_url`, `credential`, and `allow_insecure_transport`, and +refuses hand-set values. Set `dataplane.identityServiceBaseURL` to point at a non-default Identity +Service endpoint. Leave `auth.method` unset to keep the `agent_key` default. ## Bring-your-own (BYO) configuration @@ -106,7 +108,7 @@ databases: auth: agent_keys: introspection: - base_url: http://redis-agent-memory-identity-service:9200 + base_url: http://redis-agent-memory-iris-identity-service:9200 allow_insecure_transport: true product: memory credential: diff --git a/content/operate/iris/agent-memory/self-managed/deploy.md b/content/operate/iris/agent-memory/self-managed/deploy.md index da6029a575..b1ef473773 100644 --- a/content/operate/iris/agent-memory/self-managed/deploy.md +++ b/content/operate/iris/agent-memory/self-managed/deploy.md @@ -83,7 +83,7 @@ chart's defaults disagree here: the Data Plane defaults to `agent_key` authentic `workerAuth.enabled` defaults to `false`. With `agent_key`, you must configure worker identity. The values in [step 3](#3-create-ram-valuesyaml) do this. -In version 0.7.0, the Data Plane fetches the JWKS without credentials and trusts only the +The Data Plane fetches the JWKS without credentials and trusts only the certificate authority (CA) bundle in its image. What you set up depends on where the cluster's issuer is served: @@ -118,7 +118,7 @@ kubectl create clusterrolebinding redis-agent-memory-oidc-discovery \ This binding grants only `get` on `/.well-known/openid-configuration` and `/openid/v1/jwks`, which serve issuer metadata and public signing keys, not secrets. It applies to unauthenticated callers across the whole cluster. It has no effect where the API server disables anonymous authentication. -On those clusters, version 0.7.0 has no working setup for `agent_key` with workers. +On those clusters, there is no working setup for `agent_key` with workers. ## 3. Create `ram-values.yaml` @@ -197,7 +197,11 @@ memory: enabled: false session_summary_view: enabled: false +image: + tag: controlplane: + image: + tag: config: render: true configData: @@ -228,7 +232,9 @@ tls: - `tls.caCertSecret` names the CA Secret from [step 2](#prepare-worker-identity-for-your-platform). Remove it on EKS and AKS. - The chart writes the agent-key introspection settings and the Control Plane internal token for you. -- Image tags are not set, so the chart defaults (`0.7.0`) apply. +- `` is the Redis Agent Memory version you deploy. `image.tag` and + `controlplane.image.tag` are required; the chart has no default. The bundled Identity Service + takes its image tag from the pinned chart, so it needs none. {{< note >}} No external model provider? To try the API with local models, see [Evaluate locally](#evaluate-locally). @@ -246,7 +252,7 @@ its row. The rows are checklist items CL-1 to CL-6. | CL-2 | `memory.promote_session_memory.strategies.instruct.llm` is set | Worker pods `CrashLoopBackOff` (server stays Ready); log: `promote_session_memory: Strategies: strategies.instruct.llm is required.` | Add the `instruct.llm` block | | CL-3 | Each `llm.provider` names an `inference_providers` entry | Server and worker pods `CrashLoopBackOff`; log: `panic: config validation failed: promote_session_memory.strategies.instruct.llm: provider "" is not defined in inference_providers (defined: )` | Add `memory.inference_providers.` or fix the name | | CL-4 | `memory.dataplane_client.base_url` is `http://redis-agent-memory:9000` | Pods Ready, but no memories appear; worker log: `getting session memory: Get "http://:9000/v1/stores//session-memory/..."` … `dial tcp: lookup ` (the rest depends on the cluster DNS) | Set `http://redis-agent-memory:9000` | -| CL-5 | No image tag is set, or it exists on Docker Hub | Server and worker pods `ImagePullBackOff`; event: `Failed to pull image "redislabs/agent-memory:": … not found` | Remove `*.image.tag` (defaults `0.7.0`) | +| CL-5 | `image.tag` and `controlplane.image.tag` are set, and each tag exists on Docker Hub | `helm install` fails with `image.tag is required and must be set explicitly to the RAM version being deployed`; or server and worker pods `ImagePullBackOff`, event: `Failed to pull image "redislabs/agent-memory:": … not found` | Set both tags to the Redis Agent Memory version you deploy | | CL-6 | `auth.worker_identity` matches the worker (issuer, audience, subject with your namespace) | Pods Ready, but no memories appear; worker log: `memory-dataplane API error: status 401`; the Data Plane logs `401` on `GET /v1/stores//session-memory/` with no reason | Fix `issuer` from `/.well-known/openid-configuration` and the namespace in `subject`. The same symptom appears when the Data Plane cannot fetch the cluster JWKS, so also check [worker identity for your platform](#prepare-worker-identity-for-your-platform) | {{< /table-scrollable >}} @@ -256,7 +262,7 @@ its row. The rows are checklist items CL-1 to CL-6. helm repo add redis-ai https://helm.redis.io/ai helm repo update redis-ai helm install redis-agent-memory redis-ai/redis-agent-memory \ - --version 0.7.0 \ + --version 1.0.0 \ --namespace \ -f ram-values.yaml kubectl -n get pods -w @@ -268,7 +274,7 @@ If you want Helm to wait, set an explicit timeout that matches the environment: ```bash helm install redis-agent-memory redis-ai/redis-agent-memory \ - --version 0.7.0 \ + --version 1.0.0 \ --namespace \ -f ram-values.yaml \ --wait \ @@ -281,7 +287,7 @@ helm install redis-agent-memory redis-ai/redis-agent-memory \ kubectl -n get pods -l app.kubernetes.io/name=redis-agent-memory ``` -The server, worker, `redis-agent-memory-controlplane`, and `redis-agent-memory-identity-service` +The server, worker, `redis-agent-memory-controlplane`, and `redis-agent-memory-iris-identity-service` pods are Ready. Run each `port-forward` in this guide in its own terminal. Port-forward the Data Plane and check its @@ -319,9 +325,9 @@ The response carries `storeId`. Use the Identity Service control token: ```bash -kubectl -n port-forward svc/redis-agent-memory-identity-service 9200:9200 +kubectl -n port-forward svc/redis-agent-memory-iris-identity-service 9200:9200 IDS_CONTROL_TOKEN=$(kubectl -n get secret \ - redis-agent-memory-identity-service-control-token -o jsonpath='{.data.token}' | base64 -d) + redis-agent-memory-iris-identity-service-control-token -o jsonpath='{.data.token}' | base64 -d) curl -sS -X POST http://localhost:9200/v1/api-keys \ -H "Authorization: Bearer $IDS_CONTROL_TOKEN" \ -H "Content-Type: application/json" \ diff --git a/content/operate/iris/agent-memory/self-managed/operations.md b/content/operate/iris/agent-memory/self-managed/operations.md index 4b382cd4f0..1e15bca9de 100644 --- a/content/operate/iris/agent-memory/self-managed/operations.md +++ b/content/operate/iris/agent-memory/self-managed/operations.md @@ -185,24 +185,52 @@ environments where the in-cluster test path is allowed. ## FIPS-oriented posture -The chart supports an opt-in FIPS-oriented posture for regulated environments: +The chart supports an opt-in FIPS-oriented posture for regulated environments. +Set it through `global.security.profile`, which the chart and the bundled Identity Service both +read. The chart's local `security.profile` does not reach the Identity Service, and the chart +refuses a FIPS Redis Agent Memory with a non-FIPS bundled Identity Service: ```yaml -security: - profile: fips +global: + security: + profile: fips ``` +Enabling the posture also takes two `https://` endpoints. The chart's in-cluster defaults for +both are plaintext `http://`, and the posture refuses them: + +```yaml +dataplane: + # Data Plane to Identity Service agent-key introspection. + identityServiceBaseURL: https:// +identityService: + productValidation: + products: + memory: + # Identity Service to Control Plane grant validation. + baseURL: https:// +``` + +The chart doesn't terminate Transport Layer Security (TLS) for either endpoint. Your hosting +environment must provide both, for example with a service mesh, ingress, or TLS-terminating +proxy in front of the Identity Service and Control Plane Services. The calling pods must trust +both certificates; the Data Plane trusts `tls.caCertSecret` (or `global.tls.caCertSecret`) on +top of the system CAs. + You can also apply the bundled FIPS values overlay with the normal values file. -Download the chart to get `values-fips.yaml` from the chart root: +Download the chart to get `values-fips.yaml` from the chart root, and set the two endpoints in +`ram-values.yaml` or on the command line: ```bash -helm pull redis-ai/redis-agent-memory --version 0.7.0 --untar +helm pull redis-ai/redis-agent-memory --version 1.0.0 --untar helm upgrade --install redis-agent-memory redis-ai/redis-agent-memory \ --version \ --namespace \ -f ram-values.yaml \ - -f redis-agent-memory/values-fips.yaml + -f redis-agent-memory/values-fips.yaml \ + --set dataplane.identityServiceBaseURL=https:// \ + --set identityService.productValidation.products.memory.baseURL=https:// ``` When enabled, the chart sets `GODEBUG=fips140=on` on the Data Plane, worker, @@ -215,12 +243,17 @@ customer's compliance boundary. When the posture is active, the Data Plane and worker reject config that: -- enables `skip_verify` on outbound HTTP clients; or +- enables `skip_verify` on outbound HTTP clients; +- sets `allow_insecure_transport` on any authentication block, even a disabled one; or - uses non-`rediss://` URLs for Redis connections covered by the posture. The Control Plane runs under the same posture and rejects non-`rediss://` `metadata.urls` or `databases."1".urls`. +The Identity Service sends the Control Plane internal token on every grant validation. Under +the posture it refuses to start unless each product validator URL uses `https://`, and the +chart refuses to render one that doesn't. + The Redis Agent Memory API listener itself speaks HTTP inside the cluster. Edge TLS termination is owned by the hosting environment, such as ingress, service mesh, or external load balancer. Outbound TLS to Redis, embedding providers, LLM providers, and @@ -247,7 +280,7 @@ the direct caller also has a valid Redis Agent Memory credential. The chart includes `networkpolicy.reference.yaml` at the chart root as a reference manifest. Get it with -`helm pull redis-ai/redis-agent-memory --version 0.7.0 --untar`. It is not templated because allowed callers are +`helm pull redis-ai/redis-agent-memory --version 1.0.0 --untar`. It is not templated because allowed callers are environment-specific. Customize the placeholders before applying it: diff --git a/content/operate/iris/agent-memory/self-managed/prerequisites.md b/content/operate/iris/agent-memory/self-managed/prerequisites.md index 8928d190db..f6108ff098 100644 --- a/content/operate/iris/agent-memory/self-managed/prerequisites.md +++ b/content/operate/iris/agent-memory/self-managed/prerequisites.md @@ -115,15 +115,18 @@ To supply a complete configuration file in your own Secret instead, see ## Release artifacts and image tags -Chart `0.7.0` defaults every image tag to `0.7.0`. Set an image tag only to pin a mirrored image. +Chart `1.0.0` requires `image.tag` and `controlplane.image.tag` at install or upgrade time; set +each to the Redis Agent Memory version you deploy. The bundled Identity Service needs no tag of +its own: it runs the image version the pinned `iris-identity-service` chart carries. Set +`identityService.image.*` only to mirror the image or pin another tag. Use the chart version supplied by Redis for the release. The published chart is `redis-ai/redis-agent-memory` from `https://helm.redis.io/ai`. Standard customer installs use the public Docker Hub images published by the -Redis Agent Memory self-managed release: `docker.io/redislabs/agent-memory:0.7.0`, -`docker.io/redislabs/agent-memory-control-plane:0.7.0`, and -`docker.io/redislabs/iris-identity-service:0.7.0`. +Redis Agent Memory self-managed release: `docker.io/redislabs/agent-memory:` and +`docker.io/redislabs/agent-memory-control-plane:`, plus +`docker.io/redislabs/iris-identity-service` at the version the pinned chart carries. ## Air-gapped and private registry installs @@ -131,28 +134,33 @@ Mirror these images into your internal registry: | Image | Needed | | --- | --- | -| `redislabs/agent-memory:0.7.0` | Always (Data Plane and workers) | -| `redislabs/agent-memory-control-plane:0.7.0` | Always | -| `redislabs/iris-identity-service:0.7.0` | Always | +| `redislabs/agent-memory:` | Always (Data Plane and workers) | +| `redislabs/agent-memory-control-plane:` | Always | +| `redislabs/iris-identity-service:` | With the bundled Identity Service (the default); skip only with `identityService.enabled=false` | | `replicated/troubleshoot:0.131.0` | Support-bundle health checks, on by default (`supportPackage.*`) | | `hibiken/asynqmon:0.7.2` | Optional, with `controlplane.queueMonitor` | | `python:3.13-alpine` | Optional, with `tests.smoke` | +`` is the `appVersion` of the `iris-identity-service` chart the Redis Agent Memory +chart pins. Read it with +`helm show chart redis-ai/redis-agent-memory --version ` and +`helm show chart redis-ai/iris-identity-service --version `. + ```bash -docker pull redislabs/agent-memory:0.7.0 -docker tag redislabs/agent-memory:0.7.0 \ - registry.example.com/redislabs/agent-memory:0.7.0 -docker push registry.example.com/redislabs/agent-memory:0.7.0 - -docker pull redislabs/agent-memory-control-plane:0.7.0 -docker tag redislabs/agent-memory-control-plane:0.7.0 \ - registry.example.com/redislabs/agent-memory-control-plane:0.7.0 -docker push registry.example.com/redislabs/agent-memory-control-plane:0.7.0 - -docker pull redislabs/iris-identity-service:0.7.0 -docker tag redislabs/iris-identity-service:0.7.0 \ - registry.example.com/redislabs/iris-identity-service:0.7.0 -docker push registry.example.com/redislabs/iris-identity-service:0.7.0 +docker pull redislabs/agent-memory: +docker tag redislabs/agent-memory: \ + registry.example.com/redislabs/agent-memory: +docker push registry.example.com/redislabs/agent-memory: + +docker pull redislabs/agent-memory-control-plane: +docker tag redislabs/agent-memory-control-plane: \ + registry.example.com/redislabs/agent-memory-control-plane: +docker push registry.example.com/redislabs/agent-memory-control-plane: + +docker pull redislabs/iris-identity-service: +docker tag redislabs/iris-identity-service: \ + registry.example.com/redislabs/iris-identity-service: +docker push registry.example.com/redislabs/iris-identity-service: ``` If the registry requires authentication, create an image pull Secret: @@ -178,12 +186,15 @@ identityService: image: repository: registry.example.com/redislabs/iris-identity-service -imagePullSecrets: - - name: ram-registry +global: + imagePullSecrets: + - name: ram-registry ``` -Omit `imagePullSecrets` if the internal registry does not require -authentication. +Use `global.imagePullSecrets`, which the chart and the bundled Identity Service both read; the +local `imagePullSecrets` reaches only the Redis Agent Memory pods, and the chart refuses it when +the Identity Service image is mirrored. Omit pull Secrets if the internal registry does not +require authentication. For air-gapped deployments, also set: @@ -257,8 +268,10 @@ namespace. | Secret rollouts | `license.existingSecretChecksum`, `config.existingSecretChecksum`, `controlplane.config.existingSecretChecksum` | Rolling pods after externally managed Secret changes. | | Control Plane | `controlplane.image.*`, `controlplane.configData`, `controlplane.adminToken.*` | Mirroring the Control Plane image, setting its configuration, or bringing your own admin token. | | Control Plane internal token | `controlplane.internalToken.*` | Bringing your own token for Identity Service checks against the Control Plane. | -| Identity Service | `identityService.*` | Mirroring the Identity Service image, naming its metadata Secret, or bringing your own control token. | -| Security profile | `security.profile` | Selecting the [FIPS-oriented posture](/content/operate/iris/agent-memory/self-managed/operations.md#fips-oriented-posture). | +| Identity Service (bundled) | `identityService.*` | The published `iris-identity-service` subchart's contract: naming its metadata Secret, mirroring its image, bringing your own control token (`identityService.control.*`), or adding service configuration (`identityService.configData`). | +| External Identity Service | `identityService.enabled=false`, `dataplane.identityServiceBaseURL`, `externalIdentityService.*` | Installing no Identity Service objects and pointing the Data Plane at one you run elsewhere. | +| Umbrella fallbacks | `global.*` | Sharing `security.profile`, `imagePullSecrets`, and `tls.caCertSecret` with the bundled Identity Service; the chart's own `security.profile`, `imagePullSecrets`, and `tls.caCertSecret` do not reach it. | +| Security profile | `global.security.profile` | Selecting the [FIPS-oriented posture](/content/operate/iris/agent-memory/self-managed/operations.md#fips-oriented-posture) for Redis Agent Memory and the bundled Identity Service. | | Support bundles | `supportPackage.*` | Configuring support-bundle collection and its health checks. | | Preflight checks | `preflight.*` | Rendering the preflight checks spec as a ConfigMap. | | Helm tests | `tests.*` | Running [chart tests](/content/operate/iris/agent-memory/self-managed/operations.md#chart-tests) with `helm test`. | diff --git a/content/operate/iris/agent-memory/self-managed/reference.md b/content/operate/iris/agent-memory/self-managed/reference.md index 0f1bf4970b..2b1363607e 100644 --- a/content/operate/iris/agent-memory/self-managed/reference.md +++ b/content/operate/iris/agent-memory/self-managed/reference.md @@ -46,19 +46,24 @@ CSI-synced Secrets, make sure a Control Plane pod also mounts the corresponding | Symptom | Likely cause | Fix | | --- | --- | --- | | `helm search repo redis-ai/redis-agent-memory --versions` returns no results | Helm repo not added/updated, or the chart version has not been published to the repo yet | Run `helm repo add`, `helm repo update`, or install from the chart package provided by Redis. | +| `helm template` or `helm upgrade` fails with `image.tag is required and must be set explicitly to the RAM version being deployed` | Chart 1.0.0 carries no default image tags | Set `image.tag` and `controlplane.image.tag` to the Redis Agent Memory version you deploy. The bundled Identity Service takes its tag from the pinned chart. | +| `helm template` or `helm upgrade` fails with `security.profile=fips does not reach the bundled Identity Service` | FIPS is set through the chart's local `security.profile`, which the bundled Identity Service does not read | Set `global.security.profile=fips` (the bundled `values-fips.yaml` does). See [FIPS-oriented posture](/content/operate/iris/agent-memory/self-managed/operations.md#fips-oriented-posture). | +| `helm template` or `helm upgrade` fails with `security.profile=fips requires dataplane.identityServiceBaseURL to be an explicit https:// URL` | FIPS is enabled and the Data Plane would use the chart's in-cluster `http://` Identity Service default | Set `dataplane.identityServiceBaseURL` to an `https://` Identity Service endpoint. See [FIPS-oriented posture](/content/operate/iris/agent-memory/self-managed/operations.md#fips-oriented-posture). | +| `helm template` or `helm upgrade` fails with `security.profile=fips requires identityService.productValidation.products.memory.baseURL to be https://` | FIPS is enabled and the memory product validator uses the chart's in-cluster `http://` Control Plane default, or an `http://` URL | Set `identityService.productValidation.products.memory.baseURL` to an `https://` Control Plane endpoint. See [FIPS-oriented posture](/content/operate/iris/agent-memory/self-managed/operations.md#fips-oriented-posture). | +| `helm template` or `helm upgrade` fails with `identityService.enabled=false with agent-key auth … requires dataplane.identityServiceBaseURL` | The bundled Identity Service is disabled, but nothing says how the Data Plane authenticates | Set `dataplane.identityServiceBaseURL` and `externalIdentityService.credential` for an external Identity Service, or set `memory.auth.method: none` behind an infrastructure access boundary. | | Docker pull fails for the configured image tag | Image tag is wrong or has not been published to the configured registry | Use the image tag listed for the release on Docker Hub or provided by Redis. | | Server and worker pods `CrashLoopBackOff`; log: `license validation failed; refusing to start: invalid license format` | `memory.license.license_path` is not set | See [CL-1](/content/operate/iris/agent-memory/self-managed/deploy.md#cl-1). | | Worker pods `CrashLoopBackOff`; log: `strategies.instruct.llm is required` | The `instruct.llm` block is missing | See [CL-2](/content/operate/iris/agent-memory/self-managed/deploy.md#cl-2). | | Pods `CrashLoopBackOff`; log: `provider "" is not defined in inference_providers` | An `llm.provider` doesn't name an `inference_providers` entry | See [CL-3](/content/operate/iris/agent-memory/self-managed/deploy.md#cl-3). | | Worker callback cannot resolve the Data Plane host | `memory.dataplane_client.base_url` is wrong | See [CL-4](/content/operate/iris/agent-memory/self-managed/deploy.md#cl-4). | -| Pod is stuck in `ImagePullBackOff` or `ErrImagePull` | Cluster cannot pull the configured image, image tag is wrong, registry requires credentials, or `imagePullSecrets` is missing/wrong | Verify `image.repository`, `image.tag`, registry reachability, and `imagePullSecrets`; use the Redis Agent Memory release image tag. See [CL-5](/content/operate/iris/agent-memory/self-managed/deploy.md#cl-5). | +| Pod is stuck in `ImagePullBackOff` or `ErrImagePull` | Cluster cannot pull the configured image, image tag is wrong, registry requires credentials, or `imagePullSecrets` is missing/wrong | Verify `image.repository`, `image.tag`, registry reachability, and `imagePullSecrets`; use the Redis Agent Memory release image tag. The bundled Identity Service reads `identityService.imagePullSecrets` or `global.imagePullSecrets`, not the chart's local `imagePullSecrets`. See [CL-5](/content/operate/iris/agent-memory/self-managed/deploy.md#cl-5). | | Worker callbacks rejected with `401` | `auth.worker_identity` doesn't match the worker, or the Data Plane cannot fetch the cluster JWKS | See [CL-6](/content/operate/iris/agent-memory/self-managed/deploy.md#cl-6). | | `helm install --atomic --wait` times out and rolls back | Cluster is small or image pull/startup takes longer than Helm's default timeout | Install without `--atomic --wait`, or set a longer `--timeout` and ensure enough cluster capacity. | | Pods are pending during install or upgrade | CPU/memory capacity is insufficient for default replicas and rollout overlap | Add nodes/headroom or lower replicas for test deployments. | | Data Plane health fails | Pod not ready, config invalid, Redis unavailable, or license invalid | Check pod logs and call `/health`, `/health/liveness`, and `/health/readiness`. | | Data Plane fails with `agent_keys.introspection.base_url is required` | Bring-your-own (BYO) Data Plane config uses agent-key auth, the default when no auth method is set, without the `auth.agent_keys.introspection` block | Add the `auth.agent_keys.introspection` block, or set `auth.method: none`. See [Configuration](/content/operate/iris/agent-memory/self-managed/configuration.md#bring-your-own-byo-configuration). | | Data Plane fails with `removed metadata configuration keys: …` | Config uses metadata keys that version 0.7.0 removed, such as `metadata.source` or `metadata.live` | Configure `metadata.urls`, `metadata.namespace`, and `databases`, then create stores through the Control Plane. | -| Pod fails to start in FIPS posture | A Redis URL is not `rediss://` or an outbound HTTP client uses `skip_verify: true` | Update Redis URLs and HTTP client config to satisfy the posture checks. | +| Pod fails to start in FIPS posture | A Redis URL is not `rediss://`, an outbound HTTP client uses `skip_verify: true`, an authentication block sets `allow_insecure_transport: true`, or the Identity Service logs `product_validation..base_url must be an https URL in FIPS profile` | Update Redis URLs, HTTP client config, authentication blocks, and the Identity Service product validator URL to satisfy the posture checks. The error names the exact config path. | | Agent receives `401` | Key is missing, revoked, expired, or wrong | Check `Authorization` / `X-Api-Key` and key status. | | Agent receives `503` | Identity Service is unreachable, or the Data Plane's miss budget for rejected keys is exhausted | Check that the Identity Service pod is Ready and reachable from the Data Plane. | | Agent receives `403` | Key exists but lacks the required store grant or action | Update grants with `PATCH http://localhost:9200/v1/api-keys/{keyId}` and the Identity Service control token. | @@ -74,7 +79,7 @@ CSI-synced Secrets, make sure a Control Plane pod also mounts the corresponding | Need | Reference | | --- | --- | -| Helm chart values | `values.yaml` at the chart root (`helm pull redis-ai/redis-agent-memory --version 0.7.0 --untar`) | +| Helm chart values | `values.yaml` at the chart root (`helm pull redis-ai/redis-agent-memory --version 1.0.0 --untar`) | | FIPS values overlay | `values-fips.yaml` at the chart root | | NetworkPolicy reference | `networkpolicy.reference.yaml` at the chart root | | Chart README | `README.md` at the chart root | From 6d6e397a8963d387335104c0db75f4d0ee3bbd4d Mon Sep 17 00:00:00 2001 From: Ivo Bardarov Date: Wed, 21 Oct 2026 16:10:44 +0300 Subject: [PATCH 53/76] docs: MOD-19813 install Memory with a standalone Identity Service --- .../iris/agent-memory/self-managed/_index.md | 11 +- .../iris/agent-memory/self-managed/deploy.md | 146 ++++++++++++++---- .../self-managed/prerequisites.md | 84 +++++----- 3 files changed, 166 insertions(+), 75 deletions(-) diff --git a/content/operate/iris/agent-memory/self-managed/_index.md b/content/operate/iris/agent-memory/self-managed/_index.md index d7dac7eab8..bfb8962722 100644 --- a/content/operate/iris/agent-memory/self-managed/_index.md +++ b/content/operate/iris/agent-memory/self-managed/_index.md @@ -31,17 +31,18 @@ documents the self-managed admin endpoints for stores. ## What you are deploying -The chart deploys the Data Plane, the workers, and the Control Plane, and bundles the Identity -Service by default. You provide the Redis databases. To point the Data Plane at an Identity -Service the chart does not install, see the `identityService.enabled=false` entries in -[Helm values to review](/content/operate/iris/agent-memory/self-managed/prerequisites.md#helm-values-to-review). +The `redis-agent-memory` chart `0.8.0` deploys the Data Plane, workers, and Control Plane. +Install the `iris-identity-service` chart `0.10.0` as a separate release first, then connect +Memory to it. Memory does not deploy Identity Service resources or include it as a subchart. +You provide the Redis databases. Follow [Deploy](/content/operate/iris/agent-memory/self-managed/deploy.md) +to configure both releases. | Component | Purpose | Default service | | --- | --- | --- | | Redis Agent Memory Data Plane | Store-scoped runtime memory API. | `redis-agent-memory:9000` | | Redis Agent Memory worker | Background promotion, summarization, and forgetting work. | No public service | | Redis Agent Memory Control Plane | Admin API for creating and managing stores. | `redis-agent-memory-controlplane:9100` | -| Identity Service (bundled) | Mints, rotates, revokes, and checks agent keys. | `redis-agent-memory-iris-identity-service:9200` | +| Identity Service (separate release) | Mints, rotates, revokes, and checks agent keys. | `ids-iris-identity-service:9200` for release `ids` | | Store Redis | Holds session memory, long-term memory, indexes, and TTL data. | Customer-provided | | Job Redis | Holds background work for Redis Agent Memory workers. | Customer-provided | | Metadata Redis | Holds Control Plane store records. The Identity Service keeps agent-key records in the Metadata Redis named in its own metadata Secret. | Customer-provided | diff --git a/content/operate/iris/agent-memory/self-managed/deploy.md b/content/operate/iris/agent-memory/self-managed/deploy.md index b1ef473773..4ce2657b04 100644 --- a/content/operate/iris/agent-memory/self-managed/deploy.md +++ b/content/operate/iris/agent-memory/self-managed/deploy.md @@ -21,9 +21,17 @@ aliases: These steps install Redis Agent Memory with chart-rendered configuration and agent-key authentication, and end with one authenticated call to the Data Plane. -You use three credentials along the way: the Control Plane admin token, the Identity Service -control token, and an agent key. The chart generates the first two. For what each credential -unlocks, see [Credentials](/content/operate/iris/agent-memory/self-managed/authentication.md#credentials). +Install Identity Service as release `ids`, then Memory as release `redis-agent-memory`. +These steps use Memory chart `0.8.0` and Identity Service chart `0.10.0` in one namespace. +You create the shared credentials and Identity Service control token; Memory generates its +Control Plane admin token. See [Credentials](/content/operate/iris/agent-memory/self-managed/authentication.md#credentials). + +Before installing, provide two Transport Layer Security (TLS) endpoints through your hosting +environment: one forwarding to `ids-iris-identity-service:9200`, and one forwarding to +`redis-agent-memory-controlplane:9100`. Neither chart terminates TLS. Replace the endpoint +placeholders with addresses reachable from the calling pods. Obtain their certificate authority +(CA) certificates as a PEM bundle named `service-ca.crt`; use your organization's trust bundle +if the endpoints use public certificates. Before you begin, review the [prerequisites](/content/operate/iris/agent-memory/self-managed/prerequisites.md). @@ -42,8 +50,21 @@ kubectl -n create secret generic ram-secrets \ --from-file=overlay.yaml=./overlay.yaml kubectl -n create secret generic ram-ids-metadata \ --from-file=metadata.yaml=./ids-metadata.yaml +kubectl -n create secret generic memory-dp-credential \ + --from-literal=token="$(openssl rand -hex 32)" +kubectl -n create secret generic memory-internal-token \ + --from-literal=token="$(openssl rand -hex 32)" +kubectl -n create secret generic ids-control-token \ + --from-literal=token="$(openssl rand -hex 32)" +kubectl -n create secret generic ram-ca \ + --from-file=ca.crt=./service-ca.crt ``` +Both releases mount `memory-dp-credential` and `memory-internal-token`. You own these Secrets; +neither release generates them in this procedure. The runtime credential lets Memory check +agent keys. The internal token lets Identity Service validate grants against Memory's Control +Plane. The `ids-control-token` Secret is separate and is used only by Identity Service. + `overlay.yaml` holds the Redis addresses and provider credentials. The Data Plane, the workers, and the Control Plane all mount it: @@ -75,6 +96,51 @@ metadata: urls: ["redis://:6379"] ``` +### Install Identity Service first + +Save this as `values-ids.yaml`: + +```yaml +metadata: + existingSecret: ram-ids-metadata +control: + existingSecret: ids-control-token + autoGenerate: false +runtime: + unscopedGrantsProduct: memory + apiKeyTenant: + singleTenantID: default + serviceCredentials: + - name: memory-dp + subject: memory-dp + allowedOperations: [api-key-introspect] + allowedProducts: [memory] + existingSecret: memory-dp-credential + secretKey: token +productValidation: + products: + memory: + baseURL: https:// + credential: + existingSecret: memory-internal-token + secretKey: token +tls: + caCertSecret: ram-ca +``` + +```bash +helm repo add redis-ai https://helm.redis.io/ai +helm repo update redis-ai +helm install ids redis-ai/iris-identity-service \ + --version 0.10.0 \ + --namespace \ + -f values-ids.yaml +``` + +The Memory Control Plane does not exist yet. Configure its endpoint now, but wait until both +releases and the TLS routes are ready before minting keys: Identity Service calls the Control +Plane to validate each store grant. + ### Prepare worker identity for your platform With agent-key authentication, workers call the Data Plane with their projected service account @@ -89,22 +155,24 @@ issuer is served: | Cluster | `issuer` | `jwks_uri` | Extra steps | Tested | |---------|----------|------------|-------------|--------| -| Self-managed (kind, kubeadm, on-premises) | from `kubectl get --raw /.well-known/openid-configuration` (kind: `https://kubernetes.default.svc.cluster.local`) | `https://kubernetes.default.svc/openid/v1/jwks` | `cluster-ca` Secret + `tls.caCertSecret: cluster-ca` + ClusterRoleBinding (cluster admin's decision) | Yes, on kind | +| Self-managed (kind, kubeadm, on-premises) | from `kubectl get --raw /.well-known/openid-configuration` (kind: `https://kubernetes.default.svc.cluster.local`) | `https://kubernetes.default.svc/openid/v1/jwks` | `ram-ca` Secret containing the cluster CA + `tls.caCertSecret: ram-ca` + ClusterRoleBinding (cluster admin's decision) | Yes, on kind | | Amazon Elastic Kubernetes Service (EKS) | `aws eks describe-cluster --name --query cluster.identity.oidc.issuer --output text` | the `jwks_uri` field of `/.well-known/openid-configuration` | Data Plane egress to the issuer host, or the `com.amazonaws..oidc-eks` PrivateLink endpoint with private DNS. The hostname differs for IPv6 clusters and AWS China, so copy it, don't type it | No | | Azure Kubernetes Service (AKS), OpenID Connect (OIDC) issuer enabled | `az aks show -n -g --query oidcIssuerProfile.issuerUrl -o tsv` (exact, including the trailing slash) | the `jwks_uri` field of `.well-known/openid-configuration` | Data Plane egress. The issuer is on by default only for new Standard clusters on 1.34+ and on AKS Automatic. `--enable-oidc-issuer` on an existing cluster restarts the API server and cannot be undone | No | | Google Kubernetes Engine (GKE) | as self-managed | as self-managed | as self-managed, because anonymous access to the GKE `jwks` endpoint is not verified | No | | Air-gapped (any) | as self-managed | as self-managed | as self-managed | Yes, on kind | Always take `jwks_uri` from the discovery document. Don't build it from the issuer, because the AKS -path differs. On EKS and AKS, replace `issuer` and `jwks_uri` in the values, remove `tls`, and skip -the rest of this section. +path differs. On EKS and AKS, replace `issuer` and `jwks_uri` in the values and skip +the rest of this section. Keep `tls.caCertSecret: ram-ca` for the service endpoints. -On self-managed, GKE, and air-gapped clusters, create a Secret from the cluster CA so the Data -Plane trusts the API server certificate: +On self-managed, GKE, and air-gapped clusters, add the cluster CA to the service CA bundle +so the Data Plane also trusts the API server certificate: ```bash kubectl -n get configmap kube-root-ca.crt -o jsonpath="{.data.ca\.crt}" >./cluster-ca.crt -kubectl -n create secret generic cluster-ca --from-file=ca.crt=./cluster-ca.crt +cat service-ca.crt cluster-ca.crt > ram-ca.crt +kubectl -n create secret generic ram-ca \ + --from-file=ca.crt=./ram-ca.crt --dry-run=client -o yaml | kubectl apply -f - ``` The Data Plane also needs to read the JWKS without logging in. Allowing that is the cluster @@ -123,6 +191,13 @@ On those clusters, there is no working setup for `agent_key` with workers. ## 3. Create `ram-values.yaml` ```yaml +tenantId: default +dataplane: + identityServiceBaseURL: https:// +identityService: + credential: + existingSecret: memory-dp-credential + secretKey: token license: existingSecret: ram-license config: @@ -178,6 +253,7 @@ memory: type: service_account_token token_file: /var/run/secrets/redis-agent-memory-worker/token auth: + method: agent_key worker_identity: enabled: true issuer: @@ -198,10 +274,12 @@ memory: session_summary_view: enabled: false image: - tag: + tag: "0.8.0" controlplane: + internalToken: + existingSecret: memory-internal-token image: - tag: + tag: "0.8.0" config: render: true configData: @@ -214,11 +292,8 @@ controlplane: license_path: /etc/redis-agent-memory/license embedding: dimensions: 1536 -identityService: - metadata: - existingSecret: ram-ids-metadata tls: - caCertSecret: cluster-ca + caCertSecret: ram-ca ``` - `` is any chat model the endpoint serves. @@ -229,12 +304,14 @@ tls: kubectl get --raw /.well-known/openid-configuration | jq -r .issuer ``` -- `tls.caCertSecret` names the CA Secret from [step 2](#prepare-worker-identity-for-your-platform). - Remove it on EKS and AKS. -- The chart writes the agent-key introspection settings and the Control Plane internal token for you. -- `` is the Redis Agent Memory version you deploy. `image.tag` and - `controlplane.image.tag` are required; the chart has no default. The bundled Identity Service - takes its image tag from the pinned chart, so it needs none. +- `tls.caCertSecret` names the CA bundle from [step 2](#2-create-secrets). +- The chart writes the agent-key introspection settings and mounts `memory-dp-credential` at + `/etc/identity-service/runtime/memory-dp/token`. Do not hand-set the introspection endpoint or credential under `memory.auth` or `shared.auth`. +- `tenantId: default` matches `runtime.apiKeyTenant.singleTenantID` in `values-ids.yaml`. + The Control Plane creates admin-token stores in this tenant. +- The published Memory chart defaults `image.tag` and `controlplane.image.tag` to `0.8.0`; + the example pins them explicitly. Identity Service uses its own chart's application version, + `0.10.0`, independently. {{< note >}} No external model provider? To try the API with local models, see [Evaluate locally](#evaluate-locally). @@ -252,7 +329,7 @@ its row. The rows are checklist items CL-1 to CL-6. | CL-2 | `memory.promote_session_memory.strategies.instruct.llm` is set | Worker pods `CrashLoopBackOff` (server stays Ready); log: `promote_session_memory: Strategies: strategies.instruct.llm is required.` | Add the `instruct.llm` block | | CL-3 | Each `llm.provider` names an `inference_providers` entry | Server and worker pods `CrashLoopBackOff`; log: `panic: config validation failed: promote_session_memory.strategies.instruct.llm: provider "" is not defined in inference_providers (defined: )` | Add `memory.inference_providers.` or fix the name | | CL-4 | `memory.dataplane_client.base_url` is `http://redis-agent-memory:9000` | Pods Ready, but no memories appear; worker log: `getting session memory: Get "http://:9000/v1/stores//session-memory/..."` … `dial tcp: lookup ` (the rest depends on the cluster DNS) | Set `http://redis-agent-memory:9000` | -| CL-5 | `image.tag` and `controlplane.image.tag` are set, and each tag exists on Docker Hub | `helm install` fails with `image.tag is required and must be set explicitly to the RAM version being deployed`; or server and worker pods `ImagePullBackOff`, event: `Failed to pull image "redislabs/agent-memory:": … not found` | Set both tags to the Redis Agent Memory version you deploy | +| CL-5 | Any overridden image tags exist in the chosen registry | Pods `ImagePullBackOff`, event: `Failed to pull image "redislabs/agent-memory:": … not found` | Use the published defaults or valid mirrored tags for each release | | CL-6 | `auth.worker_identity` matches the worker (issuer, audience, subject with your namespace) | Pods Ready, but no memories appear; worker log: `memory-dataplane API error: status 401`; the Data Plane logs `401` on `GET /v1/stores//session-memory/` with no reason | Fix `issuer` from `/.well-known/openid-configuration` and the namespace in `subject`. The same symptom appears when the Data Plane cannot fetch the cluster JWKS, so also check [worker identity for your platform](#prepare-worker-identity-for-your-platform) | {{< /table-scrollable >}} @@ -262,7 +339,7 @@ its row. The rows are checklist items CL-1 to CL-6. helm repo add redis-ai https://helm.redis.io/ai helm repo update redis-ai helm install redis-agent-memory redis-ai/redis-agent-memory \ - --version 1.0.0 \ + --version 0.8.0 \ --namespace \ -f ram-values.yaml kubectl -n get pods -w @@ -274,7 +351,7 @@ If you want Helm to wait, set an explicit timeout that matches the environment: ```bash helm install redis-agent-memory redis-ai/redis-agent-memory \ - --version 1.0.0 \ + --version 0.8.0 \ --namespace \ -f ram-values.yaml \ --wait \ @@ -287,8 +364,15 @@ helm install redis-agent-memory redis-ai/redis-agent-memory \ kubectl -n get pods -l app.kubernetes.io/name=redis-agent-memory ``` -The server, worker, `redis-agent-memory-controlplane`, and `redis-agent-memory-iris-identity-service` -pods are Ready. +The server, worker, and `redis-agent-memory-controlplane` pods must be Ready. +Check the separate Identity Service release too: + +```bash +kubectl -n get pods -l app.kubernetes.io/instance=ids +``` + +Wait for its pod to be Ready, and verify that both configured TLS endpoints route to their +Services before creating a key. Run each `port-forward` in this guide in its own terminal. Port-forward the Data Plane and check its health. The health check doesn't need an agent key: @@ -325,9 +409,9 @@ The response carries `storeId`. Use the Identity Service control token: ```bash -kubectl -n port-forward svc/redis-agent-memory-iris-identity-service 9200:9200 +kubectl -n port-forward svc/ids-iris-identity-service 9200:9200 IDS_CONTROL_TOKEN=$(kubectl -n get secret \ - redis-agent-memory-iris-identity-service-control-token -o jsonpath='{.data.token}' | base64 -d) + ids-control-token -o jsonpath='{.data.token}' | base64 -d) curl -sS -X POST http://localhost:9200/v1/api-keys \ -H "Authorization: Bearer $IDS_CONTROL_TOKEN" \ -H "Content-Type: application/json" \ @@ -335,7 +419,7 @@ curl -sS -X POST http://localhost:9200/v1/api-keys \ "name": "my-agent-key", "grants": [ { - "tenant": "", + "tenant": "default", "product": "memory", "resourceType": "mem-store", "resourceId": "", @@ -345,8 +429,8 @@ curl -sS -X POST http://localhost:9200/v1/api-keys \ }' ``` -Use the same tenant on every grant of a key. Any string is accepted. Use `1` if you manage stores with the Control Plane admin token. The response's `token` is the agent key. It is shown -only once. +Use the same tenant as `tenantId` and `runtime.apiKeyTenant.singleTenantID`, here `default`. +The response's `token` is the agent key. It is shown only once. ## 8. Make one authenticated call diff --git a/content/operate/iris/agent-memory/self-managed/prerequisites.md b/content/operate/iris/agent-memory/self-managed/prerequisites.md index f6108ff098..52153de1af 100644 --- a/content/operate/iris/agent-memory/self-managed/prerequisites.md +++ b/content/operate/iris/agent-memory/self-managed/prerequisites.md @@ -17,7 +17,8 @@ aliases: Redis Agent Memory is distributed as container images on Docker Hub plus a Helm chart in the Redis AI Helm repository. The chart deploys the Redis Agent Memory Data -Plane, Redis Agent Memory workers, the Redis Agent Memory Control Plane, and the Identity Service. +Plane, Redis Agent Memory workers, and the Redis Agent Memory Control Plane. Install Identity +Service separately with the `iris-identity-service` chart `0.10.0` before connecting Memory chart `0.8.0`. You provide the Redis databases, provider credentials, Kubernetes exposure, and license material used by the deployment. @@ -31,7 +32,7 @@ license material used by the deployment. | Item | Where it comes from | | ---- | ------------------- | | Container images | Docker Hub: `redislabs/agent-memory`, `redislabs/agent-memory-control-plane`, and `redislabs/iris-identity-service` | -| Helm chart | `redis-agent-memory` chart in the Redis AI Helm repository, or a chart package provided by Redis | +| Helm charts | `redis-agent-memory` `0.8.0` and `iris-identity-service` `0.10.0` in the Redis AI Helm repository | | Redis databases | You provide Store Redis, Job Redis, and Metadata Redis | | License key | Contact your Redis representative or [contact sales](https://redis.io/contact/). | | Provider credentials | You provide embedding provider credentials and, when worker features are enabled, promotion or summarization LLM credentials | @@ -93,13 +94,17 @@ Plane store records, and losing the Identity Service Metadata Redis removes agen chart package or locally downloaded chart. - **Runtime access:** Redis Agent Memory pods must reach the Redis databases and any embedding or LLM provider endpoints used by the deployment. +- **Identity Service:** the Data Plane must reach its runtime endpoint. Identity Service must + reach the Memory Control Plane's grant validator and its own Metadata Redis. Provide TLS + endpoints and trust for both service hops; see [Deploy](/content/operate/iris/agent-memory/self-managed/deploy.md). - **Data Plane exposure:** use NetworkPolicy, ingress, gateway, service mesh, private load balancer, or equivalent controls to restrict API access. ## Credentials and Secrets The chart renders the configuration from Helm values (`config.render: true`). Credentials stay out -of the values: you put them in an overlay Secret. You create three Secrets: +of the values: you put them in an overlay Secret. Create the following Secrets for the +[two-release installation](/content/operate/iris/agent-memory/self-managed/deploy.md#2-create-secrets): | Secret | Default key | Holds | | --- | --- | --- | @@ -107,26 +112,24 @@ of the values: you put them in an overlay Secret. You create three Secrets: | Overlay Secret | `overlay.yaml` | Redis URLs and provider credentials. | | Identity Service metadata Secret | `metadata.yaml` | The Identity Service's Metadata Redis URL. | -The chart generates four more Secrets for the tokens and service credentials listed in -[Credentials](/content/operate/iris/agent-memory/self-managed/authentication.md#credentials). +Also create `memory-dp-credential` and `memory-internal-token`, shared by both releases, +and `ids-control-token`, read only by Identity Service. The walkthrough supplies a `ram-ca` +certificate bundle for service trust. Memory generates its Control Plane admin token unless +you provide one. See [Credentials](/content/operate/iris/agent-memory/self-managed/authentication.md#credentials). To supply a complete configuration file in your own Secret instead, see [Bring-your-own (BYO) configuration](/content/operate/iris/agent-memory/self-managed/configuration.md#bring-your-own-byo-configuration). ## Release artifacts and image tags -Chart `1.0.0` requires `image.tag` and `controlplane.image.tag` at install or upgrade time; set -each to the Redis Agent Memory version you deploy. The bundled Identity Service needs no tag of -its own: it runs the image version the pinned `iris-identity-service` chart carries. Set -`identityService.image.*` only to mirror the image or pin another tag. +The published Memory chart `0.8.0` defaults `image.tag` and `controlplane.image.tag` to `0.8.0`. +Identity Service chart `0.10.0` defaults its image to its own application version, `0.10.0`. +Set tags explicitly when pinning images or using a mirror. The releases have independent versions. -Use the chart version supplied by Redis for the release. The published chart is -`redis-ai/redis-agent-memory` from `https://helm.redis.io/ai`. - -Standard customer installs use the public Docker Hub images published by the -Redis Agent Memory self-managed release: `docker.io/redislabs/agent-memory:` and -`docker.io/redislabs/agent-memory-control-plane:`, plus -`docker.io/redislabs/iris-identity-service` at the version the pinned chart carries. +Both charts are published at `https://helm.redis.io/ai` as `redis-ai/redis-agent-memory` +and `redis-ai/iris-identity-service`. Their standard images are +`redislabs/agent-memory:0.8.0`, `redislabs/agent-memory-control-plane:0.8.0`, and +`redislabs/iris-identity-service:0.10.0`. ## Air-gapped and private registry installs @@ -136,15 +139,15 @@ Mirror these images into your internal registry: | --- | --- | | `redislabs/agent-memory:` | Always (Data Plane and workers) | | `redislabs/agent-memory-control-plane:` | Always | -| `redislabs/iris-identity-service:` | With the bundled Identity Service (the default); skip only with `identityService.enabled=false` | +| `redislabs/iris-identity-service:` | On the separate Identity Service release, required for default agent-key authentication | | `replicated/troubleshoot:0.131.0` | Support-bundle health checks, on by default (`supportPackage.*`) | | `hibiken/asynqmon:0.7.2` | Optional, with `controlplane.queueMonitor` | | `python:3.13-alpine` | Optional, with `tests.smoke` | -`` is the `appVersion` of the `iris-identity-service` chart the Redis Agent Memory -chart pins. Read it with -`helm show chart redis-ai/redis-agent-memory --version ` and -`helm show chart redis-ai/iris-identity-service --version `. +For this release, `` is `0.8.0` and `` is `0.10.0`. +Inspect each package with `helm show chart redis-ai/redis-agent-memory --version 0.8.0` +and `helm show chart redis-ai/iris-identity-service --version 0.10.0`. Memory does not pin +or install the Identity Service chart. ```bash docker pull redislabs/agent-memory: @@ -182,19 +185,25 @@ controlplane: image: repository: registry.example.com/redislabs/agent-memory-control-plane -identityService: - image: - repository: registry.example.com/redislabs/iris-identity-service +global: + imagePullSecrets: + - name: ram-registry +``` +Configure the separate Identity Service release in `values-ids.yaml`: + +```yaml +image: + repository: registry.example.com/redislabs/iris-identity-service + tag: "0.10.0" global: imagePullSecrets: - name: ram-registry ``` -Use `global.imagePullSecrets`, which the chart and the bundled Identity Service both read; the -local `imagePullSecrets` reaches only the Redis Agent Memory pods, and the chart refuses it when -the Identity Service image is mirrored. Omit pull Secrets if the internal registry does not -require authentication. +Each release reads its own `imagePullSecrets`, falling back to `global.imagePullSecrets`. +Values passed to Memory do not configure Identity Service. Omit pull Secrets if the registry +does not require authentication. For air-gapped deployments, also set: @@ -203,16 +212,13 @@ airgap: enabled: true ``` -With `airgap.enabled: true`, the chart fails if a repository still points at Docker Hub. For the -Identity Service, the error is: - -```text -airgap.enabled=true requires identityService.image.repository to point to a mirrored registry reachable from the cluster -``` +Memory's `airgap.enabled: true` checks Memory's own image repositories. It does not validate +the separate Identity Service release. Mirror that release's image and chart dependencies +separately, and confirm that every configured repository is reachable from the cluster. ## System requirements -Default chart values: +Default values for Memory and the separate Identity Service chart: | Component | Default | Purpose | | --------- | ------- | ------- | @@ -268,10 +274,10 @@ namespace. | Secret rollouts | `license.existingSecretChecksum`, `config.existingSecretChecksum`, `controlplane.config.existingSecretChecksum` | Rolling pods after externally managed Secret changes. | | Control Plane | `controlplane.image.*`, `controlplane.configData`, `controlplane.adminToken.*` | Mirroring the Control Plane image, setting its configuration, or bringing your own admin token. | | Control Plane internal token | `controlplane.internalToken.*` | Bringing your own token for Identity Service checks against the Control Plane. | -| Identity Service (bundled) | `identityService.*` | The published `iris-identity-service` subchart's contract: naming its metadata Secret, mirroring its image, bringing your own control token (`identityService.control.*`), or adding service configuration (`identityService.configData`). | -| External Identity Service | `identityService.enabled=false`, `dataplane.identityServiceBaseURL`, `externalIdentityService.*` | Installing no Identity Service objects and pointing the Data Plane at one you run elsewhere. | -| Umbrella fallbacks | `global.*` | Sharing `security.profile`, `imagePullSecrets`, and `tls.caCertSecret` with the bundled Identity Service; the chart's own `security.profile`, `imagePullSecrets`, and `tls.caCertSecret` do not reach it. | -| Security profile | `global.security.profile` | Selecting the [FIPS-oriented posture](/content/operate/iris/agent-memory/self-managed/operations.md#fips-oriented-posture) for Redis Agent Memory and the bundled Identity Service. | +| Identity Service connection | `dataplane.identityServiceBaseURL`, `identityService.credential.*`, `identityService.allowInsecureTransport` | Connecting to the separately installed runtime endpoint. The credential Secret is registered on that release. | +| Identity Service configuration | Separate `values-ids.yaml`: `metadata`, `control`, `runtime`, `productValidation`, `subjectGrants` | Configuring the Identity Service release, not Memory. | +| Shared fallbacks | `global.security.profile`, `global.imagePullSecrets`, `global.tls.caCertSecret` | Providing fallback values within each release. Pass shared values to both releases; local values take precedence. | +| Security profile | `security.profile` or `global.security.profile` | Selecting the [FIPS-oriented posture](/content/operate/iris/agent-memory/self-managed/operations.md#fips-oriented-posture); configure it separately on each release. | | Support bundles | `supportPackage.*` | Configuring support-bundle collection and its health checks. | | Preflight checks | `preflight.*` | Rendering the preflight checks spec as a ConfigMap. | | Helm tests | `tests.*` | Running [chart tests](/content/operate/iris/agent-memory/self-managed/operations.md#chart-tests) with `helm test`. | From fd30d51fe2d60678f5a3c42dba6a4e54c30e4212 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Wed, 21 Oct 2026 06:26:05 -0700 Subject: [PATCH 54/76] Update content/operate/rc/api/api-reference/openapi.json (#4315) Co-authored-by: redisdocsapp[bot] <177626021+redisdocsapp[bot]@users.noreply.github.com> --- .../operate/rc/api/api-reference/openapi.json | 76 +++++++++---------- 1 file changed, 38 insertions(+), 38 deletions(-) diff --git a/content/operate/rc/api/api-reference/openapi.json b/content/operate/rc/api/api-reference/openapi.json index 50d388708a..a6088bc486 100644 --- a/content/operate/rc/api/api-reference/openapi.json +++ b/content/operate/rc/api/api-reference/openapi.json @@ -96,7 +96,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace", + "operationId": "proxyDataIntegrationWorkspace_4", "parameters": [ { "name": "subscriptionId", @@ -166,7 +166,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_2", + "operationId": "proxyDataIntegrationWorkspace", "parameters": [ { "name": "subscriptionId", @@ -379,7 +379,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_4", + "operationId": "proxyDataIntegrationWorkspace_2", "parameters": [ { "name": "subscriptionId", @@ -452,7 +452,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_5", + "operationId": "proxyDataIntegrationWorkspace_9", "parameters": [ { "name": "subscriptionId", @@ -522,7 +522,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_7", + "operationId": "proxyDataIntegrationWorkspace_5", "parameters": [ { "name": "subscriptionId", @@ -735,7 +735,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_9", + "operationId": "proxyDataIntegrationWorkspace_7", "parameters": [ { "name": "subscriptionId", @@ -8350,9 +8350,6 @@ "412": { "description": "Precondition Failed - Feature flag for this flow is off" }, - "401": { - "description": "Unauthorized - Authentication failed for requested resource" - }, "403": { "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" }, @@ -8366,9 +8363,6 @@ } } }, - "404": { - "description": "Not Found - The resource you were trying to reach was not found or does not exist" - }, "200": { "description": "Memory stores returned.", "content": { @@ -8379,6 +8373,12 @@ } } }, + "401": { + "description": "Unauthorized - Authentication failed for requested resource" + }, + "404": { + "description": "Not Found - The resource you were trying to reach was not found or does not exist" + }, "408": { "description": "Request Timeout - The server didn't receive a complete request message within the server's allotted timeout period" }, @@ -8490,9 +8490,6 @@ "412": { "description": "Precondition Failed - Feature flag for this flow is off" }, - "401": { - "description": "Unauthorized - Authentication failed for requested resource" - }, "403": { "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" }, @@ -8506,6 +8503,9 @@ } } }, + "401": { + "description": "Unauthorized - Authentication failed for requested resource" + }, "404": { "description": "Not Found - The resource you were trying to reach was not found or does not exist" }, @@ -8565,32 +8565,32 @@ "412": { "description": "Precondition Failed - Feature flag for this flow is off" }, - "401": { - "description": "Unauthorized - Authentication failed for requested resource" - }, "403": { "description": "Forbidden - Not allowed to create requested resource (primarily due to not active resource)" }, - "201": { - "description": "API key created. The generated key value is returned once in the response body.", + "424": { + "description": "Failed Dependency - The memory store's database is unavailable or out of memory, e.g. /errors/resource-unavailable or /errors/database-out-of-memory. When the error originates from the Agent Memory service the body is an application/problem+json document (MemoryProblemDetails) carrying the service's error type and detail", "content": { - "application/json": { + "application/problem+json": { "schema": { - "$ref": "#/components/schemas/MemoryStoreApiKeyCreateResponse" + "$ref": "#/components/schemas/MemoryProblemDetails" } } } }, - "424": { - "description": "Failed Dependency - The memory store's database is unavailable or out of memory, e.g. /errors/resource-unavailable or /errors/database-out-of-memory. When the error originates from the Agent Memory service the body is an application/problem+json document (MemoryProblemDetails) carrying the service's error type and detail", + "201": { + "description": "API key created. The generated key value is returned once in the response body.", "content": { - "application/problem+json": { + "application/json": { "schema": { - "$ref": "#/components/schemas/MemoryProblemDetails" + "$ref": "#/components/schemas/MemoryStoreApiKeyCreateResponse" } } } }, + "401": { + "description": "Unauthorized - Authentication failed for requested resource" + }, "404": { "description": "Not Found - The resource you were trying to reach was not found or does not exist" }, @@ -10079,12 +10079,6 @@ "412": { "description": "Precondition Failed - Feature flag for this flow is off" }, - "401": { - "description": "Unauthorized - Authentication failed for requested resource" - }, - "403": { - "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" - }, "200": { "description": "Memory store returned.", "content": { @@ -10095,6 +10089,9 @@ } } }, + "403": { + "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" + }, "424": { "description": "Failed Dependency - The memory store's database is unavailable or out of memory, e.g. /errors/resource-unavailable or /errors/database-out-of-memory. When the error originates from the Agent Memory service the body is an application/problem+json document (MemoryProblemDetails) carrying the service's error type and detail", "content": { @@ -10105,6 +10102,9 @@ } } }, + "401": { + "description": "Unauthorized - Authentication failed for requested resource" + }, "404": { "description": "Not Found - The resource you were trying to reach was not found or does not exist" }, @@ -12804,12 +12804,6 @@ "412": { "description": "Precondition Failed - Feature flag for this flow is off" }, - "204": { - "description": "API key deleted." - }, - "401": { - "description": "Unauthorized - Authentication failed for requested resource" - }, "403": { "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" }, @@ -12823,6 +12817,12 @@ } } }, + "204": { + "description": "API key deleted." + }, + "401": { + "description": "Unauthorized - Authentication failed for requested resource" + }, "404": { "description": "Not Found - The resource you were trying to reach was not found or does not exist" }, From 04a0aa18e522bdaad6389a6476cac97f526a413c Mon Sep 17 00:00:00 2001 From: Ivo Bardarov Date: Wed, 21 Oct 2026 16:40:24 +0300 Subject: [PATCH 55/76] docs: MOD-19813 align authentication and operations with separate releases --- .../self-managed/authentication.md | 27 +++- .../self-managed/configuration.md | 69 +++++++-- .../agent-memory/self-managed/operations.md | 136 +++++++++++------- .../agent-memory/self-managed/reference.md | 27 ++-- 4 files changed, 183 insertions(+), 76 deletions(-) diff --git a/content/operate/iris/agent-memory/self-managed/authentication.md b/content/operate/iris/agent-memory/self-managed/authentication.md index 72a67e0365..ad2934664c 100644 --- a/content/operate/iris/agent-memory/self-managed/authentication.md +++ b/content/operate/iris/agent-memory/self-managed/authentication.md @@ -25,13 +25,15 @@ grants. | Credential | Who presents it to whom | Secret / key | Auto-generated? | What it unlocks | | --- | --- | --- | --- | --- | | Control Plane admin token | Operator → Control Plane (`:9100`) | `redis-agent-memory-controlplane-admin-token` / `token` | Yes (`controlplane.adminToken.autoGenerate`) | Store management: `/v1/stores`, `/v1/detectors` | -| Control Plane internal token | Identity Service → Control Plane | `redis-agent-memory-controlplane-internal-token` / `token` | Yes (`controlplane.internalToken.autoGenerate`) | Checking that the store in a grant exists. Chart-managed; you handle it only with bring-your-own (BYO) configuration | -| Identity Service control token | Operator → Identity Service (`:9200`) | `redis-agent-memory-iris-identity-service-control-token` / `token` | Yes (`identityService.control.autoGenerate`) | Creating, listing, updating, rotating and revoking agent keys | -| Data Plane service credential | Data Plane → Identity Service | `redis-agent-memory-iris-identity-service-runtime-memory-dp` / `token` | Yes (`identityService.runtime.serviceCredentials`) | Checking agent keys for `memory`. Chart-managed; you handle it only with bring-your-own (BYO) configuration | +| Control Plane internal token | Identity Service → Control Plane | `memory-internal-token` / `token` in the walkthrough | You provision it for both releases | Checking that the store in a grant exists. Set `controlplane.internalToken.existingSecret` on Memory and `productValidation.products.memory.credential.existingSecret` on Identity Service. | +| Identity Service control token | Operator → Identity Service (`:9200`) | `ids-control-token` / `token` in the walkthrough | Supplied through `control.existingSecret` on Identity Service; that chart can also generate one | Creating, listing, updating, rotating and revoking agent keys | +| Data Plane service credential | Data Plane → Identity Service | `memory-dp-credential` / `token` in the walkthrough | You provision it for both releases | Checking agent keys for `memory`. Set `identityService.credential.existingSecret` on Memory and register it in `runtime.serviceCredentials[]` on Identity Service. | | Agent key | Application → Data Plane (`:9000`) | None; returned once when minted or rotated | No | The Data Plane operations its grants allow | -Each generated Secret keeps its value across upgrades (`helm.sh/resource-policy: keep`). To bring -your own, set `existingSecret` on the matching value. +Generated token Secrets are retained across upgrades and uninstalls (`helm.sh/resource-policy: keep`). +Use existing Secrets for shared credentials and for reproducible deployment automation. +Secret names and keys must agree between releases. See +[Secret rotation](/content/operate/iris/agent-memory/self-managed/operations.md#secret-rotation). ## Control Plane admin token @@ -63,7 +65,8 @@ The Data Plane defaults to agent-key authentication whenever no auth method is s The [`deploy` walkthrough](/content/operate/iris/agent-memory/self-managed/deploy.md) uses agent-key authentication from the start. In the chart-rendered configuration, the chart writes the agent-key -introspection settings for you, so you add nothing more. +introspection settings from `dataplane.identityServiceBaseURL` and `identityService.credential`. +These must point to the standalone release and its registered `memory-dp` credential. A bring-your-own (BYO) Data Plane configuration needs the `auth.agent_keys.introspection` block, including a BYO configuration without `auth`. See @@ -83,10 +86,16 @@ Treat agent keys as opaque credentials. Do not parse their contents. Use auth-disabled mode for development and testing only: ```yaml +profile: dev auth: method: none ``` +These are application configuration fields; put them under `memory` in Helm values. +The Data Plane refuses `auth.method: none` unless `profile` is exactly `dev`. +Chart-rendered configuration fails to render otherwise. With a supplied configuration Secret, +the same rule is enforced at process startup. + > [!WARNING] > Do not expose an auth-disabled Data Plane to untrusted callers. In auth-disabled > mode, Redis Agent Memory does not authenticate or authorize Data Plane requests; any @@ -116,7 +125,11 @@ A grant names the tenant, product, store, and actions: } ``` -Use the same tenant on every grant of a key. Any string is accepted. Use `1` if you manage stores with the Control Plane admin token. +Use the installation's tenant on every grant, for example `default`. Set Memory's `tenantId` +and Identity Service's `runtime.apiKeyTenant.singleTenantID` to that same value. With a rendered +Control Plane configuration, `tenantId` sets `auth.admin_tenant_id`, so admin-token stores belong +to that tenant too. With a supplied Control Plane configuration Secret, set `auth.admin_tenant_id` +yourself. Neither release checks the other release's tenant setting. Grant actions: diff --git a/content/operate/iris/agent-memory/self-managed/configuration.md b/content/operate/iris/agent-memory/self-managed/configuration.md index cdebd5f8b2..909d1e9835 100644 --- a/content/operate/iris/agent-memory/self-managed/configuration.md +++ b/content/operate/iris/agent-memory/self-managed/configuration.md @@ -14,8 +14,9 @@ aliases: - /operate/iris/agent-memory/self-managed/data-plane-configuration/ --- -The chart renders the Data Plane, Control Plane, and Identity Service configuration from Helm values -(`config.render: true`). Credentials stay out of the values, in the overlay Secret. For a complete +The Memory chart renders Data Plane and Control Plane configuration from Helm values +(`config.render: true` and `controlplane.config.render: true`). Identity Service uses its own +release and `values-ids.yaml`. Credentials stay in existing Secrets. For a complete example, see the values in [step 3 of Deploy](/content/operate/iris/agent-memory/self-managed/deploy.md#3-create-ram-valuesyaml). To supply complete configuration files in your own Secrets instead, see @@ -28,7 +29,8 @@ To supply complete configuration files in your own Secrets instead, see | `shared` | Settings common to the Data Plane and the Control Plane, such as `databases`. The chart merges them under `memory` and `controlplane.configData`. | | `memory` | The Data Plane and worker configuration file. | | `controlplane.configData` | The Control Plane configuration file. | -| `identityService.*` | The bundled Identity Service, from the published `iris-identity-service` chart; every key is that chart's values contract. `identityService.metadata.existingSecret` names the Secret with its Metadata Redis URL, and extra service configuration goes in `identityService.configData`. | +| `dataplane.identityServiceBaseURL`, `identityService.credential.*`, `identityService.allowInsecureTransport` | Memory's connection to the standalone Identity Service. These values do not configure that service. | +| Separate `values-ids.yaml` | Identity Service's `metadata`, `control`, `runtime`, `productValidation`, and optional `subjectGrants` values. | | Overlay Secret (`secrets.secretName`, key `overlay.yaml`) | `metadata.urls`, `metadata.namespace`, `databases."1".urls`, `background_jobs.redis.urls`, and provider `api_key` values. The Data Plane, the workers, and the Control Plane all read it. | ## Data Plane settings @@ -55,8 +57,16 @@ entry. The endpoint goes in `inference_providers..endpoint`, not in the `l In the chart-rendered configuration, the chart writes the agent-key introspection settings for you: it owns `auth.agent_keys.introspection.base_url`, `credential`, and `allow_insecure_transport`, and -refuses hand-set values. Set `dataplane.identityServiceBaseURL` to point at a non-default Identity -Service endpoint. Leave `auth.method` unset to keep the `agent_key` default. +refuses hand-set values in either `memory.auth` or `shared.auth`. Cache tuning under +`auth.agent_keys.introspection.cache` is still allowed. Set `dataplane.identityServiceBaseURL` +and `identityService.credential.existingSecret`; optional `secretKey` defaults to `token`, and +`existingSecretChecksum` triggers a rollout after credential rotation. Set `memory.auth.method: agent_key` +explicitly when combining agent keys with worker or other authentication methods. + +The runtime endpoint must use `https://`. An `http://` endpoint is accepted only when +`identityService.allowInsecureTransport: true` is set and the FIPS profile is not enabled. +Subjects using the `identity_service` authority also need this connection for grant lookup, +even when another authentication method is selected. ## Bring-your-own (BYO) configuration @@ -64,10 +74,16 @@ With BYO configuration, you supply each complete configuration file in your own chart injects nothing into it. Add the keys the chart would otherwise write: - **Data Plane**: the `auth.agent_keys.introspection` block. -- **Control Plane**: `auth.internal_token.token_file: /etc/controlplane-onprem/internal/token`. +- **Control Plane**: `auth.internal_token.token_file: /etc/controlplane-onprem/internal/token` + and `auth.admin_tenant_id` matching the installation's tenant. + +With `config.existingSecret`, the chart does not inspect the supplied file. Rendering cannot +catch missing introspection settings or an invalid development profile in that file; startup +validation still applies. Connection Secrets must still be named in Helm values so they are mounted. The Data Plane defaults to `agent_key` whenever no auth method is set. A BYO Data Plane configuration -without `auth` therefore needs the `auth.agent_keys.introspection` block, or `auth.method: none`. +without `auth` therefore needs the `auth.agent_keys.introspection` block. For development only, +`auth.method: none` requires exact `profile: dev`. The `MEM_AUTH_METHOD` environment variable overrides the method. ### Data Plane configuration file @@ -106,10 +122,10 @@ databases: - redis://:6379 auth: + method: agent_key agent_keys: introspection: - base_url: http://redis-agent-memory-iris-identity-service:9200 - allow_insecure_transport: true + base_url: https:// product: memory credential: token_file: /etc/identity-service/runtime/memory-dp/token @@ -192,6 +208,7 @@ auth: token_file: /etc/controlplane-onprem/admin/token internal_token: token_file: /etc/controlplane-onprem/internal/token + admin_tenant_id: default license: license_path: /etc/redis-agent-memory/license @@ -227,9 +244,43 @@ In `ram-values.yaml`, replace `config.render`, `shared`, `memory`, `controlplane `config.existingSecret` and `config.render` are set. ```yaml +tenantId: default config: existingSecret: ram-config +identityService: + credential: + existingSecret: memory-dp-credential controlplane: + internalToken: + existingSecret: memory-internal-token config: existingSecret: ram-controlplane-config ``` + +## Connect releases across namespaces + +For another namespace, resolve the standalone Service using its full address, such as +`ids-iris-identity-service..svc:9200`, and the Memory Control Plane as +`redis-agent-memory-controlplane..svc:9100`. Configure their TLS frontends +for the two connection URLs. With a naming override, use the rendered Service name. + +Pods can mount Secrets only from their own namespace. Provision the same runtime credential +and internal token values in both namespaces, plus each namespace's trust bundle. Allow Memory +to reach Identity Service and Identity Service to reach the Control Plane, including any +TLS proxies. See [Network policy](/content/operate/iris/agent-memory/self-managed/operations.md#network-policy). + +## Optional Playbook extraction + +On the Identity Service release, allow Memory subject grants for Playbook extraction: + +```yaml +subjectGrants: + allowed: + - product: memory + resourceType: mem-store + actions: [read, read_all_owners] +``` + +Also add `subject-grant-lookup` to the `memory-dp` runtime credential's `allowedOperations` +alongside `api-key-introspect`. These are Identity Service values, not Memory's `identityService` +values. `read_all_owners` is a subject-grant action; do not add it to agent-key grants. diff --git a/content/operate/iris/agent-memory/self-managed/operations.md b/content/operate/iris/agent-memory/self-managed/operations.md index 1e15bca9de..1b6bb2b536 100644 --- a/content/operate/iris/agent-memory/self-managed/operations.md +++ b/content/operate/iris/agent-memory/self-managed/operations.md @@ -45,7 +45,7 @@ only when a key is minted or rotated. Rotation returns a new `keyId` with the new token. Use the new `keyId` for later updates, rotations, and revocations. The old key stays valid for `graceSeconds`, which defaults to 3600 and has a chart maximum of 604800 -(`identityService.apiKeys.maxRotateGraceSeconds`). The response adds +(`apiKeys.maxRotateGraceSeconds` on the Identity Service release). The response adds `oldExpiresAt`, the time the old key stops working. Revoking the old `keyId` ends its grace period early. For the full request and response, see [API examples](/content/operate/iris/agent-memory/self-managed/api-examples.md#identity-service-api-examples). @@ -61,6 +61,20 @@ kubectl -n create secret generic redis-agent-memory-controlplan -o yaml | kubectl apply -f - ``` +The runtime credential and Control Plane internal token are shared by both releases. +When rotating either, keep the values equal in every namespace that mounts them, update the +corresponding checksums, and roll the affected workloads on both releases: + +| Credential | Memory checksum value | Identity Service checksum value | +| --- | --- | --- | +| `memory-dp-credential` | `identityService.credential.existingSecretChecksum` | `existingSecretChecksum` on the matching `runtime.serviceCredentials[]` entry | +| `memory-internal-token` | `controlplane.internalToken.existingSecretChecksum` | `productValidation.products.memory.credential.existingSecretChecksum` | + +Apply `ram-values.yaml` to Memory and `values-ids.yaml` to Identity Service separately. +Coordinate rotation: mismatched runtime credentials reject authentication, and mismatched +internal tokens prevent grant validation. Identity Service's operator control token is separate; +its checksum is `control.existingSecretChecksum` on that release. + Rotate the Redis Agent Memory license by updating the license Secret and changing `license.existingSecretChecksum` so Helm rolls the Data Plane, worker, and Control Plane pods. @@ -131,7 +145,12 @@ kubectl -n rollout restart deployment ## Updates -For every update: +Memory and Identity Service have independent release versions. Upgrade each with its own +values file and verify their shared connection contract afterward. A Memory upgrade does not +upgrade Identity Service. The published pair documented here is Memory `0.8.0` and Identity +Service `0.10.0`. + +For updates within a supported release path: 1. Update chart version and image tags. 2. Recalculate Secret checksums for changed files. @@ -150,6 +169,21 @@ helm upgrade redis-agent-memory redis-ai/redis-agent-memory \ On small clusters, avoid `--atomic` unless the timeout and capacity are known to be sufficient. +### Upgrade from a bundled Identity Service + +Direct `helm upgrade` from an earlier chart with bundled Identity Service, including Memory +`0.7.0`, to `0.8.0` is unsupported. Use a fresh installation. The following manual migration +outline is untested guidance, not a supported upgrade procedure: + +1. Back up store and agent-key metadata. Preserve the existing runtime credential, Control Plane + internal token, metadata Redis addresses, and key namespaces. +2. Provision the shared Secrets with those existing values and configure a standalone Identity + Service to use the same agent-key metadata. Do not let two independent releases manage one Secret. +3. Retain the tenant used by existing stores and keys. Earlier admin-token stores used tenant `1`; + set `tenantId` and `runtime.apiKeyTenant.singleTenantID` accordingly, or migrate metadata first. +4. Plan the replacement of the old Memory release and removal of bundled resources with Redis. + Keep backups and retained Secrets until the new installation and existing keys are verified. + ## Chart tests The chart can render optional `helm test` resources when `tests.enabled=true`. @@ -185,57 +219,62 @@ environments where the in-cluster test path is allowed. ## FIPS-oriented posture -The chart supports an opt-in FIPS-oriented posture for regulated environments. -Set it through `global.security.profile`, which the chart and the bundled Identity Service both -read. The chart's local `security.profile` does not reach the Identity Service, and the chart -refuses a FIPS Redis Agent Memory with a non-FIPS bundled Identity Service: +Both releases support an opt-in Federal Information Processing Standards (FIPS) posture. +Set `security.profile: fips` or its fallback `global.security.profile: fips` on each release. +A value passed only to Memory does not configure Identity Service. + +In `ram-values.yaml`: ```yaml -global: - security: - profile: fips +security: + profile: fips +dataplane: + identityServiceBaseURL: https:// +tls: + caCertSecret: ram-ca ``` -Enabling the posture also takes two `https://` endpoints. The chart's in-cluster defaults for -both are plaintext `http://`, and the posture refuses them: +In `values-ids.yaml`: ```yaml -dataplane: - # Data Plane to Identity Service agent-key introspection. - identityServiceBaseURL: https:// -identityService: - productValidation: - products: - memory: - # Identity Service to Control Plane grant validation. - baseURL: https:// +security: + profile: fips +productValidation: + products: + memory: + baseURL: https:// + credential: + existingSecret: memory-internal-token +tls: + caCertSecret: ram-ca ``` -The chart doesn't terminate Transport Layer Security (TLS) for either endpoint. Your hosting -environment must provide both, for example with a service mesh, ingress, or TLS-terminating -proxy in front of the Identity Service and Control Plane Services. The calling pods must trust -both certificates; the Data Plane trusts `tls.caCertSecret` (or `global.tls.caCertSecret`) on -top of the system CAs. +The Memory chart refuses `identityService.allowInsecureTransport: true` under FIPS. +Identity Service refuses to start with a non-HTTPS product validator URL under FIPS. +Neither chart terminates Transport Layer Security (TLS). Your hosting environment supplies +both endpoints through a service mesh, ingress, or TLS proxy. Memory trusts its runtime +endpoint through its own `tls.caCertSecret`; Identity Service trusts the validator endpoint +through that release's `tls.caCertSecret`, each in addition to system certificate authorities. -You can also apply the bundled FIPS values overlay with the normal values file. -Download the chart to get `values-fips.yaml` from the chart root, and set the two endpoints in -`ram-values.yaml` or on the command line: +Memory also ships a `values-fips.yaml` overlay. Apply it only to Memory, and configure the +Identity Service release's posture separately: ```bash -helm pull redis-ai/redis-agent-memory --version 1.0.0 --untar - -helm upgrade --install redis-agent-memory redis-ai/redis-agent-memory \ - --version \ +helm pull redis-ai/redis-agent-memory --version 0.8.0 --untar +helm upgrade redis-agent-memory redis-ai/redis-agent-memory \ + --version 0.8.0 \ --namespace \ -f ram-values.yaml \ - -f redis-agent-memory/values-fips.yaml \ - --set dataplane.identityServiceBaseURL=https:// \ - --set identityService.productValidation.products.memory.baseURL=https:// + -f redis-agent-memory/values-fips.yaml + +helm upgrade ids redis-ai/iris-identity-service \ + --version 0.10.0 \ + --namespace \ + -f values-ids.yaml ``` -When enabled, the chart sets `GODEBUG=fips140=on` on the Data Plane, worker, -Control Plane, and Identity Service pods and enables FIPS-oriented runtime -checks. The Identity Service also gets `IDS_SECURITY_PROFILE`. +Both charts set `GODEBUG=fips140=on` on their application containers and enable +FIPS-oriented runtime checks. This is not a formal FIPS 140 compliance or validation claim. Treat it as a deployment posture and guardrail that must still be reviewed against the @@ -251,8 +290,8 @@ The Control Plane runs under the same posture and rejects non-`rediss://` `metadata.urls` or `databases."1".urls`. The Identity Service sends the Control Plane internal token on every grant validation. Under -the posture it refuses to start unless each product validator URL uses `https://`, and the -chart refuses to render one that doesn't. +the posture it refuses to start unless each product validator URL uses `https://`. +Configure that URL on the Identity Service release, not under Memory's `identityService`. The Redis Agent Memory API listener itself speaks HTTP inside the cluster. Edge TLS termination is owned by the hosting environment, such as ingress, service mesh, or external @@ -280,15 +319,15 @@ the direct caller also has a valid Redis Agent Memory credential. The chart includes `networkpolicy.reference.yaml` at the chart root as a reference manifest. Get it with -`helm pull redis-ai/redis-agent-memory --version 1.0.0 --untar`. It is not templated because allowed callers are +`helm pull redis-ai/redis-agent-memory --version 0.8.0 --untar`. It is not templated because allowed callers are environment-specific. Customize the placeholders before applying it: - ``: namespace where Redis Agent Memory is installed. - ``: Helm release name. This guide uses `redis-agent-memory`. - The release name also prefixes release-derived service and deployment names. - `nameOverride` and `fullnameOverride` change rendered resource names, but the + `fullnameOverride` fixes Memory's default resource names to `redis-agent-memory`. + Naming overrides change rendered resource names, but the `app.kubernetes.io/instance` selector remains the Helm release name. - `` and caller pod labels: ingress controller, service mesh gateway, application pod, or approved internal caller allowed to call Redis Agent Memory. @@ -298,11 +337,12 @@ traffic to server pods on port `9000` from approved callers and the worker Deployment. It also includes a Control Plane stanza for port `9100`, which always applies. -The reference policy has no Identity Service stanza. Under default-deny, add -rules that allow: - -- Data Plane to Identity Service on port `9200`, for agent-key checks. -- Identity Service to Control Plane on port `9100`. +The reference includes Control Plane ingress from the separate Identity Service release. +Set `` and `` to its namespace and +release name (`ids` in the walkthrough). It does not manage ingress for Identity Service itself. +Under default-deny, allow Data Plane requests to that release on port `9200`, and Identity +Service requests to the Control Plane on port `9100`. If TLS proxies sit between them, allow +the proxy ports and actual source pods too. Add corresponding egress rules when egress is restricted. Review the manifest against the customer's CNI, ingress path, and service mesh behavior before production use. diff --git a/content/operate/iris/agent-memory/self-managed/reference.md b/content/operate/iris/agent-memory/self-managed/reference.md index 2b1363607e..933fe27f33 100644 --- a/content/operate/iris/agent-memory/self-managed/reference.md +++ b/content/operate/iris/agent-memory/self-managed/reference.md @@ -19,7 +19,8 @@ Use these files to configure a self-managed deployment: | File | Purpose | | --- | --- | -| `ram-values.yaml` | Helm values, including the chart-rendered Data Plane, Control Plane, and Identity Service configuration. See [step 3](/content/operate/iris/agent-memory/self-managed/deploy.md#3-create-ram-valuesyaml). | +| `ram-values.yaml` | Memory configuration and its connection to Identity Service. See [step 3](/content/operate/iris/agent-memory/self-managed/deploy.md#3-create-ram-valuesyaml). | +| `values-ids.yaml` | Separate Identity Service release configuration, including runtime credential registration, tenant, and Memory grant validation. | | `overlay.yaml` | Redis addresses and provider credentials. The Data Plane, the workers, and the Control Plane all mount it. | | `ids-metadata.yaml` | Tells the Identity Service where Metadata Redis is. | | `license` | Redis Agent Memory license file provided by Redis. | @@ -28,8 +29,8 @@ Use these files to configure a self-managed deployment: If you use an external secret manager, expose the license, config, admin-token, Identity Service metadata, Identity Service control token, Data Plane service credential, -and Control Plane internal token material to the chart as Kubernetes Secrets and set the chart's -`existingSecret` values to those Secret names. +and Control Plane internal token material as Kubernetes Secrets. Set `existingSecret` on the +owning release. Shared credentials must have matching contents in both releases' namespaces. Direct CSI file mounts that bypass Kubernetes Secrets are not supported for the Redis Agent Memory license, Data Plane config, Control Plane config, or Control Plane @@ -46,22 +47,23 @@ CSI-synced Secrets, make sure a Control Plane pod also mounts the corresponding | Symptom | Likely cause | Fix | | --- | --- | --- | | `helm search repo redis-ai/redis-agent-memory --versions` returns no results | Helm repo not added/updated, or the chart version has not been published to the repo yet | Run `helm repo add`, `helm repo update`, or install from the chart package provided by Redis. | -| `helm template` or `helm upgrade` fails with `image.tag is required and must be set explicitly to the RAM version being deployed` | Chart 1.0.0 carries no default image tags | Set `image.tag` and `controlplane.image.tag` to the Redis Agent Memory version you deploy. The bundled Identity Service takes its tag from the pinned chart. | -| `helm template` or `helm upgrade` fails with `security.profile=fips does not reach the bundled Identity Service` | FIPS is set through the chart's local `security.profile`, which the bundled Identity Service does not read | Set `global.security.profile=fips` (the bundled `values-fips.yaml` does). See [FIPS-oriented posture](/content/operate/iris/agent-memory/self-managed/operations.md#fips-oriented-posture). | -| `helm template` or `helm upgrade` fails with `security.profile=fips requires dataplane.identityServiceBaseURL to be an explicit https:// URL` | FIPS is enabled and the Data Plane would use the chart's in-cluster `http://` Identity Service default | Set `dataplane.identityServiceBaseURL` to an `https://` Identity Service endpoint. See [FIPS-oriented posture](/content/operate/iris/agent-memory/self-managed/operations.md#fips-oriented-posture). | -| `helm template` or `helm upgrade` fails with `security.profile=fips requires identityService.productValidation.products.memory.baseURL to be https://` | FIPS is enabled and the memory product validator uses the chart's in-cluster `http://` Control Plane default, or an `http://` URL | Set `identityService.productValidation.products.memory.baseURL` to an `https://` Control Plane endpoint. See [FIPS-oriented posture](/content/operate/iris/agent-memory/self-managed/operations.md#fips-oriented-posture). | -| `helm template` or `helm upgrade` fails with `identityService.enabled=false with agent-key auth … requires dataplane.identityServiceBaseURL` | The bundled Identity Service is disabled, but nothing says how the Data Plane authenticates | Set `dataplane.identityServiceBaseURL` and `externalIdentityService.credential` for an external Identity Service, or set `memory.auth.method: none` behind an infrastructure access boundary. | +| Rendering fails because `identityService` has an additional property | Memory values still use the removed bundled contract, such as `identityService.enabled` or `identityService.runtime` | Move service configuration to `values-ids.yaml`. Memory accepts only `identityService.credential` and `identityService.allowInsecureTransport`; remove `externalIdentityService` too. | +| Rendering fails with `requires dataplane.identityServiceBaseURL` | Agent-key authentication or an `identity_service` subject needs the standalone runtime endpoint | Set `dataplane.identityServiceBaseURL` and `identityService.credential.existingSecret`. See [Deploy](/content/operate/iris/agent-memory/self-managed/deploy.md). | +| Rendering fails with `requires identityService.credential.existingSecret` | The endpoint is configured but the runtime credential Secret is missing | Name the existing Secret also registered on Identity Service as `memory-dp`. | +| Rendering fails with `must be https://` or `identityService.allowInsecureTransport=true is refused under the fips security profile` | The runtime endpoint uses HTTP without an allowed exception, or insecure transport is enabled under FIPS | Use an HTTPS endpoint. An HTTP exception requires `identityService.allowInsecureTransport: true` and is refused under FIPS. | +| Rendering fails with `is chart-owned` for an introspection field | An endpoint, credential, or transport flag was hand-set under `memory.auth` or `shared.auth` | Use `dataplane.identityServiceBaseURL` and `identityService.credential` for chart-rendered configuration; cache settings remain configurable. | +| Rendering or startup refuses `auth.method: none` | Authentication is disabled outside the development profile | Use agent-key authentication. For isolated development only, set both `memory.profile: dev` and `memory.auth.method: none`; supplied configuration files use `profile: dev` and `auth.method: none`. | | Docker pull fails for the configured image tag | Image tag is wrong or has not been published to the configured registry | Use the image tag listed for the release on Docker Hub or provided by Redis. | | Server and worker pods `CrashLoopBackOff`; log: `license validation failed; refusing to start: invalid license format` | `memory.license.license_path` is not set | See [CL-1](/content/operate/iris/agent-memory/self-managed/deploy.md#cl-1). | | Worker pods `CrashLoopBackOff`; log: `strategies.instruct.llm is required` | The `instruct.llm` block is missing | See [CL-2](/content/operate/iris/agent-memory/self-managed/deploy.md#cl-2). | | Pods `CrashLoopBackOff`; log: `provider "" is not defined in inference_providers` | An `llm.provider` doesn't name an `inference_providers` entry | See [CL-3](/content/operate/iris/agent-memory/self-managed/deploy.md#cl-3). | | Worker callback cannot resolve the Data Plane host | `memory.dataplane_client.base_url` is wrong | See [CL-4](/content/operate/iris/agent-memory/self-managed/deploy.md#cl-4). | -| Pod is stuck in `ImagePullBackOff` or `ErrImagePull` | Cluster cannot pull the configured image, image tag is wrong, registry requires credentials, or `imagePullSecrets` is missing/wrong | Verify `image.repository`, `image.tag`, registry reachability, and `imagePullSecrets`; use the Redis Agent Memory release image tag. The bundled Identity Service reads `identityService.imagePullSecrets` or `global.imagePullSecrets`, not the chart's local `imagePullSecrets`. See [CL-5](/content/operate/iris/agent-memory/self-managed/deploy.md#cl-5). | +| Pod is stuck in `ImagePullBackOff` or `ErrImagePull` | Cluster cannot pull the configured image, image tag is wrong, registry requires credentials, or `imagePullSecrets` is missing/wrong | Verify `image.repository`, `image.tag`, registry reachability, and `imagePullSecrets`; use the Redis Agent Memory release image tag. Configure Identity Service's image and pull Secrets on its own release. See [CL-5](/content/operate/iris/agent-memory/self-managed/deploy.md#cl-5). | | Worker callbacks rejected with `401` | `auth.worker_identity` doesn't match the worker, or the Data Plane cannot fetch the cluster JWKS | See [CL-6](/content/operate/iris/agent-memory/self-managed/deploy.md#cl-6). | | `helm install --atomic --wait` times out and rolls back | Cluster is small or image pull/startup takes longer than Helm's default timeout | Install without `--atomic --wait`, or set a longer `--timeout` and ensure enough cluster capacity. | | Pods are pending during install or upgrade | CPU/memory capacity is insufficient for default replicas and rollout overlap | Add nodes/headroom or lower replicas for test deployments. | | Data Plane health fails | Pod not ready, config invalid, Redis unavailable, or license invalid | Check pod logs and call `/health`, `/health/liveness`, and `/health/readiness`. | -| Data Plane fails with `agent_keys.introspection.base_url is required` | Bring-your-own (BYO) Data Plane config uses agent-key auth, the default when no auth method is set, without the `auth.agent_keys.introspection` block | Add the `auth.agent_keys.introspection` block, or set `auth.method: none`. See [Configuration](/content/operate/iris/agent-memory/self-managed/configuration.md#bring-your-own-byo-configuration). | +| Data Plane fails with `agent_keys.introspection.base_url is required` | Bring-your-own (BYO) Data Plane config uses agent-key auth, the default when no auth method is set, without the `auth.agent_keys.introspection` block | Add the `auth.agent_keys.introspection` block. Only isolated development can use `auth.method: none` with `profile: dev`. See [Configuration](/content/operate/iris/agent-memory/self-managed/configuration.md#bring-your-own-byo-configuration). | | Data Plane fails with `removed metadata configuration keys: …` | Config uses metadata keys that version 0.7.0 removed, such as `metadata.source` or `metadata.live` | Configure `metadata.urls`, `metadata.namespace`, and `databases`, then create stores through the Control Plane. | | Pod fails to start in FIPS posture | A Redis URL is not `rediss://`, an outbound HTTP client uses `skip_verify: true`, an authentication block sets `allow_insecure_transport: true`, or the Identity Service logs `product_validation..base_url must be an https URL in FIPS profile` | Update Redis URLs, HTTP client config, authentication blocks, and the Identity Service product validator URL to satisfy the posture checks. The error names the exact config path. | | Agent receives `401` | Key is missing, revoked, expired, or wrong | Check `Authorization` / `X-Api-Key` and key status. | @@ -69,7 +71,7 @@ CSI-synced Secrets, make sure a Control Plane pod also mounts the corresponding | Agent receives `403` | Key exists but lacks the required store grant or action | Update grants with `PATCH http://localhost:9200/v1/api-keys/{keyId}` and the Identity Service control token. | | Store created by the Control Plane is not visible to the Data Plane | Control Plane and Data Plane point at different Metadata Redis URLs or namespaces | Make `metadata.urls` / `metadata.namespace` match on the Control Plane and the Data Plane. | | Agent key is rejected by the Data Plane | Data Plane cannot reach the Identity Service, the key was rotated or revoked, or the key secret is wrong | Check Identity Service reachability and, with BYO configuration only, `auth.agent_keys.introspection`. After a rotate or revoke, pods can disagree for up to 5 minutes. Send the latest credential returned by mint or rotate. | -| Minting an agent key returns `400` for a grant | Unknown store ID, grant lacks `product: "memory"`, or invalid grant shape | Create the store first, because unknown stores fail the Control Plane check. With BYO Control Plane config, also set `auth.internal_token.token_file: /etc/controlplane-onprem/internal/token`. Use `product: "memory"`, `resourceType: "mem-store"`, and `actions: ["read"]`, `["write"]`, or both. | +| Minting an agent key returns `400` for a grant | Unknown store ID, mismatched tenant, missing `product: "memory"`, or invalid grant shape | Create the store first, match `tenantId` with Identity Service's `runtime.apiKeyTenant.singleTenantID`, and verify its `productValidation.products.memory` endpoint and internal-token Secret match Memory. With BYO Control Plane config, also set `auth.internal_token.token_file: /etc/controlplane-onprem/internal/token`. Use `product: "memory"`, `resourceType: "mem-store"`, and `actions: ["read"]`, `["write"]`, or both. | | Worker jobs fail after agent-key auth is enabled | Worker callback request has no accepted credential, the projected token is not mounted, or `auth.worker_identity` does not trust the worker subject/audience/issuer | Enable `workerAuth`, set `dataplane_client.auth.type=service_account_token`, and configure `auth.worker_identity.subjects` with the worker ServiceAccount subject and required store grants. | | Gateway path succeeds but direct external path also works | Data Plane is reachable outside the intended gateway path | Add NetworkPolicy, ingress, service mesh, or load balancer controls. | | NetworkPolicy blocks expected traffic | Placeholder namespace, release name, or caller selectors were not customized correctly | Check the Helm release label `app.kubernetes.io/instance`, caller namespace, and caller pod labels. | @@ -79,7 +81,8 @@ CSI-synced Secrets, make sure a Control Plane pod also mounts the corresponding | Need | Reference | | --- | --- | -| Helm chart values | `values.yaml` at the chart root (`helm pull redis-ai/redis-agent-memory --version 1.0.0 --untar`) | +| Helm chart values | `values.yaml` at the chart root (`helm pull redis-ai/redis-agent-memory --version 0.8.0 --untar`) | +| Identity Service chart values | `helm show values redis-ai/iris-identity-service --version 0.10.0` | | FIPS values overlay | `values-fips.yaml` at the chart root | | NetworkPolicy reference | `networkpolicy.reference.yaml` at the chart root | | Chart README | `README.md` at the chart root | From 1edb30458d555ac910df0b4f753925b687386c86 Mon Sep 17 00:00:00 2001 From: Ivo Bardarov Date: Wed, 21 Oct 2026 18:51:10 +0300 Subject: [PATCH 56/76] docs: MOD-19813 fix deployed documentation report findings --- .../iris/agent-memory/self-managed/_index.md | 6 +- .../self-managed/configuration.md | 5 ++ .../iris/agent-memory/self-managed/deploy.md | 85 +++++++++++++++++-- .../agent-memory/self-managed/operations.md | 31 ++++++- .../self-managed/prerequisites.md | 9 +- .../agent-memory/self-managed/reference.md | 3 +- 6 files changed, 122 insertions(+), 17 deletions(-) diff --git a/content/operate/iris/agent-memory/self-managed/_index.md b/content/operate/iris/agent-memory/self-managed/_index.md index bfb8962722..f29b360132 100644 --- a/content/operate/iris/agent-memory/self-managed/_index.md +++ b/content/operate/iris/agent-memory/self-managed/_index.md @@ -69,8 +69,8 @@ and the Identity Service handles agent keys. ### API surfaces -All Data Plane APIs except `/health` are scoped to a store. A store is the logical isolation -boundary for memory data. +Memory data APIs are scoped to a store, the logical isolation boundary for memory data. +Service liveness and readiness endpoints are not store-scoped. When agent-key authentication is enabled, the grants on each agent key determine which stores and actions the key can access. @@ -82,7 +82,7 @@ actions the key can access. | Namespaces | `/v1/stores/{storeId}/namespaces` | Namespace resources within a store. | | Model Context Protocol (MCP) | `/v1/stores/{storeId}/mcp` | The MCP streamable-HTTP endpoint for a store. | | Store health | `/v1/stores/{storeId}/health` | Read-only health for store-scoped features. | -| Service health | `/health` | Operational status of the Data Plane. | +| Service health | `/health/liveness`, `/health/readiness` | Process liveness and readiness to serve requests; neither requires an agent key. | | Control Plane | `/v1/stores`, `/v1/detectors` | Self-managed administration for stores, and the catalog of built-in sensitive-data detectors. | | Identity Service | `/v1/api-keys` | Self-managed administration for agent keys. | diff --git a/content/operate/iris/agent-memory/self-managed/configuration.md b/content/operate/iris/agent-memory/self-managed/configuration.md index 909d1e9835..b0fd76882b 100644 --- a/content/operate/iris/agent-memory/self-managed/configuration.md +++ b/content/operate/iris/agent-memory/self-managed/configuration.md @@ -48,6 +48,7 @@ Data Plane settings go under `memory`, or at the top level of a BYO Data Plane c | `license.license_path` | Path where the license Secret is mounted. | | `request_region` | Region used for background work routing. | | `background_jobs.redis` | Job Redis connection used by background workers. | +| `background_jobs.idempotency` | Enable execution idempotency and extraction serialization for production workers. Requires `enabled: true` and `urls`; it can share Job Redis. | | `embedders_connection_details` | Embedding provider endpoint and credentials. | | `dataplane_client` | Worker callback client configuration. | | `promote_session_memory` | Promotion strategy and LLM connection used by workers. | @@ -103,6 +104,10 @@ request_region: default: eu1 background_jobs: + idempotency: + enabled: true + urls: + - redis://:6379 redis: enabled: true queue_prefix: ram diff --git a/content/operate/iris/agent-memory/self-managed/deploy.md b/content/operate/iris/agent-memory/self-managed/deploy.md index 4ce2657b04..cbbf9b3c71 100644 --- a/content/operate/iris/agent-memory/self-managed/deploy.md +++ b/content/operate/iris/agent-memory/self-managed/deploy.md @@ -35,6 +35,30 @@ if the endpoints use public certificates. Before you begin, review the [prerequisites](/content/operate/iris/agent-memory/self-managed/prerequisites.md). +### Isolated lab without TLS + +For a network-isolated lab without the Federal Information Processing Standards (FIPS) profile, +you can use the in-cluster HTTP Services instead of TLS frontends. In `ram-values.yaml`, change: + +```yaml +dataplane: + identityServiceBaseURL: http://ids-iris-identity-service:9200 +identityService: + allowInsecureTransport: true + credential: + existingSecret: memory-dp-credential +``` + +In `values-ids.yaml`, change the validator URL to +`productValidation.products.memory.baseURL: http://redis-agent-memory-controlplane:9100`. +Keep the other settings in both files. Tokens travel over plaintext HTTP on these two hops; +use TLS for production. FIPS refuses this configuration. + +For this lab option, skip `service-ca.crt` and the initial `ram-ca` creation in step 2, and omit +`tls` from `values-ids.yaml`. For self-managed worker identity, still create `ram-ca` from the +cluster CA as described in [Prepare worker identity](#prepare-worker-identity-for-your-platform). +On EKS and AKS, omit Memory's `tls` block too unless another connection requires a private CA. + ## 1. Create the namespace ```bash @@ -57,7 +81,7 @@ kubectl -n create secret generic memory-internal-token \ kubectl -n create secret generic ids-control-token \ --from-literal=token="$(openssl rand -hex 32)" kubectl -n create secret generic ram-ca \ - --from-file=ca.crt=./service-ca.crt + --from-file=ca.crt=./service-ca.crt --save-config ``` Both releases mount `memory-dp-credential` and `memory-internal-token`. You own these Secrets; @@ -77,6 +101,8 @@ databases: background_jobs: redis: urls: ["redis://:6379"] + idempotency: + urls: ["redis://:6379"] embedders_connection_details: openai: credentials: @@ -163,7 +189,9 @@ issuer is served: Always take `jwks_uri` from the discovery document. Don't build it from the issuer, because the AKS path differs. On EKS and AKS, replace `issuer` and `jwks_uri` in the values and skip -the rest of this section. Keep `tls.caCertSecret: ram-ca` for the service endpoints. +the rest of this section. With TLS service endpoints, keep `tls.caCertSecret: ram-ca` +for their trust bundle. For the HTTP-only lab option, omit this block unless another connection +requires a private CA. On self-managed, GKE, and air-gapped clusters, add the cluster CA to the service CA bundle so the Data Plane also trusts the API server certificate: @@ -175,6 +203,9 @@ kubectl -n create secret generic ram-ca \ --from-file=ca.crt=./ram-ca.crt --dry-run=client -o yaml | kubectl apply -f - ``` +For the HTTP-only lab option, use `cp cluster-ca.crt ram-ca.crt` instead of the `cat` command; +there is no service CA file to combine. + The Data Plane also needs to read the JWKS without logging in. Allowing that is the cluster admin's decision: @@ -240,6 +271,8 @@ memory: models: default_chat_model: background_jobs: + idempotency: + enabled: true redis: enabled: true queue_prefix: ram @@ -298,6 +331,11 @@ tls: - `` is any chat model the endpoint serves. - `controlplane.configData.embedding.dimensions` must equal `memory.embedding.models.dimensions`. +- `background_jobs.idempotency.enabled` prevents successful jobs from running again after + redelivery and serializes extraction for the same session. The walkthrough enables it for + workers and supplies its Redis URL through the overlay. Keep it enabled in production; + without it, workers warn that duplicate memories remain possible. This is separate from + queue submission deduplication. - `` is the cluster's issuer. Read it with: ```bash @@ -371,21 +409,22 @@ Check the separate Identity Service release too: kubectl -n get pods -l app.kubernetes.io/instance=ids ``` -Wait for its pod to be Ready, and verify that both configured TLS endpoints route to their -Services before creating a key. +Wait for its pod to be Ready. With TLS frontends, verify that both configured endpoints route +to their Services before creating a key. For the HTTP-only lab option, verify that both +in-cluster Service addresses are reachable from the calling pods. Run each `port-forward` in this guide in its own terminal. Port-forward the Data Plane and check its health. The health check doesn't need an agent key: ```bash kubectl -n port-forward svc/redis-agent-memory 9000:9000 -curl http://localhost:9000/health +curl http://localhost:9000/health/readiness ``` The response is: ```json -{"status":"healthy"} +{"status":"ready"} ``` ## 6. Create a store @@ -515,7 +554,14 @@ kubectl -n apply -f ollama.yaml The first start downloads about 2 GB. -In `ram-values.yaml`, replace the provider blocks with: +Edit the existing `ram-values.yaml`: replace the complete `memory.embedding`, +`memory.embedders_connection_details`, `memory.inference_providers`, and +`memory.promote_session_memory` maps with the corresponding maps in this snippet. Also update +`controlplane.configData.embedding.dimensions`. Keep the other settings, including authentication, +worker callbacks, and `background_jobs.idempotency`. + +Apply these edits to the existing file. Passing the snippet as another `-f` file merges the maps +and retains old OpenAI entries. ```yaml memory: @@ -553,7 +599,30 @@ controlplane: `api_key: ollama` is a placeholder. The client requires a key, and Ollama ignores it. -In `overlay.yaml`, remove `embedders_connection_details.openai` and both `api_key` entries. +Replace `overlay.yaml` with the Redis-only overlay, removing the complete +`embedders_connection_details` and `promote_session_memory` blocks: + +```yaml +metadata: + urls: ["redis://:6379"] +databases: + "1": + urls: ["redis://:6379"] +background_jobs: + redis: + urls: ["redis://:6379"] + idempotency: + urls: ["redis://:6379"] +``` + +If Memory is already installed, update the `ram-secrets` Secret from this file, upgrade with +the edited values, and restart the Data Plane, workers, and Control Plane so they reload the +overlay. See [Secret rotation](/content/operate/iris/agent-memory/self-managed/operations.md#secret-rotation). + +Use this provider switch only in a disposable evaluation installation. Changing the settings +does not migrate existing stores or their vector indexes. After switching, create a new store +and agent key using steps 6–7 for local evaluation. Existing stores retain their original +embedding dimensions and cannot be reused with this example. ## Next steps diff --git a/content/operate/iris/agent-memory/self-managed/operations.md b/content/operate/iris/agent-memory/self-managed/operations.md index 1b6bb2b536..585e6bef46 100644 --- a/content/operate/iris/agent-memory/self-managed/operations.md +++ b/content/operate/iris/agent-memory/self-managed/operations.md @@ -32,9 +32,11 @@ aliases: ## Secret rotation -Rotate Redis Agent Memory agent keys through the Identity Service API: +Use the [Identity Service port-forward and control token](/content/operate/iris/agent-memory/self-managed/deploy.md#7-mint-an-agent-key) +from the deployment procedure to rotate agent keys: ```bash +IDS_URL=http://localhost:9200 curl -sS -X POST "$IDS_URL/v1/api-keys//rotate?graceSeconds=3600" \ -H "Authorization: Bearer $IDS_CONTROL_TOKEN" ``` @@ -70,7 +72,32 @@ corresponding checksums, and roll the affected workloads on both releases: | `memory-dp-credential` | `identityService.credential.existingSecretChecksum` | `existingSecretChecksum` on the matching `runtime.serviceCredentials[]` entry | | `memory-internal-token` | `controlplane.internalToken.existingSecretChecksum` | `productValidation.products.memory.credential.existingSecretChecksum` | -Apply `ram-values.yaml` to Memory and `values-ids.yaml` to Identity Service separately. +For example, rotate `memory-dp-credential` in the walkthrough's namespace: + +```bash +umask 077 +openssl rand -hex 32 | tr -d '\n' > memory-dp-token +kubectl -n create secret generic memory-dp-credential \ + --from-file=token=./memory-dp-token --save-config --dry-run=client -o yaml | kubectl apply -f - +RUNTIME_CREDENTIAL_CHECKSUM="$(shasum -a 256 ./memory-dp-token | awk '{print $1}')" + +helm upgrade ids redis-ai/iris-identity-service \ + --version 0.10.0 \ + --namespace \ + -f values-ids.yaml \ + --set-string "runtime.serviceCredentials[0].existingSecretChecksum=$RUNTIME_CREDENTIAL_CHECKSUM" +helm upgrade redis-agent-memory redis-ai/redis-agent-memory \ + --version 0.8.0 \ + --namespace \ + -f ram-values.yaml \ + --set-string "identityService.credential.existingSecretChecksum=$RUNTIME_CREDENTIAL_CHECKSUM" +``` + +In the walkthrough, `memory-dp` is entry `0` in `runtime.serviceCredentials`. With another list, +update the matching entry. Persist the checksum in both values files for later upgrades, and +verify the affected Deployments are Ready. For the internal token, use the corresponding paths +in the table. For multiple namespaces, provision the new token value in each namespace. + Coordinate rotation: mismatched runtime credentials reject authentication, and mismatched internal tokens prevent grant validation. Identity Service's operator control token is separate; its checksum is `control.existingSecretChecksum` on that release. diff --git a/content/operate/iris/agent-memory/self-managed/prerequisites.md b/content/operate/iris/agent-memory/self-managed/prerequisites.md index 9eb34b0882..75138b4ff0 100644 --- a/content/operate/iris/agent-memory/self-managed/prerequisites.md +++ b/content/operate/iris/agent-memory/self-managed/prerequisites.md @@ -62,7 +62,7 @@ databases. | --- | --- | --- | --- | | Store Redis | Always | `databases."1".urls` | Session memory JSON, long-term memory hashes, RediSearch indexes, vectors, and TTL-managed data. | | Metadata Redis | Always | `metadata.urls`, and optionally `metadata.namespace` (default `iris:memory`) | Store records. | -| Job Redis | Always | `background_jobs.redis.urls` | Background work, retry state, delayed jobs, and idempotency markers. | +| Job Redis | Always | `background_jobs.redis.urls` and, when execution idempotency is enabled, `background_jobs.idempotency.urls` | Background work, retry state, delayed jobs, and execution markers. The two settings can use the same Redis. | {{< /table-scrollable >}} The Identity Service reads its Metadata Redis from `metadata.urls` in its own metadata Secret and @@ -104,8 +104,8 @@ Plane store records, and losing the Identity Service Metadata Redis removes agen ## Credentials and Secrets -The chart renders the configuration from Helm values (`config.render: true`). Credentials stay out -of the values: you put them in an overlay Secret. Create the following Secrets for the +When `config.render: true` is set, as in the walkthrough, the chart renders configuration from +Helm values. This option defaults to `false`. Credentials stay in an overlay Secret. Create the following Secrets for the [two-release installation](/content/operate/iris/agent-memory/self-managed/deploy.md#2-create-secrets): | Secret | Default key | Holds | @@ -145,6 +145,7 @@ Mirror these images into your internal registry: | `replicated/troubleshoot:0.131.0` | Support-bundle health checks, on by default (`supportPackage.*`) | | `hibiken/asynqmon:0.7.2` | Optional, with `controlplane.queueMonitor` | | `python:3.13-alpine` | Optional, with `tests.smoke` | +| `curlimages/curl:8.17.0` | Optional, for `helm test` on the Identity Service release (`tests.image` in `values-ids.yaml`) | For this release, `` is `0.8.0` and `` is `0.10.0`. Inspect each package with `helm show chart redis-ai/redis-agent-memory --version 0.8.0` @@ -198,6 +199,8 @@ Configure the separate Identity Service release in `values-ids.yaml`: image: repository: registry.example.com/redislabs/iris-identity-service tag: "0.10.0" +tests: + image: registry.example.com/curlimages/curl:8.17.0 global: imagePullSecrets: - name: ram-registry diff --git a/content/operate/iris/agent-memory/self-managed/reference.md b/content/operate/iris/agent-memory/self-managed/reference.md index 933fe27f33..80d5199240 100644 --- a/content/operate/iris/agent-memory/self-managed/reference.md +++ b/content/operate/iris/agent-memory/self-managed/reference.md @@ -48,6 +48,7 @@ CSI-synced Secrets, make sure a Control Plane pod also mounts the corresponding | --- | --- | --- | | `helm search repo redis-ai/redis-agent-memory --versions` returns no results | Helm repo not added/updated, or the chart version has not been published to the repo yet | Run `helm repo add`, `helm repo update`, or install from the chart package provided by Redis. | | Rendering fails because `identityService` has an additional property | Memory values still use the removed bundled contract, such as `identityService.enabled` or `identityService.runtime` | Move service configuration to `values-ids.yaml`. Memory accepts only `identityService.credential` and `identityService.allowInsecureTransport`; remove `externalIdentityService` too. | +| Rendering fails at `/externalIdentityService` with `'not' failed` | Values contain the removed external-mode block | Remove `externalIdentityService` and use `dataplane.identityServiceBaseURL` plus `identityService.credential` for the standalone connection. | | Rendering fails with `requires dataplane.identityServiceBaseURL` | Agent-key authentication or an `identity_service` subject needs the standalone runtime endpoint | Set `dataplane.identityServiceBaseURL` and `identityService.credential.existingSecret`. See [Deploy](/content/operate/iris/agent-memory/self-managed/deploy.md). | | Rendering fails with `requires identityService.credential.existingSecret` | The endpoint is configured but the runtime credential Secret is missing | Name the existing Secret also registered on Identity Service as `memory-dp`. | | Rendering fails with `must be https://` or `identityService.allowInsecureTransport=true is refused under the fips security profile` | The runtime endpoint uses HTTP without an allowed exception, or insecure transport is enabled under FIPS | Use an HTTPS endpoint. An HTTP exception requires `identityService.allowInsecureTransport: true` and is refused under FIPS. | @@ -62,7 +63,7 @@ CSI-synced Secrets, make sure a Control Plane pod also mounts the corresponding | Worker callbacks rejected with `401` | `auth.worker_identity` doesn't match the worker, or the Data Plane cannot fetch the cluster JWKS | See [CL-6](/content/operate/iris/agent-memory/self-managed/deploy.md#cl-6). | | `helm install --atomic --wait` times out and rolls back | Cluster is small or image pull/startup takes longer than Helm's default timeout | Install without `--atomic --wait`, or set a longer `--timeout` and ensure enough cluster capacity. | | Pods are pending during install or upgrade | CPU/memory capacity is insufficient for default replicas and rollout overlap | Add nodes/headroom or lower replicas for test deployments. | -| Data Plane health fails | Pod not ready, config invalid, Redis unavailable, or license invalid | Check pod logs and call `/health`, `/health/liveness`, and `/health/readiness`. | +| Data Plane health fails | Pod not ready, config invalid, Redis unavailable, or license invalid | Check pod logs and call `/health/liveness` and `/health/readiness`. | | Data Plane fails with `agent_keys.introspection.base_url is required` | Bring-your-own (BYO) Data Plane config uses agent-key auth, the default when no auth method is set, without the `auth.agent_keys.introspection` block | Add the `auth.agent_keys.introspection` block. Only isolated development can use `auth.method: none` with `profile: dev`. See [Configuration](/content/operate/iris/agent-memory/self-managed/configuration.md#bring-your-own-byo-configuration). | | Data Plane fails with `removed metadata configuration keys: …` | Config uses metadata keys that version 0.7.0 removed, such as `metadata.source` or `metadata.live` | Configure `metadata.urls`, `metadata.namespace`, and `databases`, then create stores through the Control Plane. | | Pod fails to start in FIPS posture | A Redis URL is not `rediss://`, an outbound HTTP client uses `skip_verify: true`, an authentication block sets `allow_insecure_transport: true`, or the Identity Service logs `product_validation..base_url must be an https URL in FIPS profile` | Update Redis URLs, HTTP client config, authentication blocks, and the Identity Service product validator URL to satisfy the posture checks. The error names the exact config path. | From fb79283a6408ea5a2845270c241b659cb46dcba2 Mon Sep 17 00:00:00 2001 From: David Dougherty Date: Wed, 21 Oct 2026 09:46:20 -0700 Subject: [PATCH 57/76] OP: add 8.12-RC1 release notes (#4314) --- .../redisce/redisos-8.12-release-notes.md | 185 ++++++++++++++++++ 1 file changed, 185 insertions(+) create mode 100644 content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisce/redisos-8.12-release-notes.md diff --git a/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisce/redisos-8.12-release-notes.md b/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisce/redisos-8.12-release-notes.md new file mode 100644 index 0000000000..af72e3e554 --- /dev/null +++ b/content/operate/oss_and_stack/stack-with-enterprise/release-notes/redisce/redisos-8.12-release-notes.md @@ -0,0 +1,185 @@ +--- +Title: Redis Open Source 8.12 release notes +alwaysopen: false +categories: +- docs +- operate +- stack +description: Redis Open Source 8.12 release notes. +linkTitle: v8.12-RC1 (October 2026) +min-version-db: blah +min-version-rs: blah +weight: 5 +--- + +## Redis Open Source 8.12-RC1 (October 2026) + +This is the first Release Candidate of Redis 8.12 in Redis Open Source. + +Release Candidates are feature-complete pre-releases. Pre-releases are not suitable for production use. + +### Headlines: + +Redis 8.12 introduces new features and performance improvements. + +### Operating systems we test Redis 8.12 on + + + +- Ubuntu 22.04 (Jammy Jellyfish), 24.04 (Noble Numbat), 26.04 (Resolute Raccoon) +- Rocky Linux 8.10, 9.7, 10.1 +- AlmaLinux 8.10, 9.7, 10.1 +- Debian 12.13 (Bookworm), Debian 13.4 (Trixie) +- Alpine 3.23 +- macOS 14.8.4 (Sonoma), 15.7.4 (Sequoia), 26.3 (Tahoe) - for both Intel and ARM + +### Security fixes (compared to 8.10.0) + +- [#Q10073](https://github.com/RediSearch/RediSearch/pull/10073) Indexed value hashing is collision-resistant, preventing index degradation from hash-based denial-of-service (HashDoS) attacks. (MOD-14459) +- [#Q11202](https://github.com/RediSearch/RediSearch/pull/11202) `unicode_tolower` no longer reads out of bounds on a dangling multi-byte lead byte in indexed TEXT fields encoded with the 8-bit Unicode Transformation Format (UTF-8). (MOD-18088) + +### New Features (compared to 8.10) + +- [#15331](https://github.com/redis/redis/pull/15331) Compressed bitmaps - redis-roaring integration reduces memory usage and speeds up bitmap operations. Thanks to Antonio, its creator, for [contributing this capability](https://redisroaring.com/redis-merge/#/1). +- [#15986](https://github.com/redis/redis/pull/15986) New command: `BITRANGEINTARRAY` - returns set bit offsets within a range, equivalent to redis-roaring's `R.RANGEINTARRAY`. +- [#15938](https://github.com/redis/redis/pull/15938) Set expiration on individual set members: + - New commands: `SEXPIRE`, `SPEXPIRE`, `SEXPIREAT`, `SPEXPIREAT` - set expiration. + - New commands: `STTL`, `SPTTL`, `SEXPIRETIME`, `SPEXPIRETIME` - retrieve remaining time-to-live (TTL) values or expiration timestamps. +- [#15828](https://github.com/redis/redis/pull/15828) Bless keys: avoid eviction of specified keys; new command: `BLESS`. +- [#15829](https://github.com/redis/redis/pull/15829) `CLUSTER MIGRATION STATUS` reports progress for both the snapshot and incremental migration phases. +- [#15556](https://github.com/redis/redis/pull/15556) Configure Transport Layer Security (TLS) named groups with `tls-groups`. +- [#P1063](https://github.com/RedisBloom/RedisBloom/pull/1063) Count-min sketches support optional `CELL_SIZE` in `CMS.INITBYDIM` and `CMS.INITBYPROB` for lower memory usage. `CMS.INCRBY` accepts negative increments to decrement counts. `CMS.INFO` reports cell sizes. +- [#J1640](https://github.com/RedisJSON/RedisJSON/pull/1640) Automatically create deep paths in JavaScript Object Notation (JSON). +- [#Q11330](https://github.com/RediSearch/RediSearch/pull/11330) 8-bit scalar quantization (SQ8) compression is available for hierarchical navigable small world (HNSW) vector fields. (MOD-14958) +- [#Q10063](https://github.com/RediSearch/RediSearch/pull/10063) `COLLECT LIMIT` honors the configurable `search-max-aggregate-results`. + +### Bug fixes (compared to 8.10.0) + +- [#15569](https://github.com/redis/redis/pull/15569) Changes made by `XDELEX DELREF` for already-deleted stream entries could fail to propagate to replicas or the append-only file (AOF). +- [#15508](https://github.com/redis/redis/pull/15508) Operations using random radix-tree traversal could behave incorrectly when starting from certain leaf nodes. +- [#15604](https://github.com/redis/redis/pull/15604) Memory usage could be calculated incorrectly for hashes using shared templates. +- [#15489](https://github.com/redis/redis/pull/15489) Redis database (RDB) files could fail to load after `XSETID` reduced a stream's `entries_added` counter. +- [#15626](https://github.com/redis/redis/pull/15626) `SET` could calculate an incorrect expiration time when given an excessively large relative TTL. +- [#15628](https://github.com/redis/redis/pull/15628) Active defragmentation could incorrectly handle the shared-template registry used by compact hashes. +- [#15530](https://github.com/redis/redis/pull/15530) After failover, a replica could remain connected to another replica instead of the new primary. +- [#15733](https://github.com/redis/redis/pull/15733) Invalid cluster slot ranges could be accepted after being merged into a valid range. +- [#15756](https://github.com/redis/redis/pull/15756) `RESTORE` could calculate an incorrect expiration time when given an excessively large relative TTL. +- [#15699](https://github.com/redis/redis/pull/15699) Memory growth could be accounted for incorrectly when `SETBIT` or `BITFIELD` expanded a string. +- [#15815](https://github.com/redis/redis/pull/15815) `XTRIM` could leave a stream in a state that prevented an RDB file containing it from loading. +- [#15795](https://github.com/redis/redis/pull/15795) Clients could fail to receive a protocol error when a malformed command arrived across multiple network reads. +- [#15875](https://github.com/redis/redis/pull/15875) `ARLASTITEMS` could omit elements appearing after a gap in an array. +- [#15916](https://github.com/redis/redis/pull/15916) Data could be lost when a legacy AOF file was upgraded before loading completed. +- [#15917](https://github.com/redis/redis/pull/15917) Hiredis clients on Linux could mishandle errors when establishing Transmission Control Protocol (TCP) connections. +- [#15869](https://github.com/redis/redis/pull/15869) Redis could crash when a module thread paused input/output (I/O) threads. +- [#15468](https://github.com/redis/redis/pull/15468) Publish/subscribe (Pub/Sub) subscriptions could remain active after the subscribing user's channel permissions were revoked or the user was deleted, if the client had re-authenticated as another user. +- [#15623](https://github.com/redis/redis/pull/15623) Module actions scheduled by hash-field expiration notifications could be delayed after fields expired in the background. +- [#15680](https://github.com/redis/redis/pull/15680) Data could be lost after a replica migrated to a different shard. +- [#15587](https://github.com/redis/redis/pull/15587) `LMOVEM` could emit keyspace notifications inconsistent with those produced by `LMOVE`. +- [#15695](https://github.com/redis/redis/pull/15695) Command side effects could be propagated to replicas or the AOF despite settings that should have excluded them. +- [#Q10872](https://github.com/RediSearch/RediSearch/pull/10872) The shard no longer crashes with a segmentation fault (`SIGSEGV`) when `RESTORE` of a legacy `ft_index0` payload is issued outside an RDB load. (MOD-15685) +- [#Q10728](https://github.com/RediSearch/RediSearch/pull/10728) Arming the garbage collection (GC) timer off the main thread on ARM no longer causes random shard crashes after extended uptime. (MOD-17309) +- [#Q10767](https://github.com/RediSearch/RediSearch/pull/10767) The server no longer crashes on writes when an index `FILTER` references no schema fields. (MOD-15592) +- [#Q11303](https://github.com/RediSearch/RediSearch/pull/11303), [#Q11304](https://github.com/RediSearch/RediSearch/pull/11304), [#Q11308](https://github.com/RediSearch/RediSearch/pull/11308), [#Q11309](https://github.com/RediSearch/RediSearch/pull/11309), [#Q11310](https://github.com/RediSearch/RediSearch/pull/11310), [#Q11311](https://github.com/RediSearch/RediSearch/pull/11311) `FT.ALTER SCHEMA ADD` running concurrently with queries under `WORKERS>0` no longer crashes query execution across VECTOR, TAG, NUMERIC, GEO, ISMISSING, and GEOSHAPE nodes. (MOD-18356, MOD-18361, MOD-18366, MOD-18367, MOD-18368) +- [#Q11474](https://github.com/RediSearch/RediSearch/pull/11474) `FT.AGGREGATE` with the numeric `SORTBY` optimizer no longer crashes or returns wrong results. (MOD-6967) +- [#Q10209](https://github.com/RediSearch/RediSearch/pull/10209) The coordinator no longer crashes with Redis serialization protocol version 2 (RESP2) when `FT.PROFILE HYBRID` hits a return-strict timeout. (MOD-16396) +- [#Q10145](https://github.com/RediSearch/RediSearch/pull/10145) TEXT `PHONETIC` matches now behave correctly when non-TEXT fields precede TEXT fields in the schema. (MOD-16304) +- [#Q11262](https://github.com/RediSearch/RediSearch/pull/11262) An `FT.AGGREGATE` cursor with `FILTER` reports the correct result count. (MOD-18172) +- [#Q11282](https://github.com/RediSearch/RediSearch/pull/11282) `FT.EXPLAIN` now renders a multi-word tag value as a single tag rather than an `INTERSECT`. (MOD-8666) +- [#Q10768](https://github.com/RediSearch/RediSearch/pull/10768) NOT queries no longer leak excluded documents or drop results after a concurrent index change. (MOD-17394) +- [#Q10637](https://github.com/RediSearch/RediSearch/pull/10637) Numeric index cardinality and bounds are now correct under float compression. (MOD-16877) +- [#Q10704](https://github.com/RediSearch/RediSearch/pull/10704) Empty wildcard patterns no longer cause out-of-bounds writes or spurious empty-value matches. (MOD-17451) +- [#Q10752](https://github.com/RediSearch/RediSearch/pull/10752) Wildcard `?` on `WITHSUFFIXTRIE` TEXT fields now matches per codepoint rather than per byte. (MOD-17476) +- [#Q10112](https://github.com/RediSearch/RediSearch/pull/10112) Suffix-trie wildcard queries on TEXT fields are now case-insensitive. (MOD-16259) +- [#Q11450](https://github.com/RediSearch/RediSearch/pull/11450) `Wildcard_MatchRune` no longer hangs on a literal `*` in the indexed string. +- [#Q11267](https://github.com/RediSearch/RediSearch/pull/11267) Duplicate vector results are no longer returned during tiered indexing. (MOD-17795) +- [#Q10731](https://github.com/RediSearch/RediSearch/pull/10731) `FT.SEARCH` on a cluster returns an error for invalid `LIMIT` arguments, matching standalone behavior. (MOD-12465) +- [#Q10251](https://github.com/RediSearch/RediSearch/pull/10251) `FT.SEARCH` under `WORKERS>0` no longer returns a nil field-array for documents re-indexed mid-query. (MOD-16507) +- [#Q10921](https://github.com/RediSearch/RediSearch/pull/10921) Sort keys with embedded NUL bytes are no longer truncated in replies. +- [#Q10085](https://github.com/RediSearch/RediSearch/pull/10085) The `RETURN_STRICT` safe-loader no longer deadlocks the Redis global lock (GIL) on client timeout. (MOD-16267) +- [#Q11449](https://github.com/RediSearch/RediSearch/pull/11449) An index reference leak after queued search cancellation no longer occurs. (MOD-18682) +- [#Q11369](https://github.com/RediSearch/RediSearch/pull/11369) Memory leaks after coordinator SEARCH cancellation no longer occur. (MOD-18548) +- [#Q10881](https://github.com/RediSearch/RediSearch/pull/10881) SST replication no longer hangs and the numeric snapshot is no longer corrupted when disk GC is mid-cycle. (MOD-17197) +- [#Q10110](https://github.com/RediSearch/RediSearch/pull/10110) `FT.HYBRID` cursor setup now has a bounded channel timeout. (MOD-16145) +- [#Q10211](https://github.com/RediSearch/RediSearch/pull/10211), [#Q10745](https://github.com/RediSearch/RediSearch/pull/10745) `RETURN_STRICT` cursors are depleted on timeout; strict-timed-out hybrid cursor cycles drop their cursors at cycle end. (MOD-16091, MOD-16991) +- [#Q10947](https://github.com/RediSearch/RediSearch/pull/10947) Coordinator cursor memory is now capped via `INDEX_CURSOR_LIMIT`. (MOD-17479) +- [#Q10766](https://github.com/RediSearch/RediSearch/pull/10766), [#Q10763](https://github.com/RediSearch/RediSearch/pull/10763) Cursor reads no longer time out against an earlier read's deadline and correctly report a timeout that expires while iterators are revalidated. (MOD-17489, MOD-17488) +- [#Q10993](https://github.com/RediSearch/RediSearch/pull/10993) RDB and replication payloads no longer leak a stray heap byte. (MOD-17800) +- [#Q10074](https://github.com/RediSearch/RediSearch/pull/10074) `FT.HYBRID` returns a stable reciprocal rank fusion (RRF) result order across cluster shards. (MOD-12438) +- [#Q11027](https://github.com/RediSearch/RediSearch/pull/11027) Cluster TLS connections now use Redis node metadata, aligning with topology changes. (MOD-15869) +- [#Q10380](https://github.com/RediSearch/RediSearch/pull/10380), [#Q10430](https://github.com/RediSearch/RediSearch/pull/10430) Cluster topology is now refreshed on topology-change events rather than by polling. (MOD-15868, MOD-15870) +- [#Q10835](https://github.com/RediSearch/RediSearch/pull/10835) Coordinator fanout now preserves binary argument lengths. +- [#Q11095](https://github.com/RediSearch/RediSearch/pull/11095) Replica stalls during `FT.DROPINDEX DD` are reduced. +- [#Q10083](https://github.com/RediSearch/RediSearch/pull/10083) Internal query arguments missing during a rolling upgrade are tolerated. (MOD-16047) +- [#Q10816](https://github.com/RediSearch/RediSearch/pull/10816) Forced GC now waits for a concurrent child process instead of skipping its cycle. +- [#Q10958](https://github.com/RediSearch/RediSearch/pull/10958) The asynchronous disk read pool no longer spins when saturated; it blocks instead. +- [#Q10646](https://github.com/RediSearch/RediSearch/pull/10646) The VectorSimilarity dependency addresses unaligned metadata access on ARM/Scalable Vector Extension (SVE) paths. (MOD-15303) +- [#Q10352](https://github.com/RediSearch/RediSearch/pull/10352) `FT.PROFILE ON_TIMEOUT FAIL` now reports a pre-execution timeout as a warning rather than an error. (MOD-16660) +- [#Q10870](https://github.com/RediSearch/RediSearch/pull/10870) Crash reports now include in-flight coordinator aggregate, hybrid, and cursor-read queries. (MOD-16992) +- [#Q11089](https://github.com/RediSearch/RediSearch/pull/11089) Debug aggregate request synchronization no longer crashes. (MOD-17616) +- [#J1636](https://github.com/RedisJSON/RedisJSON/pull/1636) Fixed array-index normalization overflow for extreme negative indexes, including `i64::MIN`, affecting commands such as `JSON.ARRTRIM`. +- [#J1648](https://github.com/RedisJSON/RedisJSON/pull/1648) Corrected `JSON.MGET` key-spec metadata to identify all key arguments and exclude the trailing JSONPath argument. +- [#J1670](https://github.com/RedisJSON/RedisJSON/pull/1670) Fixed downstream compilation when using `redis_json_module_create!` with the `as-library` feature and the new RDB depth configuration. +- [#P1035](https://github.com/RedisBloom/RedisBloom/pull/1035) `TDIGEST.QUANTILE`, `TDIGEST.CDF`, and `TDIGEST.TRIMMED_MEAN` could return a not-a-number (NaN) value for a sketch with exactly one observation. +- [#P1053](https://github.com/RedisBloom/RedisBloom/pull/1053) RDB loading and `RESTORE` inconsistently rejected invalid numeric values, allocation sizes, serialized counts, or inconsistent metadata in Bloom filters, Count-Min Sketches, Top-K sketches, and t-digests. Bloom filter growth lacked overflow checks. +- [#P1067](https://github.com/RedisBloom/RedisBloom/pull/1067) Bloom filters with excessive hash-function counts could be accepted during loading and restoration, including through `BF.LOADCHUNK`. +- [#P1053](https://github.com/RedisBloom/RedisBloom/pull/1053) Top-K sketches containing nonempty items with zero counts or empty-string items with nonzero counts could be restored incorrectly. +- [#P1062](https://github.com/RedisBloom/RedisBloom/pull/1062) `CMS.MERGE` key metadata did not correctly identify the destination and all source keys, affecting cluster slot validation. +- [#P1071](https://github.com/RedisBloom/RedisBloom/pull/1071) `CF.COMPACT` command metadata was missing its argument count and key specification. + +### Performance and resource utilization improvements (compared to 8.10.0) + +- [#15618](https://github.com/redis/redis/pull/15618) Batch updates to hashes using shared templates run more efficiently. +- [#15018](https://github.com/redis/redis/pull/15018) Supported x86 systems use the processor-based monotonic clock by default, with runtime calibration to reduce timing overhead. +- [#15782](https://github.com/redis/redis/pull/15782) RDB and AOF loading provides sequential-read hints to the operating system to improve loading efficiency. +- [#Q10169](https://github.com/RediSearch/RediSearch/pull/10169) Searches on indexes using field-level expiration (`HEXPIRE`) are faster. (MOD-16350) +- [#Q10981](https://github.com/RediSearch/RediSearch/pull/10981), [#Q11065](https://github.com/RediSearch/RediSearch/pull/11065) Coordinator field lookup and row-reply serialization overhead is reduced. (MOD-17924) +- [#Q11055](https://github.com/RediSearch/RediSearch/pull/11055), [#Q11056](https://github.com/RediSearch/RediSearch/pull/11056), [#Q11057](https://github.com/RediSearch/RediSearch/pull/11057) Cluster reply conversion, result destruction, and `GROUPBY` cleanup have reduced overhead. (MOD-17908, MOD-17909, MOD-17910) +- [#Q11507](https://github.com/RediSearch/RediSearch/pull/11507), [#Q11506](https://github.com/RediSearch/RediSearch/pull/11506) Query replies now format integral numeric values without `snprintf` and declare reply collection lengths up front. (MOD-18800, MOD-18799) +- [#Q11509](https://github.com/RediSearch/RediSearch/pull/11509) Hash field loading for `RETURN` and `LOAD` no longer requires per-field allocations. (MOD-18811) +- [#Q11278](https://github.com/RediSearch/RediSearch/pull/11278) `FT.ALTER` avoids unnecessary document reindexing during schema additions. (MOD-18171) +- [#Q11384](https://github.com/RediSearch/RediSearch/pull/11384) Score/payload-only hash updates avoid reindexing. (MOD-18182) +- [#Q11230](https://github.com/RediSearch/RediSearch/pull/11230) Hash documents are no longer re-indexed when no indexed field is touched. (MOD-17688) +- [#Q10515](https://github.com/RediSearch/RediSearch/pull/10515) Coordinator requests are now allocated on the main thread, removing `CoordRequestCtx`. (MOD-16990) +- [#Q11018](https://github.com/RediSearch/RediSearch/pull/11018) Query usage is now separated from administrative index operations. (MOD-17536) +- [#J1645](https://github.com/RedisJSON/RedisJSON/pull/1645) Memory usage reporting divides a shared string's allocation, including header and padding, proportionally among its references. Previously, each reference was charged the full allocation. +- [#J1660](https://github.com/RedisJSON/RedisJSON/pull/1660) JSON parsing uses compact object allocation to reduce memory usage. +- [#J1661](https://github.com/RedisJSON/RedisJSON/pull/1661) JSONPath parsing and evaluation reduce backtracking, temporary allocations, and repeated traversal. Optimizations cache root-path operands across filter candidates, stop existence checks at the first match, and reduce regex matching and string comparison overhead. +- [#J1631](https://github.com/RedisJSON/RedisJSON/pull/1631) Improved typed-array element access through direct indexing, avoiding repeated iteration and quadratic work during full indexed scans. +- [#P1049](https://github.com/RedisBloom/RedisBloom/pull/1049) Improved t-digest compression performance for inputs containing many duplicate values. Sorting is bounded to O(n log n), avoiding quadratic behavior on pathological inputs. + +### Configuration parameters + +- [#15331](https://github.com/redis/redis/pull/15331) redis-roaring: `bitmap-default-roaring` - enables the new roaring bitmaps type; disabled by default. +- [#J1657](https://github.com/RedisJSON/RedisJSON/pull/1657) `json-legacy-rdb-depth-limit` enforces a maximum nesting depth of 128 when loading JSON from RDB or through `RESTORE`. It is disabled by default. +- [#J1640](https://github.com/RedisJSON/RedisJSON/pull/1640) `json-auto-create-deep-paths` is disabled by default and configurable only at load time. Use the same value across shard nodes and AOF replays to prevent data divergence. + +### Metrics + +- [#15374](https://github.com/redis/redis/pull/15374) `INFO keysizes` histograms include stream key sizes. +- [#15682](https://github.com/redis/redis/pull/15682) `INFO STREAMS` provides per-database histograms of consumer groups per stream, pending entries per group, and consumers per group. Enable with `stream-stats yes` (disabled by default). +- [#Q10097](https://github.com/RediSearch/RediSearch/pull/10097) `FT.INFO` now exposes the default `EF_RUNTIME`. (MOD-16147) +- [#Q10107](https://github.com/RediSearch/RediSearch/pull/10107) Disk GC `bytes_collected` and cycle statistics are reported in `FT.INFO`. (MOD-14621) +- [#Q10464](https://github.com/RediSearch/RediSearch/pull/10464) `FT.INFO` on the coordinator aggregates per-field disk metrics via the reducer. (MOD-14859) +- [#Q10204](https://github.com/RediSearch/RediSearch/pull/10204) Redis `INFO` now breaks down query timeout metrics by pipeline stage. (MOD-13192) +- [#Q11521](https://github.com/RediSearch/RediSearch/pull/11521) Index metrics are no longer collected for unrelated `INFO` sections. (MOD-18033) +- [#Q10536](https://github.com/RediSearch/RediSearch/pull/10536) `FT.PROFILE` loader-field timing resolution is improved. (MOD-17132) + +### Behavior changes + +- [#Q10109](https://github.com/RediSearch/RediSearch/pull/10109) `FT.CREATE` rejects `RERANK` on HNSW vector fields in random access memory (RAM) mode. (MOD-16015) +- [#Q10200](https://github.com/RediSearch/RediSearch/pull/10200) Disk indexes reject `DIALECT 4` and `WITHOUTCOUNT`. (MOD-15997) +- [#Q10082](https://github.com/RediSearch/RediSearch/pull/10082) `COLLECT` requires an `@` prefix for field names. (MOD-16233) +- [#Q11106](https://github.com/RediSearch/RediSearch/pull/11106) JSON `HIGHLIGHT`/`SUMMARIZE` fields with multi-value JSONPath are now rejected. (MOD-16530) +- [#Q10208](https://github.com/RediSearch/RediSearch/pull/10208) Non-cacheable commands are now marked with `dont_cache`. (MOD-16175) +- [#Q10699](https://github.com/RediSearch/RediSearch/pull/10699) Numeric command arguments are now strictly parsed. (MOD-17409) +- [#Q11335](https://github.com/RediSearch/RediSearch/pull/11335) The Search module requires Redis 8.9.241 or newer to load. (MOD-15868) +- [#P1063](https://github.com/RedisBloom/RedisBloom/pull/1063) Rejected `CMS.INCRBY` item updates now leave counters and the total count unchanged instead of saturating counters on overflow. Multi-item commands can still partially succeed. +- [#P1063](https://github.com/RedisBloom/RedisBloom/pull/1063) `CMS.MERGE` now requires matching cell sizes across the destination and all sources. + +### Modules API + +- [#15517](https://github.com/redis/redis/pull/15517) The modules application programming interface (API) adds the `REDISMODULE_NODE_PORT_TLS` flag to `RM_GetClusterNodeInfo()` to retrieve a cluster node's TLS port. +- [#J1623](https://github.com/RedisJSON/RedisJSON/pull/1623) Added `getWithPath` in shared API V9 to evaluate a precompiled JSONPath, allowing callers to compile a path once and reuse it across documents. +- [#J1666](https://github.com/RedisJSON/RedisJSON/pull/1666) Added `pathParseNoErrorMsg` in shared API V10 to compile JSONPaths without a Redis context or an allocated error message. Returns a compiled handle on success or `NULL` on failure. +- [#J1653](https://github.com/RedisJSON/RedisJSON/pull/1653) Made pre-command hooks optional and allowed hook errors to abort JSON commands before execution. Hooks now return `RedisResult<()>`; shared C API calls continue to ignore hook errors. +- [#J1640](https://github.com/RedisJSON/RedisJSON/pull/1640) Extended the Rust `Manager` interface with `clone_value` and `create_object` to support construction of missing paths. Custom implementations must implement these methods and satisfy the new `Borrow` requirement for owned values. From 4eb3e30c14e460129dd051659263bd55dcbd7077 Mon Sep 17 00:00:00 2001 From: paoloredis <158162796+paoloredis@users.noreply.github.com> Date: Thu, 22 Oct 2026 10:44:59 +0200 Subject: [PATCH 58/76] DOC-7184: Answer "Ask AI to explain this page" in a pane beside the page (#4284) * DOC-7184: Answer "Ask AI to explain this page" in a pane beside the page The link used to send the reader to redis.io/chat. A plain click now asks the same question in a Redis Copilot pane docked to the right of the page, so the page stays in view. A new-tab click still opens the full chat page. - The answer streams in as Markdown, with its sources underneath, and follow-up questions keep the conversation's context. - The pane follows the reader to the next page. The transcript is kept for the browser tab, and a page that starts with the pane open lays itself out for it before the first paint. - Collapse it with the chevron or Esc, and a tab on the right edge reopens it. Drag its left edge to resize it. - The page makes room for the pane instead of sitting under it. When less than 1280px is left, the table of contents steps aside, and below 1100px the section nav does too. Windows narrower than 1280px get an overlay. - Each answer has a thumbs up and down, as on the chat page. A downvote asks for an optional comment and a Turnstile check. Ratings are sent only from redis.io, staging builds included; a local build logs what it would have sent. The pane is a client of the chat app's own endpoints, as its page script (redis.io/chat/static/js/bubble.js) calls them. They are same-origin on redis.io and send no CORS headers, so a local build needs a proxy that forwards /chat to redis.io. The surprises were in the layout. Body is a grid, so a docked main kept its widest code block as a minimum width and ran on under the pane until it got min-width 0. And html reserves a scrollbar gutter, so clientWidth overstated the room in headless Chrome; the layout function measures the html box instead. Learned: docked main needs min-width 0 (body is a grid), and the html box, not clientWidth, is the page width Constraint: q is URI-encoded once more before form encoding, which is what /chat/ask expects Constraint: the rating row stays outside what paint() rewrites, or a late repaint wipes a downvote being typed Rejected: iframing redis.io/chat | it brings the whole chat page, header, search and facets, into a 440px pane Rejected: following the end of a streaming answer | the start of a long answer scrolls away, so the question is pinned to the top Gaps: the rating endpoint has never received a pane rating, and origin is the docs page URL rather than /chat/, pending the Copilot team Gaps: on redis.io the Qualified launcher probably covers the pane's input, which only production can show Recheck: if redis.io/chat/static/js/bubble.js changes its endpoints or fields Ticket: DOC-7184 Co-Authored-By: Claude Opus 5.5 * DOC-7184: Send focus to the Copilot tab when Esc can't reach the Ask AI link Opened from the link and then narrowed below 1280px, the pane sent focus back to the link on Esc. The site hides the link's column at that width, so focus went nowhere instead of to the tab that reopens the pane. Focus now goes to whichever of the two is on screen. Reopening the pane from the tab also sends focus back to the tab. Constraint: collapse() only returns focus to an element with client rects; document.contains() is true for hidden ones too Ticket: DOC-7184 Co-Authored-By: Claude Opus 5.5 * DOC-7184: Restore the pane on pages that come back from the back/forward cache A page restored by Back or Forward kept the conversation it had when it was left. When that page had never shown the pane, its pageshow handler returned early. So the pane didn't come back, a question asked there reset the server's conversation, and leaving the page saved its empty copy over the stored one. The handler now always reloads the saved state, and builds the pane if the page never had one. In Chrome, most docs pages never take this path, because the page rating form's Turnstile iframe keeps them out of the back/forward cache. The docs home page has no rating form, so going home, opening a page, asking, and pressing Back shows it. Learned: docs pages with the rating form aren't bfcached in Chrome (cross-origin Turnstile iframe), so test Back from the home page Constraint: a persisted pageshow reloads the stored state even when this page never built the pane Ticket: DOC-7184 Co-Authored-By: Claude Opus 5.5 * DOC-7184: Make a question wait for a Copilot reset that's still in flight New chat sent POST /chat/reset without waiting for it, and the next question, asked into the now empty pane, sent a second reset before /chat/ask. Nothing ordered the two, so a slow first reset could land after the question had gone out and wipe the conversation the pane was showing. Now only one reset is in flight at a time, and a question waits for it to reply. Constraint: a question goes out only after any reset in flight has replied Gaps: only reproduced by holding the first reset back on purpose; at normal latency the reset replies long before a second click Ticket: DOC-7184 Co-Authored-By: Claude Opus 5.5 * DOC-7184: Let the Commands and Integrate nav step aside for the pane With the pane docked and less than 1100px of page left, the section nav steps aside. The rule looked for nav#sidebar as a direct child, but the collapsed nav on the Commands and Integrate list pages puts that id on a div inside its nav, so the 24rem column stayed up there. The rule now hides whichever column of main holds #sidebar. Ticket: DOC-7184 Co-Authored-By: Claude Opus 5.5 * DOC-7184: Keep a stuck Copilot reset from freezing the pane A question waits for any reset in flight, but the reset had no time limit, and Stop and the 90-second limit only aborted the question's own request. A reset that never got a reply left the pane busy for good: Stop did nothing, and no new question could go out until the page was reloaded. A reset now gives up after 15 seconds, and Stop and the time limit end a question that's still waiting for one. Constraint: anything a question waits on must end on Stop and on the time limit Gaps: reproduced only by holding the reset with no reply on purpose Ticket: DOC-7184 Co-Authored-By: Claude Opus 5.5 * DOC-7184: Ask again from the link when the last answer didn't finish A second click on "Ask AI to explain this page" showed the earlier answer whenever the last question was this page's, even when that answer had been stopped, had failed, or was cut off. It looked like nothing happened, and retrying took New chat first. Now the link shows the earlier answer only when it finished, and asks again otherwise. Ticket: DOC-7184 Co-Authored-By: Claude Opus 5.5 * DOC-7184: Keep a question that never went out off the next page A question is shown, and was saved for the next page, as soon as it's asked, but the first one in a pane waits for a reset before it's sent. Leaving the page, or pressing Stop, in that moment carried a question Copilot never received over to the next page, shown as stopped there. Now a question is saved only once its request has gone out. Constraint: save() leaves out an answer marked unsent, together with its question Gaps: reproducible by hand only with DevTools throttling slowing the reset Ticket: DOC-7184 Co-Authored-By: Claude Opus 5.5 * DOC-7184: Call the pane "AI chatbot" in its title and collapsed tab "Copilot" promised more than the pane does, so the title at the top and the label on the collapsed tab now say "AI chatbot", as do the names screen readers announce for them. The notice, the line under the text box, and the error messages still name Redis Copilot, the service behind the chat page, as asked. Ticket: DOC-7184 Co-Authored-By: Claude Opus 5.5 * DOC-7184: Show the pane in the dark theme on Iris pages Iris pages (operate/iris and develop/ai/context-engine) are dark, but the pane stayed light, because iris-dark.css only restyles
    and the pane sits outside it. On those pages the pane now uses the same --iris-* colors: the conversation, code blocks, tables, sources, the text box, the rating form, and the collapsed tab. The Turnstile check in the rating form uses its dark theme there too. Other pages keep the light pane. Co-Authored-By: Claude Opus 5.5 * DOC-7184: Share one conversation between all tabs The chatbot keeps the conversation against its session cookie, which every tab shares until the browser closes. The pane kept its transcript per tab, so a second tab with an empty pane reset the conversation that the first tab was still showing, and the first tab's follow-ups then lost their context. The transcript is now in localStorage, so every tab shows the same conversation and updates when another tab adds to it, rates an answer, or starts a new chat. A new tab opens on the existing conversation instead of resetting it. One tab asks at a time: while another tab's answer arrives, Send waits. Whether the pane is open, and its width, stay per tab. A session cookie of the pane's own (aiChatPane=1) marks the browser session. When it's missing, the browser has restarted and the chatbot's conversation is gone, so the stored transcript is cleared too. Co-Authored-By: Claude Opus 5.5 * DOC-7184: Store a question the moment it's asked A question wasn't stored until it had gone to the chatbot, and the first one waits for /chat/reset before it goes. During that wait the other tabs couldn't see it, so one of them could ask too, and its own reset could wipe the first tab's question. A question is now stored as soon as it's asked, marked as not sent yet, so the other tabs show it coming and hold Send. If the page is left before it goes out, it's taken out again, since the chatbot never saw it. One left behind by a tab that has gone is dropped after the reset's time limit. Related: an empty saved conversation is a new chat in another tab, and ends a question in flight. Any other save that's missing the question came from a tab that hadn't seen it yet, so the asking tab keeps it and saves it again. A stored message without an id is skipped, because ids are how tabs match messages. Co-Authored-By: Claude Opus 5.5 --------- Co-authored-by: Claude Opus 5.5 --- layouts/_default/baseof.html | 1 + layouts/partials/ai-chat-pane.html | 39 + layouts/partials/meta-links.html | 6 +- static/css/ai-chat-pane.css | 823 +++++++++++++++++++ static/js/ai-chat-pane.js | 1222 ++++++++++++++++++++++++++++ 5 files changed, 2089 insertions(+), 2 deletions(-) create mode 100644 layouts/partials/ai-chat-pane.html create mode 100644 static/css/ai-chat-pane.css create mode 100644 static/js/ai-chat-pane.js diff --git a/layouts/_default/baseof.html b/layouts/_default/baseof.html index c075df9a47..8c4554091c 100644 --- a/layouts/_default/baseof.html +++ b/layouts/_default/baseof.html @@ -59,6 +59,7 @@ + {{ partial "ai-chat-pane.html" . }} {{ partial "segment.html" . }} {{ partial "fullstory.html" . }} {{ if .Params.math }} diff --git a/layouts/partials/ai-chat-pane.html b/layouts/partials/ai-chat-pane.html new file mode 100644 index 0000000000..f2585e8e8e --- /dev/null +++ b/layouts/partials/ai-chat-pane.html @@ -0,0 +1,39 @@ +{{/* Redis Copilot in a pane docked to the right of the page. The "Ask AI to + explain this page" link in meta-links.html opens it; static/js/ai-chat-pane.js + builds the pane and talks to the chat endpoints. */}} + + + diff --git a/layouts/partials/meta-links.html b/layouts/partials/meta-links.html index d735a569fd..645f30fb90 100644 --- a/layouts/partials/meta-links.html +++ b/layouts/partials/meta-links.html @@ -15,7 +15,9 @@ {{ $editURL := printf "%s/edit/%s/content/%s%s" $gh_repo $gh_branch $gh_path $gh_file }} {{ $issuesURL := printf "%s/issues/new?title=Feedback: %s&body=Page https://redis.io/docs/latest/%s%s" $gh_repo (safeURL $.Title ) $gh_path $stripped_filename }} -{{/* Build the "Ask Copilot about this page" link (opens the redis.io/chat bot) */}} +{{/* Build the "Ask Copilot about this page" link. static/js/ai-chat-pane.js turns a + plain click into the same question asked in a pane beside the page; the href + still opens the redis.io/chat bot for a new-tab click. */}} {{ $chatPrompt := printf "Explain this Redis documentation page: %q (%s)" .Title .Permalink }} {{ $chatURL := printf "/chat?%s" (querify "q" $chatPrompt) }} @@ -44,7 +46,7 @@ "label" "Download documentation" "iconClass" "w-4 h-4" "class" "group inline-flex items-center gap-1 hover:text-redis-pen-400 mt-auto self-start") "meta" }} - + {{ partial "icons/sparkles.html" }} Ask AI to explain this page diff --git a/static/css/ai-chat-pane.css b/static/css/ai-chat-pane.css new file mode 100644 index 0000000000..c1f19fa1fb --- /dev/null +++ b/static/css/ai-chat-pane.css @@ -0,0 +1,823 @@ +/* Redis Copilot pane — see static/js/ai-chat-pane.js. + + Plain CSS rather than Tailwind classes: the script builds the markup, and + Tailwind's content scan doesn't read static/js. Colors are the site palette + from tailwind.config.js. + + No vw units anywhere in here: on redis.io the Qualified widget rewrites every + viewport-width unit in page stylesheets (see assets/css/index.css). */ + +:root { + --ai-pane-w: 440px; + /* The pane starts below the sticky site header, which keeps its full width. */ + --ai-pane-top: 70px; +} + +/* ------------------------------------------------------------ page layout -- + Docked, the page gives up the pane's width instead of sitting under it. The + pinned sidebar and table of contents are position: fixed without a left or + right offset, so they follow the reflowed layout on their own. */ +html.ai-pane-docked body > main, +html.ai-pane-docked body > footer { + width: auto; + /* Body is a grid, and a grid item won't go narrower than its widest code + block unless told to. Without this, main runs on under the pane. */ + min-width: 0; + margin-right: var(--ai-pane-w); +} + +/* The workbench dock spans the window bottom; keep it beside the pane. */ +html.ai-pane-docked .rwb { + right: var(--ai-pane-w); +} + +/* When what's left of the page is narrow, the pane stands in for the right-hand + rail (table of contents and page links) rather than squeezing the text. */ +html.ai-pane-tight main > :is(section, aside):has(a[data-ai-chat-open]) { + display: none; +} +/* Narrower still, the 24rem section nav would leave the text a phone-width + column, so it steps aside too until the pane is collapsed. The sidebar id + is on the nav itself in docs-nav.html, and on a div inside the nav in + docs-nav-collapsed.html (Commands and Integrate list pages). */ +html.ai-pane-narrow body > main > :has(#sidebar) { + display: none; +} + +/* Holds the pane's place between the first paint and the script mounting the + real pane, on a page that opens with the pane already open. */ +html.ai-pane-restoring::after { + content: ''; + position: fixed; + top: var(--ai-pane-top); + right: 0; + bottom: 0; + width: var(--ai-pane-w); + z-index: 60; + background: #fff; + border-left: 1px solid #B9C2C6; +} + +html.ai-pane-resizing, +html.ai-pane-resizing * { + cursor: col-resize !important; + user-select: none !important; +} + +/* ------------------------------------------------------------------ pane -- */ +.aicp { + position: fixed; + top: var(--ai-pane-top); + right: 0; + bottom: 0; + width: var(--ai-pane-w); + /* Over the workbench dock (55) and the footer (50); under the search + modal (200). */ + z-index: 60; + display: flex; + flex-direction: column; + background: #fff; + border-left: 1px solid #B9C2C6; + color: #091A23; + font-family: 'Space Grotesk', ui-sans-serif, system-ui, sans-serif; + font-size: 14px; + line-height: 1.55; + letter-spacing: normal; + transform: translateX(100%); + transition: transform .2s ease; +} +.aicp.is-in { + transform: none; +} +.aicp[hidden] { + display: none; +} +/* Overlaying the page rather than docked beside it. */ +html:not(.ai-pane-docked) .aicp.is-in { + box-shadow: -12px 0 32px rgba(9, 26, 35, .18); +} +html.ai-pane-resizing .aicp { + transition: none; +} + +.aicp-sr { + position: absolute; + width: 1px; + height: 1px; + overflow: hidden; + clip: rect(0 0 0 0); + white-space: nowrap; +} + +/* Drag handle on the pane's left edge. */ +.aicp-resize { + position: absolute; + top: 0; + bottom: 0; + left: -4px; + width: 8px; + z-index: 1; + cursor: col-resize; + touch-action: none; +} +.aicp-resize::after { + content: ''; + position: absolute; + top: 0; + bottom: 0; + left: 3px; + width: 2px; + transition: background-color .15s; +} +.aicp-resize:hover::after, +html.ai-pane-resizing .aicp-resize::after { + background: #5961ff; +} + +/* ---------------------------------------------------------------- header -- */ +.aicp-head { + display: flex; + align-items: center; + gap: 2px; + flex: none; + padding: 10px 10px 10px 16px; + border-bottom: 1px solid #E8EBEC; +} +.aicp-title { + flex: 1; + display: flex; + align-items: center; + gap: 8px; + margin: 0; + font-size: 15px; + font-weight: 600; + line-height: 1.3; +} +.aicp-title svg, +.aicp-tab svg { + flex: none; + color: #FF4438; +} +.aicp-btn { + display: inline-flex; + align-items: center; + justify-content: center; + width: 32px; + height: 32px; + border: 0; + border-radius: 6px; + background: none; + color: #5C707A; + cursor: pointer; +} +.aicp-btn:hover { + background: #F0F0F0; + color: #091A23; +} +.aicp-btn:focus-visible, +.aicp-send:focus-visible, +.aicp-chip:focus-visible, +.aicp-copy:focus-visible, +.aicp-tab:focus-visible, +.aicp-md a:focus-visible, +.aicp-source:focus-visible, +.aicp-vote:focus-visible, +.aicp-rate-send:focus-visible, +.aicp-rate-cancel:focus-visible { + outline: 2px solid #5961ff; + outline-offset: 1px; +} + +/* ------------------------------------------------------------ transcript -- */ +.aicp-log { + position: relative; /* offsetParent for the turns, which the script scrolls to */ + flex: 1 1 0; + min-height: 0; + overflow-y: auto; + overscroll-behavior: contain; + padding: 16px; +} +.aicp-notice { + margin: 0 0 16px; + padding: 8px 10px; + border-radius: 6px; + background: #F9F9F9; + color: #5C707A; + font-size: 12px; + line-height: 1.45; +} +.aicp-turn { + display: flex; + flex-direction: column; + gap: 12px; + padding-bottom: 24px; +} +/* Room for the newest question to sit at the top of the pane while its answer + arrives below it, instead of the pane following the end of the answer. */ +.aicp-turn:last-child { + min-height: 100%; + padding-bottom: 0; +} +.aicp-user { + align-self: flex-end; + max-width: 85%; + padding: 8px 12px; + border-radius: 12px 12px 2px 12px; + background: #E8EBEC; + white-space: pre-wrap; + overflow-wrap: anywhere; +} +.aicp-bot { + overflow-wrap: break-word; +} + +.aicp-empty p { + margin: 0 0 12px; + color: #5C707A; +} +.aicp-chip { + display: inline-flex; + align-items: center; + gap: 6px; + padding: 6px 12px; + border: 1px solid #B9C2C6; + border-radius: 999px; + background: #fff; + color: #091A23; + font: inherit; + cursor: pointer; +} +.aicp-chip:hover { + border-color: #091A23; +} +.aicp-chip svg { + color: #FF4438; +} + +.aicp-dots { + display: inline-flex; + gap: 4px; + padding: 8px 0; +} +.aicp-dots i { + width: 6px; + height: 6px; + border-radius: 50%; + background: #8A99A0; + animation: aicp-pulse 1.2s infinite ease-in-out; +} +.aicp-dots i:nth-child(2) { animation-delay: .15s; } +.aicp-dots i:nth-child(3) { animation-delay: .3s; } +@keyframes aicp-pulse { + 0%, 80%, 100% { opacity: .25; } + 40% { opacity: 1; } +} + +.aicp-status { + margin: 8px 0 0; + color: #5C707A; + font-size: 12px; +} +.aicp-status.is-error { + color: #D52D1F; +} +.aicp-status a { + color: inherit; + text-decoration: underline; +} + +.aicp-sources { + display: flex; + flex-wrap: wrap; + gap: 6px; + margin-top: 12px; +} +.aicp-sources-label { + width: 100%; + color: #5C707A; + font-size: 11px; + font-weight: 600; + letter-spacing: .04em; + text-transform: uppercase; +} +.aicp-source { + max-width: 100%; + overflow: hidden; + padding: 2px 8px; + border: 1px solid #B9C2C6; + border-radius: 6px; + background: #FBFFE8; + color: #091A23; + font-size: 12px; + text-decoration: none; + text-overflow: ellipsis; + white-space: nowrap; +} +.aicp-source:hover { + text-decoration: underline; +} + +/* ---------------------------------------------------------------- rating -- */ +.aicp-rate { + display: flex; + flex-wrap: wrap; + align-items: center; + gap: 2px; + margin-top: 12px; + padding-top: 8px; + border-top: 1px solid #E8EBEC; + color: #5C707A; + font-size: 12px; +} +.aicp-rate-label { + margin-right: 6px; +} +.aicp-vote { + display: inline-flex; + align-items: center; + justify-content: center; + width: 28px; + height: 28px; + border: 0; + border-radius: 6px; + background: none; + color: #5C707A; + cursor: pointer; +} +.aicp-vote:hover:not(:disabled) { + background: #F0F0F0; + color: #091A23; +} +.aicp-vote:disabled { + cursor: default; +} +.aicp-vote:disabled:not([aria-pressed="true"]) { + opacity: .4; +} +.aicp-vote[aria-pressed="true"] { + color: #091A23; +} +.aicp-vote[aria-pressed="true"] svg { + fill: #DCFF1E; +} +.aicp-rate-note { + margin-left: 6px; +} +.aicp-rate-note:focus { + outline: none; +} +.aicp-rate-note.is-error { + color: #D52D1F; +} +.aicp-rate-form { + display: flex; + flex-basis: 100%; + flex-direction: column; + gap: 8px; + margin-top: 6px; +} +.aicp-rate-ask { + display: flex; + flex-direction: column; + gap: 4px; + color: #091A23; + font-size: 13px; +} +.aicp-rate-ask i { + color: #5C707A; + font-size: 12px; + font-style: normal; +} +.aicp-rate-ask textarea { + width: 100%; + padding: 6px 8px; + border: 1px solid #B9C2C6; + border-radius: 6px; + background: #fff; + color: inherit; + font: inherit; + resize: vertical; +} +.aicp-rate-ask textarea:focus { + border-color: #091A23; + outline: 0; + box-shadow: 0 0 0 3px rgba(89, 97, 255, .18); +} +/* Turnstile's widget is 65px tall; hold the room so the buttons don't jump. */ +.aicp-check { + min-height: 65px; +} +.aicp-rate-actions { + display: flex; + gap: 8px; +} +.aicp-rate-send, +.aicp-rate-cancel { + padding: 5px 12px; + border-radius: 6px; + font: inherit; + font-size: 13px; + cursor: pointer; +} +.aicp-rate-send { + border: 1px solid #091A23; + background: #091A23; + color: #fff; +} +.aicp-rate-send:hover:not(:disabled) { + border-color: #FF4438; + background: #FF4438; +} +.aicp-rate-send:disabled { + border-color: #E8EBEC; + background: #E8EBEC; + color: #8A99A0; + cursor: default; +} +.aicp-rate-cancel { + border: 1px solid #B9C2C6; + background: #fff; + color: #091A23; +} +.aicp-rate-cancel:hover { + border-color: #091A23; +} + +/* -------------------------------------------------------- answer content -- */ +.aicp-md > :first-child { margin-top: 0; } +.aicp-md > :last-child { margin-bottom: 0; } +.aicp-md p, +.aicp-md ul, +.aicp-md ol, +.aicp-md blockquote, +.aicp-md .aicp-code, +.aicp-md .aicp-table { + margin: 0 0 10px; +} +.aicp-md ul, +.aicp-md ol { + padding-left: 20px; +} +.aicp-md ul { list-style: disc; } +.aicp-md ol { list-style: decimal; } +.aicp-md li { margin: 3px 0; } +.aicp-md li > ul, +.aicp-md li > ol, +.aicp-md li > .aicp-code { + margin: 4px 0 6px; +} +.aicp-md h3, +.aicp-md h4, +.aicp-md h5 { + margin: 16px 0 6px; + font-weight: 600; + line-height: 1.3; +} +.aicp-md h3 { font-size: 16px; } +.aicp-md h4 { font-size: 15px; } +.aicp-md h5 { font-size: 14px; } +.aicp-md strong { font-weight: 600; } +.aicp-md a { + color: inherit; + text-decoration: underline; + text-decoration-color: #8A99A0; + text-underline-offset: 2px; +} +.aicp-md a:hover { + text-decoration-color: #FF4438; +} +.aicp-md code { + padding: 1px 4px; + border-radius: 4px; + background: #F0F0F0; + font-family: 'Space Mono', 'SF Mono', ui-monospace, monospace; + font-size: .86em; +} +.aicp-md blockquote { + padding-left: 10px; + border-left: 3px solid #B9C2C6; + color: #4C4C4C; +} +.aicp-md hr { + margin: 14px 0; + border: 0; + border-top: 1px solid #E8EBEC; +} + +.aicp-code { + overflow: hidden; + border-radius: 8px; + background: #0f172a; +} +.aicp-code-bar { + display: flex; + align-items: center; + justify-content: space-between; + padding: 4px 6px 4px 12px; + background: #1e293b; + color: #94a3b8; + font-family: 'Space Mono', 'SF Mono', ui-monospace, monospace; + font-size: 11px; +} +.aicp-copy { + padding: 1px 8px; + border: 1px solid #334155; + border-radius: 4px; + background: none; + color: #cbd5e1; + font: inherit; + cursor: pointer; +} +.aicp-copy:hover { + border-color: #64748b; + color: #fff; +} +.aicp-code pre { + margin: 0; + padding: 12px; + overflow-x: auto; + background: none; + color: #e2e8f0; + font-family: 'Space Mono', 'SF Mono', ui-monospace, monospace; + font-size: 12.5px; + line-height: 1.6; +} +.aicp-code pre code { + padding: 0; + background: none; + color: inherit; + font-size: inherit; + white-space: pre; +} + +.aicp-table { + overflow-x: auto; +} +.aicp-table table { + border-collapse: collapse; + font-size: 13px; +} +.aicp-table th, +.aicp-table td { + padding: 4px 8px; + border: 1px solid #E8EBEC; + text-align: left; + vertical-align: top; +} +.aicp-table th { + background: #F9F9F9; + font-weight: 600; +} + +/* ----------------------------------------------------------------- input -- */ +.aicp-form { + flex: none; + margin: 0; + padding: 10px 12px 12px; + border-top: 1px solid #E8EBEC; +} +.aicp-field { + display: flex; + align-items: flex-end; + gap: 6px; + padding: 6px 6px 6px 12px; + border: 1px solid #B9C2C6; + border-radius: 10px; + background: #fff; +} +.aicp-field:focus-within { + border-color: #091A23; + box-shadow: 0 0 0 3px rgba(89, 97, 255, .18); +} +.aicp-input { + flex: 1; + min-width: 0; + max-height: 160px; + margin: 0; + padding: 5px 0; + border: 0; + outline: 0; + background: none; + color: inherit; + font: inherit; + line-height: 1.45; + resize: none; +} +.aicp-input::placeholder { + color: #8A99A0; +} +.aicp-send { + flex: none; + display: inline-flex; + align-items: center; + justify-content: center; + width: 32px; + height: 32px; + border: 0; + border-radius: 8px; + background: #091A23; + color: #fff; + cursor: pointer; +} +.aicp-send:hover:not(:disabled) { + background: #FF4438; +} +.aicp-send:disabled { + background: #E8EBEC; + color: #8A99A0; + cursor: default; +} +.aicp-foot { + margin: 8px 2px 0; + color: #5C707A; + font-size: 11px; + line-height: 1.4; +} + +/* ------------------------------------------------- collapsed pane's tab -- */ +.aicp-tab { + position: fixed; + top: 50%; + right: 0; + z-index: 59; + display: flex; + flex-direction: column; + align-items: center; + gap: 8px; + padding: 12px 7px; + border: 1px solid #B9C2C6; + border-right: 0; + border-radius: 8px 0 0 8px; + background: #fff; + color: #091A23; + box-shadow: -4px 0 14px rgba(9, 26, 35, .12); + font-family: 'Space Grotesk', ui-sans-serif, system-ui, sans-serif; + font-size: 12px; + font-weight: 600; + cursor: pointer; + transform: translateY(-50%); +} +.aicp-tab[hidden] { + display: none; +} +.aicp-tab:hover { + background: #F9F9F9; +} +.aicp-tab span { + writing-mode: vertical-rl; + transform: rotate(180deg); +} + +@media (max-width: 639px) { + .aicp { + border-left: 0; + } + .aicp-resize { + display: none; + } +} + +@media (prefers-reduced-motion: reduce) { + .aicp { + transition: none; + } + .aicp-dots i { + animation: none; + opacity: .6; + } +} + +/* ------------------------------------------------------------ Iris pages -- + Iris pages are dark (body.iris-dark, see partials/is-iris-page.html), and + so is the pane on them, in the --iris-* colors that iris-dark.css sets on + body. That file only restyles
    , and the pane is outside it. */ +body.iris-dark .aicp { + border-color: var(--iris-border); + background: var(--iris-bg); + color: var(--iris-text); + color-scheme: dark; +} +/* The restore placeholder is on html, outside body and its colors, so these + are --iris-bg and --iris-border. */ +html.ai-pane-restoring:has(> body.iris-dark)::after { + border-color: #3d5866; + background: #091A23; +} + +body.iris-dark .aicp-head, +body.iris-dark .aicp-rate, +body.iris-dark .aicp-md hr, +body.iris-dark .aicp-md blockquote, +body.iris-dark .aicp-table th, +body.iris-dark .aicp-table td, +body.iris-dark .aicp-form { + border-color: var(--iris-border); +} +body.iris-dark .aicp-btn, +body.iris-dark .aicp-vote, +body.iris-dark .aicp-notice, +body.iris-dark .aicp-empty p, +body.iris-dark .aicp-status, +body.iris-dark .aicp-sources-label, +body.iris-dark .aicp-rate, +body.iris-dark .aicp-rate-ask i, +body.iris-dark .aicp-md blockquote, +body.iris-dark .aicp-foot { + color: var(--iris-muted); +} +body.iris-dark .aicp-title, +body.iris-dark .aicp-md :is(h3, h4, h5, strong), +body.iris-dark .aicp-vote[aria-pressed="true"] { + color: var(--iris-heading); +} +body.iris-dark .aicp-btn:hover, +body.iris-dark .aicp-vote:hover:not(:disabled) { + background: var(--iris-surface-hover); + color: var(--iris-heading); +} +body.iris-dark .aicp-status.is-error, +body.iris-dark .aicp-rate-note.is-error { + color: #ff6b6b; /* the error red iris-dark.css uses in the scripted demos */ +} + +body.iris-dark .aicp-notice, +body.iris-dark .aicp-md :not(pre) > code, +body.iris-dark .aicp-table th { + background: var(--iris-surface); +} +body.iris-dark .aicp-user { + background: var(--iris-surface-hover); + color: var(--iris-heading); +} + +/* Chips, source links, and fields */ +body.iris-dark .aicp-chip, +body.iris-dark .aicp-source, +body.iris-dark .aicp-rate-cancel, +body.iris-dark .aicp-rate-ask textarea, +body.iris-dark .aicp-field { + border-color: var(--iris-border); + background: var(--iris-surface); + color: var(--iris-text); +} +body.iris-dark .aicp-chip:hover, +body.iris-dark .aicp-rate-cancel:hover, +body.iris-dark .aicp-rate-ask textarea:focus, +body.iris-dark .aicp-field:focus-within { + border-color: var(--iris-muted); +} +body.iris-dark .aicp-rate-ask { + color: var(--iris-text); +} + +/* Send buttons: light on the dark pane, red on hover as on light pages. */ +body.iris-dark .aicp-send, +body.iris-dark .aicp-rate-send { + border-color: var(--iris-heading); + background: var(--iris-heading); + color: var(--iris-bg); +} +body.iris-dark .aicp-send:hover:not(:disabled), +body.iris-dark .aicp-rate-send:hover:not(:disabled) { + border-color: var(--iris-accent); + background: var(--iris-accent); + color: #fff; +} +body.iris-dark .aicp-send:disabled, +body.iris-dark .aicp-rate-send:disabled { + border-color: var(--iris-surface-hover); + background: var(--iris-surface-hover); + color: var(--iris-muted); +} + +/* Code blocks, framed like the ones on Iris pages */ +body.iris-dark .aicp-code { + border: 1px solid var(--iris-border); + background: var(--iris-surface); +} +body.iris-dark .aicp-code-bar { + background: var(--iris-surface-hover); + color: var(--iris-muted); +} +body.iris-dark .aicp-code pre { + color: var(--iris-text); +} +body.iris-dark .aicp-copy { + border-color: var(--iris-border); + color: var(--iris-text); +} +body.iris-dark .aicp-copy:hover { + border-color: var(--iris-muted); + color: var(--iris-heading); +} + +/* The collapsed pane's tab */ +body.iris-dark .aicp-tab { + border-color: var(--iris-border); + background: var(--iris-surface); + color: var(--iris-text); + box-shadow: -4px 0 14px rgba(0, 0, 0, .35); +} +body.iris-dark .aicp-tab:hover { + background: var(--iris-surface-hover); +} diff --git a/static/js/ai-chat-pane.js b/static/js/ai-chat-pane.js new file mode 100644 index 0000000000..5c6fe44bec --- /dev/null +++ b/static/js/ai-chat-pane.js @@ -0,0 +1,1222 @@ +/* ========================================================================= + Redis Copilot, beside the page + ------------------------------------------------------------------------- + "Ask AI to explain this page" (layouts/partials/meta-links.html) used to + navigate to redis.io/chat. With this script, a plain click asks the same + question in a pane docked to the right of the page, so the reader keeps the + page in view and can follow up without leaving it. The link keeps its + /chat?q= href, so a new-tab click still opens the full chat page. + + The pane is a client of the chat app's own endpoints, the ones its page + script (redis.io/chat/static/js/bubble.js) calls: + + POST /chat/ask form q, tags -> the answer, HTML-escaped Markdown, + streamed while it is generated + GET /chat/references query q, tags -> [{category, title, url}] + POST /chat/reset -> forget the conversation + POST /docusight/api/rate/copilot/ -> a thumbs up (5) or down (1) on an + answer; a downvote carries a + Turnstile token and a comment + + q is URI-encoded once more before the form encoding, because that is what + the chat page sends. The conversation lives server side, against the chat + app's session cookie, which is what lets a follow-up say "it". + + That cookie is shared by every tab and lasts until the browser closes, so + there is one conversation per browser, and the pane shows that one in + every tab. It keeps the transcript in localStorage, which every tab shares + too: the transcript follows the reader to the next page, and a tab shows + what another adds or clears as it happens. One tab asks at a time: a + question is stored the moment it's asked, and while another tab's question + is on its way, Send waits. A transcript left from an earlier + browser session goes, because the chat app's conversation went with it + (see newSession). When there's no conversation, the first question resets + the server side first, so the bot never answers from context the reader + can't see. Whether the pane is open, and its width, stay per tab, in + sessionStorage. + + /chat only exists on redis.io, and it sends no CORS headers. Staging builds + are on redis.io too, so the pane works there as is. A local build needs a + proxy in front of hugo server that forwards /chat to redis.io. + ========================================================================= */ +(function () { + 'use strict'; + + if (window.AIChatPane) return; + window.AIChatPane = {}; + + var ASK = '/chat/ask'; + var REFS = '/chat/references'; + var RESET = '/chat/reset'; + var FULL = '/chat/'; + var KEY = 'aiChatPane.v1'; // sessionStorage: this tab's pane, open or not, and its width + var TRANSCRIPT = 'aiChatPane.chat.v1'; // localStorage: the conversation, for every tab + var SESSION = 'aiChatPane'; // the cookie that marks this browser session + var DEFAULT_WIDTH = 440; + var KEEP = 40; // messages kept in the stored transcript + var MAX_SOURCES = 4; + var TIMEOUT = 90000; // end to end, for one answer + var GRACE = 5000; // past its time limit, an answer still marked as arriving belongs to a tab that's gone + var POST_TIMEOUT = 15000; // for a reset + var NOTICE = 'Redis Copilot is powered by OpenAI API and is for general info only; ' + + 'avoid inputting confidential or personal data.'; + + /* Ratings go where the chat page sends its own, but only from redis.io: like + the chat page, a local build sends nothing, and logs what it would have + sent. The site key is the chat page's, which the endpoint checks downvote + tokens against. It only works on redis.io, so elsewhere the downvote form + uses Cloudflare's always-pass test key. */ + var RATE = '/docusight/api/rate/copilot/'; + var LIVE = location.hostname === 'redis.io'; + var SITEKEY = LIVE ? '0x4AAAAAAB0a8ZJzBy7OAdPg' : '1x00000000000000000000AA'; + var TURNSTILE = 'https://challenges.cloudflare.com/turnstile/v0/api.js?render=explicit'; + + /* Links in answers point at production. Off redis.io they are pointed at this + build instead, and links to docs pages open in this tab, so the reader + takes the pane along. */ + var DOCS = 'https://redis.io/docs/latest/'; + var BASE = basePath(); + + var THUMB = '' + + ''; + var ICONS = { + sparkles: '', + external: svg(''), + compose: svg(''), + collapse: svg(''), + send: svg(''), + stop: '', + up: svg(THUMB), + down: svg('' + THUMB + '') + }; + + var html = document.documentElement; + var layout = window.aiChatPaneLayout || function () { return DEFAULT_WIDTH; }; + /* With localStorage blocked, each page keeps the conversation to itself. */ + var SHARED = (function () { try { return !!window.localStorage; } catch (e) { return false; } })(); + if (newSession()) { + try { localStorage.removeItem(TRANSCRIPT); } catch (e) {} + } + var state = load(); + var ui = null; // built on first use + var busy = null; // { ctrl, bot, node } while this tab's answer streams + var opener = null; // where focus goes back to on collapse + var owners = new WeakMap(); // answer node -> its message + var views = {}; // message id -> its answer node + + /* ------------------------------------------------------------- state -- */ + + function load() { + var s = read(sessionStorage, KEY); + return { open: !!s.open, width: Number(s.width) || DEFAULT_WIDTH, messages: loadMessages() }; + } + + function read(store, key) { + var s = null; + try { s = JSON.parse(store.getItem(key)); } catch (e) {} + return s && typeof s === 'object' ? s : {}; + } + + function loadMessages() { + var s = read(localStorage, TRANSCRIPT), out = []; + (Array.isArray(s.messages) ? s.messages : []).forEach(function (m) { + /* Every message the pane saves has an id, which is how tabs match them. */ + if (!m || typeof m.id !== 'string' || typeof m.text !== 'string' || (m.role !== 'user' && m.role !== 'assistant')) return; + /* An answer marked as arriving is being written by this tab or another, + within the time limit. Past it, the tab writing it has gone. If its + question never went out, the bot never saw it, so both go. */ + if (m.status === 'streaming' && !(Date.now() - (m.since || 0) < limitOf(m))) { + if (m.unsent) { + if (out.length && out[out.length - 1].role === 'user') out.pop(); + return; + } + m.status = 'interrupted'; + } + out.push(m); + }); + return out; + } + + /* How long an answer can be on its way. One whose question hasn't gone out + is only waiting for a reset, which gives up sooner (post). */ + function limitOf(m) { + return (m.unsent ? POST_TIMEOUT : TIMEOUT) + GRACE; + } + + function newId() { + return Date.now().toString(36) + Math.random().toString(36).slice(2, 8); + } + + /* The chat app's conversation cookie ends with the browser session, and the + conversation with it. A transcript stored in an earlier session would show + context the bot no longer has, so it goes. This cookie, also a session + cookie, is how the pane tells: it's missing in a new browser session. */ + function newSession() { + if (new RegExp('(?:^|;\\s*)' + SESSION + '=1(?:;|$)').test(document.cookie)) return false; + document.cookie = SESSION + '=1; path=/; SameSite=Lax' + (location.protocol === 'https:' ? '; Secure' : ''); + return true; + } + + /* A question still waiting to go out is saved too, the moment it's asked: + the other tabs have to see it coming, or they could ask over it. If this + page goes first, it's taken out again (pagehide). */ + function save() { + saveView(); + try { localStorage.setItem(TRANSCRIPT, JSON.stringify({ messages: state.messages.slice(-KEEP) })); } catch (e) {} + } + + /* Just this tab's pane, so opening or resizing it never writes the + conversation over what another tab has. */ + function saveView() { + try { sessionStorage.setItem(KEY, JSON.stringify({ open: state.open, width: state.width })); } catch (e) {} + } + + /* Another tab saved the conversation: show what it has. Turns are added and + answers repainted in place, so a scrolled view or a downvote comment being + typed survives; only a conversation that no longer starts the same way, + after a new chat, is drawn again. */ + function adopt(messages) { + var resave = false; + if (busy) { + var i = indexOf(messages, busy.bot.id); + if (i !== -1) messages[i] = busy.bot; + /* Only a new chat saves an empty conversation: it ends this question. */ + else if (!messages.length) { stop(); busy = null; } + /* Anything else without this question was saved by a tab that hadn't + seen it yet. Keep it, and save it again for the other tabs. */ + else { + var j = state.messages.indexOf(busy.bot); + if (j > 0 && state.messages[j - 1].role === 'user') messages.push(state.messages[j - 1]); + messages.push(busy.bot); + resave = true; + } + } + var old = state.messages; + state.messages = messages; + if (resave) save(); + expireLater(); + if (!ui) { + if (messages.length) build(); + return; + } + var continues = old.length && old.length <= messages.length && + old.every(function (m, k) { return m.id === messages[k].id; }); + if (!continues) { + var top = ui.log.scrollTop; + renderAll(); + ui.log.scrollTop = top; + } else { + messages.forEach(function (m, k) { + if (k >= old.length) addTurn(m); + else if (m === old[k]) return; + /* Unchanged: keep the message the answer's buttons act on. */ + else if (JSON.stringify(m) === JSON.stringify(old[k])) messages[k] = old[k]; + else if (m.role === 'assistant') refresh(m, old[k]); + }); + } + updateTab(); + sync(); + } + + function indexOf(messages, id) { + for (var k = 0; k < messages.length; k++) if (messages[k].id === id) return k; + return -1; + } + + /* An answer from another tab, finished, rated, or stopped since. */ + function refresh(m, was) { + var node = views[m.id]; + if (!node || !node.isConnected) return; + owners.set(node, m); + paint(node, m); + var row = node.querySelector('.aicp-rate'); + if (row && m.rating !== was.rating) row.replaceWith(rateRow(m)); + } + + /* Whether another tab is getting an answer now. */ + function answeringElsewhere() { + return state.messages.some(function (m) { return m.status === 'streaming' && !(busy && m === busy.bot); }); + } + + /* When the tab writing an answer goes without saying so, the answer is + marked as stopped once the time limit has passed. */ + var expiry = 0; + function expireLater() { + clearTimeout(expiry); + var due = Infinity; + state.messages.forEach(function (m) { + if (m.status === 'streaming' && !(busy && m === busy.bot)) due = Math.min(due, (m.since || 0) + limitOf(m)); + }); + if (due < Infinity) expiry = setTimeout(function () { adopt(loadMessages()); }, Math.max(0, due - Date.now()) + 50); + } + + /* ---------------------------------------------------------------- UI -- */ + + function build() { + var pane = el('aside', 'aicp'); + pane.id = 'ai-chat-pane'; + pane.hidden = true; + pane.setAttribute('aria-label', 'AI chatbot'); + pane.innerHTML = + '
    ' + + '
    ' + + '

    ' + ICONS.sparkles + 'AI chatbot

    ' + + '' + + ICONS.external + 'Continue in the full chat page' + + '' + + '' + + '
    ' + + '
    ' + + '
    ' + + '
    ' + + '' + + '' + + '
    ' + + '

    Verify any information from Redis Copilot before using it.

    ' + + '
    '; + + var tab = el('button', 'aicp-tab'); + tab.type = 'button'; + tab.hidden = true; + tab.title = 'Open the AI chatbot'; + tab.setAttribute('aria-controls', pane.id); + tab.setAttribute('aria-expanded', 'false'); + tab.innerHTML = ICONS.sparkles + 'AI chatbot'; + + document.body.appendChild(pane); + document.body.appendChild(tab); + ui = { + pane: pane, + tab: tab, + log: pane.querySelector('.aicp-log'), + form: pane.querySelector('.aicp-form'), + input: pane.querySelector('.aicp-input'), + send: pane.querySelector('.aicp-send'), + handle: pane.querySelector('.aicp-resize') + }; + wire(); + renderAll(); + sync(); + updateTab(); + } + + function wire() { + ui.form.addEventListener('submit', function (e) { + e.preventDefault(); + if (busy) { stop(); return; } + var q = ui.input.value.trim(); + /* The text stays put when another tab is getting an answer. */ + if (!q || !ask(q)) return; + ui.input.value = ''; + grow(); + sync(); + }); + + ui.input.addEventListener('input', function () { grow(); sync(); }); + ui.input.addEventListener('keydown', function (e) { + if (e.key !== 'Enter' || e.shiftKey || e.isComposing) return; + e.preventDefault(); + /* Enter sends; it never stops an answer that is still arriving. */ + if (busy) return; + if (ui.form.requestSubmit) ui.form.requestSubmit(); + else ui.form.dispatchEvent(new Event('submit', { cancelable: true })); + }); + + /* The search modal opens on "/" and Cmd+K from a document-level listener, + wherever they are typed. Keep the pane's keystrokes to the pane. */ + ['keydown', 'keyup', 'keypress'].forEach(function (type) { + ui.pane.addEventListener(type, function (e) { + if (type === 'keydown' && e.key === 'Escape') { + e.preventDefault(); + /* In a downvote's comment box, Escape backs out of the form only. */ + var form = e.target.closest('.aicp-rate-form'); + if (form) form.querySelector('[data-act="cancel-rating"]').click(); + else collapse(); + } + e.stopPropagation(); + }); + }); + + ui.pane.addEventListener('click', function (e) { + var t = e.target.closest('[data-act]'); + if (!t) return; + var act = t.getAttribute('data-act'); + if (act === 'collapse') collapse(); + else if (act === 'reset') reset(); + else if (act === 'copy') copy(t); + else if (act === 'explain') explain(pageLink()); + else if (act === 'up' || act === 'down') vote(t.closest('.aicp-bot'), act); + }); + + ui.tab.addEventListener('click', function () { + opener = ui.tab; + open({ focus: true }); + pinLast(true); + }); + + wireResize(); + } + + function wireResize() { + var frame = 0, x = 0; + ui.handle.addEventListener('pointerdown', function (e) { + if (e.button !== 0) return; + e.preventDefault(); + ui.handle.setPointerCapture(e.pointerId); + html.classList.add('ai-pane-resizing'); + + function move(ev) { + x = ev.clientX; + if (frame) return; + frame = requestAnimationFrame(function () { + frame = 0; + state.width = layout({ open: true, width: ui.pane.getBoundingClientRect().right - x }); + }); + } + function done() { + ui.handle.removeEventListener('pointermove', move); + ui.handle.removeEventListener('pointerup', done); + ui.handle.removeEventListener('pointercancel', done); + html.classList.remove('ai-pane-resizing'); + saveView(); + } + ui.handle.addEventListener('pointermove', move); + ui.handle.addEventListener('pointerup', done); + ui.handle.addEventListener('pointercancel', done); + }); + ui.handle.addEventListener('dblclick', function () { + state.width = DEFAULT_WIDTH; + layout(state); + saveView(); + }); + } + + function open(opts) { + opts = opts || {}; + if (!ui) build(); + state.open = true; + saveView(); + layout(state); + ui.pane.hidden = false; + if (!ui.pane.classList.contains('is-in')) { + /* Shown from display: none, the pane has no style to transition from. + Reading layout first gives it one, so it slides in. */ + if (!opts.instant) ui.pane.getBoundingClientRect(); + ui.pane.classList.add('is-in'); + } + updateTab(); + if (opts.focus) ui.input.focus({ preventScroll: true }); + } + + function collapse() { + if (!ui || !state.open) return; + var hadFocus = ui.pane.contains(document.activeElement); + state.open = false; + saveView(); + layout(state); + ui.pane.classList.remove('is-in'); + setTimeout(function () { if (!state.open) ui.pane.hidden = true; }, reducedMotion() ? 0 : 220); + updateTab(); + if (hadFocus) { + /* Back to whatever opened the pane, else the tab that reopens it, but + only to one that's on screen. Below 1280px the link's rail is hidden, + and focusing a hidden element leaves focus nowhere. */ + var back = [opener, ui.tab].filter(function (t) { + return t && document.contains(t) && t.getClientRects().length > 0; + })[0]; + if (back) back.focus({ preventScroll: true }); + } + } + + function updateTab() { + if (!ui) return; + ui.tab.hidden = state.open || !state.messages.length; + ui.tab.setAttribute('aria-expanded', String(state.open)); + } + + /* Send turns into Stop while an answer streams, and waits while another + tab's answer arrives. */ + function sync() { + if (!ui) return; + var streaming = !!busy; + var elsewhere = !streaming && answeringElsewhere(); + var label = streaming ? 'Stop' : 'Send'; + ui.send.disabled = !streaming && (elsewhere || !ui.input.value.trim()); + ui.input.placeholder = elsewhere ? 'Wait for the answer in your other tab' : 'Ask a follow-up question'; + ui.send.title = label; + ui.send.innerHTML = (streaming ? ICONS.stop : ICONS.send) + '' + label + ''; + ui.log.setAttribute('aria-busy', String(streaming)); + } + + function grow() { + ui.input.style.height = 'auto'; + ui.input.style.height = Math.min(ui.input.scrollHeight, 160) + 'px'; + } + + /* ------------------------------------------------------------ render -- */ + + function renderAll() { + ui.log.innerHTML = ''; + views = {}; + ui.log.appendChild(el('p', 'aicp-notice', NOTICE)); + if (!state.messages.length) { + ui.log.appendChild(emptyState()); + return; + } + state.messages.forEach(addTurn); + } + + /* A question starts a turn, and its answer joins it. */ + function addTurn(m) { + var turns = ui.log.querySelectorAll('.aicp-turn'), turn = turns[turns.length - 1]; + if (m.role === 'user' || !turn) turn = ui.log.appendChild(el('div', 'aicp-turn')); + turn.appendChild(m.role === 'user' ? el('div', 'aicp-user', m.label || m.text) : botNode(m)); + } + + function emptyState() { + var box = el('div', 'aicp-empty'); + box.appendChild(el('p', null, 'Ask anything about Redis.')); + if (pageLink()) { + var chip = el('button', 'aicp-chip'); + chip.type = 'button'; + chip.setAttribute('data-act', 'explain'); + chip.innerHTML = ICONS.sparkles + 'Explain this page'; + box.appendChild(chip); + } + return box; + } + + function botNode(m) { + var node = el('div', 'aicp-bot'); + node.appendChild(el('div', 'aicp-answer')); + owners.set(node, m); + views[m.id] = node; + /* Drawn again while it streams: the stream paints into the new node. */ + if (busy && busy.bot === m) busy.node = node; + paint(node, m); + return node; + } + + /* Runs on every streamed chunk. The rating row sits outside what it + rewrites, so a downvote comment being typed survives a late repaint: + sources can arrive after the answer has finished. */ + function paint(node, m) { + var body = m.text ? md(m.text) + : m.status === 'streaming' ? '' + : ''; + node.firstChild.innerHTML = '
    ' + body + '
    ' + statusLine(m) + sources(m); + if (m.status === 'done' && !node.querySelector('.aicp-rate')) node.appendChild(rateRow(m)); + } + + function statusLine(m) { + if (m.status === 'stopped') return '

    Stopped.

    '; + if (m.status === 'interrupted') return '

    This answer stopped when you left the page.

    '; + if (m.status !== 'error') return ''; + var retry = FULL + '?' + new URLSearchParams({ q: m.q || '' }); + return '

    ' + esc(m.error || "Couldn't get an answer.") + + ' Ask in the full chat page.

    '; + } + + function sources(m) { + if (m.status !== 'done' || !m.refs || !m.refs.length) return ''; + var links = m.refs.map(function (r) { + var href = safeHref(r.url); + return href ? anchor(href, esc(r.title), 'aicp-source', r.title) : ''; + }).join(''); + return links ? '
    Sources' + links + '
    ' : ''; + } + + /* Puts a turn's question at the top of the pane, so its answer reads from + the start while it arrives. The last turn is at least the pane's height + (see .aicp-turn:last-child), so there is always room to scroll that far. */ + function pin(turn, instant) { + /* The first question stays below the notice about what the bot is. */ + var first = ui.log.querySelector('.aicp-turn') === turn; + var top = first ? 0 : Math.max(0, turn.offsetTop - 16); + if (instant || reducedMotion()) ui.log.scrollTop = top; + else ui.log.scrollTo({ top: top, behavior: 'smooth' }); + } + + function pinLast(instant) { + var turns = ui.log.querySelectorAll('.aicp-turn'); + if (turns.length) pin(turns[turns.length - 1], instant); + } + + /* -------------------------------------------------------------- rate -- + The chat page's thumbs, under each finished answer. Up is sent at once. + Down first opens a box for an optional comment and a Turnstile check, + because the endpoint wants a token with a downvote. */ + + function rateRow(m) { + var row = el('div', 'aicp-rate'); + if (m.rating) { + row.innerHTML = voteButton(m.rating, true) + '' + + (LIVE ? 'Thanks for your feedback.' : 'Thanks. This is a local build, so nothing was sent.') + ''; + } else { + row.innerHTML = 'How was this answer?' + voteButton('up') + voteButton('down') + + (m.rateFailed ? 'Couldn\'t send your feedback. Try again.' : ''); + } + return row; + } + + function voteButton(kind, chosen) { + var label = kind === 'up' ? 'Good answer' : 'Bad answer'; + return ''; + } + + function vote(node, kind) { + var m = node && owners.get(node); + if (!m || m.rating) return; + if (kind === 'up') rate(node, m, 'up', '', ''); + else downvoteForm(node, m); + } + + function downvoteForm(node, m) { + var row = node.querySelector('.aicp-rate'); + if (row.querySelector('.aicp-rate-form')) return; + var up = row.querySelector('[data-act="up"]'); + var down = row.querySelector('[data-act="down"]'); + var form = el('div', 'aicp-rate-form'); + form.innerHTML = + '' + + '
    ' + + '
    ' + + '' + + '' + + '
    '; + row.appendChild(form); + down.setAttribute('aria-pressed', 'true'); + up.disabled = true; + + var comment = form.querySelector('textarea'); + var send = form.querySelector('.aicp-rate-send'); + var check = form.querySelector('.aicp-check'); + var token = '', widget = null; + + function close() { + if (widget !== null && window.turnstile) { + try { window.turnstile.remove(widget); } catch (e) {} + } + form.remove(); + } + send.addEventListener('click', function () { + if (!token) return; + var text = comment.value.trim(); + close(); + rate(node, m, 'down', text, token); + }); + form.querySelector('.aicp-rate-cancel').addEventListener('click', function () { + close(); + up.disabled = false; + down.setAttribute('aria-pressed', 'false'); + down.focus({ preventScroll: true }); + }); + + comment.focus({ preventScroll: true }); + turnstile().then(function (ts) { + if (!form.isConnected) return; + widget = ts.render(check, { + sitekey: SITEKEY, + /* Iris pages are dark, and so is the pane on them (ai-chat-pane.css). */ + theme: document.body.classList.contains('iris-dark') ? 'dark' : 'light', + callback: function (t) { token = t; send.disabled = false; }, + 'expired-callback': function () { token = ''; send.disabled = true; }, + 'error-callback': function () { + token = ''; + send.disabled = true; + check.textContent = "Couldn't check that you're not a bot. Try again later."; + } + }); + }, function () { + check.textContent = "Couldn't load the check that you're not a bot."; + }); + } + + /* Thanks straight away, as the chat page does. If the send fails, the + buttons come back with a note, so the vote isn't silently lost. */ + function rate(node, m, kind, comment, token) { + m.rating = kind; + delete m.rateFailed; + save(); + redrawRating(node, m); + sendRating(m, kind, comment, token).catch(function (err) { + console.error('[ai-chat-pane] Feedback not sent:', err); + delete m.rating; + m.rateFailed = true; + save(); + if (node.isConnected) redrawRating(node, m); + }); + } + + function redrawRating(node, m) { + var old = node.querySelector('.aicp-rate'), row = rateRow(m); + var focusLost = !document.activeElement || document.activeElement === document.body || + (old && old.contains(document.activeElement)); + if (old) old.replaceWith(row); else node.appendChild(row); + var note = row.querySelector('.aicp-rate-note'); + if (note && focusLost) note.focus({ preventScroll: true }); + } + + /* The fields the chat page sends. origin is the page the rating was given + on, as for the docs' own page ratings (feedback.html); the chat page sends + its own URL there. */ + function sendRating(m, kind, comment, token) { + var data = { + origin: location.origin + location.pathname, + question: m.q || '', + answer: decode(m.text), + comment: comment, + rating: kind === 'up' ? 5 : 1, + 'cf-turnstile-response': token + }; + if (!LIVE) { + console.info('[ai-chat-pane] Feedback not sent from a local build:', data); + return Promise.resolve(); + } + return fetch(RATE, { + method: 'POST', + credentials: 'same-origin', + headers: { 'X-Requested-With': 'XMLHttpRequest' }, + body: new URLSearchParams(data) + }).then(function (r) { if (!r.ok) throw httpError(r.status); }); + } + + var turnstileLoading = null; + + /* Docs pages already load Turnstile for the page rating form, and loading it + twice makes it complain, so wait for that copy when there is one. */ + function turnstile() { + if (window.turnstile) return Promise.resolve(window.turnstile); + if (!turnstileLoading) { + if (!document.querySelector('script[src*="challenges.cloudflare.com/turnstile/"]')) { + var script = document.createElement('script'); + script.src = TURNSTILE; + script.async = true; + document.head.appendChild(script); + } + turnstileLoading = new Promise(function (resolve, reject) { + var tries = 0; + (function wait() { + if (window.turnstile) resolve(window.turnstile); + else if (++tries > 150) { turnstileLoading = null; reject(new Error('Turnstile did not load')); } + else setTimeout(wait, 100); + })(); + }); + } + return turnstileLoading; + } + + /* --------------------------------------------------------------- ask -- */ + + function pageLink() { + return document.querySelector('a[data-ai-chat-open]'); + } + + function promptOf(link) { + try { return new URL(link.href, location.href).searchParams.get('q'); } catch (e) { return null; } + } + + function explain(link) { + var q = link && promptOf(link); + if (!q) return; + var title = link.getAttribute('data-ai-chat-title'); + ask(q, title ? 'Explain this page: ' + title : 'Explain this page'); + } + + /* True if the question went in: it waits while another tab gets an answer. */ + function ask(q, label) { + if (busy || !q) return false; + if (!ui) build(); + /* Another tab may have moved the conversation on since this one last heard. */ + if (SHARED) adopt(loadMessages()); + if (answeringElsewhere()) return false; + var fresh = !state.messages.length; + var user = { id: newId(), role: 'user', text: q }; + if (label) user.label = label; + var bot = { id: newId(), role: 'assistant', q: q, text: '', status: 'streaming', since: Date.now(), refs: [], unsent: true }; + state.messages.push(user, bot); + save(); + + /* The stream paints into job.node, which a redraw replaces (botNode). */ + var ctrl = new AbortController(); + var job = busy = { ctrl: ctrl, bot: bot, node: null }; + if (fresh) renderAll(); + else { addTurn(user); addTurn(bot); } + var turn = ui.log.lastElementChild; + pin(turn, fresh); + updateTab(); + sync(); + var timedOut = false; + var timer = setTimeout(function () { timedOut = true; ctrl.abort(); }, TIMEOUT); + var frame = 0; + function repaint() { + if (frame) return; + frame = requestAnimationFrame(function () { frame = 0; paint(job.node, bot); }); + } + + untilAborted(fresh ? forget() : resetting || Promise.resolve(), ctrl.signal) + .then(function () { + references(q, ctrl.signal).then(function (refs) { + bot.refs = refs; + if (bot.status === 'done') { paint(job.node, bot); save(); } + }); + delete bot.unsent; + save(); + return fetch(ASK, { + method: 'POST', + credentials: 'same-origin', + headers: { 'X-Requested-With': 'XMLHttpRequest' }, + body: new URLSearchParams({ q: encodeURIComponent(q), tags: '' }), + signal: ctrl.signal + }); + }) + .then(function (res) { + if (!res.ok) throw httpError(res.status); + /* A Cloudflare challenge is an HTML page where the answer should be. */ + if (/text\/html/i.test(res.headers.get('content-type') || '')) throw httpError(403); + if (!res.body || !res.body.getReader) return res.text().then(function (t) { bot.text = t; }); + var reader = res.body.getReader(); + var decoder = new TextDecoder(); + return (function pump() { + return reader.read().then(function (r) { + if (r.done) { bot.text += decoder.decode(); return; } + bot.text += decoder.decode(r.value, { stream: true }); + repaint(); + return pump(); + }); + })(); + }) + .then(function () { + if (bot.text.trim()) bot.status = 'done'; + else { bot.status = 'error'; bot.error = 'Redis Copilot sent an empty answer.'; } + }, function (err) { + if (ctrl.signal.aborted && !timedOut) { bot.status = 'stopped'; return; } + bot.status = 'error'; + bot.error = timedOut ? 'Redis Copilot took too long to answer.' : describe(err); + }) + .then(function () { + clearTimeout(timer); + if (frame) { cancelAnimationFrame(frame); frame = 0; } + if (busy === job) busy = null; + paint(job.node, bot); + save(); + sync(); + }); + return true; + } + + function stop() { + if (busy) busy.ctrl.abort(); + } + + function reset() { + stop(); + busy = null; + state.messages = []; + save(); + renderAll(); + updateTab(); + sync(); + forget(); + ui.input.focus({ preventScroll: true }); + } + + var resetting = null; // the POST /chat/reset in flight, if any + + /* Clears the server side of the conversation. One reset at a time, and a + question waits for it: a reset that landed after the next question was + sent would wipe the context the pane is showing. */ + function forget() { + if (!resetting) { + resetting = post(RESET).catch(function () {}).then(function () { resetting = null; }); + } + return resetting; + } + + function references(q, signal) { + var url = REFS + '?' + new URLSearchParams({ q: encodeURIComponent(q), tags: '' }); + return fetch(url, { credentials: 'same-origin', headers: { Accept: 'application/json' }, signal: signal }) + .then(function (r) { return r.ok ? r.json() : []; }) + .then(function (list) { + var seen = {}, out = []; + (Array.isArray(list) ? list : []).forEach(function (r) { + if (!r || !r.url || !r.title || out.length >= MAX_SOURCES) return; + /* The same page often comes back once per product version. */ + var title = String(r.title), key = title.toLowerCase(); + if (seen[key] || seen[r.url]) return; + seen[key] = seen[r.url] = true; + out.push({ title: title, url: String(r.url) }); + }); + return out; + }) + .catch(function () { return []; }); + } + + /* Gives up after a while, so a request that never gets a reply can't hold + a question back for good. */ + function post(url) { + var ctrl = new AbortController(); + var timer = setTimeout(function () { ctrl.abort(); }, POST_TIMEOUT); + return fetch(url, { method: 'POST', credentials: 'same-origin', headers: { 'X-Requested-With': 'XMLHttpRequest' }, signal: ctrl.signal }) + .then(function (r) { clearTimeout(timer); if (!r.ok) throw httpError(r.status); return r; }, + function (err) { clearTimeout(timer); throw err; }); + } + + /* Settles as promise does, or rejects as soon as signal aborts: Stop and the + time limit have to work while a question is still waiting for a reset. */ + function untilAborted(promise, signal) { + return new Promise(function (resolve, reject) { + function abort() { reject(new Error('Aborted')); } + if (signal.aborted) return abort(); + signal.addEventListener('abort', abort); + promise.then(resolve, reject); + }); + } + + function httpError(status) { + var e = new Error('HTTP ' + status); + e.status = status; + return e; + } + + function describe(err) { + var status = err && err.status; + if (status === 404 && /^(localhost|127\.0\.0\.1|\[::1\])$/.test(location.hostname)) { + return 'This local site has no /chat endpoint. Browse it through a proxy that forwards /chat to redis.io.'; + } + if (status === 403 || status === 429) return 'Redis Copilot turned this request away (HTTP ' + status + ').'; + if (status) return "Redis Copilot couldn't answer (HTTP " + status + ').'; + return "Couldn't reach Redis Copilot."; + } + + function copy(button) { + var code = button.closest('.aicp-code').querySelector('code').textContent; + var done = function (label) { + button.textContent = label; + setTimeout(function () { button.textContent = 'Copy'; }, 1500); + }; + if (!navigator.clipboard) { done('Copy failed'); return; } + navigator.clipboard.writeText(code).then(function () { done('Copied'); }, function () { done('Copy failed'); }); + } + + /* ---------------------------------------------------------- Markdown -- + Answers are Markdown. This renders the subset the bot writes: fenced code, + headings, lists nested by indentation, tables, quotes, rules, links, + emphasis and inline code. Text is escaped before any tag is added, and only + these tags are ever produced, so nothing in an answer reaches the page as + markup. It runs on every streamed chunk, so half-written constructs, such + as a code fence not closed yet, have to render sensibly too. */ + + var LIST_ITEM = /^([ \t]*)([-*+]|\d{1,9}[.)])[ \t]+(.*)$/; + var TABLE_RULE = /^[ \t]*\|?[ \t]*:?-+:?[ \t]*(\|[ \t]*:?-+:?[ \t]*)+\|?[ \t]*$/; + + function md(src) { + /* NUL is the placeholder inline() uses; an answer has no business with it. */ + return blocks(decode(src).replace(/\u0000/g, '').replace(/\r\n?/g, '\n').split('\n')); + } + + function indentOf(line) { + return line.match(/^[ \t]*/)[0].replace(/\t/g, ' ').length; + } + + function blocks(lines) { + var out = [], para = [], lists = [], i = 0, m, afterBlank = false; + + function flushPara() { + if (para.length) out.push('

    ' + inline(para.join('\n')) + '

    '); + para = []; + } + function closeLists(indent) { + while (lists.length && lists[lists.length - 1].indent > indent) { + out.push(''); + } + } + function flush() { flushPara(); closeLists(-1); } + + while (i < lines.length) { + var line = lines[i]; + var indent = indentOf(line); + var inItem = lists.length && indent > lists[lists.length - 1].indent; + + if ((m = /^[ \t]*(`{3,}|~{3,})[ \t]*([^`\s]*)/.exec(line))) { + flushPara(); + if (!inItem) closeLists(-1); + var close = new RegExp('^[ \\t]*\\' + m[1][0] + '{' + m[1].length + ',}[ \\t]*$'); + var code = []; + for (i++; i < lines.length && !close.test(lines[i]); i++) { + code.push(lines[i].slice(Math.min(indent, indentOf(lines[i])))); + } + i++; + out.push(codeBlock(code.join('\n'), m[2])); + afterBlank = false; + continue; + } + + if (!line.trim()) { + flushPara(); + /* A list carries on past a blank line if what follows still belongs to it. */ + var next = ''; + for (var j = i + 1; j < lines.length && !next; j++) next = lines[j].trim() ? lines[j] : ''; + if (lists.length && !(next && (LIST_ITEM.test(next) || indentOf(next) > lists[0].indent))) closeLists(-1); + afterBlank = true; + i++; + continue; + } + + if ((m = /^[ \t]*(#{1,6})[ \t]+(.*?)[ \t]*#*[ \t]*$/.exec(line))) { + flush(); + var level = m[1].length <= 2 ? 3 : m[1].length === 3 ? 4 : 5; + out.push('' + inline(m[2]) + ''); + } else if (/^[ \t]*([-*_])([ \t]*\1){2,}[ \t]*$/.test(line)) { + flush(); + out.push('
    '); + } else if (line.indexOf('|') !== -1 && i + 1 < lines.length && TABLE_RULE.test(lines[i + 1])) { + flush(); + var head = cells(line), rows = []; + for (i += 2; i < lines.length && lines[i].indexOf('|') !== -1 && lines[i].trim(); i++) rows.push(cells(lines[i])); + out.push(table(head, rows)); + afterBlank = false; + continue; + } else if (/^[ \t]*>/.test(line)) { + flush(); + var quote = []; + for (; i < lines.length && /^[ \t]*>/.test(lines[i]); i++) quote.push(lines[i].replace(/^[ \t]*> ?/, '')); + out.push('
    ' + blocks(quote) + '
    '); + afterBlank = false; + continue; + } else if ((m = LIST_ITEM.exec(line))) { + flushPara(); + var tag = /\d/.test(m[2]) ? 'ol' : 'ul'; + closeLists(indent); + var top = lists[lists.length - 1]; + if (top && top.indent === indent && top.tag !== tag) { + out.push(''); + top = lists[lists.length - 1]; + } + if (!top || indent > top.indent) { + var start = tag === 'ol' ? parseInt(m[2], 10) : 1; + out.push('<' + tag + (start > 1 ? ' start="' + start + '"' : '') + '>'); + lists.push({ tag: tag, indent: indent }); + } else { + out.push(''); + } + out.push('
  • ' + inline(m[3])); + } else if (lists.length && (inItem || !afterBlank)) { + /* More of the current list item. */ + out.push((afterBlank ? '

    ' : ' ') + inline(line.trim())); + } else { + para.push(line.trim()); + } + afterBlank = false; + i++; + } + flush(); + return out.join(''); + } + + function cells(row) { + var escaped = '\u0001'; + return row.trim().replace(/^\|/, '').replace(/\|$/, '').replace(/\\\|/g, escaped).split('|').map(function (c) { + return c.trim().split(escaped).join('|'); + }); + } + + function table(head, rows) { + return '
    ' + + head.map(function (c) { return ''; }).join('') + + '' + + rows.map(function (r) { + return '' + head.map(function (_, k) { return ''; }).join('') + ''; + }).join('') + + '
    ' + inline(c) + '
    ' + inline(r[k] || '') + '
    '; + } + + function codeBlock(code, lang) { + return '
    ' + esc(lang || 'code') + '' + + '
    ' + + '
    ' + esc(code) + '
    '; + } + + function inline(text) { + var slots = []; + function keep(markup) { + slots.push(markup); + return '\u0000' + (slots.length - 1) + '\u0000'; + } + text = text + .replace(/(`+)([^`]|[^`][\s\S]*?[^`])\1(?!`)/g, function (_, ticks, code) { + return keep('' + esc(code.replace(/^ ([\s\S]*) $/, '$1')) + ''); + }) + .replace(/\[([^\]\n]+)\]\(\s*()]+(?:\([^\s<>()]*\)[^\s<>()]*)*)>?(?:\s+["'][^"'\n]*["'])?\s*\)/g, function (all, label, url) { + var href = safeHref(url); + return href ? keep(anchor(href, emphasis(esc(label)))) : label; + }) + .replace(/<(https?:\/\/[^\s<>\u0000]+)>|\bhttps?:\/\/[^\s<>()[\]"'`\u0000]*[^\s<>()[\]"'`\u0000.,;:!?*_]/g, function (all, bracketed) { + var url = bracketed || all, href = safeHref(url); + return href ? keep(anchor(href, esc(url))) : all; + }); + text = emphasis(esc(text)) + .replace(/(?: {2,}|\\)\n/g, '
    ') + .replace(/\n/g, ' '); + /* Slots can hold slots (a code span in a link's text). */ + while (/\u0000\d+\u0000/.test(text)) { + text = text.replace(/\u0000(\d+)\u0000/g, function (_, n) { return slots[+n]; }); + } + return text; + } + + function emphasis(s) { + return s + .replace(/\*\*(?=\S)([\s\S]*?\S)\*\*/g, '$1') + .replace(/(^|[^\w])__(?=\S)([\s\S]*?\S)__(?!\w)/g, '$1$2') + .replace(/~~(?=\S)([\s\S]*?\S)~~/g, '$1') + .replace(/(^|[^*\w])\*(?=[^\s*])([^*\n]*?[^\s*])\*(?![*\w])/g, '$1$2') + .replace(/(^|[^\w])_(?=[^\s_])([^_\n]*?[^\s_])_(?!\w)/g, '$1$2'); + } + + function anchor(href, label, cls, title) { + var here = href.indexOf(BASE) === 0 || href.indexOf(DOCS) === 0; + return '' + label + ''; + } + + /* Only web and mail links survive, whatever the bot writes. */ + function safeHref(url) { + var u; + try { u = new URL(String(url).trim(), location.href); } catch (e) { return null; } + if (!/^(https?|mailto):$/.test(u.protocol)) return null; + if (location.hostname !== 'redis.io' && u.href.indexOf(DOCS) === 0) return BASE + u.href.slice(DOCS.length); + if (u.origin === location.origin) return u.pathname + u.search + u.hash; + return u.href; + } + + var NAMED = { amp: '&', lt: '<', gt: '>', quot: '"', apos: "'", nbsp: ' ' }; + + /* The answer arrives HTML-escaped. Undo that as text; never by parsing it. */ + function decode(s) { + return s.replace(/&(#[xX][0-9a-fA-F]+|#[0-9]+|[a-zA-Z]+);/g, function (all, e) { + if (e[0] !== '#') return Object.prototype.hasOwnProperty.call(NAMED, e.toLowerCase()) ? NAMED[e.toLowerCase()] : all; + var n = /^#x/i.test(e) ? parseInt(e.slice(2), 16) : parseInt(e.slice(1), 10); + return n > 0 && n < 0x110000 ? String.fromCodePoint(n) : all; + }); + } + + /* ------------------------------------------------------------ helpers -- */ + + function esc(s) { + return String(s).replace(/&/g, '&').replace(//g, '>') + .replace(/"/g, '"').replace(/'/g, '''); + } + + function el(tag, cls, text) { + var n = document.createElement(tag); + if (cls) n.className = cls; + if (text != null) n.textContent = text; + return n; + } + + function svg(paths) { + return ''; + } + + function reducedMotion() { + return window.matchMedia && window.matchMedia('(prefers-reduced-motion: reduce)').matches; + } + + function basePath() { + try { return new URL(window.HUGO_BASEURL || '/', location.href).pathname.replace(/\/?$/, '/'); } catch (e) { return '/'; } + } + + /* -------------------------------------------------------------- start -- */ + + /* The page link. A click that asks for a new tab or window keeps the href. */ + document.addEventListener('click', function (e) { + var link = e.target.closest && e.target.closest('a[data-ai-chat-open]'); + if (!link || e.defaultPrevented || e.button !== 0 || e.metaKey || e.ctrlKey || e.shiftKey || e.altKey) return; + var q = promptOf(link); + if (!q) return; + e.preventDefault(); + opener = link; + open({ focus: true }); + /* Asked about this page already and answered: show that answer rather than + ask again. An answer that was stopped, failed or cut off is asked again. */ + var last = state.messages[state.messages.length - 1]; + var answered = last && last.role === 'assistant' && last.q === q && last.status === 'done'; + if (busy || answered) pinLast(false); + else explain(link); + }); + + window.addEventListener('resize', (function () { + var frame = 0; + return function () { + if (frame) return; + frame = requestAnimationFrame(function () { frame = 0; layout(state); }); + }; + })()); + + /* Keep whatever has arrived of an answer when the reader moves on. Leaving + ends the request, so the other tabs see the answer as stopped. A question + still waiting to go out never reached the bot, so it goes, with its + answer. With nothing in flight there's nothing new to keep, and a tab + that's been in the background mustn't write its copy over the + conversation. */ + window.addEventListener('pagehide', function () { + if (!busy) return; + var i = state.messages.indexOf(busy.bot); + if (!busy.bot.unsent) busy.bot.status = 'interrupted'; + else if (i > 0 && state.messages[i - 1].role === 'user') state.messages.splice(i - 1, 2); + else if (i !== -1) state.messages.splice(i, 1); + save(); + }); + + /* Another tab changed the conversation. */ + if (SHARED) { + window.addEventListener('storage', function (e) { + if (e.storageArea === localStorage && (e.key === TRANSCRIPT || e.key === null)) adopt(loadMessages()); + }); + } + + /* Back/forward cache: the page comes back as it was left, but the pages + since may have moved the conversation on, or started one. Take the stored + state whether or not this page had built the pane. Its own copy is stale, + and the pagehide save would write it over the conversation. */ + window.addEventListener('pageshow', function (e) { + if (!e.persisted) return; + stop(); + busy = null; + state = load(); + expireLater(); + if (!ui) { + if (!state.open && !state.messages.length) return; + build(); + } else { + renderAll(); + sync(); + } + if (state.open) { open({ instant: true }); pinLast(true); } + else { layout(state); ui.pane.classList.remove('is-in'); ui.pane.hidden = true; updateTab(); } + }); + + expireLater(); + if (state.open || state.messages.length) build(); + if (state.open) { + open({ instant: true }); + pinLast(true); + } + html.classList.remove('ai-pane-restoring'); + + window.AIChatPane.open = open; + window.AIChatPane.collapse = collapse; + window.AIChatPane.ask = ask; + window.AIChatPane.reset = reset; +})(); From 717a4dd2fc33ce2c713539aac91fddde41075176 Mon Sep 17 00:00:00 2001 From: andy-stark-redis <164213578+andy-stark-redis@users.noreply.github.com> Date: Thu, 22 Oct 2026 10:20:16 +0100 Subject: [PATCH 59/76] DOC-7198 Fix Go vecsets for hugot v0.7.7 and close harness pom gaps (#4288) Fixes the Go vecsets example for current hugot and closes gaps in the example-test harness poms that were found during DOC-7196 and DOC-7197. Go vecsets: home_vecsets.go and the page used hugot's pre-v0.7 API (NewGoSession() without a context, RunPipeline(inputs)), so they didn't build on any current release, and the page's `go get` was unpinned. It now targets hugot v0.7.7, the newest release that builds on Go 1.26, matching vecsearch from DOC-7197. That means a ctx argument on NewGoSession, DownloadModel and all six RunPipeline calls. The page pins `go get ...hugot@v0.7.7` (Go 1.26.5 or later), and its RunPipeline() description now matches what the call returns. Verified: the original file fails to build on v0.7.7. The new one builds with Go 1.26.9 and 1.27.2 and runs three times on Redis 8.10.2 with identical output, matching every comment and both output blocks. Reruns are stable, because VADD and VSETATTR overwrite the same eight elements. Harness poms: the portable and fidelity Lettuce async/reactive poms and both Jedis poms gain DJL 0.33.0 (api, tokenizers, pytorch-engine), which the vector examples import. The fidelity Lettuce async/reactive pins go from lettuce-core 7.4.0 to 7.7.0.RELEASE, matching portable. 7.4.0 has no HashImport, so fidelity mode couldn't compile any Lettuce HIMPORT example after DOC-7196. A run.sh sweep in both modes against Redis 8.10.2 passed geoindex, search_quickstart, hash_import and bf_tutorial for go-redis, Jedis and Lettuce async/reactive, except Jedis hash_import, which fails on the missing HashImport class. The Lettuce vecsearch and HomeJsonExample files also compile and pass with the new portable poms, staged the way run.sh stages Java. Constraint: Go vecsets pins hugot v0.7.7, like vecsearch; v0.7.8 and later need Go 1.27, and v0.7.7 uses hugot.NewPipeline(session, config) and creates ./models itself Constraint: fidelity Lettuce async/reactive poms must stay at lettuce-core >= 7.7.0 (HashImport) and below 7.9.0, whose search API generics break the docs examples Learned: run.sh has no mapping for the client-specific index/query sets (home_query_vec, *_home_json, HomeQueryVec), so the harness never runs those examples; it resolves local_examples/// plus a few hand-mapped tutorial files Gaps: harness Jedis pins (portable 7.5.3, fidelity 7.4.0) predate Jedis 8.0.0, so Jedis HIMPORT examples (hash_import) fail under run.sh in both modes Ticket: DOC-7198 Co-Authored-By: Claude Opus 5.5 --- .../example-test-harness/fidelity/pom-jedis.xml | 15 +++++++++++++++ .../fidelity/pom-lettuce-async.xml | 17 ++++++++++++++++- .../fidelity/pom-lettuce-reactive.xml | 17 ++++++++++++++++- build/example-test-harness/pom-jedis.xml | 3 +++ .../example-test-harness/pom-lettuce-async.xml | 3 +++ .../pom-lettuce-reactive.xml | 3 +++ content/develop/clients/go/vecsets.md | 8 +++++--- .../client-specific/go/home_vecsets.go | 15 ++++++++------- 8 files changed, 69 insertions(+), 12 deletions(-) diff --git a/build/example-test-harness/fidelity/pom-jedis.xml b/build/example-test-harness/fidelity/pom-jedis.xml index b6c867193c..bb58bd3f0e 100644 --- a/build/example-test-harness/fidelity/pom-jedis.xml +++ b/build/example-test-harness/fidelity/pom-jedis.xml @@ -27,6 +27,21 @@ 5.10.0 test + + ai.djl + api + 0.33.0 + + + ai.djl.huggingface + tokenizers + 0.33.0 + + + ai.djl.pytorch + pytorch-engine + 0.33.0 + diff --git a/build/example-test-harness/fidelity/pom-lettuce-async.xml b/build/example-test-harness/fidelity/pom-lettuce-async.xml index 1567f2d5da..7a138dcbe6 100644 --- a/build/example-test-harness/fidelity/pom-lettuce-async.xml +++ b/build/example-test-harness/fidelity/pom-lettuce-async.xml @@ -19,7 +19,22 @@ io.lettuce lettuce-core - 7.4.0.RELEASE + 7.7.0.RELEASE + + + ai.djl + api + 0.33.0 + + + ai.djl.huggingface + tokenizers + 0.33.0 + + + ai.djl.pytorch + pytorch-engine + 0.33.0 org.junit.jupiter diff --git a/build/example-test-harness/fidelity/pom-lettuce-reactive.xml b/build/example-test-harness/fidelity/pom-lettuce-reactive.xml index 4867ac6afa..3ee0392865 100644 --- a/build/example-test-harness/fidelity/pom-lettuce-reactive.xml +++ b/build/example-test-harness/fidelity/pom-lettuce-reactive.xml @@ -19,13 +19,28 @@ io.lettuce lettuce-core - 7.4.0.RELEASE + 7.7.0.RELEASE io.projectreactor reactor-core 3.5.0 + + ai.djl + api + 0.33.0 + + + ai.djl.huggingface + tokenizers + 0.33.0 + + + ai.djl.pytorch + pytorch-engine + 0.33.0 + org.junit.jupiter junit-jupiter diff --git a/build/example-test-harness/pom-jedis.xml b/build/example-test-harness/pom-jedis.xml index d442649664..fe0fd65968 100644 --- a/build/example-test-harness/pom-jedis.xml +++ b/build/example-test-harness/pom-jedis.xml @@ -5,6 +5,9 @@ 17UTF-8 redis.clientsjedis7.5.3 + ai.djlapi0.33.0 + ai.djl.huggingfacetokenizers0.33.0 + ai.djl.pytorchpytorch-engine0.33.0 org.junit.jupiterjunit-jupiter5.10.2test org.assertjassertj-core3.25.3test diff --git a/build/example-test-harness/pom-lettuce-async.xml b/build/example-test-harness/pom-lettuce-async.xml index b887da8f00..350d36b5f0 100644 --- a/build/example-test-harness/pom-lettuce-async.xml +++ b/build/example-test-harness/pom-lettuce-async.xml @@ -6,6 +6,9 @@ io.lettucelettuce-core7.7.0.RELEASE com.fasterxml.jackson.corejackson-databind2.17.1 + ai.djlapi0.33.0 + ai.djl.huggingfacetokenizers0.33.0 + ai.djl.pytorchpytorch-engine0.33.0 org.junit.jupiterjunit-jupiter5.10.2test org.assertjassertj-core3.25.3test diff --git a/build/example-test-harness/pom-lettuce-reactive.xml b/build/example-test-harness/pom-lettuce-reactive.xml index b1fb7d1c25..4ccc212093 100644 --- a/build/example-test-harness/pom-lettuce-reactive.xml +++ b/build/example-test-harness/pom-lettuce-reactive.xml @@ -6,6 +6,9 @@ io.lettucelettuce-core7.7.0.RELEASE com.fasterxml.jackson.corejackson-databind2.17.1 + ai.djlapi0.33.0 + ai.djl.huggingfacetokenizers0.33.0 + ai.djl.pytorchpytorch-engine0.33.0 org.junit.jupiterjunit-jupiter5.10.2test org.assertjassertj-core3.25.3test diff --git a/content/develop/clients/go/vecsets.md b/content/develop/clients/go/vecsets.md index 5a7f57186d..4bb4ffc130 100644 --- a/content/develop/clients/go/vecsets.md +++ b/content/develop/clients/go/vecsets.md @@ -40,10 +40,10 @@ Start by [installing](/content/develop/clients/go/_index.md#install) `go-redis` [v9.10.0](https://github.com/redis/go-redis/releases/tag/v9.10.0) or later to use vector sets. -Also, install `hugot`: +Also, install v0.7.7 of `hugot`, which requires Go 1.26.5 or later: ```bash -go get github.com/knights-analytics/hugot +go get github.com/knights-analytics/hugot@v0.7.7 ``` In a new Go file, add the required imports: @@ -81,7 +81,9 @@ elements to a vector set called `famousPeople`. Use the `RunPipeline()` method of `embeddingPipeline` to generate the -embedding as an array of `float32` values, then use a loop like the one +embedding. This method takes a context and an array of strings and returns a +single `FeatureExtractionOutput` object. Its `Embeddings` field contains one array of `float32` +values for each input string. Use a loop like the one shown below to convert the `float32` array to a `float64` array. You can then pass this array to the [`VAdd()`](/content/commands/vadd.md) command to set the embedding. diff --git a/local_examples/client-specific/go/home_vecsets.go b/local_examples/client-specific/go/home_vecsets.go index 1e13e105dd..09ff0ba7c8 100644 --- a/local_examples/client-specific/go/home_vecsets.go +++ b/local_examples/client-specific/go/home_vecsets.go @@ -80,7 +80,7 @@ func main() { // STEP_START model // Create a new Hugot session - session, err := hugot.NewGoSession() + session, err := hugot.NewGoSession(ctx) if err != nil { panic(err) } @@ -95,6 +95,7 @@ func main() { downloadOptions := hugot.NewDownloadOptions() downloadOptions.OnnxFilePath = "onnx/model.onnx" modelPath, err := hugot.DownloadModel( + ctx, "sentence-transformers/all-MiniLM-L6-v2", "./models/", downloadOptions, @@ -127,7 +128,7 @@ func main() { // STEP_START add_data for name, details := range peopleData { // Generate embeddings using Hugot - result, err := embeddingPipeline.RunPipeline([]string{details.Description}) + result, err := embeddingPipeline.RunPipeline(ctx, []string{details.Description}) if err != nil { panic(err) } @@ -159,7 +160,7 @@ func main() { // STEP_START basic_query queryValue := "actors" - queryResult, err := embeddingPipeline.RunPipeline([]string{queryValue}) + queryResult, err := embeddingPipeline.RunPipeline(ctx, []string{queryValue}) if err != nil { panic(err) } @@ -185,7 +186,7 @@ func main() { // STEP_START limited_query queryValue = "actors" - queryResult, err = embeddingPipeline.RunPipeline([]string{queryValue}) + queryResult, err = embeddingPipeline.RunPipeline(ctx, []string{queryValue}) if err != nil { panic(err) } @@ -211,7 +212,7 @@ func main() { // STEP_START entertainer_query queryValue = "entertainer" - queryResult, err = embeddingPipeline.RunPipeline([]string{queryValue}) + queryResult, err = embeddingPipeline.RunPipeline(ctx, []string{queryValue}) if err != nil { panic(err) } @@ -236,7 +237,7 @@ func main() { queryValue = "science" - queryResult, err = embeddingPipeline.RunPipeline([]string{queryValue}) + queryResult, err = embeddingPipeline.RunPipeline(ctx, []string{queryValue}) if err != nil { panic(err) } @@ -259,7 +260,7 @@ func main() { // STEP_START filtered_query queryValue = "science" - queryResult, err = embeddingPipeline.RunPipeline([]string{queryValue}) + queryResult, err = embeddingPipeline.RunPipeline(ctx, []string{queryValue}) if err != nil { panic(err) } From 95ce8badaa880e0cc78ada4d81c1cbbe9a8e9747 Mon Sep 17 00:00:00 2001 From: andy-stark-redis <164213578+andy-stark-redis@users.noreply.github.com> Date: Thu, 22 Oct 2026 10:20:28 +0100 Subject: [PATCH 60/76] DOC-7198 Sort query3 by city in all clients, fix Go vecsets steps, and make examples rerunnable (#4290) Makes the queryjson query3 aggregation deterministic in all eight clients, fixes the Go vecsets page's steps and wording, and makes two examples rerunnable. query3 grouped by @city with a COUNT reducer and no SORTBY, so the server's group order wasn't guaranteed. Real runs printed Tel Aviv first while the comments showed London first. Every client now sends SORTBY 2 @city ASC with its own API: - redis-py: sort_by(Asc('@city')). - node-redis: a STEPS SORTBY entry. - go-redis: an FTAggregateSortByStep. - redis-rb: sort_by(Search::Asc...). - Jedis: sortByAsc. - Lettuce: sortBy(..., SortDirection.ASC). - NRedisStack: SortBy(SortedField.Asc(...)). - Predis: sortBy(0, '@city', 'ASC'). Visible client-side sorting in Go and Ruby is gone, the hidden asserts now check the server's order instead of sorting first, and each page says the results are sorted by city. go-redis query3 moved to Steps (a GroupBy step, then a SortBy step), because v9.23.0 deprecates the top-level GroupBy/SortBy fields and they can't be mixed with Steps. Its visible "City: London - 1" comment now matches the printed "London - 1". Go vecsets: - The science query had no step markers, so the page hard-coded its output. It's now a science_query step rendered with clients-example. - The page and the add_data description credited VAdd() with setting the born/died attributes; a separate VSetAttr() call does that. - ctx is now defined inside the first rendered step that uses it. Rerunnability: - Jedis LandingExample gains its missing org.junit.jupiter.api.Test import and a hidden reset of user-session:123, so a second run prints 4, not 0. - PHP vecsearch drops vector_json_idx before creating it, as it already did for vector_idx, so a second run no longer fails with SEARCH_INDEX_EXISTS. Verified on Redis 8.10.2. Every changed file ran twice in a row, printing London then Tel Aviv each time, with redis-py 8.1.0, node-redis 6.3.0, go-redis 9.23.0 (Go 1.26.9), redis-rb 6.0.0, Jedis 8.0.2, Lettuce 7.7.0, SE.Redis 3.3.1 / NRedisStack 1.6.0 and Predis 3.7.0. MONITOR or SLOWLOG confirmed every client sends SORTBY. Both Go vector programs, and the PHP vecsearch page assembled from its inline blocks, also ran twice. The site build is clean. Constraint: query3 must keep SORTBY @city ASC in every client; without it, the documented London-then-Tel-Aviv order is incidental, and hidden asserts no longer sort client-side Constraint: go-redis query3 uses FTAggregateOptions.Steps (GroupBy then SortBy); the top-level GroupBy/SortBy fields are deprecated in v9.23.0 and can't be combined with Steps Learned: an unsorted FT.AGGREGATE can return the "expected" order by chance, so prove a sort with MONITOR/SLOWLOG, not output Gaps: PHP HomeJsonTest.php never drops idx:users or hash-idx:users, so a rerun reuses whatever index exists; the PHP vecsearch JSON section shows an output block its code never prints Recheck: the redis-py queryjson Thebe notebook (BINDER_ID) may need regenerating to pick up the sort Ticket: DOC-7198 Co-Authored-By: Claude Opus 5.5 --- .../clients/dotnet/nredisstack/queryjson.md | 2 +- content/develop/clients/go/queryjson.md | 2 +- content/develop/clients/go/vecsets.md | 15 +++++----- content/develop/clients/jedis/queryjson.md | 2 +- content/develop/clients/lettuce/queryjson.md | 2 +- content/develop/clients/nodejs/queryjson.md | 2 +- content/develop/clients/php/queryjson.md | 2 +- content/develop/clients/php/vecsearch.md | 4 +++ content/develop/clients/redis-py/queryjson.md | 2 +- content/develop/clients/ruby/queryjson.md | 2 +- .../dotnet-sync/HomeJsonExample.cs | 8 ++--- .../go/home_json_example_test.go | 30 +++++++++++-------- .../client-specific/go/home_vecsets.go | 5 +++- .../jedis/HomeJsonExample.java | 3 +- .../client-specific/jedis/LandingExample.java | 6 ++++ .../lettuce-async/HomeJsonExample.java | 5 ++-- .../lettuce-reactive/HomeJsonExample.java | 5 ++-- .../client-specific/nodejs/home-query.js | 4 ++- .../client-specific/php/HomeJsonTest.php | 3 +- .../client-specific/redis-py/home_json.py | 4 +-- .../client-specific/ruby/home_json.rb | 8 ++--- 21 files changed, 68 insertions(+), 48 deletions(-) diff --git a/content/develop/clients/dotnet/nredisstack/queryjson.md b/content/develop/clients/dotnet/nredisstack/queryjson.md index 688bc981f2..46b420b2b1 100644 --- a/content/develop/clients/dotnet/nredisstack/queryjson.md +++ b/content/develop/clients/dotnet/nredisstack/queryjson.md @@ -108,7 +108,7 @@ Specify query options to return only the `city` field: Use an [aggregation query](/content/develop/ai/search-and-query/query/aggregation.md) -to count all users in each city. +to count all users in each city, with the results sorted by city. {{< clients-example set="cs_home_json" step="query3" description="Aggregation queries: Use GROUP BY and COUNT operations to summarize and analyze indexed data" difficulty="advanced" >}} {{< /clients-example >}} diff --git a/content/develop/clients/go/queryjson.md b/content/develop/clients/go/queryjson.md index e3459fd166..6c17142870 100644 --- a/content/develop/clients/go/queryjson.md +++ b/content/develop/clients/go/queryjson.md @@ -148,7 +148,7 @@ returning the documents themselves. Use an [aggregation query](/content/develop/ai/search-and-query/query/aggregation.md) -to count all users in each city. +to count all users in each city, with the results sorted by city. {{< clients-example set="go_home_json" step="query3" description="Aggregation: Perform aggregation queries to group and count documents by field values using FTAggregateWithArgs" difficulty="intermediate" >}} {{< /clients-example >}} diff --git a/content/develop/clients/go/vecsets.md b/content/develop/clients/go/vecsets.md index 4bb4ffc130..aa26e29615 100644 --- a/content/develop/clients/go/vecsets.md +++ b/content/develop/clients/go/vecsets.md @@ -88,11 +88,12 @@ shown below to convert the `float32` array to a `float64` array. You can then pass this array to the [`VAdd()`](/content/commands/vadd.md) command to set the embedding. -The call to `VAdd()` also adds the `born` and `died` values from the -original map as attribute data. You can access this during a query -or by using the [`VGetAttr()`](/content/commands/vgetattr.md) method. +A separate call to [`VSetAttr()`](/content/commands/vsetattr.md) then adds the +`born` and `died` values from the original map as attribute data. You can +access this during a query or by using the +[`VGetAttr()`](/content/commands/vgetattr.md) method. -{{< clients-example set="home_vecsets" step="add_data" lang_filter="Go" description="Foundational: Generate embeddings and add data to a vector set with VAdd including attribute metadata" difficulty="intermediate" >}} +{{< clients-example set="home_vecsets" step="add_data" lang_filter="Go" description="Foundational: Generate embeddings and add data to a vector set with VAdd, then set attribute metadata with VSetAttr" difficulty="intermediate" >}} {{< /clients-example >}} ## Query the vector set @@ -138,10 +139,8 @@ highest in the results list: Similarly, if you use "science" as a query, you get the following results: -``` -'science': Marie Curie, Linus Pauling, Maryam Mirzakhani, Paul Erdos, -Marie Fredriksson, Freddie Mercury, Masako Natsume, Chaim Topol -``` +{{< clients-example set="home_vecsets" step="science_query" lang_filter="Go" description="Semantic search: Rank elements by similarity to a science query so related fields rank together" difficulty="intermediate" >}} +{{< /clients-example >}} The scientists are ranked highest but they are then followed by the mathematicians. This seems reasonable given the connection between mathematics diff --git a/content/develop/clients/jedis/queryjson.md b/content/develop/clients/jedis/queryjson.md index 00f06e2762..f82cb25233 100644 --- a/content/develop/clients/jedis/queryjson.md +++ b/content/develop/clients/jedis/queryjson.md @@ -108,7 +108,7 @@ Specify query options to return only the `city` field: Use an [aggregation query](/content/develop/ai/search-and-query/query/aggregation.md) -to count all users in each city. +to count all users in each city, with the results sorted by city. {{< clients-example set="java_home_json" step="query3" description="Aggregation query: Group and count results by field values to analyze data patterns" difficulty="intermediate" >}} {{< /clients-example >}} diff --git a/content/develop/clients/lettuce/queryjson.md b/content/develop/clients/lettuce/queryjson.md index da43d93fe8..d87aa19b07 100644 --- a/content/develop/clients/lettuce/queryjson.md +++ b/content/develop/clients/lettuce/queryjson.md @@ -92,7 +92,7 @@ Specify query options to return only the `city` field: Use an [aggregation query](/content/develop/ai/search-and-query/query/aggregation.md) -to count all users in each city. +to count all users in each city, with the results sorted by city. {{< clients-example set="lettuce_home_json" step="query3" description="Aggregation: Use aggregation queries to group and count results, performing server-side data analysis" difficulty="advanced" >}} {{< /clients-example >}} diff --git a/content/develop/clients/nodejs/queryjson.md b/content/develop/clients/nodejs/queryjson.md index f811666985..d591e70a75 100644 --- a/content/develop/clients/nodejs/queryjson.md +++ b/content/develop/clients/nodejs/queryjson.md @@ -112,7 +112,7 @@ Specify query options to return only the `city` field: Use an [aggregation query](/content/develop/ai/search-and-query/query/aggregation.md) -to count all users in each city. +to count all users in each city, with the results sorted by city. {{< clients-example set="js_home_query" step="query3" lang_filter="Node.js" description="Aggregate query results: Use aggregation queries to group and count results by field values for analytics" difficulty="advanced" >}} {{< /clients-example >}} diff --git a/content/develop/clients/php/queryjson.md b/content/develop/clients/php/queryjson.md index 62cd1f6d40..f49a335596 100644 --- a/content/develop/clients/php/queryjson.md +++ b/content/develop/clients/php/queryjson.md @@ -104,7 +104,7 @@ Specify query options to return only the `city` field: Use an [aggregation query](/content/develop/ai/search-and-query/query/aggregation.md) -to count all users in each city. +to count all users in each city, with the results sorted by city. {{< clients-example set="php_home_json" step="query3" description="Aggregation: Use aggregation queries to group and count results from indexed documents" difficulty="advanced" >}} {{< /clients-example >}} diff --git a/content/develop/clients/php/vecsearch.md b/content/develop/clients/php/vecsearch.md index 3689edc357..24857115de 100644 --- a/content/develop/clients/php/vecsearch.md +++ b/content/develop/clients/php/vecsearch.md @@ -355,6 +355,10 @@ The code below shows these differences, but the index is otherwise very similar the one created previously for hashes: ```php +try { + $client->ftdropindex("vector_json_idx"); +} catch (Exception $e){} + $jsonSchema = [ new TextField("$.content", "content"), new TagField("$.genre", "genre"), diff --git a/content/develop/clients/redis-py/queryjson.md b/content/develop/clients/redis-py/queryjson.md index 712b7bb813..c4b2bb7e7d 100644 --- a/content/develop/clients/redis-py/queryjson.md +++ b/content/develop/clients/redis-py/queryjson.md @@ -108,7 +108,7 @@ Specify query options to return only the `city` field: Use an [aggregation query](/content/develop/ai/search-and-query/query/aggregation.md) -to count all users in each city. +to count all users in each city, with the results sorted by city. {{< jupyter-example set="py_home_json" lang_filter="Python" step="query3" depends="import" description="Aggregation queries: Use GROUP BY and COUNT operations to summarize and analyze indexed data" difficulty="advanced" />}} diff --git a/content/develop/clients/ruby/queryjson.md b/content/develop/clients/ruby/queryjson.md index 92567ad677..730563a85b 100644 --- a/content/develop/clients/ruby/queryjson.md +++ b/content/develop/clients/ruby/queryjson.md @@ -121,7 +121,7 @@ the `city` field: Use an [aggregation query](/content/develop/ai/search-and-query/query/aggregation.md) -to count all users in each city. +to count all users in each city, with the results sorted by city. {{< clients-example set="ruby_home_json" step="query3" description="Aggregation query: Group and count results by field values to analyze data patterns" difficulty="intermediate" >}} {{< /clients-example >}} diff --git a/local_examples/client-specific/dotnet-sync/HomeJsonExample.cs b/local_examples/client-specific/dotnet-sync/HomeJsonExample.cs index 7c80f81dd0..9f31df4cf0 100644 --- a/local_examples/client-specific/dotnet-sync/HomeJsonExample.cs +++ b/local_examples/client-specific/dotnet-sync/HomeJsonExample.cs @@ -141,7 +141,8 @@ public void Run() // STEP_START query3 AggregationRequest aggRequest = new AggregationRequest("*") - .GroupBy("@city", Reducers.Count().As("count")); + .GroupBy("@city", Reducers.Count().As("count")) + .SortBy(SortedField.Asc("@city")); AggregationResult aggResult = db.FT().Aggregate("idx:users", aggRequest); IReadOnlyList> resultsList = @@ -158,12 +159,11 @@ public void Run() // REMOVE_START Assert.Equal(2, resultsList.Count); - var sortedResults = resultsList.OrderBy(x => x["city"]); - Dictionary testItem = sortedResults.ElementAt(0); + Dictionary testItem = resultsList.ElementAt(0); Assert.Equal("London", testItem["city"]); Assert.Equal(1, testItem["count"]); - testItem = sortedResults.ElementAt(1); + testItem = resultsList.ElementAt(1); Assert.Equal("Tel Aviv", testItem["city"]); Assert.Equal(2, testItem["count"]); // REMOVE_END diff --git a/local_examples/client-specific/go/home_json_example_test.go b/local_examples/client-specific/go/home_json_example_test.go index bbdccf3541..75c4abacf9 100644 --- a/local_examples/client-specific/go/home_json_example_test.go +++ b/local_examples/client-specific/go/home_json_example_test.go @@ -181,13 +181,22 @@ func ExampleClient_search_json() { // STEP_START query3 aggOptions := redis.FTAggregateOptions{ - GroupBy: []redis.FTAggregateGroupBy{ + Steps: []redis.FTAggregateStep{ { - Fields: []interface{}{"@city"}, - Reduce: []redis.FTAggregateReducer{ - { - Reducer: redis.SearchCount, - As: "count", + GroupBy: &redis.FTAggregateGroupBy{ + Fields: []interface{}{"@city"}, + Reduce: []redis.FTAggregateReducer{ + { + Reducer: redis.SearchCount, + As: "count", + }, + }, + }, + }, + { + SortBy: &redis.FTAggregateSortByStep{ + Fields: []redis.FTAggregateSortBy{ + {FieldName: "@city", Asc: true}, }, }, }, @@ -205,18 +214,13 @@ func ExampleClient_search_json() { panic(err) } - sort.Slice(aggResult.Rows, func(i, j int) bool { - return aggResult.Rows[i].Fields["city"].(string) < - aggResult.Rows[j].Fields["city"].(string) - }) - for _, row := range aggResult.Rows { fmt.Printf("%v - %v\n", row.Fields["city"], row.Fields["count"], ) } - // >>> City: London - 1 - // >>> City: Tel Aviv - 2 + // >>> London - 1 + // >>> Tel Aviv - 2 // STEP_END // Output: diff --git a/local_examples/client-specific/go/home_vecsets.go b/local_examples/client-specific/go/home_vecsets.go index 09ff0ba7c8..197c0469b0 100644 --- a/local_examples/client-specific/go/home_vecsets.go +++ b/local_examples/client-specific/go/home_vecsets.go @@ -76,9 +76,9 @@ var peopleData = map[string]PersonData{ // STEP_END func main() { + // STEP_START model ctx := context.Background() - // STEP_START model // Create a new Hugot session session, err := hugot.NewGoSession(ctx) if err != nil { @@ -235,6 +235,7 @@ func main() { // Maryam Mirzakhani, Marie Curie // STEP_END + // STEP_START science_query queryValue = "science" queryResult, err = embeddingPipeline.RunPipeline(ctx, []string{queryValue}) @@ -257,6 +258,8 @@ func main() { fmt.Printf("'science': %v\n", strings.Join(scienceResults, ", ")) // >>> 'science': Marie Curie, Linus Pauling, Maryam Mirzakhani, Paul Erdos, // Marie Fredriksson, Freddie Mercury, Masako Natsume, Chaim Topol + // STEP_END + // STEP_START filtered_query queryValue = "science" diff --git a/local_examples/client-specific/jedis/HomeJsonExample.java b/local_examples/client-specific/jedis/HomeJsonExample.java index 4116b56dd8..9bc2d0a90d 100644 --- a/local_examples/client-specific/jedis/HomeJsonExample.java +++ b/local_examples/client-specific/jedis/HomeJsonExample.java @@ -135,6 +135,7 @@ public void run() { AggregationResult aggResult = jedis.ftAggregate("idx:users", new AggregationBuilder("*") .groupBy("@city", Reducers.count().as("count")) + .sortByAsc("@city") ); System.out.println(aggResult.getTotalResults()); // >>> 2 @@ -151,7 +152,7 @@ public void run() { new String[] {"London - 1", "Tel Aviv - 2"}, aggResult.getRows().stream() .map(r -> r.getString("city") + " - " + r.getString("count")) - .sorted().toArray()); + .toArray()); // REMOVE_END // STEP_START cleanup_hash diff --git a/local_examples/client-specific/jedis/LandingExample.java b/local_examples/client-specific/jedis/LandingExample.java index 4f328a0610..49d61259d3 100644 --- a/local_examples/client-specific/jedis/LandingExample.java +++ b/local_examples/client-specific/jedis/LandingExample.java @@ -1,5 +1,8 @@ // EXAMPLE: landing // BINDER_ID jedis-landing +// REMOVE_START +import org.junit.jupiter.api.Test; +// REMOVE_END // STEP_START import import redis.clients.jedis.RedisClient; import java.util.HashMap; @@ -29,6 +32,9 @@ public void run() { hash.put("company", "Redis"); hash.put("age", "29"); + // REMOVE_START + jedis.del("user-session:123"); + // REMOVE_END Long res3 = jedis.hset("user-session:123", hash); System.out.println(res3); // >>> 4 diff --git a/local_examples/client-specific/lettuce-async/HomeJsonExample.java b/local_examples/client-specific/lettuce-async/HomeJsonExample.java index fe719e21b1..f429fac296 100644 --- a/local_examples/client-specific/lettuce-async/HomeJsonExample.java +++ b/local_examples/client-specific/lettuce-async/HomeJsonExample.java @@ -153,7 +153,8 @@ NumericFieldArgs. builder().name("$.age").as("age").build(), // STEP_START query3 AggregateArgs aggArgs = AggregateArgs. builder() - .groupBy(GroupBy. of("@city").reduce(Reducer. count().as("count"))).build(); + .groupBy(GroupBy. of("@city").reduce(Reducer. count().as("count"))) + .sortBy("@city", SortDirection.ASC).build(); CompletableFuture> query3 = searchCommands.ftAggregate("idx:users", "*", aggArgs) .thenApply(res -> { List> replies = res.getReplies(); @@ -171,7 +172,7 @@ NumericFieldArgs. builder().name("$.age").as("age").build(), assertThat(replies.get(0).getResults().stream().map(result -> { return String.format("City: %s, Count: %s", result.getFields().get("city"), result.getFields().get("count")); - }).sorted().toArray()).containsExactly("City: London, Count: 1", "City: Tel Aviv, Count: 2"); + }).toArray()).containsExactly("City: London, Count: 1", "City: Tel Aviv, Count: 2"); // REMOVE_END return res; }).toCompletableFuture(); diff --git a/local_examples/client-specific/lettuce-reactive/HomeJsonExample.java b/local_examples/client-specific/lettuce-reactive/HomeJsonExample.java index 2d598de364..6292ab4f63 100644 --- a/local_examples/client-specific/lettuce-reactive/HomeJsonExample.java +++ b/local_examples/client-specific/lettuce-reactive/HomeJsonExample.java @@ -139,7 +139,8 @@ NumericFieldArgs. builder().name("$.age").as("age").build(), // STEP_START query3 AggregateArgs aggArgs = AggregateArgs. builder() - .groupBy(GroupBy. of("@city").reduce(Reducer. count().as("count"))).build(); + .groupBy(GroupBy. of("@city").reduce(Reducer. count().as("count"))) + .sortBy("@city", SortDirection.ASC).build(); Mono> query3 = searchCommands.ftAggregate("idx:users", "*", aggArgs) .doOnNext(res -> { @@ -158,7 +159,7 @@ NumericFieldArgs. builder().name("$.age").as("age").build(), assertThat(replies.get(0).getResults().stream().map(result -> { return String.format("City: %s, Count: %s", result.getFields().get("city"), result.getFields().get("count")); - }).sorted().toArray()).containsExactly("City: London, Count: 1", "City: Tel Aviv, Count: 2"); + }).toArray()).containsExactly("City: London, Count: 1", "City: Tel Aviv, Count: 2"); // REMOVE_END }); // STEP_END diff --git a/local_examples/client-specific/nodejs/home-query.js b/local_examples/client-specific/nodejs/home-query.js index c73ab385de..8d4b6f6c61 100644 --- a/local_examples/client-specific/nodejs/home-query.js +++ b/local_examples/client-specific/nodejs/home-query.js @@ -124,6 +124,9 @@ let aggResult = await client.ft.aggregate('idx:users', '*', { type: FT_AGGREGATE_GROUP_BY_REDUCERS.COUNT, AS: 'count' }] + }, { + type: FT_AGGREGATE_STEPS.SORTBY, + BY: { BY: '@city', DIRECTION: 'ASC' } }] }); @@ -137,7 +140,6 @@ aggResult.results.forEach(result => { // STEP_END // REMOVE_START assert.equal(aggResult.total, 2); -aggResult.results.sort((a, b) => a.city.localeCompare(b.city)); assert.deepEqual(aggResult.results.map(result => result.city), [ 'London', 'Tel Aviv' diff --git a/local_examples/client-specific/php/HomeJsonTest.php b/local_examples/client-specific/php/HomeJsonTest.php index 8f8443c51e..494a89f3b6 100644 --- a/local_examples/client-specific/php/HomeJsonTest.php +++ b/local_examples/client-specific/php/HomeJsonTest.php @@ -102,7 +102,8 @@ public function testHomeJson() { // STEP_START query3 $ftAggregateArguments = (new AggregateArguments()) ->groupBy('@city') - ->reduce('COUNT', true, 'count'); + ->reduce('COUNT', true, 'count') + ->sortBy(0, '@city', 'ASC'); $res = $r->ftAggregate('idx:users', '*', $ftAggregateArguments); echo json_encode($res), PHP_EOL; diff --git a/local_examples/client-specific/redis-py/home_json.py b/local_examples/client-specific/redis-py/home_json.py index 2dc3e93275..6891a0506e 100644 --- a/local_examples/client-specific/redis-py/home_json.py +++ b/local_examples/client-specific/redis-py/home_json.py @@ -118,15 +118,13 @@ # STEP_START query3 req = aggregations.AggregateRequest("*").group_by( '@city', reducers.count().alias('count') -) +).sort_by(aggregations.Asc('@city')) aggResult = r.ft("idx:users").aggregate(req).rows print(aggResult) # >>> [['city', 'London', 'count', '1'], ['city', 'Tel Aviv', 'count', '2']] # STEP_END # REMOVE_START -aggResult.sort(key=lambda row: row[1]) - assert str(aggResult) == ( "[['city', 'London', 'count', '1'], ['city', 'Tel Aviv', 'count', '2']]" ) diff --git a/local_examples/client-specific/ruby/home_json.rb b/local_examples/client-specific/ruby/home_json.rb index ff7c5d7e17..dbb4a30f62 100644 --- a/local_examples/client-specific/ruby/home_json.rb +++ b/local_examples/client-specific/ruby/home_json.rb @@ -109,19 +109,19 @@ def assert_equal(expected, actual) # STEP_START query3 request = Search::AggregateRequest.new('*') .group_by('@city', Search::Reducers.count.as('count')) + .sort_by(Search::Asc.new('@city')) agg_result = index.aggregate(request) -agg_result.rows.sort_by { |row| row['city'] }.each do |row| +agg_result.rows.each do |row| puts "#{row['city']} - #{row['count']}" end # >>> London - 1 # >>> Tel Aviv - 2 # STEP_END # REMOVE_START -sorted_rows = agg_result.rows.sort_by { |row| row['city'] } -assert_equal(['London', 'Tel Aviv'], sorted_rows.map { |row| row['city'] }) -assert_equal(%w[1 2], sorted_rows.map { |row| row['count'] }) +assert_equal(['London', 'Tel Aviv'], agg_result.rows.map { |row| row['city'] }) +assert_equal(%w[1 2], agg_result.rows.map { |row| row['count'] }) # REMOVE_END # STEP_START cleanup_hash From 115a3c5ec1fb12e2aebd1da80e93bcd8ace25fd3 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Thu, 22 Oct 2026 06:20:54 -0700 Subject: [PATCH 61/76] Update content/operate/rc/api/api-reference/openapi.json (#4322) Co-authored-by: redisdocsapp[bot] <177626021+redisdocsapp[bot]@users.noreply.github.com> --- .../operate/rc/api/api-reference/openapi.json | 122 +++++++++--------- 1 file changed, 61 insertions(+), 61 deletions(-) diff --git a/content/operate/rc/api/api-reference/openapi.json b/content/operate/rc/api/api-reference/openapi.json index a6088bc486..e1f1a7f7fe 100644 --- a/content/operate/rc/api/api-reference/openapi.json +++ b/content/operate/rc/api/api-reference/openapi.json @@ -96,7 +96,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_4", + "operationId": "proxyDataIntegrationWorkspace_1", "parameters": [ { "name": "subscriptionId", @@ -166,7 +166,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace", + "operationId": "proxyDataIntegrationWorkspace_3", "parameters": [ { "name": "subscriptionId", @@ -237,7 +237,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_1", + "operationId": "proxyDataIntegrationWorkspace_2", "parameters": [ { "name": "subscriptionId", @@ -308,7 +308,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_3", + "operationId": "proxyDataIntegrationWorkspace", "parameters": [ { "name": "subscriptionId", @@ -379,7 +379,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_2", + "operationId": "proxyDataIntegrationWorkspace_4", "parameters": [ { "name": "subscriptionId", @@ -452,7 +452,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_9", + "operationId": "proxyDataIntegrationWorkspace_6", "parameters": [ { "name": "subscriptionId", @@ -522,7 +522,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_5", + "operationId": "proxyDataIntegrationWorkspace_8", "parameters": [ { "name": "subscriptionId", @@ -593,7 +593,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_6", + "operationId": "proxyDataIntegrationWorkspace_7", "parameters": [ { "name": "subscriptionId", @@ -664,7 +664,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_8", + "operationId": "proxyDataIntegrationWorkspace_5", "parameters": [ { "name": "subscriptionId", @@ -735,7 +735,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_7", + "operationId": "proxyDataIntegrationWorkspace_9", "parameters": [ { "name": "subscriptionId", @@ -8353,15 +8353,11 @@ "403": { "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" }, - "424": { - "description": "Failed Dependency - The memory store's database is unavailable or out of memory, e.g. /errors/resource-unavailable or /errors/database-out-of-memory. When the error originates from the Agent Memory service the body is an application/problem+json document (MemoryProblemDetails) carrying the service's error type and detail", - "content": { - "application/problem+json": { - "schema": { - "$ref": "#/components/schemas/MemoryProblemDetails" - } - } - } + "401": { + "description": "Unauthorized - Authentication failed for requested resource" + }, + "404": { + "description": "Not Found - The resource you were trying to reach was not found or does not exist" }, "200": { "description": "Memory stores returned.", @@ -8373,11 +8369,15 @@ } } }, - "401": { - "description": "Unauthorized - Authentication failed for requested resource" - }, - "404": { - "description": "Not Found - The resource you were trying to reach was not found or does not exist" + "424": { + "description": "Failed Dependency - The memory store's database is unavailable or out of memory, e.g. /errors/resource-unavailable or /errors/database-out-of-memory. When the error originates from the Agent Memory service the body is an application/problem+json document (MemoryProblemDetails) carrying the service's error type and detail", + "content": { + "application/problem+json": { + "schema": { + "$ref": "#/components/schemas/MemoryProblemDetails" + } + } + } }, "408": { "description": "Request Timeout - The server didn't receive a complete request message within the server's allotted timeout period" @@ -8493,6 +8493,12 @@ "403": { "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" }, + "401": { + "description": "Unauthorized - Authentication failed for requested resource" + }, + "404": { + "description": "Not Found - The resource you were trying to reach was not found or does not exist" + }, "424": { "description": "Failed Dependency - The memory store's database is unavailable or out of memory, e.g. /errors/resource-unavailable or /errors/database-out-of-memory. When the error originates from the Agent Memory service the body is an application/problem+json document (MemoryProblemDetails) carrying the service's error type and detail", "content": { @@ -8503,12 +8509,6 @@ } } }, - "401": { - "description": "Unauthorized - Authentication failed for requested resource" - }, - "404": { - "description": "Not Found - The resource you were trying to reach was not found or does not exist" - }, "408": { "description": "Request Timeout - The server didn't receive a complete request message within the server's allotted timeout period" }, @@ -8568,15 +8568,11 @@ "403": { "description": "Forbidden - Not allowed to create requested resource (primarily due to not active resource)" }, - "424": { - "description": "Failed Dependency - The memory store's database is unavailable or out of memory, e.g. /errors/resource-unavailable or /errors/database-out-of-memory. When the error originates from the Agent Memory service the body is an application/problem+json document (MemoryProblemDetails) carrying the service's error type and detail", - "content": { - "application/problem+json": { - "schema": { - "$ref": "#/components/schemas/MemoryProblemDetails" - } - } - } + "401": { + "description": "Unauthorized - Authentication failed for requested resource" + }, + "404": { + "description": "Not Found - The resource you were trying to reach was not found or does not exist" }, "201": { "description": "API key created. The generated key value is returned once in the response body.", @@ -8588,11 +8584,15 @@ } } }, - "401": { - "description": "Unauthorized - Authentication failed for requested resource" - }, - "404": { - "description": "Not Found - The resource you were trying to reach was not found or does not exist" + "424": { + "description": "Failed Dependency - The memory store's database is unavailable or out of memory, e.g. /errors/resource-unavailable or /errors/database-out-of-memory. When the error originates from the Agent Memory service the body is an application/problem+json document (MemoryProblemDetails) carrying the service's error type and detail", + "content": { + "application/problem+json": { + "schema": { + "$ref": "#/components/schemas/MemoryProblemDetails" + } + } + } }, "408": { "description": "Request Timeout - The server didn't receive a complete request message within the server's allotted timeout period" @@ -10079,6 +10079,9 @@ "412": { "description": "Precondition Failed - Feature flag for this flow is off" }, + "403": { + "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" + }, "200": { "description": "Memory store returned.", "content": { @@ -10089,8 +10092,11 @@ } } }, - "403": { - "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" + "401": { + "description": "Unauthorized - Authentication failed for requested resource" + }, + "404": { + "description": "Not Found - The resource you were trying to reach was not found or does not exist" }, "424": { "description": "Failed Dependency - The memory store's database is unavailable or out of memory, e.g. /errors/resource-unavailable or /errors/database-out-of-memory. When the error originates from the Agent Memory service the body is an application/problem+json document (MemoryProblemDetails) carrying the service's error type and detail", @@ -10102,12 +10108,6 @@ } } }, - "401": { - "description": "Unauthorized - Authentication failed for requested resource" - }, - "404": { - "description": "Not Found - The resource you were trying to reach was not found or does not exist" - }, "429": { "description": "Too Many Requests - Too many resources are concurrently created / updated / deleted in the account. Please re-submit API requests after resource changes are completed)" }, @@ -12804,9 +12804,18 @@ "412": { "description": "Precondition Failed - Feature flag for this flow is off" }, + "204": { + "description": "API key deleted." + }, "403": { "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" }, + "401": { + "description": "Unauthorized - Authentication failed for requested resource" + }, + "404": { + "description": "Not Found - The resource you were trying to reach was not found or does not exist" + }, "424": { "description": "Failed Dependency - The memory store's database is unavailable or out of memory, e.g. /errors/resource-unavailable or /errors/database-out-of-memory. When the error originates from the Agent Memory service the body is an application/problem+json document (MemoryProblemDetails) carrying the service's error type and detail", "content": { @@ -12817,15 +12826,6 @@ } } }, - "204": { - "description": "API key deleted." - }, - "401": { - "description": "Unauthorized - Authentication failed for requested resource" - }, - "404": { - "description": "Not Found - The resource you were trying to reach was not found or does not exist" - }, "408": { "description": "Request Timeout - The server didn't receive a complete request message within the server's allotted timeout period" }, From 54f487642718490342e76290ffd7bc7f1c23467a Mon Sep 17 00:00:00 2001 From: mich-elle-luna Date: Thu, 22 Oct 2026 14:23:39 -0700 Subject: [PATCH 62/76] Link Redis Iris from the rest of the docs Add one-sentence links to Redis Iris and Redis Agent Memory where readers are solving a problem Iris solves, using each page's existing Resources, Learn more, or Related info section. - Develop: RAG and GenAI get-started pages, vector search concepts, ecosystem integrations (Google ADK and Agent Memory), and the client vector search pages through a new shared embed, iris-for-developers.md. - Operate: Redis Cloud landing and create-database pages, and the Redis Software and Redis Software for Kubernetes landing pages. - Integrate: LangChain and Amazon Bedrock. - Memory layer use case: name the managed service Redis Agent Memory, not Redis Agent Memory Server, which is the open source project. Links to Agent Playbooks and Context Gateway wait for their GA pages. Co-Authored-By: Claude Opus 5.5 (1M context) --- content/develop/ai/ecosystem-integrations.md | 1 + content/develop/ai/search-and-query/vectors/_index.md | 1 + content/develop/clients/dotnet/nredisstack/vecsearch.md | 2 ++ content/develop/clients/go/vecsearch.md | 2 ++ content/develop/clients/jedis/vecsearch.md | 2 ++ content/develop/clients/lettuce/vecsearch.md | 2 ++ content/develop/clients/nodejs/vecsearch.md | 2 ++ content/develop/clients/php/vecsearch.md | 2 ++ content/develop/clients/redis-py/vecsearch.md | 2 ++ content/develop/clients/ruby/vecsearch.md | 2 ++ content/develop/get-started/rag.md | 1 + content/develop/get-started/redis-in-ai.md | 2 +- content/develop/use-cases/memory-layer/_index.md | 2 +- content/embeds/iris-for-developers.md | 1 + content/integrate/amazon-bedrock/_index.md | 1 + content/integrate/langchain-redis/_index.md | 2 ++ content/operate/kubernetes/_index.md | 1 + content/operate/rc/_index.md | 2 ++ content/operate/rc/databases/create-database/_index.md | 2 ++ content/operate/rs/_index.md | 1 + 20 files changed, 31 insertions(+), 2 deletions(-) create mode 100644 content/embeds/iris-for-developers.md diff --git a/content/develop/ai/ecosystem-integrations.md b/content/develop/ai/ecosystem-integrations.md index 10aa37a438..d5fc957f15 100644 --- a/content/develop/ai/ecosystem-integrations.md +++ b/content/develop/ai/ecosystem-integrations.md @@ -23,6 +23,7 @@ Redis integrates with a wide range of AI frameworks, platforms, and tools to enh ## Additional integrations +- [**Google ADK & Redis Agent Memory**](/content/integrate/google-adk/redis-agent-memory.md): Give Google Agent Development Kit (ADK) agents session and long-term memory with Redis Agent Memory. - [**AISIX AI Gateway & Redis**](https://docs.api7.ai/ai-gateway/traffic-controls/caching): Use Redis to share response cache entries and coordinate [rate-limit counters](https://docs.api7.ai/ai-gateway/traffic-controls/rate-limits) across AISIX gateway instances. - [**Microsoft Semantic Kernel**](https://learn.microsoft.com/en-us/semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/redis-connector?pivots=programming-language-csharp): Use Redis as a vector store connector with Microsoft's Semantic Kernel framework. - [**DocArray**](https://docs.docarray.org/user_guide/storing/index_redis/): Leverage Redis as a document store and vector database with Jina AI's DocArray. diff --git a/content/develop/ai/search-and-query/vectors/_index.md b/content/develop/ai/search-and-query/vectors/_index.md index 9c6d347d0f..dbec3cb85f 100644 --- a/content/develop/ai/search-and-query/vectors/_index.md +++ b/content/develop/ai/search-and-query/vectors/_index.md @@ -669,6 +669,7 @@ Here are some additional resources that apply vector search for different use ca - [Retrieval augmented generation from scratch](https://github.com/redis-developer/redis-ai-resources/blob/main/python-recipes/RAG/01_redisvl.ipynb) - [Semantic caching](https://github.com/redis-developer/redis-ai-resources/blob/main/python-recipes/semantic-cache/00_semantic_caching_gemini.ipynb) +- [Agent memory with Redis Agent Memory](/content/develop/ai/context-engine/agent-memory/_index.md), which uses vector search to recall long-term memories ## Continue learning with Redis University diff --git a/content/develop/clients/dotnet/nredisstack/vecsearch.md b/content/develop/clients/dotnet/nredisstack/vecsearch.md index c53b378f9f..7dafd4f53e 100644 --- a/content/develop/clients/dotnet/nredisstack/vecsearch.md +++ b/content/develop/clients/dotnet/nredisstack/vecsearch.md @@ -647,3 +647,5 @@ See [Vector search](/content/develop/ai/search-and-query/query/vector-search.md) for more information about the indexing options, distance metrics, and query format for vectors. + +{{< embed-md "iris-for-developers.md" >}} diff --git a/content/develop/clients/go/vecsearch.md b/content/develop/clients/go/vecsearch.md index 276bfb3034..9999ab01d9 100644 --- a/content/develop/clients/go/vecsearch.md +++ b/content/develop/clients/go/vecsearch.md @@ -251,3 +251,5 @@ See [Vector search](/content/develop/ai/search-and-query/query/vector-search.md) for more information about the indexing options, distance metrics, and query format for vectors. + +{{< embed-md "iris-for-developers.md" >}} diff --git a/content/develop/clients/jedis/vecsearch.md b/content/develop/clients/jedis/vecsearch.md index ed1316c685..31ac097c67 100644 --- a/content/develop/clients/jedis/vecsearch.md +++ b/content/develop/clients/jedis/vecsearch.md @@ -292,3 +292,5 @@ See [Vector search](/content/develop/ai/search-and-query/query/vector-search.md) for more information about the indexing options, distance metrics, and query format for vectors. + +{{< embed-md "iris-for-developers.md" >}} diff --git a/content/develop/clients/lettuce/vecsearch.md b/content/develop/clients/lettuce/vecsearch.md index 370a0351ba..d41c9e8a20 100644 --- a/content/develop/clients/lettuce/vecsearch.md +++ b/content/develop/clients/lettuce/vecsearch.md @@ -312,3 +312,5 @@ See [Vector search](/content/develop/ai/search-and-query/query/vector-search.md) for more information about the indexing options, distance metrics, and query format for vectors. + +{{< embed-md "iris-for-developers.md" >}} diff --git a/content/develop/clients/nodejs/vecsearch.md b/content/develop/clients/nodejs/vecsearch.md index 1575dc7e5e..3e3214be10 100644 --- a/content/develop/clients/nodejs/vecsearch.md +++ b/content/develop/clients/nodejs/vecsearch.md @@ -211,3 +211,5 @@ jdoc:3: 'Today is a sunny day', Score: 1.50889515877 See [Vector search](/content/develop/ai/search-and-query/query/vector-search.md) for more information about indexing options, distance metrics, and query format. + +{{< embed-md "iris-for-developers.md" >}} diff --git a/content/develop/clients/php/vecsearch.md b/content/develop/clients/php/vecsearch.md index 3689edc357..79f59b60e8 100644 --- a/content/develop/clients/php/vecsearch.md +++ b/content/develop/clients/php/vecsearch.md @@ -476,3 +476,5 @@ See [Vector search](/content/develop/ai/search-and-query/query/vector-search.md) for more information about the indexing options, distance metrics, and query format for vectors. + +{{< embed-md "iris-for-developers.md" >}} diff --git a/content/develop/clients/redis-py/vecsearch.md b/content/develop/clients/redis-py/vecsearch.md index cd222ead74..f844a57e1f 100644 --- a/content/develop/clients/redis-py/vecsearch.md +++ b/content/develop/clients/redis-py/vecsearch.md @@ -273,3 +273,5 @@ See [Vector search](/content/develop/ai/search-and-query/query/vector-search.md) for more information about the indexing options, distance metrics, and query format for vectors. + +{{< embed-md "iris-for-developers.md" >}} diff --git a/content/develop/clients/ruby/vecsearch.md b/content/develop/clients/ruby/vecsearch.md index 6e3e86ec2e..efdc35e52d 100644 --- a/content/develop/clients/ruby/vecsearch.md +++ b/content/develop/clients/ruby/vecsearch.md @@ -268,3 +268,5 @@ See [Vector search](/content/develop/ai/search-and-query/query/vector-search.md) for more information about the indexing options, distance metrics, and query format for vectors. + +{{< embed-md "iris-for-developers.md" >}} diff --git a/content/develop/get-started/rag.md b/content/develop/get-started/rag.md index 728b14fe90..514a2cefba 100644 --- a/content/develop/get-started/rag.md +++ b/content/develop/get-started/rag.md @@ -74,6 +74,7 @@ In summary, Redis offers a powerful and efficient platform for implementing RAG. - [RAG with Redis and SpringAI](https://redis.io/blog/building-a-rag-application-with-redis-and-spring-ai/) - [Build a multimodal RAG app with LangChain and Redis](https://redis.io/blog/explore-the-new-multimodal-rag-template-from-langchain-and-redis/) - [Get hands-on with advanced Redis AI Recipes](https://github.com/redis-developer/redis-ai-resources) +- [Redis Iris](/content/develop/ai/context-engine/_index.md), for managed memory and context for AI agents ## Continue learning with Redis University diff --git a/content/develop/get-started/redis-in-ai.md b/content/develop/get-started/redis-in-ai.md index b670e65233..28e6f794d8 100644 --- a/content/develop/get-started/redis-in-ai.md +++ b/content/develop/get-started/redis-in-ai.md @@ -33,7 +33,7 @@ Explore how Redis optimizes various GenAI applications through specific use case ### Optimizing AI Agent Performance -Redis improves session persistence and caching for conversational agents managing high interaction volumes. See the [Flowise Conversational Agent with Redis](https://redis.io/learn/howtos/solutions/flowise/conversational-agent) tutorial and demo for implementation details. +Redis improves session persistence and caching for conversational agents managing high interaction volumes. See the [Flowise Conversational Agent with Redis](https://redis.io/learn/howtos/solutions/flowise/conversational-agent) tutorial and demo for implementation details. For managed session and long-term memory, see [Redis Agent Memory](/content/develop/ai/context-engine/agent-memory/_index.md). ### Chatbot Development and Management diff --git a/content/develop/use-cases/memory-layer/_index.md b/content/develop/use-cases/memory-layer/_index.md index 304e6975c6..cf8a531acd 100644 --- a/content/develop/use-cases/memory-layer/_index.md +++ b/content/develop/use-cases/memory-layer/_index.md @@ -64,7 +64,7 @@ The following libraries, frameworks, and managed services build on Redis for a m - **Frameworks**: [LangChain](/content/integrate/langchain-redis/_index.md) supports Redis as a chat history and memory backend, and [LangGraph & Redis](https://redis.io/blog/langgraph-redis-build-smarter-ai-agents-with-memory-persistence/) ships a Redis checkpointer for persisting graph state across runs. - **AWS**: [Amazon Bedrock](/content/integrate/amazon-bedrock/_index.md) agent runtimes integrate with Redis for memory persistence and vector search. - **Any language**: standard Redis client libraries cover the pattern below for custom agent loops. -- **Managed**: [Redis Agent Memory Server](/content/develop/ai/context-engine/agent-memory/_index.md) is a managed agent memory service with REST and MCP interfaces, working and long-term memory tiers, deduplication, summarization, and background consolidation — useful when you'd rather not build and operate the pattern below yourself. +- **Managed**: [Redis Agent Memory](/content/develop/ai/context-engine/agent-memory/_index.md) is a managed agent memory service with REST and MCP interfaces, working and long-term memory tiers, deduplication, summarization, and background consolidation — useful when you'd rather not build and operate the pattern below yourself. ## Code examples to build your own Redis memory layer diff --git a/content/embeds/iris-for-developers.md b/content/embeds/iris-for-developers.md new file mode 100644 index 0000000000..de061049ee --- /dev/null +++ b/content/embeds/iris-for-developers.md @@ -0,0 +1 @@ +To add managed memory and context to AI agents without building them yourself, see [Redis Iris](/content/develop/ai/context-engine/_index.md). diff --git a/content/integrate/amazon-bedrock/_index.md b/content/integrate/amazon-bedrock/_index.md index 9165908fe0..0d7c7781f0 100644 --- a/content/integrate/amazon-bedrock/_index.md +++ b/content/integrate/amazon-bedrock/_index.md @@ -39,3 +39,4 @@ To fully set up Bedrock with Redis Cloud, you will need to do the following: - [Amazon Bedrock integration blog post](https://redis.io/blog/amazon-bedrock-integration-with-redis-enterprise/) - [Detailed steps](https://github.com/redis-applied-ai/aws-redis-bedrock-stack/blob/main/README.md) +- [Redis Agent Memory](/content/develop/ai/context-engine/agent-memory/_index.md), for managed session and long-term memory for agents diff --git a/content/integrate/langchain-redis/_index.md b/content/integrate/langchain-redis/_index.md index 0457f6b9db..23e9ed95d5 100644 --- a/content/integrate/langchain-redis/_index.md +++ b/content/integrate/langchain-redis/_index.md @@ -36,3 +36,5 @@ Redis integration with LangChain enables you to build sophisticated AI applicati ## Getting Started Learn how to [build AI chatbots with Redis and LangChain](https://redis.io/learn/howtos/solutions/vector/gen-ai-chatbot) for enhanced AI experiences. + +For managed session and long-term memory that your agents keep across conversations, see [Redis Agent Memory](/content/develop/ai/context-engine/agent-memory/_index.md). diff --git a/content/operate/kubernetes/_index.md b/content/operate/kubernetes/_index.md index fe5eda4f46..ccbd085dc7 100644 --- a/content/operate/kubernetes/_index.md +++ b/content/operate/kubernetes/_index.md @@ -108,4 +108,5 @@ Stay informed about new features, enhancements, and fixes. - [Redis Enterprise Software](/content/operate/rs/_index.md) - [Redis Cloud](/content/operate/rc/_index.md) - [Redis Open Source](/content/operate/oss_and_stack/_index.md) +- [Redis Iris](/content/operate/iris/_index.md), to run AI agent services such as Agent Memory with your Redis Software for Kubernetes deployment - [Glossary](/content/glossary/_index.md) \ No newline at end of file diff --git a/content/operate/rc/_index.md b/content/operate/rc/_index.md index ed4c16bab0..536826d678 100644 --- a/content/operate/rc/_index.md +++ b/content/operate/rc/_index.md @@ -28,6 +28,8 @@ Use the [Quick start](/content/operate/rc/rc-quickstart.md) to learn how to crea You can also use [Vercel's Redis Cloud integration](https://vercel.com/marketplace/redis-cloud) or [Heroku's Redis Cloud add-on](https://elements.heroku.com/addons/rediscloud) to quickly add a Redis database to your project. +To add AI agent services, such as Agent Memory, to your Redis Cloud account, see [Redis Iris](/content/operate/iris/_index.md). + ## Databases Create and manage [Redis databases](/content/operate/rc/databases/_index.md) in the cloud. - [Create database](/content/operate/rc/databases/create-database/_index.md) diff --git a/content/operate/rc/databases/create-database/_index.md b/content/operate/rc/databases/create-database/_index.md index 2d36d11be3..961024337f 100644 --- a/content/operate/rc/databases/create-database/_index.md +++ b/content/operate/rc/databases/create-database/_index.md @@ -27,4 +27,6 @@ A **Pro** plan is an hourly price based on capacity. It supports more databases, - [Create a Pro database in an existing subscription](/content/operate/rc/databases/create-database/create-pro-database-existing.md) - [Create an Active-Active database](/content/operate/rc/databases/active-active/create-active-active-database.md) +Redis Agent Memory stores its data in a Redis Cloud database. After you create one, see [Create an Agent Memory service](/content/operate/iris/agent-memory/create-service.md). + diff --git a/content/operate/rs/_index.md b/content/operate/rs/_index.md index c44c038ab7..f5d754bd90 100644 --- a/content/operate/rs/_index.md +++ b/content/operate/rs/_index.md @@ -75,6 +75,7 @@ To view documentation earlier than version 7.4, see the archived website: - [Redis Cloud](/content/operate/rc/_index.md) - [Redis Open Source](/content/operate/oss_and_stack/_index.md) - [Redis Stack](/content/operate/oss_and_stack/stack-with-enterprise/_index.md) +- [Redis Iris](/content/operate/iris/_index.md), to run AI agent services such as Agent Memory with your Redis Software deployment - [Glossary](/content/glossary/_index.md) ## Continue learning with Redis University From deca3a09ee63372109316194084acc549454fafc Mon Sep 17 00:00:00 2001 From: paoloredis <158162796+paoloredis@users.noreply.github.com> Date: Fri, 23 Oct 2026 13:36:09 +0200 Subject: [PATCH 63/76] DOC-7218 Fix AI chatbot pane ratings missing from DocuSight (#4334) The pane now sends the chat page's URL as a rating's origin, as the chat page does, instead of the docs page's URL. DocuSight's copilot rating endpoint keeps a rating only if its origin starts with the chat page's URL. It answers 200 to every other rating and drops it, so every rating the pane sent was lost while the pane said thanks. DocuSight still records which docs page a rating came from, because it stores the Referer header too. That field is in its CSV export, not in the dashboard table. Unlike docs page ratings, ratings given on a staging build are now kept too. Checked before and after on the live PING page, with this branch's script swapped in and ratings sent to DocuSight run locally: the live script's rating was dropped, this one's was kept. Learned: DocuSight answers 200 even to ratings it drops, so the pane said thanks for every rating it lost Constraint: origin must start with https://redis.io/chat/, or DocuSight's handle_chat_feedback (redis-learn/docusight, app/routes.py) drops the rating Gaps: checked against DocuSight main (1a85c58) run locally, not against the deployed service Ticket: DOC-7218 Co-authored-by: Claude Opus 5.5 --- static/js/ai-chat-pane.js | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/static/js/ai-chat-pane.js b/static/js/ai-chat-pane.js index 5c6fe44bec..75d844b50d 100644 --- a/static/js/ai-chat-pane.js +++ b/static/js/ai-chat-pane.js @@ -664,12 +664,13 @@ if (note && focusLost) note.focus({ preventScroll: true }); } - /* The fields the chat page sends. origin is the page the rating was given - on, as for the docs' own page ratings (feedback.html); the chat page sends - its own URL there. */ + /* The same fields the chat page sends. origin is the chat page's URL too. + DocuSight keeps a rating only if its origin starts with + https://redis.io/chat/, and drops any other while still answering 200. + It still records the page, because it stores the Referer header too. */ function sendRating(m, kind, comment, token) { var data = { - origin: location.origin + location.pathname, + origin: location.origin + '/chat/', question: m.q || '', answer: decode(m.text), comment: comment, From 296b88b1a7a7d98f0f89e1036edff43d5f7bffae Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Fri, 23 Oct 2026 06:33:05 -0700 Subject: [PATCH 64/76] Update content/operate/rc/api/api-reference/openapi.json (#4328) Co-authored-by: redisdocsapp[bot] <177626021+redisdocsapp[bot]@users.noreply.github.com> --- .../operate/rc/api/api-reference/openapi.json | 122 +++++++++--------- 1 file changed, 61 insertions(+), 61 deletions(-) diff --git a/content/operate/rc/api/api-reference/openapi.json b/content/operate/rc/api/api-reference/openapi.json index e1f1a7f7fe..a6088bc486 100644 --- a/content/operate/rc/api/api-reference/openapi.json +++ b/content/operate/rc/api/api-reference/openapi.json @@ -96,7 +96,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_1", + "operationId": "proxyDataIntegrationWorkspace_4", "parameters": [ { "name": "subscriptionId", @@ -166,7 +166,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_3", + "operationId": "proxyDataIntegrationWorkspace", "parameters": [ { "name": "subscriptionId", @@ -237,7 +237,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_2", + "operationId": "proxyDataIntegrationWorkspace_1", "parameters": [ { "name": "subscriptionId", @@ -308,7 +308,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace", + "operationId": "proxyDataIntegrationWorkspace_3", "parameters": [ { "name": "subscriptionId", @@ -379,7 +379,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_4", + "operationId": "proxyDataIntegrationWorkspace_2", "parameters": [ { "name": "subscriptionId", @@ -452,7 +452,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_6", + "operationId": "proxyDataIntegrationWorkspace_9", "parameters": [ { "name": "subscriptionId", @@ -522,7 +522,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_8", + "operationId": "proxyDataIntegrationWorkspace_5", "parameters": [ { "name": "subscriptionId", @@ -593,7 +593,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_7", + "operationId": "proxyDataIntegrationWorkspace_6", "parameters": [ { "name": "subscriptionId", @@ -664,7 +664,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_5", + "operationId": "proxyDataIntegrationWorkspace_8", "parameters": [ { "name": "subscriptionId", @@ -735,7 +735,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_9", + "operationId": "proxyDataIntegrationWorkspace_7", "parameters": [ { "name": "subscriptionId", @@ -8353,32 +8353,32 @@ "403": { "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" }, - "401": { - "description": "Unauthorized - Authentication failed for requested resource" - }, - "404": { - "description": "Not Found - The resource you were trying to reach was not found or does not exist" - }, - "200": { - "description": "Memory stores returned.", + "424": { + "description": "Failed Dependency - The memory store's database is unavailable or out of memory, e.g. /errors/resource-unavailable or /errors/database-out-of-memory. When the error originates from the Agent Memory service the body is an application/problem+json document (MemoryProblemDetails) carrying the service's error type and detail", "content": { - "application/json": { + "application/problem+json": { "schema": { - "$ref": "#/components/schemas/MemoryStoreListResponse" + "$ref": "#/components/schemas/MemoryProblemDetails" } } } }, - "424": { - "description": "Failed Dependency - The memory store's database is unavailable or out of memory, e.g. /errors/resource-unavailable or /errors/database-out-of-memory. When the error originates from the Agent Memory service the body is an application/problem+json document (MemoryProblemDetails) carrying the service's error type and detail", + "200": { + "description": "Memory stores returned.", "content": { - "application/problem+json": { + "application/json": { "schema": { - "$ref": "#/components/schemas/MemoryProblemDetails" + "$ref": "#/components/schemas/MemoryStoreListResponse" } } } }, + "401": { + "description": "Unauthorized - Authentication failed for requested resource" + }, + "404": { + "description": "Not Found - The resource you were trying to reach was not found or does not exist" + }, "408": { "description": "Request Timeout - The server didn't receive a complete request message within the server's allotted timeout period" }, @@ -8493,12 +8493,6 @@ "403": { "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" }, - "401": { - "description": "Unauthorized - Authentication failed for requested resource" - }, - "404": { - "description": "Not Found - The resource you were trying to reach was not found or does not exist" - }, "424": { "description": "Failed Dependency - The memory store's database is unavailable or out of memory, e.g. /errors/resource-unavailable or /errors/database-out-of-memory. When the error originates from the Agent Memory service the body is an application/problem+json document (MemoryProblemDetails) carrying the service's error type and detail", "content": { @@ -8509,6 +8503,12 @@ } } }, + "401": { + "description": "Unauthorized - Authentication failed for requested resource" + }, + "404": { + "description": "Not Found - The resource you were trying to reach was not found or does not exist" + }, "408": { "description": "Request Timeout - The server didn't receive a complete request message within the server's allotted timeout period" }, @@ -8568,32 +8568,32 @@ "403": { "description": "Forbidden - Not allowed to create requested resource (primarily due to not active resource)" }, - "401": { - "description": "Unauthorized - Authentication failed for requested resource" - }, - "404": { - "description": "Not Found - The resource you were trying to reach was not found or does not exist" - }, - "201": { - "description": "API key created. The generated key value is returned once in the response body.", + "424": { + "description": "Failed Dependency - The memory store's database is unavailable or out of memory, e.g. /errors/resource-unavailable or /errors/database-out-of-memory. When the error originates from the Agent Memory service the body is an application/problem+json document (MemoryProblemDetails) carrying the service's error type and detail", "content": { - "application/json": { + "application/problem+json": { "schema": { - "$ref": "#/components/schemas/MemoryStoreApiKeyCreateResponse" + "$ref": "#/components/schemas/MemoryProblemDetails" } } } }, - "424": { - "description": "Failed Dependency - The memory store's database is unavailable or out of memory, e.g. /errors/resource-unavailable or /errors/database-out-of-memory. When the error originates from the Agent Memory service the body is an application/problem+json document (MemoryProblemDetails) carrying the service's error type and detail", + "201": { + "description": "API key created. The generated key value is returned once in the response body.", "content": { - "application/problem+json": { + "application/json": { "schema": { - "$ref": "#/components/schemas/MemoryProblemDetails" + "$ref": "#/components/schemas/MemoryStoreApiKeyCreateResponse" } } } }, + "401": { + "description": "Unauthorized - Authentication failed for requested resource" + }, + "404": { + "description": "Not Found - The resource you were trying to reach was not found or does not exist" + }, "408": { "description": "Request Timeout - The server didn't receive a complete request message within the server's allotted timeout period" }, @@ -10079,9 +10079,6 @@ "412": { "description": "Precondition Failed - Feature flag for this flow is off" }, - "403": { - "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" - }, "200": { "description": "Memory store returned.", "content": { @@ -10092,11 +10089,8 @@ } } }, - "401": { - "description": "Unauthorized - Authentication failed for requested resource" - }, - "404": { - "description": "Not Found - The resource you were trying to reach was not found or does not exist" + "403": { + "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" }, "424": { "description": "Failed Dependency - The memory store's database is unavailable or out of memory, e.g. /errors/resource-unavailable or /errors/database-out-of-memory. When the error originates from the Agent Memory service the body is an application/problem+json document (MemoryProblemDetails) carrying the service's error type and detail", @@ -10108,6 +10102,12 @@ } } }, + "401": { + "description": "Unauthorized - Authentication failed for requested resource" + }, + "404": { + "description": "Not Found - The resource you were trying to reach was not found or does not exist" + }, "429": { "description": "Too Many Requests - Too many resources are concurrently created / updated / deleted in the account. Please re-submit API requests after resource changes are completed)" }, @@ -12804,18 +12804,9 @@ "412": { "description": "Precondition Failed - Feature flag for this flow is off" }, - "204": { - "description": "API key deleted." - }, "403": { "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" }, - "401": { - "description": "Unauthorized - Authentication failed for requested resource" - }, - "404": { - "description": "Not Found - The resource you were trying to reach was not found or does not exist" - }, "424": { "description": "Failed Dependency - The memory store's database is unavailable or out of memory, e.g. /errors/resource-unavailable or /errors/database-out-of-memory. When the error originates from the Agent Memory service the body is an application/problem+json document (MemoryProblemDetails) carrying the service's error type and detail", "content": { @@ -12826,6 +12817,15 @@ } } }, + "204": { + "description": "API key deleted." + }, + "401": { + "description": "Unauthorized - Authentication failed for requested resource" + }, + "404": { + "description": "Not Found - The resource you were trying to reach was not found or does not exist" + }, "408": { "description": "Request Timeout - The server didn't receive a complete request message within the server's allotted timeout period" }, From 992808f637d10b0148da24444f8fd8450d479cb8 Mon Sep 17 00:00:00 2001 From: andy-stark-redis <164213578+andy-stark-redis@users.noreply.github.com> Date: Fri, 23 Oct 2026 14:35:12 +0100 Subject: [PATCH 65/76] DOC-7208 Remove the license section from the RDI FAQ Delete "Which license does RDI use?" from the RDI FAQ, in both the current docs and the 1.19.1 version. Per agreement with the VP of Pricing, licensing and pricing details stay out of the RDI technical docs because RDI is a sales-driven tool. The removed answer also said the staging database needs two extra Redis Enterprise shards. That requirement already appears on the installation requirements summary page, so nothing was relocated. No pages linked to the removed anchor. The 1.19.1 tree was edited alongside latest because it is still maintained, not a frozen archive. Directive: keep licensing and pricing details out of RDI docs (VP of Pricing agreement) Ticket: DOC-7208 Co-Authored-By: Claude Opus 5.5 --- content/integrate/redis-data-integration/1.19.1/faq.md | 5 ----- content/integrate/redis-data-integration/faq.md | 5 ----- 2 files changed, 10 deletions(-) diff --git a/content/integrate/redis-data-integration/1.19.1/faq.md b/content/integrate/redis-data-integration/1.19.1/faq.md index 356a8f60d6..c06b567897 100644 --- a/content/integrate/redis-data-integration/1.19.1/faq.md +++ b/content/integrate/redis-data-integration/1.19.1/faq.md @@ -17,11 +17,6 @@ weight: 50 url: '/integrate/redis-data-integration/1.19.1/faq/' --- -## Which license does RDI use? - -You must purchase a commercial license for RDI with Redis Enterprise. This includes two extra -Redis Enterprise shards (primary and replica) for the staging database. - ## How does RDI track data changes in the source database? RDI uses change data capture (CDC) mechanisms that are specific to each of the diff --git a/content/integrate/redis-data-integration/faq.md b/content/integrate/redis-data-integration/faq.md index 98e7fcfdc7..2f67e83527 100644 --- a/content/integrate/redis-data-integration/faq.md +++ b/content/integrate/redis-data-integration/faq.md @@ -17,11 +17,6 @@ type: integration weight: 50 --- -## Which license does RDI use? - -You must purchase a commercial license for RDI with Redis Enterprise. This includes two extra -Redis Enterprise shards (primary and replica) for the staging database. - ## How does RDI track data changes in the source database? RDI uses change data capture (CDC) mechanisms that are specific to each of the From 612ff69435dcfe5808a1d077b05dc6292a39ec34 Mon Sep 17 00:00:00 2001 From: andy-stark-redis <164213578+andy-stark-redis@users.noreply.github.com> Date: Fri, 23 Oct 2026 16:02:16 +0100 Subject: [PATCH 66/76] DOC-7216 Correct the VisuaLeaf and Beekeeper Studio third-party tool entries Re-vetting both live entries against vendor docs and source found claims the evidence contradicts. VisuaLeaf: the pricing page lists Redis connections as "Not included" in the free Community Edition (Basic, Professional, or the 14-day trial needed), so "with a free tier and paid plans" and "Download for free" misled. The SSH tunnel claim is dropped: the Redis product page says SSH tunnelling for Redis is not available yet, while the docs page says a standalone server can be reached through one. Restore it, scoped, once the vendor reconciles the two. Beekeeper Studio: SSH tunnels are not available for Redis ('server:ssh' is in the Redis dialect's disabledFeatures in lib/db/clients/index.ts at v6.1.6, and RedisForm.vue omits the common-advanced SSH section other forms include), even though docs/user_guide/connecting/redis.md mentions SSH tunneling. The AI Shell is paid-only (docs/user_guide/sql-ai-shell.md: "included in all paid versions"), the Redis client only calls createClient (no Cluster or Sentinel), and RedisForm.vue labels Redis support "still in beta". The key browser loops SCAN to cursor 0 before slicing each page (scanAll, called from the key listing), values load whole (LRANGE 0 -1, SMEMBERS, HGETALL, XRANGE - +), and read-only mode only blocks commands it finds in its command table without IS_READ_ONLY, so unknown commands pass. Dropped the unverified "full feature parity" claim and the stray line break. Gaps: VisuaLeaf's Lettuce, replica-read, client-certificate, and ElastiCache/Azure claims remain unverified, not contradicted, so they stay Ticket: DOC-7216 Recheck: when VisuaLeaf's Redis page and docs agree on SSH tunnel support Co-Authored-By: Claude Opus 5.5 --- content/develop/tools/_index.md | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/content/develop/tools/_index.md b/content/develop/tools/_index.md index d145b20bf2..13a125762e 100644 --- a/content/develop/tools/_index.md +++ b/content/develop/tools/_index.md @@ -55,7 +55,7 @@ The tools above are maintained by Redis. The tools below are maintained by their ### VisuaLeaf -[VisuaLeaf](https://visualeaf.com/database/redis/) is a desktop database GUI for macOS, Windows, and Linux, with a free tier and paid plans. It presents Redis in the same interface, and the same connection list, as the MongoDB and SQL databases a team already runs, so a cache and the application database behind it can be inspected in one place. The Redis support is built on [Lettuce](https://github.com/redis/lettuce) and provides: +[VisuaLeaf](https://visualeaf.com/database/redis/) is a desktop database GUI for macOS, Windows, and Linux. Redis connections need a paid license or the 14-day trial: the free Community Edition doesn't include them. It presents Redis in the same interface, and the same connection list, as the MongoDB and SQL databases a team already runs, so a cache and the application database behind it can be inspected in one place. The Redis support is built on [Lettuce](https://github.com/redis/lettuce) and provides: * A command console and script editor that sends any command through a generic dispatch path — including module commands such as `JSON.GET` and `FT.SEARCH` — with command help, saved scripts, batch runs where each command's reply is shown separately, and command history. * A key browser that discovers keys with `SCAN` (never `KEYS`), groups them by prefix on a delimiter you choose, and filters by type or value. Large values are paged rather than fetched whole: lists and sorted sets by index window, hashes and sets by `HSCAN` / `SSCAN` cursor. TTLs are shown and editable, including per-field hash TTLs on servers that support `HTTL`. @@ -63,9 +63,9 @@ The tools above are maintained by Redis. The tools below are maintained by their * Security management: ACL users and permissions over the data-plane `ACL` commands, `requirepass` changes, and user management on Redis Enterprise and Redis Cloud through their REST APIs. * Per-key JSON export and restore for string, hash, list, set, and sorted set keys, preserving TTL. RDB parsing is out of scope. -It connects to standalone servers, Cluster deployments via seed nodes, and Sentinel via a master name and sentinel nodes, over plain TCP or TLS (`rediss://`, with optional client certificates), directly or through an SSH tunnel. Reads can be routed to replicas. Redis-compatible servers — Valkey, KeyDB, ElastiCache, Azure Cache — connect the same way. Commands carry the privileges of the user you connect as, so read-only access has to come from a Redis ACL rather than from the tool. +It connects to standalone servers, Cluster deployments via seed nodes, and Sentinel via a master name and sentinel nodes, over plain TCP or TLS (`rediss://`, with optional client certificates). Reads can be routed to replicas. Redis-compatible servers — Valkey, KeyDB, ElastiCache, Azure Cache — connect the same way. Commands carry the privileges of the user you connect as, so read-only access has to come from a Redis ACL rather than from the tool. -[Download for free on their website](https://visualeaf.com/database/redis/) +[Download from their website](https://visualeaf.com/database/redis/) ### LibreDB Studio @@ -79,16 +79,15 @@ It connects to a single standalone node over an unencrypted connection: Cluster, ### Beekeeper Studio -[Beekeeper Studio](https://www.beekeeperstudio.io/db/redis-client/) is a free, open-source database manager that supports a range of databases like Redis, MongoDB, PostgreSQL, MySQL and more. -Beekeeper Studio is a desktop app, and is available for Linux, macOS, and Windows with full feature parity across all operating systems. +[Beekeeper Studio](https://www.beekeeperstudio.io/db/redis-client/) is an open-source desktop database manager for Linux, macOS, and Windows that supports a range of databases like Redis, MongoDB, PostgreSQL, MySQL and more. Redis is included in the free Community Edition, and the app labels its Redis support as beta. Redis support includes -* A command console with autocomplete, history, saved commands, and multi-tab editing, plus an AI assistant that can turn natural-language questions into Redis commands. +* A command console with autocomplete, history, saved commands, and multi-tab editing, plus, in the paid editions, an AI assistant that can turn natural-language questions into Redis commands. * A spreadsheet-style key browser for browsing, filtering, and editing values, including JSON documents via the ReJSON module. * One-click export of query results to CSV, Excel, or JSON. -It connects to a Redis server directly or over an SSH tunnel, with TLS/SSL supported, and authenticates with either a password or a username and password against Redis 6+ ACLs. +It connects to a Redis server with TLS/SSL supported, and authenticates with either a password or a username and password against Redis 6+ ACLs. Only standalone servers are supported: Cluster, Sentinel, and SSH tunnels are not. The key browser runs `SCAN` across the whole keyspace on every page and loads each value whole, so large databases are slow to browse. Read-only mode blocks the write commands the tool recognizes and passes unrecognized commands through. [Download for free on their website](https://www.beekeeperstudio.io/db/redis-client/) From 683d384d4cca99b5c9909c9af962bfcc6098fea9 Mon Sep 17 00:00:00 2001 From: andy-stark-redis <164213578+andy-stark-redis@users.noreply.github.com> Date: Fri, 23 Oct 2026 16:19:11 +0100 Subject: [PATCH 67/76] DOC-7221 Pin RDI 1.19.1 install docs to v1.19.1 downloads The archived 1.19.1 install, K8s and upgrade pages used {{< rdi-version >}}, which reads rdi_current_version from config.toml (now 2.0.0). So the 1.19.1 docs sent readers to the 2.0.0 installer, Helm chart and CLI binaries. The export RDI_VERSION lines were the worst case: they aren't links, so a reader who copy-pasted them got 2.0.0 even after checking the URLs. All 14 uses in links and shell snippets are now the literal 1.19.1. The 1.19.1 artifacts were confirmed by hand to still be at the same S3 paths. The shortcode remains only in the new version note, where pointing at the current release is the intent. The note is the same on all three pages, including upgrade.md. It asks readers to check the release notes before moving to the latest version, because 1.19.x to 2.0.0 isn't guaranteed to suit every setup. It links the release notes index rather than rdi-2-0-0.md, so it doesn't go stale when the current version changes. Learned: rdi-version always resolves to the current release, so it is wrong in any archived snapshot outside the version note Constraint: archived RDI pages use literal version numbers in download URLs and RDI_VERSION exports Directive: archives are copied by hand; when 2.0.0 is archived, replace rdi-version in its links and snippets the same way Ticket: DOC-7221 Co-Authored-By: Claude Opus 5.5 --- .../1.19.1/installation/install-k8s.md | 22 ++++++++++++------- .../1.19.1/installation/install-vm.md | 9 +++++--- .../1.19.1/installation/upgrade.md | 14 ++++++++---- 3 files changed, 30 insertions(+), 15 deletions(-) diff --git a/content/integrate/redis-data-integration/1.19.1/installation/install-k8s.md b/content/integrate/redis-data-integration/1.19.1/installation/install-k8s.md index 25e9058684..5a1f869b97 100644 --- a/content/integrate/redis-data-integration/1.19.1/installation/install-k8s.md +++ b/content/integrate/redis-data-integration/1.19.1/installation/install-k8s.md @@ -21,6 +21,12 @@ This guide explains how to use the RDI [Helm chart](https://helm.sh/docs/topics/ to install on [Kubernetes](https://kubernetes.io/) (K8s). You can also [Install RDI on VMs](/content/integrate/redis-data-integration/1.19.1/installation/install-vm.md). +> [!NOTE] +> These docs are for RDI v1.19.1. The latest version is RDI v{{< rdi-version >}}, and we +> recommend you upgrade to it if you can. Before you do, check the +> [release notes](/content/integrate/redis-data-integration/release-notes/_index.md) +> to make sure the latest version still supports your setup. + The installation creates the following K8s objects: - A K8s [namespace](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/) named `rdi`. @@ -55,10 +61,10 @@ Complete the following steps before installing the RDI Helm chart: more information). - Download the RDI Helm chart tar file from the - [Redis download center](https://redis-enterprise-software-downloads.s3.amazonaws.com/redis-di/rdi-{{< rdi-version >}}.tgz) (in the *Modules, Tools & Integration* category) . + [Redis download center](https://redis-enterprise-software-downloads.s3.amazonaws.com/redis-di/rdi-1.19.1.tgz) (in the *Modules, Tools & Integration* category) . ```bash - export RDI_VERSION={{< rdi-version >}} + export RDI_VERSION=1.19.1 wget https://redis-enterprise-software-downloads.s3.amazonaws.com/redis-di/rdi-$RDI_VERSION.tgz ``` @@ -163,16 +169,16 @@ which to choose, run `uname -sm`: `Linux x86_64` is Linux amd64, `Linux aarch64` | Platform | Download | | :-- | :-- | -| Linux, x86-64 (amd64) | [`redis-di`](https://redis-enterprise-software-downloads.s3.amazonaws.com/redis-di/cli/{{< rdi-version >}}/bin/linux-amd64/redis-di) | -| Linux, ARM64 (aarch64) | [`redis-di`](https://redis-enterprise-software-downloads.s3.amazonaws.com/redis-di/cli/{{< rdi-version >}}/bin/linux-arm64/redis-di) | -| macOS, Intel (amd64) | [`redis-di`](https://redis-enterprise-software-downloads.s3.amazonaws.com/redis-di/cli/{{< rdi-version >}}/bin/darwin-amd64/redis-di) | -| macOS, Apple silicon (arm64) | [`redis-di`](https://redis-enterprise-software-downloads.s3.amazonaws.com/redis-di/cli/{{< rdi-version >}}/bin/darwin-arm64/redis-di) | -| Windows, x86-64 (amd64) | [`redis-di.exe`](https://redis-enterprise-software-downloads.s3.amazonaws.com/redis-di/cli/{{< rdi-version >}}/bin/windows-amd64/redis-di.exe) | +| Linux, x86-64 (amd64) | [`redis-di`](https://redis-enterprise-software-downloads.s3.amazonaws.com/redis-di/cli/1.19.1/bin/linux-amd64/redis-di) | +| Linux, ARM64 (aarch64) | [`redis-di`](https://redis-enterprise-software-downloads.s3.amazonaws.com/redis-di/cli/1.19.1/bin/linux-arm64/redis-di) | +| macOS, Intel (amd64) | [`redis-di`](https://redis-enterprise-software-downloads.s3.amazonaws.com/redis-di/cli/1.19.1/bin/darwin-amd64/redis-di) | +| macOS, Apple silicon (arm64) | [`redis-di`](https://redis-enterprise-software-downloads.s3.amazonaws.com/redis-di/cli/1.19.1/bin/darwin-arm64/redis-di) | +| Windows, x86-64 (amd64) | [`redis-di.exe`](https://redis-enterprise-software-downloads.s3.amazonaws.com/redis-di/cli/1.19.1/bin/windows-amd64/redis-di.exe) | For example, to download the CLI for Linux amd64, make it executable, and put it on your `PATH`: ```bash -export RDI_VERSION={{< rdi-version >}} +export RDI_VERSION=1.19.1 wget https://redis-enterprise-software-downloads.s3.amazonaws.com/redis-di/cli/$RDI_VERSION/bin/linux-amd64/redis-di chmod +x redis-di sudo mv redis-di /usr/local/bin/ diff --git a/content/integrate/redis-data-integration/1.19.1/installation/install-vm.md b/content/integrate/redis-data-integration/1.19.1/installation/install-vm.md index 33857da003..ebd2804557 100644 --- a/content/integrate/redis-data-integration/1.19.1/installation/install-vm.md +++ b/content/integrate/redis-data-integration/1.19.1/installation/install-vm.md @@ -22,7 +22,10 @@ your source database. You can also [Install RDI on Kubernetes](/content/integrate/redis-data-integration/1.19.1/installation/install-k8s.md). > [!NOTE] -> We recommend you always use the latest version, which is RDI v{{< rdi-version >}}. +> These docs are for RDI v1.19.1. The latest version is RDI v{{< rdi-version >}}, and we +> recommend you upgrade to it if you can. Before you do, check the +> [release notes](/content/integrate/redis-data-integration/release-notes/_index.md) +> to make sure the latest version still supports your setup. ## Create the RDI database @@ -151,12 +154,12 @@ Follow the steps below for each of your VMs. > [Using the mount command](https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/5/html/deployment_guide/chap-using_the_mount_command) in the Red Hat documentation to learn how to remount a partition. 1. Download the RDI installer from the - [Redis download center](https://redis-enterprise-software-downloads.s3.amazonaws.com/redis-di/rdi-installation-{{< rdi-version >}}.tar.gz) + [Redis download center](https://redis-enterprise-software-downloads.s3.amazonaws.com/redis-di/rdi-installation-1.19.1.tar.gz) (from the *Modules, Tools & Integration* category) and extract it to your preferred installation folder. ```bash - export RDI_VERSION={{< rdi-version >}} + export RDI_VERSION=1.19.1 wget https://redis-enterprise-software-downloads.s3.amazonaws.com/redis-di/rdi-installation-$RDI_VERSION.tar.gz tar -xvf rdi-installation-$RDI_VERSION.tar.gz ``` diff --git a/content/integrate/redis-data-integration/1.19.1/installation/upgrade.md b/content/integrate/redis-data-integration/1.19.1/installation/upgrade.md index b2e622075f..53fbeac649 100644 --- a/content/integrate/redis-data-integration/1.19.1/installation/upgrade.md +++ b/content/integrate/redis-data-integration/1.19.1/installation/upgrade.md @@ -17,18 +17,24 @@ weight: 30 url: '/integrate/redis-data-integration/1.19.1/installation/upgrade/' --- +> [!NOTE] +> These docs are for RDI v1.19.1. The latest version is RDI v{{< rdi-version >}}, and we +> recommend you upgrade to it if you can. Before you do, check the +> [release notes](/content/integrate/redis-data-integration/release-notes/_index.md) +> to make sure the latest version still supports your setup. + ## Upgrading a VM installation Follow the steps below to upgrade an existing [VM installation](/content/integrate/redis-data-integration/1.19.1/installation/install-vm.md) of RDI: -1. Download the RDI installer from the [Redis download center](https://redis-enterprise-software-downloads.s3.amazonaws.com/redis-di/rdi-installation-{{< rdi-version >}}.tar.gz) +1. Download the RDI installer from the [Redis download center](https://redis-enterprise-software-downloads.s3.amazonaws.com/redis-di/rdi-installation-1.19.1.tar.gz) (in the *Modules, Tools & Integration* category) and extract it to your preferred installation folder. ```bash - export RDI_VERSION={{< rdi-version >}} + export RDI_VERSION=1.19.1 wget https://redis-enterprise-software-downloads.s3.amazonaws.com/redis-di/rdi-installation-$RDI_VERSION.tar.gz tar -xvf rdi-installation-$RDI_VERSION.tar.gz ``` @@ -116,11 +122,11 @@ installation of RDI: [Using a private image registry](/content/integrate/redis-data-integration/1.19.1/installation/install-k8s.md#using-a-private-image-registry) and add them to your local registry. -1. Download the RDI Helm chart tar file from the [Redis download center](https://redis-enterprise-software-downloads.s3.amazonaws.com/redis-di/rdi-{{< rdi-version >}}.tgz) +1. Download the RDI Helm chart tar file from the [Redis download center](https://redis-enterprise-software-downloads.s3.amazonaws.com/redis-di/rdi-1.19.1.tgz) (in the *Modules, Tools & Integration* category). ```bash - export RDI_VERSION={{< rdi-version >}} + export RDI_VERSION=1.19.1 wget https://redis-enterprise-software-downloads.s3.amazonaws.com/redis-di/rdi-$RDI_VERSION.tgz ``` From 10adbdee047760fdf1862c4f9e9e3bd03700ba8b Mon Sep 17 00:00:00 2001 From: andy-stark-redis <164213578+andy-stark-redis@users.noreply.github.com> Date: Fri, 23 Oct 2026 16:43:25 +0100 Subject: [PATCH 68/76] DOC-7219 Add a /vet-redis-tool skill for vetting third-party Redis tools MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds /vet-redis-tool, the counterpart of /vet-redis-client, for submissions to the Third-party tools section of the Client tools page. PR #4313 (DBine) prompted it. DBine looked legitimate, but its repo and domain were both created on 2026-09-28 and one person wrote every commit. The client rubric had nothing that covered an installed application holding database credentials. The skill scores 19 criteria (38 points), each band citing a URL and a concrete fact. Only sections A–C (maturity, adoption, trust and safety; 24 points) decide the verdict: include at 17 or more, discuss at 12–16, decline below 12. The page is a directory, so section D's Redis-fit rows decide which verified gaps the entry must state, and E1/E2 must reach Pass before merge. Gates cover tools under 6 months old ("Decline (not yet)"), generic key-value adapters, and scores resting on more than 3 blind Warns. Hard stops must be confirmed before they force a Decline. The design came from calibration runs against DBine (expected to wait), Beekeeper Studio (expected to include), and VisuaLeaf (closed source). The first draft declined all three, two of them through hard stops on weak evidence, and that showed the verdict was measuring the wrong thing. Re-scored under the final rubric: Beekeeper 19/24 include, DBine 16/24 but gated on age, VisuaLeaf 13/24 discuss. A Codex review of the rubric's mechanics and five Bugbot passes then closed band overlaps and gaps. Most were enumerated Warn lists that missed a case, and "nothing observed" Fail bands that every closed-source tool fell into. The last pass caught a merge-blocking one: E2's Warn band included zero missing elements, so Pass was unreachable. The calibration runs also found errors in two live entries, fixed in DOC-7216 / #4333. Learned: a rubric that declines every listed tool is measuring the wrong thing; split "safe to list" (A–C) from "what the entry must say" (D) Learned: write Warn bands as "neither Pass nor Fail" plus examples, and say where a "nothing observed" Fail is observable; enumerated Warns and unscoped absences caused nearly every band defect Constraint: A1 under 6 months is a gate, not just a score; that is what declines DBine Constraint: a hard stop resting only on vendor docs or conflicting sources is "Needs discussion (unconfirmed hard stop)" until source, artifact inspection, a run, or the vendor confirms it Constraint: a default path verified bad decides C2 and C3; a blind path never rescues a confirmed Fail Constraint: the entry states only verified limitations; an unverified one is dropped from the entry, never asserted Rejected: OSI license as a hard stop | VisuaLeaf (closed source, paid Redis) is already listed Rejected: recoverable credential storage as a hard stop | it would decline most of the GUI category, so it scores C3 Fail instead Rejected: any shipped critical/high dependency advisory as C6 Fail | 900-package Electron apps always ship some; Fail needs a shown path from the tool's own code Directive: read pricing and feature matrices from the rendered page or the source, never a WebFetch summary; it gave two different wrong answers for one table Gaps: cut-offs (17/12) calibrated on three desktop GUIs only; no run yet against a CLI, web, or container tool; later Bugbot passes may find further edge cases; human review was a skim, so the PR's threshold questions are unanswered Recheck: after the next real tool submission is scored with this skill Ticket: DOC-7219 Co-Authored-By: Claude Opus 5.5 --- .claude/skills/vet-redis-tool/SKILL.md | 223 +++++++++++++++++++++++++ 1 file changed, 223 insertions(+) create mode 100644 .claude/skills/vet-redis-tool/SKILL.md diff --git a/.claude/skills/vet-redis-tool/SKILL.md b/.claude/skills/vet-redis-tool/SKILL.md new file mode 100644 index 0000000000..6817a5daf5 --- /dev/null +++ b/.claude/skills/vet-redis-tool/SKILL.md @@ -0,0 +1,223 @@ +--- +name: vet-redis-tool +description: Score a third-party Redis tool (GUI, desktop database manager, web console, CLI) against the docs team's quality checklist. Use when reviewing a PR or suggestion to add an entry to the Third-party tools section of the Client tools page (content/develop/tools/_index.md), when re-vetting an entry already on that page, or when asked to vet/evaluate a Redis tool. For client libraries, use /vet-redis-client instead. +--- + +# Vet a third-party Redis tool + +Score a third-party Redis tool against the docs team's 19-criterion checklist and produce an evidence-backed verdict (Recommend include / Needs discussion / Decline). Every score must cite a URL plus the specific fact observed (e.g. "repo created 2026-09-28"). Reviews, blog posts, vendor-published comparisons, and social-media popularity are deliberately excluded as signals — only verifiable repo, release, and artifact facts count. + +This is the tools sibling of `/vet-redis-client`, and keeps its scoring discipline. It differs in three ways: + +- **A tool is an installed application that holds credentials to someone's database**, so trust and safety (section C) carry weight that has no counterpart for a library. +- **There is no package registry**, so adoption is read from weaker, independent signals. +- **Closed-source tools are eligible.** The page already lists one (VisuaLeaf), so an OSI license is not a hard stop here. + +Most of the tools on this page are multi-database managers. **The criteria in sections A and B measure the project; section D measures its Redis support.** A mature project can carry young, thin, or beta Redis support, and D is where that shows. + +## 1. Identify the tool + +The input is the PR adding the entry, the tool's repository, its website, or an entry already on the page (a re-vet). From any of these, establish: + +- The source repository, if public. For a closed-source tool, the publisher (company or person) and where the binaries are published. +- The **entry text**: the PR diff for a new entry, or the current subsection of `content/develop/tools/_index.md` for a re-vet. Section E scores it, and its claims are hypotheses to verify, not evidence. +- The submitter's relationship to the tool. Record it, but **do not score it**: most entries are submitted by the tool's own developer, and that is fine. Note it for the reviewer (section 5). + +If you cannot find either public source or an identifiable publisher, stop and report that — an unattributable binary is disqualifying on its own. + +## 2. Gather evidence + +`gh`, `whois`, and large release downloads need an unsandboxed call in this environment. `$TMPDIR` differs between sandboxed and unsandboxed calls, so keep scratch files under one explicit absolute path (the session scratchpad) rather than `$TMPDIR`. Vendor websites are often refused by the sandbox; a refusal is the boundary, so note the gap and use WebFetch or the browser tools rather than finding another route. Check all of the following before scoring: + +- **Repo metadata** — `gh api repos//`: `created_at`, `pushed_at`, stars, forks, license, archived flag. +- **Releases** — `gh api repos///releases`: tags, `published_at`, and per-asset `download_count`. +- **Commits** — author names over the last 12 months, and which accounts are the same person. +- **Issues and PRs** — the A4 sample, and who opens them. +- **Source tree** — `gh api 'repos///git/trees/HEAD?recursive=1'` to find the Redis code, then the contents API to read it. Do not rely on GitHub code search (see the traps). +- **CI** — the workflow list (`…/contents/.github/workflows`), and the run that built the latest release. Its logs often include the signing and notarization verification that C1 needs (`gh api …/actions/jobs//logs`). +- **Updater metadata** — for Electron and Tauri apps, `latest.yml`, `latest-mac.yml`, or `latest.json` next to the release artifacts. C2 reads it. +- **Security** — `…/security-advisories`, SECURITY.md or `/.well-known/security.txt`, the vendor's security or status page, and an NVD keyword search for the product name. For dependency advisories, list the packages the **shipped artifact** contains (for Electron, the `package.json` files inside `resources/app.asar`), not the lockfile, which includes development dependencies. Check each hit against OSV and confirm the shipped version is inside the advisory's affected range; OSV has false positives. +- **Vendor material** — license or terms, pricing, privacy policy, telemetry statement, and the Redis docs page. **Read pricing and feature matrices from the rendered page** (the Playwright browser tools) or from edition gating in the source, never from a WebFetch summary. +- **The domain's registration date** (`whois `), as a cross-check on project age. +- **Distribution listings** — official Homebrew, winget-pkgs, Flathub, the app stores, distro repositories, and the AUR. + +**Signature inspection without execution is allowed.** You may download an installer to the scratch path and inspect it without running it: + +- **macOS:** mount the `.dmg` with `hdiutil attach -nobrowse -readonly`, then run `codesign -dv --verbose=4`, `codesign --verify --deep --strict`, and `xcrun stapler validate` on the `.app`. `spctl -a -vv` is also useful, but on a machine with Gatekeeper disabled it reports `override=security disabled`; the `codesign` and `stapler` results are the evidence. +- **Windows:** `osslsigncode verify` on the installer. If no Authenticode verifier is available, the Windows check is blind — do not improvise one. +- **Electron updater config:** extract `resources/app-update.yml` from the installer. C2 needs it. + +Detach mounted images and delete the downloads afterwards. Never launch the app. + +**Do not install or run the tool without asking first.** A hands-on run is a legitimate way to verify rows that source cannot (section 3), but it means executing untrusted code with network access. If the user agrees, run it against a throwaway Redis on a spare port holding no real data — never against 6379 or any server with credentials worth stealing. + +### Verification traps + +- **Commit dates are set by the author, not by GitHub.** History can be imported or backdated. Date the project from server-side timestamps: repo `created_at`, the first release's `published_at`. The domain registration counts only with evidence that it hosted this product at that date (an archive capture) — domains are bought and reused. Use an earlier date only on positive evidence of an earlier public life, and cite it. +- **One person can appear as several accounts.** An org or bot account committing alongside its owner is still one maintainer. Merge accounts that share a name, email, or profile before counting committers. Exclude dependency bots and AI co-author accounts (`claude`, `copilot`, `*[bot]`) from every count, and commits whose author name is an AI agent even when they sit under a human login. +- **Release download counts include non-installer assets.** Updater metadata (`latest*.yml`, `latest.json`), `.blockmap` files, Electron's macOS `*-mac.zip` updater payloads, Tauri's `.app.tar.gz`, checksums, signatures, and on-demand plugin or driver bundles are fetched by the app itself, often far more than the installers. Count installer assets only (`.dmg`, `.exe`, `.msi`, `.deb`, `.rpm`, `.AppImage`, `.flatpak`, `.snap`, `.pacman`), and leave out releases that exist only to host bundles. +- **GitHub code search fails silently.** New repositories are not indexed yet, so a search returning nothing is no evidence of absence. Walk the git tree instead. +- **WebFetch summaries are unreliable for exact facts.** A summarizing fetch has returned two different wrong answers for the same pricing table. For pricing, feature matrices, and any vendor sentence a score rests on, read the rendered page or quote the raw text, and re-check before scoring. +- **Website analytics are not app telemetry.** A site that runs Google Analytics says nothing about whether the app phones home. Score C4 from the app's source, docs, or privacy policy. + +### Evidence strength + +From strongest to weakest: **source code** and **inspection of the shipped artifact** — where these two conflict, the artifact wins, because it is what users run — then **a hands-on run**, then **vendor product documentation** (docs pages, settings references, privacy policy), then **vendor marketing pages**, then **README and entry claims**. + +- A capability supported only by a marketing page, a README, or the entry text is unverified: it can reach Warn at most. +- A vendor statement of a *limitation* (an admission against interest, such as "SSH tunneling is not available yet") counts as verified absence, wherever it appears — including the entry text. +- **Where sources conflict**, cite both, score from the stronger source, and if they are equally strong score the lower band and say the evidence conflicts. A hard stop resting on conflicting or vendor-docs-only evidence is unconfirmed (section 4). + +## 3. Score against the checklist + +Score each criterion Pass (2) / Warn (1) / Fail (0). Every criterion's evidence must cite a URL and a concrete fact. + +**Each criterion's three bands must be mutually exclusive and exhaustive** — numeric ones as half-open intervals, qualitative ones as cases. If a plausible tool matches two bands, or none, the row is broken rather than the reviewer's judgement: fix the row. + +**Where a band offers alternative routes ("X **or** Y"), every lower band must explicitly exclude those routes**, or the tie-break below silently kills the alternative. + +**If more than one band still applies, score the lowest one** and say in the evidence cell that the row needs repair. Prefer restructuring a row around the single axis it measures over appending another "or" clause. + +**A Fail requires positive evidence of the deficiency.** Where the figure does not exist, or the only instrument available is blind (closed source, a feature you could not exercise), score **Warn**, mark the evidence cell **(blind)**, and name the gap — not Fail, and not Pass. This rule takes precedence over every band below. Blind Warns are counted in section 4, so that a tool which documents nothing cannot score like one measured as mediocre. + +**Closed source and source-reading rows.** Rows that read source code (C2–C4, D2, D4) can instead be satisfied by vendor product documentation that states the behavior explicitly, or by a hands-on run. Say which instrument you used. A3, C1, and D1 are excluded on purpose: their own bands say what closed-source evidence can reach. + +**Checklist rows with unverified items** (D2, D4): count only verified items. If resolving the unverified items could change the band, score Warn. + +**Time windows.** "The last 12 months" means the project's whole life when it is younger than that; say so in the evidence cell. + +### A. Project maturity and maintenance + +| # | Criterion | Pass (2) | Warn (1) | Fail (0) | Where to look | +|---|-----------|----------|----------|----------|---------------| +| A1 | Project age | ≥ 12 months since the earliest public evidence of the project | ≥ 6 and < 12 months | < 6 months | Repo `created_at`, first release `published_at`, an archived capture of the product site — per the traps | +| A2 | Activity | Not archived, and a release in the last 6 months | Not archived, no release in the last 6 months, and a release or commit within the last 18 months — **or** no release history is observable (closed source with no changelog or dated releases, blind) | Archived, or an observable release and commit history shows nothing within the last 18 months | `…/releases`, `…/commits`, vendor changelog for closed source. A burst of releases in a tool's first weeks is recency, not cadence — A1 already covers the missing history | +| A3 | Bus factor | ≥ 2 distinct people each authored ≥ 10% of commits in the last 12 months | Commits exist in the last 12 months but fewer than 2 people reach 10%, **or** the team is not observable (closed source, blind) | A public repository with no commits in the last 12 months | Commit author names, after merging accounts and excluding bots, per the traps. A vendor-named team does not reach Pass, since it cannot be checked | +| A4 | Responsiveness | ≥ 80% of a sample of ≥ 5 items got a maintainer response within 14 days | ≥ 30% and < 80% of a sample of ≥ 5 items, **or** a public inbound channel exists but fewer than 5 items qualify (give the count), **or** the only channel is private (support email, blind) | < 30% of a sample of ≥ 5 items, or no inbound channel at all, public or private | The sample: the 20 most recent issues and PRs opened by people outside the project (not maintainers, not bots), each at least 14 days old. Count only channels the project offers — a project with no PR channel is scored on issues alone. A **response** is a comment, review, close, or merge by a maintainer; a label alone is not. **Maintainers** are A3's ≥ 10% committers plus any account seen merging or closing others' items | + +### B. Adoption + +| # | Criterion | Pass (2) | Warn (1) | Fail (0) | Where to look | +|---|-----------|----------|----------|----------|---------------| +| B1 | Independent usage | ≥ 2 of the 4 signals below are met | Exactly 1 is met, **or** none is met and at least one could not be measured (blind) | All 4 were measured and none is met | Repo front page, issues, release assets, distribution listings | +| B2 | Distribution | Accepted into a channel that reviews submissions: official Homebrew (`homebrew/core` or `homebrew/cask`), winget-pkgs, Flathub, the Mac App Store, the Microsoft Store, or a distribution's official repositories | Installable, but not accepted into any reviewed channel: the maintainer's own Homebrew tap, the AUR, a website download, or a submission still pending | No installable distribution — build from source only | `brew info`, the winget-pkgs and Flathub repos, store listings, the README's install section. Only merged or published listings count | + +**B1 signals:** + +1. ≥ 200 GitHub stars (all-time). +2. ≥ 10 distinct accounts outside the project opening issues or PRs in the last 12 months. +3. ≥ 10,000 installer-asset downloads across releases in the last 12 months (installers only, per the traps). Self-hosted binaries with no published counts are blind. +4. Packaged by an independent third party — someone outside the project maintains it in a distribution channel (for example an AUR package or a distro package). + +**Outside the project** means not a maintainer (A4's definition), not a member of the project's GitHub organization, and showing no affiliation with the project or vendor (profile company, vendor email domain, or signing agreements or releases on the vendor's behalf). Count only accounts you can establish as outside; say how many you excluded as affiliated. + +State each number you found and whether it met the signal. These thresholds are a starting point; a reviewer may override one with a stated alternative. A1 and B1 both hold back a brand-new tool, and that is deliberate: A1 measures elapsed time and B1 measures uptake, and a young tool with real uptake still passes B1. Stars can be bought — a star history with a single spike is worth a reviewer note, but is not scored. + +### C. Trust and safety + +| # | Criterion | Pass (2) | Warn (1) | Fail (0) | Where to look | +|---|-----------|----------|----------|----------|---------------| +| C1 | Binary provenance | A signature check shows the current release is signed on every OS it ships desktop installers for where signing is the norm (macOS: notarized Developer ID, or Mac App Store distribution; Windows: valid Authenticode), by the person or organization that controls the project | Not Pass, but either signed on some such OS, **or** built by a public CI workflow from tagged source, **or** signing was not checked (blind) | Inspection shows the release unsigned on every such OS, and there is no CI-build trace to source | Your own inspection of a downloaded installer (section 2), or the release CI run's verification log for the same release. A workflow that *configures* signing, or a CI log that says signing or notarization succeeded without verifying the result, is not a signature check. A person's certificate (for example a Developer ID in the founder's name) counts when that person owns or maintains the project. An unverifiable Windows signature is blind. Linux has no platform-signing norm and is excluded. For a tool with no macOS or Windows installer, see the note after this table | +| C2 | Runtime code fetch | Every code artifact the app fetches by default (updates, drivers, engines) is verified against a signature, a hash pinned in the installed app, or a hash from an index signed with a pinned key — or the app fetches none. Updates verified against the app's code signature count: Squirrel.Mac on macOS, and electron-updater on Windows only when the installer's `resources/app-update.yml` names a `publisherName` — without it, electron-updater checks only the unsigned hash in `latest.yml` | Neither Pass nor Fail: every fetch path not verified as Pass is verified only against an unsigned hash from the same location, **or** only when a hash happens to be present, **or** its verification cannot be determined (blind) | Any default fetch path is verified to fetch and run code with no verification at all — this decides the row whatever the other paths show. A path that verifies whenever a hash is present is Warn, not Fail | Updater metadata, driver and plugin loaders, docs. Name what is fetched. User-installed third-party extensions are recorded but not scored. If the Redis support itself is a plugin or extension, its install and update path counts as a default fetch path and is scored in the bands like any other | +| C3 | Credential storage | Saved secrets, or the key that encrypts them, are held in the OS credential store (Keychain, Credential Manager, Secret Service) or an external secret manager by default — or secrets are not saved by default | Neither Pass nor Fail — for example, secrets encrypted under a key derived from a user-supplied master password, or a storage path that cannot be determined (blind) | Any default storage path is verified recoverable by anyone who can read the user's files — this decides the row whatever the other paths show: plaintext by default (including encryption the user must opt in to), or a key stored beside the data or derived from a constant in the app or its source | Connection-storage and key-loading code, docs, a config file written during a test run. Record whether "remember password" is on by default | +| C4 | Telemetry and data egress | No telemetry, or documented opt-in only; **and** every path that sends Redis data, key names, or schemas off the machine (AI assistant, cloud sync, built-in MCP server) is opt-in and documented; **and** no service listens beyond loopback by default | Neither Pass nor Fail — for example, documented opt-out telemetry, an opt-in data path that is undocumented, or behavior that cannot be determined (blind) | Telemetry that is undisclosed or cannot be disabled; Redis data sent off the machine by default; or a service listening beyond loopback by default | Analytics or crash-reporting dependencies in the source, privacy policy, settings docs, AI provider config. **Opt-in** means an explicit setting that is off by default, or a per-use action that names the destination | +| C5 | License and terms | An OSI-approved license covering the Redis support, **or** published proprietary or source-available terms with a free tier that includes the Redis support | Published proprietary or source-available terms, but the Redis support needs a paid tier or a time-limited trial, **or** a closed-source tool whose terms could not be found (blind) | A public repository with no license — no license file, and GitHub's license API returns none — and no terms linked from the repository or the project website | Repo license, EULA, pricing page (rendered, per section 2). For open-core projects, check that the Redis code is outside the proprietary directories and not edition-gated | +| C6 | Security | A security-specific reporting channel exists (SECURITY.md, `security.txt`, or a security contact or page), **and** the current release ships no advisory of the kinds in the Fail band, **and** no other shipped dependency advisory | No advisory of the Fail kinds, but either no security-specific channel, **or** a shipped dependency advisory that is not of the Fail kinds — any severity with no shown path, or medium or low with one — **or** advisory status cannot be established (blind) | The current release is affected by an advisory against the tool itself, or ships a critical or high dependency advisory with a shown path from the tool's own code | `…/security-advisories` (published advisories only), the vendor's security or scanner page, NVD, and the shipped-artifact package list checked against OSV (section 2). "Affected" means the shipped version is in the advisory's range, whether or not upstream has a fix. A **shown path** is a call from the tool's own code to the affected function, found in source, or a vendor or upstream statement that the tool is affected. OSV and NVD establish the affected version, not reachability, so without that evidence an advisory has no shown path. A large Electron or Node app nearly always ships transitive highs; without a shown path they are Warn, not Fail. Patched historical CVEs are a reviewer note, not a score | + +**C1 for tools with no macOS or Windows installer** (web apps, container images, CLIs, Linux-only packages): score provenance of what the tool does ship. Pass: the published artifacts carry a verifiable signature or provenance attestation tying them to the project (for example cosign-signed images, npm provenance, or a SLSA attestation). Warn: not Pass, but built by a public CI workflow from tagged source, **or** not checked (blind). Fail: inspection shows no signature or attestation, and there is no CI-build trace to source. + +### D. Redis fit + +Score these on how the tool handles Redis, not how many databases it supports. Apart from D1, these rows do not affect the verdict: they decide which gaps the entry must state (section 4). + +| # | Criterion | Pass (2) | Warn (1) | Fail (0) | Where to look | +|---|-----------|----------|----------|----------|---------------| +| D1 | Depth of Redis support | Not Fail, and either tests that CI runs against a real Redis server **or** Redis-specific behavior demonstrated across ≥ 4 data types in a hands-on run | Not Fail, and neither Pass route holds: no tests that CI runs against a real Redis server (none, mocks or fakeredis only, or not run — for example `#[ignore]` with no Redis service in CI) and no run covering ≥ 4 types — **or** closed source with only vendor docs (blind) | Redis is handled by a generic key-value adapter with no Redis-specific behavior. This takes precedence: a generic adapter that displays several types in a run is still Fail | The Redis driver/adapter directory, its tests, and the CI workflow that runs them | +| D2 | Production safety | Key discovery is verified not to use `KEYS`, **and** ≥ 4 of the 5 behaviors below are verified present | Neither Pass nor Fail | Key discovery uses `KEYS`, **or** ≥ 3 of the 5 verified absent | Key-browser, value-loading, and command-dispatch code, and the console UI | +| D3 | Topology | Both Cluster and Sentinel are supported | Exactly one is supported, **or** support cannot be determined (blind) | Verified standalone-only: neither is supported | Connection code and forms, docs. A vendor statement that a topology is not supported counts as verified absence | +| D4 | Connection and type coverage | ≥ 4 of the 5 items below verified supported | Neither Pass nor Fail | At most 1 item could be supported: verified-supported items plus unverified items total ≤ 1 | Connection code and forms, value views, docs | +| D5 | Maturity of the Redis support | In a stable public release for ≥ 12 months, and not labeled beta, preview, or experimental | In a stable public release for ≥ 6 months, and either < 12 months or still labeled beta, preview, or experimental — **or** the release history needed to date it is not observable (blind) | Verified not to be in a stable public release, or verified to have been in one for < 6 months | The first stable (not beta or pre-release) release whose changelog or source includes the Redis support; labels in the UI source, the connection form, and the docs. Record who wrote the Redis code in the evidence cell — a single author of a mature project's Redis driver is worth a reviewer note | + +**D2 behaviors:** + +1. **Bounded key discovery** — `SCAN` page by page, or with an explicit cap. Walking the whole keyspace on every view does not count, even with `SCAN`. +2. **Bounded value loading** — values of every type loaded in pages or with a size cap; whole-value `HGETALL`, `GET`, or `LRANGE 0 -1` on any type means absent. +3. **Confirmation before destructive commands** (`FLUSHDB`, `FLUSHALL`, bulk deletes), including when typed into the tool's console. Staged edits that must be applied are confirmation for edits, not for the console. The console path alone settles it: if a typed `FLUSHALL` reaches the server unconfirmed, the behavior is absent. +4. **A default-deny read-only mode** — when enabled, commands the tool does not recognize as read-only are blocked, not passed through. +5. **`MONITOR` only on explicit user action.** A tool that never issues `MONITOR` has this behavior. + +**D4 items:** TLS; TLS client certificates; ACL username-and-password authentication; a stream view (type-aware display of stream entries with their IDs — a console that can send `XRANGE` does not count); a JSON view (type-aware display of JSON keys — a console that can send `JSON.GET` does not count). + +### E. The entry + +Score these against the entry text from section 1. Both must reach Pass before merge (section 4). + +| # | Criterion | Pass (2) | Warn (1) | Fail (0) | Where to look | +|---|-----------|----------|----------|----------|---------------| +| E1 | Claims match the evidence | No claim is contradicted, every capability and limitation claim is verified, **and** the entry states the verified Redis gaps found in D2–D5 | No claim is contradicted, but some claims could not be verified **or** known gaps are omitted | At least one claim is contradicted by the evidence | The entry, checked line by line against sections C and D | +| E2 | Fits the page | Has all four elements of the established pattern and needs no edits | Exactly one of the four elements is missing, **or** all four are present but edits are needed: marketing words, commands not named in the capability list, length over 1.5× the longest neighboring entry | Two or more of the four elements are missing | The other entries in the Third-party tools section | + +**Reading claims for E1.** Read each claim as a typical reader would. Pricing and availability are claims: a feature listed under an unqualified "free" or "open-source" opening that needs a paid tier, or a download link labeled "free" for a tool whose Redis support is paid, is a contradiction, not an omission. A missing gap statement is scored here only, not again in E2. + +**The established pattern** for an entry has four elements: an opening paragraph naming the platforms and the license or pricing; a bulleted capability list that names the Redis commands behind each feature (`SCAN`, `SLOWLOG GET`, `CLIENT LIST`); a connection paragraph covering topologies, TLS, authentication, and what is *not* supported; and a download link. Apply the repo's style rules (`AGENTS.md`) — no "seamless", "powerful", "simply", "easily". + +## 4. Total and verdict + +The Third-party tools section is a directory: it lists tools that work with Redis and says plainly what each one does not do. So the scores split into three jobs: + +- **Sections A–C decide the listing** — is the tool alive, adopted, and safe to point readers at? 12 criteria × 2 points = **24 max**. +- **Section D decides what the entry must say.** Thin Redis support is not a reason to decline; an entry that hides it is. Every *verified* gap behind a D2–D5 score below Pass is one the entry must state, which is what E1's Pass condition checks. A blind Warn is not a gap: the entry must neither claim nor deny the unverified capability, and the reviewer notes list it as unverified. D1 is the exception — see the gates. +- **Section E decides whether the entry is ready to merge.** E1 and E2 must both reach Pass before merge, whatever the verdict. The edits from section 5 are a merge condition, not a reason to decline. + +| Verdict | A–C total (of 24) | +|---------|-------------------| +| **Recommend include** | ≥ 17 | +| **Needs discussion** (list the specific Warn/Fail items to resolve) | 12–16 | +| **Decline** (cite the failing criteria in the PR response) | < 12 | + +Record the full 38-point total too. It is not part of the verdict, but it makes reviews comparable over time. + +**Gates, regardless of the A–C total:** + +- **Too young:** A1 Fail (under 6 months) → **Decline (not yet)**. Give the date the tool turns 6 months old, so the reply can invite a resubmission then. +- **No real Redis support:** D1 Fail (a generic key-value adapter) → **Decline**. There is nothing Redis-specific to list. +- **Too little evidence:** more than 3 blind Warns in sections A–C → at most **Needs discussion**, and the discussion item is the missing evidence. +- **Hard stops:** archived repo; a public repository with no license and no terms (C5 Fail); an advisory of the kinds in C6's Fail band; unverified runtime code fetch (C2 Fail) → **Decline**. Check these first. If one applies, still complete the scorecard for the record. + +**A hard stop must be confirmed.** If it rests only on vendor documentation or marketing, or on sources that conflict, the verdict is **Needs discussion (unconfirmed hard stop)** until source code, inspection of the shipped artifact, a hands-on run, or the vendor confirms it. Confirmed, the verdict is Decline no matter the total. + +## 5. Output format + +Produce, in this order: + +1. A header naming the tool, its repo URL (or publisher, if closed source), its website, the version observed, the review date, and whether this is a new entry or a re-vet. +2. A scorecard table with columns: criterion | score | evidence. Every evidence cell must contain at least one URL and the concrete fact observed. Name the instrument for rows scored on docs or a test run instead of source, and mark blind Warns **(blind)**. +3. The A–C total (of 24), the full total (of 38), the count of blind Warns in A–C, and the verdict per section 4, noting explicitly whether any gate or hard stop applies and whether a hard stop is confirmed. +4. A 2–3 sentence summary of the standout strengths and weaknesses. +5. If E1 or E2 scored below Pass, the specific edits the entry needs before merge, as replacement text where you can — including a sentence for every verified D2–D5 gap. Never write an unverified limitation into the entry; drop the unverified claim instead. +6. **Reviewer notes**, kept separate: the submitter's affiliation, star-history anomalies, patched historical CVEs, and anything else that is not a criterion. These carry no score. Keep them out of anything published — in a public reply they read as a judgement on the author rather than on the tool. + +## 6. Record the verdict + +Do not save the scorecard as a file in this repo: most of its facts (download counts, "released last week", the total itself) are false within months. A dated snapshot belongs somewhere understood to be a snapshot. + +**On an accept**, put it in the squash commit message of the PR that adds the entry, so `git blame` on the entry leads to the evidence. Write it as a snapshot: name the review date and the version observed, keep the non-Pass criteria in full with their evidence, and compress the passes to a single line. Anchor it with the same trailers as client vetting, so both sets stay enumerable with one query (`git log --grep='^Vetted:'`): + +``` +Vetted: /24 (/38) +Gaps: +Recheck: +``` + +This repo squashes with `squash_merge_commit_message = COMMIT_MESSAGES`, so the default body is the contributor's commits concatenated and the scorecard is silently discarded. Override it explicitly: + +``` +gh pr merge --squash --subject "" --body-file +``` + +**On a decline**, the PR reply comment is the record. Lead with the criteria the tool can fix by waiting or working — age, adoption, signing — and say what would change the verdict, since a young tool that fails A1 today may pass in six months. + +**On a re-vet of a live entry**, the outcome is one of two things, and neither is removal by default: + +- **Entry corrections** (E1 or E2 below Pass): propose them as a docs PR, and put the scorecard in that PR's squash commit message with the trailers above. +- **Needs discussion or Decline**: report it to the docs owner with the scorecard. Removing a listed tool is the docs team's decision, and usually starts with asking the vendor about the failing criteria. From 40ce405ea0739ac6a031dfda63d279092541a163 Mon Sep 17 00:00:00 2001 From: mich-elle-luna Date: Fri, 23 Oct 2026 12:10:02 -0700 Subject: [PATCH 69/76] Fix context-map diagram links missing the /docs/latest/ prefix The render hook set data-base-path with {{ "/" | relURL }}, but relURL returns a leading-slash path unchanged, so the base was always "/" and node links on the Iris request-flow page resolved to /develop/... instead of /docs/latest/develop/.... Passing "" gives the real subpath. Co-Authored-By: Claude Opus 5.5 (1M context) --- layouts/_default/_markup/render-codeblock-context-map.html | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/layouts/_default/_markup/render-codeblock-context-map.html b/layouts/_default/_markup/render-codeblock-context-map.html index a37ca70838..65c08eabfd 100644 --- a/layouts/_default/_markup/render-codeblock-context-map.html +++ b/layouts/_default/_markup/render-codeblock-context-map.html @@ -9,7 +9,7 @@ {{- /* basePath lets the JS resolve the site-relative docsUrl/link.url values authored in the YAML against wherever this build is actually deployed (e.g. a /docs/latest/ subpath), since those values are static strings the JS parses, not relref-resolved paths. */ -}} - 
    {{ .Inner | htmlEscape | safeHTML }}
    + 
    {{ .Inner | htmlEscape | safeHTML }}
    {{- /* Extract id and scope from YAML for metadata (only top-level fields) */ -}} {{- $lines := split .Inner "\n" -}} From ecac70b506a6abbe472f19348fc278ef38ba0b51 Mon Sep 17 00:00:00 2001 From: David Dougherty Date: Fri, 23 Oct 2026 12:42:54 -0700 Subject: [PATCH 70/76] DEV: (cmds) correct PING return info (#4340) --- content/commands/ping.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/content/commands/ping.md b/content/commands/ping.md index 5ee1da30c7..e2201474b0 100644 --- a/content/commands/ping.md +++ b/content/commands/ping.md @@ -35,7 +35,7 @@ syntax_fmt: PING [message] title: PING --- -This command returns the bulk string `PONG` if no argument is provided, otherwise it returns a copy of the +This command returns the simple string `PONG` if no argument is provided, otherwise it returns a copy of the provided `message` as a bulk string. This command is useful for: @@ -43,10 +43,10 @@ This command is useful for: 1. Verifying the server's ability to serve data - an error is returned when this isn't the case (for example, during load from persistence data or accessing a stale replica). 1. Measuring latency. -If the client is subscribed to a channel or a pattern, it will instead return a -multi-bulk string with `PONG` in the first position and an empty bulk string in the second -position, unless an argument is provided, in which case it returns a copy -of the argument. +If a RESP2 client is subscribed to a channel, pattern, or shard channel, PING returns a +two-element array instead. The first element is the bulk string `pong` (lowercase). The +second is a copy of `message` if you provide one, or an empty bulk string if you don't. +RESP3 clients get the standard reply whether or not they're subscribed. ## Optional arguments @@ -80,6 +80,7 @@ redis> PING "hello world" Any of the following: * [Simple string reply](../../develop/reference/protocol-spec#simple-strings): `PONG` when no argument is provided. * [Bulk string reply](../../develop/reference/protocol-spec#bulk-strings): the provided argument. +* [Array reply](../../develop/reference/protocol-spec#arrays): "`pong`" and either the provided argument or an empty bulk string, when the client is in subscribed mode. -tab-sep- From 4fe762f8dab40f0b661207374378ed52113c9fe3 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Sat, 24 Oct 2026 17:54:53 -0700 Subject: [PATCH 71/76] Update content/operate/rc/api/api-reference/openapi.json (#4344) --- .../operate/rc/api/api-reference/openapi.json | 40 +++++++++---------- 1 file changed, 20 insertions(+), 20 deletions(-) diff --git a/content/operate/rc/api/api-reference/openapi.json b/content/operate/rc/api/api-reference/openapi.json index a6088bc486..185075da7b 100644 --- a/content/operate/rc/api/api-reference/openapi.json +++ b/content/operate/rc/api/api-reference/openapi.json @@ -8350,9 +8350,6 @@ "412": { "description": "Precondition Failed - Feature flag for this flow is off" }, - "403": { - "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" - }, "424": { "description": "Failed Dependency - The memory store's database is unavailable or out of memory, e.g. /errors/resource-unavailable or /errors/database-out-of-memory. When the error originates from the Agent Memory service the body is an application/problem+json document (MemoryProblemDetails) carrying the service's error type and detail", "content": { @@ -8363,6 +8360,9 @@ } } }, + "403": { + "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" + }, "200": { "description": "Memory stores returned.", "content": { @@ -8490,9 +8490,6 @@ "412": { "description": "Precondition Failed - Feature flag for this flow is off" }, - "403": { - "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" - }, "424": { "description": "Failed Dependency - The memory store's database is unavailable or out of memory, e.g. /errors/resource-unavailable or /errors/database-out-of-memory. When the error originates from the Agent Memory service the body is an application/problem+json document (MemoryProblemDetails) carrying the service's error type and detail", "content": { @@ -8503,6 +8500,9 @@ } } }, + "403": { + "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" + }, "401": { "description": "Unauthorized - Authentication failed for requested resource" }, @@ -8565,9 +8565,6 @@ "412": { "description": "Precondition Failed - Feature flag for this flow is off" }, - "403": { - "description": "Forbidden - Not allowed to create requested resource (primarily due to not active resource)" - }, "424": { "description": "Failed Dependency - The memory store's database is unavailable or out of memory, e.g. /errors/resource-unavailable or /errors/database-out-of-memory. When the error originates from the Agent Memory service the body is an application/problem+json document (MemoryProblemDetails) carrying the service's error type and detail", "content": { @@ -8578,6 +8575,9 @@ } } }, + "403": { + "description": "Forbidden - Not allowed to create requested resource (primarily due to not active resource)" + }, "201": { "description": "API key created. The generated key value is returned once in the response body.", "content": { @@ -10079,12 +10079,12 @@ "412": { "description": "Precondition Failed - Feature flag for this flow is off" }, - "200": { - "description": "Memory store returned.", + "424": { + "description": "Failed Dependency - The memory store's database is unavailable or out of memory, e.g. /errors/resource-unavailable or /errors/database-out-of-memory. When the error originates from the Agent Memory service the body is an application/problem+json document (MemoryProblemDetails) carrying the service's error type and detail", "content": { - "application/json": { + "application/problem+json": { "schema": { - "$ref": "#/components/schemas/MemoryStoreResponse" + "$ref": "#/components/schemas/MemoryProblemDetails" } } } @@ -10092,12 +10092,12 @@ "403": { "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" }, - "424": { - "description": "Failed Dependency - The memory store's database is unavailable or out of memory, e.g. /errors/resource-unavailable or /errors/database-out-of-memory. When the error originates from the Agent Memory service the body is an application/problem+json document (MemoryProblemDetails) carrying the service's error type and detail", + "200": { + "description": "Memory store returned.", "content": { - "application/problem+json": { + "application/json": { "schema": { - "$ref": "#/components/schemas/MemoryProblemDetails" + "$ref": "#/components/schemas/MemoryStoreResponse" } } } @@ -12804,9 +12804,6 @@ "412": { "description": "Precondition Failed - Feature flag for this flow is off" }, - "403": { - "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" - }, "424": { "description": "Failed Dependency - The memory store's database is unavailable or out of memory, e.g. /errors/resource-unavailable or /errors/database-out-of-memory. When the error originates from the Agent Memory service the body is an application/problem+json document (MemoryProblemDetails) carrying the service's error type and detail", "content": { @@ -12817,6 +12814,9 @@ } } }, + "403": { + "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" + }, "204": { "description": "API key deleted." }, From fcc15c76a48baeb0dc08fc839242a7799c077080 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Sun, 25 Oct 2026 17:38:26 -0700 Subject: [PATCH 72/76] Update content/operate/rc/api/api-reference/openapi.json (#4345) --- .../operate/rc/api/api-reference/openapi.json | 98 +++++++++---------- 1 file changed, 49 insertions(+), 49 deletions(-) diff --git a/content/operate/rc/api/api-reference/openapi.json b/content/operate/rc/api/api-reference/openapi.json index 185075da7b..5ef3e02827 100644 --- a/content/operate/rc/api/api-reference/openapi.json +++ b/content/operate/rc/api/api-reference/openapi.json @@ -96,7 +96,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_4", + "operationId": "proxyDataIntegrationWorkspace_3", "parameters": [ { "name": "subscriptionId", @@ -166,7 +166,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace", + "operationId": "proxyDataIntegrationWorkspace_1", "parameters": [ { "name": "subscriptionId", @@ -237,7 +237,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_1", + "operationId": "proxyDataIntegrationWorkspace_4", "parameters": [ { "name": "subscriptionId", @@ -308,7 +308,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_3", + "operationId": "proxyDataIntegrationWorkspace_2", "parameters": [ { "name": "subscriptionId", @@ -379,7 +379,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_2", + "operationId": "proxyDataIntegrationWorkspace", "parameters": [ { "name": "subscriptionId", @@ -452,7 +452,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_9", + "operationId": "proxyDataIntegrationWorkspace_8", "parameters": [ { "name": "subscriptionId", @@ -522,7 +522,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_5", + "operationId": "proxyDataIntegrationWorkspace_6", "parameters": [ { "name": "subscriptionId", @@ -593,7 +593,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_6", + "operationId": "proxyDataIntegrationWorkspace_9", "parameters": [ { "name": "subscriptionId", @@ -664,7 +664,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_8", + "operationId": "proxyDataIntegrationWorkspace_7", "parameters": [ { "name": "subscriptionId", @@ -735,7 +735,7 @@ ], "summary": "Proxy a Data Integration workspace request", "description": "Forwards the request to the Data Integration workspace owned by the specified subscription. Path under `data-integration-workspace/**` is forwarded as-is to the Cloud RDI service.", - "operationId": "proxyDataIntegrationWorkspace_7", + "operationId": "proxyDataIntegrationWorkspace_5", "parameters": [ { "name": "subscriptionId", @@ -8350,6 +8350,12 @@ "412": { "description": "Precondition Failed - Feature flag for this flow is off" }, + "404": { + "description": "Not Found - The resource you were trying to reach was not found or does not exist" + }, + "403": { + "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" + }, "424": { "description": "Failed Dependency - The memory store's database is unavailable or out of memory, e.g. /errors/resource-unavailable or /errors/database-out-of-memory. When the error originates from the Agent Memory service the body is an application/problem+json document (MemoryProblemDetails) carrying the service's error type and detail", "content": { @@ -8360,8 +8366,8 @@ } } }, - "403": { - "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" + "401": { + "description": "Unauthorized - Authentication failed for requested resource" }, "200": { "description": "Memory stores returned.", @@ -8373,12 +8379,6 @@ } } }, - "401": { - "description": "Unauthorized - Authentication failed for requested resource" - }, - "404": { - "description": "Not Found - The resource you were trying to reach was not found or does not exist" - }, "408": { "description": "Request Timeout - The server didn't receive a complete request message within the server's allotted timeout period" }, @@ -8490,6 +8490,12 @@ "412": { "description": "Precondition Failed - Feature flag for this flow is off" }, + "404": { + "description": "Not Found - The resource you were trying to reach was not found or does not exist" + }, + "403": { + "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" + }, "424": { "description": "Failed Dependency - The memory store's database is unavailable or out of memory, e.g. /errors/resource-unavailable or /errors/database-out-of-memory. When the error originates from the Agent Memory service the body is an application/problem+json document (MemoryProblemDetails) carrying the service's error type and detail", "content": { @@ -8500,15 +8506,9 @@ } } }, - "403": { - "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" - }, "401": { "description": "Unauthorized - Authentication failed for requested resource" }, - "404": { - "description": "Not Found - The resource you were trying to reach was not found or does not exist" - }, "408": { "description": "Request Timeout - The server didn't receive a complete request message within the server's allotted timeout period" }, @@ -8565,6 +8565,12 @@ "412": { "description": "Precondition Failed - Feature flag for this flow is off" }, + "404": { + "description": "Not Found - The resource you were trying to reach was not found or does not exist" + }, + "403": { + "description": "Forbidden - Not allowed to create requested resource (primarily due to not active resource)" + }, "424": { "description": "Failed Dependency - The memory store's database is unavailable or out of memory, e.g. /errors/resource-unavailable or /errors/database-out-of-memory. When the error originates from the Agent Memory service the body is an application/problem+json document (MemoryProblemDetails) carrying the service's error type and detail", "content": { @@ -8575,9 +8581,6 @@ } } }, - "403": { - "description": "Forbidden - Not allowed to create requested resource (primarily due to not active resource)" - }, "201": { "description": "API key created. The generated key value is returned once in the response body.", "content": { @@ -8591,9 +8594,6 @@ "401": { "description": "Unauthorized - Authentication failed for requested resource" }, - "404": { - "description": "Not Found - The resource you were trying to reach was not found or does not exist" - }, "408": { "description": "Request Timeout - The server didn't receive a complete request message within the server's allotted timeout period" }, @@ -10079,12 +10079,15 @@ "412": { "description": "Precondition Failed - Feature flag for this flow is off" }, - "424": { - "description": "Failed Dependency - The memory store's database is unavailable or out of memory, e.g. /errors/resource-unavailable or /errors/database-out-of-memory. When the error originates from the Agent Memory service the body is an application/problem+json document (MemoryProblemDetails) carrying the service's error type and detail", + "404": { + "description": "Not Found - The resource you were trying to reach was not found or does not exist" + }, + "200": { + "description": "Memory store returned.", "content": { - "application/problem+json": { + "application/json": { "schema": { - "$ref": "#/components/schemas/MemoryProblemDetails" + "$ref": "#/components/schemas/MemoryStoreResponse" } } } @@ -10092,12 +10095,12 @@ "403": { "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" }, - "200": { - "description": "Memory store returned.", + "424": { + "description": "Failed Dependency - The memory store's database is unavailable or out of memory, e.g. /errors/resource-unavailable or /errors/database-out-of-memory. When the error originates from the Agent Memory service the body is an application/problem+json document (MemoryProblemDetails) carrying the service's error type and detail", "content": { - "application/json": { + "application/problem+json": { "schema": { - "$ref": "#/components/schemas/MemoryStoreResponse" + "$ref": "#/components/schemas/MemoryProblemDetails" } } } @@ -10105,9 +10108,6 @@ "401": { "description": "Unauthorized - Authentication failed for requested resource" }, - "404": { - "description": "Not Found - The resource you were trying to reach was not found or does not exist" - }, "429": { "description": "Too Many Requests - Too many resources are concurrently created / updated / deleted in the account. Please re-submit API requests after resource changes are completed)" }, @@ -12804,6 +12804,12 @@ "412": { "description": "Precondition Failed - Feature flag for this flow is off" }, + "404": { + "description": "Not Found - The resource you were trying to reach was not found or does not exist" + }, + "403": { + "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" + }, "424": { "description": "Failed Dependency - The memory store's database is unavailable or out of memory, e.g. /errors/resource-unavailable or /errors/database-out-of-memory. When the error originates from the Agent Memory service the body is an application/problem+json document (MemoryProblemDetails) carrying the service's error type and detail", "content": { @@ -12814,17 +12820,11 @@ } } }, - "403": { - "description": "Forbidden - Not allowed to access requested resource (primarily due to client source IP API Key restrictions)" - }, - "204": { - "description": "API key deleted." - }, "401": { "description": "Unauthorized - Authentication failed for requested resource" }, - "404": { - "description": "Not Found - The resource you were trying to reach was not found or does not exist" + "204": { + "description": "API key deleted." }, "408": { "description": "Request Timeout - The server didn't receive a complete request message within the server's allotted timeout period" From 687c6e44252ae2e7649a6671656d1ac5cf65e57e Mon Sep 17 00:00:00 2001 From: Kaitlyn Michael <76962844+kaitlynmichael@users.noreply.github.com> Date: Mon, 26 Oct 2026 08:59:20 -0500 Subject: [PATCH 73/76] RS: Add upgrade warning for deprecated gradual sync parameters (#4275) * RS: Add upgrade warning for deprecated gradual sync parameters * max sources not included --- .../operate/rs/release-notes/rs-8-2-releases/_index.md | 10 ++++++++++ .../rs/release-notes/rs-8-2-releases/rs-8-2-0-25.md | 10 ++++++++++ 2 files changed, 20 insertions(+) diff --git a/content/operate/rs/release-notes/rs-8-2-releases/_index.md b/content/operate/rs/release-notes/rs-8-2-releases/_index.md index 0f9e9a39be..24e860ff33 100644 --- a/content/operate/rs/release-notes/rs-8-2-releases/_index.md +++ b/content/operate/rs/release-notes/rs-8-2-releases/_index.md @@ -99,6 +99,16 @@ See [Ports and port ranges used by Redis Software](/content/operate/rs/networkin - The `gradual_src_mode`, `gradual_sync_mode`, and `gradual_sync_max_shards_per_source` parameters are deprecated. Use `gradual_sync_policy` instead. You can set it with [`rladmin tune db`](/content/operate/rs/references/cli-utilities/rladmin/tune.md) or the REST API. + > [!WARNING] + > Upgrading to Redis Software version 8.2 can overwrite these settings. Contact Redis Support before you upgrade if any of them differ from these values: + > + > - `gradual_sync_mode`: `auto` or `enabled` + > - `gradual_src_mode`: `disabled` + > - `gradual_src_max_sources`: `1` + > - `gradual_sync_max_shards_per_source`: `1` + > + > To check the current values, run [`rladmin info db`](/content/operate/rs/references/cli-utilities/rladmin/info.md#info-db). + - The `old_password` field in `POST /v1/users/password` and `PUT /v1/users/password` requests is deprecated and ignored. It was optional and had no effect when adding or resetting a password, so you can remove it from these requests. `old_password` is still required for `DELETE /v1/users/password`, which uses it to identify the password to delete. - The following REST API fields, deprecated since Redis Software version 6.4.2, are removed in this release: diff --git a/content/operate/rs/release-notes/rs-8-2-releases/rs-8-2-0-25.md b/content/operate/rs/release-notes/rs-8-2-releases/rs-8-2-0-25.md index c2999632bd..9ff1d34690 100644 --- a/content/operate/rs/release-notes/rs-8-2-releases/rs-8-2-0-25.md +++ b/content/operate/rs/release-notes/rs-8-2-releases/rs-8-2-0-25.md @@ -221,6 +221,16 @@ See [Ports and port ranges used by Redis Software](/content/operate/rs/networkin - The `gradual_src_mode`, `gradual_sync_mode`, and `gradual_sync_max_shards_per_source` parameters are deprecated. Use `gradual_sync_policy` instead. You can set it with [`rladmin tune db`](/content/operate/rs/references/cli-utilities/rladmin/tune.md) or the REST API. + > [!WARNING] + > Upgrading to Redis Software version 8.2 can overwrite these settings. Contact Redis Support before you upgrade if any of them differ from these values: + > + > - `gradual_sync_mode`: `auto` or `enabled` + > - `gradual_src_mode`: `disabled` + > - `gradual_src_max_sources`: `1` + > - `gradual_sync_max_shards_per_source`: `1` + > + > To check the current values, run [`rladmin info db`](/content/operate/rs/references/cli-utilities/rladmin/info.md#info-db). + - The `old_password` field in `POST /v1/users/password` and `PUT /v1/users/password` requests is deprecated and ignored. It was optional and had no effect when adding or resetting a password, so you can remove it from these requests. `old_password` is still required for `DELETE /v1/users/password`, which uses it to identify the password to delete. - The following REST API fields, deprecated since Redis Software version 6.4.2, are removed in this release: From a37507e54758f25dcb7bcb711d8f065abcdcd78c Mon Sep 17 00:00:00 2001 From: Kaitlyn Michael <76962844+kaitlynmichael@users.noreply.github.com> Date: Mon, 26 Oct 2026 08:59:53 -0500 Subject: [PATCH 74/76] RS: gradual sync policies (#4253) * RS: gradual sync policies * RS: mark legacy gradual sync fields deprecated in the REST API bdb object Co-Authored-By: Claude Opus 5.5 (1M context) --------- Co-authored-by: Claude Opus 5.5 (1M context) --- .../references/cli-utilities/rladmin/tune.md | 23 ++++++++++++++++--- .../references/rest-api/objects/bdb/_index.md | 6 ++--- 2 files changed, 23 insertions(+), 6 deletions(-) diff --git a/content/operate/rs/references/cli-utilities/rladmin/tune.md b/content/operate/rs/references/cli-utilities/rladmin/tune.md index bbd375dca4..ae1edc2239 100644 --- a/content/operate/rs/references/cli-utilities/rladmin/tune.md +++ b/content/operate/rs/references/cli-utilities/rladmin/tune.md @@ -146,6 +146,8 @@ rladmin tune db { db: | } [ gradual_src_mode { enabled | disabled } ] [ gradual_sync_mode { enabled | disabled | auto } ] [ gradual_sync_max_shards_per_source ] + [ gradual_sync_policy { default | non-gradual | migration | replica | active-active | custom } ] + [ gradual_sync_policy_parameters { src_endpoint_limit | dst_endpoint_limit | dst_shard_limit | src_node_limit | dst_node_limit } ] [ module_name ] [ module_config_params ] [ crdt_xadd_id_uniqueness_mode { liberal | semi-strict | strict } ] [ metrics_export_all { enabled | disabled } ] @@ -174,9 +176,11 @@ rladmin tune db { db: | } | crdt_xadd_id_uniqueness_mode | `liberal`
    `semi-strict`
    `strict` | XADD's behavior in an Active-Active database, defined as liberal, semi-strict, or strict (see descriptions below) | | data_internode_encryption | `enabled`
    `disabled` | Activates or deactivates [internode encryption](/content/operate/rs/security/encryption/internode-encryption.md) for the database | | db_conns_auditing | `enabled`
    `disabled` | Activates or deactivates database [connection auditing](/content/operate/rs/security/audit-events.md) for a database | -| gradual_src_mode | `enabled`
    `disabled` | Activates or deactivates gradual sync of sources | -| gradual_sync_max_shards_per_source | integer | Number of shards per sync source that can be replicated in parallel (positive integer) | -| gradual_sync_mode | `enabled`
    `disabled`
    `auto` | Activates, deactivates, or automatically determines gradual sync of source shards | +| gradual_src_mode | `enabled`
    `disabled` | Activates or deactivates gradual sync of sources (deprecated as of Redis Software v8.2.0; use `gradual_sync_policy` instead) | +| gradual_sync_max_shards_per_source | integer | Number of shards per sync source that can be replicated in parallel (positive integer) (deprecated as of Redis Software v8.2.0; use `gradual_sync_policy` instead) | +| gradual_sync_mode | `enabled`
    `disabled`
    `auto` | Activates, deactivates, or automatically determines gradual sync of source shards (deprecated as of Redis Software v8.2.0; use `gradual_sync_policy` instead) | +| gradual_sync_policy | `default`
    `non-gradual`
    `migration`
    `replica`
    `active-active`
    `custom` | Policy for gradual (parallel) full sync in Replica Of and Active-Active databases (default: `default`). Gradual sync limits how many shards sync in parallel, so full syncs complete reliably with minimal impact on client applications. It affects only multi-node or multi-shard databases; smaller databases sync one shard at a time.
    Every policy except `custom` applies preset limits. See [Gradual sync policies](#gradual-sync-policies). | +| gradual_sync_policy_parameters | `src_endpoint_limit`
    `dst_endpoint_limit`
    `dst_shard_limit`
    `src_node_limit`
    `dst_node_limit` | Sets one limit for the `custom` gradual sync policy. You can set these limits only when `gradual_sync_policy` is `custom`; other policies reject them. Each limit is `-1` (no limit) or a positive integer; `0` is not valid.
    • `src_endpoint_limit`: maximum parallel full syncs per source endpoint
    • `dst_endpoint_limit`: maximum parallel full syncs per destination endpoint
    • `dst_shard_limit`: maximum parallel full syncs per destination shard
    • `src_node_limit`: maximum parallel full syncs per source node
    • `dst_node_limit`: maximum parallel full syncs per destination node | | master_persistence | `enabled`
    `disabled` | If enabled, persists the primary shard in addition to replica shards in a replicated and persistent database. | | max_aof_file_size | size in MB | Maximum size (in MB, if not specified) of [AoF](/content/glossary/_index.md#letter-a) file (minimum value is 10 GB) | | max_aof_load_time | time in seconds | Time limit in seconds to load a shard from an append-only file (AOF). If exceeded, an AOF rewrite is initiated to decrease future load time.
    Minimum: 2700 seconds (45 minutes)
    Default: 3600 seconds (1 hour) | @@ -212,6 +216,19 @@ rladmin tune db { db: | } | semi-strict | Allows a full ID. Partial IDs are completed with the unique database instance ID (not recommended, allows duplicate IDs). | | strict | XADD fails if a full ID is given. Partial IDs are completed using the unique database instance ID. | +#### Gradual sync policies + +Each policy except `custom` sets these limits on parallel full syncs: + +| Policy | Use case | src_endpoint_limit | dst_endpoint_limit | dst_shard_limit | src_node_limit | dst_node_limit | +|--------|----------|--------------------|--------------------|-----------------|----------------|----------------| +| `default` | Applies when you don't select another policy | 1 | No limit | No limit | 1 | 1 | +| `non-gradual` | Syncs all shards at once. Fastest, but the load on the cluster can disrupt client applications. | No limit | No limit | No limit | No limit | No limit | +| `migration` | Protects the source database. Use when the destination serves no traffic, such as during a migration. | 1 | No limit | 1 | 1 | No limit | +| `replica` | Protects both the source and destination databases. Use for Replica Of when the destination serves live read traffic. | No limit | 1 | No limit | 1 | No limit | +| `active-active` | Tuned for full syncs between participating clusters in an Active-Active database, which are rare but can have a large impact | No limit | No limit | 1 | 1 | No limit | +| `custom` | Uses the limits you set with `gradual_sync_policy_parameters` | – | – | – | – | – | + ### Returns Returns `Finished successfully` if the database configuration was changed. Otherwise, it returns an error. diff --git a/content/operate/rs/references/rest-api/objects/bdb/_index.md b/content/operate/rs/references/rest-api/objects/bdb/_index.md index fe57419b02..ab608273e6 100644 --- a/content/operate/rs/references/rest-api/objects/bdb/_index.md +++ b/content/operate/rs/references/rest-api/objects/bdb/_index.md @@ -95,9 +95,9 @@ An API object that represents a managed database in the cluster. | export_status | Status of manually triggered export process (read-only)
    Values:
    'exporting'
    'succeeded'
    'failed' | | generate_text_monitor | boolean; Enable/disable generation of syncer monitoring information | | gradual_src_max_sources | integer (default: 1); Sync a maximum N sources in parallel (gradual_src_mode should be enabled for this to take effect) | -| gradual_src_mode | Indicates if gradual sync (of sync sources) should be activated
    Values:
    'enabled'
    'disabled' | -| gradual_sync_max_shards_per_source | integer (default: 1); Sync a maximum of N shards per source in parallel (gradual_sync_mode should be enabled for this to take effect) | -| gradual_sync_mode | Indicates if gradual sync (of source shards) should be activated ('auto' for automatic decision)
    Values:
    'enabled'
    'disabled'
    'auto' | +| gradual_src_mode | Indicates if gradual sync (of sync sources) should be activated (deprecated as of Redis Software v8.2.0; use `gradual_sync_policy` instead)
    Values:
    'enabled'
    'disabled' | +| gradual_sync_max_shards_per_source | integer (default: 1); Sync a maximum of N shards per source in parallel (gradual_sync_mode should be enabled for this to take effect) (deprecated as of Redis Software v8.2.0; use `gradual_sync_policy` instead) | +| gradual_sync_mode | Indicates if gradual sync (of source shards) should be activated ('auto' for automatic decision) (deprecated as of Redis Software v8.2.0; use `gradual_sync_policy` instead)
    Values:
    'enabled'
    'disabled'
    'auto' | | gradual_sync_policy | string (default: default); Managed gradual (parallel) full sync policy for replica synchronization. Non-custom policies own and normalize the limit fields in `gradual_sync_policy_parameters`.
    Values:
    'default'
    'non-gradual'
    'migration'
    'replica'
    'active-active'
    'custom' | | gradual_sync_policy_parameters | object; Gradual full sync limits, applied when `gradual_sync_policy` is `custom`. Each limit accepts `-1` (no limit) or a positive integer; `0` is not valid. Includes `src_endpoint_limit` (default: 1), `dst_endpoint_limit`, `dst_shard_limit`, and `src_node_limit` (default: -1). | | hash_slots_policy | The policy used for hash slots handling
    Values:
    **'legacy'**: slots range is '1-4096'
    **'16k'**: slots range is '0-16383' | From e5667ac1aedea39b50273b0450bf225b962d9363 Mon Sep 17 00:00:00 2001 From: David Dougherty Date: Mon, 26 Oct 2026 12:50:56 -0700 Subject: [PATCH 75/76] OP/RS: correct Redis Search limitation with Redis Software (#4346) --- .../oss_and_stack/stack-with-enterprise/search/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/operate/oss_and_stack/stack-with-enterprise/search/_index.md b/content/operate/oss_and_stack/stack-with-enterprise/search/_index.md index 912a394142..f5824f7afd 100644 --- a/content/operate/oss_and_stack/stack-with-enterprise/search/_index.md +++ b/content/operate/oss_and_stack/stack-with-enterprise/search/_index.md @@ -66,7 +66,7 @@ Because the index on the new shard is created synchronously though, it's expecte ## Limitations -- You cannot use Redis Search capabilities with the [OSS Cluster API](/content/operate/rs/databases/configure/oss-cluster-api.md). This limitation was fixed in Redis Software version 8.0. +- Redis Search capabilities are not supported with the [OSS Cluster API](/content/operate/rs/databases/configure/oss-cluster-api.md) before Redis Software 8.0. To use Redis Search with the OSS Cluster API on Redis Software 8.0 or later, the database must also be version 8.0 or later. ## More info From 46c971d5aa830a4845217a7407f1914790180a91 Mon Sep 17 00:00:00 2001 From: Kaitlyn Michael Date: Mon, 26 Oct 2026 16:47:16 -0500 Subject: [PATCH 76/76] K8s: Eden supported distros update --- .../reference/supported_k8s_distributions.md | 52 +++++++++---------- 1 file changed, 26 insertions(+), 26 deletions(-) diff --git a/content/operate/kubernetes/reference/supported_k8s_distributions.md b/content/operate/kubernetes/reference/supported_k8s_distributions.md index 21c2593d5e..c782a4eb0d 100644 --- a/content/operate/kubernetes/reference/supported_k8s_distributions.md +++ b/content/operate/kubernetes/reference/supported_k8s_distributions.md @@ -34,7 +34,7 @@ The tables below use the following status icons: ### Latest release -Operator release **8.2.0-12** supports the following Kubernetes and OpenShift versions. +Operator release **8.2.6-21** supports the following Kubernetes and OpenShift versions. | OpenShift | Kubernetes | Support status | |---|---|---| @@ -48,30 +48,30 @@ Operator release **8.2.0-12** supports the following Kubernetes and OpenShift ve The following table lists the Kubernetes versions supported by every operator version. Operator versions run from newest (left) to oldest (right). -{{}}| | Redis operator | **8.2.0-12** | **8.0.20-21** | **8.0.18-11** | **8.0.10-15** | **8.0.6-6** | **8.0.2-2** | **7.22.2-21** | **7.22.0-15** | **7.22.0-7** | **7.8.6-1** | **7.8.4-9** | **7.8.4-8** | **7.8.2-6** | **7.4.6-2** | **7.4.2-12** | **7.4.2-2** | **7.2.4-12** | **7.2.4-7** | **7.2.4-2** | **6.4.2-8** | **6.4.2-6** | **6.4.2-5** | **6.4.2-4** | **6.2.18-41** | **6.2.18-3** | **6.2.12-1** | **6.2.10-45** | **6.2.10-34** | **6.2.10-4** | **6.2.8-15** | **6.2.8-11** | **6.2.8-2** | **6.2.4-1** | -|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---| -| | | July 2026 | May 2026 | March 2026 | Jan 2026 | Dec 2025 | Oct 2025 | Oct 2025 | July 2025 | April 2025 | March 2025 | March 2025 | Feb 2025 | Nov 2024 | July 2024 | May 2024 | March 2024 | Dec 2023 | Oct 2023 | Aug 2023 | July 2023 | June 2023 | April 2023 | March 2023 | Jan 2023 | Nov 2022 | Sept 2022 | July 2022 | May 2022 | March 2022 | Jan 2022 | Jan 2022 | Nov 2021 | Sept 2021 | -| OpenShift | Kubernetes | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | -| — | 1.36 | ✅ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | -| 4.22 | 1.35 | ✅ | ✅ | ✅ | ✅ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | -| 4.21 | 1.34 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | | | | | | | | | | | | | | | | | | | | | | | -| 4.20 | 1.33 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | | | | | | | | | | | | | | | | | | | | | -| 4.19 | 1.32 | ⚠️ | ⚠️ | ⚠️ | ⚠️ | ⚠️ | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | | | | | | | | | | | | | | | | | | | -| 4.18 | 1.31 | ❌ | ⚠️ | ⚠️ | ⚠️ | ⚠️ | ⚠️ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | | | | | | | | | | | | | | | | -| 4.17 | 1.30 | | | | | | ❌ | ⚠️ | ⚠️ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | | | | | | | | | | | | | | | -| 4.16 | 1.29 | | | | | | | | | ❌ | ⚠️ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | | | | | | | | | | | | | -| 4.15 | 1.28 | | | | | | | | | ❌ | ⚠️ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | | | | | | | | | | | | -| 4.14 | 1.27 | | | | | | | | | | | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | | | | | | | | -| 4.13 | 1.26 | | | | | | | | | | | | | | ⚠️ | ⚠️ | ⚠️ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | | | | | | -| 4.12 | 1.25 | | | | | | | | | | | | | | | | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | | | | -| 4.11 | 1.24 | | | | | | | | | | | | | | | | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | | -| 4.10 | 1.23 | | | | | | | | | | | | | | | | | | | ❌ | ⚠️ | ⚠️ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | -| 4.09 | 1.22 | | | | | | | | | | | | | | | | | | | | | ❌ | ⚠️ | ⚠️ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | -| 4.08 | 1.21 | | | | | | | | | | | | | | | | | | | | | | | | | | ❌ | ⚠️ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | -| 4.07 | 1.20 | | | | | | | | | | | | | | | | | | | | | | | | | | | ❌ | ⚠️ | ✅ | ✅ | ✅ | ✅ | ✅ | -| 4.06 | 1.19 | | | | | | | | | | | | | | | | | | | | | | | | | | | ❌ | ⚠️ | ✅ | ✅ | ✅ | ✅ | ✅ | -| 4.05 | 1.18 | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ✅ | ✅ | ✅ | ✅ | ✅ | -| 4.04 | 1.17 | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ❌ | ⚠️ | -| 4.03 | 1.16 | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ❌ | ⚠️ | +{{}}| | Redis operator | **8.2.6-21** | **8.2.0-12** | **8.0.20-21** | **8.0.18-11** | **8.0.10-15** | **8.0.6-6** | **8.0.2-2** | **7.22.2-21** | **7.22.0-15** | **7.22.0-7** | **7.8.6-1** | **7.8.4-9** | **7.8.4-8** | **7.8.2-6** | **7.4.6-2** | **7.4.2-12** | **7.4.2-2** | **7.2.4-12** | **7.2.4-7** | **7.2.4-2** | **6.4.2-8** | **6.4.2-6** | **6.4.2-5** | **6.4.2-4** | **6.2.18-41** | **6.2.18-3** | **6.2.12-1** | **6.2.10-45** | **6.2.10-34** | **6.2.10-4** | **6.2.8-15** | **6.2.8-11** | **6.2.8-2** | **6.2.4-1** | +|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---| +| | | Oct 2026 | July 2026 | May 2026 | March 2026 | Jan 2026 | Dec 2025 | Oct 2025 | Oct 2025 | July 2025 | April 2025 | March 2025 | March 2025 | Feb 2025 | Nov 2024 | July 2024 | May 2024 | March 2024 | Dec 2023 | Oct 2023 | Aug 2023 | July 2023 | June 2023 | April 2023 | March 2023 | Jan 2023 | Nov 2022 | Sept 2022 | July 2022 | May 2022 | March 2022 | Jan 2022 | Jan 2022 | Nov 2021 | Sept 2021 | +| OpenShift | Kubernetes | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | +| — | 1.36 | ✅ | ✅ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | +| 4.22 | 1.35 | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | +| 4.21 | 1.34 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | | | | | | | | | | | | | | | | | | | | | | | +| 4.20 | 1.33 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | | | | | | | | | | | | | | | | | | | | | +| 4.19 | 1.32 | ⚠️ | ⚠️ | ⚠️ | ⚠️ | ⚠️ | ⚠️ | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | | | | | | | | | | | | | | | | | | | +| 4.18 | 1.31 | | ❌ | ⚠️ | ⚠️ | ⚠️ | ⚠️ | ⚠️ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | | | | | | | | | | | | | | | | +| 4.17 | 1.30 | | | | | | | ❌ | ⚠️ | ⚠️ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | | | | | | | | | | | | | | | +| 4.16 | 1.29 | | | | | | | | | | ❌ | ⚠️ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | | | | | | | | | | | | | +| 4.15 | 1.28 | | | | | | | | | | ❌ | ⚠️ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | | | | | | | | | | | | +| 4.14 | 1.27 | | | | | | | | | | | | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | | | | | | | | +| 4.13 | 1.26 | | | | | | | | | | | | | | | ⚠️ | ⚠️ | ⚠️ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | | | | | | +| 4.12 | 1.25 | | | | | | | | | | | | | | | | | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | | | | +| 4.11 | 1.24 | | | | | | | | | | | | | | | | | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | | +| 4.10 | 1.23 | | | | | | | | | | | | | | | | | | | | ❌ | ⚠️ | ⚠️ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | +| 4.09 | 1.22 | | | | | | | | | | | | | | | | | | | | | | ❌ | ⚠️ | ⚠️ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | +| 4.08 | 1.21 | | | | | | | | | | | | | | | | | | | | | | | | | | | ❌ | ⚠️ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| 4.07 | 1.20 | | | | | | | | | | | | | | | | | | | | | | | | | | | | ❌ | ⚠️ | ✅ | ✅ | ✅ | ✅ | ✅ | +| 4.06 | 1.19 | | | | | | | | | | | | | | | | | | | | | | | | | | | | ❌ | ⚠️ | ✅ | ✅ | ✅ | ✅ | ✅ | +| 4.05 | 1.18 | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ✅ | ✅ | ✅ | ✅ | ✅ | +| 4.04 | 1.17 | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ❌ | ⚠️ | +| 4.03 | 1.16 | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ❌ | ⚠️ | {{}}