[docs](lance) Document index prewarm through WARM UP SELECT

i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/lakehouse/catalogs/lance-catalog.mdx

@@ -12,7 +12,7 @@
 :::note
 这是一个实验性功能。
 
-Lance Catalog 自 Apache Doris 4.2 版本开始支持。
+Lance Catalog 自 Apache Doris 4.1.4 版本开始支持。
 :::
 
 Lance 是面向分析和 AI 场景的列式数据格式。Doris 可以通过 Lance Catalog 发现 Lance Namespace 中的数据库和表，并直接查询存储在本地文件系统、S3 兼容对象存储或阿里云 OSS 中的 Lance 数据集。
@@ -856,7 +856,7 @@
 
 #### 准备数据并执行查询
 
 使用上文的 AWS S3 Filesystem Catalog 示例，Catalog 名称为 `lance_fs_s3`，warehouse 为 `s3://my-bucket/lance`。在写入环境中安装 Lance Python SDK（`pylance`）和 `pyarrow`，配置 S3 凭证和 region，例如设置 `AWS_ACCESS_KEY_ID`、`AWS_SECRET_ACCESS_KEY` 和 `AWS_DEFAULT_REGION`。写入端需要写权限，Doris 需要同一位置的读权限。使用自定义 endpoint 或其他认证方式时，按 Catalog 配置对应的 SDK [对象存储选项](https://lance.org/guide/object_store/)。
 
 将 `documents.lance` 直接写在 warehouse 下，Doris 会将其发现为 `lance_fs_s3.default.documents`。请同时替换 Catalog 和 Python 示例中的 bucket，并使用尚不存在的数据集路径：
 
@@ -989,7 +989,7 @@
 
 ## FE 表访问缓存
 
 该配置要求 Doris 构建版本包含[表访问缓存变更](https://github.com/apache/doris/pull/68305)。
 
 每个 FE 可以在 Catalog 内缓存表解析后的 Dataset URI 和访问配置，避免查询规划时重复执行文件系统发现或 REST `describeTable` 请求。每个 Catalog 客户端最多保留 10,000 个条目。`lance.table_access_cache_ttl_seconds` 默认为 `60` 秒，命中缓存不会延长有效期。此缓存不保存 Dataset 版本：每次读取仍会打开 Dataset 并选择快照。
 
@@ -1031,6 +1031,82 @@
 
 数据缓存按需填充，不会预先下载整个 Dataset。未命中时读取源存储，并将数据交给后台写入磁盘缓存；首次读取仍会产生源存储 I/O 和缓存填充开销。缓存命中可以减少后续读取的源存储访问，但不消除解码、过滤和排序开销。
 
+### 索引预热 {#index-prewarm}
+
+在延迟敏感的查询前，使用 `WARM UP SELECT ... PROPERTIES ("read_index_only" = "true")` 将已有 Lance 索引加载到目标 BE 共享的索引内存缓存中。语句同步执行：只有所有选中索引在每个目标 BE 上都完成预热后才返回成功。该模式读取必要的元数据和索引内容，不扫描或物化表数据。
+
+#### 前置条件
+
+- 按照[配置 Catalog](#配置-catalog)配置 Lance Catalog，并注册已经通过 Lance 工具建立索引的 Dataset。以下示例假设 `example_lance.default.items` 的 `embedding` 列已经建立索引。
+- FE 和所有目标 BE 必须使用支持仅预热索引的 Doris 构建。这是 Lance 的实验性功能；支持普通数据预热不代表支持该模式。
+- 执行用户需要全局 `ADMIN_PRIV` 和目标表的 `SELECT_PRIV`。存算分离模式下，还需要当前计算组的 `USAGE_PRIV`。
+- 目标 BE 必须能够使用 Catalog 的存储访问配置读取 Dataset 及其索引。通过 `lance_index_cache_size_bytes` 规划索引内存容量，参见 [BE 配置](#be-配置)。
+
+#### 语法和选项
+
+```sql
+WARM UP SELECT <column_list_or_star>
+FROM <lance_catalog_table>
+PROPERTIES ("read_index_only" = "true");
+```
+
+| 选项 | 类型 | 是否必选 | 默认值 | 作用 |
+|---|---|---|---|---|
+| `read_index_only` | Boolean（`true` 或 `false`，可加引号） | 否 | `false` | `true` 预热投影列所选择的索引；省略或设为 `false` 时保留原有数据预热流程及扫描/缓存统计。该选项仅作用于当前语句，不是 Session 变量或 Catalog 属性。 |
+
+投影决定预热哪些**逻辑索引**：
+
+- `*` 选择已解析快照中的全部逻辑索引，包含每个索引的全部有效 Index Segment。
+- 列列表选择与任意指定列关联的索引。重复列或同一索引的多个 Segment 不会产生重复的逻辑索引结果。列名不区分大小写，未知或存在歧义的列名会被拒绝。
+- 没有匹配索引时返回错误，包括对没有索引的表使用 `*`。如果其他选中列存在索引，列表中的无索引列不会增加预热工作。
+
+仅预热索引模式支持单个 Lance Catalog 表，以及普通列名或 `*`。不支持文件 TVF、表达式、列别名、`WHERE`、`EXPLAIN`、JOIN、子查询、分组、排序或 `LIMIT`。`*` 必须单独使用，不能带 `EXCEPT` 或 `REPLACE`。未知选项以及 `true`、`false` 以外的取值会被拒绝。索引格式必须受内置 Lance SDK 的预热实现支持；支持查询某种格式并不保证支持预热该格式。
+
+#### 示例与返回结果
+
+预热全部索引：
+
+```sql
+WARM UP SELECT * FROM example_lance.default.items
+PROPERTIES ("read_index_only" = "true");
+```
+
+每个逻辑索引返回一行。若只需要预热 `embedding` 列关联的索引：
+
+```sql
+WARM UP SELECT embedding FROM example_lance.default.items
+PROPERTIES ("read_index_only" = "true");
+```
+
+假设该列只有一个名为 `embedding_idx` 的索引，返回结果示意如下：
+
+```text
+Table                            | Index         | DatasetVersion | BackendCount | ElapsedMs
+`example_lance`.`default`.`items` | embedding_idx | 7              | 3            | 250
+```
+
+实际索引名、版本、BE 数量和耗时取决于 Dataset 与集群。如果该列有多个索引，则每个索引各返回一行。
+
+| 返回列 | 含义 |
+|---|---|
+| `Table` | 完整限定的表名。 |
+| `Index` | 逻辑索引名。 |
+| `DatasetVersion` | 本次语句固定的 Dataset 快照版本，所有返回行使用同一版本。 |
+| `BackendCount` | 该索引完成预热的可用目标 BE 数量，所有返回行使用同一目标集合。 |
+| `ElapsedMs` | 该索引在后端预热阶段的耗时，单位毫秒；不包含初始元数据解析时间，也不表示缓存命中数或字节数。 |
+
+语句使用当前 Session 的计算组，并预热该执行上下文中全部符合条件的 BE。预热其他计算组前，应先在 Session 中切换计算组。语句不会预热无关计算组，也不会包含目标集合确定后新增的 BE。全部索引使用同一个 Dataset 快照和语句超时期限；并发更新 Dataset 不会改变正在预热的快照。
+
+#### 错误处理与缓存生命周期
+
+任意目标 BE 或选中索引失败时，语句返回错误，不返回部分成功行。重试前应检查表和计算组权限、可用 BE、存储访问及索引格式兼容性。不支持预热的旧 BE 不会被静默计为成功。`query_timeout` 和查询/连接取消会终止客户端等待，但不保证立即中断 SDK I/O。失败或取消不会回滚已加载的缓存条目，可以重新执行语句。
+
+仅预热索引模式使用 Lance 索引缓存，独立于通用 File Cache 的 Session 开关和单独的 Lance 磁盘数据缓存，不需要开启数据文件缓存。省略 `read_index_only` 或设为 `false` 时，保留原有[数据预热](../data-cache.md#缓存预热)行为及其缓存前置条件。
+
+已加载的索引条目可由同一 BE 上的后续查询共享，SQL Session 结束不会清除这些条目。条目仍然可以被淘汰：预热成功不代表索引被固定在缓存中，也不保证全部索引持续驻留。BE 重启或替换会丢失内存缓存；新增 BE、新建/替换的索引 Segment 或不同计算组可能需要再次预热。
+
+预热会消耗每个目标 BE 的存储带宽和内存，并可能淘汰其他热索引。只查询部分索引时优先指定所需列，避免使用 `*`；为并发查询预留内存，将大规模预热安排在低峰期。预热不会消除向量距离/Top-K 计算、输出列回读或无索引数据读取。按照[查看缓存效果](#查看缓存效果)对比后续查询 Profile 与源存储 I/O；数据缓存字节计数不能用于衡量索引预热流量。
+
 ### BE 配置
 
 以下参数均为可选项，在每个 BE 的 `be.conf` 中配置，修改后需要重启对应 BE。容量均为单个 BE 的容量，单位为字节。

i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/lakehouse/data-cache.md

@@ -215,6 +215,8 @@
 
 > 该功能自 4.0.2 版本支持。
 
+预热 Lance 索引时，使用 `WARM UP SELECT ... PROPERTIES ("read_index_only" = "true")`；支持条件、权限、索引选择规则和每个索引的返回列，参见 [Lance 索引预热](catalogs/lance-catalog.mdx#index-prewarm)。该模式加载独立的 Lance 索引内存缓存，不扫描表数据。省略该设置或设为 `false` 时，保留下文的数据预热行为。上述 4.0.2 版本说明针对普通数据预热，不代表该版本支持 Lance 仅预热索引扩展。
+
 ### 语法
 
 ```sql
@@ -263,7 +265,7 @@
   FROM hive_db.tpch100_parquet.lineitem
   WHERE l_orderkey = 123456;
   ```
 
 ### 执行返回结果
 
 执行 `WARM UP SELECT` 后，FE 会下发任务至各 BE。BE 扫描远端数据并写入 Data Cache。

versioned_docs/version-4.x/lakehouse/catalogs/lance-catalog.mdx

@@ -1,4 +1,4 @@
 ---
 {
     "title": "Lance Catalog",
     "language": "en",
@@ -12,7 +12,7 @@
 :::note
 This is an experimental feature.
 
-Lance Catalog is supported starting from Apache Doris 4.2.
+Lance Catalog is supported starting from Apache Doris 4.1.4.
 :::
 
 Lance is a columnar data format designed for analytics and AI workloads. Doris can use a Lance Catalog to discover databases and tables in a Lance Namespace and directly query Lance datasets stored on a local file system, S3-compatible object storage, or Alibaba Cloud OSS.
@@ -856,7 +856,7 @@
 
 #### Prepare Data and Run a Query
 
 Use the AWS S3 Filesystem Catalog example above, with catalog `lance_fs_s3` and warehouse `s3://my-bucket/lance`. Install the Lance Python SDK (`pylance`) and `pyarrow` in the writer environment. Configure its S3 credentials and region, for example through `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, and `AWS_DEFAULT_REGION`. The writer needs write access, and Doris needs read access to the same location. For custom endpoints or other authentication methods, configure the SDK's [object-store options](https://lance.org/guide/object_store/) to match the Catalog.
 
 Create `documents.lance` directly under the warehouse so that Doris discovers it as `lance_fs_s3.default.documents`. Replace the bucket in both the Catalog and Python example. Use a new dataset location:
 
@@ -989,7 +989,7 @@
 
 ## FE Table Access Cache
 
 This setting requires a Doris build containing [the table access cache change](https://github.com/apache/doris/pull/68305).
 
 Each FE can cache a table's resolved Dataset URI and access options within a Catalog, avoiding repeated filesystem discovery or REST `describeTable` requests during query planning. Each Catalog client holds at most 10,000 entries. `lance.table_access_cache_ttl_seconds` defaults to `60`; cache hits do not extend the lifetime. This cache does not store Dataset versions: each read still opens the Dataset and selects its snapshot.
 
@@ -1031,6 +1031,82 @@
 
 The data cache fills on demand without downloading the entire Dataset in advance. On a miss, it reads from the source storage and enqueues the data for background disk-cache writes. The first read still incurs source I/O and cache-fill overhead. Cache hits can reduce subsequent source reads, but do not eliminate decoding, filtering, or sorting costs.
 
+### Index Prewarm {#index-prewarm}
+
+Use `WARM UP SELECT ... PROPERTIES ("read_index_only" = "true")` before latency-sensitive queries to load existing Lance indexes into the shared index memory cache on the target BEs. The statement runs synchronously: it returns success only after all selected indexes have completed prewarm on every target BE. It reads the required metadata and index contents without scanning or materializing table data.
+
+#### Prerequisites
+
+- Configure a Lance Catalog as described in [Configure a Catalog](#configure-a-catalog), and register a dataset that already contains indexes created with Lance tooling. The examples below assume `example_lance.default.items` has an index on `embedding`.
+- Use a Doris build whose FE and all target BEs support index-only prewarm. This is an experimental Lance feature; the availability of ordinary data warmup alone does not imply support for this mode.
+- The executing user needs global `ADMIN_PRIV` and `SELECT_PRIV` on the table. In compute-storage decoupled mode, the user also needs `USAGE_PRIV` on the current compute group.
+- The target BEs must be able to read the dataset and its indexes with the Catalog's storage access configuration. Budget index memory with `lance_index_cache_size_bytes`; see [BE Configuration](#be-configuration).
+
+#### Syntax and Options
+
+```sql
+WARM UP SELECT <column_list_or_star>
+FROM <lance_catalog_table>
+PROPERTIES ("read_index_only" = "true");
+```
+
+| Option | Type | Required | Default | Effect |
+|---|---|---|---|---|
+| `read_index_only` | Boolean (`true` or `false`, optionally quoted) | No | `false` | `true` prewarms the indexes selected by the projection. Omitted or `false` retains the existing data warmup path and its scan/cache statistics. This is a statement option, not a session variable or Catalog property. |
+
+The projection determines which **logical indexes** are prewarmed:
+
+- `*` selects every logical index in the table's resolved snapshot, including all its active index segments.
+- A column list selects each index associated with any of those columns. Repeated columns or multiple segments do not cause duplicate logical-index results. Column names are matched case-insensitively; unknown or ambiguous names are rejected.
+- A selection with no matching indexes, including `*` on a table without indexes, returns an error. An unindexed column does not add work when other selected columns have indexes.
+
+Index-only mode supports one Lance Catalog table and plain column names or `*`. It does not support file TVFs, expressions, column aliases, `WHERE`, `EXPLAIN`, joins, subqueries, grouping, sorting, or `LIMIT`. Use `*` alone, without `EXCEPT` or `REPLACE`. Unknown properties and values other than `true` or `false` are rejected. Index formats must be supported by the embedded Lance SDK's prewarm implementation; query support alone is not a guarantee that a format supports prewarm.
+
+#### Examples and Results
+
+Prewarm all indexes:
+
+```sql
+WARM UP SELECT * FROM example_lance.default.items
+PROPERTIES ("read_index_only" = "true");
+```
+
+This returns one row for each logical index. To prewarm only indexes associated with `embedding`:
+
+```sql
+WARM UP SELECT embedding FROM example_lance.default.items
+PROPERTIES ("read_index_only" = "true");
+```
+
+For a table with one index named `embedding_idx` on that column, an illustrative result is:
+
+```text
+Table                            | Index         | DatasetVersion | BackendCount | ElapsedMs
+`example_lance`.`default`.`items` | embedding_idx | 7              | 3            | 250
+```
+
+Actual index names, versions, backend counts, and times depend on the dataset and cluster. If the column has multiple indexes, the result contains one row per index.
+
+| Result column | Meaning |
+|---|---|
+| `Table` | Fully qualified table name. |
+| `Index` | Logical index name. |
+| `DatasetVersion` | Dataset snapshot pinned for this statement. All result rows use the same version. |
+| `BackendCount` | Number of eligible target BEs on which this index completed prewarm. All rows use the same target set. |
+| `ElapsedMs` | Elapsed milliseconds for this index's backend prewarm phase; excludes initial metadata resolution and is not a cache-hit or byte count. |
+
+The statement uses the current session's compute group and warms every eligible BE in that execution context. To prewarm another group, select it in the session first. It does not warm unrelated groups or BEs added after the target set was resolved. All indexes use the same dataset snapshot and statement timeout; a concurrent dataset update does not change the snapshot being warmed.
+
+#### Errors and Cache Lifetime
+
+If any target BE or selected index fails, the statement returns an error instead of partial success rows. Check table/compute-group permissions, available BEs, storage access, and index-format compatibility before retrying. Older BEs that do not support prewarm cannot silently count as successful targets. `query_timeout` and query/connection cancellation stop the client wait, but may not immediately interrupt SDK I/O. Failure or cancellation does not roll back cache entries already loaded; the statement can be retried.
+
+Index-only mode uses the Lance index cache independently of the general File Cache session switches and the separate Lance disk data cache. It does not require enabling data-file caching. Omitting `read_index_only` or setting it to `false` keeps the existing [data warmup](../data-cache.md#cache-warmup) behavior and its cache prerequisites.
+
+Loaded index entries are shared by later queries on the same BE and survive the end of the SQL session. They remain evictable: successful prewarm does not pin an index or guarantee that the entire index stays resident. BE restart or replacement loses the in-memory cache; new BEs, newly built/replaced index segments, or a different compute group may need another prewarm.
+
+Prewarm consumes storage bandwidth and memory on each target BE and can evict other hot indexes. Prefer the required columns over `*` when only a subset of indexes will be queried, leave memory for concurrent queries, and schedule large warmups outside peak traffic. It does not eliminate vector distance/Top-K computation, output-column fetches, or reads of unindexed data. Compare subsequent query Profiles and source I/O using [Checking Cache Effectiveness](#checking-cache-effectiveness); the data-cache byte counters do not measure index-prewarm traffic.
+
 ### BE Configuration
 
 All parameters below are optional. Set them in each BE's `be.conf` and restart that BE for changes to take effect. Capacities are per BE and expressed in bytes.

versioned_docs/version-4.x/lakehouse/data-cache.md

@@ -1,4 +1,4 @@
 ---
 {
     "title": "Data Cache",
     "language": "en",
@@ -215,6 +215,8 @@
 
 > This feature is supported since version 4.0.2.
 
+For Lance indexes, use `WARM UP SELECT ... PROPERTIES ("read_index_only" = "true")`; see [Lance Index Prewarm](catalogs/lance-catalog.mdx#index-prewarm) for availability, permissions, index selection, and the per-index result schema. This mode loads the separate Lance index memory cache without a table-data scan. Omitting the setting or specifying `false` retains the data warmup behavior described below. The 4.0.2 version note above applies to ordinary data warmup, not the Lance index-only extension.
+
 ### Syntax
 
 ```sql
@@ -263,7 +265,7 @@
   FROM hive_db.tpch100_parquet.lineitem
   WHERE l_orderkey = 123456;
   ```
 
 ### Execution Results
 
 After executing `WARM UP SELECT`, the FE dispatches tasks to each BE. The BE scans remote data and writes it to Data Cache.