Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 48 additions & 0 deletions RELEASE_NOTES.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,54 @@

-->

# Apache TsFile 2.5.0

## New Features

- [Go] Added table-model read and write APIs backed by the C++ library, with Tablet and Arrow batch writes, time-range and TAG filtering, pagination, and Arrow batch reads. (#929, #949)
- [Python] Added opt-in persistent dataset indexes and a native query runtime for `TsFileDataFrame`, supporting indexed row slicing and time-range queries across multiple TsFile files. (#911)
- [Java/C++/Python] Added read and write support for binary file-level properties. (#882, #897)
- [Java] Added optional table point counting and a tool to inspect and repair stored table point counts. (#882)
- [C++/Python] Added seekable random-access data sources, including Python binary file objects and remote files opened through libraries such as `fsspec`. (#939)
- [C++/Python] Added an optional memory-mapped local file read backend with `PREAD`, `MMAP`, and `AUTO` modes. (#920)
- [C++] Added CHIMP, RLBE, and CAMEL encodings, Zstandard compression, and optional LZMA2 compression, with Java/C++ compatibility coverage. (#905)
- [C++] Added `tsfile-cli`, a command-line tool for tree-model and table-model inspection, filtered reads, CSV/NDJSON export, CSV import, and physical file layout inspection with `sketch`. (#829, #838, #915, #994, #996, #997, #1002)
- [Tools] Added reusable agent skills for TsFile development and `tsfile-cli` workflows. (#907)

## Improvements

- [Python] Accelerated dataset-index metadata lookups with Cython and descriptor caching, added configurable read-cache and concurrency limits, and added an index-trust option for immutable datasets. (#959, #1004)
- [Python] Aligned Python exceptions and error codes with the native library for consistent error reporting. (#870)
- [C++] Improved batch read throughput with faster Gorilla decoding, bulk column output, and aligned row-offset pushdown. (#873, #898, #863)
- [C++] Added static-library builds and configurable system or verified source-archive dependency sourcing, including offline builds from cached archives. (#894, #905)
- [Java] Added buffered input and configurable `TsFileInput` construction for readers. (#868, #869)
- [Java] Streamlined data-type handling, cached page-reader type services, and moved type dispatch outside batch read and write loops. (#872, #955, #963)
- [Java] Reduced query merge overhead and boxing allocations with primitive long heaps and sets. (#906)
- [Java] Optimized small bitmaps with a long-backed implementation and added range-query APIs. (#871, #884)
- [Java] Added batched null writes and independent bitmap offsets to `ValuePageWriter` for efficient nullable-column writes. (#948)

## Bug Fixes

- [Java] Propagated `IOException` from table-writer `close()` so flush and file-finalization failures are reported to callers instead of being swallowed. (#899)
- [C++] Fixed loss of buffered data when closing writers by flushing pending rows before finalizing the file. (#969)
- [C++/Python] Fixed silent data loss from table-read failures by propagating metadata, decoding, and I/O errors to callers. (#970)
- [C++] Fixed scans ending prematurely when a decoded page contains no rows matching the filter, allowing later matching pages and chunks to be read. (#975)
- [C++] Fixed value/timestamp misalignment when aligned records or Tablets omit measurements, including null padding and synchronized page sealing across columns. (#968)
- [C++] Fixed empty aligned value-page headers and page advancement so leading, intermediate, and consecutive all-null pages do not hide or corrupt subsequent data. (#976, #1001)
- [C++/Python] Fixed residual row-offset handling in non-aligned page decoding, correcting indexed dataset slices across page and chunk boundaries. (#1007)
- [C++] Stopped writing empty chunks for registered but unwritten non-aligned measurements, preventing Java readers from rejecting otherwise valid files as crashed. (#909)
- [C++] Fixed UTF-8 file paths on Windows for reads, writes, and recovery, and prevented existing output files from being overwritten. (#927)
- [C++] Fixed TS_2DIFF FLOAT/DOUBLE page decoding, wide INT64 decoding, SIMD min/max statistics, and Gorilla sentinel-value handling. (#901, #947, #883)
- [Java/C++] Fixed RLBE decoding of malformed blocks by validating block sizes, segment widths, and run lengths. (#913)
- [C++] Fixed reader schema reporting and append recovery to use the encoding and compression stored in the latest chunk instead of library defaults or stale chunk settings. (#926)
- [C++] Preserved shared time statistics in filtered table metadata so FIELD null counts can be calculated correctly. (#989)
- [Java] Fixed `VectorMeasurementSchema` stream deserialization for per-column compressors and partial stream reads. (#941)
- [Java] Fixed dictionary primitive-value lookup through dictionary IDs, corrected `TimeColumn` position counts, and completed its typed accessors. (#961)
- [Python] Fixed omitted BOOLEAN and TIMESTAMP dataset fields and preserved null field values during Arrow-to-NumPy conversion. (#935, #942)
- [Python] Fixed DataFrame subsets closing shared readers and guarded dataset-index access after close to prevent invalid reads and crashes. (#946, #999)
- [C++] Fixed double-free, memory-leak, and null-dereference paths in readers, compressors, Tablets, containers, and C API string getters. (#879, #881)
- [Java] Strengthened encryption metadata length and implementation-type validation while preserving compatibility with custom encryption implementations. (#925, #956)

# Apache TsFile 2.4.0

## New Features
Expand Down
2 changes: 1 addition & 1 deletion cpp/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ endif ()
if (POLICY CMP0074)
cmake_policy(SET CMP0074 NEW)
endif ()
set(TsFile_CPP_VERSION 2.3.2.dev)
set(TsFile_CPP_VERSION 2.5.1.dev)

include(${CMAKE_CURRENT_SOURCE_DIR}/cmake/DependencySource.cmake)
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/cmake")
Expand Down
2 changes: 1 addition & 1 deletion cpp/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
<parent>
<groupId>org.apache.tsfile</groupId>
<artifactId>tsfile-parent</artifactId>
<version>2.4.1-SNAPSHOT</version>
<version>2.5.1-SNAPSHOT</version>
</parent>
<artifactId>tsfile-cpp</artifactId>
<packaging>pom</packaging>
Expand Down
9 changes: 9 additions & 0 deletions doap_tsfile.rdf
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@
<programming-language>C++</programming-language>
<programming-language>C</programming-language>
<programming-language>Python</programming-language>
<programming-language>Go</programming-language>

<category rdf:resource="https://projects.apache.org/category/big-data" />
<category rdf:resource="https://projects.apache.org/category/iot" />
Expand All @@ -47,6 +48,14 @@
<category rdf:resource="http://projects.apache.org/category/c++"/>
<category rdf:resource="http://projects.apache.org/category/c"/>

<release>
<Version>
<name>Apache TsFile</name>
<created>2026-10-14</created>
<revision>2.5.0</revision>
</Version>
</release>

<release>
<Version>
<name>Apache TsFile</name>
Expand Down
2 changes: 1 addition & 1 deletion java/common/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
<parent>
<groupId>org.apache.tsfile</groupId>
<artifactId>tsfile-java</artifactId>
<version>2.4.1-SNAPSHOT</version>
<version>2.5.1-SNAPSHOT</version>
</parent>
<artifactId>common</artifactId>
<name>TsFile: Java: Common</name>
Expand Down
4 changes: 2 additions & 2 deletions java/examples/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
<parent>
<groupId>org.apache.tsfile</groupId>
<artifactId>tsfile-java</artifactId>
<version>2.4.1-SNAPSHOT</version>
<version>2.5.1-SNAPSHOT</version>
</parent>
<artifactId>examples</artifactId>
<name>TsFile: Java: Examples</name>
Expand All @@ -36,7 +36,7 @@
<dependency>
<groupId>org.apache.tsfile</groupId>
<artifactId>tsfile</artifactId>
<version>2.4.1-SNAPSHOT</version>
<version>2.5.1-SNAPSHOT</version>
</dependency>
</dependencies>
<build>
Expand Down
4 changes: 2 additions & 2 deletions java/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,10 @@
<parent>
<groupId>org.apache.tsfile</groupId>
<artifactId>tsfile-parent</artifactId>
<version>2.4.1-SNAPSHOT</version>
<version>2.5.1-SNAPSHOT</version>
</parent>
<artifactId>tsfile-java</artifactId>
<version>2.4.1-SNAPSHOT</version>
<version>2.5.1-SNAPSHOT</version>
<packaging>pom</packaging>
<name>TsFile: Java</name>
<modules>
Expand Down
6 changes: 3 additions & 3 deletions java/tools/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -24,15 +24,15 @@
<parent>
<groupId>org.apache.tsfile</groupId>
<artifactId>tsfile-java</artifactId>
<version>2.4.1-SNAPSHOT</version>
<version>2.5.1-SNAPSHOT</version>
</parent>
<artifactId>tools</artifactId>
<name>TsFile: Java: Tools</name>
<dependencies>
<dependency>
<groupId>org.apache.tsfile</groupId>
<artifactId>common</artifactId>
<version>2.4.1-SNAPSHOT</version>
<version>2.5.1-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>commons-cli</groupId>
Expand All @@ -41,7 +41,7 @@
<dependency>
<groupId>org.apache.tsfile</groupId>
<artifactId>tsfile</artifactId>
<version>2.4.1-SNAPSHOT</version>
<version>2.5.1-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
Expand Down
4 changes: 2 additions & 2 deletions java/tsfile/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
<parent>
<groupId>org.apache.tsfile</groupId>
<artifactId>tsfile-java</artifactId>
<version>2.4.1-SNAPSHOT</version>
<version>2.5.1-SNAPSHOT</version>
</parent>
<artifactId>tsfile</artifactId>
<name>TsFile: Java: TsFile</name>
Expand All @@ -38,7 +38,7 @@
<dependency>
<groupId>org.apache.tsfile</groupId>
<artifactId>common</artifactId>
<version>2.4.1-SNAPSHOT</version>
<version>2.5.1-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>com.github.luben</groupId>
Expand Down
2 changes: 1 addition & 1 deletion pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
</parent>
<groupId>org.apache.tsfile</groupId>
<artifactId>tsfile-parent</artifactId>
<version>2.4.1-SNAPSHOT</version>
<version>2.5.1-SNAPSHOT</version>
<packaging>pom</packaging>
<name>Apache TsFile Project Parent POM</name>
<properties>
Expand Down
2 changes: 1 addition & 1 deletion python/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
<parent>
<groupId>org.apache.tsfile</groupId>
<artifactId>tsfile-parent</artifactId>
<version>2.4.1-SNAPSHOT</version>
<version>2.5.1-SNAPSHOT</version>
</parent>
<artifactId>tsfile-python</artifactId>
<packaging>pom</packaging>
Expand Down
2 changes: 1 addition & 1 deletion python/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "tsfile"
version = "2.3.2.dev"
version = "2.5.1.dev"
requires-python = ">=3.9"
description = "TsFile Python"
readme = {file = "README.md", content-type = "text/markdown"}
Expand Down
2 changes: 1 addition & 1 deletion python/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@
ROOT = Path(__file__).parent.resolve()
PKG = ROOT / "tsfile"

version = "2.3.2.dev"
version = "2.5.1.dev"


def _find_cpp_build():
Expand Down
6 changes: 3 additions & 3 deletions skills/tsfile/references/docs-map.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,10 @@ path_bases:
skill_baseline:
scope: current-source
source_ref: develop
maven_project_version: "2.3.2-SNAPSHOT"
maven_project_version: "2.5.1-SNAPSHOT"
java_release: "17"
cpp_sdk_version: "2.3.2.dev"
python_package_version: "2.3.2.dev"
cpp_sdk_version: "2.5.1.dev"
python_package_version: "2.5.1.dev"

official_url_prefixes:
- https://tsfile.apache.org/
Expand Down
Loading