diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.md index 81ca4c67a..c29348d4a 100644 --- a/RELEASE_NOTES.md +++ b/RELEASE_NOTES.md @@ -19,6 +19,54 @@ --> +# Apache TsFile 2.5.0 + +## New Features + +- [Go] Added table-model read and write APIs backed by the C++ library, with Tablet and Arrow batch writes, time-range and TAG filtering, pagination, and Arrow batch reads. (#929, #949) +- [Python] Added opt-in persistent dataset indexes and a native query runtime for `TsFileDataFrame`, supporting indexed row slicing and time-range queries across multiple TsFile files. (#911) +- [Java/C++/Python] Added read and write support for binary file-level properties. (#882, #897) +- [Java] Added optional table point counting and a tool to inspect and repair stored table point counts. (#882) +- [C++/Python] Added seekable random-access data sources, including Python binary file objects and remote files opened through libraries such as `fsspec`. (#939) +- [C++/Python] Added an optional memory-mapped local file read backend with `PREAD`, `MMAP`, and `AUTO` modes. (#920) +- [C++] Added CHIMP, RLBE, and CAMEL encodings, Zstandard compression, and optional LZMA2 compression, with Java/C++ compatibility coverage. (#905) +- [C++] Added `tsfile-cli`, a command-line tool for tree-model and table-model inspection, filtered reads, CSV/NDJSON export, CSV import, and physical file layout inspection with `sketch`. (#829, #838, #915, #994, #996, #997, #1002) +- [Tools] Added reusable agent skills for TsFile development and `tsfile-cli` workflows. (#907) + +## Improvements + +- [Python] Accelerated dataset-index metadata lookups with Cython and descriptor caching, added configurable read-cache and concurrency limits, and added an index-trust option for immutable datasets. (#959, #1004) +- [Python] Aligned Python exceptions and error codes with the native library for consistent error reporting. (#870) +- [C++] Improved batch read throughput with faster Gorilla decoding, bulk column output, and aligned row-offset pushdown. (#873, #898, #863) +- [C++] Added static-library builds and configurable system or verified source-archive dependency sourcing, including offline builds from cached archives. (#894, #905) +- [Java] Added buffered input and configurable `TsFileInput` construction for readers. (#868, #869) +- [Java] Streamlined data-type handling, cached page-reader type services, and moved type dispatch outside batch read and write loops. (#872, #955, #963) +- [Java] Reduced query merge overhead and boxing allocations with primitive long heaps and sets. (#906) +- [Java] Optimized small bitmaps with a long-backed implementation and added range-query APIs. (#871, #884) +- [Java] Added batched null writes and independent bitmap offsets to `ValuePageWriter` for efficient nullable-column writes. (#948) + +## Bug Fixes + +- [Java] Propagated `IOException` from table-writer `close()` so flush and file-finalization failures are reported to callers instead of being swallowed. (#899) +- [C++] Fixed loss of buffered data when closing writers by flushing pending rows before finalizing the file. (#969) +- [C++/Python] Fixed silent data loss from table-read failures by propagating metadata, decoding, and I/O errors to callers. (#970) +- [C++] Fixed scans ending prematurely when a decoded page contains no rows matching the filter, allowing later matching pages and chunks to be read. (#975) +- [C++] Fixed value/timestamp misalignment when aligned records or Tablets omit measurements, including null padding and synchronized page sealing across columns. (#968) +- [C++] Fixed empty aligned value-page headers and page advancement so leading, intermediate, and consecutive all-null pages do not hide or corrupt subsequent data. (#976, #1001) +- [C++/Python] Fixed residual row-offset handling in non-aligned page decoding, correcting indexed dataset slices across page and chunk boundaries. (#1007) +- [C++] Stopped writing empty chunks for registered but unwritten non-aligned measurements, preventing Java readers from rejecting otherwise valid files as crashed. (#909) +- [C++] Fixed UTF-8 file paths on Windows for reads, writes, and recovery, and prevented existing output files from being overwritten. (#927) +- [C++] Fixed TS_2DIFF FLOAT/DOUBLE page decoding, wide INT64 decoding, SIMD min/max statistics, and Gorilla sentinel-value handling. (#901, #947, #883) +- [Java/C++] Fixed RLBE decoding of malformed blocks by validating block sizes, segment widths, and run lengths. (#913) +- [C++] Fixed reader schema reporting and append recovery to use the encoding and compression stored in the latest chunk instead of library defaults or stale chunk settings. (#926) +- [C++] Preserved shared time statistics in filtered table metadata so FIELD null counts can be calculated correctly. (#989) +- [Java] Fixed `VectorMeasurementSchema` stream deserialization for per-column compressors and partial stream reads. (#941) +- [Java] Fixed dictionary primitive-value lookup through dictionary IDs, corrected `TimeColumn` position counts, and completed its typed accessors. (#961) +- [Python] Fixed omitted BOOLEAN and TIMESTAMP dataset fields and preserved null field values during Arrow-to-NumPy conversion. (#935, #942) +- [Python] Fixed DataFrame subsets closing shared readers and guarded dataset-index access after close to prevent invalid reads and crashes. (#946, #999) +- [C++] Fixed double-free, memory-leak, and null-dereference paths in readers, compressors, Tablets, containers, and C API string getters. (#879, #881) +- [Java] Strengthened encryption metadata length and implementation-type validation while preserving compatibility with custom encryption implementations. (#925, #956) + # Apache TsFile 2.4.0 ## New Features diff --git a/cpp/CMakeLists.txt b/cpp/CMakeLists.txt index c39bcbc28..0e498e3e0 100755 --- a/cpp/CMakeLists.txt +++ b/cpp/CMakeLists.txt @@ -32,7 +32,7 @@ endif () if (POLICY CMP0074) cmake_policy(SET CMP0074 NEW) endif () -set(TsFile_CPP_VERSION 2.3.2.dev) +set(TsFile_CPP_VERSION 2.5.1.dev) include(${CMAKE_CURRENT_SOURCE_DIR}/cmake/DependencySource.cmake) list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/cmake") diff --git a/cpp/pom.xml b/cpp/pom.xml index 30c70b858..d7db7ba01 100644 --- a/cpp/pom.xml +++ b/cpp/pom.xml @@ -22,7 +22,7 @@ org.apache.tsfile tsfile-parent - 2.4.1-SNAPSHOT + 2.5.1-SNAPSHOT tsfile-cpp pom diff --git a/doap_tsfile.rdf b/doap_tsfile.rdf index ea006da21..6b9b385d1 100644 --- a/doap_tsfile.rdf +++ b/doap_tsfile.rdf @@ -39,6 +39,7 @@ C++ C Python + Go @@ -47,6 +48,14 @@ + + + Apache TsFile + 2026-10-14 + 2.5.0 + + + Apache TsFile diff --git a/java/common/pom.xml b/java/common/pom.xml index ea093dbe2..937e0e784 100644 --- a/java/common/pom.xml +++ b/java/common/pom.xml @@ -24,7 +24,7 @@ org.apache.tsfile tsfile-java - 2.4.1-SNAPSHOT + 2.5.1-SNAPSHOT common TsFile: Java: Common diff --git a/java/examples/pom.xml b/java/examples/pom.xml index a56625c3d..8492736d4 100644 --- a/java/examples/pom.xml +++ b/java/examples/pom.xml @@ -24,7 +24,7 @@ org.apache.tsfile tsfile-java - 2.4.1-SNAPSHOT + 2.5.1-SNAPSHOT examples TsFile: Java: Examples @@ -36,7 +36,7 @@ org.apache.tsfile tsfile - 2.4.1-SNAPSHOT + 2.5.1-SNAPSHOT diff --git a/java/pom.xml b/java/pom.xml index c5d31ade5..892fd420c 100644 --- a/java/pom.xml +++ b/java/pom.xml @@ -24,10 +24,10 @@ org.apache.tsfile tsfile-parent - 2.4.1-SNAPSHOT + 2.5.1-SNAPSHOT tsfile-java - 2.4.1-SNAPSHOT + 2.5.1-SNAPSHOT pom TsFile: Java diff --git a/java/tools/pom.xml b/java/tools/pom.xml index 4d6bc9f71..a5057446b 100644 --- a/java/tools/pom.xml +++ b/java/tools/pom.xml @@ -24,7 +24,7 @@ org.apache.tsfile tsfile-java - 2.4.1-SNAPSHOT + 2.5.1-SNAPSHOT tools TsFile: Java: Tools @@ -32,7 +32,7 @@ org.apache.tsfile common - 2.4.1-SNAPSHOT + 2.5.1-SNAPSHOT commons-cli @@ -41,7 +41,7 @@ org.apache.tsfile tsfile - 2.4.1-SNAPSHOT + 2.5.1-SNAPSHOT ch.qos.logback diff --git a/java/tsfile/pom.xml b/java/tsfile/pom.xml index 0bcca3e4f..072f1bdb6 100644 --- a/java/tsfile/pom.xml +++ b/java/tsfile/pom.xml @@ -24,7 +24,7 @@ org.apache.tsfile tsfile-java - 2.4.1-SNAPSHOT + 2.5.1-SNAPSHOT tsfile TsFile: Java: TsFile @@ -38,7 +38,7 @@ org.apache.tsfile common - 2.4.1-SNAPSHOT + 2.5.1-SNAPSHOT com.github.luben diff --git a/pom.xml b/pom.xml index 0816e8137..c5fa08d0c 100644 --- a/pom.xml +++ b/pom.xml @@ -28,7 +28,7 @@ org.apache.tsfile tsfile-parent - 2.4.1-SNAPSHOT + 2.5.1-SNAPSHOT pom Apache TsFile Project Parent POM diff --git a/python/pom.xml b/python/pom.xml index 8de3d2fbb..1e11b3634 100644 --- a/python/pom.xml +++ b/python/pom.xml @@ -22,7 +22,7 @@ org.apache.tsfile tsfile-parent - 2.4.1-SNAPSHOT + 2.5.1-SNAPSHOT tsfile-python pom diff --git a/python/pyproject.toml b/python/pyproject.toml index cd28b3d54..a69422c25 100644 --- a/python/pyproject.toml +++ b/python/pyproject.toml @@ -30,7 +30,7 @@ build-backend = "setuptools.build_meta" [project] name = "tsfile" -version = "2.3.2.dev" +version = "2.5.1.dev" requires-python = ">=3.9" description = "TsFile Python" readme = {file = "README.md", content-type = "text/markdown"} diff --git a/python/setup.py b/python/setup.py index fe19fc91c..4736c46bd 100644 --- a/python/setup.py +++ b/python/setup.py @@ -30,7 +30,7 @@ ROOT = Path(__file__).parent.resolve() PKG = ROOT / "tsfile" -version = "2.3.2.dev" +version = "2.5.1.dev" def _find_cpp_build(): diff --git a/skills/tsfile/references/docs-map.yaml b/skills/tsfile/references/docs-map.yaml index ed6e21305..51153b215 100644 --- a/skills/tsfile/references/docs-map.yaml +++ b/skills/tsfile/references/docs-map.yaml @@ -25,10 +25,10 @@ path_bases: skill_baseline: scope: current-source source_ref: develop - maven_project_version: "2.3.2-SNAPSHOT" + maven_project_version: "2.5.1-SNAPSHOT" java_release: "17" - cpp_sdk_version: "2.3.2.dev" - python_package_version: "2.3.2.dev" + cpp_sdk_version: "2.5.1.dev" + python_package_version: "2.5.1.dev" official_url_prefixes: - https://tsfile.apache.org/