Repository navigation
fix: writeAtomically fails for file names longer than 217 bytes - #20493
Merged
FrankChen021 merged 1 commit intoOct 8, 2026
Merged
Conversation
FileUtils.writeAtomically named its temporary file ".<fileName>.<uuid>", adding 38 characters to the target name. File names between 218 and 255 bytes were valid, but their temporary names were not, so the write failed with "File name too long". WorkerTaskManager persists tasks this way using the task ID as the file name, so tasks with such IDs, such as kill tasks on datasources with long names, could never start. Name the temporary file ".<uuid>" instead. The UUID keeps it unique, and the target file keeps its full name after the move. Fixes apache#20492.
RiccardoSale
commented
Oct 6, 2026
FrankChen021
approved these changes
Oct 7, 2026
FrankChen021
reviewed
Oct 7, 2026
FrankChen021
left a comment
Member
There was a problem hiding this comment.
🟢 Approval recommended
Reviewed all three changed files in the prepared diff:
indexing-service/src/test/java/org/apache/druid/indexing/worker/WorkerTaskManagerTest.javaprocessing/src/main/java/org/apache/druid/java/util/common/FileUtils.javaprocessing/src/test/java/org/apache/druid/java/util/common/FileUtilsTest.java
The filename-independent temporary name preserves the existing CREATE_NEW, write/fsync, atomic replacement, and cleanup flow while allowing targets at the filesystem component-length limit. I traced task assignment and completion persistence and searched the writeAtomically call sites; no affected behavior or compatibility issue surfaced.
Validation: git diff --check 87525a9c6d50474efc94cbc01706fa28c99b1dc1...HEAD passed. Static review only; no tests were run.
This is an automated review by Codex GPT-5.6-Luna(max)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #20492.
Description
FileUtils.writeAtomicallywrites to a temporary file named.<fileName>.<uuid>before moving it to the target. The temporary name adds 38 bytes. On filesystems with a filename limit of 255 bytes, valid filenames of 218–255 bytes therefore cause the write to fail withFile name too long.WorkerTaskManageruses this method to persist assigned and completed tasks, with the task ID as the filename. Tasks with IDs in this range cannot start on a MiddleManager or Indexer.Kill tasks reach this limit sooner because their IDs are longer than ingestion task IDs for the same datasource:
index_kafka_<dataSource>_<hash>_<random>index_parallel_<dataSource>_<random>_<createdTime>coordinator-issued_kill_<dataSource>_<random>_<intervalStart>_<intervalEnd>_<createdTime>As a result, a datasource can ingest successfully while its kill tasks consistently fail to start.
Changes
.<uuid>as the temporary filename so its length is independent of the target filename. The UUID preserves uniqueness, and the target retains its full name after the move. This allows task IDs up to the filesystem limit of 255 bytes.WorkerTaskManager.Proposed follow-up: kill task IDs without the interval
After this fix, kill tasks still support shorter datasource names than ingestion tasks: 147 characters, compared with 206 for batch ingestion and 218 for Kafka ingestion.
Would it make sense to aim for parity between ingestion and kill tasks in the datasource name lengths they support, so that any datasource that can be ingested can also be cleaned up?
The interval is already stored in the task payload.
Release note
Fixed: Tasks with IDs longer than 217 characters no longer fail to start on MiddleManagers and Indexers with
File name too long.Key changed/added classes in this PR
FileUtilsFileUtilsTestWorkerTaskManagerTestThis PR has: