[helm] Support multi-disk data.dirs for tablet servers - #4114
Draft
morazow wants to merge 2 commits into
Draft
Conversation
Add tablet.storage.volumes, a list of {name, size, storageClass} entries.
Each entry renders its own volumeClaimTemplate (or emptyDir when storage
is disabled) mounted at /tmp/fluss/<name>, and the chart emits data.dirs
with all mount paths in server.yaml so tablet servers balance data across
the disks. With the list empty, the chart renders exactly what it does
today: one 'data' claim sized by tablet.storage.size and no data.dirs
line, so existing releases see no StatefulSet storage change on upgrade.
Template-time validation rejects entries missing name or size, duplicate
or chart-reserved volume names, and a configurationOverrides data.dirs
that would conflict with the generated one.
Review findings on apache#4114: - Guard the data.dirs conflict check with 'default dict' so a nulled-out configurationOverrides renders instead of crashing hasKey (matches the chart's existing _metrics.tpl pattern). - Reject volume names that are not lowercase DNS-1123 labels at template time; previously a bad name passed lint and failed only at the API server, and a comma would silently corrupt the joined data.dirs value. - Emit claim size and storageClass via toYaml so string values that look like another YAML type (e.g. storageClass "1234") stay strings; nil storageClass still renders empty, keeping single-disk output identical. - Lock the volumeMounts order in the extraVolumeMounts test with a full equal assert instead of order-insensitive contains.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
tablet.storage.volumesto the Helm chart: a list of{name, size, storageClass}entries, each rendering its ownvolumeClaimTemplatesentry (oremptyDirwhentablet.storage.enabled=false) mounted at/tmp/fluss/<name>.data.dirswith all mount paths inserver.yaml, so tablet servers balance log and KV storage across the disks.data.dirstakes precedence overdata.dirserver-side, so the defaultconfigurationOverrides.data.dirstays harmless.main- onedataclaim sized bytablet.storage.size, nodata.dirsline - so existing releases see no StatefulSet storage change on upgrade.nameorsize, duplicate or chart-reserved volume names (fluss-conf,sasl-config,secret-*), and aconfigurationOverrides."data.dirs"that would conflict with the generated one.deploying-with-helm.md, including the migration caveat thatvolumeClaimTemplatesare immutable (orphan-delete the StatefulSet or install a fresh release).Test Plan
helm lint ./helm && helm unittest ./helm(whathelm-chart.yamlCI runs): 159 tests pass, including the newhelm/tests/storage_test.yamlcovering default emptyDir, legacy single-PVC, multi-volume claims + mounts +data.dirs, single-entry list, disabled-storage multi-emptyDir, coexistence withextraVolumes/extraVolumeMounts, and all five validation failures.helm templateoutput is byte-identical tomainfor default values and for the legacy single-disk configuration (enabled=true+size+storageClass).🤖 AI-assisted changes - reviewed by human developer