Skip to content

Commit dcb4216

Browse files
committed
[SPARK-56493][K8S] Replace GzipCompressorOutputStream with Java GZIPOutputStream
### What changes were proposed in this pull request? This PR replaces `org.apache.commons.compress.compressors.gzip.GzipCompressorOutputStream` with Java's built-in `java.util.zip.GZIPOutputStream` in K8s integration test `Utils`, and adds a Scalastyle rule to ban `org.apache.commons.compress.compressors` package. ### Why are the changes needed? To reduce reliance on the `commons-compress` library by using the Java standard library equivalent. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Pass the CIs with the existing tests. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 4.6) Closes #55357 from dongjoon-hyun/SPARK-56493. Authored-by: Dongjoon Hyun <dongjoon@apache.org> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
1 parent a1f99b4 commit dcb4216

2 files changed

Lines changed: 7 additions & 3 deletions

File tree

resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/Utils.scala

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -20,14 +20,13 @@ import java.io.{Closeable, File, FileInputStream, FileOutputStream, PrintWriter}
2020
import java.nio.charset.StandardCharsets
2121
import java.nio.file.{Files, Path}
2222
import java.util.concurrent.CountDownLatch
23-
import java.util.zip.{ZipEntry, ZipOutputStream}
23+
import java.util.zip.{GZIPOutputStream, ZipEntry, ZipOutputStream}
2424

2525
import scala.jdk.CollectionConverters._
2626

2727
import io.fabric8.kubernetes.client.dsl.ExecListener
2828
import io.fabric8.kubernetes.client.dsl.ExecListener.Response
2929
import org.apache.commons.compress.archivers.tar.{TarArchiveEntry, TarArchiveOutputStream}
30-
import org.apache.commons.compress.compressors.gzip.GzipCompressorOutputStream
3130
import org.apache.commons.io.output.ByteArrayOutputStream
3231

3332
import org.apache.spark.{SPARK_VERSION, SparkException}
@@ -158,7 +157,7 @@ object Utils extends Logging {
158157
) { fis =>
159158
Utils.tryWithResource(
160159
new TarArchiveOutputStream(
161-
new GzipCompressorOutputStream(
160+
new GZIPOutputStream(
162161
new FileOutputStream(oFile)))
163162
) { tOut =>
164163
val tarEntry = new TarArchiveEntry(fileToTarGz, fileToTarGz.getName)

scalastyle-config.xml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -412,6 +412,11 @@ This file is divided into 3 sections:
412412
<customMessage>Use contentEquals of SparkFileUtils or Utils instead.</customMessage>
413413
</check>
414414

415+
<check customId="commonscompresscompressors" level="error" class="org.scalastyle.file.RegexChecker" enabled="true">
416+
<parameters><parameter name="regex">org\.apache\.commons\.compress\.compressors\b</parameter></parameters>
417+
<customMessage>Use Java API instead</customMessage>
418+
</check>
419+
415420
<check customId="commonslang2" level="error" class="org.scalastyle.file.RegexChecker" enabled="true">
416421
<parameters><parameter name="regex">org\.apache\.commons\.lang\.</parameter></parameters>
417422
<customMessage>Use Commons Lang 3 classes (package org.apache.commons.lang3.*) instead

0 commit comments

Comments
 (0)