Skip to content

Gradle WrapperDownloader: move URL & sha into properties #16521

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
dsmiley wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Gradle WrapperDownloader: move URL & sha into properties #16521

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6b86907
Gradle WrapperDownloader: move URL & sha into properties
dsmiley Aug 18, 2026
21f1588
UTF8 properties
dsmiley Aug 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/actions/prepare-for-build/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ runs:
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: gradle/wrapper/gradle-wrapper.jar
key: gradle-wrapper-${{ hashFiles('gradle/wrapper/gradle-wrapper.jar.sha256') }}
key: gradle-wrapper-${{ hashFiles('gradle/wrapper/gradle-wrapper.properties') }}

# This includes "smart" caching of gradle dependencies.
- name: Set up Gradle
Expand Down
84 changes: 18 additions & 66 deletions build-tools/build-infra/src/main/java/org/apache/lucene/gradle/WrapperDownloader.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.io.Reader;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
Expand All @@ -35,10 +36,8 @@
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.Locale;
import java.util.Objects;
import java.util.Properties;
import java.util.concurrent.TimeUnit;
import java.util.logging.Logger;
import java.util.regex.Pattern;

/**
* Standalone class used to download the {@code gradle-wrapper.jar}.
Expand Down Expand Up @@ -81,70 +80,19 @@ public static void checkVersion() {
}

public void run(Path destination) throws IOException, NoSuchAlgorithmException {
var expectedFileName = destination.getFileName().toString();
Path checksumPath = destination.resolveSibling(expectedFileName + ".sha256");
if (!Files.exists(checksumPath)) {
throw new IOException("Checksum file not found: " + checksumPath);
}

String expectedChecksum;
try (var lines = Files.lines(checksumPath, StandardCharsets.UTF_8)) {
expectedChecksum =
lines
.map(
line -> {
// "The default mode is to print a line with: checksum, a space,
// a character indicating input mode ('*' for binary, ' ' for text
// or where binary is insignificant), and name for each FILE."
var spaceIndex = line.indexOf(" ");
if (spaceIndex != -1 && spaceIndex + 2 < line.length()) {
var mode = line.charAt(spaceIndex + 1);
String fileName = line.substring(spaceIndex + 2);
if (mode == '*' && fileName.equals(expectedFileName)) {
return line.substring(0, spaceIndex);
}
}

Logger.getLogger(WrapperDownloader.class.getName())
.warning(
"Something is wrong with the checksum file. Regenerate with "
+ "'sha256sum -b gradle-wrapper.jar > gradle-wrapper.jar.sha256'");
return null;
})
.filter(Objects::nonNull)
.findFirst()
.orElse(null);

if (expectedChecksum == null) {
throw new IOException(
"The checksum file did not contain the expected checksum for '"
+ expectedFileName
+ "'?");
}
}

Path wrapperProperties =
destination.resolveSibling(
destination.getFileName().toString().replace(".jar", ".properties"));
if (!Files.exists(wrapperProperties)) {
throw new IOException("Wrapper property file not found: " + wrapperProperties);
}

Pattern versionPattern = Pattern.compile("gradle-(?<version>.+?)-bin.zip");

@dsmiley dsmiley Aug 18, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: this pattern was flawed because the . can match a \

@dweiss dweiss Aug 19, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm.... What do you mean? It doesn't have a DOTALL flag, it should work just fine?

@dsmiley dsmiley Aug 19, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The regexp is used with matcher.find(), which returns the LEFTMOST match. The non-greedy
.+? doesn't save you here, because . also matches / -- so the capture happily runs across path separators.

@dweiss dweiss Aug 20, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know what it does but I can't see how you're hitting a problem here - I suspect you're modifying this in place:

distributionUrl=https\://services.gradle.org/distributions/gradle-9.7.0-bin.zip

and your corporate url doesn't match the pattern, right? Sorry for being dim. Can you give me an example of when this pattern fails to work?

Anyway, this isn't a solution in the long term because this file is versioned... It'll be a nightmare for you to have to adjust it everywhere.

I've gone through gradle's docs, issues and the wrapper code and I wonder how anybody in a corporate environment is solving the problem of auto-gradle-distro-installation (wrapper jar aside). There seem to be only two options that I see:

  1. use http proxy props and a proxy server to redirect gradle's "official" URLs to another location; this is tricky with https certs,
  2. store the binary distribution with the code (yes, it's possible). This dodges the download problem entirely.

It's interesting to me that there seems to be no way of redirecting these URLs "dynamically" -- seems like people behind firewalls need to install gradle manually (?). I guess adding a property to download something from arbitrary locations may be perceived as a security issue, don't know.

Looking at what's inside the gradle wrapper code, we may indeed try to simulate the same runtime behavior (using much less code) but skipping gradle-wrapper.jar entirely will break higher-level tools that depend on it (like intellij) so I think it needs to be there.

String wrapperVersion =
Files.readAllLines(wrapperProperties, StandardCharsets.UTF_8).stream()
.map(
line -> {
var matcher = versionPattern.matcher(line);
if (matcher.find()) {
return matcher.group("version");
} else {
return null;
}
})
.filter(Objects::nonNull)
.findAny()
.orElseThrow();
Properties properties = new Properties();
try (Reader in = Files.newBufferedReader(wrapperProperties, StandardCharsets.UTF_8)) {
properties.load(in);
}
String wrapperUrl = requireProperty(properties, wrapperProperties, "wrapperUrl");
String expectedChecksum = requireProperty(properties, wrapperProperties, "wrapperSha256");

MessageDigest digest = MessageDigest.getInstance("SHA-256");

Expand All @@ -159,12 +107,7 @@ public void run(Path destination) throws IOException, NoSuchAlgorithmException {
}
}

URL url =
URI.create(
"https://raw.githubusercontent.com/gradle/gradle/v"
+ wrapperVersion
+ "/gradle/wrapper/gradle-wrapper.jar")
.toURL();
URL url = URI.create(wrapperUrl).toURL();
System.err.println("Downloading gradle-wrapper.jar from " + url);

// Zero-copy save the jar to a temp file
Expand Down Expand Up @@ -242,6 +185,15 @@ public void run(Path destination) throws IOException, NoSuchAlgorithmException {
}
}

private static String requireProperty(Properties properties, Path source, String key)
throws IOException {
String value = properties.getProperty(key);
if (value == null || value.isBlank()) {
throw new IOException("Missing required property '" + key + "' in " + source);
}
return value.trim();
}

@SuppressForbidden(reason = "Valid use of thread.sleep.")
private static void sleep(long millis) throws InterruptedException {
Thread.sleep(millis);
Expand Down
1 change: 0 additions & 1 deletion gradle/wrapper/gradle-wrapper.jar.sha256
View file
Open in desktop

This file was deleted.

5 changes: 5 additions & 0 deletions gradle/wrapper/gradle-wrapper.properties
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,8 @@ retryBackOffMs=500
validateDistributionUrl=true
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists

# Read by Lucene's WrapperDownloader (not by the actual wrapper) to bootstrap gradle-wrapper.jar.
wrapperSha256=497c8c2a7e5031f6aa847f88104aa80a93532ec32ee17bdb8d1d2f67a194a9c7
wrapperUrl=https\://raw.githubusercontent.com/gradle/gradle/v9.6.1/gradle/wrapper/gradle-wrapper.jar
# To self download, try %> curl -Lo gradle/wrapper/gradle-wrapper.jar <wrapperUrl> (from the repo root)

@dsmiley dsmiley Aug 18, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this comment is helpful to anyone who wants to download it themselves (me). I have proxy env vars.

@dweiss dweiss Aug 19, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't put anything in here. This is controlled by gradle - when you update the wrapper, it'll likely get overwritten and people use llms more and more for such stuff... Likely to break in my opinion.

9 changes: 2 additions & 7 deletions gradlew
View file
Open in desktop

@dsmiley dsmiley Aug 18, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer not validating that an existing gradle wrapper matches the sha. Why bother checking this? If the user has a wrapper (whatever version/origin) that continues to work for them, let them continue to use it. If they don't like it anymore, they are free to remove/replace it.

Admittedly I was primarily motivated by the simplicity of using gradle-wrapper.properties for everything the downloader and the wrapper itself needs to know..

@dweiss dweiss Aug 19, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The sha checking code is there because if you're switching branches (for example, the wrapper version changes), you want to update the wrapper jar too. It is a stricter check than just file timestamps.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

14 changes: 2 additions & 12 deletions gradlew.bat
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading