Skip to content

Fix ML container publishing in push3XX tasks - #40343

Merged
damccorm merged 1 commit into
apache:masterfrom
damccorm:fix-ml-container-push
Sep 29, 2026
Merged

damccorm merged 1 commit into
apache:masterfrom
damccorm:fix-ml-container-push

Conversation

@damccorm

Copy link
Copy Markdown
Contributor

Context & Root Cause

ML containers haven't been published since 2.73.0 because of a lazy task-configuration closure bug in sdks/python/container/build.gradle:

for(int i=min_python_version; i<=max_python_version; ++i) {
  String min_version = "3" + min_python_version
  String cur = "3" + i
  String prev = "3" + (i-1)
  tasks.register("push" + cur) {
    ...
    dependsOn ':sdks:python:container:py' + cur + ':docker'
    // 3.14 does not have dependencies for ML yet.
    if (project.hasProperty("include-ml") && i < 14) {
      dependsOn ':sdks:python:container:ml:push' + cur
    }
    if (project.hasProperty("include-distroless")) {
      dependsOn ':sdks:python:container:distroless:push' + cur
    }
  1. tasks.register("push" + cur) { ... } configures the task lazily—the closure only executes when the task (e.g. :sdks:python:container:push310) is realized, which happens after the for loop has already finished.
  2. While String cur = "3" + i is a fresh local variable scoped to each loop iteration, the loop counter i is a single mutable variable shared across the loop and captured by reference in the closure.
  3. Since max_python_version is 14, by the time the tasks.register closure runs for any push3XX task, the loop has completed and i is 15.
  4. As a result, i < 14 evaluates 15 < 14 (false) for every Python version (push310, push311, push312, push313), so :sdks:python:container:ml:push3XX is never added to dependsOn.

This was introduced in commit 86ca83e81dd (#38221 on April 17, 2026), which is why 2.73.0 (cut April 16, 2026) was the last release with tags pushed to apache/beam_python3.*_sdk_ml on Docker Hub.

Fix

Check cur != "314" instead of i < 14 inside the tasks.register closure so the iteration-scoped cur variable is used when the task is realized.

@damccorm

Copy link
Copy Markdown
Contributor Author

R: @jrmccluskey

@github-actions

Copy link
Copy Markdown
Contributor

Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment assign set of reviewers

@damccorm
damccorm merged commit 7df6f2f into apache:master Sep 29, 2026
90 of 91 checks passed
@damccorm
damccorm deleted the fix-ml-container-push branch September 29, 2026 19:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants