Skip to content

fix(up): check a depends_on condition before the first poll tick - #14385

Merged
ndeloof merged 2 commits into
docker:mainfrom
htoyoda18:fix/wait-dependency-check-first
Oct 27, 2026
Merged

ndeloof merged 2 commits into
docker:mainfrom
htoyoda18:fix/wait-dependency-check-first

Conversation

@htoyoda18

@htoyoda18 htoyoda18 commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

What this PR does

When a service waits on a dependency with condition: service_healthy or service_completed_successfully, Compose always waited for one 500ms poll interval before it checked the dependency for the first time, even when the dependency already satisfied the condition. This PR checks these two conditions as soon as the wait starts, so a no-op docker compose up -d goes from ~0.6s to ~0.07s. The running_or_healthy check used by up --wait still waits for the first tick, so --wait keeps reporting a container that exits right after starting as a failure.

Related issue

Fixes #14384

Changes made

  • pkg/compose/service_containers.go: waitDependency checks service_healthy and service_completed_successfully on entry, without waiting for a tick. running_or_healthy (used by --wait, and for dependencies of a project rebuilt from container labels) is still first checked on the first tick. It accepts a running container that has no healthcheck, so a check right after start would report a container that exits within the interval as running. If the context is already done on entry, the wait still ends through the existing ctx.Done() branch without inspecting the dependency: a cancellation stays silent, and a deadline keeps its current meaning.
  • pkg/compose/service_containers_test.go: add TestWaitDependencyFirstCheck, which uses testing/synctest to lock when the first check of an already satisfied dependency happens: after 0s for service_healthy and service_completed_successfully, and after exactly 500ms for running_or_healthy with a running container that has no healthcheck. TestWaitDependencyDeadline and TestOptionalDependencyInspectFailure assumed that a 50ms timeout always expires before any check. They now let the check on entry find the dependency still starting, so they still cover a deadline that expires between two polls.
  • pkg/compose/executor_start_test.go: rewrite a comment that said the wait "never" checks on entry.
  • pkg/e2e/testdata/TestUpWithDependencyExit/compose.yaml: give db a healthcheck. Without one, the test's expected exited (1) only won over has no healthcheck configured because the first check came 500ms after start.

Testing done

  • Unit tests pass: make test
  • E2E tests pass: make e2e-compose or make e2e-compose-standalone
  • Linting passes: make lint
  • Code formatted: make fmt
  • Manually tested (describe):

AI tool used (if applicable)

AI Tool:

Additional context

(not mandatory) A picture of a cute animal, if possible in relation to what you did

image

@htoyoda18
htoyoda18 force-pushed the fix/wait-dependency-check-first branch from 23db0f0 to 60cc2d6 Compare October 24, 2026 15:36
@htoyoda18
htoyoda18 marked this pull request as ready for review October 25, 2026 11:03
@htoyoda18
htoyoda18 requested review from a team as code owners October 25, 2026 11:03
@htoyoda18
htoyoda18 requested review from glours and ndeloof October 25, 2026 11:03
@ndeloof

ndeloof commented Oct 26, 2026

Copy link
Copy Markdown
Contributor

LGTM, thanks for the fix and for locking the first-check timing in a test.

I went through the side effects I could think of and found none: the check* functions are untouched, only the instant of their first call changes for service_healthy and service_completed_successfully, and running_or_healthy (--wait) keeps waiting for the first tick, which is the right call.

Could you rebase on main? #14376 has been merged since and it reworked the loop in waitDependency, so the PR now conflicts in service_containers.go. One thing to keep when resolving: on main the select on the ticker and ctx.Done() is followed by an if ctx.Err() != nil, so that a canceled wait is never run through one more check when both are ready together. The check on entry can simply skip the select on the first iteration and fall through to that same ctx.Err() test.

I tried that resolution locally on top of main: the build, go test ./pkg/compose/ -race and the lint (only the two existing errcheck) are all fine, and the wait/executor tests are stable over repeated runs.

@htoyoda18
htoyoda18 force-pushed the fix/wait-dependency-check-first branch from 60cc2d6 to 48a805a Compare October 26, 2026 09:43
@htoyoda18

Copy link
Copy Markdown
Contributor Author

@ndeloof Thanks for the review! Rebased on main and resolved the conflict as you suggested: on the first iteration, service_healthy and service_completed_successfully skip the select and fall through to the same ctx.Err() test, so the extra ctx.Err() != nil condition from the previous version is gone. running_or_healthy still waits for the first tick. I also trimmed the comments I had added.

@ndeloof
ndeloof enabled auto-merge (rebase) October 26, 2026 10:23
ndeloof
ndeloof previously approved these changes Oct 26, 2026
auto-merge was automatically disabled October 26, 2026 10:47

Head branch was pushed to by a user without write access

@htoyoda18
htoyoda18 force-pushed the fix/wait-dependency-check-first branch from 48a805a to 8f35601 Compare October 26, 2026 10:47
@htoyoda18

Copy link
Copy Markdown
Contributor Author

@ndeloof

Follow-up: CI caught something my local run missed. TestUpWithDependencyExit waits with service_healthy on a dependency that has no healthcheck and exits at once. The first check now often runs before the exit, so it reports has no healthcheck configured instead of exited (1). up still fails, only the message differs.

I gave db a healthcheck in the test's compose file, since the scenario is "exits before turning healthy". With it, the test reports exited (1) whatever the timing.

@htoyoda18
htoyoda18 requested a review from ndeloof October 26, 2026 10:51
Signed-off-by: hiroto.toyoda <hiroto.toyoda@dena.com>
@htoyoda18
htoyoda18 force-pushed the fix/wait-dependency-check-first branch from 44c5500 to 87917c0 Compare October 26, 2026 18:57
@ndeloof
ndeloof enabled auto-merge (rebase) October 27, 2026 07:51
@codecov

codecov Bot commented Oct 27, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@ndeloof ndeloof left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @htoyoda18, the change itself is good and everything else checks out. One blocking point though: this PR makes an existing test flaky.

TestExecutePlanCanceledCallerReportsNothingMore (pkg/compose/executor_start_test.go) now fails intermittently under -race with api: [Starting] instead of no event:

  • on main: 0 failures out of 150 runs (-race, GOMAXPROCS=2)
  • on this branch: 20 failures out of 300 (same conditions), about 1 in 300 without the CPU constraint

Repro: GOMAXPROCS=2 go test ./pkg/compose/ -race -count=300 -run 'TestExecutePlanCanceledCallerReportsNothingMore$'

Why: the test cancels the caller's context from inside the first ContainerInspect of the wait. Before this PR that inspection only happened after the first 500ms tick, so the api start node was long parked in its dependency select and ctx.Done() woke it up first. Now the inspection happens on entry, so the cancel can land before that goroutine reaches its select. When it does, done and ctx.Done() are both ready, select picks at random, and api emits a Starting event after the cancellation. The test was relying on that timing without saying so.

The race in runNode itself is not introduced by this PR (it is the same pattern waitDependencyReached fixed by letting ctx win), and I will handle it separately. For this PR the test just needs to stop depending on scheduling. One way: run it under synctest.Test as you already do in TestWaitDependencyFirstCheck, make the mocked inspection wait on a channel, call synctest.Wait() from the test goroutine (it returns once the api node is durably blocked on its dependency), then cancel() and release the inspection.

Please also run the test with -race -count=300 and GOMAXPROCS=2 once fixed.

…rst inspection

Signed-off-by: hiroto.toyoda <hiroto.toyoda@dena.com>
auto-merge was automatically disabled October 27, 2026 08:35

Head branch was pushed to by a user without write access

@htoyoda18

Copy link
Copy Markdown
Contributor Author

@ndeloof Thanks! I fixed the test under synctest.Test as you suggested, in a separate commit so the re-indentation is easy to review (happy to squash before merge). With GOMAXPROCS=2 -race -count=300, it now passes 600/600 (28/300 failed before).

@htoyoda18
htoyoda18 requested a review from ndeloof October 27, 2026 08:37
@ndeloof
ndeloof merged commit b1375cb into docker:main Oct 27, 2026
62 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] depends_on health/completion waits always sleep 500ms before the first check

2 participants