Repository navigation
Track starting and suspending states for the team broker agent - #8917
Merged
Merged
Conversation
A suspend arriving while start was still launching the agent found nothing to stop, then deleted the record, leaving the agent running untracked. The agent now stays starting until the driver has launched it and suspend replies 409 meanwhile; suspend marks it suspending first so a start in that window gets a 409 too. A failed launch removes the record instead of leaving it stuck in starting.
This was referenced Oct 14, 2026
cstns
marked this pull request as ready for review
October 14, 2026 10:19
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #8917 +/- ##
==========================================
+ Coverage 79.07% 79.15% +0.07%
==========================================
Files 476 476
Lines 25890 25903 +13
Branches 6886 6889 +3
==========================================
+ Hits 20473 20504 +31
+ Misses 5417 5399 -18
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
hardillb
approved these changes
Oct 14, 2026
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #8619
Merge order: FlowFuse/driver-localfs#363, FlowFuse/driver-docker#254 and FlowFuse/driver-k8s#415 go first and get released, then this one together with bumping the driver versions. On its own this PR is no worse than today, but it only closes the race once the drivers wait for the launch.
Starting the team broker and suspending it right after could leave the schema agent running with no record of it. The suspend landed while start was still launching the agent, so there was no pid to kill yet, and then the record got deleted. Only possible when the two calls overlap within a few ms, so API callers, not the UI.
The agent row now carries the state through the gap. Start creates it as
starting, waits for the driver to launch the agent, then setsrunning. A suspend while it'sstartinggets a 409agent_startinginstead of trying to stop something that isn't there yet. Suspend marks itsuspendingbefore stopping it, and a start in that window (before the row is removed) gets a 409agent_suspendingrather than seeing a "running" agent and doing nothing. If the launch fails the row is removed so it can't get stuck instarting, and a second start while one is alreadystartingis a no-op.Things worth a look:
TeamId. Adding one needs a look at existing data first (a migration would fail on startup if prod has duplicates), so left out.suspendingstill relies on the existing 1.5s timer that deletes the row.Added route tests for the starting/suspending states, the 409s and the failed launch, all failing before the change. Team broker specs pass. Also raced start and suspend against the local app (localfs) at offsets from 2 to 30ms: with this plus the driver change 0 orphans out of 24 tries and the overlapping suspends got 409s. With only the driver change it was 3 orphans out of 24, with only this one 1 out of 18, so both are needed.