Skip to content

fix: fail reporter worker when final result flush cannot send results - #354

Merged
dtice25 merged 3 commits into
masterfrom
devin/1787260876-reporter-flush-fail
Aug 21, 2026
Merged

fix: fail reporter worker when final result flush cannot send results#354
dtice25 merged 3 commits into
masterfrom
devin/1787260876-reporter-flush-fail

Conversation

@dtice25

@dtice25 dtice25 commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

Summary

BenchmarkReporter.flush catches sendTaskResults errors and leaves unsent records in this.pending, but finish still called completeWorker if flush(true) did not throw. This allowed a worker (and therefore the run) to be marked complete before all of its task results reached the platform, which could also make a run appear complete sooner than it actually was.

Changes:

  • Track flushFailed on the reporter and set it when any batch fails to send.
  • finish now checks flushFailed and pending.length after the final flush. If any result remains unsent, it calls failWorker instead of completeWorker.
  • Only call completeWorker when the final flush has accepted every pending result.
if (this.flushFailed || this.pending.length > 0) {
  await this.client.failWorker(..., new Error('Failed to flush task results'));
  return;
}
await this.client.completeWorker(...);

Prevents a worker from reporting success when its result batch could not be delivered.

Link to Devin session: https://app.devin.ai/sessions/06813b4b9fb64b60b1f47bd8e1e8f152
Requested by: @dtice25


Open in Devin Review

…ails

BenchmarkReporter.flush catches send errors and leaves the unsent batch in
pending, but finish would call completeWorker anyway if the final flush did
not throw. This could mark a worker (and therefore the run) completed before
all of its task results reached the platform.

Track flushFailed on the reporter. finish now calls failWorker when the final
flush leaves pending results or any batch failed to send, and only calls
completeWorker when every result was accepted.

Refs the investigation in https://app.devin.ai/sessions/06813b4b9fb64b60b1f47bd8e1e8f152

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@open-cla

open-cla Bot commented Aug 20, 2026

Copy link
Copy Markdown

Contributor License Agreement

All contributors are covered by a CLA.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Open in Devin Review

dtice25 and others added 2 commits August 21, 2026 15:14
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
BenchmarkReporter.flush catches send errors and leaves the unsent batch in
pending, but finish would call completeWorker anyway if the final flush did
not throw. This could mark a worker (and therefore the run) completed before
all of its task results reached the platform.

The reporter implementation has moved to @benchsdk/worker; apply the fix there.
Track flushFailed on the reporter, and have finish call failWorker when the
final flush leaves pending results or any batch failed to send. Only call
completeWorker when every result was accepted.

Refs the investigation in https://app.devin.ai/sessions/06813b4b9fb64b60b1f47bd8e1e8f152

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@dtice25
dtice25 merged commit 5820742 into master Aug 21, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant