Skip to content

fix: preserve SDK error results through typed teardown exceptions - #85

Merged
svozza merged 1 commit into
mainfrom
fix/sdk-result-error-recovery
Sep 28, 2026
Merged

svozza merged 1 commit into
mainfrom
fix/sdk-result-error-recovery

Conversation

@svozza

@svozza svozza commented Sep 28, 2026 •

Copy link
Copy Markdown
Owner

After SDK 0.2.160, a Bedrock API failure can emit an error ResultMessage followed by the SDK's new ResultError. Aceiro recognized only the previous bare-exception wrapper. Because ResultError inherits ProcessError, the run incorrectly reported “without a result envelope,” skipped its API-error record and retry path, and could discard an already verified artifact.

This occurred in scheduled run 36433006668: 41/42 review scenarios and 6/6 plan scenarios passed; sandbox_credential_read submitted incomplete arguments, then exhausted ten CLI retries on Bedrock 503s. The failed sample remains invalid and its evidence is retained.

Recognize the typed SDK wrapper only after capturing an error result, so the existing retry, permanent-error and verified-artifact rules can handle it. Plain process crashes, programming errors and absent/successful envelopes remain failures. No model, prompt, fixture, grader or retry-budget changes.

Validation:

  • 2,376 deterministic tests passed; two fixture-dependent skips. Type checking passed.
  • Regression tests failed on main before the fix, then passed for retrying 503s, preserving verified artifacts and diagnosing non-retryable permission errors.
  • A custom in-memory transport exercises the installed SDK's actual message reader and exit-wrapper behavior, in addition to scripted recovery tests.
  • All 42 review and six remediation scenarios passed three repetitions with unchanged inputs: 126/126 review and 18/18 plan samples using Opus 5.5/high.
  • GitHub CI passed on the same commit, including 42/42 review and 6/6 plan samples (run 36436709851).
  • One additional 503 recovered inside the CLI during local validation; the deterministic regression tests cover exhausted internal retries and the resulting SDK exception.

The api_error metric continues to count terminal session API failures. Internal CLI retries remain visible in the captured stream.

@svozza
svozza deployed to ai-pr-review September 28, 2026 14:32 — with GitHub Actions Active
@svozza
svozza deployed to ai-pr-review-runtime September 28, 2026 14:34 — with GitHub Actions Active
@svozza
svozza marked this pull request as ready for review September 28, 2026 14:52
@svozza
svozza merged commit 07ee7ca into main Sep 28, 2026
11 checks passed

This branch was successfully deployed

2 active deployments
ai-pr-review-runtime — 94d67d78 Deployed Sep 28, 2026 by svozza via evals #190
ai-pr-review — 94d67d78 Deployed Sep 28, 2026 by svozza via eval_approve #190
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant