Repository navigation
feat(changelog): Make the AI summary timeout configurable - #967
Merged
Merged
Conversation
Large changelogs that summarize every scope and category can need more than the fixed 120-second budget; getsentry/toolkit's CLI 0.48.0 release ran out of it during the category stage and fell back to the original entries. Add aiSummary.timeout (seconds, default 120), report a single slow call as request-timeout instead of deadline, and log each model call's duration at debug level. Co-Authored-By: BYK <126780+BYK@users.noreply.github.com>
Contributor
|
BYK
approved these changes
Nov 5, 2026
BYK
left a comment
BYK
left a comment
Member
There was a problem hiding this comment.
I think we should be able to disable the timeout too (infinite)
BYK
pushed a commit
that referenced
this pull request
Nov 6, 2026
Follow-up to #967, from review feedback asking for a way to turn the total timeout off. - `aiSummary.timeout: 0` removes the total budget. The 30-second per-request limit and the 64-request cap still apply. - Other values must still be whole numbers of at least 30. I dropped the 1800-second cap, since a limit like that no longer makes sense once there's no limit at all. - Updated the docs table and the limits section. Tests: the long-running summary test now also runs with `timeout: 0`. The config tests accept 0, 30, 600 and 3600 and reject -1, 29, 1.5 and `"600"`. `tsc` and `eslint` pass, as do the AI summary, config, changelog, HTTP and action suites. <!-- junior-session-footer:start --> <!-- junior-conversation-id:local%3Aweb%3Aaaa2b8ba1c1b8915ae7daa2e --> -- [View Session in Sentry](https://sentry.sentry.io/explore/conversations/local%3Aweb%3Aaaa2b8ba1c1b8915ae7daa2e/?project=4510944073809921) <!-- junior-session-footer:end --> Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com>
8 tasks done
BYK
added a commit
to getsentry/toolkit
that referenced
this pull request
Nov 6, 2026
CLI 0.48.0 shipped without AI changelog summaries. In the [prepare run](https://github.com/getsentry/toolkit/actions/runs/42097652931), Craft 3.10.0 started summarizing at 19:50:34.8 and gave up exactly 120s later: ``` [warn] AI changelog summary failed; keeping original entries. (stage=category; reason=deadline) ``` That 120s is a fixed total budget in Craft. The model and the token are fine: scope summaries completed. With `levels: [release, category, scope]`, `minItems: 0` and ~330 entries since 0.47.0, the run just needs more than 120s. This PR sets the new `aiSummary.timeout: 600` for the CLI workspace and bumps `minVersion` to `3.11.0`. **Draft: merge only after getsentry/craft#967 ships in a Craft release (expected 3.11.0).** Craft validates `aiSummary` strictly, so if this lands before that release, CLI prepare and publish will reject the config. <!-- junior-request-attribution:start --> via @BYK. <!-- junior-request-attribution:end --> <!-- junior-session-footer:start --> <!-- junior-conversation-id:local%3Aweb%3Aaaa2b8ba1c1b8915ae7daa2e --> -- [View Session in Sentry](https://sentry.sentry.io/explore/conversations/local%3Aweb%3Aaaa2b8ba1c1b8915ae7daa2e/?project=4510944073809921) <!-- junior-session-footer:end --> Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com> Co-authored-by: BYK <126780+BYK@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AI changelog summaries had a fixed 120-second total budget. That's too short for large changelogs that summarize every scope and category.
getsentry/toolkit's CLI 0.48.0 prepare (run) used
levels: [release, category, scope],minItems: 0and covered ~330 entries across ~30 scope groups. The summary started at 19:50:34.8 and failed at 19:52:34.9, exactly 120s later, with:So scope summaries finished, and the run ran out of total budget during the category stage. No provider or validation error was reported.
Changes
aiSummary.timeoutsetting: the total budget in seconds. Integer, 30–1800, default120, so existing configs behave the same.reason=request-timeoutinstead ofreason=deadline.deadlinenow always means the total budget ran out. Before this change the two looked the same in the warning.After this ships, toolkit can set
timeout(and bumpminVersion) for its CLI workspace.Tests
timeout: 600. It fails with the default."600".request-timeout.tsc,eslintand the AI summary, config, changelog and HTTP suites pass. In the sandbox,prepare-dry-run.e2e,npm.testandgit.testfail the same way onmaster, because of a Git identity rule and the environment. These changes don't cause those failures.via @BYK.
--
View Session in Sentry