fix: emit specific error message when API rate limit retries are exhausted by devin-ai-integration[bot] · Pull Request #1004 · airbytehq/airbyte-python-cdk

devin-ai-integration · 2026-04-28T05:07:09Z

Summary

When HTTP 429 retries are exhausted, the CDK currently produces a generic Exhausted available request attempts. Please see logs for more details. Exception: HTTP Status Code: 429. Error: Too many requests. message — which the platform often overwrites with Something went wrong within the source connector. This gives users no indication that rate limiting is the root cause.

This PR detects 429 retry exhaustion in HttpClient._send_with_retry and raises a targeted AirbyteTracedException with:

message: API rate limit exceeded. (follows error message guidelines)
failure_type: transient_error (rate limiting is transient, not a system error)
internal_message: includes the raw exception for debugging

Non-rate-limit retry exhaustion is unchanged.

Resolves https://github.com/airbytehq/airbyte-internal-issues/issues/16258
Related to https://github.com/airbytehq/oncall/issues/12105

Review & Testing Checklist for Human

Verify the new rate-limit detection logic in http_client.py:303-306 correctly identifies 429 responses via e.response.status_code == requests.codes.too_many_requests
Confirm the user-facing message API rate limit exceeded. meets the error message guidelines and is a net improvement over the old generic message
Verify the transient_error FailureType is correct for rate-limit exhaustion (rate limits are transient, not config or system errors)

Notes

The fix checks the HTTP response status code (429) rather than the exception type, so it covers both exit_on_rate_limit=True (DefaultBackoffException) and exit_on_rate_limit=False (RateLimitBackoffException) paths
This is a CDK-level fix that benefits all connectors, not just source-gitlab
The original issue was triggered by source-gitlab, which uses the concurrent CDK with HTTPAPIBudget rate limiting

Link to Devin session: https://app.devin.ai/sessions/9f9470b028bc44949d86e9339f195908

…usted When HTTP 429 retries are exhausted, replace the generic 'Exhausted available request attempts' message with 'API rate limit exceeded.' and ensure the failure type is transient_error. This gives users a clear signal that the failure is due to rate limiting rather than an opaque internal error. Co-Authored-By: bot_apk <apk@cognition.ai>

devin-ai-integration · 2026-04-28T05:07:10Z

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

Disable automatic comment and CI monitoring

github-actions · 2026-04-28T05:07:18Z

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

Testing This CDK Version

You can test this version of the CDK using the following:

# Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@devin/1777352642-fix-rate-limit-error-message#egg=airbyte-python-cdk[dev]' --help

# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch devin/1777352642-fix-rate-limit-error-message

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

/autofix - Fixes most formatting and linting issues
/poetry-lock - Updates poetry.lock file
/test - Runs connector tests with the updated CDK
/prerelease - Triggers a prerelease publish with default arguments
/poe build - Regenerate git-committed build artifacts, such as the pydantic models which are generated from the manifest JSON schema in YAML.
/poe <command> - Runs any poe command in the CDK environment

📚 Show Repo Guidance

Helpful Resources

CDK API Reference

📝 Edit this welcome message.

github-actions · 2026-04-28T05:15:34Z

PyTest Results (Fast)

4 034 tests ±0 4 023 ✅ ±0 7m 44s ⏱️ -18s
1 suites ±0 11 💤 ±0
1 files ±0 0 ❌ ±0

Results for commit 7221854. ± Comparison against base commit 60bae81.

♻️ This comment has been updated with latest results.

Co-Authored-By: bot_apk <apk@cognition.ai>

github-actions · 2026-04-28T05:30:56Z

PyTest Results (Full)

4 037 tests ±0 4 025 ✅ ±0 11m 38s ⏱️ +41s
1 suites ±0 12 💤 ±0
1 files ±0 0 ❌ ±0

Results for commit 7221854. ± Comparison against base commit 60bae81.

Copilot

Pull request overview

This PR improves user-facing error reporting when HTTP 429 (rate limiting) retries are exhausted in the CDK HttpClient, emitting a targeted AirbyteTracedException so the root cause isn’t obscured by generic platform errors.

Changes:

Detects retry exhaustion caused by HTTP 429 in HttpClient._send_with_retry and raises an AirbyteTracedException with message API rate limit exceeded. and FailureType.transient_error.
Keeps existing behavior for non-429 retry exhaustion unchanged.
Updates unit tests to assert the new user-facing message for 429 exhaustion scenarios.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
`airbyte_cdk/sources/streams/http/http_client.py`	Adds 429 exhaustion detection and throws a more specific traced exception with transient failure type.
`unit_tests/sources/streams/http/test_http.py`	Updates expectations to match the new rate-limit error message.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-28T16:25:57Z

    with pytest.raises(
-        AirbyteTracedException, match="Exception: HTTP Status Code: 429. Error: Too many requests."
+        AirbyteTracedException,
+        match="API rate limit exceeded.",
    ):


pytest.raises(..., match=...) uses a regex. The trailing . in "API rate limit exceeded." will match any character, so this assertion could pass even if the message changes slightly. Consider escaping the dot (e.g., matching a literal period) or using re.escape(...) to ensure the test asserts the exact message.

Copilot · 2026-04-28T16:25:57Z

    with pytest.raises(
        AirbyteTracedException,
-        match="Exhausted available request attempts. Please see logs for more details. Exception: HTTP Status Code: 429. Error: Too many requests.",
+        match="API rate limit exceeded.",
    ):


pytest.raises(..., match=...) treats match as a regex; the . at the end of "API rate limit exceeded." matches any character. Escaping the period (or using re.escape) would make this assertion stricter and avoid accidental false positives.

devin-ai-integration Bot and others added 2 commits April 28, 2026 05:16

fix: update remaining test asserting old 429 message

0d87243

Co-Authored-By: bot_apk <apk@cognition.ai>

style: fix ruff formatting

7221854

Co-Authored-By: bot_apk <apk@cognition.ai>

Patrick Nilan (pnilan) approved these changes Apr 28, 2026

View reviewed changes

Patrick Nilan (pnilan) marked this pull request as ready for review April 28, 2026 16:22

Copilot AI review requested due to automatic review settings April 28, 2026 16:22

Patrick Nilan (pnilan) merged commit 3f13144 into main Apr 28, 2026
29 checks passed

Patrick Nilan (pnilan) deleted the devin/1777352642-fix-rate-limit-error-message branch April 28, 2026 16:23

Copilot started reviewing on behalf of Patrick Nilan (pnilan) April 28, 2026 16:23

View session

Copilot AI reviewed Apr 28, 2026

View reviewed changes

devin-ai-integration Bot mentioned this pull request May 18, 2026

fix(source-trello): Improve exhausted request retry errors airbytehq/airbyte#78156

Draft

1 of 2 tasks

Navigation Menu

Repository navigation

fix: emit specific error message when API rate limit retries are exhausted - #1004

fix: emit specific error message when API rate limit retries are exhausted#1004
Patrick Nilan (pnilan) merged 3 commits into
mainfrom
devin/1777352642-fix-rate-limit-error-message

devin-ai-integration Bot commented Apr 28, 2026

Uh oh!

devin-ai-integration Bot commented Apr 28, 2026

Uh oh!

github-actions Bot commented Apr 28, 2026

Testing This CDK Version

PR Slash Commands

Helpful Resources

Uh oh!

github-actions Bot commented Apr 28, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Apr 28, 2026

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 28, 2026

Uh oh!

Copilot AI Apr 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

devin-ai-integration Bot commented Apr 28, 2026

Summary

Review & Testing Checklist for Human

Notes

Uh oh!

devin-ai-integration Bot commented Apr 28, 2026

🤖 Devin AI Engineer

Uh oh!

github-actions Bot commented Apr 28, 2026

👋 Greetings, Airbyte Team Member!

Testing This CDK Version

PR Slash Commands

Helpful Resources

Uh oh!

github-actions Bot commented Apr 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PyTest Results (Fast)

Uh oh!

github-actions Bot commented Apr 28, 2026

PyTest Results (Full)

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

github-actions Bot commented Apr 28, 2026 •

edited

Loading