Skip to content

feat(declarative): add RESET_OPAQUE_TOKEN recovery for OpaqueTokenCursor (2/2) - #1235

Draft
devin-ai-integration[bot] wants to merge 2 commits into
devin/1792189821-opaque-token-cursorfrom
devin/1792189821-opaque-token-reset
Draft

devin-ai-integration[bot] wants to merge 2 commits into
devin/1792189821-opaque-token-cursorfrom
devin/1792189821-opaque-token-reset

Conversation

@devin-ai-integration

Copy link
Copy Markdown
Contributor

Summary

Adds recovery for rejected or expired opaque tokens, for example Ashby sync_token_expired. This is PR 2 of 2, stacked on #1234 (base branch devin/1792189821-opaque-token-cursor).

Resolves https://github.com/airbytehq/airbyte-internal-issues/issues/17213:

Close on merge: yes — internally filed, no customer to confirm with.

Changes

  • ResponseAction.RESET_OPAQUE_TOKEN, plus the schema and model enum entries, so an HttpResponseFilter can map a token rejection to the new action:
    - type: HttpResponseFilter
      action: RESET_OPAQUE_TOKEN
      predicate: "{{ response.get('errorInfo', {}).get('code') in ['sync_token_expired', 'sync_token_invalid'] }}"
  • HttpClient raises OpaqueTokenResetRequiredException for that action, the same way RESET_PAGINATION does. CompositeErrorHandler keeps the action instead of falling through.
  • SimpleRetriever._read_records_or_split_request_window catches the exception and calls token_reset_handler, which the factory binds to OpaqueTokenCursor.restart_without_token. It then re-reads the partition from page 1 with a fresh read context (paginator, page-size reducer, record parser).
    • The handler drops that partition's pending token and returns a slice with cursor_slice={}, so the re-read sends initial_token or nothing.
    • The committed token changes only if the re-read completes.
  • A WARN names the stream, the API's reason, and how many records may be emitted again (at-least-once delivery).
  • A second reset in the same partition read is a system_error. That includes a rejection of a read that sent no stored token.
  • RESET_OPAQUE_TOKEN on a stream without OpaqueTokenCursor raises OpaqueTokenResetNotSupportedException (config_error).

Declarative-First Evaluation

This is a CDK feature, not a connector change. RESET_PAGINATION restarts pagination but keeps the stored token, and no existing action drops the token. The new action is a built-in declarative option for adopters.

Test Coverage

  • E2E tests (HttpMocker, exact request bodies), recovery rows of the issue's test plan:
    • T-RESET: reset on page 3; the tokenless re-read commits a new token; WARN with the emitted count.
    • T-RESET-CTX: a record filter and a transformation read stream_interval, after a page-size reduction.
    • T-RESET2: a second reset raises a system_error.
    • T-RPAG: RESET_PAGINATION alone and after a token reset; a second pagination reset fails.
    • T-COMPOSITE: CompositeErrorHandler keeps the action.
  • Unit tests in test_http_client.py and test_composite_error_handler.py.
  • The new tests fail on the PR 1 branch and pass with this change.

Breaking change

None. This adds an opt-in action, and existing actions and streams are unchanged.

Link to Devin session: https://app.devin.ai/sessions/94b41efe75db43e78eb5a4f8d5f58484
Open in Devin Desktop: https://app.devin.ai/desktop/session/94b41efe75db43e78eb5a4f8d5f58484?variant=devin

devin-ai-integration Bot and others added 2 commits October 16, 2026 23:38
Co-Authored-By: bot_apk <apk@cognition.ai>
Co-Authored-By: bot_apk <apk@cognition.ai>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@github-actions

Copy link
Copy Markdown

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

Testing This CDK Version

You can test this version of the CDK using the following:

# Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@devin/1792189821-opaque-token-reset#egg=airbyte-python-cdk[dev]' --help

# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch devin/1792189821-opaque-token-reset

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

  • /autofix - Fixes most formatting and linting issues
  • /poetry-lock - Updates poetry.lock file
  • /test - Runs connector tests with the updated CDK
  • /prerelease - Triggers a prerelease publish with default arguments
  • /poe build - Regenerate git-committed build artifacts, such as the pydantic models which are generated from the manifest JSON schema in YAML.
  • /poe <command> - Runs any poe command in the CDK environment
📚 Show Repo Guidance

Helpful Resources

📝 Edit this welcome message.

@github-actions

Copy link
Copy Markdown

PyTest Results (Fast)

3 106 tests   - 2 692   3 084 ✅  - 2 683   11m 30s ⏱️ -56s
    1 suites ±    0      21 💤  -    10 
    1 files   ±    0       1 ❌ +    1 

For more details on these failures, see this check.

Results for commit 6bca0ac. ± Comparison against base commit 9808890.

@github-actions

Copy link
Copy Markdown

PyTest Results (Full)

5 810 tests  +9   5 779 ✅ +9   17m 43s ⏱️ +34s
    1 suites ±0      31 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit 6bca0ac. ± Comparison against base commit 9808890.

This branch was successfully deployed

2 active deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant