Skip to content

feat(test): persist configs rotated during standard tests to secrets/updated_configurations - #1199

Draft
Serhii Lazebnyi (lazebnyi) wants to merge 2 commits into
mainfrom
feat/persist-updated-configs
Draft

Serhii Lazebnyi (lazebnyi) wants to merge 2 commits into
mainfrom
feat/persist-updated-configs

Conversation

@lazebnyi

@lazebnyi Serhii Lazebnyi (lazebnyi) commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

What

Restores the "config update" emitter that was lost when the old Connector Acceptance Tests were deleted. Connectors that rotate credentials during a run (single-use OAuth refresh tokens, for example) report the new config as a CONTROL/CONNECTOR_CONFIG message. The standard test harness used to persist those to secrets/updated_configurations/ so CI could push them back to GSM; since the move to airbyte-cdk connector test nothing captured them, so the next run started from an already-consumed token (see airbytehq/airbyte-internal-issues#17023 and the source-greenhouse acceptance config).

How

  • New airbyte_cdk/test/standard_tests/_updated_configs.py: extracts CONNECTOR_CONFIG control messages from an EntrypointOutput and writes each config that differs from the current one to <config dir>/updated_configurations/<stem>|<emitted_at_ms>.json (same layout the old CAT used, mode 0600). Persistence never raises.
  • _job_runner.run_test_job calls it right after _run_command, before any assertion can short-circuit, so a rotated token survives a failing test.
  • docker_base.py: the check/discover/read image tests go through a small wrapper that runs the container, persists updates from control messages and from in-place edits to the mounted config file, and only then raises on errors.
  • ConnectorTestScenario.get_config_dict now prefers the newest persisted update over the original file, so later tests in the same session use the rotated credential instead of the stale one. New helpers resolve_config_path() / get_effective_config_path() and module-level find_latest_updated_config_file(). id and requires_creds still derive from the original path.

updated_configurations is already gitignored in airbytehq/airbyte.

Related PRs

Verification

  • poetry run pytest unit_tests/test unit_tests/cli (119 passed, 9 skipped), including new unit_tests/test/test_updated_configs.py
  • poetry run ruff check, ruff format, and mypy on the touched modules pass

🤖 Generated with Claude Code

Comment thread airbyte_cdk/test/standard_tests/docker_base.py Fixed
@github-actions

github-actions Bot commented Oct 5, 2026

Copy link
Copy Markdown

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

Testing This CDK Version

You can test this version of the CDK using the following:

# Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@feat/persist-updated-configs#egg=airbyte-python-cdk[dev]' --help

# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch feat/persist-updated-configs

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

  • /autofix - Fixes most formatting and linting issues
  • /poetry-lock - Updates poetry.lock file
  • /test - Runs connector tests with the updated CDK
  • /prerelease - Triggers a prerelease publish with default arguments
  • /poe build - Regenerate git-committed build artifacts, such as the pydantic models which are generated from the manifest JSON schema in YAML.
  • /poe <command> - Runs any poe command in the CDK environment
📚 Show Repo Guidance

Helpful Resources

📝 Edit this welcome message.

@github-actions

github-actions Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

PyTest Results (Fast)

4 988 tests  +8   4 976 ✅ +8   10m 56s ⏱️ -9s
    1 suites ±0      12 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit 56df78c. ± Comparison against base commit b8f465f.

♻️ This comment has been updated with latest results.

@github-actions

github-actions Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

PyTest Results (Full)

4 991 tests  +8   4 979 ✅ +8   16m 8s ⏱️ + 2m 28s
    1 suites ±0      12 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit 56df78c. ± Comparison against base commit b8f465f.

♻️ This comment has been updated with latest results.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant