Repository navigation
feat(test): persist configs rotated during standard tests to secrets/updated_configurations - #1199
Draft
Serhii Lazebnyi (lazebnyi) wants to merge 2 commits into
Draft
Serhii Lazebnyi (lazebnyi) wants to merge 2 commits into
Serhii Lazebnyi (lazebnyi) wants to merge 2 commits into
Conversation
1 of 2 tasks
Serhii Lazebnyi (lazebnyi)
force-pushed
the
feat/persist-updated-configs
branch
from
October 5, 2026 15:04
8b87efe to
bbd2ddb
Compare
👋 Greetings, Airbyte Team Member!Here are some helpful tips and reminders for your convenience. 💡 Show Tips and TricksTesting This CDK VersionYou can test this version of the CDK using the following: # Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@feat/persist-updated-configs#egg=airbyte-python-cdk[dev]' --help
# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch feat/persist-updated-configsPR Slash CommandsAirbyte Maintainers can execute the following slash commands on your PR:
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Restores the "config update" emitter that was lost when the old Connector Acceptance Tests were deleted. Connectors that rotate credentials during a run (single-use OAuth refresh tokens, for example) report the new config as a
CONTROL/CONNECTOR_CONFIGmessage. The standard test harness used to persist those tosecrets/updated_configurations/so CI could push them back to GSM; since the move toairbyte-cdk connector testnothing captured them, so the next run started from an already-consumed token (see airbytehq/airbyte-internal-issues#17023 and thesource-greenhouseacceptance config).How
airbyte_cdk/test/standard_tests/_updated_configs.py: extractsCONNECTOR_CONFIGcontrol messages from anEntrypointOutputand writes each config that differs from the current one to<config dir>/updated_configurations/<stem>|<emitted_at_ms>.json(same layout the old CAT used, mode0600). Persistence never raises._job_runner.run_test_jobcalls it right after_run_command, before any assertion can short-circuit, so a rotated token survives a failing test.docker_base.py: thecheck/discover/readimage tests go through a small wrapper that runs the container, persists updates from control messages and from in-place edits to the mounted config file, and only then raises on errors.ConnectorTestScenario.get_config_dictnow prefers the newest persisted update over the original file, so later tests in the same session use the rotated credential instead of the stale one. New helpersresolve_config_path()/get_effective_config_path()and module-levelfind_latest_updated_config_file().idandrequires_credsstill derive from the original path.updated_configurationsis already gitignored inairbytehq/airbyte.Related PRs
airbyte-ops secrets push, which uploads these files to GSM.connector-ci-checks.ymlafter the integration tests.Verification
poetry run pytest unit_tests/test unit_tests/cli(119 passed, 9 skipped), including newunit_tests/test/test_updated_configs.pypoetry run ruff check,ruff format, andmypyon the touched modules pass🤖 Generated with Claude Code