Skip to content

fix: improve oneOf/anyOf config validation error messages - #972

Closed
devin-ai-integration[bot] wants to merge 1 commit into
mainfrom
devin/1775052273-improve-oneof-validation-error
Closed

devin-ai-integration[bot] wants to merge 1 commit into
mainfrom
devin/1775052273-improve-oneof-validation-error

Conversation

@devin-ai-integration

Copy link
Copy Markdown
Contributor

Summary

Improves the user-facing error message produced when config validation fails on a oneOf or anyOf schema (e.g., authentication method selection). Previously, jsonschema.validate() used best_match internally, which often surfaced a misleading error from the last non-matching branch — for example, "Config validation error: 'Service' was expected" when the user intended OAuth but was missing client_secret. This affects all connectors using oneOf with const discriminators (source-google-sheets, source-google-analytics-data-api, source-mailchimp, etc.).

The fix switches from validate() to Draft7Validator.iter_errors(), which exposes the full error tree. When a oneOf/anyOf error is detected, the new _format_oneof_error helper formats a message listing each branch by its title (or "option N") with the specific validation failures for that branch.

Before: Config validation error: 'Service' was expected

After: Config validation error: the provided configuration for "credentials" does not match any supported option. OAuth Credentials: 'client_secret' is a required property | Service Account Key: 'Service' was expected; 'service_account_info' is a required property.

For non-oneOf validation errors, behavior is preserved: the first error's message is used directly.

Resolves https://github.com/airbytehq/airbyte-internal-issues/issues/16138
Related to https://github.com/airbytehq/oncall/issues/11841

Review & Testing Checklist for Human

  • Non-oneOf error regression: The old code used best_match (via validate()) to pick the most relevant error; the new code uses errors[0] from iter_errors(). Verify that common non-oneOf validation failures (e.g., missing required top-level field, wrong type) still produce clear messages. These may differ subtly from the old behavior.
  • No secret leakage in internal_message: The new internal_message includes sub.message and list(sub.absolute_path). Confirm that absolute_path only contains field names (keys), never config values, so secrets are not exposed.
  • Edge cases in _format_oneof_error: Consider schemas with nested oneOf, anyOf (not oneOf), branches without title, or empty context. The function handles missing titles (falls back to "option N") but doesn't have explicit test coverage.
  • Add unit tests: No new tests were added for _format_oneof_error or the oneOf-specific path in check_config_against_spec_or_exit. Consider adding tests for: (a) oneOf with titled branches, (b) oneOf without titles, (c) anyOf, (d) non-oneOf errors preserved.

Suggested manual test: Run a connector that uses oneOf auth (e.g., source-google-sheets) with an intentionally incomplete OAuth config and verify the improved error message surfaces in the check output.

Notes

  • FailureType remains config_error — no change in retry/classification behavior, so this is not a breaking change.
  • CDK versioning is handled automatically by semantic-pr-release-drafter based on the fix: PR title prefix.

Link to Devin session: https://app.devin.ai/sessions/ef29e8d43f4e405d969a118acacb723d

Replace validate() with Draft7Validator.iter_errors() to detect
oneOf/anyOf validation failures and produce per-branch error details
instead of surfacing a misleading error from the last non-matching
branch (e.g. "'Service' was expected" when the user intended OAuth).

The improved message lists each oneOf option by title with the specific
validation errors for that branch, making it actionable for users to
identify which required fields are missing in their chosen auth method.

Co-Authored-By: bot_apk <apk@cognition.ai>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@github-actions

github-actions Bot commented Apr 1, 2026

Copy link
Copy Markdown

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

Testing This CDK Version

You can test this version of the CDK using the following:

# Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@devin/1775052273-improve-oneof-validation-error#egg=airbyte-python-cdk[dev]' --help

# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch devin/1775052273-improve-oneof-validation-error

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

  • /autofix - Fixes most formatting and linting issues
  • /poetry-lock - Updates poetry.lock file
  • /test - Runs connector tests with the updated CDK
  • /prerelease - Triggers a prerelease publish with default arguments
  • /poe build - Regenerate git-committed build artifacts, such as the pydantic models which are generated from the manifest JSON schema in YAML.
  • /poe <command> - Runs any poe command in the CDK environment
📚 Show Repo Guidance

Helpful Resources

📝 Edit this welcome message.

@github-actions

github-actions Bot commented Apr 1, 2026

Copy link
Copy Markdown

PyTest Results (Fast)

3 975 tests  ±0   3 964 ✅ ±0   7m 23s ⏱️ -9s
    1 suites ±0      11 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit cf497c0. ± Comparison against base commit 69cd63d.

@github-actions

github-actions Bot commented Apr 1, 2026

Copy link
Copy Markdown

PyTest Results (Full)

3 978 tests  ±0   3 966 ✅ ±0   11m 6s ⏱️ -14s
    1 suites ±0      12 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit cf497c0. ± Comparison against base commit 69cd63d.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant