fix(low-code): interpolate PredicateValidator.value and allow plain strings in ValidateAdheresToSchema

[devin-ai-integration]

Summary

👉 TL;DR: Makes the declarative config validators actually usable: a predicate check can now compute its value from the user's config, and a schema check can validate a plain text value such as a hostname. Specifically, PredicateValidator now implements Validator.validate(input_data) and interpolates value against the config passed by Spec.validate_config, and ValidateAdheresToSchema falls back to validating the raw string when a str value is not JSON.

Resolves https://github.com/airbytehq/airbyte-internal-issues/issues/17719:

• https://github.com/airbytehq/airbyte-internal-issues/issues/17719

Close on merge: yes — internally filed, no customer to confirm with.

Parent epic: https://github.com/airbytehq/airbyte-internal-issues/issues/17702 (adopter migrations for source-amazon-seller-partner and source-jira are tracked separately and are not part of this PR).

Changes

• PredicateValidator (validators/predicate_validator.py)
  • Now subclasses Validator and accepts the config: validate(self, input_data: Optional[Mapping] = None). Previously validate(self) took no argument, so Spec.validate_config -> validator.validate(config) raised TypeError: PredicateValidator.validate() takes 1 positional argument but 2 were given for any manifest using it under config_normalization_rules.validations.
  • value is interpolated with JinjaInterpolation.eval(value, config) at validate time (string leaves, recursing into lists and object values; non-string literals pass through unchanged). Because Spec runs config transformations before validations, values can read keys added by ConfigAddFields. JinjaInterpolation literal-parses the rendered output, so "{{ config['report_options_list'] | map(attribute='stream_name') | list }}" reaches the strategy as a list, not "['a', 'b']".
  • Calling validate() with no argument still works (interpolates against {}), keeping the existing unit tests unchanged.
• ValidateAdheresToSchema.validate: a str value is still JSON-decoded first; on JSONDecodeError the raw string is validated instead of raising ValueError("Invalid JSON string: ..."). This makes DpathValidator(field_path=["domain"]) + {type: string, pattern: ...} work. Failures are still raised as ValueError.
• Schema/model: PredicateValidator.value description and examples updated (adds the map(attribute=...) | list example). No structural schema change; ModelToComponentFactory.create_predicate_validator is unchanged since interpolation now happens in the component against the runtime config.

Example manifest snippet that now works:

config_normalization_rules:
  validations:
    - type: PredicateValidator
      value: "{{ config['report_options_list'] | map(attribute='stream_name') | list }}"
      validation_strategy:
        type: ValidateAdheresToSchema
        base_schema: { type: array, uniqueItems: true }
    - type: DpathValidator
      field_path: [domain]
      validation_strategy:
        type: ValidateAdheresToSchema
        base_schema: { type: string, pattern: "^[a-zA-Z0-9-]+\\.atlassian\\.net$" }

Declarative-First Evaluation

Not a connector fix: this changes the CDK's own declarative components (PredicateValidator, ValidateAdheresToSchema) so manifests can use them as documented. No custom component is added; adopters keep using the built-in PredicateValidator / DpathValidator + ValidateAdheresToSchema from YAML.

Breaking Change Evaluation

Not breaking. No schema, spec, stream, or state change. PredicateValidator previously crashed when used through Spec, and plain-string inputs to ValidateAdheresToSchema previously always raised; both now validate. Valid-JSON strings and non-string values behave as before.

Review Spotlight

Reviewers with limited time, please review first:

• predicate_validator.py: interpolation and literal parsing of value
• validate_adheres_to_schema.py#L30-L35: raw-string fallback

Risks

• Interpolated/literal string values go through JinjaInterpolation's ast.literal_eval step (same as other interpolated CDK fields), so e.g. a value rendering "123" is validated as the int 123. This matches existing interpolation semantics elsewhere in the CDK.
• ValidateAdheresToSchema still JSON-decodes strings first, so a string that happens to be valid JSON (e.g. "123", "true") is validated as the decoded value, not as a string. Plain non-JSON strings (hostnames, URLs) are now validated as-is.
• For a non-JSON string validated against a non-string schema (e.g. the source-google-search-console-style custom_reports array check given malformed JSON), the error is still a ValueError, but the message changes from Invalid JSON string: ... to JSON schema validation error: '...' is not of type 'array'. Valid JSON strings and non-string values behave exactly as before.
• PredicateValidator had no working runtime path before (it raised TypeError), and no connector in airbytehq/airbyte uses type: PredicateValidator today.

Follow-ups

• Latent, not addressed here: ValidateAdheresToSchema.base_schema declares interpolation_context: [config] but create_validate_adheres_to_schema casts it directly without interpolation. Interpolating it naively would run every schema string (descriptions, patterns) through Jinja + literal_eval, which is not low-risk, so it is left for a separate change.

Test plan

• New factory-level tests in unit_tests/sources/declarative/spec/test_spec.py build a Spec from a manifest snippet via ModelToComponentFactory and call validate_config:
  • interpolated list + {type: array, uniqueItems: true}: pass (unique) and fail (duplicate)
  • PredicateValidator reads a key from the passed config (pass/fail)
  • DpathValidator + hostname pattern schema: valid hostname passes; URL and non-matching host fail with ValueError
• New unit tests in test_predicate_validator.py (interpolation, list rendering, literal non-string preservation, nested interpolation) and test_validate_adheres_to_schema.py (non-JSON string pass/fail).
• All 12 new tests fail on main and pass with this change; existing validator/spec tests pass unchanged.
• poetry run pytest unit_tests/sources/declarative (2357 passed, 1 skipped), ruff check, ruff format --check, mypy on validators/. The full unit_tests/ suite is left to CI.

Link to Devin session: https://app.devin.ai/sessions/b07c9708bfc1453a93f32490f11a5806
Open in Devin Desktop: https://app.devin.ai/desktop/session/b07c9708bfc1453a93f32490f11a5806?variant=devin

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[github-actions]

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

Testing This CDK Version

You can test this version of the CDK using the following:

# Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@devin/1791298421-fix-predicate-validator#egg=airbyte-python-cdk[dev]' --help

# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch devin/1791298421-fix-predicate-validator

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

• /autofix - Fixes most formatting and linting issues
• /poetry-lock - Updates poetry.lock file
• /test - Runs connector tests with the updated CDK
• /prerelease - Triggers a prerelease publish with default arguments
• /poe build - Regenerate git-committed build artifacts, such as the pydantic models which are generated from the manifest JSON schema in YAML.
• /poe <command> - Runs any poe command in the CDK environment

📚 Show Repo Guidance

Helpful Resources

• CDK API Reference

📝 Edit this welcome message.

[github-actions]

PyTest Results (Fast)

5 037 tests  +12   5 025 ✅ +12   11m 9s ⏱️ +5s
    1 suites ± 0      12 💤 ± 0 
    1 files   ± 0       0 ❌ ± 0 

Results for commit 70200a3. ± Comparison against base commit 5381b8f.

[github-actions]

PyTest Results (Full)

5 040 tests  +12   5 028 ✅ +12   15m 44s ⏱️ + 3m 4s
    1 suites ± 0      12 💤 ± 0 
    1 files   ± 0       0 ❌ ± 0 

Results for commit 70200a3. ± Comparison against base commit 5381b8f.