fix(low-code): pass a custom component's decoder to its nested requester

[devin-ai-integration]

Summary

Resolves https://github.com/airbytehq/airbyte-internal-issues/issues/17738:

• https://github.com/airbytehq/airbyte-internal-issues/issues/17738

Close on merge: yes — internally filed, no customer to confirm with.

Part of https://github.com/airbytehq/airbyte-internal-issues/issues/17702.

When a CustomRetriever (or any custom component) declares a decoder field, its nested HttpRequester, RecordSelector/extractor and paginator were built independently by create_custom_component and always fell back to JsonDecoder. So HttpRequester.stream_response was False even with a streaming decoder (e.g. JsonlDecoder → CompositeRawDecoder), requests read the body eagerly, and the streaming decoder reading response.raw silently yielded zero records. create_simple_retriever already builds the decoder first and passes it down; the custom-component path now does the same.

Changes in ModelToComponentFactory:

• create_custom_component: builds the decoder field first; if the component class declares decoder and it resolves to a Decoder, it is forwarded via kwargs to the nested subcomponents (nested_component_kwargs.setdefault("decoder", ...), so a decoder passed by a parent still wins).
• create_http_requester: decoder is now keyword-only and Optional (default None → fresh JsonDecoder), so _create_nested_component can fill it like it already does for create_record_selector / create_default_paginator (pattern from fix(low-code): build session authenticators and paginators nested in SelectiveAuthenticator or custom components #500). This also removes the shared mutable JsonDecoder default. No positional callers exist in the CDK (create_simple_retriever and the other callers pass decoder=).

# before: requester under a CustomRetriever with decoder: {type: JsonlDecoder}
retriever.decoder           -> CompositeRawDecoder (is_stream_response True)
retriever.requester.decoder -> JsonDecoder, stream_response False
extractor.decoder           -> JsonDecoder
# after
retriever.requester.decoder is retriever.decoder, stream_response True
extractor.decoder is retriever.decoder

Unchanged: custom components without a decoder field (or with no decoder set) keep the JsonDecoder defaults; under emit_connector_builder_messages=True streaming decoders are still downgraded to buffered, and the nested requester follows the same decoder (stream_response=False).

This should let source-google-ads drop its workarounds (forcing stream_response = True in GoogleAdsHttpRequester and copying the decoder into the extractor in GoogleAdsRetriever.__post_init__) in a follow-up sub-issue; no connector change here.

Reproduction

At main (5381b8f) a SimpleRetriever subclass with decoder: Optional[Decoder] used as a CustomRetriever with decoder: {type: JsonlDecoder} built requester.decoder = JsonDecoder, requester.stream_response = False, extractor.decoder = JsonDecoder. Against a local HTTP server returning 2 JSONL rows, CompositeRawDecoder(JsonLineParser(), stream_response=True).decode() yields [] for a stream=False response and both rows for stream=True.

Declarative-First Evaluation

Not applicable to a connector: this is a CDK factory fix, no custom component is added to any connector. The change makes the declarative decoder field on custom components behave like it does on SimpleRetriever, which reduces the need for custom-component workarounds.

Test Coverage

New tests in unit_tests/sources/declarative/parsers/test_model_to_component_factory.py (with TestingCustomRetrieverWithDecoder in testing_components.py):

• streaming decoder reaches nested requester (stream_response is True) and extractor (same instance)
• decoder reaches a nested DefaultPaginator
• Connector Builder mode: decoder propagated, requester stays buffered
• no decoder set → JsonDecoder defaults, stream_response False
• custom component without a decoder field → manifest decoder is not propagated
• decoder passed by the parent takes precedence

The propagation tests fail on the unfixed factory (assert JsonDecoder() is CompositeRawDecoder(), assert False is True) and pass with the fix.

Ran locally: poetry run ruff check ., poetry run ruff format --check ., poetry run mypy --config-file mypy.ini airbyte_cdk/sources/declarative/parsers/model_to_component_factory.py, poetry run pytest unit_tests/sources/declarative unit_tests/connector_builder (2414 passed, 1 skipped), and the rest of unit_tests/ (1736 passed; 28 modules not collected locally only because optional extras — fastavro, fastapi, langchain_core, pyarrow, sqlalchemy, unstructured — aren't installed in this env; CI covers them).

Requested via the /ai-fix workflow for the issue above (filed by Anatolii Yatsuk).

Link to Devin session: https://app.devin.ai/sessions/d39a6ff2f24f44a691121ab965bf039c
Open in Devin Desktop: https://app.devin.ai/desktop/session/d39a6ff2f24f44a691121ab965bf039c?variant=devin

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[github-actions]

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

Testing This CDK Version

You can test this version of the CDK using the following:

# Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@devin/1791298668-custom-component-decoder-propagation#egg=airbyte-python-cdk[dev]' --help

# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch devin/1791298668-custom-component-decoder-propagation

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

• /autofix - Fixes most formatting and linting issues
• /poetry-lock - Updates poetry.lock file
• /test - Runs connector tests with the updated CDK
• /prerelease - Triggers a prerelease publish with default arguments
• /poe build - Regenerate git-committed build artifacts, such as the pydantic models which are generated from the manifest JSON schema in YAML.
• /poe <command> - Runs any poe command in the CDK environment

📚 Show Repo Guidance

Helpful Resources

• CDK API Reference

📝 Edit this welcome message.

[github-actions]

PyTest Results (Fast)

5 031 tests  +6   5 019 ✅ +6   8m 58s ⏱️ - 2m 6s
    1 suites ±0      12 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit a55b1c8. ± Comparison against base commit 5381b8f.

♻️ This comment has been updated with latest results.

[github-actions]

PyTest Results (Full)

5 034 tests  +6   5 022 ✅ +6   16m 14s ⏱️ + 3m 34s
    1 suites ±0      12 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit a55b1c8. ± Comparison against base commit 5381b8f.

♻️ This comment has been updated with latest results.