perf: defer pandas import in ResponseToFileExtractor

[devin-ai-integration]

Summary

response_to_file_extractor.py imported pandas at module level. That module is always loaded on the declarative / SDM import path (extractors/__init__.py, model_to_component_factory.py, http_job_repository.py), so every manifest-only connector process imported pandas and numpy at start-up on every command, though none of them use it. pandas is only used in ResponseToFileExtractor._read_with_chunks.

- import pandas as pd   # module level
  def _read_with_chunks(self, path, file_encoding, chunk_size=100):
+     import pandas as pd   # before `try:` so `except pd.errors.EmptyDataError` can resolve it
      try:
          ...

This follows the same lazy-import approach as unstructured in file_based/file_types/unstructured_parser.py. Behavior is unchanged, including preserve_na_values. The only consumer in the monorepo is source-salesforce, a Python connector. Its tests patch source_salesforce.streams.ResponseToFileExtractor, not the module's pd.

Measured locally on main:

• Before: import airbyte_cdk.cli.source_declarative_manifest._run → 'pandas' in sys.modules, 'numpy' in sys.modules = True True, and -X importtime attributes about 198 ms cumulative to response_to_file_extractor.
• After: False False, and response_to_file_extractor takes about 0.4 ms cumulative.

Trade-off: a broken pandas/numpy install now fails on the first CSV parse instead of at process start. The existing test_response_to_file_extractor.py tests still run pd.read_csv, so CI catches this.

Resolves https://github.com/airbytehq/airbyte-internal-issues/issues/17921:

• https://github.com/airbytehq/airbyte-internal-issues/issues/17921

Close on merge: yes — internally filed, no customer to confirm with.

Part of https://github.com/airbytehq/airbyte-internal-issues/issues/17920 (faster check for manifest-only connectors).

Test Coverage

• New unit_tests/source_declarative_manifest/test_source_declarative_manifest_imports.py: runs SDM spec, check (HTTP mocked with requests_mock) and discover on the PokeAPI test manifest in a subprocess, so other tests can't pollute sys.modules. It then asserts that neither pandas nor numpy was imported. Verified it fails without the fix (assert '["numpy", "pandas"]' == '[]') and passes with it.
• unit_tests/source_declarative_manifest/, unit_tests/sources/declarative/extractors/test_response_to_file_extractor.py, unit_tests/sources/declarative/requesters/: 536 passed.
• ruff check, ruff format, mypy on the changed file all pass.

Declarative-First Evaluation

N/A. This is a CDK-internal import change, not a change to a connector manifest or custom component.

Breaking Change Evaluation

Not breaking: no schema, spec, state, stream or data-scope change. CDK versions come from release tooling, so this PR does not bump a version.

Link to Devin session: https://app.devin.ai/sessions/919705a00f754ff494dff1d2ed6f9a61
Open in Devin Desktop: https://app.devin.ai/desktop/session/919705a00f754ff494dff1d2ed6f9a61?variant=devin

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[github-actions]

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

Testing This CDK Version

You can test this version of the CDK using the following:

# Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@devin/1791494270-defer-pandas-import#egg=airbyte-python-cdk[dev]' --help

# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch devin/1791494270-defer-pandas-import

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

• /autofix - Fixes most formatting and linting issues
• /poetry-lock - Updates poetry.lock file
• /test - Runs connector tests with the updated CDK
• /prerelease - Triggers a prerelease publish with default arguments
• /poe build - Regenerate git-committed build artifacts, such as the pydantic models which are generated from the manifest JSON schema in YAML.
• /poe <command> - Runs any poe command in the CDK environment

📚 Show Repo Guidance

Helpful Resources

• CDK API Reference

📝 Edit this welcome message.

[github-actions]

PyTest Results (Fast)

778 tests   - 4 256   766 ✅  - 4 256   6m 7s ⏱️ - 4m 34s
  1 suites ±    0    11 💤  -     1 
  1 files   ±    0     1 ❌ +    1 

For more details on these failures, see this check.

Results for commit dfc8407. ± Comparison against base commit 916d6f6.

This pull request removes 4257 and adds 1 tests. Note that renamed tests count towards both.

unit_tests.sources.declarative.auth.test_jwt.TestJwtAuthenticator ‑ test_get_request_headers[test_get_request_headers]
unit_tests.sources.declarative.auth.test_jwt.TestJwtAuthenticator ‑ test_get_request_headers[test_with_default_authorization_header]
unit_tests.sources.declarative.auth.test_jwt.TestJwtAuthenticator ‑ test_get_request_options[test_get_request_headers1]
unit_tests.sources.declarative.auth.test_jwt.TestJwtAuthenticator ‑ test_get_request_options[test_get_request_headers2]
unit_tests.sources.declarative.auth.test_oauth.TestOauth2Authenticator ‑ test_client_secret_empty
unit_tests.sources.declarative.auth.test_oauth.TestOauth2Authenticator ‑ test_custom_component_parse_token_expiration_date_override_only_gets_raw_values
unit_tests.sources.declarative.auth.test_oauth.TestOauth2Authenticator ‑ test_custom_component_returning_raw_expires_in
unit_tests.sources.declarative.auth.test_oauth.TestOauth2Authenticator ‑ test_custom_component_returning_unparseable_expires_in_stores_it_as_is
unit_tests.sources.declarative.auth.test_oauth.TestOauth2Authenticator ‑ test_error_handling
unit_tests.sources.declarative.auth.test_oauth.TestOauth2Authenticator ‑ test_error_on_refresh_token_grant_without_refresh_token
…

unit_tests.source_declarative_manifest.test_source_declarative_manifest_imports ‑ test_spec_check_discover_do_not_import_pandas_or_numpy

[github-actions]

PyTest Results (Full)

5 038 tests  +1   5 026 ✅ +1   16m 7s ⏱️ +35s
    1 suites ±0      12 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit dfc8407. ± Comparison against base commit 916d6f6.