fix(file-based): skip empty CSV files during schema inference and honor ! glob exclusions

[devin-ai-integration]

Summary

Fixes two file-based CDK bugs surfaced by an Azure Blob Storage customer whose container (~1,100 CSVs, 10 sampled for discovery) regularly receives empty CSV files:

1. Empty CSV files fail the whole sync. CsvParser.infer_schema raised EmptyFileSchemaInferenceError when a sampled file had no data rows, which DefaultFileBasedStream turned into a fatal SchemaInferenceError. Now an empty (0-byte or header-only) file logs a warning and contributes {} to merge_schemas, so the schema comes from the non-empty sampled files. A 0-byte file also used to blow up earlier in _read_and_validate_headers / _auto_generate_headers (next(reader) → StopIteration), so those now return [] headers. If every sampled file is empty, the existing InvalidSchemaError still fires, with the details message extended to say so. The EmptyFileSchemaInferenceError class is left in place for compatibility.

2. !pattern glob exclusions were silently ignored. file_matches_globs evaluated each glob independently:

any(globmatch(file.uri, g, flags=GLOBSTAR) for g in globs)

Without wcmatch's NEGATE flag, !inbound/.../x.csv is a literal pattern that never matches, so the customer's exclusions had no effect. Now:

globmatch(file.uri, globs, flags=GLOBSTAR | NEGATE)

which is equivalent for positive-only lists and applies ! exclusions across the list. get_prefixes_from_globs skips ! globs so we don't list under a bogus !inbound/... prefix, and the globs field description documents the syntax (spec snapshot in csv_scenarios.py updated accordingly).

Tests added for both paths (parser-level empty/header-only files, stream-level merge with one empty file, glob negation + prefix handling). unit_tests/sources/file_based passes except two test_parse_start_date cases that fail identically on main.

Link to Devin session: https://app.devin.ai/sessions/99491084666f4c3dbb1596d660e1f7bc
Open in Devin Desktop: https://app.devin.ai/desktop/session/99491084666f4c3dbb1596d660e1f7bc?variant=devin

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[github-actions]

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

Testing This CDK Version

You can test this version of the CDK using the following:

# Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@devin/1790355270-file-based-empty-csv-and-glob-negation#egg=airbyte-python-cdk[dev]' --help

# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch devin/1790355270-file-based-empty-csv-and-glob-negation

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

• /autofix - Fixes most formatting and linting issues
• /poetry-lock - Updates poetry.lock file
• /test - Runs connector tests with the updated CDK
• /prerelease - Triggers a prerelease publish with default arguments
• /poe build - Regenerate git-committed build artifacts, such as the pydantic models which are generated from the manifest JSON schema in YAML.
• /poe <command> - Runs any poe command in the CDK environment

📚 Show Repo Guidance

Helpful Resources

• CDK API Reference

📝 Edit this welcome message.

[devin-ai-integration]

Closing at requester's request.