Repository navigation
fix(file-based): surface actionable errors instead of full Python tracebacks during connection check - #1012
Conversation
…cebacks during connection check
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
👋 Greetings, Airbyte Team Member!Here are some helpful tips and reminders for your convenience. 💡 Show Tips and TricksTesting This CDK VersionYou can test this version of the CDK using the following: # Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@devin/1777763056-file-based-actionable-error-messages#egg=airbyte-python-cdk[dev]' --help
# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch devin/1777763056-file-based-actionable-error-messagesPR Slash CommandsAirbyte Maintainers can execute the following slash commands on your PR:
|
There was a problem hiding this comment.
Pull request overview
This PR improves file-based connector connection-check UX by ensuring actionable, connector-specific error messages are surfaced to the platform instead of full Python tracebacks, while still logging tracebacks for debugging.
Changes:
- Re-raise
AirbyteTracedExceptionfrom_check_list_filesso connector-provided actionable messages andfailure_typepropagate unchanged. - Return
str(CheckAvailabilityError)(and log vialogger.exception) instead of returningtraceback.format_exc()as the user-facing failure reason. - Add unit tests verifying (a) no traceback text is returned in the reason and (b)
AirbyteTracedExceptionpropagates from both availability-check methods.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
airbyte_cdk/sources/file_based/availability_strategy/default_file_based_availability_strategy.py |
Stops returning traceback strings as reasons; logs exceptions and preserves actionable error messages; re-raises AirbyteTracedException from file listing. |
unit_tests/sources/file_based/availability_strategy/test_default_file_based_availability_strategy.py |
Adds regression tests ensuring actionable reasons are returned and AirbyteTracedException is not swallowed/wrapped. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…teTracedException
Summary
When a file-based connector's connection check fails, the platform was rendering the full Python traceback as the user-facing error message. Patrick reported this for the Google Drive Agent Connector — both "service account key in the wrong format" and "no files match the default glob" surfaced as raw stack traces in the Engine UI popup instead of an actionable message.
Two issues in
default_file_based_availability_strategy.pyare responsible for this across all file-based connectors (S3, GCS, Azure, SFTP, Google Drive, ...):_check_list_files's broadexcept Exceptionwas swallowingAirbyteTracedExceptions raised by stream readers. For example,source-google-drive's_build_google_servicealready raises a perfectAirbyteTracedException(failure_type=config_error, message="Could not authenticate with Google drive. Please check your credentials."), but it got re-wrapped as a genericCheckAvailabilityError(ERROR_LISTING_FILES, ...), losing the actionable text.check_availabilityandcheck_availability_and_parsabilityreturned"".join(traceback.format_exc())as the failure reason.FileBasedSource.check_connectionthen propagated that traceback string verbatim intoAirbyteTracedException.message, which the platform exposes asexternal_messageand the Engine renders in the popup.This PR:
AirbyteTracedExceptionfrom_check_list_filesso connector-specific actionable messages propagate unchanged to the platform.str(exc)from the twoCheckAvailability*methods instead of the full traceback. The actionable message comes fromBaseFileBasedSourceError's string representation, which already includes theFileBasedSourceErrorenum text (e.g. theEMPTY_STREAMsentence). The full traceback is still emitted to the connector log vialogger.exception(...)for debugging.User-visible result for the two reported cases (no behavior change for the happy path):
Unable to connect to stream files - Could not authenticate with Google drive. Please check your credentials.No files were identified in the stream. This may be because there are no files in the specified container, or because your glob patterns did not match any files. Please verify that your source contains files last modified after the start_date and that your glob patterns are not overly strict. Contact Support if you need assistance. stream=filesBoth strings already exist and comply with the error-message guidelines — they were just buried in tracebacks.
Review & Testing Checklist for Human
reasonstring starting withTraceback ...(a quick grep acrossairbytehq/airbyteconnectors andairbytehq/airbyte-platform-internalfor substrings like"Traceback (most recent call last)"against the file-based check reason should show no positive consumers).source-s3,source-sftp) by triggering a check failure and verifying the popup shows the actionable string rather than a stack trace.logger.exception("Stream availability check failed")produces the expected log entry in connector stdout — the full traceback should still be in the logs for debugging.Notes
test_default_file_based_availability_strategy.pycover both fixes:test_not_available_given_no_files/test_check_availability_returns_actionable_reason_when_no_filesassert the reason contains the actionable enum text and does not containTracebackorraise CheckAvailabilityError.test_airbyte_traced_exception_from_stream_reader_propagates/test_airbyte_traced_exception_propagates_in_check_availabilityassert thatAirbyteTracedExceptions raised by the stream reader propagate unchanged with their originalmessageandfailure_type.poetry run pytest unit_tests/sources/file_based/ -x -qpasses.#ask-devin-aithread (https://airbytehq-team.slack.com/archives/C08BHPUMEPJ/p1777762733898279). A follow-up PR inairbytehq/airbytecould differentiate Google Drive's_build_google_servicefailure modes (json.JSONDecodeError→ "Service Account Key must contain valid JSON" /ValueErrorfromload_pem_private_key→ "Service Account Key is not a valid PEM-encoded private key") for an even more specific message — happy to do that as a separate small PR if desired.Link to Devin session: https://app.devin.ai/sessions/7ddfcdf1232643c48efb777a5bc66e26
Important
Auto-merge enabled.
This PR is set to merge automatically when all requirements are met.