Repository navigation
fix(memory-monitor): raise system_error instead of transient_error on critical memory threshold - #1005
Conversation
… critical memory threshold Co-Authored-By: patrick.nilan@airbyte.io <patrick.nilan@airbyte.io>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
👋 Greetings, Airbyte Team Member!Here are some helpful tips and reminders for your convenience. 💡 Show Tips and TricksTesting This CDK VersionYou can test this version of the CDK using the following: # Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@devin/1777475498-memory-monitor-system-error#egg=airbyte-python-cdk[dev]' --help
# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch devin/1777475498-memory-monitor-system-errorPR Slash CommandsAirbyte Maintainers can execute the following slash commands on your PR:
|
There was a problem hiding this comment.
Pull request overview
Updates MemoryMonitor to classify critical near-OOM memory conditions as system_error (instead of transient_error) so the platform receives an accurate signal for non-retriable internal failures.
Changes:
- Update
MemoryMonitorcritical-thresholdAirbyteTracedExceptionto useFailureType.system_error. - Update
MemoryMonitordocstring to reflect the new failure classification. - Update the unit test assertion to expect
FailureType.system_error.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
airbyte_cdk/utils/memory_monitor.py |
Switches critical memory-threshold exception failure type to system_error and updates documentation accordingly. |
unit_tests/utils/test_memory_monitor.py |
Adjusts the fail-fast test to assert system_error for critical memory exhaustion. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Summary
Changes the
FailureTyperaised byMemoryMonitorwhen exceeding the critical memory threshold fromtransient_errortosystem_error.An OOM-level memory exhaustion is an unexpected internal failure, not a transient issue that will resolve on retry. Classifying it as
system_errorgives the platform accurate signal for how to handle the failure.Changes
airbyte_cdk/utils/memory_monitor.py: Changedfailure_type=FailureType.transient_errortofailure_type=FailureType.system_errorin the critical threshold exception, and updated the class docstring accordingly.unit_tests/utils/test_memory_monitor.py: Updated the test assertion to expectFailureType.system_error.Review & Testing Checklist for Human
system_errorfrom the memory monitor (e.g., retry policies, user-facing messaging)Notes
Requested by Patrick Nilan (@pnilan).
Link to Devin session: https://app.devin.ai/sessions/5aa5b30d63484e35bd4314b32fac0f54
Note
Auto-merge may have been disabled. Please check the PR status to confirm.
Important
Auto-merge enabled.
This PR is set to merge automatically when all requirements are met.