Skip to content

make tests order independent - #40953

Merged
Abacn merged 3 commits into
apache:masterfrom
aIbrahiim:fix-dep-postcommit-flakes
Nov 3, 2026
Merged

Abacn merged 3 commits into
apache:masterfrom
aIbrahiim:fix-dep-postcommit-flakes

Conversation

@aIbrahiim

@aIbrahiim aIbrahiim commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Fixes: #30799

PostCommit Python Dependency failing on embeddings + tft-121.

HF: with a token set the provider can still reject, and it was treated {"error":...} as embeddings ((5, False)). Raise on that, and skip the live test if the API isn't usable.

TFT: vocab stdout tests flake on print order so switch those two to assertCountEqual.

@github-actions github-actions Bot added python examples size/s S: 10-49 lines changed labels Nov 2, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📋 Review Summary

This PR improves test stability by making stdout assertions in mltransform_test.py order-independent and adding proper HTTP status checking, error payload validation, and pre-flight availability checks for the Hugging Face Inference API in huggingface.py and huggingface_test.py. The changes are clean, well-scoped, and effectively address potential test flakes.

🔍 General Feedback

  • In mltransform_test.py, switching to assertCountEqual for vocabulary computation tests makes sense since pipeline execution does not guarantee element print ordering (consider applying the same to test_mltransform_scale_to_0_1 if it can also experience out-of-order execution).
  • The error handling additions in _InferenceAPIHandler.run_inference (raise_for_status() and checking for 'error' in the response dict) prevent API error payloads from silently passing through as embedding outputs.

💡 Tip: Comment /gemini review on this pull request to trigger a new Gemini code review.

@github-actions

github-actions Bot commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

Assigning reviewers:

R: @jrmccluskey for label python.
Backup reviewer: tvalentyn (if the main reviewer is unresponsive, you can escalate to them by tagging their username)

Note: If you would like to opt out of this review, comment assign to next reviewer.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

@aIbrahiim

Copy link
Copy Markdown
Contributor Author

R: @Abacn

@aIbrahiim

Copy link
Copy Markdown
Contributor Author

assign to next reviewer

@github-actions

github-actions Bot commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment assign set of reviewers

@codecov

codecov Bot commented Nov 3, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.09%. Comparing base (12bbb17) to head (895e93e).
⚠️ Report is 46 commits behind head on master.

Files with missing lines Patch % Lines
...pache_beam/ml/transforms/embeddings/huggingface.py 0.00% 5 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #40953      +/-   ##
==========================================
+ Coverage   79.83%   80.09%   +0.26%     
==========================================
  Files         494      494              
  Lines       83575    83614      +39     
==========================================
+ Hits        66722    66972     +250     
+ Misses      16853    16642     -211     
Flag Coverage Δ
python 80.09% <0.00%> (+0.26%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@aIbrahiim

Copy link
Copy Markdown
Contributor Author

assign set of reviewers

@github-actions

github-actions Bot commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

Assigning reviewers:

R: @jrmccluskey for label python.
Backup reviewer: tvalentyn (if the main reviewer is unresponsive, you can escalate to them by tagging their username)

Note: If you would like to opt out of this review, comment assign to next reviewer.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

@aIbrahiim

Copy link
Copy Markdown
Contributor Author

PostCommit Python Dependency checks passed

@Abacn
Abacn merged commit c9fbc95 into apache:master Nov 3, 2026
88 of 89 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The PostCommit Python Dependency job is flaky

2 participants