Repository navigation
TPC-DS suite follow ups - #40741
TPC-DS suite follow ups#40741
Conversation
* Some 10 MB test failed due to filtered dataset has no result, either intermediate or final query result, and may cause ZeroDivision, etc. Fixed dataset to add more items so they query result has non-zero row * Fix known error to check whole error stack instead of the first line * Fix log spam on Flink and Spark runners * Enable query90,98
There was a problem hiding this comment.
📋 Review Summary
This PR updates the TPC-DS benchmark suite in Java and Python to enable queries 90 and 98, suppress excessive runner log spam on Flink and Spark via logging configuration files, and preserve the full error stack trace in TpcdsQueryResult for regression checks while still summarizing errors in the printed table.
🔍 General Feedback
- Preserving the full exception string in
TpcdsQueryResult.error_messagewhile moving the single-line extraction toformat_summary_tableis a good improvement for makingverify_no_regressionsreliable across multi-line stack traces. - Watch out for an
IndexErroredge case inrun_single_querywhenstr(exc)is empty.
💡 Tip: Comment /gemini review on this pull request to trigger a new Gemini code review.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #40741 +/- ##
==========================================
- Coverage 80.04% 79.92% -0.13%
==========================================
Files 490 493 +3
Lines 82916 83408 +492
==========================================
+ Hits 66373 66661 +288
- Misses 16543 16747 +204
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Assigning reviewers: R: @chamikaramj for label java. Note: If you would like to opt out of this review, comment Available commands:
The PR bot will only process comments in the main thread (not review comments). |
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
|
R: @kennknowles @aIbrahiim (bring newly added PostCommit green: https://github.com/apache/beam/actions/runs/39878843497/job/120171031464) |
|
Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment |
|
The local environment test is failing in a way I have seen before but I don't remember. I think it is printing the Java version then exiting, but not sure why it doesn't give a full error message. I've seen this across many PRs and I assume it is an environmental issue. |
Some 10 MB test failed due to filtered dataset has no result, either intermediate or final query result, and may cause ZeroDivision, etc. Fixed dataset to add more items so they query result has non-zero row
Fix known error to check whole error stack instead of the first line
Fix log spam on Flink and Spark runners
Enable query90,98