Repository navigation
Add Prism integration tests for Delta IO - #39764
goutamadwant wants to merge 4 commits into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #39764 +/- ##
=============================================
- Coverage 80.04% 58.17% -21.88%
- Complexity 0 13101 +13101
=============================================
Files 490 2523 +2033
Lines 82916 265755 +182839
Branches 0 10795 +10795
=============================================
+ Hits 66373 154612 +88239
- Misses 16543 105332 +88789
- Partials 0 5811 +5811 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Assigning reviewers: R: @ahmedabu98 for label java. Note: If you would like to opt out of this review, comment Available commands:
The PR bot will only process comments in the main thread (not review comments). |
|
Reminder, please take a look at this pr: @ahmedabu98 @liferoad |
|
Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment R: @Abacn for label java. Available commands:
|
|
R: @chamikaramj |
|
Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment |
| @@ -0,0 +1,141 @@ | |||
| /* | |||
| * Licensed to the Apache Software Foundation (ASF) under one | |||
There was a problem hiding this comment.
Instead of adding new tests, can we update the config to run existing tests with the Prism runner ?
There was a problem hiding this comment.
reused the four existing Managed IO tests and removed this new test class @chamikaramj
| - name: run Delta IO Prism integration tests | ||
| uses: ./.github/actions/gradle-command-self-hosted-action | ||
| with: | ||
| gradle-command: :sdks:java:io:delta:prismIntegrationTest |
There was a problem hiding this comment.
Please also do a small update to the trigger file so that the test suite get triggered for this PR.
https://github.com/apache/beam/blob/master/.github/trigger_files/beam_PreCommit_Java_Delta_IO_Direct.json
There was a problem hiding this comment.
done @chamikaramj the Delta suite ran and passed, including the Prism step.
|
waiting on author |
|
@chamikaramj review fixes are pushed. can you PTAL when you get a chance? thanks! |
|
@chamikaramj also could you rerun the failed Dataflow Examples check? WordCountIT hit a GCS read timeout while staging a JAR. unrelated to my change and I don't have access for it. |
Adds Prism coverage by running four existing Managed Delta IO tests: snapshot reads, version and timestamp reads, and change-data reads.
The dedicated Gradle task builds the local Prism binary and runs these tests in LOOPBACK mode using their existing local fixtures. No new test class or cloud credentials are needed. The existing DirectRunner and Dataflow test selections are unchanged.
The Delta IO precommit workflow runs the Prism task, and the trigger file is updated to run the suite for this PR.
Fixes #38557
Testing
:sdks:java:io:delta:prismIntegrationTest: 4 tests passed with Java 17 and the locally built Prism binary.:beam-test-gha:preCommit: passed.Checklist
CHANGES.mdis not needed because this change only adds test and CI coverage.