Repository navigation
Conversation
…ython Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Automated Reviewer SuggestionsBased on the
|
|
| config | throughput | MB/s | latency | max Δ latest / 7d | |
|---|---|---|---|---|---|
| 🔴 | bs=10 sw=10 sl=64 | 519 | 0.317 | 18,855/23,785/23,785 us | 🔴 +16.3% / 🔴 +94.6% |
| 🔴 | bs=100 sw=10 sl=64 | 1,217 | 0.743 | 78,252/120,539/120,539 us | 🔴 +13.9% / 🔴 +38.3% |
| ⚪ | bs=1000 sw=10 sl=64 | 1,439 | 0.878 | 692,198/775,517/775,517 us | ⚪ within ±5% / 🟢 -11.3% |
Baseline details
Latest main 8fbd4f2 from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---|---|---|---|---|
| bs=10 sw=10 sl=64 | throughput | 519 tuples/sec | 568 tuples/sec | 1,019 tuples/sec | -8.6% | -49.1% |
| bs=10 sw=10 sl=64 | MB/s | 0.317 MB/s | 0.347 MB/s | 0.622 MB/s | -8.6% | -49.0% |
| bs=10 sw=10 sl=64 | p50 | 18,855 us | 16,213 us | 10,144 us | +16.3% | +85.9% |
| bs=10 sw=10 sl=64 | p95 | 23,785 us | 24,119 us | 12,225 us | -1.4% | +94.6% |
| bs=10 sw=10 sl=64 | p99 | 23,785 us | 24,119 us | 15,527 us | -1.4% | +53.2% |
| bs=100 sw=10 sl=64 | throughput | 1,217 tuples/sec | 1,232 tuples/sec | 1,371 tuples/sec | -1.2% | -11.2% |
| bs=100 sw=10 sl=64 | MB/s | 0.743 MB/s | 0.752 MB/s | 0.837 MB/s | -1.2% | -11.2% |
| bs=100 sw=10 sl=64 | p50 | 78,252 us | 78,450 us | 79,910 us | -0.3% | -2.1% |
| bs=100 sw=10 sl=64 | p95 | 120,539 us | 105,791 us | 87,155 us | +13.9% | +38.3% |
| bs=100 sw=10 sl=64 | p99 | 120,539 us | 105,791 us | 106,643 us | +13.9% | +13.0% |
| bs=1000 sw=10 sl=64 | throughput | 1,439 tuples/sec | 1,442 tuples/sec | 1,415 tuples/sec | -0.2% | +1.7% |
| bs=1000 sw=10 sl=64 | MB/s | 0.878 MB/s | 0.88 MB/s | 0.864 MB/s | -0.2% | +1.6% |
| bs=1000 sw=10 sl=64 | p50 | 692,198 us | 693,654 us | 780,407 us | -0.2% | -11.3% |
| bs=1000 sw=10 sl=64 | p95 | 775,517 us | 749,780 us | 817,814 us | +3.4% | -5.2% |
| bs=1000 sw=10 sl=64 | p99 | 775,517 us | 749,780 us | 848,683 us | +3.4% | -8.6% |
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,385.09,200,128000,519,0.317,18854.51,23785.19,23785.19
1,100,10,64,20,1642.87,2000,1280000,1217,0.743,78251.84,120539.34,120539.34
2,1000,10,64,20,13900.10,20000,12800000,1439,0.878,692197.93,775516.62,775516.62
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #9154 +/- ##
=========================================
Coverage 91.81% 91.81%
- Complexity 5069 5070 +1
=========================================
Files 1271 1271
Lines 55208 55207 -1
Branches 6987 6987
=========================================
+ Hits 50688 50689 +1
+ Misses 2800 2799 -1
+ Partials 1720 1719 -1
*This pull request uses carry forward flags. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
@carloea2 This removes the explanatory comments that the merged export PRs left in exported Python, and keeps a short note in Scala where the reason matters. The generated statements are unchanged across all 1,098 verification scripts. Could you take a look? |
What changes were proposed in this PR?
The Python that "Export to Python" writes carried notes meant for whoever maintains the generator: why a frame is copied, why a row is dropped before scoring, why an empty cell keeps its row. Someone running the exported script does not need them, and they made the script longer to read. This PR takes those comments out of the exported code in 24 operators, and keeps a one or two line note in Scala above the string where the reason is worth keeping.
Only comments go. The translator's own markers stay: the
# [Operator]heading over each section,# --- Output ---, and the# TODOit writes for an operator it cannot export yet. The Python that operators run inside the engine is not touched.Any related issues, documentation, discussions?
The comments came in with the export PRs #8365, #8367, #8368, #8504 and #8340.
#9035 also changes Unnest String's export. Whichever of the two merges second keeps the other's change.
How was this PR tested?
No generated statement changes, only comment lines. Running the verification suite before and after, the 1,098 scripts it generates for 142 operators are identical once comment lines and temporary paths are set aside, and none of them has an explanatory comment left. All of
WorkflowOperator/testpasses (2,912 tests), as doscalafmtCheckandscalafixAll --check.Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Opus 5.5)
🤖 Generated with Claude Code