Skip to content

test(verify): run a Python operator the way the engine runs it #8357

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
kz930 wants to merge 20 commits into
base: main
Choose a base branch
from
Draft

test(verify): run a Python operator the way the engine runs it #8357

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
91f80d2
test(verify): run a Python operator the way the engine runs it
kz930 Sep 2, 2026
d2e8495
build: read the fork options once, outside the grouping loop
kz930 Sep 3, 2026
4659852
test(verify): call the plan preparation next door instead of copying it
kz930 Sep 3, 2026
1fb3581
test(verify): say it once, in the shape the code does not already give
kz930 Sep 3, 2026
9f51cd5
fix(verify): drop an import nothing uses
kz930 Sep 4, 2026
c2c89a4
test(verify): finalize each emitted tuple against the output schema
kz930 Sep 9, 2026
e59d0d8
test(verify): finalize each yield where it is yielded
kz930 Sep 10, 2026
cf3544a
test(verify): tell the operator what an empty port was carrying
kz930 Sep 10, 2026
0c9c6f4
test(verify): cut the doc to what the code cannot say
kz930 Sep 10, 2026
76e3e82
test(verify): write this spec's one row without the curated handlers
kz930 Sep 11, 2026
72c3570
Merge branch 'main' into feat/verify-run-operator-python
kz930 Sep 15, 2026
f4cdf6a
test(verify): carry a large binary column through the Python harness
kz930 Sep 18, 2026
153db9c
test(verify): stop handing the driver each port's declared schema
kz930 Sep 22, 2026
b210ba1
test(verify): hand a pickled binary cell to the operator as the object
kz930 Sep 23, 2026
1f0a0ef
test(verify): hand a real fitted model to the operator in the harness…
kz930 Sep 24, 2026
a2f9668
test(verify): hand a Python operator the timestamp the engine sends, …
kz930 Sep 27, 2026
135b9be
test(verify): hand a port that carried no rows its declared columns
kz930 Sep 30, 2026
b581ae0
test(verify): pickle the harness spec's model from a script file
kz930 Oct 13, 2026
a739b78
test(verify): give the executor each finished port's declared schema
kz930 Oct 13, 2026
3f7a878
test(verify): write a Python operator's infinities as the engine's wr…
kz930 Oct 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions build.sbt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -228,8 +228,14 @@
Test / fork := true,
Test / forkOptions := (Test / forkOptions).value
.withWorkingDirectory((ThisBuild / baseDirectory).value),
Test / testGrouping := (Test / definedTests).value.map { suite =>
Tests.Group(suite.name, Seq(suite), Tests.SubProcess((Test / forkOptions).value))
Test / testGrouping := {
// Read once, outside the loop: sbt lifts a `.value` written inside a lambda to

Check warning on line 232 in build.sbt

View workflow job for this annotation

GitHub Actions / Bench 

The evaluation of `/` inside an anonymous function is prohibited.
// the top of the task anyway, so leaving it there reads as one lookup per suite
// when it is not, and sbt warns about exactly that.
val forked = (Test / forkOptions).value
(Test / definedTests).value.map { suite =>
Tests.Group(suite.name, Seq(suite), Tests.SubProcess(forked))
}
}
)

Expand All @@ -240,6 +246,7 @@
.dependsOn(WorkflowOperator)
lazy val WorkflowCompilingService = (project in file("workflow-compiling-service"))
.dependsOn(WorkflowCompiler, Auth, Config, Resource)
.dependsOn(WorkflowOperator % "test->test") // reuse PythonWorkerPool in verify tests
.settings(commonModuleSettings)
.settings(
dependencyOverrides ++= Seq(
Expand Down
Loading
Loading