Repository navigation

feat(workflow-operator): export the CSV sources as Python - #8341

Merged

aglinxinyuan merged 61 commits into

apache:mainfrom

kz930:feat/standalone-sources

Oct 26, 2026

Merged

feat(workflow-operator): export the CSV sources as Python#8341
aglinxinyuan merged 61 commits into
apache:mainfrom
kz930:feat/standalone-sources

Commits on Sep 1, 2026
- feat(workflow-compiling-service): export a workflow as a standalone Python script
  
  kz930
  and
  claude
  committed
  5b8dd57
- test(workflow-compiling-service): run an operator both ways and compare the files
  
  kz930
  and
  claude
  committed
  a7558eb
- test(workflow-compiling-service): verify a generated script against the operator it came from
  
  kz930
  and
  claude
  committed
  cba725f
Commits on Sep 2, 2026
- Merge remote-tracking branch 'upstream/main' into feat/standalone-sources
  kz930
  committed
  97e05ec
- feat(workflow-operator): export the source operators as Python
  
  kz930
  and
  claude
  committed
  41724ff
- ci: give the verify spec a job with an interpreter, and keep it out of the one without
  
  kz930
  and
  claude
  committed
  bc8fc17
- Merge remote-tracking branch 'myfork/feat/standalone-verify-harness' into feat/standalone-sources
  kz930
  committed
  9be0dbd
- test(verify): pin the no-generator case to an operator that can never have one
  
  kz930
  and
  claude
  committed
  883ea72
- chore: leave the harness to the change that introduces it
  
  kz930
  and
  claude
  committed
  5a44c3c
- Merge upstream/main
  kz930
  committed
  3f07020
- chore: leave the two verify files to #8327 as well
  
  kz930
  and
  claude
  committed
  df10bd8
Commits on Sep 9, 2026
- fix(operator): slice the raw lines before converting them
  
  kz930
  and
  claude
  committed
  73375b9
Commits on Sep 10, 2026
- fix(operator): read a CSV the way the parser does, not the way pandas does
  
  kz930
  and
  claude
  committed
  cb23c1e
- fix(operator): take the row's first string field as the file to scan
  
  kz930
  and
  claude
  committed
  c2e2eb3
- fix(operator): decode a fetched body by replacement, as the executor does
  
  kz930
  and
  claude
  committed
  1c0405f
- fix(operator): slice a JSONL file's lines before parsing them
  
  kz930
  and
  claude
  committed
  208816d
- fix(operator): take a CSV's column names from the schema
  
  kz930
  and
  claude
  committed
  566aa15
- test(operator): assert the column names the CSV reader now writes
  
  kz930
  and
  claude
  committed
  3d12fae
Commits on Sep 11, 2026
- Merge remote-tracking branch 'upstream/main' into HEAD
  kz930
  committed
  14b5571
Commits on Sep 15, 2026
- Merge branch 'main' into feat/standalone-sources
  kz930
  authored
  832b914
Commits on Sep 18, 2026
- fix(operator): read an Arrow file into the dtypes that keep its nulls
  
  kz930
  and
  claude
  committed
  edf46a4
- test(operator): pin that an Arrow read keeps a missing value apart from a NaN
  
  kz930
  and
  claude
  committed
  fe5e8c0
- fix(workflow-operator): follow the native readers in the exported sources
  
  kz930
  and
  claude
  committed
  32173ef
- Merge remote-tracking branch 'myfork/feat/standalone-sources' into feat/standalone-sources
  kz930
  committed
  292ddbe
- feat(workflow-operator): clamp the exported scan window at zero
  
  kz930
  and
  claude
  committed
  d938d35
- fix(workflow-operator): give a flattened array the executor's column names
  
  kz930
  and
  claude
  committed
  a2290cd
Commits on Sep 19, 2026
- fix(operator): order the exported JSONL read the way the schema does
  
  kz930
  and
  claude
  committed
  2ef4c15
- fix(operator): read an Arrow file's narrow numeric widths on both paths
  
  kz930
  and
  claude
  committed
  46e62a8
- fix(operator): read an Arrow file's unsigned integers as the numbers they count to
  
  kz930
  and
  claude
  committed
  b310855
- fix(workflow-operator): count the end of an exported scan window in Long
  
  kz930
  and
  claude
  committed
  3700e19
- fix(operator): decode a file scan with the charset its Encoding field names
  
  kz930
  and
  claude
  committed
  d8a9d13
- fix(operator): name the file each line came from when a file scan was asked to
  
  kz930
  and
  claude
  committed
  95ab40c
- fix(operator): type a flattened JSONL timestamp as the schema declares it
  
  kz930
  and
  claude
  committed
  f3d0db2
- fix(operator): keep a file's columns when its scan window asks for no rows
  
  kz930
  and
  claude
  committed
  3d1a4be
Commits on Sep 21, 2026
- fix(operator): read every column an Arrow file states, index or not
  
  kz930
  and
  claude
  committed
  ce52ee2
- fix(operator): read an Arrow file's columns under the names it states
  
  kz930
  and
  claude
  committed
  ce2e484
Commits on Sep 23, 2026
- fix(operator): read a zoned Arrow timestamp as the wall clock it holds
  
  kz930
  and
  claude
  committed
  719cb93
- fix(operator): ask pandas for a CSV column's type by its position
  
  kz930
  and
  claude
  committed
  5bc1a0d
- fix(operator): keep a JSONL boolean with a missing key boolean
  
  kz930
  and
  claude
  committed
  053a6fd
Commits on Sep 24, 2026
- Merge upstream/main into feat/standalone-sources
  
  kz930
  and
  claude
  committed
  6d36af8
Commits on Sep 25, 2026
- fix(operator): read what the executor reads in the exported sources
  
  kz930
  and
  claude
  committed
  07798b7
Commits on Sep 27, 2026
- fix(operator): read timestamps as the engine does, and hand Python a zoneless microsecond timestamp
  
  kz930
  and
  claude
  committed
  f874af5
- refactor(operator): leave the JSONL, Arrow and line sources to the two PRs that follow
  
  kz930
  and
  claude
  committed
  7ed4f40
Commits on Oct 1, 2026
- Merge upstream/main into feat/standalone-sources
  
  kz930
  and
  claude
  committed
  6b0a3b5
Commits on Oct 5, 2026
- Merge upstream/main into feat/standalone-sources
  
  kz930
  and
  claude
  committed
  37f8cc2
Commits on Oct 8, 2026
- fix(workflow-operator): read no rows when a headerless CSV offset passes the end
  
  kz930
  and
  claude
  committed
  280aaeb
Commits on Oct 13, 2026
- fix(workflow-operator): read a CSV DOUBLE column the way the engine's toDouble does
  
  kz930
  and
  claude
  committed
  1c9569b
Commits on Oct 14, 2026
- fix(workflow-operator): keep a CSV "NaN" a value when its column has no empty cell
  
  kz930
  and
  claude
  committed
  5acfad6
- fix(workflow-operator): read an overflowing hex double as an infinity
  
  kz930
  and
  claude
  committed
  1b126e0
Commits on Oct 23, 2026
- feat(workflow-operator): keep the double readers' notes out of the exported script
  
  kz930
  and
  claude
  committed
  d0c137f
Commits on Oct 24, 2026
- feat(workflow-operator): read a wide table's typed columns in a loop
  
  kz930
  and
  claude
  committed
  7fe7662
- feat(workflow-operator): type an empty CSV read without restating its columns
  
  kz930
  and
  claude
  committed
  d53e50a
- feat(workflow-operator): share one Java trim, int and double parser across the exports
  
  kz930
  and
  claude
  committed
  bfe9997
- feat(workflow-operator): state a CSV read's columns once, with their types
  
  kz930
  and
  claude
  committed
  7700f83
- feat(workflow-operator): order doubles by one shared Double.compareTo key
  
  kz930
  and
  claude
  committed
  a9f2143
Commits on Oct 25, 2026
- feat(workflow-operator): number engine keys in one shared helper
  
  kz930
  and
  claude
  committed
  9787117
Commits on Oct 26, 2026
- feat(workflow-operator): read a cast boolean the way the engine trims and parses it
  
  kz930
  and
  claude
  committed
  5eb8071
- feat(workflow-operator): read a cast double the way the engine parses it
  
  kz930
  and
  claude
  committed
  1c80ab6
- feat(workflow-operator): let the shared key codes tell a signed zero apart
  
  kz930
  and
  claude
  committed
  915fae5
- feat(workflow-operator): space the shared cast helpers by one blank line
  
  kz930
  and
  claude
  committed
  377b4bb
- feat(workflow-operator): infer each CSV source's schema once per export
  
  kz930
  and
  claude
  committed
  253241c

Commits

Commits on Sep 1, 2026

Commits on Sep 2, 2026

Commits on Sep 9, 2026

Commits on Sep 10, 2026

Commits on Sep 11, 2026

Commits on Sep 15, 2026

Commits on Sep 18, 2026

Commits on Sep 19, 2026

Commits on Sep 21, 2026

Commits on Sep 23, 2026

Commits on Sep 24, 2026

Commits on Sep 25, 2026

Commits on Sep 27, 2026

Commits on Oct 1, 2026

Commits on Oct 5, 2026

Commits on Oct 8, 2026

Commits on Oct 13, 2026

Commits on Oct 14, 2026

Commits on Oct 23, 2026

Commits on Oct 24, 2026

Commits on Oct 25, 2026

Commits on Oct 26, 2026