[PySpark] - Add partial support for window function#318
Conversation
|
I don't know how but the next step is to implement proper handling of Column expressions in ordering (including asc/desc). |
06feddd to
9718dcd
Compare
|
Hi @binste I would love to see this land. Thanks for the foundation here. If you're not actively working on it, happy to pick it up. I can:
The plan is to introspect the underlying @evertlammerts: is this PR still on the roadmap? If a fresh take is welcome, I can put up a continuation PR crediting the original author. |
|
Hi @aaron-ang ! Maybe a mix-up, I think I was never involved in this PR (and am also not a duckdb maintainer) but fully agree that it would be great to see this land. Good luck and thanks for picking it up! |
|
hi @binste, apologies for the mixup. i meant to tag @mariotaddeucci. |
Initial support for window functions by introducing a PySpark-like
WindowAPI and aWindowSpecclass.The implementation is partial: partitioning, basic ordering-by-names, and frame specification (rowsBetween / rangeBetween) are supported and converted to SQL window clauses, but extracting ordering direction from Column expressions is not yet implemented (see the TODO in WindowSpec._columns_as_str). Using Column expressions in orderBy currently raises a
ContributionsAcceptedError.Additionally, a set of functions was added for convenient use with .over(window):