[e-strauss]

Summary

Two correctness bugs in the polars/pandas map impls.

1. polars .dt reads returned different values than pandas. polars counts weekdays from Monday=1 and nanoseconds past the second; pandas counts from Monday=0 and only 0..999 past the microsecond. Both polars paths (PolarsAccessorMapOp and DtExpr.to_polars, used by AssignMapOp and mask selections) now go through one shared helper, like polars_string_method for .str:

polars_dt_attribute(obj, attr)          # logical/_map_ops.py
  is_month_end   -> obj.dt.month_end() == obj
  name = POLARS_ATTR_NAME_MAP.get(attr, attr)
  weekday        -> obj.dt.weekday() - 1      # dayofweek, day_of_week, weekday
  nanosecond     -> obj.dt.nanosecond() % 1000
  otherwise      -> obj.dt.<name>()

POLARS_ATTR_NAME_MAP also gains the pandas aliases polars lacks: day_of_week, day_of_year, daysinmonth. Before, these crashed in PolarsAccessorMapOp and refused polars in DtExpr.

2. pd.to_datetime ignored plan-fed options. Both DatetimeConversionOp impls passed self.args/self.kwargs straight through, so an option computed in the plan reached pandas/polars as an unresolved OperandRef. They now resolve through _extract_args_and_kwargs(inputs) like the other CallMapOp impls, so polars_datetime_kwargs sees the real values.

 PandasDatetimeConversionOp.process
-  pd.to_datetime(inputs[0], *self.args, **self.kwargs)
+  obj, args, kwargs = self._extract_args_and_kwargs(inputs)
+  pd.to_datetime(obj, *args, **kwargs)

Evidence

• Before:

  x["d"].dt.dayofweek            # 2024-01-01 (Mon), 2024-01-07 (Sun)
    pandas -> [0, 6]
    polars -> [1, 7]
  pd.to_datetime(x["s"], format=skrub.deferred(lambda: "%Y-%m-%d")())
    pandas -> TypeError: expected str, got OperandRef
    polars -> TypeError: argument of type 'OperandRef' is not a container or iterable
  

  30 of the new test cases failed on main.
  After: polars returns [0, 6], and both backends parse with the computed format.

New tests in tests/optimizer/logical/test_map_ops.py:

• test_dt_attribute_matches_pandas[attr-path-backend]: dayofweek, day_of_week, weekday, nanosecond, day_of_year, daysinmonth x standalone accessor / assign / mask selection x pandas / polars. Each case checks the result against eager pandas, and on polars checks that the result is a polars object, so a pandas fallback cannot hide the bug.
• test_to_datetime_resolves_graph_fed_format[pandas|polars]: the repro above, run end to end.
• TestDatetimeConversionOp.test_resolves_graph_fed_kwarg / _positional: OperandRef options at the op level, on both backends.
• TestAccessorMapOp.test_polars_matches_pandas_values replaces test_polars_dayofweek, which asserted the off-by-one value.

Full suite locally: 1914 passed, 13 failed. The same 13 fail on a clean checkout of main. They are the tests/adapters encoders and the random forest histogram tests, which need the compiled Rust runtime.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

Flag	Coverage Δ
unittests	92.73% <100.00%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
stratum/optimizer/logical/_column_expr.py	97.15% <100.00%> (-0.03%)	⬇️
stratum/optimizer/logical/_map_ops.py	94.91% <100.00%> (+0.22%)	⬆️
stratum/optimizer/physical/_map_execs.py	91.95% <100.00%> (-1.15%)	⬇️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.