fix(freshcore): fall back for missforest, per-column strategies, non-finite outliers and nullable bools; honour native duplicate counts

[kevincostner17]

Summary

The FreshCore adapter ran several configs natively that its v1 kernels silently get wrong. They now fall back to the pandas reference through the existing fallback path, so fallback_policy="error" raises FallbackError and reports record the event.

• FreshCore engine runs impute='missforest' and impute_strategy natively and imputes nothing #322: impute="missforest" and per-column impute_strategy fall back, with the same reasons as PlanGenerator.fallback_reason(), so fd.plan() and the recorded event agree. PlanGenerator now treats an empty impute_strategy as a no-op, like the pandas imputer.
• FreshCore outlier fences include ±inf, so zscore/IQR flag nothing when a column holds inf #334: when outliers is set and a float column holds ±inf, the run falls back. Native fences don't exclude non-finite values, so zscore and IQR flagged nothing. The check is a vectorised np.isinf over float columns.
• FreshCore imputation skips nullable boolean columns that the pandas engine fills #335: impute="mode"/"auto" falls back when a nullable boolean column has missing values, which the native kernel never fills.
• FreshCore engine skips duplicate detection; duplicate_ratio_action='error' never raises #323 (adapter side): when the native result has duplicates_detected, it goes through report_detected_duplicates(), in pandas action order and using the row count after empty-row removal. With drop_duplicates=True, the native drop count goes through check_duplicate_ratio(), so duplicate_ratio_action="error" raises and high ratios warn. If the key is missing, detection is skipped, except under duplicate_ratio_action="error", where the run falls back so the escalation isn't lost.
• docs/fallback-matrix.md, docs/backends.md and docs/freshcore.md list the new triggers.

Known gap: with fix_dtypes=True, native string casts can still create boolean columns with missing values or float columns holding inf, which the input-dtype checks don't see.

Tests

New tests/test_execution/test_freshcore_fallbacks.py, using a fake native module:

• Each issue repro records the fallback reason, matches the pandas output and actions, and raises under fallback_policy="error".
• fd.plan() shows the same reason as the recorded fallback.
• Supported data stays native.
• FreshCore engine skips duplicate detection; duplicate_ratio_action='error' never raises #323: a reported duplicates_detected produces the pandas detection action, warning and recommendation, and raises DuplicateRatioError under "error". A missing key skips detection, or falls back under "error". drop_duplicates=True escalates and warns like pandas.

Verification

• ruff check . clean; mypy src/freshdata clean (202 files)
• Python 3.12: 4271 passed, 6 skipped
• Python 3.9: 4267 passed, 10 skipped
• 19 of the 26 new tests fail against main's src

Closes #322
Closes #334
Closes #335
Refs #323 (Python adapter only; the native engine still needs to report duplicates)

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 83530d0a-42a1-433d-9781-92f06eb470e8

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

FreshData benchmark report — performance

• freshdata: ?
• python: ?
• platform: ?

fixture	n_rows	n_cols	p50 s	p95 s	peak MB	repair %	false-repair %	preserve %	trust	monotonic	export %

Authored-code reduction (Metric 6)