fix(context): quoted allowed values, dotted names, explicit confidence phrases, strict protection after resolution

[kevincostner17]

Summary

Fixes four context-compiler parsing and compile bugs.

• Allowed-values lists (Context allowed-values lists split single values on 'and', 'or' and '/' #301): split_value_list keeps quoted values whole and never splits on /. When , or ; is present, only those separate values; and/or split only the last item (a, b or c). Without commas, and/or still separate values (Deduplicate by email and phone). Trinidad and Tobago, Chile and N/A, OK now compile correctly, and fd.validate no longer reports those values as violations. The docs say ambiguous values need quoting.
• Dotted column names (Context sentence splitter breaks dotted column names and binds the rule to another column #302): a sentence ends at . only before whitespace or end of line, so file.name is unique. binds to file.name instead of splitting and attaching the rule to name. Decimals and U.S., U.K., stay in one sentence.
• Confidence gates (compile_context reads any bare number as an imputation confidence gate #303): parse_confidence accepts a number only after a confidence word (confidence, certainty, probability, sure) or with %/percent. only if 3 neighbours agree and when age > 50 are now unparsed (and raise under strict) instead of becoming gates of 0.03 and 0.5.
• Strict protection conflicts (strict=True does not raise protection_conflict for a schema-free policy passed to fd.clean #304): resolve_policy re-runs the restated-rule and protection-conflict checks after resolving a schema-free policy, merging new issues without duplicates. fd.clean(df, policy=schema_free_policy, strict=True) now raises protection_conflict, like the columns= and context= flows. Non-strict mode records the issue and demotes the repair.

Behaviour notes: / no longer splits dedup keys either; sentences with no space after a period are no longer split; unquoted abbreviations in value lists lose their final period (quote them to keep it).

Tests

New tests/context/test_compiler_parsing_fixes.py:

• the repro from each issue;
• list-splitting cases (Oxford comma, ;, brackets, CA, OR, WA, quotes, apostrophes);
• the dedup-key split is unchanged;
• dotted names with and without column renaming; decimals and abbreviations;
• bare numbers are unparsed and raise under strict, while explicit confidence phrasings still parse;
• schema-free vs schema-bound flows give the same result; non-strict still protects the column;
• a rule restated in different words is caught at resolution without duplicate issues.

Verification

• ruff check .: clean
• mypy src/freshdata: no issues (202 files)
• py3.12: 4287 passed, 6 skipped
• py3.9: 4283 passed, 10 skipped
• Golden and TruthBench expectations unchanged

Closes #301
Closes #302
Closes #303
Closes #304

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 3f66f571-8950-4918-bca4-ea160d792543

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

FreshData benchmark report — performance

• freshdata: ?
• python: ?
• platform: ?

fixture	n_rows	n_cols	p50 s	p95 s	peak MB	repair %	false-repair %	preserve %	trust	monotonic	export %

Authored-code reduction (Metric 6)