Fail on columns the contract does not declare, when the schema says so

[markhm]

Fixes #1722.

Problem

A contract cannot say that a file must have no columns besides the ones it declares. An extra column passes unnoticed. ODCS v3.2.0 has no field for this.

What this changes

A schema opts in with a custom property, named after the same idea in JSON Schema:

schema:
  - name: orders
    customProperties:
      - property: additionalProperties
        value: false

• The schema then gets a check, "Check that orders has no fields the contract does not declare" (type model_no_additional_fields, dimension conformity). It fails on columns the schema does not declare and names them: Fields not in the contract: note, with additional_fields in the diagnostics.
• Names compare case-insensitively, like the existing field presence check.
• It reads only the column names, never the rows, so it also runs with --metadata-only. On a CSV file it reads the {model}__raw__ view; elsewhere the table itself.
• It works on every server read through ibis: CSV, JSON and Parquet files, and database tables. SAP HANA has its own engine and is not covered.
• The value may be the boolean false or the text "false".
• Without the custom property nothing changes.

Tests

tests/test_test_additional_fields.py: 9 tests, covering an extra column, exactly the declared columns, case, no custom property, text values, a JSON file, --metadata-only, and a DuckDB database table. uv run pytest: all pass. ruff: clean.

Docs

schema.md: a row in the checks table and a short section on the setting. CHANGELOG entry under Added.

Note

This PR and #1724 touch a comment in check_spec.py and the same bullet in schema.md, so the one merged second needs a small rebase. I'll do that.

• Tests pass (uv run pytest)
• Code formatted (uv run ruff check --fix && uv run ruff format)
• Docs updated (if relevant)
• CHANGELOG.md entry added

🤖 Generated with Claude Code

[jschoedl]

Thanks. I think it would make sense to propose this to ODCS so it becomes part of the standard instead of a customProperty. Someone already did this a year ago (bitol-io/open-data-contract-standard#201), but the issue was closed as stale.

As soon as this PR and the one for the fixed order are merged, you could create a new ODCS issue for both additions. I guess it helps to reference the PRs and describe why you need a fixed layout CSV in your concrete setup.

[jschoedl]

On a Kafka server this check always passes. JSON messages are decoded straight into the contract's columns (_decode_json builds the arrow table from to_arrow_schema(schema_obj.properties)), and unframed Avro is read with the schema derived from the contract, so undeclared fields are gone before the check sees the table.

We either support Kafka or at least report the check as a warning (metric = MetricType.UNSUPPORTED, preset_result = "warning", like the unenforced checks above), and mention it in the docs.