Skip to content

Validate property values on export to prevent writing invalid files#832

Merged
ehennestad merged 21 commits into
mainfrom
validate-properties-on-export
Jul 2, 2026
Merged

Validate property values on export to prevent writing invalid files#832
ehennestad merged 21 commits into
mainfrom
validate-properties-on-export

Conversation

@ehennestad

@ehennestad ehennestad commented Jun 19, 2026

Copy link
Copy Markdown
Collaborator

⚠️ Depends on — merge these first:

Motivation

Background

MatNWB v2.9.0 fixed some validation bugs rendering existing NWB files unreadable because of strict validation on read. In issue #776, the consensus is to relax validation on read, showing warnings instead of errors if validation fails.

PR #836 (merged) and PR #830 is ongoing work to support that.

Problem

However, relaxing validation on read creates a new problem. MatNWB currently validates property values only on construction and modification, not on export. If users are allowed to read an NWB file with invalid values, they are also free to re-export those invalid values to new NWB files.

Solution

This PR adds an export-time guard, ensuring that all properties are re-validated on file export.

Changes

  • Add types.untyped.MetaClass/validateProperties, invoked in export alongside the existing required-property and custom-constraint checks. For each set property that has a generated validate_<prop> method, it re-runs the validator and discards the result, so there are no setter side effects such as type coercion or column-name syncing. If the value is invalid it raises NWB:Export:InvalidPropertyValue, including the property name and file location, and preserves the original validator error as a cause.
  • Empty/unset optional properties are skipped, matching what export writes.
  • Validators run in the default (strict) context here, so a schema violation is an error on export even when the same check is lenient during a read.

Relationship to other work

This is the write-side counterpart to the read-side permissiveness in #776 — read lenient (warn, keep value), write strict (error). It generalizes the pattern already used for DynamicTable (checkConfig via the custom-constraint path) to all property validators. Its user-visible effect lands together with #776, so this is best reviewed and merged alongside that work.

How to test

tests.unit.types.MetaClassValidatePropertiesTest exercises the guard with a small MetaClass test double: an invalid property value raises NWB:Export:InvalidPropertyValue with the property name, location, and original cause; an unset (empty) property is skipped; a valid value passes.

Checklist

  • Have you ensured the PR description clearly describes the problem and solutions?
  • Have you checked to ensure that there aren't other open or previously closed Pull Requests for the same change?
  • If this PR fixes an issue, is the first line of the PR description fix #XX where XX is the issue number?

🤖 Generated with Claude Code

@codecov

codecov Bot commented Jun 19, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.14286% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 95.29%. Comparing base (d8f55c8) to head (900f47d).

Files with missing lines Patch % Lines
+types/+untyped/MetaClass.m 97.14% 1 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main     #832   +/-   ##
=======================================
  Coverage   95.28%   95.29%           
=======================================
  Files         219      219           
  Lines        7702     7737   +35     
=======================================
+ Hits         7339     7373   +34     
- Misses        363      364    +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ehennestad ehennestad changed the base branch from fix-read-only-validator to main June 24, 2026 19:52
ehennestad and others added 10 commits June 24, 2026 23:16
- A scalar datetime was wrapped in a cell. This is an unnecessary mutation which is removed here
- A cell array of datetimes where all values either have TimeZone info or do not, is now output as a datetime array.
Undo non-scalar cell array to datetime array. Mutating outputs should be minimised
Add test ensuring scalar dataset is NOT read as datastub
Add test checking that scalar datetime is exported to a scalar dataspace
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
… test

Co-authored-by: ehennestad <17237719+ehennestad@users.noreply.github.com>
@ehennestad ehennestad force-pushed the validate-properties-on-export branch from 484da14 to a35e1c4 Compare June 24, 2026 21:22
@ehennestad ehennestad changed the base branch from main to fix-scalar-datetime-export June 24, 2026 21:22
@bendichter

Copy link
Copy Markdown
Contributor

I don't understand the type of invalid workflow you are trying to prevent

@bendichter bendichter closed this Jun 30, 2026
@bendichter bendichter reopened this Jun 30, 2026
@ehennestad ehennestad changed the base branch from fix-scalar-datetime-export to main July 2, 2026 14:09
@ehennestad ehennestad enabled auto-merge July 2, 2026 14:10
@ehennestad ehennestad added this pull request to the merge queue Jul 2, 2026
Merged via the queue into main with commit 0343320 Jul 2, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants