Skip to content

Unnamed group mixin improvements#784

Open
ehennestad wants to merge 22 commits into
mainfrom
unnamed-group-mixin-improvements
Open

Unnamed group mixin improvements#784
ehennestad wants to merge 22 commits into
mainfrom
unnamed-group-mixin-improvements

Conversation

@ehennestad

@ehennestad ehennestad commented Jan 24, 2026

Copy link
Copy Markdown
Collaborator

Motivation

Background — PR #705 added matnwb.mixin.HasUnnamedGroups, a convenience layer that lets you add, dot-access, and neatly display the members of a container's unnamed subgroup (e.g. ProcessingModule) instead of going through the raw types.untyped.Set API.

Problem — For a user building a table, that convenience stopped short:

  • Types whose unnamed members are datasets rather than groups — the vectordata columns of a DynamicTable, the baseimage set of an Images — still exposed a bare [1×1 types.untyped.Set] property with no add, no dot-access, and no grouped display.
  • AlignedDynamicTable already grouped its own dynamictable set, but its empty-group hint pointed users at a generic add method rather than the idiomatic addColumn / addCategory.

Solution — Extend the convenience layer to unnamed datasets and add a routing hook so add() dispatches to each type's idiomatic method (DynamicTableaddColumn, AlignedDynamicTableaddCategory).

What changed

  • DynamicTable and Images now expose their unnamed members through add / dot-access / grouped display instead of a raw types.untyped.Set property.
  • add() routes to the type-specific method: vectordata additions go through addColumn, category additions through addCategory.
  • Empty-group display hints are now type-specific — AlignedDynamicTable points at addColumn (for columns) and addCategory (for category tables) instead of a generic add.
Implementation notes
  • The mixin is matnwb.mixin.HasUnnamedGroups. The generator now applies it to types with unnamed datasets (declaring GroupPropertyNames, e.g. ["vectordata"] on DynamicTable, ["baseimage"] on Images) — previously only unnamed groups qualified.
  • Routing is via a protected handleUnnamedGroupAdd(groupName, name, value) hook. matnwb.neurodata.DynamicTableBase routes vectordataaddColumn; matnwb.neurodata.AlignedDynamicTableBase calls the superclass hook first, then routes category tables → addCategory.
  • AlignedDynamicTable is the first type to inherit an unnamed group (vectordata, now on DynamicTable) while declaring its own (dynamictable). matnwb.neurodata.internal.collectConstantPropertiesAcrossHierarchy gathers GroupPropertyNames across the class hierarchy so both groups resolve. This logic affects no pre-existing type — the two-level case only arises once DynamicTable gains the mixin in this PR.

Examples

Example 1 — types.core.Images

image_data = randi(255, [3, 200, 200], 'uint8');  % dims: [RGB, width, height]
rgb_image = types.core.RGBImage('data', image_data, 'resolution', 70.0, 'description', 'RGB image');
image_collection = types.core.Images('description', 'A collection of images presented to the subject.');

Before — add/get go through the raw Set:

image_collection.baseimage.set('rgb_image', rgb_image);
rgb_image = image_collection.baseimage.get('rgb_image');
image_collection
image_collection = 

  Images with properties:

          baseimage: [1×1 types.untyped.Set]
        description: 'A collection of images presented to the subject.'
    order_of_images: []

Afteradd / dot-access, grouped display:

image_collection.add('rgb_image', rgb_image);
rgb_image = image_collection.rgb_image;
image_collection
image_collection = 

  Images with properties:

        description: 'A collection of images presented to the subject.'
    order_of_images: []

   baseimage group with 1 entry:
          rgb_image: [1×1 types.core.RGBImage]

Example 2 — types.hdmf_common.DynamicTable

dt = types.hdmf_common.DynamicTable( ...
    'description', 'my table', ...
    'ColumnA', types.hdmf_common.VectorData('description', 'a column'), ...
    'colnames', 'ColumnA')

Beforevectordata is a raw Set property:

dt = 

  DynamicTable with properties:

           colnames: {'ColumnA'}
        description: 'my table'
                 id: [1×1 types.hdmf_common.ElementIdentifiers]
    meanings_tables: [0×1 types.untyped.Set]
         vectordata: [1×1 types.untyped.Set]

Aftervectordata is a group, dt.add(...) routes to addColumn:

dt = 

  DynamicTable with properties:

           colnames: {'ColumnA'}
        description: 'my table'
                 id: [1×1 types.hdmf_common.ElementIdentifiers]
    meanings_tables: [0×1 types.untyped.Set]

   vectordata group with 1 entry:
            ColumnA: [1×1 types.hdmf_common.VectorData]

Example 3 — types.hdmf_common.AlignedDynamicTable

An AlignedDynamicTable has two unnamed groups with different idiomatic add-methods.

adt = types.hdmf_common.AlignedDynamicTable('description', 'aligned', 'categories', {}, 'colnames', {})

Beforedynamictable is grouped but hints at a generic add; vectordata is a raw Set:

adt = 

  AlignedDynamicTable with properties:

         categories: ''
           colnames: ''
        description: 'aligned'
                 id: []
    meanings_tables: [0×1 types.untyped.Set]
         vectordata: [0×1 types.untyped.Set]

   dynamictable: (no entries)
    Tip: Use the 'add' method to add data objects to this group.

After — both groups shown, each with its own hint:

adt = 

  AlignedDynamicTable with properties:

         categories: ''
           colnames: ''
        description: 'aligned'
                 id: []
    meanings_tables: [0×1 types.untyped.Set]

   dynamictable: (no entries)
    Tip: Use the 'addCategory' method to add category tables.

   vectordata: (no entries)
    Tip: Use the 'addColumn' method to add column data.

How to test

% Dataset-backed unnamed group now uses the convenience interface + routing
dt = types.hdmf_common.DynamicTable(...
    'description', 't', ...
    'colnames', 'ColumnA', ...
    'ColumnA', types.hdmf_common.VectorData('description', 'a column', 'data', (10:12)'));
    
assert(isprop(dt, 'ColumnA'))                       % dot-access to the column

dt.add('ColumnB',  types.hdmf_common.VectorData('description', 'b', 'data', (1:3)'));

assert(any(strcmp(dt.colnames, 'ColumnB')))         % add() routed through addColumn

% Two-level unnamed-group hierarchy resolves both groups with type-specific hints
adt = types.hdmf_common.AlignedDynamicTable('description', 'a');
disp(adt)   % shows dynamictable (addCategory) + vectordata (addColumn) tips

Checklist

  • Have you ensured the PR description clearly describes the problem and solutions?
  • Have you checked to ensure that there aren't other open or previously closed Pull Requests for the same change?
  • If this PR fixes an issue, is the first line of the PR description fix #XX where XX is the issue number?

🤖 Generated with Claude Code

@ehennestad ehennestad marked this pull request as draft January 24, 2026 20:30
@ehennestad ehennestad marked this pull request as ready for review April 13, 2026 09:30
@codecov

codecov Bot commented Apr 13, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 92.00000% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 95.18%. Comparing base (13b0f9a) to head (fe01dd8).

Files with missing lines Patch % Lines
+matnwb/+neurodata/DynamicTableBase.m 50.00% 5 Missing ⚠️
+matnwb/+neurodata/AlignedDynamicTableBase.m 78.57% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #784      +/-   ##
==========================================
- Coverage   95.43%   95.18%   -0.25%     
==========================================
  Files         228      229       +1     
  Lines        8064     8124      +60     
==========================================
+ Hits         7696     7733      +37     
- Misses        368      391      +23     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ehennestad ehennestad marked this pull request as draft April 15, 2026 20:39
@ehennestad ehennestad force-pushed the unnamed-group-mixin-improvements branch 3 times, most recently from f12e5db to b9517fb Compare June 25, 2026 06:47
@ehennestad ehennestad force-pushed the unnamed-group-mixin-improvements branch 2 times, most recently from 6552e99 to fafc9bf Compare June 26, 2026 15:10
@ehennestad ehennestad changed the base branch from main to add-dynamictable-base-class June 26, 2026 15:11
@bendichter

Copy link
Copy Markdown
Contributor

Could use more motivation and examples

@ehennestad ehennestad force-pushed the add-dynamictable-base-class branch from 9ad5c40 to ea8b815 Compare July 1, 2026 12:00
@ehennestad ehennestad force-pushed the unnamed-group-mixin-improvements branch from 7c1e3c1 to 525159e Compare July 2, 2026 13:26
@ehennestad ehennestad marked this pull request as ready for review July 2, 2026 13:51
@ehennestad ehennestad changed the base branch from add-dynamictable-base-class to codex/aligned-dynamic-table-add-category July 2, 2026 13:51
@ehennestad ehennestad marked this pull request as draft July 2, 2026 13:51
@bendichter bendichter marked this pull request as ready for review July 2, 2026 14:28
@ehennestad ehennestad changed the base branch from codex/aligned-dynamic-table-add-category to main July 2, 2026 14:28
@ehennestad ehennestad changed the base branch from main to to-table-support-aligneddynamictable July 2, 2026 14:58
@ehennestad ehennestad changed the base branch from to-table-support-aligneddynamictable to codex/aligned-dynamic-table-add-category July 2, 2026 14:58
@ehennestad ehennestad force-pushed the unnamed-group-mixin-improvements branch from 930c148 to 234ecdf Compare July 2, 2026 19:13
@ehennestad ehennestad changed the base branch from codex/aligned-dynamic-table-add-category to main July 2, 2026 19:23
@ehennestad ehennestad changed the base branch from main to codex/aligned-dynamic-table-add-category July 2, 2026 19:24
@ehennestad ehennestad marked this pull request as draft July 2, 2026 20:02
@ehennestad ehennestad marked this pull request as ready for review July 2, 2026 20:02
@ehennestad ehennestad changed the base branch from codex/aligned-dynamic-table-add-category to main July 2, 2026 21:57
- Replaces the abstract protected GroupPropertyNames property with a private dependent property and a static protected method for retrieval.
- Adds caching for GroupPropertyNames
-Implements getGroupPropertyNamesAcrossTypeHierarchy to aggregate unnamed group property names across the class hierarchy.
- Updates fillClass to generate appropriate method overrides for classes with anonymous groups or data.
Refactored the logic to add 'matnwb.mixin.HasUnnamedGroups' as a superclass by combining checks for 'hasAnonGroups' and 'hasAnonData' into a single condition.
ehennestad and others added 18 commits July 3, 2026 00:23
Add methods that are present on the Set class, to support accessing Anon values from the HasUnnamedGroups mixin.
- Moved mixin setup and DynamicTable validation logic from fillBody and fillCheck into the main constructor generation flow.
- Introduced isDynamicTableDescendent helper for cleaner ancestry checks.
- Removed the now-redundant fillCheck function.
Fix warning suppression in sprintf statement (add escape char)
- Renamed getStaticProperties to getNonDynamicProperties with internal caching
- Added isDynamicProperty helper
setupHasUnnamedGroupsMixin() was gated on a class directly declaring
unnamed (anonymous/constrained) groups. Subclasses that inherit the
mixin from an ancestor without declaring their own groups (e.g.
DynamicTable subclasses such as Units, TimeIntervals, PlaneSegmentation)
therefore never ran the setup, because the ancestor's call is guarded to
execute only for the leaf class being constructed.

Derive the gate from the full processed type hierarchy so the call is
emitted whenever the class or any ancestor uses the mixin. Regenerate the
affected core types.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Simplify, use string and shape validators to normalize input
Fix failing test. The HasUnnamedGroups mixin adds every element as a dynamic property, so the old verification is not valid anymore
Routes vectordata additions through addColumn, mirroring the existing
hook in AlignedDynamicTableBase that routes category additions through
addCategory. AlignedDynamicTableBase now inherits from DynamicTableBase
(matching the NWB schema hierarchy) and delegates to the superclass hook
before applying its own category logic.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
For alignedDynamicTable, display tip now recommends addCategroy and addVector
Regenerate core classes from the merged generator so the HasUnnamedGroups
private-constant handling and the AlignedDynamicTable/DynamicTable base-class
integration are reflected consistently in the generated +types.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The mixin work renamed the hidden TypeName property to a getTypeShortName method
(in types.untyped.MetaClass), but ensureAlignedTableConsistency still referenced
obj.TypeName, which no longer exists. Route it through obj.getTypeShortName().

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Function was moved and generalised
@ehennestad ehennestad force-pushed the unnamed-group-mixin-improvements branch from 53d10d0 to fe01dd8 Compare July 2, 2026 22:24
@ehennestad ehennestad requested a review from bendichter July 2, 2026 22:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants