Skip to content

fix(dataset): preserve Weka unions through HF Arrow null padding - #60

Draft
rawsh wants to merge 1 commit into
SemiAnalysisAI:masterfrom
rawsh:rob/agentx-hf-null-padding-20261022
Draft

rawsh wants to merge 1 commit into
SemiAnalysisAI:masterfrom
rawsh:rob/agentx-hf-null-padding-20261022

Conversation

@rawsh

@rawsh rawsh commented Oct 22, 2026

Copy link
Copy Markdown

The public393-trace AgentX HF loader currently rejects row0 with1077 strict-union errors because Arrow fills absent fields from other request variants with null. Normalize only schema-unknown null padding at the HF boundary; required nulls and non-null extras still fail, and file-loader validation is unchanged.

Validation:24 targeted loader tests pass; all393 SHA-pinned raw JSON traces equal their normalized HF model objects; full pre-commit passes. The input is semianalysisai/cc-traces-weka-062126@23f152f6f0f9399a85901b89a6458def0ef16729. No corpus filtering or unsafe scenario override.

Signed-off-by: Robert Washbourne <rawashbourne@gmail.com>
@cursor

cursor Bot commented Oct 22, 2026

Copy link
Copy Markdown

Bugbot is paused — your team has an unpaid invoice

Bugbot is paused because your team has an unpaid invoice.

Pay it from the billing page in the Cursor dashboard and Bugbot resumes automatically.

(requestId: ca2a0324-846e-4e69-a295-7a1ff29c9ba2)

@github-actions

Copy link
Copy Markdown

Try out this PR

Quick install:

pip install --upgrade --force-reinstall git+https://github.com/ai-dynamo/aiperf.git@fce650617faf5cacf0a2c35a40d3ccc4e1cd36dd

Recommended with virtual environment (using uv):

uv venv --python 3.12 && source .venv/bin/activate
uv pip install --upgrade --force-reinstall git+https://github.com/ai-dynamo/aiperf.git@fce650617faf5cacf0a2c35a40d3ccc4e1cd36dd

Last updated for commit: fce6506 • Browse code

@github-actions github-actions Bot added the fix label Oct 22, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant