Skip to content

fix: read skill files as UTF-8 in validator - #10

Merged
BramVR merged 2 commits into
mainfrom
claude/validator-utf8
Jul 22, 2026
Merged

fix: read skill files as UTF-8 in validator#10
BramVR merged 2 commits into
mainfrom
claude/validator-utf8

Conversation

@BramVR

@BramVR BramVR commented Jul 22, 2026

Copy link
Copy Markdown
Owner
- text = File.read(path)
+ text = File.read(path, encoding: "UTF-8")

Why

scripts/validate-skills read every SKILL.md with Ruby's default external encoding, which follows the process locale. In any shell without LANG set — including the non-interactive shells agents run in, and therefore the hooks/pre-commit gate — Ruby defaults to US-ASCII and every skill containing a non-ASCII byte fails YAML parsing with invalid byte sequence in US-ASCII.

Nineteen of the current skills trip it, since the house style uses em dashes and in descriptions. The failure looks like broken front matter but is purely environmental: the same files validate fine under LANG=en_US.UTF-8. Every commit in this repo today needed a LANG prefix to get past the hook.

Upstream steipete/agent-scripts fixed this; the same one-line fix was already sitting unmerged on the local claude/quirky-bartik-cdc137 branch. This cherry-picks that commit onto current main, preserving authorship, and adds a changelog entry.

Verification

Before, with the locale unset:

$ env -u LANG -u LC_ALL ./scripts/validate-skills
Skill validation failed:
- skills/oracle/SKILL.md: YAML: invalid byte sequence in US-ASCII
  ... 18 more
exit 1

After:

$ env -u LANG -u LC_ALL ruby -e 'puts Encoding.default_external'
US-ASCII
$ env -u LANG -u LC_ALL ./scripts/validate-skills
Validated 45 skill(s).
exit 0

Still US-ASCII as the process default — the fix is in the read, not the environment.

Note on the count: 45 here versus 47 in the main checkout. skills/gog and skills/wacrawl are relative symlinks (../../gogcli/...) that only resolve from the real checkout, so they dangle in a worktree and are skipped. Pre-existing behaviour, unrelated to this change.

BramVR added 2 commits July 22, 2026 15:09
The Ruby validator used File.read's locale-default encoding, so under a
non-UTF-8 locale (LANG=C) the frontmatter regex threw 'invalid byte
sequence in US-ASCII' and blocked commits via the pre-commit hook. Read
SKILL.md files as UTF-8 explicitly so validation is locale-independent.
Copilot AI review requested due to automatic review settings July 22, 2026 13:10
@BramVR
BramVR merged commit cd0e6c6 into main Jul 22, 2026
2 checks passed
@BramVR
BramVR deleted the claude/validator-utf8 branch July 22, 2026 13:10

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes locale-dependent failures in the skill validator by forcing UTF-8 when reading SKILL.md, ensuring the pre-commit hook and CI behave consistently even when LANG/LC_ALL are unset.

Changes:

  • Read skill files with File.read(..., encoding: "UTF-8") in scripts/validate-skills.
  • Add a changelog entry documenting the validator behavior change and the resolved error mode.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
scripts/validate-skills Forces UTF-8 reads for SKILL.md to avoid US-ASCII default-external encoding failures when locale env vars are unset.
CHANGELOG.md Documents the validator UTF-8 read fix and its impact on pre-commit/validation reliability.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants