Skip to content

feat: add PowerShell license header support - #285

Merged
wu-sheng merged 4 commits into
apache:mainfrom
linghengqian:pwsh
Oct 10, 2026
Merged

wu-sheng merged 4 commits into
apache:mainfrom
linghengqian:pwsh

Conversation

@linghengqian

Copy link
Copy Markdown
Member

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The new PowerShell workflow lacks regression coverage across style lookup, generation, insertion, and normalization.

1 open finding
What changed in this PR

Adds PowerShell license-header checking and fixing support.

Changes:

  • Defines PowerShell <# ... #> header formatting.
  • Maps PowerShell files to the new style.
  • Normalizes PowerShell block-comment delimiters.
File Description
pkg/​license/​norm.go Recognizes PowerShell comment delimiters.
assets/​styles.yaml Defines the PowerShell header style.
assets/​languages.yaml Enables the style for PowerShell files.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread assets/styles.yaml

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The implementation is focused, consistent with existing styles, and adequately covered by regression tests.

0 open findings

1 resolved since last review

🧠 Review effort: Balanced

@wu-sheng

wu-sheng commented Oct 9, 2026

Copy link
Copy Markdown
Member

[P2] Preserve PowerShell file encoding when enabling header fixes

The new mapping at assets/languages.yaml:4310 enables header fix for UTF-16LE PowerShell files, but the existing fixer prepends a UTF-8 header before their BOM and original UTF-16 bytes. This produces a mixed-encoding file while reporting a successful fix. Subsequent checks silently skip the damaged file as binary.

This affects .ps1 scripts and .psd1 manifests, including manifests generated by Windows PowerShell's New-ModuleManifest, which writes UTF-16LE (Microsoft documentation). Before this change, fixing these files returned unsupported file and left their contents untouched.

I verified the following reproduction against 85baf45. Run from the PR checkout with Go and Python 3; it creates the fixture in a temporary directory:

go build -o /tmp/skywalking-eyes-pr285-license-eye ./cmd/license-eye
python3 - <<'PY'
from pathlib import Path
from tempfile import TemporaryDirectory
import subprocess

with TemporaryDirectory() as tmp:
    root = Path(tmp)
    script = root / "test.ps1"
    script.write_bytes(b"\xff\xfe" + "Write-Host 'Hello'\n".encode("utf-16le"))
    (root / ".licenserc.yaml").write_text(
        "header:\n"
        "  license:\n"
        "    content: Apache License 2.0\n"
        "  paths:\n"
        "    - test.ps1\n",
        encoding="utf-8",
    )
    binary = "/tmp/skywalking-eyes-pr285-license-eye"
    print("BOM offset before fix:", script.read_bytes().find(b"\xff\xfe"), flush=True)
    subprocess.run([binary, "header", "fix"], cwd=root, check=True)
    data = script.read_bytes()
    print("BOM offset after fix:", data.find(b"\xff\xfe"), flush=True)
    print("First 29 bytes after fix:", repr(data[:29]), flush=True)
    subprocess.run([binary, "-v", "debug", "header", "check"], cwd=root, check=True)
PY

Relevant output:

BOM offset before fix: 0
INFO Totally checked 1 files, valid: 0, invalid: 1, ignored: 0, fixed: 1
BOM offset after fix: 27
First 29 bytes after fix: b'<#\n Apache License 2.0\n#>\n\n\xff\xfe'
DEBUG Ignoring file: test.ps1 ; type: application/octet-stream
INFO Totally checked 0 files, valid: 0, invalid: 0, ignored: 0, fixed: 0

Expected: preserve the file's encoding and keep its BOM at byte 0, or reject the unsupported encoding before writing. Please add a UTF-16LE/BOM regression case alongside the UTF-8 tests.

The existing suites pass: go test ./pkg/comments ./pkg/header ./pkg/license ./test.

- add a PowerShellStyle using <# ... #> block comments
- preserve BOM encodings (UTF-8, UTF-16LE/BE) in header check, diff and fix

@linghengqian linghengqian left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • It looks like the ASF runner queue is full again, just like before. We'll likely have to wait until tomorrow to see the CI results.

@linghengqian linghengqian left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@wu-sheng

Copy link
Copy Markdown
Member

The original UTF-16 reproduction now passes on 04a53f1: the BOM stays at byte 0, the encoding is preserved, and the fixed file is checked successfully.

[P2] Detect UTF-32LE before matching the UTF-16LE BOM

There is still an encoding corruption case in pkg/header/encoding.go:50-52. A UTF-32LE BOM is FF FE 00 00, so it matches the two-byte UTF-16LE prefix. The decoder treats the UTF-32 source as UTF-16, and header fix writes a UTF-16 header ahead of source text containing interleaved NUL characters. The command reports success, and the subsequent license check passes even though the script's encoding is damaged. PowerShell supports this encoding through -Encoding utf32 (Microsoft documentation).

Verified reproduction against 04a53f1, from the PR checkout with Go and Python 3. The fixture is created in a temporary directory:

go build -o /tmp/skywalking-eyes-pr285-04a53f1-license-eye ./cmd/license-eye
python3 - <<'PY'
from pathlib import Path
from tempfile import TemporaryDirectory
import subprocess

with TemporaryDirectory() as tmp:
    root = Path(tmp)
    script = root / "test.ps1"
    script.write_bytes(b"\xff\xfe\x00\x00" + "Write-Host 'Hello'\n".encode("utf-32le"))
    (root / ".licenserc.yaml").write_text(
        "header:\n"
        "  license:\n"
        "    content: Apache License 2.0\n"
        "  comment: never\n"
        "  paths:\n"
        "    - test.ps1\n",
        encoding="utf-8",
    )
    binary = "/tmp/skywalking-eyes-pr285-04a53f1-license-eye"
    print("Original prefix:", script.read_bytes()[:4].hex(" "), flush=True)
    print("Original text:", repr(script.read_bytes().decode("utf-32")), flush=True)
    subprocess.run([binary, "header", "fix"], cwd=root, check=True)
    data = script.read_bytes()
    print("After-fix prefix:", data[:4].hex(" "), flush=True)
    try:
        print("Decoded as UTF-32:", repr(data.decode("utf-32")), flush=True)
    except UnicodeDecodeError as error:
        print("UTF-32 decoding failed:", error, flush=True)
    print("Decoded as UTF-16:", repr(data.decode("utf-16")), flush=True)
    subprocess.run([binary, "header", "check"], cwd=root, check=True)
PY

Relevant output:

Original prefix: ff fe 00 00
Original text: "Write-Host 'Hello'\n"
INFO Totally checked 1 files, valid: 0, invalid: 1, ignored: 0, fixed: 1
After-fix prefix: ff fe 3c 00
UTF-32 decoding failed: 'utf-32-le' codec can't decode bytes in position 0-3: code point not in range(0x110000)
Decoded as UTF-16: "<#\n Apache License 2.0\n#>\n\n\x00W\x00r\x00i\x00t\x00e\x00-\x00H\x00o\x00s\x00t\x00 \x00'\x00H\x00e\x00l\x00l\x00o\x00'\x00\n\x00"
INFO Totally checked 1 files, valid: 1, invalid: 0, ignored: 0, fixed: 0

Please detect the four-byte UTF-32 BOM before the UTF-16 prefixes and either preserve its encoding or reject/skip it without modifying the file. Full UTF-32 support is not required to avoid this corruption. A regression fixture should verify that an unsupported encoding is left untouched.

Validation otherwise passed: go test ./pkg/comments ./pkg/header ./pkg/license ./test, plus 24 UTF-8/UTF-16 combinations covering .ps1, .psm1, .psd1, LF/CRLF, Unicode text, check/diff/fix, shebang placement, and repeated fixes.

@linghengqian linghengqian left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • It seems there's nothing left for me to do?

CheckFile now logs a warning and marks a file it cannot decode (e.g.
UTF-32) as invalid, so the rest of the check, fix and diff still run.
Fixing such a file is still rejected by InsertComment without touching it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qcqgfw4N7r3bzX86dXrWxD
@wu-sheng

Copy link
Copy Markdown
Member
  • It seems there's nothing left for me to do?

Yes. I did update for the nits. Will merge after CI passed.

@wu-sheng

wu-sheng commented Oct 10, 2026 •

Copy link
Copy Markdown
Member

Final behavior as of 40de3cd. On top of the UTF-32 fix, I pushed two more commits: an undecodable file is now reported per file instead of stopping the whole run, and fix no longer rewrites malformed UTF-16 files.

PowerShell

.ps1, .psm1 and .psd1 use <# ... #> block headers. When a file starts with a #! line, the header goes directly after it.

Encoding handling in header check / diff / fix

File encoding check diff fix
No BOM Same as main: MIME sniffing, non-text files are ignored Same as main Same as main
UTF-8 with BOM Matched on the text after the BOM Diffed on the decoded text Header written after the BOM, which stays at byte 0
UTF-16LE / UTF-16BE with BOM Decoded to UTF-8 for matching Diffed on the decoded text Re-encoded in the original encoding, with the BOM at byte 0. A malformed file (an odd trailing byte or an unpaired surrogate) is not rewritten: fix returns malformed UTF-16 content and leaves it untouched
UTF-32LE / UTF-32BE BOM Logs a WARNING Failed to decode file: ... unsupported encoding: UTF-32 and reports the file as invalid. Other files are still checked Returns a per-file error, and other files are still diffed Returns a per-file error and leaves the file untouched. Other files are still fixed

The UTF-32 BOM is matched before the UTF-16 BOMs, so FF FE 00 00 is no longer decoded as UTF-16LE.

Behavior changes compared with main

  • A UTF-32 file in paths makes header check exit 1 until it is added to paths-ignore. main already reported UTF-32LE files as invalid, but it skipped UTF-32BE files, and any file starting with 00 00 FE FF, as binary.
  • A malformed UTF-16 file is never rewritten. The UTF-16 decoder replaces bad bytes with U+FFFD instead of failing, so fix first checks that the decoded text converts back to the original bytes, and otherwise returns a per-file error. check and diff still decode such a file leniently, so a malformed file that already has a header still passes.

Verification on 40de3cd

  • CI passes: Build LicenseEye (lint, license check, make test, build, docker) and Lint.
  • go test ./... passes. TestFixPreservesFileEncoding covers the UTF-8 BOM and UTF-16LE/BE fix round trips. TestHeaderOperationsRejectUTF32 covers check, diff, fix and insert for UTF-32LE/BE, and asserts that the file stays byte-for-byte unchanged. TestFixRejectsMalformedUTF16 covers an odd trailing byte and an unpaired surrogate in both byte orders. It fails without the round-trip check.
  • I ran the CLI on a repo with a UTF-32LE or UTF-32BE a.ps1 and a b.sh without a header. header check lists both as invalid. header fix leaves a.ps1 unchanged, adds the header to b.sh, and exits 1 with failed to decode a.ps1: unsupported encoding: UTF-32.
  • I also ran the CLI on malformed UTF-16LE files. Without a header, header fix exits 1 and leaves the file unchanged. With a header, header check and header fix both report it valid.

wu-sheng
wu-sheng previously approved these changes Oct 10, 2026
The UTF-16 decoder replaces an odd trailing byte or an unpaired surrogate
with U+FFFD instead of failing, so fixing such a file silently changed
bytes outside the license header. InsertComment now requires the decoded
content to round-trip back to the original bytes, and otherwise returns a
per-file error without writing. Check and diff keep the lenient decoding,
so a malformed file that already has a header still passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qcqgfw4N7r3bzX86dXrWxD
@wu-sheng
wu-sheng merged commit f6b7f44 into apache:main Oct 10, 2026
2 checks passed
@linghengqian
linghengqian deleted the pwsh branch October 10, 2026 01:52

@linghengqian linghengqian left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • It looks like the GitHub milestone for 0.9.0 is incorrect? After all, the 0.9.0 version has already been released.

@kezhenxu94 kezhenxu94 modified the milestones: 0.9.0, 0.10.0 Oct 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants