Skip to content

Skill metadata drift: wrong requiresMcp for ns-generate-sbom / ns-optimize-function / ns-audit-dependencies, and README says 17 skills (there are 18) #71

Open

Description

@brycebaril

Summary

bundle.json is the canonical skill descriptor (CLAUDE.md: "canonical descriptor of every skill (name, path, description, requiresMcp)"). Several requiresMcp values do not match what the skills actually use, and the README's skill count is stale.

I cross-checked every skill's files for backticked MCP tool names from each server:

  • nsolid-console: all of its tools, including sbom, information-dashboard and asset-summary
  • ncm: getPackageQuality, getPackageVersions
  • ns-benchmark: run_benchmark, compare_benchmarks, get_benchmark_result
Skill requiresMcp in bundle.json What the skill actually uses Should be
ns-generate-sbom (bundle.json:67-70) ["ncm"] nsolid-console only: information-dashboard (SKILL.md:11), sbom (SKILL.md:15, :20); global-filter is mentioned only to say not to call it (SKILL.md:12). No NCM tool is used. ["nsolid-console"]
ns-optimize-function (bundle.json:85-88) (none) Delegates to ns-benchmark-run (SKILL.md:23) and ns-validate-optimization (SKILL.md:32-35), both of which require ns-benchmark. It names run_benchmark/get_benchmark_result/compare_benchmarks only to say not to call them directly (SKILL.md:35). It transitively requires ns-benchmark. ["ns-benchmark"]
ns-audit-dependencies (bundle.json:37-40) ["ncm"] No NCM MCP tool is used. audit-dependencies.cjs calls the NCM GraphQL API directly (https://api.ncm.nodesource.com, :11; /ncm2/api/v2/graphql, :155) with the service token from ~/.agents/.nodesource-auth.json (:84-96). [] (needs credentials, not the MCP server), unless requiresMcp is meant to mean "service dependency"

The other 15 skills are consistent: the 9 nsolid-console skills, ns-benchmark-run and ns-validate-optimization (ns-benchmark), ns-node-upgrade, ns-replace-package and ns-upgrade-package (ncm), and ns-switch-org (none).

Separately, README.md:19 says "17 Node.js operations skills". bundle.json and .claude-plugin/plugin.json list 18. The count was written when ns-switch-org was added (72ad7a3, #58), and ns-download-asset (#64) made it 18. The prose list on the same line also omits asset download and function optimization. docs/console-gap-analysis.md:9 already says 18. The published npm 1.0.3 does have 17, but main and npm 1.1.0 have 18.

Impact

  • Runtime: none today. requiresMcp is only schema-validated as an optional array of unique strings (packages/core/src/validate.ts:8-11) and typed (packages/core/src/types.ts:24). Nothing in packages/core/src, scripts/ or packages/pi-plugin reads it, and it is not emitted into the generated Claude/Codex/Antigravity manifests. It is also not checked against mcpServers[].name. The install-time warning in the archived design (openspec/changes/archive/2026-07-21-cross-harness-plugin-installer/design.md:315-322) was never implemented, and docs/pi-mcp-research.md:168 refers to "requiresMcp warnings" that don't exist.
  • Documentation and tooling: misleading. bundle.json ships in the npm package and is the canonical descriptor, so anyone documenting skill-to-server dependencies from it (as the N|Solid docs plugin page does) gets them wrong. For example, it says SBOM needs NCM when it needs a connected N|Solid Console. If the planned warning is ever implemented, these values would produce false warnings for SBOM and miss real ones for optimize-function.
  • The README undercounts and under-describes the skill set.

Steps to reproduce

  1. jq '.skills[] | {name, requiresMcp}' bundle.json
  2. grep -n 'sbom\|information-dashboard' skills/ns-generate-sbom/SKILL.md shows nsolid-console tools, and no getPackage* references.
  3. grep -n 'ns-benchmark-run\|ns-validate-optimization' skills/ns-optimize-function/SKILL.md
  4. grep -rn 'requiresMcp' packages/core/src scripts packages/pi-plugin returns only validate.ts and types.ts.
  5. jq '.skills | length' bundle.json gives 18, while sed -n 19p README.md says 17.

Expected

requiresMcp lists exactly the MCP servers each skill's workflow needs, including through delegated skills. The README count matches bundle.json.

Actual

See the table above. The README says 17.

Suggested fix

  • In bundle.json: set ns-generate-sbom to ["nsolid-console"], add "requiresMcp": ["ns-benchmark"] to ns-optimize-function, and decide on ns-audit-dependencies (drop it, or document that the field means "NodeSource service dependency"). Then run pnpm --filter nsolid-plugin bundle:sync so packages/core/bundle.json matches.
  • While there, ns-generate-sbom's bundle description ("…for project dependency provenance") could say it targets a running or registered N|Solid app, to match SKILL.md. Other bundle descriptions have drifted from the SKILL.md frontmatter too, for example ns-benchmark-run ("against running production instances") and ns-analyze-event ("event loop delays").
  • Add a validation step that each requiresMcp entry names a server in bundle.mcpServers. Optionally, add a test that scans each SKILL.md for known tool names and checks them against requiresMcp. Alternatively, implement or remove the design-doc warning so the field has a defined consumer.
  • README.md:19: change to "18" and add downloading saved assets and optimizing a selected function to the list. Better still, drop the hardcoded number or generate it.
  • README.md:315 ("Manual uninstall / cleanup") says "Credentials are preserved." That's no longer accurate: uninstall deletes ~/.agents/.nodesource-auth.json when no NodeSource installs remain, unless --keep-credentials is passed (see the uninstall case in packages/core/src/cli.ts). Update the sentence and mention --keep-credentials.

Found while writing the plugin docs page in nodesource/nsdocs#110.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingdocumentationImprovements or additions to documentationlaunch: stretchStretch goal for the N|Solid 7.0 release

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions