Skip to content

fix(shared): share the HTTP loaders' path guards and catch encoded traversal - #3617

Open
Toyenzo wants to merge 1 commit into
finos:mainfrom
Toyenzo:fix/document-loader-traversal-guard
Open

Toyenzo wants to merge 1 commit into
finos:mainfrom
Toyenzo:fix/document-loader-traversal-guard

Conversation

@Toyenzo

@Toyenzo Toyenzo commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

Description

This covers items 1 and 2 of #3024. DirectUrlDocumentLoader and CalmHubDocumentLoader now share one request-path allowlist and one directory-traversal guard, in document-loader/request-guards.ts. It follows the response-origin.ts precedent. A future tightening now applies to both loaders.

While writing the shared guard I found that the old check, documentId.includes('/..'), only caught a literal ... The WHATWG URL parser also resolves these forms to ..:

  • %2e%2e, %2E%2E, .%2e, %2e.
  • \..\ in http(s) URLs, where the parser treats \ as /
  • dots split by a tab or newline, which the parser strips

By the time either loader read url.pathname, the parser had already resolved the segment away. So https://calm.finos.org/calm/%2e%2e/secret.json passed the guard and was requested as /secret.json. The same happened with calm:/schemas/2025-03/%2e%2e/%2e%2e/admin/... against CalmHub. The shared guard applies the same rewrites as the parser before it looks for /... Plain URLs behave as before.

Notes:

  • The two allowlists were not actually different. The CalmHub pattern's optional (\.json)? suffix was already matched by the character class, so one pattern keeps today's behaviour for both loaders.
  • The CalmHub error messages now read "rejected a path ...", the same wording as the direct-URL loader. Nothing in the repo matched the old wording.
  • Items 3 (DNS resolution and pinning), 4 (axios debug hooks) and 5 (double loadSchemas()) are left for follow-ups. Item 3 needs the Node/browser design agreed first, as the comment on the issue says. I used "Refs" so the issue stays open.

Refs #3024

AI-assisted: drafted with Claude Code; reviewed and tested by me.

Type of Change

  • 🐛 Bug fix (non-breaking change which fixes an issue)
  • ✨ New feature (non-breaking change which adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • 📚 Documentation update
  • 🎨 Code style/formatting changes
  • ♻️ Refactoring (no functional changes)
  • ⚡ Performance improvements
  • ✅ Test additions or updates
  • 🔧 Chore (maintenance, dependencies, CI, etc.)

Affected Components

  • CLI (cli/)
  • Schema (calm/)
  • CALM AI (calm-ai/)
  • CALM Hub (calm-hub/)
  • CALM Hub UI (calm-hub-ui/)
  • CALM Lab (calm-lab/)
  • CALM Models (calm-models/)
  • CALM Server (calm-server/)
  • CALM Studio (calm-studio/)
  • CALM Widgets (calm-widgets/)
  • CALMGuard (calm-guard/)
  • Documentation (docs/)
  • Shared (shared/)
  • VS Code Extension (calm-plugins/vscode/)
  • Dependencies
  • CI/CD

Commit Message Format ✅

Testing

  • I have tested my changes locally

  • I have added/updated unit tests

  • All existing tests pass

  • 29 new tests: 19 unit tests for the shared guards, plus 5 per loader for the parser-resolved traversal forms. The loader tests also check that no HTTP request is sent.

  • All 10 new loader tests fail against the old inline checks. The old code sent the request and failed later with "Failed to load document ...".

  • npm run build:cli passes. npm test --workspace shared passes 1573/1573, with coverage thresholds met and the browser-entry check OK.

  • CLI: 982 unit tests pass. cli.e2e passes 60/60 when run with --hookTimeout=300000. With the default 30s hook timeout it times out locally while packing and installing the CLI, which is unrelated to this change.

  • npm run lint: 0 errors. The only warnings are existing ones in untouched packages.

Checklist

  • My commits follow the conventional commit format
  • I have updated documentation if necessary (shared/AGENTS.md, Document Loader section)
  • I have added tests for my changes (if applicable)
  • My changes follow the project's coding standards

🤖 Generated with Claude Code

…aversal

DirectUrlDocumentLoader and CalmHubDocumentLoader each carried a copy of
the directory-traversal check and of the request-path allowlist, and the
two allowlists had drifted apart (finos#3024 items 1 and 2). Both now use one
shared helper in document-loader/request-guards.ts, following the
response-origin.ts precedent.

The old check, documentId.includes('/..'), only saw a literal "..". The
WHATWG URL parser also resolves "%2e%2e", ".%2E", "\.." (http/https) and
dots split by a tab or newline, so those forms reached the server as an
already-resolved path and bypassed the guard. The shared guard applies
the same rewrites before looking for "/..".

The CalmHub allowlist's optional "(\.json)?" suffix was already covered
by the character class, so a single pattern keeps today's behaviour.

Refs finos#3024

Signed-off-by: Toyenzo <64788123+Toyenzo@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant