Skip to content

Ship all JavaScript adapters in one npm package - #1070

Open
zlareb1 wants to merge 6 commits into
mainfrom
refactor/single-npm-package
Open

zlareb1 wants to merge 6 commits into
mainfrom
refactor/single-npm-package

Conversation

@zlareb1

@zlareb1 zlareb1 commented Nov 20, 2026 •

Copy link
Copy Markdown
Member

Related: #1076

Changelog category (leave one):

  • Build/Testing/Packaging Improvement

Changelog entry (one or two sentences a user of Loom can read without the diff; the release pull request collects these into the release notes):

The @clickhouse/loom npm package includes the native n8n nodes and credentials alongside the TypeScript SDK and its framework adapters. Self-hosted n8n loads the nodes through N8N_CUSTOM_EXTENSIONS.

Why

All JavaScript integrations need one package version and one publication path. A separate n8n package requires its own registry setup and trusted publisher. The SDK build can include the native files and validate them before its existing publisher uploads the tarball.

The tradeoff is installation through n8n's custom-node directory: its Community Nodes settings installer requires an n8n-nodes- package name. The setup documentation gives the shared-package command. The adapter targets self-hosted n8n.

Scope and risk

Move the n8n source, icons, licenses and behavior tests into the TypeScript SDK. Export the classes through @clickhouse/loom/integrations/n8n and emit the CommonJS files that the native loader expects. Remove the standalone package, its publishing workflow, unused pnpm bootstrap and node-CLI toolchain patch.

n8n-workflow remains an optional host peer. Plain SDK installs gain no required runtime dependencies. The tarball tests check this in a clean consumer, then install a locked n8n host and exercise the native files and both module formats. Separate test installs accommodate the frameworks' different Zod requirements. SDK development and CI installs skip dependency lifecycle scripts because adapter builds and tests do not use the host expression engine.

The n8n framework keeps its UUID 11.1.1 override when moving into the SDK; a regression test checks that undersized UUID output buffers are rejected.

The artifact grows to include the native bundles. Packaging, module loading and host dependency resolution are the main risks. The sender census classifies history listing, lookup and deletion as identity-bound references and pins their identity builders; message content writes retain their admission class. The release skill, release documentation and publisher contract test record the single-package policy. Generated documentation and the operation reach manifest follow the source move.

Compatibility, rollout and rollback

  • Wire: no operations, fields or defaults change.
  • Operator steps: install @clickhouse/loom in the npm project running n8n, set N8N_CUSTOM_EXTENSIONS to its bundled node directory, and restart each n8n worker. The documentation includes the commands. No migration or backfill is required.
  • Publication: this PR changes the contents of the next SDK release; it does not republish an existing npm version.
  • Rollback: revert the packaging change before publication, or publish a subsequent SDK version after publication. There are no stored-data changes.

Tests

  • make format, make lint, make typecheck locally; CI verifies make format-check.
  • npm --prefix sdk/typescript run check and npm --prefix sdk/typescript run check:dist: source tests, README examples, native adapter tests against the packed tarball, publint and declaration resolution.
  • npm exec --yes --package=node@22 -- node scripts/test-n8n.mjs from sdk/typescript: the packed artifact loads and the adapter tests pass on Node 22.
  • Targeted Python contract tests for the sender census, publication, native tooling, agent instructions, documentation links, public copy, release preparation, operation reach and test inventory pass.
  • pytest tests/unit/test_install_script.py -q: 212 passed, one platform skip. pytest bench -q: 251 passed.
  • The broader local make test-unit run encounters SQL failures, parser subprocess/resource-limit errors, unavailable Linux /proc identity proofs and native CLI subprocess timeouts. Representative failures reproduce on unchanged main; the local ClickHouse binary does not support the schema's enable_quantized_codec setting. CI runs the full suite on the supported environment.
  • Cell acceptance reaches the fixture's 117-minute work deadline. The PR run and base run fail at the same check. The server execution paths and native harness are unchanged by this PR.
  • Existing tests passed after one rerun in earlier CI attempts: the WSL session-close test missed its one-second deadline; Windows hook cleanup raced detached work (ENOTEMPTY); unit cancellation cleanup exceeded half a second. These paths are unchanged by this PR.
  • A WSL Ubuntu setup stalled before tests started; one targeted job restart passed on the same commit.
  • A macOS plugin test failed its token-exchange assertion. One targeted rerun passed on the same commit. The unchanged helper has short request and process deadlines; the logs do not show which stage failed.
  • A Vue parser cost test measured 8.04 times the smaller sample against an 8-times limit. Its shard passed after one rerun on the same commit; the parser, test and limit are unchanged by this PR.
  • A worker terminated with SIGSEGV during a translation check. The isolated check and a later full shard passed; the translation code and tests are unchanged by this PR.
  • A SQL-splitter cost check measured 11.78 seconds against its calculated 10.67-second limit. The isolated test and one shard rerun on the same commit passed; the extractor, test and limit are unchanged by this PR.
  • A Cell acceptance run completed functional verification and cleanup but exceeded the latency gate: its trigger-enabled p99 delta is 31.5 ms against a 10 ms limit. The gate and trigger implementation are unchanged by this PR. The same gate passed in a later CI run.
  • A knowledge-ingest setup attempt read no dataset after a worker pass; later full-shard runs passed. Its helper accepts nonterminal worker results. The test and ingestion implementation are unchanged by this PR.
  • A live n8n application and a live Loom server were not started; the artifact tests use real host libraries with a fake HTTP transport.

AI assistance in the code (leave one; writing this description with AI is fine):

  • AI-generated: Codex wrote the packaging changes and checked the artifact contents, adapter behavior, module resolution and generated files with the commands above.

  • I can explain every line of this diff and how it interacts with the rest of Loom.

Checklist

  • Local format, lint and type checks pass. Focused SDK and packaging checks passed on the unchanged SDK source. No additional local unit run is required; the full unit and integration results are reported by CI.
  • Generated files are regenerated in this PR, not by hand.
  • No migrations are added or changed.
  • No coding-agent plugin runtime files change; plugin versions are unchanged.
  • No frontend files change.
  • Docs describe the package and operator steps.
  • Repository text passes the public-copy checks.
  • Commits are signed off.

@vercel

vercel Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
loom-docs Ready Ready Preview Nov 21, 2026 6:46pm UTC

Request Review

@zlareb1
zlareb1 force-pushed the refactor/single-npm-package branch 3 times, most recently from fd8ae44 to 809aa32 Compare November 20, 2026 21:14
@zlareb1
zlareb1 force-pushed the refactor/single-npm-package branch 25 times, most recently from 004ba15 to 1e08255 Compare November 21, 2026 18:09
Bundle the native n8n nodes in @clickhouse/loom and publish them through the SDK release workflow. Qualify the packed artifact with an isolated n8n host and record the single-package release policy.

Signed-off-by: Lareb Zafar <lareb.zafar@clickhouse.com>
Keep the adapter toolchain on UUID 11.1.1 when moving it into the shared SDK. Check that its UUID helper rejects an undersized output buffer.

Signed-off-by: Lareb Zafar <lareb.zafar@clickhouse.com>
The SDK scan includes the moved n8n history adapter. Review its identity-bound list, get and delete payloads and pin their scope builders without changing content-write admission.

Signed-off-by: Lareb Zafar <lareb.zafar@clickhouse.com>
Skip unused expression-engine install scripts in source-build instructions, scope the browser compatibility statement to the core SDK, and remove the unused pnpm bootstrap.

Signed-off-by: Lareb Zafar <lareb.zafar@clickhouse.com>
The bundled adapter uses the SDK build directory and no longer needs a standalone package directory.

Signed-off-by: Lareb Zafar <lareb.zafar@clickhouse.com>
The integration job uses SDK clients without the host expression engine. Suppress dependency lifecycle scripts there as in the SDK and release jobs.

Signed-off-by: Lareb Zafar <lareb.zafar@clickhouse.com>
@zlareb1
zlareb1 force-pushed the refactor/single-npm-package branch from 1e08255 to 47f2293 Compare November 21, 2026 18:44

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant