Skip to content

feat(*): turn on Skill Hub retrieval and add a settings switch - #1141

Merged
LivXue merged 14 commits into
mainfrom
fix/skillhub_endpoint_seed
Oct 30, 2026
Merged

LivXue merged 14 commits into
mainfrom
fix/skillhub_endpoint_seed

Conversation

@LivXue

@LivXue LivXue commented Oct 28, 2026 •

Copy link
Copy Markdown
Member

Summary

Fresh installs never searched the Skill Hub from a conversation. Onboarding
used to seed skillForge.router.hub.endpoint, but the wizard rewrite retired
init_extension_block_defaults and took the seed with it; the schema default
stayed None so programmatic loads remain Hub-disabled. The Skill Hub page
kept working on its own default URL, which made the gap easy to miss.

This restores the seed as a narrow writer instead of the retired defaults
block: onboarding writes the shared endpoint, keeps an endpoint the config
already names, leaves apiKey to the user, and does not re-enable a source a
config turned off.

It also adds skillForge.router.hub.enabled (default true) and a Skills-page
switch, so the source can be turned off without discarding a custom endpoint.
Both readers -- the router source and the shared client behind read_skill,
use_skill and find_skill -- require endpoint and enabled. The default
keeps endpoint-only configs working after an upgrade. The agent's self-config
surface lists the same key, and the row reads "Use 110k+ skills from EverMind
Skill Hub".

Review of the switch found that the settings page read only the camelCase
spelling of a top-level config block, while the loader accepts both spellings
and the writer keeps the one the file already carries. The switch is fixed in
this PR, and so are two same-class defects that predate it: the Skills page's
blocklist and the gate and title model-role slots read hard-coded camelCase
paths. Under a snake_case block the switch showed off and sent true on every
click (so the page could not turn the source off), every skill counted as on,
and those two roles drew as following the chat model. The blocklist and role
defects are folded into this PR at the maintainer's request because one
resolver covers all three. A top-level segment now takes the camel spelling
unless it is null, then the snake one, matching the loader's first non-null
rule and the writer's spelling-preserving update.

The second review round found the switch was not live: it wrote the file while
the serving loop kept its Hub client and router source, so a person who
switched remote retrieval off saw it off while turns kept sending their text
to the Hub. Both readers and the Hub tools now ask the live config before
each call, and the client is built whenever an endpoint exists, so both
directions apply without a rebuild. The web wizard also seeds the endpoint at
finish now -- the installer's default entrance used to leave the source off --
without re-enabling a config that switched it off. The setting is marked
sensitive.

An independent review round found the on/off state decided in four places
that could disagree, and this PR now answers it from one predicate,
skill_hub_on, which reads the file the way the loader does:

  • The loader migrates two legacy shapes in memory only (a top-level
    skillRouter, the retired hub.prefetchBodies), so a file keeps them for
    good. The live reader skipped those migrations and switched off a Hub the
    runtime had built; it now runs them on a copy, once per change of the file.
    The switch writer updates a legacy router in place, instead of creating
    skillForge.router beside it and hiding its endpoint and key on the next
    load.
  • The settings page draws the switch from settings.get (skill_hub_on,
    skill_hub_awaiting_reload) instead of re-deriving it from the raw file; a
    page-side copy of the boolean rules disagreed with pydantic on "y" / "t".
  • find_skill renders its description per call, so the 110k-library
    sentence appears only while the Hub is searched; the client alone no longer
    decides it.
  • raven_config reads show what the switch runs on (in_effect) beside the
    stored flag, which defaults to true and alone reported a Hub with no
    endpoint as on.
  • The reload notice follows the serving loop (it holds no Hub client), not
    whether the write seeded the endpoint, so a second switch-on before the
    reload still says so, and the Skills row keeps saying it after the toast.
  • Both wizards say what turning the source on sends and how to stop it, the
    Skills row says it, and the docs page describes it in both languages.
  • A read that fails (a file the migration cannot walk, a loop that failed to
    build) answers off and no wait instead of raising.

A later independent review found two accounting defects in the same
surface. The web picker and the onboarding wizard persist the language
through config.set, which wrote the file but never moved the process
language the sentences Raven composes itself are rendered in, so a
reader who picked Chinese in the wizard could still be shown the new
Skill Hub notice in English; config.set and config.unset now move it
like settings.set already did. And the switch was still declared a
reload-only setting after it had become live: the self-config surface
told the model a reload was owed for a switch-off that stops the next
search, while a switch-on whose write had seeded the endpoint could be
reported as already serving. The setting is now declared live, and
raven_config relays the settings service's own warning instead of
dropping it, so the one write that waits still says so.

Type

  • Fix
  • Feature
  • Docs
  • CI / tooling
  • Refactor
  • Other

Verification

  • Relevant tests pass locally

  • Relevant lint / type checks pass locally

  • User-facing docs or screenshots are updated when needed

  • This round, in the PR worktree (its own uv sync --frozen --all-extras
    venv): the 19 affected Python test files with -n 4, 1954 passed
    (repeated). One ordering fails at origin/main too and is not this PR's:
    run serially with tests/test_rpc_settings.py first, three env-mirror
    tests in tests/test_cli_onboard_commands.py fail identically at
    6298840e0, at the previous head and here; CI deals files to shards, and
    this round adds no test file.

  • ui-web vitest run: 261 files, 4490 tests, no errors; tsc --noEmit and
    eslint over the changed files exit 0; gen-rpc-client.mjs --check,
    ui-tui gen-rpc-types.mjs --check and gen-i18n.mjs --check in sync.

  • Mutations, each restored and byte-compared, 20 of 20 reddened a test:
    skipping the migration in the reader, the writer ignoring the legacy
    router, previous from the raw block, dropping the per-file cache, either
    read-failure guard, settings.get answering off, the page ignoring it,
    find_skill ignoring the switch or keeping the registered schema, the
    wiring dropping its reader, in_effect hidden from get or describe, the
    wait ignoring the loop, the wizard notice dropped or untranslated, the CLI
    line dropped or printed when nothing turned on, the row's disclosure
    dropped, the snapshot dropping the wait.

  • ruff check, ruff format --check and lint-imports (10 kept): pass.
    check_source_language.py and check_large_files.py over
    origin/main...<worktree snapshot>: exit 0; a snapshot with one CJK line
    added to raven/config/live.py is reported, as a control.

  • After rebasing onto 11ca10d84 (a ui-web shell change, no file in
    common): ui-web src/features/settings, src/features/onboard, src/rpc,
    src/chrome and src/state (84 files, 1243 tests), tsc --noEmit, and
    seven of the Python files (687 passed) re-run: pass.
    scripts/check_commit_messages.py origin/main..HEAD: exit 0.

  • The two fixes above, in the PR worktree: the four touched test files,
    743 passed; each new test was watched failing at the previous tree
    first. The full unit suite with -n 4: 37080 passed, 8 failed -- the
    same 8 IDs on the merge base 11ca10d8 in this box's own checkout
    (five provider proxy cases, the LibreOffice xls conversion, npm on
    PATH, and the root-readable file), so none is this PR's.

  • Not re-run this round: native Windows (the two fixes are
    platform-neutral: process-language state and a tool's reply string),
    ty check, vite build / ui-web/build.py, the docs build.

Risk

  • Security impact considered
  • Backward compatibility considered
  • Rollback path is clear for risky changes

New network behavior: an onboarded config now queries skillhub.evermind.ai on
qualifying turns. Existing configs without the endpoint stay off until they
onboard or flip the switch. No key is written, the endpoint remains
operator-configurable, and switching off keeps the URL.

The new field defaults to true, so endpoint-only configs behave as before
after an upgrade. Configs written by the switch carry hub.enabled; a
downgrade to a binary that predates the field would refuse it (the config
models forbid unknown keys), so roll back by reverting the release rather
than downgrading. Reverting the commit is otherwise safe.

The settings page now reads a snake_case top-level block as well as the
camelCase one, matching what the loader already accepted; no config file
needs a migration. settings.get gains two required result fields
(skill_hub_on, skill_hub_awaiting_reload); the page and the server ship
together.

The switch now applies live in both directions while an endpoint is present.
A first enable on a config that had no endpoint at loop start waits for a
reload and says so. The web wizard's finish seeds the shared endpoint unless
the config already has one, does not re-enable a config that switched the
source off, and says what turning it on sends. A legacy top-level
skillRouter endpoint now counts as on, as it already did for the runtime.

Related Issues

N/A

@LivXue
LivXue requested review from 0xKT and gloryfromca October 28, 2026 04:40

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: the settings switch misreports supported snake-case configs and must read the active alias.

I reviewed the full diff and the surrounding onboarding, raw-config writer, runtime construction, settings RPC/source, and web rendering paths. I found one concrete backward-compatibility defect inline.

Verification:

  • uv run pytest tests/test_config_update.py tests/test_rpc_settings.py tests/test_cli_onboard_commands.py tests/test_context_engine_factory.py tests/test_agent_loop_skill_tools.py tests/test_config_raven_sections.py -k 'not test_the_login_shell_capture_actually_sees_the_keys' - 730 passed, 1 deselected.
  • The non-deselected run failed only at that unrelated login-shell environment test after 620 passes because the spawned shell did not expose its generated keys.
  • Focused Hub selection - 41 passed.
  • npm test -- --run src/features/settings/pages/Skills.test.tsx - 8 passed.
  • npm run type-check - passed.

Coverage included AGENTS.md and the runtime/web context rules, the diff, callers, relevant alias-preservation history, backward compatibility, test-strength review, and the RPC boundary architecture. I found no CLAUDE.md in scope.

@LivXue

LivXue commented Oct 28, 2026

Copy link
Copy Markdown
Member Author

Windows compatibility review — PR #1141 @ cebc1dc678019a37944a3acd04f990c84bc7cab7

Scope: the full base-to-head diff at this head (19 files, +501/−37) — the onboarding seed, the new skillForge.router.hub.enabled field and its writers, both runtime readers, the settings RPC, and the Skills-page switch — reviewed for Windows behaviour and exercised on native Windows.

Result: no Windows-specific defect found. The two new writers (initialize_skill_hub_endpoint, set_skill_hub_enabled) go through the existing atomic_update path, which is the repo's Windows-hardened primitive (portalocker LockFileEx sidecar, UTF-8 reads and writes, mode/DACL carry-over onto the replacement, read-only tmp cleanup); they inherit that handling instead of re-implementing it. Reads go through read_raw_or_raise (UTF-8); the JSON round-trip keeps non-ASCII values (ensure_ascii=False) and writes CRLF on Windows exactly as every other config writer already does, which stays valid JSON. _skill_hub_block honours the existing skill_forge/skillForge spelling in place, so a config keeps a single loadable block. No path, subprocess, signal, socket, permission, or line-ending assumption is introduced beyond what the shared primitive already covers.

Windows verification on native Windows 11, detached worktree D:\tmp\rv-1141 at the reviewed head, offline venv (uv sync --frozen --offline --all-extras --no-extra segmenter --dev), scratch HOME/USERPROFILE, PYTHONUTF8=1, PYTHONPATH pinned to the worktree:

  • uv run --no-sync pytest tests/test_config_update.py tests/test_rpc_settings.py tests/test_context_engine_factory.py tests/test_agent_loop_skill_tools.py tests/test_config_raven_sections.py -q → 5 failed, 368 passed; the same five files at base 06509f8e → 5 failed, 327 passed; failing-ID sets are identical (the a2a chmod/mode cases and the /somewhere/theirs POSIX-path refusal). All 41 tests added by this PR pass on Windows.
  • uv run --no-sync pytest tests/test_cli_onboard_commands.py -q → 3 failed, 355 passed; base → 3 failed, 353 passed; the same three inherited login-shell/environment IDs on both.
  • npx --no-install vitest run src/features/settings/pages/Skills.test.tsx → 8 passed.
  • End-to-end onboarding probe on a path with a space and CJK characters (D:\tmp\rv-1141 home 目录): _bootstrap_empty_config() created ...\.raven\config.json, wrote hub = {'endpoint': 'https://skillhub.evermind.ai'}, and the file parsed back as valid UTF-8 JSON (with the CRLF endings other config writers produce on Windows).

Coverage gaps: the full suite, the settings page's other flows, and the PR's Playwright/Chromium checks were not rerun here; ESLint is unavailable in this environment. The blocking snake-case finding already raised inline by gloryfromca (ui-web/src/features/settings/pages/Skills.tsx:32 — settings.get may return skill_forge) is a cross-platform backward-compatibility issue and is outside this Windows review's scope.

@LivXue
LivXue requested a review from gloryfromca October 28, 2026 06:50

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blockers; this can merge as far as I am concerned.

The revision fixes the supported skill_forge config case by resolving the active top-level spelling before reading Hub state. Its alias precedence matches the loader and config writers, and the regression test covers both displayed state and the value sent when the active switch is clicked. The extracted helper does not regress its other settings callers. My prior thread is replied to and resolved.

Verification:

  • npm test -- --run src/features/settings/pages/Skills.test.tsx src/features/settings/providers/Roles.test.tsx src/features/settings/source.test.ts - 68 passed.
  • npm run type-check - passed.

Coverage included AGENTS.md and the runtime/web context rules, the full current diff and revision delta, affected callers, relevant alias-preservation history, backward compatibility, test-strength review, and the settings RPC architecture boundary. No CLAUDE.md is present in scope.

@LivXue

LivXue commented Oct 28, 2026

Copy link
Copy Markdown
Member Author

Windows compatibility re-review — PR #1141 @ 984455024448f1c7584575017c4208b94ad489e3

Scope: the delta since the previously reviewed head cebc1dc6 — one UI commit (6 files, +106/−16): dig now resolves a top-level config block under either spelling (skillForge/skill_forge), Skills.tsx and Roles.tsx read the block through it, and the two settings tests gain snake-case cases. The rest of the PR (config writers, onboarding seed, settings RPC, runtime readers) is unchanged from the head reviewed in the previous round.

Result: no Windows-specific defect. The change is pure TypeScript object-shape logic with no path, filesystem, subprocess, permission, or encoding surface. toSnake maps camelCase JSON keys to their snake_case aliases; that is object-key matching, not filesystem naming, so there is no case-sensitivity difference between Windows and POSIX. This commit also closes the cross-platform snake-case finding gloryfromca raised inline on Skills.tsx:32, and its new tests cover both spellings; the Python side of that issue is unaffected.

Verification on native Windows 11, detached worktree D:\tmp\rv-1141b at the reviewed head, ui-web/node_modules junctioned read-only from the main checkout:

  • npx --no-install vitest run src/features/settings/pages/Skills.test.tsx src/features/settings/providers/Roles.test.tsx → 2 passed files, 47 passed tests
  • npx --no-install vitest run src/features/settings → 18 passed files, 244 passed tests
  • No failures, so no base comparison was needed. The Python suites were last exercised at the previous head; this commit touches only ui-web.

Coverage gaps: the full ui-web suite, tsc --noEmit, and the PR's Playwright/Chromium check were not rerun here.

Comment thread raven/rpc/methods/console.py Resolved
@0xKT

0xKT commented Oct 28, 2026

Copy link
Copy Markdown
Member

Not a blocker - non-blocking items from acceptance on 98445502 (onto 959cd0a9).

Severity of this one finding, not a verdict on the pull request. Any blocking findings from this pass are review threads on the changed files; GitHub renders those collapsed, as a file name with no text.

  1. The seed reaches only the CLI wizard. initialize_skill_hub_endpoint has one production caller, _bootstrap_empty_config (raven/cli/onboard_commands.py:361). Both installers end on raven web --foreground, and the page's own onboarding (model.save_key, config.set model) never reaches the seed. Once setup.status reports configured, the TUI gate skips the CLI wizard. Measured: a config onboarded through the web wizard's RPCs ends with no skillForge block and the Hub effectively off, while the CLI bootstrap on the same tree seeds https://skillhub.evermind.ai. So the default differs by entrance, and the installer's default entrance is the one that stays off. The switch can turn it on, and set_skill_hub_enabled seeds. CONTEXT.md's new "Onboarding seeds the shared catalog URL" holds for one of the two wizards.
  2. The new self-config entry is not marked sensitive. skillForge.router.hub.enabled (raven/config/self_surface.py:549-555) carries no sensitive=. So in an attended smart-mode turn the reviewer may approve the agent turning remote retrieval back on after the user switched it off. With the default skillForge.autoInstall='auto', that also re-opens hub skill installs. The neighbouring skillForge.autoInstall entry is marked sensitive for that reason.
  3. Disclosure. For installs onboarded through the CLI or TUI, the default now sends the turn's message text to skillhub.evermind.ai. That is a deliberate posture change; please state it where onboarding and the Skills page describe the Hub.
  4. Smaller:
    • "Is the Hub source on" is decided at more than one site. The factory copy and set_skill_hub_enabled read lax booleans (0, "false") differently from the loader, so the switch can show ON for a value the loader reads as false.
    • The PR's guarantee that "a repeat onboard does not re-enable a config that turned the source off" has no test that fails when it is broken.

Tests here on the merged tree: the 6 changed Python test files via the board runner gave 731 passed. ui-web Skills.test.tsx + Roles.test.tsx: 47 passed. lint-imports: 10 kept.

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: the Skill Hub switch must update the live runtime or require and surface a reload before displaying the source as disabled.

The newly reported runtime-effect thread changes my prior clean stance. I independently verified its mechanism: this key is declared reload-only, settings.set only persists it, the existing loop retains both the Hub client and router source, and the RPC returns no warning even though the web settings source already displays one when supplied. I am not opening a duplicate finding or resolving another reviewer's thread.

Verification:

  • uv run pytest tests/test_rpc_settings.py -k 'hub_switch' - 4 passed; these cover persisted config state, not live-loop recomposition.
  • npm test -- --run src/features/settings/pages/Skills.test.tsx src/features/settings/source.test.ts - 31 passed; the current UI tests confirm the saved state is redrawn immediately.

@LivXue
LivXue force-pushed the fix/skillhub_endpoint_seed branch from 9844550 to c23c4bd Compare October 28, 2026 12:38
@LivXue
LivXue requested review from 0xKT and gloryfromca October 28, 2026 12:39

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blockers; suggestions only, and they are marked inline.

The live switch fix closes the previously reproduced privacy failure: an already-built router checks the current config before every Hub search, Hub-only read/install calls do the same, and retaining the client whenever an endpoint exists makes both off and re-enable live. The existing blocking thread belongs to its author to close; I have not resolved or re-graded it.

Named nonblocking follow-up: the new web-wizard skillForge.router.hub.init path seeds an endpoint after the first-run stack has already been assembled without a Hub client, but returns no reload warning. The wizard can therefore finish with the switch visually on while Hub retrieval remains inactive until a gateway restart. A restart is a working escape hatch, so at this late review round I am not holding the change for it; the init path should eventually surface the same reload cost as an endpoint-seeding switch write or arrange recomposition.

Verification:

  • Focused backend Hub/config/onboarding selection - 141 passed.
  • Focused web onboarding/settings selection - 61 passed.
  • npm run type-check - passed.

Coverage included AGENTS.md and the runtime/web context rules, the rebased full diff via range-diff, the new commit, affected callers and lifecycle, backward compatibility, test-strength review, and the settings RPC/source boundary. No CLAUDE.md is present in scope.

@LivXue
LivXue requested a review from gloryfromca October 28, 2026 12:45

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blockers; suggestions only, and they are marked inline.

The new revision only adds the missing localized sensitive-confirmation sentence. Its key matches skillForge.router.hub.enabled, both locales are present, and the checked-in generated catalogue matches the source. It introduces no new issue.

The previously named nonblocking web-wizard endpoint-seed reload follow-up is unchanged. The other reviewer's blocking thread remains theirs to close; I have not resolved or re-graded it.

Verification:

  • npm run lint:i18n --prefix ui-tui - passed; generated catalogue is up to date.
  • npm test -- --run src/features/composer/approve.test.ts - 52 passed.
  • make check-source-language could not run because make is unavailable; invoking uv run python scripts/check_source_language.py directly exited successfully but reported no commit range to check.

@LivXue

LivXue commented Oct 28, 2026

Copy link
Copy Markdown
Member Author

Windows compatibility re-review — PR #1141 @ 89a964fcea6038fa1d98b982cf5e409e8e250053

Scope: the branch is rebased onto current main (which now includes #1102); the PR's own diff at this head is 855a412fa..89a964fc (35 files, +1079/−56). The two commits since the previously reviewed head 98445502 change the feature materially: the switch now applies live (the router source and the read_skill/use_skill tools ask the live config on every call, and the client is built whenever an endpoint exists, so off stops the next query; only a write that had to seed the endpoint returns a reload warning), the web wizard seeds the endpoint through settings.set like the CLI bootstrap, the setting is marked sensitive with both locale notes, and the page reads a hand-written enabled the way the loader does.

Result: no Windows-specific defect. Both new config writers still go through atomic_update (portalocker LockFileEx sidecar, UTF-8, DACL/mode carry onto the replacement), which is the platform-hardened path reviewed in this PR's earlier heads; the live readers re-parse the config on byte change only and involve no path, subprocess, signal, socket, permission, or encoding logic. hub_source_on answers "off" for a block the loader would refuse, which is the safe direction for a switch that stops text leaving the machine.

Verification on native Windows 11, detached worktree D:\tmp\rv-1141c at the reviewed head, offline venv (uv sync --frozen --offline --all-extras --no-extra segmenter --dev), scratch HOME/USERPROFILE, PYTHONUTF8=1, PYTHONPATH pinned:

  • uv run --no-sync pytest tests/test_agent_loop_skill_tools.py tests/test_cli_onboard_commands.py tests/test_config_live.py tests/test_config_raven_sections.py tests/test_config_update.py tests/test_context_engine_factory.py tests/test_rpc_settings.py tests/test_skill_hub_source.py tests/test_skill_hub_tools.py -q → 8 failed, 926 passed
  • the same nine files at base 855a412fa → 8 failed, 855 passed
  • failing-ID sets are identical in both directions (the inherited Windows login-shell/path cases); the 71 added tests pass
  • node ui-tui/scripts/gen-i18n.mjs --check → i18n: generated catalogue is up to date (exit 0), and gui.confirm.cfg.why.skillForge.router.hub.enabled is present with both en and zh in i18n/messages.json

Coverage gaps: the ui-web test files this PR changes (Skills.test.tsx, Roles.test.tsx, the onboard source/store/page tests) could not be executed. The shared D:/projects/Raven/ui-web/node_modules is currently empty (0 entries; it held 196 at the end of an earlier review run today), no intact copy exists elsewhere on this box, and I did not run an install. That leaves the UI half — the live switch rendering, the wizard's seeding call, and the sensitive-note catalogue gate — verified by code reading only. The full suite was not rerun either.

@LivXue
LivXue force-pushed the fix/skillhub_endpoint_seed branch from 89a964f to 521b0fc Compare October 29, 2026 17:13
@LivXue

LivXue commented Oct 29, 2026

Copy link
Copy Markdown
Member Author

Follow-up fixed at 521b0fcb6 (rebased onto 6298840e0).

settings.set skillForge.router.hub.init now returns the same reload warning the endpoint-seeding switch write returns, and only when it actually had to seed the endpoint; the wizard's finish shows it as a toast. So the first-run loop that was assembled without a Hub client no longer leaves the switch reading on with no notice. Tests pin the RPC reply (warning when seeding, none when the endpoint already exists) and the toast; mutating either construct reddens one test.

@LivXue
LivXue requested a review from gloryfromca October 29, 2026 17:13

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blockers; this can merge as far as I am concerned.

The new commit resolves the remaining web-wizard follow-up: skillForge.router.hub.init now returns the reload warning only when it actually seeds a missing endpoint, and onboarding surfaces that warning through the persistent toast host before closing. The four previously reviewed PR commits are unchanged across the rebase.

Verification: uv run pytest tests/test_rpc_settings.py -k 'hub_init or hub_switch' -x (8 passed); the three focused onboarding suites (30 passed); and npm run type-check (passed). I covered the repository rules and relevant context, the delta and range-diff/history, the RPC-to-onboarding callers, compatibility with already-configured/custom endpoints, test-strength changes, and the live-config/reload architecture. My earlier thread remains resolved; the other reviewer's blocker is also resolved by its author.

@LivXue

LivXue commented Oct 29, 2026

Copy link
Copy Markdown
Member Author

Windows compatibility re-review — PR #1141 @ 521b0fcb65ee647e3c6797e03d12b0984ad353eb

Scope: the branch is rebased onto current main; the PR's own diff at this head is 6298840e..521b0fcb (35 files, +1128/−56). The delta since the previously reviewed head 89a964fc is the rebase plus one commit (521b0fcb): the web wizard's finish seeds the Hub endpoint after the first-run stack may already exist, so settings.set's init path now returns the same reload warning the switch write returns when it actually seeds, and the wizard shows it as a toast. Reviewed and run on native Windows.

Result: no Windows-specific issue. The two config writers still go through atomic_update (portalocker LockFileEx sidecar, UTF-8, DACL/mode carry), and the new commit is RPC/UI text plus one shared constant; the live readers are in-memory/file-reparse logic with no path, subprocess, signal, or encoding surface beyond the already-reviewed writers.

Verification on native Windows 11, detached worktree D:\tmp\rv-1141d at the reviewed head, offline venv (uv sync --frozen --offline --all-extras --no-extra segmenter --dev), scratch HOME/USERPROFILE, PYTHONUTF8=1, PYTHONPATH pinned:

  • uv run --no-sync pytest tests/test_agent_loop_skill_tools.py tests/test_cli_onboard_commands.py tests/test_config_live.py tests/test_config_raven_sections.py tests/test_config_update.py tests/test_context_engine_factory.py tests/test_rpc_settings.py tests/test_skill_hub_source.py tests/test_skill_hub_tools.py -q → 8 failed, 930 passed
  • the same nine files at base 6298840e → 8 failed, 859 passed
  • failing-ID sets are identical in both directions (the inherited Windows login-shell/path cases); the 71 added tests pass.

Coverage gaps: the ui-web tests this PR changes (Skills.test.tsx, Roles.test.tsx, the onboard source/store/page tests) still cannot run — the shared D:/projects/Raven/ui-web/node_modules is empty (0 entries) — so the switch UI, the wizard seeding call and the new toast are checked by reading only. The full suite was not rerun.

Onboarding stopped seeding skillForge.router.hub.endpoint when the wizard
rewrite retired init_extension_block_defaults, while the endpoint's schema
default stayed None so programmatic loads remain Hub-disabled. A fresh install
therefore never searched the Skill Hub from a conversation, even though the
Skill Hub page kept working on its own default URL.

Restore the seed as a narrow writer rather than the whole retired defaults
block: onboarding writes the shared endpoint, keeps an endpoint the config
already names, and leaves apiKey to the user. A repeat onboard does not
re-enable a config that turned the source off.

Add skillForge.router.hub.enabled (default true) so the settings page can
switch the source off without discarding a custom endpoint. Both readers (the
router source and the shared client behind read_skill, use_skill and
find_skill) now require endpoint and enabled; the default keeps
endpoint-only configs working after an upgrade. The Skills page gains the
switch, and the agent's self-config surface can set the same key.
settings.get returns the config file as written, and both camelCase and
snake_case spellings load. The Skill Hub switch and the skill blocklist
read only raw.skillForge, so a snake_case skill_forge block left the
switch showing off (and sending true on every click) and every skill
counting as on. The gate and title roles read hard-coded camelCase paths
for the same reason.

Route all three through one dotted-path read in the settings source: the
top-level segment takes the camel spelling unless it is null, then the
snake one, matching the loader's first non-null rule and the writer's
spelling-preserving update. Tests cover the switch, the blocklist and the
two role paths under a snake_case block, and the switch's off click.
The switch only wrote config: the running loop kept its Hub client and router
source, settings.set answered applied with no warning, and the page showed off
while turns kept sending their text to the Hub. Both readers now ask the live
config per search and per tool call, and the client is built whenever an
endpoint exists, so off stops the next query and turning it back on is live
too. Only a write that had to seed the endpoint needs a reload, and the RPC
returns the warning the page already knows how to show.

The web wizard never reached the CLI seed, so the installer's default entrance
left the source off. Its finish now seeds the endpoint through settings.set,
non-clobbering like the CLI bootstrap. The setting is marked sensitive so an
attended agent does not turn remote retrieval back on, the page reads a
hand-written enabled the way the loader does, and a repeat onboard is pinned
not to re-enable a switched-off source.
The switch is marked sensitive, and the approval card asks the catalogue for
gui.confirm.cfg.why.<path>. Only the setting and its sentence existed, so the
zh page would have fallen back to the English note and the sensitive-notes
gate failed the unit suite.
The wizard's finish seeds the endpoint after the first-run stack may already
have been assembled, so the switch could read on while Hub retrieval waited
for a restart. settings.set now returns the same reload warning the switch
write returns when the init path actually seeds, and the wizard shows it as
a toast.
The loader migrates two legacy shapes in memory and never writes them
back: a top-level skillRouter is nested under skillForge, and the retired
hub.prefetchBodies is dropped. The live reader validated the raw block, so
a Hub the runtime built from such a file answered off: searches returned
nothing and read_skill / use_skill refused with "turned off in Settings".

skill_hub_on now runs the loader's migrations on a copy, once per change
of the file's bytes. The switch writer updates a legacy router in place
while skillForge has none of its own, instead of creating
skillForge.router beside it, which would hide the legacy endpoint and key
from the next load. The reply's previous state comes from the same
predicate.
The Skills page re-derived the switch from the raw file with its own copy
of the loader's boolean rules. That copy disagreed with pydantic ("y",
"t", padded strings) and missed the legacy shapes, so the page could show
the source off while turns were sending text to the Hub. settings.get now
returns skill_hub_on from the runtime's predicate, and the page draws it.
The Hub client is built while the switch is off, so the client alone no
longer says whether the Hub is searched, and find_skill kept promising
the 110k-skill library with the source switched off. It now reads the
switch when its description renders and authors to_schema, so the
registry serves the live text rather than the copy taken at
registration.
skillForge.router.hub.enabled defaults to true, but the source runs only
with an endpoint, so a raven_config get reported a Hub with no endpoint
as on. A setting can now name how the runtime reads it (in_effect); get
and describe show that value beside the stored one.
The notice came only with the write that seeded the endpoint, so a second
switch-on before the reload said nothing while the loop still held no Hub
client, and the page showed the switch on once the toast was gone. The
notice now follows the serving loop: settings.set returns it while the
source is on and the loop holds no client, settings.get reports the wait,
and the Skills row says so. No loop at all is no wait, because the first
one is built from the file.
Onboarding is the opt-in for conversational Skill Hub search, and neither
wizard said that it sends text from the conversation to the Hub. The CLI
bootstrap prints it when its seed turns the source on, the web wizard's
finish returns it as the notice it already shows, and the Skills row,
CONTEXT.md and the docs page describe it. The server notices go through
raven.i18n.
The predicate walks the whole file through the loader's migrations, which
raise on a section they cannot walk (tools as a list), and the stack's
loop factory raises its build error when no loop could be built. Either
turned settings.get, or every Hub search, into an error. A file the
loader cannot walk now answers off, and a loop that failed to build is no
wait.
@LivXue
LivXue force-pushed the fix/skillhub_endpoint_seed branch from 521b0fc to 916e7ed Compare October 30, 2026 03:50
@LivXue

LivXue commented Oct 30, 2026

Copy link
Copy Markdown
Member Author

@0xKT re the non-blocking notes on 98445502:

  • Item 3 (disclosure): addressed at 9beb8d7ce. Both wizards now say what turning Skill Hub search on sends and how to stop it: the CLI bootstrap prints it when its seed turns the source on, and the web wizard's finish shows it as its notice, in the reply language. Nothing is said when the seed turns nothing on (an explicit off, a null endpoint, an endpoint already configured). The Skills page row carries the same sentence, and docs-site/docs/skills-and-extensions.md and its zh page describe the data flow and the switch.
  • Item 4a (lax booleans): the page-side copy of the boolean rules added for this note still disagreed with pydantic on "y", "t" and padded strings. Since 39ab2162f the page draws skill_hub_on from settings.get, computed by the predicate the runtime reads, so no second copy is left to drift.

The same push answers the on/off state from that one predicate everywhere; it also runs the loader's in-memory migrations, so a legacy top-level skillRouter reads as on, as it does for the runtime. The description lists the round with its verification.

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blockers; this can merge as far as I am concerned.

The five previously reviewed commits are patch-equivalent after the rebase. The seven new follow-ups consistently derive Hub state through the loader migrations, return the serving loop's reload state over RPC, render that server answer in Settings, keep the tool description live, and disclose the outbound conversation text in both onboarding paths and documentation. I found no concrete regression or remaining plain error.

Verification: focused Python behavior tests (80 passed), RPC contract-shape tests (51 passed), affected web tests (79 passed), npm run type-check (passed), and the TUI i18n catalogue check (passed). I covered AGENTS.md and the relevant CONTEXT.md update (no repository CLAUDE.md), the delta and range-diff/history, runtime/RPC/UI callers, legacy-config and older-client compatibility, whether tests were weakened, and the live-preference/control-plane boundary. Both review threads are resolved, including mine.

@LivXue

LivXue commented Oct 30, 2026

Copy link
Copy Markdown
Member Author

Windows compatibility re-review — PR #1141 @ 916e7ed7f6425b51e192cf07ce4d86c9e47ee13e

Scope: the branch is rebased onto current main; the delta since the previously reviewed head 521b0fcb is seven commits: the switch is now read through the loader's own migrations (on a deep copy, cached per parsed file), raven_config shows the stored value and what is in effect, the skill-hub tools describe and gate on the live switch, the reload notice is keyed on the serving loop, and a switch state that cannot be read answers off; docs and i18n are updated. Reviewed and run on native Windows.

Result: no Windows-specific issue. The new read path is pure dict handling (_migrate_config(copy.deepcopy(raw)), no file I/O); the only writes remain initialize_skill_hub_endpoint and set_skill_hub_enabled, both still going through atomic_update (portalocker LockFileEx sidecar, UTF-8, DACL/mode carry onto the replacement). _skill_hub_block now prefers a legacy top-level skillRouter over creating a new skillForge.router, which keeps an existing endpoint and key visible to the next load — again pure mapping work. The per-file answer cache is keyed on the LiveConfig and its raw dict via weakrefs, platform-neutral.

Verification on native Windows 11, detached worktree D:\tmp\rv-1141e at the reviewed head, offline venv (uv sync --frozen --offline --all-extras --no-extra segmenter --dev), scratch HOME/USERPROFILE, PYTHONUTF8=1, PYTHONPATH pinned:

  • uv run --no-sync pytest tests/test_agent_loop_skill_tools.py tests/test_cli_onboard_commands.py tests/test_config_live.py tests/test_config_update.py tests/test_context_scent.py tests/test_raven_config_tool.py tests/test_rpc_settings.py tests/test_skill_hub_source.py tests/test_skill_hub_tools.py tests/test_config_raven_sections.py tests/test_context_engine_factory.py -q → 8 failed, 1142 passed
  • the same eleven files at base 11ca10d8 → 8 failed, 1030 passed
  • failing-ID sets are identical in both directions; the 112 added tests pass. The inherited failures are the known Windows login-shell/path cases.

Coverage gaps: the ui-web tests this PR changes still cannot run — the shared D:/projects/Raven/ui-web/node_modules is empty (0 entries) — so the switch UI, the wizard seeding call and the notices are checked by reading only. The full suite was not rerun.

@LivXue
LivXue force-pushed the fix/skillhub_endpoint_seed branch from 916e7ed to 1a00cbe Compare October 30, 2026 07:36

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blockers; this can merge as far as I am concerned.

This head has the exact same Git tree as the previously reviewed clean revision (14c034766822cd88f7d3c83578e0009c70a2f035); the range-diff contains commit-metadata changes only. uv run python scripts/check_commit_messages.py github/main..HEAD passed, and both review threads remain resolved. With no code or test delta, the prior coverage of repository rules/context, callers and history, backward compatibility, test strength, and architecture still applies.

@LivXue

LivXue commented Oct 30, 2026

Copy link
Copy Markdown
Member Author

Windows compatibility re-review — PR #1141 @ 1a00cbeb2aa68d47090c1b44fa25268cbeeba164

Scope: the delta since the previously reviewed head 916e7ed7. git range-diff 11ca10d8..916e7ed7 11ca10d8..1a00cbeb shows that only commits 1, 3 and 4 differ, and only by dropping Co-Authored-By trailers from their messages; the trees are otherwise identical, and the branch's base is unchanged at 11ca10d8. No code changed since the last review.

Result: no Windows-specific issue; the conclusions of the previous review stand — both config writers go through atomic_update (portalocker LockFileEx, UTF-8, DACL/mode carry), the live read goes through the loader's migrations on a deep copy, and the per-file answer cache is keyed on the LiveConfig and its raw dict via weakrefs.

Verification on native Windows 11, fresh detached worktree D:\tmp\rv-1141f at this exact head, offline venv (uv sync --frozen --offline --all-extras --no-extra segmenter --dev), scratch HOME/USERPROFILE, PYTHONUTF8=1, PYTHONPATH pinned:

  • uv run --no-sync pytest tests/test_agent_loop_skill_tools.py tests/test_cli_onboard_commands.py tests/test_config_live.py tests/test_config_update.py tests/test_context_scent.py tests/test_raven_config_tool.py tests/test_rpc_settings.py tests/test_skill_hub_source.py tests/test_skill_hub_tools.py tests/test_config_raven_sections.py tests/test_context_engine_factory.py -q → 8 failed, 1142 passed, matching the previous head exactly.
  • The failing IDs are the inherited Windows set seen throughout this PR: three test_cli_onboard_commands.py login-shell/TTY cases, four test_config_update.py a2a chmod/mode cases, and one test_rpc_settings.py POSIX-path case; at base 11ca10d8 the same files give 8 failed, 1030 passed with identical IDs, and the 112 added tests pass.

Coverage gaps: the ui-web tests remain unrunnable here — the shared D:/projects/Raven/ui-web/node_modules is empty (0 entries) — unchanged from the previous review.

The web picker and the onboarding wizard persist the language through
config.set, but only settings.set moved the process language that the
sentences Raven composes itself are rendered in. A reader who picked
Chinese in the wizard could therefore still be shown the new Skill Hub
notice in English. config.set and config.unset now move it too, and an
unset returns to the default.
The switch stops the next search when it is turned off and is live in
both directions while the endpoint is present, but it was still declared
a reload-only setting: raven_config told the model a reload was owed for
a write that had already applied, and a switch-on whose write had to seed
the endpoint could be reported as serving. The setting is now declared
live, and the tool relays the settings service's own warning instead of
dropping it, so the one write that waits still says so.
@ZuyiZhou
ZuyiZhou self-requested a review October 30, 2026 08:22

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blockers; this can merge as far as I am concerned.

Reviewed the two-commit delta from 1a00cbeb2aa6 to b85965e3262b: config.set / config.unset now keep process i18n state aligned with the validated file write, and raven_config reports the Skill Hub switch as live while preserving the settings RPC's conditional reload warning. I checked the changed callers, the settings RPC contract, the self-configuration catalog, history, backward compatibility, and the Runtime rules for Live preferences, Generations, RPC ownership, and process-scoped i18n. Existing tests were not weakened.

Verification:

  • uv run pytest tests/test_rpc_config.py tests/test_raven_config_tool.py tests/test_config_self_surface.py -q — 343 passed
  • uv run pytest tests/test_rpc_contract_shapes.py -q — 51 passed
  • uv run pytest tests/test_i18n_boundary.py -q — 2 passed
  • Source-language checker, commit-message checker, and git diff --check passed. (make is unavailable in this environment, so I ran the underlying source-language script directly with github/main..HEAD.)

Both existing review threads are resolved.

@LivXue
LivXue merged commit 7c12b93 into main Oct 30, 2026
25 checks passed
@LivXue
LivXue deleted the fix/skillhub_endpoint_seed branch October 30, 2026 08:32
LivXue added a commit that referenced this pull request Oct 30, 2026
## Summary

Three storage layers of the trace store, each with an idempotent, opt-in
migration:

1. **Payloads.** Every captured artifact and every content-addressed
message is written as a zstd frame under the path it already had. `raven
tracing compress` migrates an existing store in place and moves each
artifact hard link onto the one compressed blob; `--dry-run` compresses
in memory to report the real saving and writes nothing.
2. **Cold logs.** A rotated span or event log is compressed one rotation
later, never the log just rotated (another writer may still be appending
its last line to that inode), and by `raven tracing compress --logs`,
which leaves the newest rotated log of each kind for that next rotation;
a compressed log keeps its mtime, which is how the viewer orders logs.
The span-log readers decode by the frame and keep working in plaintext
byte offsets, which is the only coordinate the session index cursor and
the viewer shard sidecars record. Compression gives a log a new inode,
so the session index recognises it by the sha1 of its first 64 plaintext
bytes and carries the cursor over instead of reading it again.
3. **Message packs.** Content-addressed messages are packed into
multi-megabyte zstd frames with an `sha1 -> (pack, plaintext offset,
length)` index. `raven tracing pack-messages` writes the packs, commits
the index atomically, and only then removes the message files; a reader
that finds no file reads its slice.

A fourth commit bounds what a stored frame may decode to, after the
automated security review flagged decompression bombs. A later review
round fixed six defects, each with a regression test that failed before
its change: a frame without a content-size field was measured from the
wrong offset (a span log then reset the session index on every other
pass); a per-shell budget cut inside a multi-byte character raised
instead of degrading; the viewer accepted Node 23.0-23.7, which have no
zstd; the pack reader's streaming skip was unbounded and packs past the
cache budget were decoded whole rather than streamed; the pack cleanup
deleted a message file whose pack no longer decoded; and CONTEXT.md
still said the message store had no reclaim path. A third round fixed
four more the same way: compressing a rotated log replaced its inode, so
the session index read it again from offset 0 (a record without ids was
listed twice, a rotation mid-scan reset the index, as did ext4 handing
the freed inode to the next log, and a spilled trace was counted as
lost); compressing the log just rotated dropped a line another writer
had opened the file to append, at rotation and in `raven tracing
compress --logs` alike; the writer did not write a packed message again
when its pack was gone or cut short, so a store copied incompletely
never healed; and a PATH Node without zstd hid a zstd-capable one later
in the search order, such as the runtime under `~/.raven/runtime`. The
command table also stopped calling packs archives.

Why: the live store on this machine was 1357.7 MB, most of it block
slack and uncompressed JSON. On a clone of that store the archive logs
fell from 94.0 MB to 8.2 MB, and 40,056 messages fell from 225 MB of
disk to 12 packs (23.6 MB) plus a 3.9 MB index. Projected whole-store
result: about 548 MB, roughly 60 percent saved.

Key decisions:

- **Paths never change, the frame decides.** A span records
`artifact_path` when it is written and those records are the audit
trail, so a migration cannot rename anything. Readers sniff the zstd
magic, which is also what keeps a store written before this PR readable
and a half-migrated one coherent.
- **ACP frame journals stay plaintext on purpose.** A span records a
plaintext byte range into its journal (`subagent.external.frames.start`
/ `end`), and #1147's frames replay reads exactly those offsets.
Re-encoding a journal would silently invalidate that audit record, so
neither the migration nor rotation touches it.
- **Pack migration order is crash-safe.** Packs, then the index via
atomic replace, then deletion, then an orphan sweep. A crash at any
point leaves every address readable from one side or the other. The
writer skips an address whose pack still decodes and covers its slice,
so a resumed or repeated turn does not recreate files that were packed,
while an address whose pack was lost or cut short is written again. The
deletion pass keeps a file whose pack no longer decodes, so a store
copied incompletely does not lose its last readable copy.
- **Reader caches are bounded.** The pack index is cached per process
until its size or mtime changes; at most 32 MB of decompressed packs are
kept, and a pack larger than the whole budget is streamed per read.

Relationship to #1147: the two branches overlap in four files
(`CONTEXT.md`, `raven/trajectory/conversation.py`,
`raven/trajectory/details.py`, `tests/test_subagent_acp.py`), none of
them touched by the third-round fixes, and a merge-tree dry run against
its head `1950ad79` has no conflict. #1147 does not touch
`pyproject.toml` or `uv.lock`. This branch was rebased onto main; the
only conflict was main's `openai` dependency from #1108, resolved by
keeping both entries and re-locking. A second rebase, onto `7c12b93f`
(#1141), had no conflict.

Deployment: deploy this code before running either migration. The
bundled viewer needs Node 22.15 or newer (23.8 on the 23.x line) for the
zstd decoder, the same requirement its compressed-artifact path already
has. Code older than this PR cannot read a compressed payload, and the
write path compresses new artifacts by default, so the deployment is
one-way once anything new is written; the migrations themselves are
manual and opt-in.

Measurements on a clone of the live store (2026-10-29):

- Archive span/event logs: 94.0 MB -> 8.2 MB (11.4x); 74,098 spans read
identically before and after; a second pass reports `compressed 0,
already 33`.
- Messages: 40,056 addresses, 225 MB of disk -> 12 packs (23.6 MB) + 3.9
MB index; all 40,056 read-backs match the stored plaintext; a real v2
artifact resolves identically from files and from packs.
- Read overhead: +0.01 to 0.11 ms for small and medium payloads, +0.45
to 0.49 ms for a 2.1 MB payload read with a 512 KiB cap; 40,056 packed
reads in 1.1 s.
- Migration speed: 4,973 files/s, 38 MB/s; full dry run 24.3 s; 40,056
messages packed in 1.7 s.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

Commands and results:

- `uv run pytest tests/test_tracing_store_compression.py
tests/test_trajectory_index.py tests/test_cli_tracing_commands.py
tests/test_tracing_api.py tests/test_subagent_acp.py
tests/test_trajectory_conversation.py tests/test_trajectory_details.py
tests/test_trajectory_selection.py tests/test_trajectory_bundle.py
tests/test_trajectory_export.py tests/test_rpc_cli_dispatch.py
tests/test_utils_mirrors.py tests/test_node_server.py
tests/test_tracing_message_pack.py -q -n 4`: 1127 passed, 1 skipped.
- Review-fix round, the affected tracing and trajectory suites plus the
CLI, RPC and kernel-closure tests, before and after the rebase onto
main: 978 passed, 1 skipped. Each of the six fixes has a regression test
that failed at the reviewed head and passes now.
- Third review round: each of the four fixes has a regression test that
failed at `57447ef8` and passes now, and removing each fix again (nine
mutants: no cursor adoption, adopting only for a key that vanished, no
moved-key lookup in the rescan, compressing the just-rotated log, no
mtime kept at rotation, the migration compressing the newest log, the
migration not keeping mtimes, an index-only packed check, the TUI floor
in the viewer's Node search) turns exactly its own tests red. `uv run
pytest tests/test_trajectory_index.py
tests/test_tracing_store_compression.py
tests/test_tracing_compression_edges.py
tests/test_tracing_message_pack.py tests/test_cli_tracing_commands.py
-q`: 188 passed, and the same files pass on each of the six fix commits'
own tree.
- Fourth review round: the writer now verifies that a pack still decodes
and covers a packed address's slice, so a pack cut short heals like a
missing one. The new test in `tests/test_tracing_message_pack.py` failed
before the change (the message files were not written again) and passes
now; the decoded length is streamed and cached per pack, so a healthy
store pays one decompression per pack rather than one per address. The
affected tracing, trajectory, CLI and RPC set passed: 1050 tests. Full
suite at the fixed head: 37157 passed, 8 failed - the same known local
baselines as before.
- `uv run pytest tests/integration/test_tracing_viewer_e2e.py -m
integration -q`: 28 passed.
- `uv run pytest tests/integration/test_kernel_wheel_smoke.py -q`: 1
passed.
- Full suite at the fixed head: 37157 passed, 8 failed, 255 skipped, 2
xfailed in 7:29, after the rebase onto `7c12b93f`. The eight are the
known local baselines on this box (five proxy-environment tests, the
Excel parser, the install script, and the node runtime probe); the same
eight IDs also fail at `57447ef8` here, and none of the new tests is
among them. The new index tests also pass with the test temp directory
on ext4, which reuses a freed inode at once, and on tmpfs, which does
not.
- `uv run ruff check` and `uv run ruff format --check` on the changed
trees: clean.
- `uv run --frozen --all-extras ty check raven/tracing raven/trajectory
raven/cli/tracing_commands.py raven/cli/tui_commands.py
raven/rpc/methods/cli_dispatch.py`: all checks passed.
- `uv run --frozen --all-extras lint-imports`: 10 kept, 0 broken.
- `scripts/check_commit_messages.py origin/main..HEAD`, `commitlint
--from origin/main --to HEAD`, `scripts/check_source_language.py
origin/main...HEAD` and `scripts/check_large_files.py
origin/main...HEAD`: exit 0.
- The three feature commits were each verified on their own tree before
the next one was built; the fourth commit bounds decode output and is
covered by the same batch above.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

- Security: local files only; no new network surface, credentials, or
permissions. A trace store can be imported from another machine, so
decompression is bounded rather than trusted: every decode path stops at
256 MiB (including the pack reader's offset skip), far above the 32.9 MB
largest captured payload, and treats an oversized or corrupt frame as
unreadable, the viewer passes `maxOutputLength` to its zstd decode, and
`resolve_payload` carries a per-shell total budget so a shell naming
many addresses cannot pull in more than one bounded amount. The pack
index is validated (40-hex addresses, integer offsets) before any path
is built.
- Backward compatibility: readers tell the encoding from the frame, so
plain and half-migrated stores read. Older code cannot read a migrated
payload, which is why both migrations are manual and the deployment
order is stated above. ACP journals are not migrated at all.
- Performance: measured read overhead is sub-millisecond; the pack cache
is capped at 32 MB per process and the index is 3.9 MB for 40,000
messages. A rotation compresses at most one earlier log, and finding it
among 2,191 archives took 9 ms.
- Rollback: the write path stores every new artifact compressed, so a
revert is only clean if no artifact has been written by this version.
After any new write or a migration there is no decompress command in
this PR; any current reader still returns the original plaintext, and a
decompress command can be a follow-up if one is wanted.

## Related Issues

N/A. The relationship to #1147 is described in Summary.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants