Skip to content

test(skill-install): serialize harness tests around the home env vars - #293

Merged
iuyo5678 merged 1 commit into
Tencent:mainfrom
lyingbug:fix/harness-env-test-lock
Sep 20, 2026
Merged

iuyo5678 merged 1 commit into
Tencent:mainfrom
lyingbug:fix/harness-env-test-lock

Conversation

@lyingbug

Copy link
Copy Markdown
Contributor

Fixes #292.

Problem

hermes_skills_dir_honors_hermes_home_env overrides the process-wide HERMES_HOME without holding any lock, while detects_hermes_from_home_layout and skills_dirs_match_harness_spec resolve Hermes through that same variable on other libtest threads. On main (d04c752), cargo test -p bsk --lib skill_install failed 6 times in 30 runs, always detects_hermes_from_home_layout reading the other test's temp directory:

thread '...detects_hermes_from_home_layout' panicked at crates/bsk-cli/src/skill_install/harness.rs:552:9:
assertion `left == right` failed
  left: "/var/folders/.../.tmpnVRcYf/custom-hermes/skills"
 right: "/var/folders/.../.tmpaX6clB/.hermes/skills"

The Kimi Code tests already share kimi_env_lock, but skills_dirs_match_harness_spec reads KIMI_CODE_HOME without taking it, so the same failure was latent there. Both unsafe blocks carried SAFETY comments whose stated invariant did not hold: the hazard is any concurrent reader, not only another writer, which is also why edition 2024 marks set_var unsafe.

Change

Test-only. kimi_env_lock becomes harness_env_lock, covering both variables, with a lock_harness_env() helper; all five tests that read or write HERMES_HOME / KIMI_CODE_HOME now take it, and the two SAFETY comments state the invariant the lock actually provides.

Validation

  • cargo test -p bsk --lib skill_install, 30 runs on the same machine: 6 failures before, 0 after.
  • cargo test -p bsk --lib, 30 runs after the change: 0 failures. The same race hit skills_dirs_match_harness_spec there at roughly 1 run in 10 before.
  • cargo test --workspace --locked passes. One unrelated failure appeared once during validation, daemon::probe::readiness_tests::readiness_retries_pid_mismatch_and_discovery_changes (probe_tests.rs:177), and did not recur in 30 further runs; I did not investigate it here.
  • cargo fmt --all -- --check and cargo clippy --workspace --all-targets --locked -- -D warnings pass.
  • No production code is touched.

Possible follow-up

The lock covers this module. Tests elsewhere in the crate reach these variables only by iterating HarnessId::ALL (for example sync_installed_harnesses) and none of them assert on the resolved home, but they do still call getenv while an override is installed. Removing the unsafe mutation entirely — resolvers taking an explicit override argument, with the environment read at the call site — would close that window too. Happy to send it if you want that shape.

hermes_skills_dir_honors_hermes_home_env overrode the process-wide
HERMES_HOME without any lock, while detects_hermes_from_home_layout and
skills_dirs_match_harness_spec resolved Hermes through that variable on
other libtest threads: the detection test read the override's temp
directory and failed about six times in thirty runs. The Kimi tests took
kimi_env_lock, but skills_dirs_match_harness_spec read KIMI_CODE_HOME
without it, leaving the same failure latent there.

Widen that lock to harness_env_lock, take it in every test that reads or
writes HERMES_HOME or KIMI_CODE_HOME, and replace the two SAFETY comments
whose invariant did not hold: the hazard is a concurrent reader, not only
another writer.
@iuyo5678
iuyo5678 merged commit 907b455 into Tencent:main Sep 20, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flaky test: HERMES_HOME mutation races concurrent readers in skill_install::harness

2 participants