diff --git a/Cargo.lock b/Cargo.lock index 1ad534707..157eb613c 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1781,7 +1781,6 @@ dependencies = [ "axum", "base64", "bytes", - "chrono", "clap", "clap_complete", "console 0.16.3", diff --git a/README.md b/README.md index db0977d96..3a033090d 100644 --- a/README.md +++ b/README.md @@ -28,7 +28,7 @@ evaluation products, see [the Ecosystem guide](https://docs.nvidia.com/nemo/rela | Goal | Start With | |---|---| -| Observe Codex, Claude Code, or Hermes locally with the CLI | [Quick Start CLI](https://docs.nvidia.com/nemo/relay/nemo-relay-cli/about) | +| Observe Codex or Claude Code locally with the CLI | [Quick Start CLI](https://docs.nvidia.com/nemo/relay/nemo-relay-cli/about) | | Instrument app-owned LLM or tool calls | [Quick Start Application](https://docs.nvidia.com/nemo/relay/getting-started/quick-start) | | Use LangChain, LangGraph, Deep Agents, or OpenClaw | [Supported Integrations](https://docs.nvidia.com/nemo/relay/supported-integrations/about) | | Build a framework or provider integration | [Integrate into Frameworks](https://docs.nvidia.com/nemo/relay/integrate-into-frameworks/about) | @@ -36,6 +36,9 @@ evaluation products, see [the Ecosystem guide](https://docs.nvidia.com/nemo/rela | Package reusable middleware or exporters | [Build Plugins](https://docs.nvidia.com/nemo/relay/build-plugins/about) | | Develop or test this repository from source | [CONTRIBUTING.md](CONTRIBUTING.md) | +Hermes Agent understands NeMo Relay plugin configurations. NeMo Relay is built +into Hermes Agent, so no separate observability plugin or Relay CLI setup is +required. ## Quick Start CLI @@ -45,7 +48,7 @@ trajectory file, you have concrete data to inspect, debug, and build on. ### Local Agent Trajectory This walkthrough shows an end-to-end quick success setup. Install the -NeMo Relay CLI, turn on local exporters, run Codex, Claude Code, or Hermes +NeMo Relay CLI, turn on local exporters, run Codex or Claude Code through Relay, and check that Relay wrote both raw events and normalized trajectories. @@ -129,12 +132,6 @@ For Claude Code, run: nemo-relay claude -- "Summarize this repository." ``` -For Hermes, run: - -```bash -nemo-relay hermes -- -z "Summarize this repository." -``` - Refer to the full [Quick Start CLI](https://docs.nvidia.com/nemo/relay/nemo-relay-cli/about) docs for more options. The transparent wrapper starts a local Relay gateway, injects host-specific hook @@ -318,7 +315,7 @@ coverage. |:--|:--:|:--:|:--:|:--| | Claude Code | Yes | Yes | Partial | Hook forwarding, pre-tool blocking, and gateway-routed LLM observability are supported. | | Codex | Yes | Yes | Partial | Persistent install verifies the exact plugin hooks. Each `Stop` finalizes a turn snapshot; the supported generated schema does not install `SessionEnd`. | -| Hermes Agent | Yes | Yes | Partial | User config installs the shared native MCP gateway lifecycle plus exact trusted hooks; gateway-routed or hook-backed LLM observability is supported. | +| Hermes Agent | Yes | Yes | Partial | NeMo Relay is built into Hermes Agent, and Hermes Agent understands NeMo Relay plugin configurations. No separate observability plugin or Relay CLI setup is required. | ### Public API Integrations diff --git a/crates/cli/Cargo.toml b/crates/cli/Cargo.toml index e04210891..3dba3df83 100644 --- a/crates/cli/Cargo.toml +++ b/crates/cli/Cargo.toml @@ -34,7 +34,6 @@ async-stream = "0.3" axum = "0.8" base64 = "0.22" bytes = "1" -chrono = "0.4" clap = { version = "4", features = ["derive", "env"] } clap_complete = "4" console = "0.16" diff --git a/crates/cli/README.md b/crates/cli/README.md index 088998622..dc78c9519 100644 --- a/crates/cli/README.md +++ b/crates/cli/README.md @@ -28,8 +28,8 @@ with the installed `nemo-relay` command rather than link against the crate. The CLI is designed for these tasks: -- **Observe existing coding agents**: Run Claude Code, Codex, or Hermes - Agent through a local NeMo Relay gateway without changing the agent +- **Observe existing coding agents**: Run Claude Code or Codex through a local + NeMo Relay gateway without changing the agent itself. - **Configure transparent runs interactively**: Use the setup wizard to write project or user configuration for supported agents. @@ -47,14 +47,14 @@ The CLI provides these capabilities: Cargo package. - **First-run setup**: Bare `nemo-relay` launches setup when no config exists, then runs doctor once config is present. -- **Agent shortcuts**: `nemo-relay claude`, `nemo-relay codex`, and - `nemo-relay hermes` start observed agent runs. +- **Agent shortcuts**: `nemo-relay claude` and `nemo-relay codex` start + observed agent runs. - **Config-driven launch**: `nemo-relay run` resolves config, environment, and CLI overrides for deterministic non-interactive use. - **Hook forwarding server**: A local gateway accepts agent hook events and provider-shaped OpenAI or Anthropic requests. -- **Persistent agent integration**: `nemo-relay install` configures Codex, - Claude Code, or Hermes Agent with one generated MCP bootstrap and the host's +- **Persistent agent integration**: `nemo-relay install` configures Codex or + Claude Code with one generated MCP bootstrap and the host's canonical lifecycle hooks. - **Shared gateway lifecycle**: Every persistent integration launches the same host-neutral `nemo-relay mcp` client. Concurrent clients share one native diff --git a/crates/cli/src/agents/claude/mod.rs b/crates/cli/src/agents/claude/mod.rs index b08ab9b11..cb39a3295 100644 --- a/crates/cli/src/agents/claude/mod.rs +++ b/crates/cli/src/agents/claude/mod.rs @@ -35,7 +35,6 @@ pub(super) const DESCRIPTOR: AgentDescriptor = AgentDescriptor { "PostCompact", "SessionEnd", ], - direct_hook_entries: false, }; pub(super) fn parse_version(raw: &str) -> Option { diff --git a/crates/cli/src/agents/codex/mod.rs b/crates/cli/src/agents/codex/mod.rs index c3db8c5a5..542515982 100644 --- a/crates/cli/src/agents/codex/mod.rs +++ b/crates/cli/src/agents/codex/mod.rs @@ -32,7 +32,6 @@ pub(super) const DESCRIPTOR: AgentDescriptor = AgentDescriptor { "PreCompact", "PostCompact", ], - direct_hook_entries: false, }; pub(super) fn parse_version(raw: &str) -> Option { diff --git a/crates/cli/src/agents/hermes/adapter.rs b/crates/cli/src/agents/hermes/adapter.rs deleted file mode 100644 index 6465d4513..000000000 --- a/crates/cli/src/agents/hermes/adapter.rs +++ /dev/null @@ -1,354 +0,0 @@ -// SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. -// SPDX-License-Identifier: Apache-2.0 - -use axum::http::HeaderMap; -use serde_json::{Map, Value, json}; - -use crate::agents::shared::adapters::{ - AdapterOutcome, ClassificationRules, HERMES_PAYLOAD_EXTRACTOR, classify, common_session_event, - event_name, metadata, normalize_name, session_id, -}; -use crate::events::json_path::value_at; -use crate::events::{AgentKind, LlmEvent, NormalizedEvent}; - -/// Normalizes Hermes shell hook payloads without emitting control directives. -/// -/// Hermes hooks are installed as shell commands and may run outside `run`, so this adapter keeps -/// responses minimal and relies on the forwarder fail-open/fail-closed setting to decide whether -/// hook delivery problems affect the invoking agent. -pub(crate) fn adapt(payload: Value, headers: &HeaderMap) -> AdapterOutcome { - let event_name = event_name(&payload, &HERMES_PAYLOAD_EXTRACTOR); - let normalized = normalize_name(&event_name); - if normalized == "preapirequest" { - return AdapterOutcome { - events: vec![crate::events::NormalizedEvent::LlmStarted( - hermes_llm_event(&payload, headers, &event_name), - )], - response: json!({}), - }; - } - if normalized == "postapirequest" { - return AdapterOutcome { - events: vec![crate::events::NormalizedEvent::LlmEnded(hermes_llm_event( - &payload, - headers, - &event_name, - ))], - response: json!({}), - }; - } - if normalized == "apirequesterror" { - return AdapterOutcome { - events: vec![crate::events::NormalizedEvent::LlmEnded(hermes_llm_event( - &payload, - headers, - &event_name, - ))], - response: json!({}), - }; - } - if normalized == "pretoolcall" && !hermes_pre_tool_call_is_correlatable(&payload, headers) { - return AdapterOutcome { - events: Vec::new(), - response: json!({}), - }; - } - - // `on_session_end` is a Hermes per-turn boundary, not user-visible trajectory content. - // Emitting it as both HookMark and TurnEnded polluted ATIF with system rows whose only purpose - // was to trigger a snapshot. Keep the snapshot signal and leave the agent scope open. - if normalized == "onsessionend" { - return AdapterOutcome { - events: vec![NormalizedEvent::TurnEnded(common_session_event( - &payload, - headers, - AgentKind::Hermes, - &HERMES_PAYLOAD_EXTRACTOR, - ))], - response: json!({}), - }; - } - - let events = classify( - &payload, - headers, - &HERMES_PAYLOAD_EXTRACTOR, - &ClassificationRules { - kind: AgentKind::Hermes, - agent_start: &["on_session_start", "sessionStart"], - agent_end: &["on_session_finalize", "on_session_reset"], - subagent_start: &["subagent_start", "subagentStart"], - subagent_end: &["subagent_stop", "subagentStop"], - tool_start: &["pre_tool_call", "preToolCall"], - tool_end: &["post_tool_call", "postToolCall"], - }, - ); - AdapterOutcome { - events, - response: json!({}), - } -} - -fn hermes_llm_event(payload: &Value, headers: &HeaderMap, event_name: &str) -> LlmEvent { - let session_id = session_id(payload, headers, &HERMES_PAYLOAD_EXTRACTOR); - let api_call_id = hermes_api_call_id(payload, &session_id); - let provider = hermes_string_at(payload, "provider") - .or_else(|| hermes_string_at(payload, "api_mode")) - .unwrap_or_else(|| "hermes_api_request".to_string()); - let model_name = - hermes_string_at(payload, "response_model").or_else(|| hermes_string_at(payload, "model")); - let payload_exact = hermes_payload_exact(payload, event_name); - let mut event_metadata = metadata( - payload, - headers, - AgentKind::Hermes, - event_name, - &HERMES_PAYLOAD_EXTRACTOR, - ); - if let Value::Object(ref mut object) = event_metadata { - object.insert("api_call_id".into(), json!(api_call_id.clone())); - object.insert("provider_payload_exact".into(), json!(payload_exact)); - object.insert( - "fidelity_source".into(), - json!(if payload_exact { - "hermes_api_hooks_sanitized" - } else { - "hermes_api_hooks" - }), - ); - } - LlmEvent { - session_id, - agent_kind: AgentKind::Hermes, - event_name: event_name.to_string(), - api_call_id, - provider, - model_name, - request: hermes_llm_request(payload), - response: hermes_llm_response(payload), - metadata: event_metadata, - } -} - -fn hermes_api_call_id(payload: &Value, session_id: &str) -> String { - // Newer Hermes request-scoped hooks emit a stable per-attempt ID. Prefer it so pre, post, - // error, tool, and approval telemetry can join without depending on turn-local counters. - // Older Hermes payloads do not have it, so keep the synthesized ID for compatibility. - if let Some(api_request_id) = hermes_string_at(payload, "api_request_id") { - return api_request_id; - } - let task_id = hermes_string_at(payload, "task_id").unwrap_or_default(); - let api_call_count = hermes_string_at(payload, "api_call_count").unwrap_or_default(); - format!("{session_id}:{task_id}:{api_call_count}") -} - -fn hermes_llm_request(payload: &Value) -> Value { - // Prefer first-party sanitized request bodies from newer Hermes telemetry hooks. This is still - // observer-only data: NeMo Relay is not intercepting or rewriting Hermes execution here. When the - // exact payload is absent or was truncated by Hermes, fall back to the legacy summary shape. - if let Some(request) = hermes_exact_request(payload) { - return request; - } - let mut object = Map::new(); - for key in [ - "task_id", - "session_id", - "platform", - "model", - "provider", - "base_url", - "api_mode", - "api_call_count", - "message_count", - "tool_count", - "approx_input_tokens", - "request_char_count", - "max_tokens", - ] { - if let Some(value) = hermes_value_at(payload, key) { - object.insert(key.into(), value); - } - } - object.insert( - "fidelity".into(), - json!({ - "provider_payload_exact": false, - "source": "hermes_pre_api_request" - }), - ); - Value::Object(object) -} - -fn hermes_llm_response(payload: &Value) -> Value { - // Prefer first-party sanitized response bodies from newer Hermes telemetry hooks. Older Hermes - // versions only send summary fields, which remain useful for latency/token accounting but not - // full ATIF reconstruction. - if let Some(response) = hermes_exact_response(payload) { - return response; - } - let mut object = Map::new(); - for key in [ - "task_id", - "session_id", - "platform", - "model", - "provider", - "base_url", - "api_mode", - "api_call_count", - "api_duration", - "finish_reason", - "message_count", - "response_model", - "usage", - "assistant_content_chars", - "assistant_tool_call_count", - "status_code", - "retry_count", - "max_retries", - "retryable", - "reason", - "error", - ] { - if let Some(value) = hermes_value_at(payload, key) { - object.insert(key.into(), value); - } - } - Value::Object(object) -} - -fn hermes_payload_exact(payload: &Value, event_name: &str) -> bool { - // The fallback is automatic and per-event: exact sanitized hook payloads get marked as - // provider_payload_exact=true, while missing/truncated payloads retain the lossy summary marker. - // Consumers can inspect these metadata fields to decide whether the trace is reconstruction - // grade or summary-only. - // - // Follow-up: once the Hermes middleware branch that emits sanitized request/response hook - // payloads is available in the smoke environment, rerun the Hermes Harbor smoke against that - // version to validate the exact hook-telemetry path end to end. Until then, the smoke mainly - // exercises the legacy summary fallback. - match normalize_name(event_name).as_str() { - "preapirequest" => hermes_exact_request(payload).is_some(), - "postapirequest" => hermes_exact_response(payload).is_some(), - _ => false, - } -} - -fn hermes_pre_tool_call_is_correlatable(payload: &Value, headers: &HeaderMap) -> bool { - // Public Hermes releases can emit `pre_tool_call` with only a turn/task id. Treating that - // `task_id` as a session opens a synthetic session that is later closed as `gateway_shutdown`. - // Keep pre-tool spans only when they can be routed to a real session and paired with a stable - // tool call id. The matching `post_tool_call` still records the tool result. - has_explicit_hermes_session_id(payload, headers) && has_explicit_hermes_tool_call_id(payload) -} - -fn has_explicit_hermes_session_id(payload: &Value, headers: &HeaderMap) -> bool { - header_has_value(headers, "x-nemo-relay-session-id") - || header_has_value(headers, "x-claude-code-session-id") - || hermes_string_at(payload, "session_id").is_some() - || hermes_string_at(payload, "sessionId").is_some() - || value_at(payload, &["session", "id"]).is_some() - || hermes_string_at(payload, "conversation_id").is_some() - || hermes_string_at(payload, "conversationId").is_some() - || hermes_string_at(payload, "parent_session_id").is_some() -} - -fn has_explicit_hermes_tool_call_id(payload: &Value) -> bool { - hermes_string_at(payload, "tool_call_id").is_some() - || hermes_string_at(payload, "toolCallId").is_some() - || hermes_string_at(payload, "tool_use_id").is_some() - || hermes_string_at(payload, "call_id").is_some() - || value_at(payload, &["tool", "id"]).is_some() - || value_at(payload, &["tool_input", "id"]).is_some() - || hermes_string_at(payload, "id").is_some() -} - -fn header_has_value(headers: &HeaderMap, name: &str) -> bool { - headers - .get(name) - .and_then(|value| value.to_str().ok()) - .is_some_and(|value| !value.trim().is_empty()) -} - -fn hermes_exact_request(payload: &Value) -> Option { - let request = hermes_value_at(payload, "request")?; - // Hermes bounds hook payload size before invoking plugins. A truncated payload is intentionally - // not treated as exact, because ATIF/ATOF reconstruction would otherwise trust partial context. - if request.is_null() || is_truncated_payload(&request) { - return None; - } - request - .get("body") - .filter(|body| !body.is_null()) - .cloned() - .or(Some(request)) -} - -fn hermes_exact_response(payload: &Value) -> Option { - let response = hermes_value_at(payload, "response")?; - // Same rule as requests: truncated response telemetry is useful as a diagnostic, but it is not - // exact provider payload evidence. - if is_truncated_payload(&response) { - return None; - } - if let Some(raw_response) = response - .get("raw_response") - .filter(|raw_response| !raw_response.is_null() && !is_truncated_payload(raw_response)) - { - return Some(raw_response.clone()); - } - if response.get("choices").is_some() - || response.get("output").is_some() - || response.get("content").is_some() - { - return Some(response); - } - let assistant_message = response.get("assistant_message")?; - let mut object = Map::new(); - if let Some(content) = assistant_message.get("content") { - object.insert("content".into(), content.clone()); - } - if let Some(tool_calls) = assistant_message.get("tool_calls") { - object.insert("tool_calls".into(), tool_calls.clone()); - } - if let Some(usage) = response - .get("usage") - .cloned() - .or_else(|| hermes_value_at(payload, "usage")) - { - object.insert("usage".into(), usage); - } - for key in ["model", "finish_reason"] { - if let Some(value) = response - .get(key) - .cloned() - .or_else(|| hermes_value_at(payload, key)) - { - object.insert(key.into(), value); - } - } - (!object.is_empty()).then_some(Value::Object(object)) -} - -fn is_truncated_payload(value: &Value) -> bool { - value - .get("_truncated") - .and_then(Value::as_bool) - .unwrap_or(false) -} - -fn hermes_string_at(payload: &Value, key: &str) -> Option { - value_at(payload, &[key]) - .or_else(|| value_at(payload, &["extra", key])) - .and_then(|value| match value { - Value::String(value) => Some(value), - Value::Number(value) => Some(value.to_string()), - Value::Bool(value) => Some(value.to_string()), - _ => None, - }) - .filter(|value| !value.is_empty()) -} - -fn hermes_value_at(payload: &Value, key: &str) -> Option { - value_at(payload, &[key]).or_else(|| value_at(payload, &["extra", key])) -} diff --git a/crates/cli/src/agents/hermes/alignment.rs b/crates/cli/src/agents/hermes/alignment.rs deleted file mode 100644 index 85e514bff..000000000 --- a/crates/cli/src/agents/hermes/alignment.rs +++ /dev/null @@ -1,243 +0,0 @@ -// SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. -// SPDX-License-Identifier: Apache-2.0 - -//! Hermes-specific trace alignment. -//! -//! Hermes reports subagent lifecycle through parent-session hooks, while the child worker also -//! opens its own Hermes session. The parent `subagent_start` payload carries the bridge fields -//! (`child_session_id`, `child_subagent_id`, and `parent_turn_id`) needed to route later child -//! session events back under the parent subagent scope. - -use serde_json::{Map, Value, json}; - -use crate::agents::shared::alignment::{ - SessionAlias, insert_optional, json_string_at, merge_metadata, -}; -use crate::events::{AgentKind, SessionEvent, SubagentEvent}; - -#[derive(Debug, Clone)] -pub(crate) struct SubagentContext { - pub(crate) parent_session_id: String, - pub(crate) subagent_id: String, - pub(crate) child_session_id: String, - parent_turn_id: Option, - child_role: Option, - child_goal: Option, -} - -#[derive(Debug, Clone)] -pub(crate) struct ExplicitSubagentAlias { - pub(crate) child_session_id: String, - pub(crate) alias: SessionAlias, - pub(crate) scope_metadata: Value, -} - -pub(crate) fn subagent_context(event: &SessionEvent) -> Option { - if event.agent_kind != AgentKind::Hermes { - return None; - } - context_from_values(&event.session_id, &event.payload, &event.metadata) - .filter(|context| context.parent_session_id != event.session_id) -} - -pub(crate) fn explicit_subagent_alias(event: &SubagentEvent) -> Option { - if event.agent_kind != AgentKind::Hermes { - return None; - } - let context = context_from_values(&event.session_id, &event.payload, &event.metadata)?; - if context.child_session_id == event.session_id { - return None; - } - let scope_metadata = scope_metadata(event.metadata.clone(), &context); - let alias = alias_for_context(&context); - Some(ExplicitSubagentAlias { - child_session_id: context.child_session_id, - alias, - scope_metadata, - }) -} - -pub(crate) fn child_session_id_for_subagent_event(event: &SubagentEvent) -> Option { - if event.agent_kind != AgentKind::Hermes { - return None; - } - context_from_values(&event.session_id, &event.payload, &event.metadata) - .map(|context| context.child_session_id) -} - -pub(crate) fn augment_subagent_metadata(metadata: Value, context: &SubagentContext) -> Value { - scope_metadata(metadata, context) -} - -pub(crate) fn subagent_start_event( - event: &SessionEvent, - context: &SubagentContext, -) -> SubagentEvent { - SubagentEvent { - session_id: context.parent_session_id.clone(), - agent_kind: event.agent_kind, - event_name: event.event_name.clone(), - subagent_id: context.subagent_id.clone(), - payload: event.payload.clone(), - metadata: scope_metadata(event.metadata.clone(), context), - } -} - -pub(crate) fn alias_for_child_session( - _child_session_id: String, - context: &SubagentContext, -) -> SessionAlias { - alias_for_context(context) -} - -pub(crate) fn llm_owner_metadata(scope_metadata: Option<&Value>) -> Value { - let Some(Value::Object(scope_metadata)) = scope_metadata else { - return Value::Null; - }; - let mut metadata = Map::new(); - for key in [ - "thread_source", - "subagent_id", - "subagent_session_id", - "hermes_parent_session_id", - "hermes_subagent_session_id", - "hermes_child_subagent_id", - "hermes_parent_turn_id", - "child_role", - "child_goal", - ] { - if let Some(value) = scope_metadata.get(key) - && !value.is_null() - { - metadata.insert(key.to_string(), value.clone()); - } - } - if metadata.is_empty() { - Value::Null - } else { - Value::Object(metadata) - } -} - -fn context_from_values( - default_parent_session_id: &str, - payload: &Value, - metadata: &Value, -) -> Option { - let child_session_id = child_session_id(payload).or_else(|| child_session_id(metadata))?; - let parent_session_id = parent_session_id(payload) - .or_else(|| parent_session_id(metadata)) - .unwrap_or_else(|| default_parent_session_id.to_string()); - let subagent_id = subagent_id(payload) - .or_else(|| subagent_id(metadata)) - .unwrap_or_else(|| child_session_id.clone()); - Some(SubagentContext { - parent_session_id, - subagent_id, - child_session_id, - parent_turn_id: optional_string(payload, metadata, "parent_turn_id"), - child_role: optional_string(payload, metadata, "child_role"), - child_goal: optional_string(payload, metadata, "child_goal"), - }) -} - -fn alias_for_context(context: &SubagentContext) -> SessionAlias { - SessionAlias::new( - context.parent_session_id.clone(), - context.subagent_id.clone(), - alias_metadata(context), - ) -} - -fn alias_metadata(context: &SubagentContext) -> Value { - Value::Object(base_metadata(context)) -} - -fn scope_metadata(metadata: Value, context: &SubagentContext) -> Value { - let mut object = base_metadata(context); - object.insert("session_id".into(), json!(context.child_session_id.clone())); - merge_metadata(metadata, Value::Object(object)) -} - -fn base_metadata(context: &SubagentContext) -> Map { - let mut object = Map::new(); - object.insert("thread_source".into(), json!("subagent")); - object.insert("subagent_id".into(), json!(context.subagent_id.clone())); - object.insert( - "subagent_session_id".into(), - json!(context.child_session_id.clone()), - ); - object.insert( - "hermes_parent_session_id".into(), - json!(context.parent_session_id.clone()), - ); - object.insert( - "hermes_subagent_session_id".into(), - json!(context.child_session_id.clone()), - ); - object.insert( - "hermes_child_subagent_id".into(), - json!(context.subagent_id.clone()), - ); - insert_optional( - &mut object, - "hermes_parent_turn_id", - context.parent_turn_id.as_deref(), - ); - insert_optional(&mut object, "child_role", context.child_role.as_deref()); - insert_optional(&mut object, "child_goal", context.child_goal.as_deref()); - object -} - -fn parent_session_id(value: &Value) -> Option { - json_string_at( - value, - &[ - &["parent_session_id"][..], - &["parentSessionId"][..], - &["parent", "session_id"][..], - &["extra", "parent_session_id"][..], - &["extra", "parentSessionId"][..], - &["extra", "parent", "session_id"][..], - ], - ) -} - -fn child_session_id(value: &Value) -> Option { - json_string_at( - value, - &[ - &["child_session_id"][..], - &["childSessionId"][..], - &["subagent_session_id"][..], - &["subagentSessionId"][..], - &["extra", "child_session_id"][..], - &["extra", "childSessionId"][..], - &["extra", "subagent_session_id"][..], - &["extra", "subagentSessionId"][..], - ], - ) -} - -fn subagent_id(value: &Value) -> Option { - json_string_at( - value, - &[ - &["child_subagent_id"][..], - &["childSubagentId"][..], - &["subagent_id"][..], - &["subagentId"][..], - &["agent_id"][..], - &["extra", "child_subagent_id"][..], - &["extra", "childSubagentId"][..], - &["extra", "subagent_id"][..], - &["extra", "subagentId"][..], - &["extra", "agent_id"][..], - ], - ) -} - -fn optional_string(payload: &Value, metadata: &Value, key: &str) -> Option { - json_string_at(payload, &[&[key][..], &["extra", key][..]]) - .or_else(|| json_string_at(metadata, &[&[key][..], &["extra", key][..]])) -} diff --git a/crates/cli/src/agents/hermes/config.rs b/crates/cli/src/agents/hermes/config.rs deleted file mode 100644 index 06c3d413b..000000000 --- a/crates/cli/src/agents/hermes/config.rs +++ /dev/null @@ -1,346 +0,0 @@ -// SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. -// SPDX-License-Identifier: Apache-2.0 - -//! Pure Hermes YAML generation, migration, and ownership recognition. - -use std::path::{Path, PathBuf}; - -use serde_json::{Map, Value, json}; - -use crate::error::CliError; -use crate::hooks::{GeneratedHookCommands, generated_policy_hooks, merge_hooks}; - -pub(super) use crate::mcp::SERVER_NAME as MCP_SERVER_NAME; - -pub(super) fn user_config_path_with_override( - default_home: &Path, - hermes_home: Option, -) -> PathBuf { - hermes_home - .filter(|value| !value.is_empty()) - .map(PathBuf::from) - .unwrap_or_else(|| default_home.join(".hermes")) - .join("config.yaml") -} - -/// Rewrites the Relay-owned portion of a Hermes config for a transparent run. The fixed MCP -/// client is removed because the wrapper already owns a dynamic gateway. -pub(crate) fn transparent_config( - existing: &str, - relay: &Path, - gateway_url: &str, -) -> Result { - let mut root = parse_yaml_object(Some(existing), "Hermes config")?; - let owned = owned_install_command(&root, relay, None)?; - strip_owned_hooks(&mut root, owned.as_ref())?; - remove_owned_mcp(&mut root, owned.is_some())?; - let commands = crate::hooks::transparent_hook_forward_commands( - relay, - crate::agents::CodingAgent::Hermes, - gateway_url, - ) - .map_err(CliError::Install)?; - let mut root = merge_hooks( - root, - generated_policy_hooks(crate::agents::CodingAgent::Hermes, &commands), - )?; - let object = root - .as_object_mut() - .ok_or_else(|| CliError::Launch("Hermes config must be a YAML mapping".into()))?; - let mut model = match object.remove("model") { - Some(Value::Object(model)) => model, - Some(Value::String(default)) => { - Map::from_iter([("default".into(), Value::String(default))]) - } - Some(Value::Null) | None => Map::new(), - Some(_) => { - return Err(CliError::Launch( - "Hermes model config must be a string or mapping".into(), - )); - } - }; - model.insert("provider".into(), Value::String("custom".into())); - model.insert( - "base_url".into(), - Value::String(format!("{}/v1", gateway_url.trim_end_matches('/'))), - ); - model.insert( - "api_key".into(), - Value::String(format!( - "${{{}}}", - crate::provider_auth::TRANSPARENT_PROXY_CREDENTIAL_ENV - )), - ); - object.insert("model".into(), Value::Object(model)); - serde_yaml::to_string(&root).map_err(|error| CliError::Install(error.to_string())) -} - -pub(crate) fn persistent_hook_commands( - relay: &Path, - generation: &Path, - generation_token: &str, -) -> Result { - crate::hooks::persistent_hook_forward_commands( - relay, - crate::agents::CodingAgent::Hermes, - generation, - generation_token, - ) -} - -#[cfg(test)] -pub(super) fn persistent_hook_commands_for_platform( - relay: &Path, - generation: &Path, - generation_token: &str, - windows: bool, -) -> GeneratedHookCommands { - crate::hooks::persistent_hook_forward_commands_for_platform( - relay, - crate::agents::CodingAgent::Hermes, - generation, - generation_token, - windows, - ) -} - -pub(super) fn persistent_config( - existing: Option<&str>, - relay: &Path, - commands: &GeneratedHookCommands, - generation: &Path, - generation_token: &str, - environment: &[String], -) -> Result { - let mut root = parse_yaml_object(existing, "Hermes config")?; - let owned = owned_install_command(&root, relay, Some(generation))?; - if root - .pointer(&format!("/mcp_servers/{MCP_SERVER_NAME}")) - .is_some() - && owned.is_none() - { - return Err(CliError::Install(format!( - "Hermes MCP server `{MCP_SERVER_NAME}` already exists and is not managed by Relay; rename or remove it before installing the Relay integration" - ))); - } - strip_owned_hooks(&mut root, owned.as_ref())?; - root = merge_hooks( - root, - generated_policy_hooks(crate::agents::CodingAgent::Hermes, commands), - )?; - let servers = object_field_mut(&mut root, "mcp_servers", "mcp_servers")?; - servers.insert( - MCP_SERVER_NAME.into(), - expected_mcp_server(relay, generation, generation_token, environment), - ); - Ok(root) -} - -pub(super) fn expected_mcp_server( - relay: &Path, - generation: &Path, - generation_token: &str, - environment: &[String], -) -> Value { - let mut server = crate::mcp::persistent_server(relay, generation, generation_token); - let forwarded = server - .get_mut("env") - .and_then(Value::as_object_mut) - .expect("persistent MCP server environment is an object"); - for name in environment { - forwarded.insert(name.clone(), json!(format!("${{{name}}}"))); - } - server -} - -pub(super) fn forwarded_environment_names( - environment: &[String], - plugin_config: Option<&Value>, -) -> Vec { - crate::mcp_environment::forwarded_names(environment.iter().cloned(), plugin_config) -} - -pub(super) fn strip_owned_hooks( - root: &mut Value, - owned_commands: Option<&GeneratedHookCommands>, -) -> Result<(), CliError> { - let Some(hooks) = root.get_mut("hooks") else { - return Ok(()); - }; - let remove_hooks = { - let hooks = hooks - .as_object_mut() - .ok_or_else(|| CliError::Install("Hermes hooks must be an object".into()))?; - let mut empty = Vec::new(); - for (event, groups) in hooks.iter_mut() { - let groups = groups.as_array_mut().ok_or_else(|| { - CliError::Install(format!("Hermes {event} hooks must be an array")) - })?; - groups.retain(|group| { - group - .get("command") - .and_then(Value::as_str) - .is_none_or(|command| { - owned_commands.is_none_or(|commands| !commands.contains(command)) - }) - }); - if groups.is_empty() { - empty.push(event.clone()); - } - } - for event in empty { - hooks.remove(&event); - } - hooks.is_empty() - }; - if remove_hooks { - root.as_object_mut() - .expect("Hermes config root checked as object") - .remove("hooks"); - } - Ok(()) -} - -pub(super) fn remove_owned_mcp(root: &mut Value, owned: bool) -> Result<(), CliError> { - let Some(servers) = root.get_mut("mcp_servers") else { - return Ok(()); - }; - let servers = servers - .as_object_mut() - .ok_or_else(|| CliError::Install("Hermes mcp_servers must be an object".into()))?; - if owned { - servers.remove(MCP_SERVER_NAME); - } - if servers.is_empty() { - root.as_object_mut() - .expect("Hermes config root checked as object") - .remove("mcp_servers"); - } - Ok(()) -} - -pub(super) fn owned_install_command( - root: &Value, - relay: &Path, - expected_generation: Option<&Path>, -) -> Result, CliError> { - let Some(server) = root.pointer(&format!("/mcp_servers/{MCP_SERVER_NAME}")) else { - return Ok(None); - }; - if server.get("command") != Some(&json!(relay)) { - return Ok(None); - } - let env = server.get("env").and_then(Value::as_object); - if server.get("args") == Some(&json!(["mcp"])) - && env.and_then(|env| env.get("NEMO_RELAY_GATEWAY_BIND")) - == Some(&json!(crate::bootstrap::DEFAULT_BIND)) - { - let generation = env - .and_then(|env| env.get(crate::installation::generation::GENERATION_FILE_ENV)) - .and_then(Value::as_str); - let token = env - .and_then(|env| env.get(crate::installation::generation::GENERATION_TOKEN_ENV)) - .and_then(Value::as_str); - if let (Some(generation), Some(token)) = (generation, token) - && !token.is_empty() - && expected_generation.is_none_or(|expected| Path::new(generation) == expected) - { - let commands = persistent_hook_commands(relay, Path::new(generation), token) - .map_err(CliError::Install)?; - return Ok(Some(commands)); - } - } - legacy_owned_command(root, relay) -} - -fn legacy_owned_command( - root: &Value, - relay: &Path, -) -> Result, CliError> { - let server = &root["mcp_servers"][MCP_SERVER_NAME]; - if server.get("args") != Some(&json!(["mcp", "--agent", "hermes"])) { - return Ok(None); - } - let Some(hooks) = root.get("hooks").and_then(Value::as_object) else { - return Ok(None); - }; - let mut common = None; - for event in crate::agents::CodingAgent::Hermes.hook_events() { - let commands = hooks - .get(*event) - .and_then(Value::as_array) - .into_iter() - .flatten() - .filter_map(|entry| entry.get("command").and_then(Value::as_str)) - .filter(|command| legacy_command_uses_relay(command, relay)) - .collect::>(); - if commands.len() != 1 || common.is_some_and(|value| value != commands[0]) { - return Ok(None); - } - common = Some(commands[0]); - } - Ok(common.map(GeneratedHookCommands::uniform)) -} - -fn legacy_command_uses_relay(command: &str, relay: &Path) -> bool { - let relay = relay.to_string_lossy(); - let quoted = crate::agents::shell_quote_arg_for_platform(&relay, cfg!(windows)); - [relay.as_ref(), quoted.as_str()].into_iter().any(|prefix| { - command.strip_prefix(prefix).is_some_and(|arguments| { - [" hook-forward hermes", " plugin-shim hook hermes"] - .iter() - .any(|marker| arguments.starts_with(marker)) - }) - }) -} - -pub(super) fn relay_is_executable(path: &Path) -> bool { - if !path.is_file() { - return false; - } - #[cfg(unix)] - { - use std::os::unix::fs::PermissionsExt; - std::fs::metadata(path) - .map(|metadata| metadata.permissions().mode() & 0o111 != 0) - .unwrap_or(false) - } - #[cfg(not(unix))] - { - true - } -} - -pub(super) fn parse_yaml_object(raw: Option<&str>, description: &str) -> Result { - let value = match raw.filter(|raw| !raw.trim().is_empty()) { - Some(raw) => serde_yaml::from_str(raw) - .map_err(|error| CliError::Install(format!("invalid {description}: {error}")))?, - None => json!({}), - }; - if value.is_object() { - Ok(value) - } else { - Err(CliError::Install(format!( - "{description} must contain an object" - ))) - } -} - -pub(super) fn yaml_bytes(value: &Value) -> Result, CliError> { - serde_yaml::to_string(value) - .map(String::into_bytes) - .map_err(|error| CliError::Install(error.to_string())) -} - -fn object_field_mut<'a>( - root: &'a mut Value, - field: &str, - description: &str, -) -> Result<&'a mut Map, CliError> { - root.as_object_mut() - .expect("config root checked as object") - .entry(field) - .or_insert_with(|| json!({})) - .as_object_mut() - .ok_or_else(|| CliError::Install(format!("Hermes {description} must be an object"))) -} diff --git a/crates/cli/src/agents/hermes/doctor.rs b/crates/cli/src/agents/hermes/doctor.rs deleted file mode 100644 index 0f8b19509..000000000 --- a/crates/cli/src/agents/hermes/doctor.rs +++ /dev/null @@ -1,13 +0,0 @@ -// SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. -// SPDX-License-Identifier: Apache-2.0 - -use std::path::Path; - -pub(crate) fn hook_status(hooks_path: Option<&Path>) -> Result { - match hooks_path { - Some(path) => super::diagnose_persistent(path).map_err(|error| { - format!("persistent MCP/hooks: {error}; run `nemo-relay install hermes --force`") - }), - None => Ok("hooks: injected through an isolated HERMES_HOME during run".into()), - } -} diff --git a/crates/cli/src/agents/hermes/files.rs b/crates/cli/src/agents/hermes/files.rs deleted file mode 100644 index 7cbdfd2c5..000000000 --- a/crates/cli/src/agents/hermes/files.rs +++ /dev/null @@ -1,207 +0,0 @@ -// SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. -// SPDX-License-Identifier: Apache-2.0 - -//! Serialized, rollback-capable filesystem operations for the Hermes integration. - -use std::fs::{self, File, OpenOptions}; -use std::path::{Path, PathBuf}; -use std::thread; -use std::time::{Duration, Instant}; - -use crate::error::CliError; -use crate::filesystem::{LockAttempt, try_lock_exclusive}; -use crate::installation::generation::GENERATION_FILE_NAME; - -const ALLOWLIST_FILE_NAME: &str = "shell-hooks-allowlist.json"; -const INSTALL_LOCK_FILE_NAME: &str = ".nemo-relay-operation.lock"; -const INSTALL_LOCK_RETRY: Duration = Duration::from_millis(25); -pub(super) const INSTALL_LOCK_TIMEOUT: Duration = Duration::from_secs(5); - -#[derive(Clone, Debug, PartialEq, Eq)] -pub(super) struct PersistentPaths { - pub(super) config: PathBuf, - pub(super) allowlist: PathBuf, - pub(super) generation: PathBuf, -} - -impl PersistentPaths { - pub(super) fn for_config(config: PathBuf) -> Result { - let home = config.parent().ok_or_else(|| { - CliError::Install(format!( - "Hermes config path {} has no parent directory", - config.display() - )) - })?; - Ok(Self { - allowlist: home.join(ALLOWLIST_FILE_NAME), - generation: home.join(GENERATION_FILE_NAME), - config, - }) - } - - pub(super) fn all(&self) -> [PathBuf; 3] { - [ - self.config.clone(), - self.allowlist.clone(), - self.generation.clone(), - ] - } -} - -pub(super) fn acquire_install_lock(config: &Path, timeout: Duration) -> Result { - let home = config.parent().ok_or_else(|| { - format!( - "Hermes config path {} has no parent directory", - config.display() - ) - })?; - acquire_lock_file( - &home.join(INSTALL_LOCK_FILE_NAME), - timeout, - "another Hermes integration update", - ) -} - -/// Uses Hermes's own sibling allowlist lock so Relay cannot lose an unrelated approval that -/// Hermes records concurrently. -pub(super) fn acquire_allowlist_lock(allowlist: &Path, timeout: Duration) -> Result { - let mut lock = allowlist.as_os_str().to_os_string(); - lock.push(".lock"); - acquire_lock_file( - &PathBuf::from(lock), - timeout, - "a Hermes shell-hook approval update", - ) -} - -fn acquire_lock_file(path: &Path, timeout: Duration, contention: &str) -> Result { - if let Some(parent) = path.parent() { - fs::create_dir_all(parent) - .map_err(|error| format!("failed to create {}: {error}", parent.display()))?; - } - let mut options = OpenOptions::new(); - options.create(true).truncate(false).read(true).write(true); - #[cfg(unix)] - { - use std::os::unix::fs::OpenOptionsExt; - options.mode(0o600); - } - let file = options.open(path).map_err(|error| { - format!( - "failed to open Hermes install lock {}: {error}", - path.display() - ) - })?; - let deadline = Instant::now() + timeout; - loop { - match try_lock_exclusive(&file) { - Ok(LockAttempt::Acquired) => return Ok(file), - Ok(LockAttempt::Contended) if Instant::now() < deadline => { - thread::sleep( - INSTALL_LOCK_RETRY.min(deadline.saturating_duration_since(Instant::now())), - ); - } - Ok(LockAttempt::Contended) => { - return Err(format!( - "timed out waiting for {contention} at {}; wait for it to finish and retry", - path.display() - )); - } - Err(error) => { - return Err(format!( - "failed to lock Hermes integration state {}: {error}", - path.display() - )); - } - } - } -} - -pub(super) fn read_optional_utf8(path: &Path) -> Result, CliError> { - match fs::read_to_string(path) { - Ok(raw) => Ok(Some(raw)), - Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(None), - Err(error) => Err(CliError::Install(format!( - "failed to read {}: {error}", - path.display() - ))), - } -} - -pub(super) fn replace_optional_file( - path: &Path, - bytes: Option<&[u8]>, - write: &mut W, -) -> Result<(), String> -where - W: FnMut(&Path, &[u8]) -> Result<(), String>, -{ - match bytes { - Some(bytes) => write(path, bytes), - None => remove_optional_file(path), - } -} - -pub(super) fn remove_optional_file(path: &Path) -> Result<(), String> { - match fs::remove_file(path) { - Ok(()) => Ok(()), - Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(()), - Err(error) => Err(format!("failed to remove {}: {error}", path.display())), - } -} - -pub(super) struct FileSnapshot { - path: PathBuf, - bytes: Option>, - permissions: Option, -} - -impl FileSnapshot { - pub(super) fn capture(path: &Path) -> Result { - match fs::read(path) { - Ok(bytes) => { - let permissions = fs::metadata(path) - .map(|metadata| metadata.permissions()) - .map_err(|error| { - CliError::Install(format!( - "failed to snapshot permissions on {}: {error}", - path.display() - )) - })?; - Ok(Self { - path: path.to_path_buf(), - bytes: Some(bytes), - permissions: Some(permissions), - }) - } - Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(Self { - path: path.to_path_buf(), - bytes: None, - permissions: None, - }), - Err(error) => Err(CliError::Install(format!( - "failed to snapshot {}: {error}", - path.display() - ))), - } - } - - pub(super) fn restore(&self, write: &mut W) -> Result<(), String> - where - W: FnMut(&Path, &[u8]) -> Result<(), String>, - { - if let Some(bytes) = self.bytes.as_deref() { - write(&self.path, bytes)?; - if let Some(permissions) = self.permissions.as_ref() { - fs::set_permissions(&self.path, permissions.clone()).map_err(|error| { - format!( - "failed to restore permissions on {}: {error}", - self.path.display() - ) - })?; - } - return Ok(()); - } - remove_optional_file(&self.path) - } -} diff --git a/crates/cli/src/agents/hermes/install.rs b/crates/cli/src/agents/hermes/install.rs deleted file mode 100644 index d656dab3f..000000000 --- a/crates/cli/src/agents/hermes/install.rs +++ /dev/null @@ -1,191 +0,0 @@ -// SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. -// SPDX-License-Identifier: Apache-2.0 - -use std::path::PathBuf; -use std::process::ExitCode; - -use crate::agents::CodingAgent; -use crate::error::CliError; -use crate::installation::marketplace::HostPluginReadiness; -use crate::installation::marketplace::host::{ - CommandRunner, RealCommandRunner, require_host_cli, require_relay, validate_host_version, - validate_relay_hook_forward, validate_relay_mcp, -}; -use crate::installation::marketplace::state::PluginInstallOptions; -use crate::installation::{InstallRequest, UninstallRequest}; - -pub(crate) fn install(command: InstallRequest) -> Result { - let options = options(command.dry_run, command.skip_doctor, command.force); - let runner = RealCommandRunner; - require_host_cli(CodingAgent::Hermes, &options, &runner).map_err(CliError::Install)?; - validate_host_version(CodingAgent::Hermes, &options, &runner).map_err(CliError::Install)?; - let relay = require_relay(&options, &runner).map_err(CliError::Install)?; - validate_relay_hook_forward(&relay, &options, &runner).map_err(CliError::Install)?; - validate_relay_mcp(&relay, &options, &runner).map_err(CliError::Install)?; - let config = config_path().map_err(CliError::Install)?; - if options.dry_run { - println!("configure Hermes MCP and hooks at {}", config.display()); - return Ok(ExitCode::SUCCESS); - } - super::install_persistent(&config, &relay) - .map_err(|error| CliError::Install(error.to_string()))?; - if !options.skip_doctor { - super::diagnose_persistent(&config).map_err(CliError::Install)?; - } - println!("installed Hermes integration"); - Ok(ExitCode::SUCCESS) -} - -pub(crate) fn uninstall(command: UninstallRequest) -> Result { - let config = config_path().map_err(CliError::Install)?; - if command.dry_run { - println!( - "remove Relay-owned Hermes MCP and hooks from {}", - config.display() - ); - return Ok(ExitCode::SUCCESS); - } - super::uninstall_persistent(&config).map_err(|error| CliError::Install(error.to_string()))?; - println!("uninstalled Hermes integration"); - Ok(ExitCode::SUCCESS) -} - -pub(crate) fn config_path() -> Result { - std::env::var_os("HOME") - .or_else(|| std::env::var_os("USERPROFILE")) - .map(PathBuf::from) - .map(|home| super::user_config_path(&home)) - .ok_or_else(|| "cannot determine home directory (set HOME or USERPROFILE)".into()) -} - -fn options(dry_run: bool, skip_doctor: bool, force: bool) -> PluginInstallOptions { - PluginInstallOptions { - install_dir: PathBuf::new(), - operation_lock_dir: PathBuf::new(), - force, - dry_run, - skip_doctor, - } -} - -pub(crate) fn doctor( - options: &PluginInstallOptions, - runner: &dyn CommandRunner, -) -> Result<(), String> { - let report = doctor_json_value(options, runner)?; - for check in report["readiness_checks"] - .as_array() - .expect("Hermes readiness checks are an array") - { - println!( - "{}: {} ({})", - check["name"].as_str().unwrap_or_default(), - if check["ok"] == serde_json::json!(true) { - "ok" - } else { - "failed" - }, - check["details"].as_str().unwrap_or_default() - ); - } - (report["ok"] == serde_json::json!(true)) - .then_some(()) - .ok_or_else(|| { - format!( - "Hermes integration doctor checks failed; remediation: {}", - report["remediation"].as_str().unwrap_or_default() - ) - }) -} - -pub(crate) fn doctor_json_value( - options: &PluginInstallOptions, - runner: &dyn CommandRunner, -) -> Result { - let config = config_path()?; - let readiness = collect_readiness(&config, options, runner); - Ok(serde_json::json!({ - "ok": readiness.ok(), - "host": readiness.host, - "remediation": readiness.remediation, - "config": config, - "readiness_checks": readiness.checks - })) -} - -pub(crate) fn collect_readiness( - config: &std::path::Path, - options: &PluginInstallOptions, - runner: &dyn CommandRunner, -) -> HostPluginReadiness { - let mut readiness = HostPluginReadiness { - host: CodingAgent::Hermes.install_arg().into(), - remediation: format!( - "nemo-relay install {} --force", - CodingAgent::Hermes.install_arg() - ), - state_path: config.to_path_buf(), - marketplace: None, - plugin: None, - checks: Vec::new(), - relay: None, - host_plugin_registered: None, - host_marketplace_registered: None, - plugin_setup: None, - }; - - let host_cli = require_host_cli(CodingAgent::Hermes, options, runner); - readiness.push( - "Host CLI", - host_cli - .as_ref() - .map(|_| "hermes is available".into()) - .map_err(Clone::clone), - ); - let version = validate_host_version(CodingAgent::Hermes, options, runner); - if version.is_err() { - readiness.remediation = format!( - "upgrade to {}, then run `nemo-relay install {} --force`", - CodingAgent::Hermes.version_requirement(), - CodingAgent::Hermes.install_arg() - ); - } - readiness.push( - "Hermes Agent version", - version.map(|_| format!("{} is installed", CodingAgent::Hermes.version_requirement())), - ); - - let relay = super::configured_relay_executable(config); - readiness.push( - "Configured Relay binary", - relay - .as_ref() - .map(|path| format!("found at {}", path.display())) - .map_err(Clone::clone), - ); - match relay { - Ok(relay) => { - readiness.relay = Some(relay.clone()); - readiness.push( - "Relay hook support", - validate_relay_hook_forward(&relay, options, runner) - .map(|_| "hook-forward is supported".into()), - ); - readiness.push( - "Relay MCP support", - validate_relay_mcp(&relay, options, runner) - .map(|_| "native mcp subcommand is supported".into()), - ); - } - Err(error) => { - let unavailable = || format!("cannot verify configured Relay capabilities: {error}"); - readiness.push("Relay hook support", Err(unavailable())); - readiness.push("Relay MCP support", Err(unavailable())); - } - } - readiness.push( - "Hermes MCP, hooks, and trust", - super::diagnose_persistent(config), - ); - readiness -} diff --git a/crates/cli/src/agents/hermes/integration.rs b/crates/cli/src/agents/hermes/integration.rs deleted file mode 100644 index fbba5a6b3..000000000 --- a/crates/cli/src/agents/hermes/integration.rs +++ /dev/null @@ -1,634 +0,0 @@ -// SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. -// SPDX-License-Identifier: Apache-2.0 - -//! Hermes-owned MCP and lifecycle-hook configuration. - -use std::env; -use std::fs; -use std::path::{Path, PathBuf}; -use std::time::SystemTime; - -use serde_json::{Map, Value, json}; - -#[cfg(test)] -use super::config::persistent_hook_commands_for_platform; -use super::config::{ - MCP_SERVER_NAME, expected_mcp_server, forwarded_environment_names, owned_install_command, - parse_yaml_object, persistent_config, relay_is_executable, remove_owned_mcp, strip_owned_hooks, - user_config_path_with_override, yaml_bytes, -}; -pub(crate) use super::config::{persistent_hook_commands, transparent_config}; -use super::files::{ - FileSnapshot, INSTALL_LOCK_TIMEOUT, PersistentPaths, acquire_allowlist_lock, - acquire_install_lock, read_optional_utf8, remove_optional_file, replace_optional_file, -}; -use super::trust::{json_bytes, parse_json_object, trusted_hooks, verify_trust}; -use crate::agents::CodingAgent; -use crate::bootstrap::DEFAULT_BIND; -use crate::error::CliError; -use crate::filesystem::atomic_write; -use crate::hooks::GeneratedHookCommands; -#[cfg(test)] -use crate::installation::generation::GENERATION_FILE_NAME; -use crate::installation::generation::{ - GENERATION_FILE_ENV, GENERATION_TOKEN_ENV, GenerationRetirement, InstallGeneration, -}; - -/// Hermes host configuration is user-owned. -pub(crate) fn user_config_path(default_home: &Path) -> PathBuf { - user_config_path_with_override(default_home, env::var_os("HERMES_HOME")) -} - -pub(crate) fn install_persistent(config: &Path, relay: &Path) -> Result, CliError> { - let relay = relay.canonicalize().unwrap_or_else(|_| relay.to_path_buf()); - let relay = crate::agents::portable_executable_path(relay); - if !relay_is_executable(&relay) { - return Err(CliError::Install(format!( - "nemo-relay executable is missing or not executable at {}", - relay.display() - ))); - } - let paths = PersistentPaths::for_config(config.to_path_buf())?; - let _lock = - acquire_install_lock(&paths.config, INSTALL_LOCK_TIMEOUT).map_err(CliError::Install)?; - let _allowlist_lock = acquire_allowlist_lock(&paths.allowlist, INSTALL_LOCK_TIMEOUT) - .map_err(CliError::Install)?; - let plugin_config = crate::configuration::user_plugin_runtime_config()?; - let environment = env::vars_os() - .filter_map(|(name, _)| name.into_string().ok()) - .collect::>(); - let mut retirement = retire_generation_before_gateway_stop(&paths)?; - let result = install_persistent_with_generation( - paths, - &relay, - &environment, - plugin_config.as_ref(), - retirement.as_ref(), - SystemTime::now(), - atomic_write, - ); - finish_generation_mutation(result, retirement.as_mut(), "install") -} - -pub(crate) fn persistent_state_exists(config: &Path) -> bool { - PersistentPaths::for_config(config.to_path_buf()) - .ok() - .and_then(|paths| persistent_paths_have_managed_state(&paths).ok()) - .unwrap_or(false) -} - -pub(crate) fn uninstall_persistent(config: &Path) -> Result, CliError> { - let paths = PersistentPaths::for_config(config.to_path_buf())?; - if !persistent_paths_have_managed_state(&paths)? { - return Ok(Vec::new()); - } - let _lock = - acquire_install_lock(&paths.config, INSTALL_LOCK_TIMEOUT).map_err(CliError::Install)?; - let _allowlist_lock = acquire_allowlist_lock(&paths.allowlist, INSTALL_LOCK_TIMEOUT) - .map_err(CliError::Install)?; - if !persistent_paths_have_managed_state(&paths)? { - return Ok(Vec::new()); - } - let mut retirement = retire_generation_before_gateway_stop(&paths)?; - let result = uninstall_persistent_with(paths, atomic_write); - finish_generation_mutation(result, retirement.as_mut(), "uninstall") -} - -fn retire_generation_before_gateway_stop( - paths: &PersistentPaths, -) -> Result, CliError> { - let mut retirement = - GenerationRetirement::acquire(&paths.generation).map_err(CliError::Install)?; - if let Some(retirement) = retirement.as_mut() { - retirement - .invalidate_for_replacement() - .map_err(CliError::Install)?; - } - if let Err(error) = crate::agents::stop_plugin_gateway() { - if let Some(retirement) = retirement.as_mut() - && let Err(restore_error) = retirement.restore_after_rollback() - { - return Err(CliError::Install(format!( - "{error}; additionally failed to restore the Hermes MCP generation: {restore_error}" - ))); - } - return Err(CliError::Install(error)); - } - Ok(retirement) -} - -fn finish_generation_mutation( - result: Result, - retirement: Option<&mut GenerationRetirement>, - operation: &str, -) -> Result { - match result { - Ok(value) => { - if let Some(retirement) = retirement { - retirement.commit_replacement(); - } - Ok(value) - } - Err(error) => { - let Some(retirement) = retirement else { - return Err(error); - }; - match retirement.restore_after_rollback() { - Ok(()) => Err(error), - Err(restore_error) => Err(CliError::Install(format!( - "{error}; additionally failed to restore the Hermes MCP generation after {operation}: {restore_error}" - ))), - } - } - } -} - -fn persistent_paths_have_managed_state(paths: &PersistentPaths) -> Result { - if paths.generation.exists() { - return Ok(true); - } - if let Some(raw) = read_optional_utf8(&paths.config)? { - let config = parse_yaml_object(Some(&raw), "Hermes config")?; - if config_has_managed_state(&config) { - return Ok(true); - } - } - if let Some(raw) = read_optional_utf8(&paths.allowlist)? { - let allowlist = parse_json_object(Some(&raw), "Hermes shell-hook allowlist")?; - if allowlist_has_owned_command(&allowlist, None) { - return Ok(true); - } - } - Ok(false) -} - -fn config_has_managed_state(config: &Value) -> bool { - owned_command_from_config(config, None).is_some() -} - -fn allowlist_has_owned_command( - allowlist: &Value, - commands: Option<&GeneratedHookCommands>, -) -> bool { - allowlist - .get("approvals") - .and_then(Value::as_array) - .into_iter() - .flatten() - .filter_map(|entry| entry.get("command").and_then(Value::as_str)) - .any(|candidate| { - commands.is_some_and(|commands| commands.contains(candidate)) - || (commands.is_none() && is_persistent_relay_hook_command(candidate)) - }) -} - -fn is_persistent_relay_hook_command(command: &str) -> bool { - #[cfg(any(windows, test))] - if let Some(arguments) = crate::hooks::decode_windows_hook_command(command) { - let arguments = arguments.as_slice(); - let base = match arguments { - [base @ .., policy] if matches!(policy.as_str(), "--fail-open" | "--fail-closed") => { - base - } - base => base, - }; - return matches!( - base, - [ - _, - hook_forward, - agent, - gateway_flag, - gateway_url, - generation_file_flag, - _, - generation_token_flag, - generation_token, - ] if hook_forward == "hook-forward" - && agent == "hermes" - && gateway_flag == "--gateway-url" - && gateway_url == crate::bootstrap::DEFAULT_URL - && generation_file_flag == "--generation-file" - && generation_token_flag == "--generation-token" - && !generation_token.is_empty() - ); - } - command.contains("hook-forward") - && command.contains("hermes") - && command.contains("--gateway-url") - && command.contains(crate::bootstrap::DEFAULT_URL) - && command.contains("--generation-file") - && command.contains("--generation-token") -} - -fn owned_command_from_config( - config: &Value, - generation: Option<&Path>, -) -> Option { - let relay = config - .pointer(&format!("/mcp_servers/{MCP_SERVER_NAME}/command")) - .and_then(Value::as_str) - .map(PathBuf::from)?; - owned_install_command(config, &relay, generation) - .ok() - .flatten() -} - -pub(crate) fn diagnose_persistent(config_path: &Path) -> Result { - let paths = PersistentPaths::for_config(config_path.to_path_buf()) - .map_err(|error| error.to_string())?; - let raw = fs::read_to_string(&paths.config) - .map_err(|error| format!("failed to read {}: {error}", paths.config.display()))?; - let config = parse_yaml_object(Some(&raw), "Hermes config").map_err(|e| e.to_string())?; - let relay = relay_executable_from_config(&config)?; - if !relay_is_executable(&relay) { - return Err(format!( - "configured nemo-relay executable is missing or not executable at {}", - relay.display() - )); - } - let generation = InstallGeneration::capture(paths.generation.clone())?; - let commands = persistent_hook_commands(&relay, &paths.generation, generation.token())?; - verify_hook_definitions(&config, &commands)?; - verify_trust(&paths.allowlist, &commands)?; - - let mcp_env = config["mcp_servers"][MCP_SERVER_NAME] - .get("env") - .and_then(Value::as_object) - .ok_or_else(|| "Hermes Relay MCP environment is missing".to_string())?; - if mcp_env.get("NEMO_RELAY_GATEWAY_BIND") != Some(&json!(DEFAULT_BIND)) { - return Err(format!( - "Hermes Relay MCP must use the shared gateway bind {DEFAULT_BIND}" - )); - } - let configured_generation = mcp_env - .get(GENERATION_FILE_ENV) - .and_then(Value::as_str) - .ok_or_else(|| "Hermes Relay MCP generation fence is missing".to_string())?; - if Path::new(configured_generation) != paths.generation { - return Err("Hermes Relay MCP generation fence points at the wrong file".into()); - } - let configured_token = mcp_env - .get(GENERATION_TOKEN_ENV) - .and_then(Value::as_str) - .ok_or_else(|| "Hermes Relay MCP expected generation identity is missing".to_string())?; - if configured_token != generation.token() { - return Err("Hermes Relay MCP expected generation identity is stale".into()); - } - - let plugin_config = - crate::configuration::user_plugin_runtime_config().map_err(|e| e.to_string())?; - let environment = env::vars_os() - .filter_map(|(name, _)| name.into_string().ok()) - .collect::>(); - let environment = forwarded_environment_names(&environment, plugin_config.as_ref()); - let expected = expected_mcp_server(&relay, &paths.generation, generation.token(), &environment); - let expected_env = expected - .get("env") - .and_then(Value::as_object) - .expect("expected MCP environment is an object"); - let missing = environment - .into_iter() - .filter(|name| mcp_env.get(name) != expected_env.get(name)) - .collect::>(); - if !missing.is_empty() { - return Err(format!( - "Hermes Relay MCP is missing environment names {}; run `nemo-relay install hermes --force`", - missing.join(", ") - )); - } - Ok(format!( - "MCP lifecycle and {} hooks trusted at {}", - CodingAgent::Hermes.hook_events().len(), - paths.config.display() - )) -} - -/// Returns the exact Relay binary configured for Hermes's managed MCP client. -/// -/// Doctor uses this path instead of the currently running binary so it verifies the executable -/// that Hermes will actually launch. -pub(crate) fn configured_relay_executable(config_path: &Path) -> Result { - let raw = fs::read_to_string(config_path) - .map_err(|error| format!("failed to read {}: {error}", config_path.display()))?; - let config = parse_yaml_object(Some(&raw), "Hermes config").map_err(|e| e.to_string())?; - let relay = relay_executable_from_config(&config)?; - if !relay_is_executable(&relay) { - return Err(format!( - "configured nemo-relay executable is missing or not executable at {}", - relay.display() - )); - } - Ok(relay) -} - -fn relay_executable_from_config(config: &Value) -> Result { - let server = config - .get("mcp_servers") - .and_then(|servers| servers.get(MCP_SERVER_NAME)) - .ok_or_else(|| format!("Hermes MCP server `{MCP_SERVER_NAME}` is missing"))?; - let relay = PathBuf::from( - server - .get("command") - .and_then(Value::as_str) - .ok_or_else(|| "Hermes Relay MCP command is missing".to_string())?, - ); - if owned_install_command(config, &relay, None) - .map_err(|error| error.to_string())? - .is_none() - { - return Err(format!( - "Hermes MCP server `{MCP_SERVER_NAME}` is not a managed Relay MCP client" - )); - } - Ok(relay) -} - -#[cfg(test)] -fn install_persistent_with( - paths: PersistentPaths, - relay: &Path, - environment: &[String], - plugin_config: Option<&Value>, - now: SystemTime, - write: W, -) -> Result, CliError> -where - W: FnMut(&Path, &[u8]) -> Result<(), String>, -{ - install_persistent_with_generation(paths, relay, environment, plugin_config, None, now, write) -} - -fn install_persistent_with_generation( - paths: PersistentPaths, - relay: &Path, - environment: &[String], - plugin_config: Option<&Value>, - generation_transaction: Option<&GenerationRetirement>, - now: SystemTime, - mut write: W, -) -> Result, CliError> -where - W: FnMut(&Path, &[u8]) -> Result<(), String>, -{ - let snapshots = paths - .all() - .iter() - .map(|path| FileSnapshot::capture(path)) - .collect::, _>>()?; - let existing_config = read_optional_utf8(&paths.config)?; - let existing_allowlist = read_optional_utf8(&paths.allowlist)?; - let previous_command = match existing_config.as_deref() { - Some(raw) => { - let root = parse_yaml_object(Some(raw), "Hermes config")?; - owned_install_command(&root, relay, Some(&paths.generation))? - } - None => None, - }; - let environment = forwarded_environment_names(environment, plugin_config); - let token = uuid::Uuid::now_v7().to_string(); - let commands = - persistent_hook_commands(relay, &paths.generation, &token).map_err(CliError::Install)?; - let config = persistent_config( - existing_config.as_deref(), - relay, - &commands, - &paths.generation, - &token, - &environment, - )?; - let allowlist = trusted_hooks( - existing_allowlist.as_deref(), - previous_command.as_ref(), - &commands, - relay, - now, - )?; - let config = yaml_bytes(&config)?; - let allowlist = json_bytes(&allowlist)?; - let generation = format!("{token}\n").into_bytes(); - - let result = (|| { - // Trust is published before config so Hermes never observes a configured hook without - // its exact approval. The config write is the transaction's commit point. - write(&paths.generation, &generation)?; - write(&paths.allowlist, &allowlist)?; - write(&paths.config, &config)?; - verify_install( - &paths, - relay, - &commands, - &environment, - &token, - generation_transaction, - ) - })(); - if let Err(error) = result { - return rollback_error("install", error, &snapshots, &mut write); - } - Ok(paths.all().into_iter().collect()) -} - -fn uninstall_persistent_with( - paths: PersistentPaths, - mut write: W, -) -> Result, CliError> -where - W: FnMut(&Path, &[u8]) -> Result<(), String>, -{ - let affected = paths - .all() - .into_iter() - .filter(|path| path.exists()) - .collect::>(); - let snapshots = paths - .all() - .iter() - .map(|path| FileSnapshot::capture(path)) - .collect::, _>>()?; - let config = read_optional_utf8(&paths.config)? - .map(|raw| { - let mut root = parse_yaml_object(Some(&raw), "Hermes config")?; - let owned = owned_command_from_config(&root, Some(&paths.generation)); - strip_owned_hooks(&mut root, owned.as_ref())?; - remove_owned_mcp(&mut root, owned.is_some())?; - if root.as_object().is_some_and(Map::is_empty) { - Ok(None) - } else { - yaml_bytes(&root).map(Some) - } - }) - .transpose()? - .flatten(); - let owned = read_optional_utf8(&paths.config)? - .and_then(|raw| parse_yaml_object(Some(&raw), "Hermes config").ok()) - .and_then(|root| owned_command_from_config(&root, Some(&paths.generation))); - let allowlist = read_optional_utf8(&paths.allowlist)? - .map(|raw| { - let mut root = parse_json_object(Some(&raw), "Hermes shell-hook allowlist")?; - let object = root - .as_object_mut() - .expect("allowlist root checked as object"); - if let Some(approvals) = object.get_mut("approvals") { - let approvals = approvals.as_array_mut().ok_or_else(|| { - CliError::Install( - "Hermes shell-hook allowlist approvals must be an array".into(), - ) - })?; - approvals.retain(|entry| { - entry - .get("command") - .and_then(Value::as_str) - .is_none_or(|command| { - owned.as_ref().map_or_else( - || !is_persistent_relay_hook_command(command), - |commands| !commands.contains(command), - ) - }) - }); - if approvals.is_empty() { - object.remove("approvals"); - } - } - if object.is_empty() { - Ok(None) - } else { - json_bytes(&root).map(Some) - } - }) - .transpose()? - .flatten(); - - let result = (|| { - remove_optional_file(&paths.generation)?; - replace_optional_file(&paths.allowlist, allowlist.as_deref(), &mut write)?; - replace_optional_file(&paths.config, config.as_deref(), &mut write)?; - verify_uninstall(&paths, owned.as_ref()) - })(); - if let Err(error) = result { - return rollback_error("uninstall", error, &snapshots, &mut write); - } - Ok(affected) -} - -fn rollback_error( - operation: &str, - error: String, - snapshots: &[FileSnapshot], - write: &mut W, -) -> Result -where - W: FnMut(&Path, &[u8]) -> Result<(), String>, -{ - let rollback_errors = snapshots - .iter() - .rev() - .filter_map(|snapshot| snapshot.restore(write).err()) - .collect::>(); - let rollback = if rollback_errors.is_empty() { - String::new() - } else { - format!("; rollback also failed: {}", rollback_errors.join("; ")) - }; - Err(CliError::Install(format!( - "failed to {operation} Hermes MCP integration: {error}{rollback}" - ))) -} - -fn verify_install( - paths: &PersistentPaths, - relay: &Path, - commands: &GeneratedHookCommands, - environment: &[String], - token: &str, - generation_transaction: Option<&GenerationRetirement>, -) -> Result<(), String> { - let raw = fs::read_to_string(&paths.config) - .map_err(|error| format!("failed to verify {}: {error}", paths.config.display()))?; - let config = parse_yaml_object(Some(&raw), "Hermes config").map_err(|e| e.to_string())?; - let expected = expected_mcp_server(relay, &paths.generation, token, environment); - if config.pointer("/mcp_servers/nemo-relay") != Some(&expected) { - return Err("Hermes MCP server did not persist exactly".into()); - } - verify_hook_definitions(&config, commands)?; - verify_trust(&paths.allowlist, commands)?; - - let actual_token = match generation_transaction { - Some(transaction) => transaction.active_visible_token()?, - None => InstallGeneration::capture(paths.generation.clone())? - .token() - .to_owned(), - }; - if actual_token != token { - return Err("Hermes MCP generation did not persist exactly".into()); - } - Ok(()) -} - -fn verify_hook_definitions(config: &Value, commands: &GeneratedHookCommands) -> Result<(), String> { - for event in CodingAgent::Hermes.hook_events() { - let groups = config - .pointer(&format!("/hooks/{event}")) - .and_then(Value::as_array) - .ok_or_else(|| format!("Hermes hook {event} is missing"))?; - let matching = groups - .iter() - .filter(|group| { - group.get("command").and_then(Value::as_str) == Some(commands.for_event(event)) - }) - .count(); - if matching != 1 { - return Err(format!( - "Hermes hook {event} expected exactly one trusted Relay handler" - )); - } - } - for (event, groups) in config - .get("hooks") - .and_then(Value::as_object) - .into_iter() - .flat_map(Map::iter) - { - let groups = groups - .as_array() - .ok_or_else(|| format!("Hermes {event} hooks must be an array"))?; - if !CodingAgent::Hermes.hook_events().contains(&event.as_str()) - && groups.iter().any(|group| { - group - .get("command") - .and_then(Value::as_str) - .is_some_and(|command| commands.contains(command)) - }) - { - return Err("Hermes config contains an unexpected Relay hook handler".into()); - } - } - Ok(()) -} - -fn verify_uninstall( - paths: &PersistentPaths, - owned_commands: Option<&GeneratedHookCommands>, -) -> Result<(), String> { - if paths.generation.exists() { - return Err("Hermes MCP generation fence still exists".into()); - } - if let Some(raw) = read_optional_utf8(&paths.config).map_err(|error| error.to_string())? { - let config = parse_yaml_object(Some(&raw), "Hermes config").map_err(|e| e.to_string())?; - if config_has_managed_state(&config) { - return Err("managed Hermes Relay config still exists".into()); - } - } - if let Some(raw) = read_optional_utf8(&paths.allowlist).map_err(|error| error.to_string())? { - let allowlist = parse_json_object(Some(&raw), "Hermes shell-hook allowlist") - .map_err(|e| e.to_string())?; - if allowlist_has_owned_command(&allowlist, owned_commands) { - return Err("managed Hermes Relay trust approval still exists".into()); - } - } - Ok(()) -} - -#[cfg(test)] -#[path = "../../../tests/coverage/agents/hermes_tests.rs"] -mod tests; diff --git a/crates/cli/src/agents/hermes/launch.rs b/crates/cli/src/agents/hermes/launch.rs deleted file mode 100644 index 077a6d8ad..000000000 --- a/crates/cli/src/agents/hermes/launch.rs +++ /dev/null @@ -1,196 +0,0 @@ -// SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. -// SPDX-License-Identifier: Apache-2.0 - -use std::path::{Path, PathBuf}; - -use crate::error::CliError; -use crate::process::PreparedAgentLaunch; - -pub(crate) fn prepare( - launch: &mut PreparedAgentLaunch, - hooks_path: Option<&Path>, - dry_run: bool, -) -> Result<(), CliError> { - let source_config = hooks_path_for_launch(hooks_path)?; - let gateway_url = launch - .env - .iter() - .find_map(|(name, value)| { - (name == crate::configuration::GATEWAY_URL_ENV).then_some(value.as_str()) - }) - .expect("transparent runs always define their gateway URL") - .to_owned(); - launch.env.push(("HERMES_ACCEPT_HOOKS".into(), "1".into())); - launch.env.push(( - "OPENAI_BASE_URL".into(), - format!("{}/v1", gateway_url.trim_end_matches('/')), - )); - if dry_run { - launch.notes.push(format!( - "would create an isolated Hermes config overlay for {}", - source_config.display() - )); - return Ok(()); - } - let source_home = source_config.parent().ok_or_else(|| { - CliError::Launch(format!( - "Hermes config path {} has no parent directory", - source_config.display() - )) - })?; - let overlay_home = create_overlay(source_home, &source_config, &gateway_url)?; - launch - .env - .push(("HERMES_HOME".into(), overlay_home.display().to_string())); - launch.notes.push(format!( - "using an isolated Hermes config overlay for {}", - source_config.display() - )); - launch.temp_dirs.push(overlay_home); - Ok(()) -} - -fn create_overlay( - source_home: &Path, - source_config: &Path, - gateway_url: &str, -) -> Result { - ensure_durable_state_paths(source_home)?; - let overlay = source_home - .parent() - .filter(|parent| parent.is_dir()) - .and_then(|parent| { - crate::filesystem::temp::private_temp_dir(parent, ".nemo-relay-hermes-home").ok() - }) - .map(Ok) - .unwrap_or_else(|| { - crate::filesystem::temp::private_system_temp_dir("nemo-relay-hermes-home") - })?; - if let Err(error) = populate_overlay(&overlay, source_home, source_config, gateway_url) { - let _ = std::fs::remove_dir_all(&overlay); - return Err(error); - } - Ok(overlay) -} - -fn ensure_durable_state_paths(source_home: &Path) -> Result<(), CliError> { - std::fs::create_dir_all(source_home.join("sessions"))?; - let state_db = source_home.join("state.db"); - match std::fs::OpenOptions::new() - .write(true) - .create_new(true) - .open(&state_db) - { - Ok(_) => Ok(()), - Err(error) if error.kind() == std::io::ErrorKind::AlreadyExists && state_db.is_file() => { - Ok(()) - } - Err(error) => Err(CliError::Io(error)), - } -} - -pub(crate) fn populate_overlay( - overlay: &Path, - source_home: &Path, - source_config: &Path, - gateway_url: &str, -) -> Result<(), CliError> { - let absolute_overlay = overlay - .canonicalize() - .unwrap_or_else(|_| overlay.to_path_buf()); - match std::fs::read_dir(source_home) { - Ok(entries) => { - for entry in entries { - let entry = entry?; - let name = entry.file_name(); - if name == "config.yaml" || name == "shell-hooks-allowlist.json" { - continue; - } - let source = entry.path(); - let absolute_source = source.canonicalize().unwrap_or_else(|_| source.clone()); - if absolute_overlay.starts_with(absolute_source) { - continue; - } - link_state(&source, &overlay.join(name), entry.file_type()?.is_dir())?; - } - } - Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} - Err(error) => return Err(CliError::Io(error)), - } - let existing = match std::fs::read_to_string(source_config) { - Ok(raw) => raw, - Err(error) if error.kind() == std::io::ErrorKind::NotFound => String::new(), - Err(error) => return Err(CliError::Io(error)), - }; - let relay = std::env::current_exe() - .map(|path| path.canonicalize().unwrap_or(path)) - .map(crate::agents::portable_executable_path) - .unwrap_or_else(|_| PathBuf::from("nemo-relay")); - let contents = crate::agents::hermes::transparent_config(&existing, &relay, gateway_url)?; - std::fs::write(overlay.join("config.yaml"), contents)?; - Ok(()) -} - -fn link_state(source: &Path, destination: &Path, directory: bool) -> Result<(), CliError> { - #[cfg(unix)] - { - let _ = directory; - std::os::unix::fs::symlink(source, destination)?; - Ok(()) - } - #[cfg(windows)] - { - if directory { - create_windows_junction(source, destination)?; - } else if std::fs::hard_link(source, destination).is_err() { - std::fs::copy(source, destination)?; - } - Ok(()) - } - #[cfg(not(any(unix, windows)))] - { - let _ = directory; - std::fs::copy(source, destination)?; - Ok(()) - } -} - -#[cfg(windows)] -fn create_windows_junction(source: &Path, destination: &Path) -> Result<(), CliError> { - use std::os::windows::process::CommandExt; - - let mut command = std::process::Command::new( - std::env::var_os("COMSPEC").unwrap_or_else(|| std::ffi::OsString::from("cmd.exe")), - ); - command.args(["/d", "/e:on", "/v:off", "/s", "/c"]); - command - .raw_arg(r#""mklink /J "%NEMO_RELAY_JUNCTION_DEST%" "%NEMO_RELAY_JUNCTION_SOURCE%" >nul""#); - let status = command - .env("NEMO_RELAY_JUNCTION_SOURCE", source) - .env("NEMO_RELAY_JUNCTION_DEST", destination) - .status()?; - if status.success() { - Ok(()) - } else { - Err(CliError::Launch(format!( - "failed to create Hermes state junction {} -> {}: {status}", - destination.display(), - source.display() - ))) - } -} - -pub(crate) fn hooks_path_for_launch(configured: Option<&Path>) -> Result { - if let Some(path) = configured { - return Ok(path.to_path_buf()); - } - if let Some(home) = std::env::var_os("HERMES_HOME").filter(|value| !value.is_empty()) { - return Ok(PathBuf::from(home).join("config.yaml")); - } - let home = std::env::var_os("HOME") - .or_else(|| std::env::var_os("USERPROFILE")) - .ok_or_else(|| { - CliError::Launch("could not resolve home directory for Hermes hooks".into()) - })?; - Ok(PathBuf::from(home).join(".hermes").join("config.yaml")) -} diff --git a/crates/cli/src/agents/hermes/mod.rs b/crates/cli/src/agents/hermes/mod.rs deleted file mode 100644 index 3b6577dc7..000000000 --- a/crates/cli/src/agents/hermes/mod.rs +++ /dev/null @@ -1,51 +0,0 @@ -// SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. -// SPDX-License-Identifier: Apache-2.0 - -use semver::Version; - -use super::AgentDescriptor; - -pub(super) const DESCRIPTOR: AgentDescriptor = AgentDescriptor { - argument: "hermes", - install_argument: "hermes", - label: "Hermes Agent", - executable: "hermes", - hook_path: "/hooks/hermes", - version_product: "Hermes Agent", - minimum_version: (0, 18, 2), - hook_events: &[ - "on_session_start", - "on_session_end", - "on_session_finalize", - "on_session_reset", - "pre_llm_call", - "post_llm_call", - "pre_api_request", - "post_api_request", - "api_request_error", - "pre_tool_call", - "post_tool_call", - "subagent_start", - "subagent_stop", - ], - direct_hook_entries: true, -}; - -pub(super) fn parse_version(raw: &str) -> Option { - Version::parse( - raw.strip_prefix("Hermes Agent v")? - .split_whitespace() - .next()?, - ) - .ok() -} - -mod config; -pub(crate) mod doctor; -mod files; -pub(crate) mod install; -mod integration; -pub(crate) mod launch; -mod trust; - -pub(crate) use integration::*; diff --git a/crates/cli/src/agents/hermes/trust.rs b/crates/cli/src/agents/hermes/trust.rs deleted file mode 100644 index 4dd529dff..000000000 --- a/crates/cli/src/agents/hermes/trust.rs +++ /dev/null @@ -1,129 +0,0 @@ -// SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. -// SPDX-License-Identifier: Apache-2.0 - -//! Exact Hermes shell-hook trust generation and verification. - -use std::fs; -use std::path::Path; -use std::time::SystemTime; - -use chrono::{DateTime, SecondsFormat, Utc}; -use serde_json::{Value, json}; - -use crate::agents::CodingAgent; -use crate::error::CliError; -use crate::hooks::GeneratedHookCommands; - -pub(super) fn trusted_hooks( - existing: Option<&str>, - previous_commands: Option<&GeneratedHookCommands>, - commands: &GeneratedHookCommands, - relay: &Path, - now: SystemTime, -) -> Result { - let mut root = parse_json_object(existing, "Hermes shell-hook allowlist")?; - let approvals = root - .as_object_mut() - .expect("JSON root checked as object") - .entry("approvals") - .or_insert_with(|| json!([])) - .as_array_mut() - .ok_or_else(|| { - CliError::Install("Hermes shell-hook allowlist approvals must be an array".into()) - })?; - approvals.retain(|entry| { - entry - .get("command") - .and_then(Value::as_str) - .is_none_or(|candidate| { - previous_commands.is_none_or(|commands| !commands.contains(candidate)) - }) - }); - let approved_at = timestamp(now); - let script_mtime_at_approval = fs::metadata(relay) - .and_then(|metadata| metadata.modified()) - .ok() - .map(timestamp); - approvals.extend(CodingAgent::Hermes.hook_events().iter().map(|event| { - json!({ - "event": event, - "command": commands.for_event(event), - "approved_at": approved_at, - "script_mtime_at_approval": script_mtime_at_approval, - }) - })); - Ok(root) -} - -fn timestamp(time: SystemTime) -> String { - DateTime::::from(time).to_rfc3339_opts(SecondsFormat::Micros, true) -} - -pub(super) fn verify_trust( - allowlist_path: &Path, - commands: &GeneratedHookCommands, -) -> Result<(), String> { - let raw = fs::read_to_string(allowlist_path) - .map_err(|error| format!("failed to read {}: {error}", allowlist_path.display()))?; - let allowlist = - parse_json_object(Some(&raw), "Hermes shell-hook allowlist").map_err(|e| e.to_string())?; - let approvals = allowlist - .get("approvals") - .and_then(Value::as_array) - .ok_or_else(|| "Hermes shell-hook approvals are missing".to_string())?; - for event in CodingAgent::Hermes.hook_events() { - let matching = approvals - .iter() - .filter(|entry| { - entry.get("event").and_then(Value::as_str) == Some(event) - && entry.get("command").and_then(Value::as_str) - == Some(commands.for_event(event)) - }) - .count(); - if matching != 1 { - return Err(format!( - "Hermes hook {event} expected exactly one trust approval" - )); - } - } - for entry in approvals { - let Some(command) = entry.get("command").and_then(Value::as_str) else { - continue; - }; - if !commands.contains(command) { - continue; - } - let event = entry - .get("event") - .and_then(Value::as_str) - .ok_or_else(|| "Hermes Relay hook approval is missing its event".to_string())?; - if !CodingAgent::Hermes.hook_events().contains(&event) - || command != commands.for_event(event) - { - return Err("Hermes allowlist contains an unexpected Relay hook approval".into()); - } - } - Ok(()) -} - -pub(super) fn parse_json_object(raw: Option<&str>, description: &str) -> Result { - let value = match raw.filter(|raw| !raw.trim().is_empty()) { - Some(raw) => serde_json::from_str(raw) - .map_err(|error| CliError::Install(format!("invalid {description}: {error}")))?, - None => json!({}), - }; - if value.is_object() { - Ok(value) - } else { - Err(CliError::Install(format!( - "{description} must contain a JSON object" - ))) - } -} - -pub(super) fn json_bytes(value: &Value) -> Result, CliError> { - let mut bytes = - serde_json::to_vec_pretty(value).map_err(|error| CliError::Install(error.to_string()))?; - bytes.push(b'\n'); - Ok(bytes) -} diff --git a/crates/cli/src/agents/mod.rs b/crates/cli/src/agents/mod.rs index 376387797..133b8ad98 100644 --- a/crates/cli/src/agents/mod.rs +++ b/crates/cli/src/agents/mod.rs @@ -5,7 +5,6 @@ pub(crate) mod claude; pub(crate) mod codex; -pub(crate) mod hermes; pub(crate) mod shared; use semver::Version; @@ -16,7 +15,6 @@ pub(crate) enum CodingAgent { /// `claude-code` remains an input alias for older Relay configuration. ClaudeCode, Codex, - Hermes, } #[derive(Debug, Clone, Copy)] @@ -29,17 +27,15 @@ pub(super) struct AgentDescriptor { version_product: &'static str, minimum_version: (u64, u64, u64), hook_events: &'static [&'static str], - direct_hook_entries: bool, } impl CodingAgent { - pub(crate) const ALL: [Self; 3] = [Self::ClaudeCode, Self::Codex, Self::Hermes]; + pub(crate) const ALL: [Self; 2] = [Self::ClaudeCode, Self::Codex]; const fn descriptor(self) -> AgentDescriptor { match self { Self::ClaudeCode => claude::DESCRIPTOR, Self::Codex => codex::DESCRIPTOR, - Self::Hermes => hermes::DESCRIPTOR, } } @@ -73,11 +69,6 @@ impl CodingAgent { self.descriptor().hook_events } - /// Hermes stores direct command entries; plugin hosts use nested command-hook groups. - pub(crate) const fn uses_direct_hook_entries(self) -> bool { - self.descriptor().direct_hook_entries - } - pub(crate) fn minimum_version(self) -> Version { let (major, minor, patch) = self.descriptor().minimum_version; Version::new(major, minor, patch) @@ -117,7 +108,6 @@ impl CodingAgent { match self { Self::ClaudeCode => claude::parse_version(raw), Self::Codex => codex::parse_version(raw), - Self::Hermes => hermes::parse_version(raw), } } @@ -139,7 +129,6 @@ impl CodingAgent { match name { "claude" | "claude-code" => Some(Self::ClaudeCode), "codex" => Some(Self::Codex), - "hermes" | "hermes-agent" => Some(Self::Hermes), _ => None, } } @@ -170,7 +159,6 @@ impl crate::installation::marketplace::MarketplaceHost for CodingAgent { match self { Self::Codex => &[".agents", "plugins", "marketplace.json"], Self::ClaudeCode => &[".claude-plugin", "marketplace.json"], - Self::Hermes => unreachable!("Hermes does not use marketplace layout"), } } @@ -178,7 +166,6 @@ impl crate::installation::marketplace::MarketplaceHost for CodingAgent { match self { Self::Codex => &[".codex-plugin", "plugin.json"], Self::ClaudeCode => &[".claude-plugin", "plugin.json"], - Self::Hermes => unreachable!("Hermes does not use marketplace layout"), } } @@ -219,7 +206,6 @@ impl crate::installation::marketplace::MarketplaceHost for CodingAgent { "--scope".into(), "user".into(), ], - Self::Hermes => unreachable!("Hermes does not register marketplace plugins"), } } @@ -227,7 +213,6 @@ impl crate::installation::marketplace::MarketplaceHost for CodingAgent { match self { Self::Codex => vec!["plugin".into(), "remove".into(), plugin_id.into()], Self::ClaudeCode => vec!["plugin".into(), "uninstall".into(), plugin_name.into()], - Self::Hermes => unreachable!("Hermes does not register marketplace plugins"), } } @@ -243,7 +228,6 @@ impl crate::installation::marketplace::MarketplaceHost for CodingAgent { Self::ClaudeCode => { crate::installation::marketplace::host::claude_registration_report(options, runner) } - Self::Hermes => unreachable!("Hermes does not register marketplace plugins"), } } @@ -259,7 +243,6 @@ impl crate::installation::marketplace::MarketplaceHost for CodingAgent { Self::ClaudeCode => format!( "cannot safely replace or uninstall an existing Claude Code plugin because its MCP generation marker {problem}; close all Claude Code clients and standalone `nemo-relay mcp` processes, run `claude plugin uninstall nemo-relay-plugin` and `claude plugin marketplace remove nemo-relay-local`, remove the stale marketplace and state from the selected install directory, then run `nemo-relay install claude-code --force` to create a fenced install (and `nemo-relay uninstall claude-code` afterward if removal was intended)" ), - Self::Hermes => unreachable!("Hermes does not use marketplace generations"), } } @@ -285,7 +268,6 @@ impl crate::installation::marketplace::MarketplaceHost for CodingAgent { || plugin_root.join(".mcp.json").exists() || generation_fence.exists() } - Self::Hermes => unreachable!("Hermes does not use marketplace installs"), } } @@ -347,7 +329,6 @@ pub(crate) fn marketplace_manifest( match agent { CodingAgent::Codex => codex::assets::marketplace_manifest(marketplace, plugin), CodingAgent::ClaudeCode => claude::assets::marketplace_manifest(marketplace, plugin), - CodingAgent::Hermes => unreachable!("Hermes does not install a marketplace plugin"), } } @@ -355,7 +336,6 @@ pub(crate) fn plugin_manifest(agent: CodingAgent, plugin: &str) -> serde_json::V match agent { CodingAgent::Codex => codex::assets::plugin_manifest(plugin), CodingAgent::ClaudeCode => claude::assets::plugin_manifest(plugin), - CodingAgent::Hermes => unreachable!("Hermes does not install a marketplace plugin"), } } @@ -366,7 +346,6 @@ pub(crate) fn plugin_mcp_config( match agent { CodingAgent::Codex => codex::assets::mcp_config(server), CodingAgent::ClaudeCode => Ok(claude::assets::mcp_config(server)), - CodingAgent::Hermes => unreachable!("Hermes does not install a marketplace plugin"), } } @@ -382,7 +361,7 @@ pub(crate) fn prepare_launch( agent: CodingAgent, launch: &mut crate::process::PreparedAgentLaunch, gateway_url: &str, - resolved: &crate::configuration::ResolvedConfig, + _resolved: &crate::configuration::ResolvedConfig, proxy_credential: &crate::provider_auth::TransparentProxyCredential, dry_run: bool, ) -> Result<(), crate::error::CliError> { @@ -395,17 +374,11 @@ pub(crate) fn prepare_launch( CodingAgent::ClaudeCode => { claude::launch::prepare(launch, gateway_url, proxy_credential, dry_run) } - CodingAgent::Hermes => hermes::launch::prepare( - launch, - resolved.agents.hermes.hooks_path.as_deref(), - dry_run, - ), } } pub(crate) fn configured(agent: CodingAgent, configs: &crate::configuration::AgentConfigs) -> bool { config(agent, configs).command.is_some() - || matches!(agent, CodingAgent::Hermes) && configs.hermes.hooks_path.is_some() } pub(crate) const fn config( @@ -415,18 +388,16 @@ pub(crate) const fn config( match agent { CodingAgent::ClaudeCode => &configs.claude, CodingAgent::Codex => &configs.codex, - CodingAgent::Hermes => &configs.hermes, } } pub(crate) fn hook_status( agent: CodingAgent, - configs: &crate::configuration::AgentConfigs, + _configs: &crate::configuration::AgentConfigs, ) -> Result { match agent { CodingAgent::Codex => codex::doctor::hook_status(), CodingAgent::ClaudeCode => claude::doctor::hook_status(), - CodingAgent::Hermes => hermes::doctor::hook_status(configs.hermes.hooks_path.as_deref()), } } @@ -457,7 +428,6 @@ pub(crate) fn snapshot_setup(agent: CodingAgent) -> Result snapshot_codex_setup().map(SetupSnapshot::Codex), CodingAgent::ClaudeCode => snapshot_claude_setup().map(SetupSnapshot::Claude), - CodingAgent::Hermes => unreachable!("Hermes does not use marketplace setup"), } } @@ -479,7 +449,6 @@ pub(crate) fn setup_marketplace_plugin( install_codex_plugin_with_generation(gateway_url, plugin_root, generation_token) } CodingAgent::ClaudeCode => enable_claude_provider(gateway_url), - CodingAgent::Hermes => unreachable!("Hermes does not use marketplace setup"), } } @@ -491,7 +460,6 @@ pub(crate) fn uninstall_marketplace_plugin( match agent { CodingAgent::Codex => uninstall_codex_plugin(gateway_url, plugin_root), CodingAgent::ClaudeCode => restore_claude_provider(gateway_url), - CodingAgent::Hermes => unreachable!("Hermes does not use marketplace setup"), } } @@ -509,7 +477,6 @@ pub(crate) fn doctor_marketplace_plugin( generation_token, ), CodingAgent::ClaudeCode => doctor_plugin(CodingAgent::ClaudeCode, gateway_url, plugin_root), - CodingAgent::Hermes => unreachable!("Hermes does not use marketplace setup"), } } @@ -523,7 +490,6 @@ pub(crate) fn doctor_marketplace_plugin_json( CodingAgent::ClaudeCode => { doctor_plugin_json(CodingAgent::ClaudeCode, gateway_url, plugin_root) } - CodingAgent::Hermes => unreachable!("Hermes does not use marketplace setup"), } } @@ -532,7 +498,6 @@ pub(crate) fn install_integration( command: crate::installation::InstallRequest, ) -> Result { match agent { - CodingAgent::Hermes => hermes::install::install(command), CodingAgent::Codex => codex::install::install(command), CodingAgent::ClaudeCode => claude::install::install(command), } @@ -543,7 +508,6 @@ pub(crate) fn uninstall_integration( command: crate::installation::UninstallRequest, ) -> Result { match agent { - CodingAgent::Hermes => hermes::install::uninstall(command), CodingAgent::Codex => codex::install::uninstall(command), CodingAgent::ClaudeCode => claude::install::uninstall(command), } @@ -567,12 +531,8 @@ pub(crate) fn installed_integrations( candidates .iter() .copied() - .filter(|agent| match agent { - CodingAgent::Codex | CodingAgent::ClaudeCode => { - crate::installation::marketplace::persisted_state_exists(*agent, &install_dir) - } - CodingAgent::Hermes => hermes::install::config_path() - .is_ok_and(|path| hermes::persistent_state_exists(&path)), + .filter(|agent| { + crate::installation::marketplace::persisted_state_exists(*agent, &install_dir) }) .collect() } @@ -581,31 +541,14 @@ pub(crate) fn doctor_integration( agent: CodingAgent, options: &crate::installation::marketplace::state::PluginInstallOptions, ) -> Result<(), crate::error::CliError> { - match agent { - CodingAgent::Codex | CodingAgent::ClaudeCode => { - crate::installation::marketplace::doctor_marketplace_integration(agent, options) - } - CodingAgent::Hermes => { - let runner = crate::installation::marketplace::host::RealCommandRunner; - hermes::install::doctor(options, &runner).map_err(crate::error::CliError::Install) - } - } + crate::installation::marketplace::doctor_marketplace_integration(agent, options) } pub(crate) fn doctor_integration_report( agent: CodingAgent, options: &crate::installation::marketplace::state::PluginInstallOptions, ) -> Result { - match agent { - CodingAgent::Codex | CodingAgent::ClaudeCode => { - crate::installation::marketplace::doctor_marketplace_report(agent, options) - } - CodingAgent::Hermes => { - let runner = crate::installation::marketplace::host::RealCommandRunner; - hermes::install::doctor_json_value(options, &runner) - .map_err(crate::error::CliError::Install) - } - } + crate::installation::marketplace::doctor_marketplace_report(agent, options) } struct PendingIntegrationReadiness { @@ -682,15 +625,7 @@ fn spawn_integration_readiness( agent: CodingAgent, install_dir: PathBuf, ) -> PendingIntegrationReadiness { - let state_path = match agent { - CodingAgent::Codex | CodingAgent::ClaudeCode => { - crate::installation::marketplace::marketplace_state_path(agent, &install_dir) - } - CodingAgent::Hermes => { - hermes::install::config_path().unwrap_or_else(|_| install_dir.join("hermes.json")) - } - }; - let worker_state_path = state_path.clone(); + let state_path = crate::installation::marketplace::marketplace_state_path(agent, &install_dir); let worker_install_dir = install_dir.clone(); let (sender, receiver) = std::sync::mpsc::sync_channel(1); std::thread::spawn(move || { @@ -702,16 +637,9 @@ fn spawn_integration_readiness( skip_doctor: true, }; let runner = crate::installation::marketplace::host::RealCommandRunner; - let readiness = match agent { - CodingAgent::Codex | CodingAgent::ClaudeCode => { - crate::installation::marketplace::collect_marketplace_readiness( - agent, &options, &runner, - ) - } - CodingAgent::Hermes => { - hermes::install::collect_readiness(&worker_state_path, &options, &runner) - } - }; + let readiness = crate::installation::marketplace::collect_marketplace_readiness( + agent, &options, &runner, + ); let _ = sender.send(readiness); }); PendingIntegrationReadiness { @@ -727,20 +655,14 @@ fn failed_integration_readiness( install_dir: &Path, details: &str, ) -> crate::installation::marketplace::HostPluginReadiness { - let (marketplace, plugin) = match agent { - CodingAgent::Codex | CodingAgent::ClaudeCode => { - let (marketplace, plugin) = - crate::installation::marketplace::marketplace_install_roots(agent, install_dir); - (Some(marketplace), Some(plugin)) - } - CodingAgent::Hermes => (None, None), - }; + let (marketplace, plugin) = + crate::installation::marketplace::marketplace_install_roots(agent, install_dir); let mut readiness = crate::installation::marketplace::HostPluginReadiness { host: agent.install_arg().to_string(), remediation: format!("nemo-relay install {} --force", agent.install_arg()), state_path, - marketplace, - plugin, + marketplace: Some(marketplace), + plugin: Some(plugin), checks: Vec::new(), relay: None, host_plugin_registered: None, @@ -752,6 +674,7 @@ fn failed_integration_readiness( } pub(crate) use crate::process::portable_executable_path; +#[cfg(any(not(windows), test))] pub(crate) use crate::process::shell_quote_arg_for_platform; #[cfg(test)] pub(crate) use crate::process::strip_windows_verbatim_prefix; @@ -786,10 +709,6 @@ pub(crate) fn install_codex_plugin_with_generation( .map(|_| ()) } -pub(crate) fn stop_plugin_gateway() -> Result<(), String> { - crate::bootstrap::state::stop_owned_and_reset(crate::bootstrap::DEFAULT_URL) -} - pub(crate) fn uninstall_codex_plugin(gateway_url: &str, plugin_root: &Path) -> Result<(), String> { uninstall_codex(gateway_url, &plugin_root.join("hooks").join("hooks.json")).map(|_| ()) } @@ -870,12 +789,6 @@ pub(crate) fn doctor_plugin_json( Some(trust), ) } - other => { - return Err(format!( - "plugin doctor supports claude and codex, got {}", - other.as_arg() - )); - } }; let mut report = json!({ "ok": ok, @@ -935,12 +848,6 @@ fn doctor_ok( print_info("codex hook trust", &trust.summary()); } } - other => { - return Err(format!( - "plugin doctor supports claude and codex, got {}", - other.as_arg() - )); - } } Ok(ok) } diff --git a/crates/cli/src/agents/shared/adapters.rs b/crates/cli/src/agents/shared/adapters.rs index 38d4b0794..8d5b3aff5 100644 --- a/crates/cli/src/agents/shared/adapters.rs +++ b/crates/cli/src/agents/shared/adapters.rs @@ -5,8 +5,6 @@ pub(crate) mod claude_code; #[path = "../codex/adapter.rs"] pub(crate) mod codex; -#[path = "../hermes/adapter.rs"] -pub(crate) mod hermes; pub(crate) const SKILL_LOAD_SOURCE_KEY: &str = "skill_load_source"; pub(crate) const SKILL_LOAD_SOURCE_PROMPT_EXPANSION: &str = "prompt_expansion"; @@ -114,9 +112,7 @@ pub(crate) trait AgentPayloadExtractor { } /// Tool payload paths (call id, name, arguments, result, status). - fn tool_paths(&self) -> &'static ToolPathSet { - TOOL_PATHS - } + fn tool_paths(&self) -> &'static ToolPathSet; // -- Shared behavior (derived from the deviation hooks above) ------------ @@ -186,12 +182,10 @@ pub(crate) trait AgentPayloadExtractor { pub(super) struct ClaudeCodePayloadExtractor; pub(super) struct CodexPayloadExtractor; -pub(super) struct HermesPayloadExtractor; pub(super) static CLAUDE_CODE_PAYLOAD_EXTRACTOR: ClaudeCodePayloadExtractor = ClaudeCodePayloadExtractor; pub(super) static CODEX_PAYLOAD_EXTRACTOR: CodexPayloadExtractor = CodexPayloadExtractor; -pub(super) static HERMES_PAYLOAD_EXTRACTOR: HermesPayloadExtractor = HermesPayloadExtractor; /// Claude Code reports its native tool identifier as `tool_use_id`, so it uses /// a tool path set that prefers that key. Every other hook field matches the @@ -219,16 +213,6 @@ impl AgentPayloadExtractor for CodexPayloadExtractor { } } -/// Hermes always runs nested under another agent, so the `child_subagent_id` -/// signal is the most reliable owner and is preferred over the generic -/// session-scoped subagent id. Session, event, and tool extraction match the -/// canonical defaults. -impl AgentPayloadExtractor for HermesPayloadExtractor { - fn subagent_id_paths(&self) -> &'static [&'static [&'static str]] { - HERMES_SUBAGENT_ID_PATHS - } -} - pub(crate) struct ToolPathSet { call_id: &'static [&'static [&'static str]], name: &'static [&'static [&'static str]], @@ -318,25 +302,6 @@ const CODEX_SUBAGENT_ID_PATHS: &[&[&str]] = &[ &["extra", "agent", "id"], ]; -/// Hermes deviation: prefers the `child_subagent_id` owner signal before the -/// generic session-scoped subagent id. -const HERMES_SUBAGENT_ID_PATHS: &[&[&str]] = &[ - &["child_subagent_id"], - &["childSubagentId"], - &["subagent_id"], - &["subagentId"], - &["agent_id"], - &["subagent", "id"], - &["agent", "id"], - &["extra", "child_subagent_id"], - &["extra", "childSubagentId"], - &["extra", "subagent_id"], - &["extra", "subagentId"], - &["extra", "agent_id"], - &["extra", "subagent", "id"], - &["extra", "agent", "id"], -]; - /// Claude Code deviation: its native tool identifier is `tool_use_id`, checked /// before the generic `tool_call_id` shapes. const CLAUDE_TOOL_CALL_ID_PATHS: &[&[&str]] = &[ @@ -351,8 +316,7 @@ const CLAUDE_TOOL_CALL_ID_PATHS: &[&[&str]] = &[ &["id"], ]; -/// Canonical tool-call-id precedence for harnesses that report the generic -/// `tool_call_id` first (Codex and Hermes). +/// Codex tool-call-id precedence. const TOOL_CALL_ID_PATHS: &[&[&str]] = &[ &["tool_call_id"], &["toolCallId"], @@ -373,8 +337,7 @@ const TOOL_NAME_PATHS: &[&[&str]] = &[ &["name"], ]; -/// Canonical argument precedence for harnesses that nest tool input under -/// `tool_input` first (Claude Code and Hermes). +/// Claude Code argument precedence. const TOOL_ARGUMENT_PATHS: &[&[&str]] = &[&["tool_input"], &["input"], &["arguments"], &["args"]]; /// Codex deviation: sends tool arguments under `arguments`/`args` first. @@ -390,15 +353,6 @@ const TOOL_RESULT_PATHS: &[&[&str]] = &[ ]; const TOOL_STATUS_PATHS: &[&[&str]] = &[&["status"], &["decision"], &["permission"]]; -/// Canonical tool path set used by harnesses that report generic tool shapes -/// (Hermes). Name, result, and status precedence is shared by every harness. -const TOOL_PATHS: &ToolPathSet = &ToolPathSet { - call_id: TOOL_CALL_ID_PATHS, - name: TOOL_NAME_PATHS, - arguments: TOOL_ARGUMENT_PATHS, - result: TOOL_RESULT_PATHS, - status: TOOL_STATUS_PATHS, -}; const CLAUDE_TOOL_PATHS: &ToolPathSet = &ToolPathSet { call_id: CLAUDE_TOOL_CALL_ID_PATHS, name: TOOL_NAME_PATHS, @@ -525,19 +479,6 @@ fn agent_tool_call( } } -/// Derive a stable session identifier from extracted facts and compatibility fallbacks. -/// -/// Header and payload precedence lives in the selected extractor. This boundary -/// applies the final synthetic ID fallback so sparse payloads stay observable. -fn session_id( - payload: &Value, - headers: &HeaderMap, - extractor: &dyn AgentPayloadExtractor, -) -> String { - let fallback_session_id = fallback_session_id(); - session_id_with_fallback(payload, headers, extractor, &fallback_session_id) -} - fn fallback_session_id() -> String { format!("hook-{}", Uuid::now_v7()) } @@ -589,20 +530,6 @@ fn metadata( extractor.metadata(payload, headers, kind, event_name) } -/// Create a root session event using the common extraction rules. -/// -/// Lifecycle, marks, notifications, and compaction events all carry identical -/// session-id and metadata correlation fields. -pub(crate) fn common_session_event( - payload: &Value, - headers: &HeaderMap, - kind: AgentKind, - extractor: &dyn AgentPayloadExtractor, -) -> SessionEvent { - let fallback_session_id = fallback_session_id(); - common_session_event_with_fallback(payload, headers, kind, extractor, &fallback_session_id) -} - fn common_session_event_with_fallback( payload: &Value, headers: &HeaderMap, diff --git a/crates/cli/src/agents/shared/alignment.rs b/crates/cli/src/agents/shared/alignment.rs index 6694d526d..94e03af54 100644 --- a/crates/cli/src/agents/shared/alignment.rs +++ b/crates/cli/src/agents/shared/alignment.rs @@ -17,14 +17,12 @@ use crate::configuration::header_string; pub(crate) use crate::events::json_path::{ string_at_any as json_string_at, value_at_any as json_value_at, }; -use crate::events::{AgentKind, LlmEvent, NormalizedEvent, SessionEvent, SubagentEvent, ToolEvent}; +use crate::events::{AgentKind, NormalizedEvent, SessionEvent, SubagentEvent, ToolEvent}; #[path = "../claude/alignment.rs"] pub(crate) mod claude_code; #[path = "../codex/alignment.rs"] pub(crate) mod codex; -#[path = "../hermes/alignment.rs"] -pub(crate) mod hermes; const REQUEST_AFFINITY_KEY_MIN_CHARS: usize = 24; const REQUEST_AFFINITY_KEY_MAX_CHARS: usize = 4096; @@ -32,14 +30,12 @@ const REQUEST_AFFINITY_KEY_MAX_CHARS: usize = 4096; #[derive(Debug, Clone)] pub(crate) enum SubagentSessionContext { Codex(codex::SubagentContext), - Hermes(hermes::SubagentContext), } impl SubagentSessionContext { pub(crate) fn parent_session_id(&self) -> &str { match self { Self::Codex(context) => &context.parent_session_id, - Self::Hermes(context) => &context.parent_session_id, } } } @@ -268,17 +264,13 @@ impl PendingSubagentStart { #[derive(Debug, Default)] pub(crate) struct SessionAlignmentState { aliases: HashMap, - completed_aliases: HashMap, pending_subagents: HashMap, - task_sessions: HashMap>, } impl SessionAlignmentState { pub(crate) fn clear(&mut self) { self.aliases.clear(); - self.completed_aliases.clear(); self.pending_subagents.clear(); - self.task_sessions.clear(); } pub(crate) fn alias_for_session(&self, session_id: &str) -> Option { @@ -316,17 +308,12 @@ impl SessionAlignmentState { } pub(crate) fn route_event(&mut self, event: NormalizedEvent) -> NormalizedEvent { - self.record_task_session(&event); - let event = self.route_task_session_event(event); let (event, finished_alias) = route_event_through_alias(event, &self.aliases); let session_id = event.session_id().to_string(); if let Some(child_session_id) = finished_alias.as_ref() { // Remove aliases before terminal skip checks so a late child AgentEnd, or a child // TurnEnded used as a subagent completion signal, cannot leave stale reparenting state. - if let Some(alias) = self.aliases.remove(child_session_id) { - self.completed_aliases - .insert(child_session_id.clone(), alias); - } + self.aliases.remove(child_session_id); self.pending_subagents.remove(child_session_id); } if matches!(&event, NormalizedEvent::AgentEnded(_)) { @@ -335,31 +322,6 @@ impl SessionAlignmentState { event } - pub(crate) fn align_explicit_subagent_end(&mut self, event: &mut NormalizedEvent) { - let NormalizedEvent::SubagentEnded(subagent_event) = event else { - return; - }; - let Some(child_session_id) = hermes::child_session_id_for_subagent_event(subagent_event) - else { - return; - }; - let Some(alias) = self - .aliases - .get(&child_session_id) - .or_else(|| self.completed_aliases.get(&child_session_id)) - .cloned() - else { - return; - }; - if subagent_event.session_id != alias.parent_session_id { - return; - } - subagent_event.subagent_id = alias.subagent_id.clone(); - subagent_event.metadata = merge_metadata(subagent_event.metadata.clone(), alias.metadata()); - self.aliases.remove(&child_session_id); - self.completed_aliases.remove(&child_session_id); - } - pub(crate) fn pending_for_parent( &mut self, parent_session_id: &str, @@ -386,14 +348,9 @@ impl SessionAlignmentState { self.aliases.retain(|child_session_id, alias| { child_session_id != session_id && alias.parent_session_id != session_id }); - self.completed_aliases.retain(|child_session_id, alias| { - child_session_id != session_id && alias.parent_session_id != session_id - }); self.pending_subagents.retain(|child_session_id, pending| { child_session_id != session_id && pending.parent_session_id() != session_id }); - self.task_sessions.remove(session_id); - prune_task_sessions(&mut self.task_sessions, session_id); } pub(crate) fn clear_for_ended_subagent(&mut self, parent_session_id: &str, subagent_id: &str) { @@ -407,73 +364,7 @@ impl SessionAlignmentState { && !(pending.parent_session_id() == parent_session_id && pending.event.session_id == subagent_id) }); - self.task_sessions - .retain(|session_id, _| session_id != subagent_id); - prune_task_sessions(&mut self.task_sessions, subagent_id); - } - - fn record_task_session(&mut self, event: &NormalizedEvent) { - if normalized_event_agent_kind(event) != AgentKind::Hermes { - return; - } - let Some(task_id) = event_task_id(event) else { - return; - }; - let session_id = event.session_id(); - if session_id == task_id { - return; - } - self.task_sessions - .entry(session_id.to_string()) - .or_default() - .insert(task_id, session_id.to_string()); } - - fn route_task_session_event(&self, event: NormalizedEvent) -> NormalizedEvent { - let should_route = matches!( - event, - NormalizedEvent::ToolStarted(_) | NormalizedEvent::ToolEnded(_) - ) && normalized_event_agent_kind(&event) == AgentKind::Hermes; - if !should_route { - return event; - } - - let task_id = event_task_id(&event).unwrap_or_else(|| event.session_id().to_string()); - let session_scope = event_task_session_scope(&event); - let Some(session_id) = self.session_for_task(&task_id, session_scope.as_deref()) else { - return event; - }; - route_task_session_event(event, task_id, session_id) - } - - fn session_for_task(&self, task_id: &str, session_scope: Option<&str>) -> Option { - if let Some(session_scope) = session_scope { - return self - .task_sessions - .get(session_scope) - .and_then(|tasks| tasks.get(task_id)) - .cloned(); - } - - let mut matches = self - .task_sessions - .values() - .filter_map(|tasks| tasks.get(task_id).cloned()); - let session_id = matches.next()?; - matches.next().is_none().then_some(session_id) - } -} - -fn prune_task_sessions( - task_sessions: &mut HashMap>, - session_id: &str, -) { - task_sessions.values_mut().for_each(|tasks| { - tasks.retain(|task_id, mapped_session_id| { - task_id != session_id && mapped_session_id != session_id - }); - }); - task_sessions.retain(|_, tasks| !tasks.is_empty()); } // Resolves the session id for a gateway request in precedence order: @@ -632,7 +523,6 @@ pub(crate) async fn subagent_session_context( codex::subagent_context(event) .await .map(SubagentSessionContext::Codex) - .or_else(|| hermes::subagent_context(event).map(SubagentSessionContext::Hermes)) } /// Convert an agent start into a pending child-session record when possible. @@ -673,9 +563,6 @@ pub(crate) fn augment_subagent_session_metadata( SubagentSessionContext::Codex(context) => { codex::augment_subagent_metadata(metadata, context) } - SubagentSessionContext::Hermes(context) => { - hermes::augment_subagent_metadata(metadata, context) - } } } @@ -689,7 +576,6 @@ pub(crate) fn subagent_start_event( ) -> SubagentEvent { match context { SubagentSessionContext::Codex(context) => codex::subagent_start_event(event, context), - SubagentSessionContext::Hermes(context) => hermes::subagent_start_event(event, context), } } @@ -705,34 +591,15 @@ pub(crate) fn alias_for_child_session( SubagentSessionContext::Codex(context) => { codex::alias_for_child_session(child_session_id, context) } - SubagentSessionContext::Hermes(context) => { - hermes::alias_for_child_session(child_session_id, context) - } } } -/// Extract an explicit child-session alias from a subagent-start event. -pub(crate) fn explicit_subagent_alias( - event: &mut NormalizedEvent, -) -> Option<(String, SessionAlias)> { - let NormalizedEvent::SubagentStarted(subagent_event) = event else { - return None; - }; - let explicit = hermes::explicit_subagent_alias(subagent_event)?; - subagent_event.metadata = - merge_metadata(subagent_event.metadata.clone(), explicit.scope_metadata); - Some((explicit.child_session_id, explicit.alias)) -} - /// Recover provider-specific metadata that should follow owned LLM spans. /// /// Codex contributes thread identifiers today; other harnesses can add filters /// here without changing session ownership code. pub(crate) fn llm_owner_metadata(scope_metadata: Option<&Value>) -> Value { - merge_metadata( - codex::llm_owner_metadata(scope_metadata), - hermes::llm_owner_metadata(scope_metadata), - ) + codex::llm_owner_metadata(scope_metadata) } /// Build a route-specific affinity key from provider request user task text. @@ -773,7 +640,6 @@ pub(crate) fn aliased_turn_subagent_id(event: &SessionEvent) -> Option { json_string_at( &event.metadata, &[ - &["hermes_child_subagent_id"][..], &["subagent_id"][..], &["codex_subagent_session_id"][..], &["subagent_session_id"][..], @@ -852,14 +718,6 @@ pub(crate) fn route_event_through_alias( event.metadata = merge_metadata(event.metadata, metadata); (NormalizedEvent::LlmHint(event), None) } - NormalizedEvent::LlmStarted(mut event) => { - route_llm_event(&mut event, &alias, metadata); - (NormalizedEvent::LlmStarted(event), None) - } - NormalizedEvent::LlmEnded(mut event) => { - route_llm_event(&mut event, &alias, metadata); - (NormalizedEvent::LlmEnded(event), None) - } NormalizedEvent::ToolStarted(mut event) => { route_tool_event(&mut event, &alias, metadata); (NormalizedEvent::ToolStarted(event), None) @@ -886,24 +744,6 @@ fn route_subagent_event(event: &mut SubagentEvent, alias: &SessionAlias, metadat event.metadata = merge_metadata(event.metadata.clone(), metadata); } -// Rewrites hook-originated LLM events from aliased child sessions. `LlmEvent` does not have a -// first-class subagent id field, so the alias owner is stamped into metadata where the session -// manager's hook-LLM path can recover it and choose the subagent scope. -fn route_llm_event(event: &mut LlmEvent, alias: &SessionAlias, metadata: Value) { - event.session_id = alias.parent_session_id.clone(); - event.metadata = merge_metadata( - event.metadata.clone(), - merge_metadata( - metadata, - json!({ - "llm_correlation_status": "session_alias", - "llm_correlation_source": "session_alias", - "llm_correlation_subagent_id": alias.subagent_id.clone(), - }), - ), - ); -} - // Rewrites tool calls emitted by an aliased child session so they attach under the aliased // subagent. This is the common case for Codex child-thread tool activity that would otherwise show // up as root-agent tool calls. @@ -913,142 +753,6 @@ fn route_tool_event(event: &mut ToolEvent, alias: &SessionAlias, metadata: Value event.metadata = merge_metadata(event.metadata.clone(), metadata); } -fn route_task_session_event( - event: NormalizedEvent, - task_id: String, - session_id: String, -) -> NormalizedEvent { - let metadata = json!({ - "session_correlation_status": "task_session_alias", - "session_correlation_source": "task_id", - "hermes_task_id": task_id, - "hermes_session_id": session_id, - }); - match event { - NormalizedEvent::ToolStarted(mut event) => { - event.session_id = session_id; - event.metadata = merge_metadata(event.metadata, metadata); - NormalizedEvent::ToolStarted(event) - } - NormalizedEvent::ToolEnded(mut event) => { - event.session_id = session_id; - event.metadata = merge_metadata(event.metadata, metadata); - NormalizedEvent::ToolEnded(event) - } - event => event, - } -} - -fn normalized_event_agent_kind(event: &NormalizedEvent) -> AgentKind { - match event { - NormalizedEvent::AgentStarted(event) - | NormalizedEvent::AgentEnded(event) - | NormalizedEvent::TurnEnded(event) - | NormalizedEvent::PromptSubmitted(event) - | NormalizedEvent::Compaction(event) - | NormalizedEvent::Notification(event) - | NormalizedEvent::HookMark(event) => event.agent_kind, - NormalizedEvent::SubagentStarted(event) | NormalizedEvent::SubagentEnded(event) => { - event.agent_kind - } - NormalizedEvent::LlmHint(event) => event.agent_kind, - NormalizedEvent::LlmStarted(event) | NormalizedEvent::LlmEnded(event) => event.agent_kind, - NormalizedEvent::ToolStarted(event) | NormalizedEvent::ToolEnded(event) => event.agent_kind, - } -} - -fn event_task_id(event: &NormalizedEvent) -> Option { - match event { - NormalizedEvent::AgentStarted(event) - | NormalizedEvent::AgentEnded(event) - | NormalizedEvent::TurnEnded(event) - | NormalizedEvent::PromptSubmitted(event) - | NormalizedEvent::Compaction(event) - | NormalizedEvent::Notification(event) - | NormalizedEvent::HookMark(event) => { - task_id_from_payload_and_metadata(&event.payload, &event.metadata) - } - NormalizedEvent::SubagentStarted(event) | NormalizedEvent::SubagentEnded(event) => { - task_id_from_payload_and_metadata(&event.payload, &event.metadata) - } - NormalizedEvent::LlmHint(event) => { - task_id_from_payload_and_metadata(&event.payload, &event.metadata) - } - NormalizedEvent::LlmStarted(event) | NormalizedEvent::LlmEnded(event) => { - task_id_from_llm_event(event) - } - NormalizedEvent::ToolStarted(event) | NormalizedEvent::ToolEnded(event) => { - task_id_from_payload_and_metadata(&event.payload, &event.metadata) - } - } -} - -fn event_task_session_scope(event: &NormalizedEvent) -> Option { - match event { - NormalizedEvent::AgentStarted(event) - | NormalizedEvent::AgentEnded(event) - | NormalizedEvent::TurnEnded(event) - | NormalizedEvent::PromptSubmitted(event) - | NormalizedEvent::Compaction(event) - | NormalizedEvent::Notification(event) - | NormalizedEvent::HookMark(event) => { - session_scope_from_payload_and_metadata(&event.payload, &event.metadata) - } - NormalizedEvent::SubagentStarted(event) | NormalizedEvent::SubagentEnded(event) => { - session_scope_from_payload_and_metadata(&event.payload, &event.metadata) - } - NormalizedEvent::LlmHint(event) => { - session_scope_from_payload_and_metadata(&event.payload, &event.metadata) - } - NormalizedEvent::LlmStarted(event) | NormalizedEvent::LlmEnded(event) => { - session_scope_from_llm_event(event) - } - NormalizedEvent::ToolStarted(event) | NormalizedEvent::ToolEnded(event) => { - session_scope_from_payload_and_metadata(&event.payload, &event.metadata) - } - } -} - -fn task_id_from_llm_event(event: &LlmEvent) -> Option { - task_id_from_payload_and_metadata(&event.request, &event.metadata) - .or_else(|| task_id_from_payload_and_metadata(&event.response, &event.metadata)) -} - -fn task_id_from_payload_and_metadata(payload: &Value, metadata: &Value) -> Option { - json_string_at(payload, TASK_ID_PATHS).or_else(|| json_string_at(metadata, TASK_ID_PATHS)) -} - -fn session_scope_from_llm_event(event: &LlmEvent) -> Option { - session_scope_from_payload_and_metadata(&event.request, &event.metadata) - .or_else(|| session_scope_from_payload_and_metadata(&event.response, &event.metadata)) -} - -fn session_scope_from_payload_and_metadata(payload: &Value, metadata: &Value) -> Option { - json_string_at(payload, TASK_SESSION_SCOPE_PATHS) - .or_else(|| json_string_at(metadata, TASK_SESSION_SCOPE_PATHS)) -} - -const TASK_ID_PATHS: &[&[&str]] = &[ - &["task_id"], - &["taskId"], - &["extra", "task_id"], - &["extra", "taskId"], -]; - -const TASK_SESSION_SCOPE_PATHS: &[&[&str]] = &[ - &["session_id"], - &["sessionId"], - &["session", "id"], - &["conversation_id"], - &["conversationId"], - &["parent_session_id"], - &["parentSessionId"], - &["extra", "session_id"], - &["extra", "sessionId"], - &["extra", "parent_session_id"], - &["extra", "parentSessionId"], -]; - fn messages_user_task_text(payload: &Value) -> Option { payload .get("messages") diff --git a/crates/cli/src/commands/configure/mod.rs b/crates/cli/src/commands/configure/mod.rs index 01ca40b90..974f2cb2d 100644 --- a/crates/cli/src/commands/configure/mod.rs +++ b/crates/cli/src/commands/configure/mod.rs @@ -22,8 +22,7 @@ pub(crate) struct ConfigCommand { pub(crate) command: Option, #[arg(value_enum)] pub(crate) agent: Option, - /// Reset user Relay configuration. Persistent Hermes integration state is - /// managed separately with `nemo-relay uninstall hermes`. + /// Reset user Relay configuration. #[arg(long)] pub(crate) reset: bool, } diff --git a/crates/cli/src/commands/configure/model.rs b/crates/cli/src/commands/configure/model.rs index 2c1bb514c..396d72e1e 100644 --- a/crates/cli/src/commands/configure/model.rs +++ b/crates/cli/src/commands/configure/model.rs @@ -285,7 +285,6 @@ pub(crate) fn read_agents_from_doc(doc: &DocumentMut) -> Vec { let agent = match key { "claude" => Some(CodingAgent::ClaudeCode), "codex" => Some(CodingAgent::Codex), - "hermes" => Some(CodingAgent::Hermes), _ => None, }; if let Some(agent) = agent { diff --git a/crates/cli/src/commands/configure/wizard/prompt.rs b/crates/cli/src/commands/configure/wizard/prompt.rs index 544fe46f7..0d9f8ff62 100644 --- a/crates/cli/src/commands/configure/wizard/prompt.rs +++ b/crates/cli/src/commands/configure/wizard/prompt.rs @@ -197,11 +197,7 @@ fn ask_agents( detected: &[CodingAgent], configured: &[CodingAgent], ) -> Result, CliError> { - let all_supported = [ - CodingAgent::ClaudeCode, - CodingAgent::Codex, - CodingAgent::Hermes, - ]; + let all_supported = [CodingAgent::ClaudeCode, CodingAgent::Codex]; let labels: Vec = all_supported .iter() .map(|a| { diff --git a/crates/cli/src/commands/diagnostics.rs b/crates/cli/src/commands/diagnostics.rs index ccf6cbe08..d541723a6 100644 --- a/crates/cli/src/commands/diagnostics.rs +++ b/crates/cli/src/commands/diagnostics.rs @@ -66,7 +66,7 @@ fn execute_plugin_doctor( }; if agents.is_empty() { return Err(CliError::Install( - "no installed Claude Code, Codex, or Hermes integration state was found".into(), + "no installed Claude Code or Codex integration state was found".into(), )); } let options = crate::installation::marketplace::plugin_doctor_options(install_dir); diff --git a/crates/cli/src/commands/install.rs b/crates/cli/src/commands/install.rs index eaf4e1512..a090beaf6 100644 --- a/crates/cli/src/commands/install.rs +++ b/crates/cli/src/commands/install.rs @@ -39,7 +39,6 @@ pub(crate) enum InstallTarget { Codex, #[value(name = "claude-code", alias = "claude")] ClaudeCode, - Hermes, All, } @@ -48,12 +47,7 @@ impl InstallTarget { match self { Self::Codex => vec![CodingAgent::Codex], Self::ClaudeCode => vec![CodingAgent::ClaudeCode], - Self::Hermes => vec![CodingAgent::Hermes], - Self::All => vec![ - CodingAgent::Codex, - CodingAgent::ClaudeCode, - CodingAgent::Hermes, - ], + Self::All => vec![CodingAgent::Codex, CodingAgent::ClaudeCode], } } @@ -93,7 +87,7 @@ pub(super) fn install(command: InstallCommand) -> Result { }; if agents.is_empty() { return Err(CliError::Install( - "no supported Claude Code, Codex, or Hermes host CLI was detected".into(), + "no supported Claude Code or Codex host CLI was detected".into(), )); } run_agent_operations(agents, "install", |agent| { @@ -112,7 +106,7 @@ pub(super) fn uninstall(command: UninstallCommand) -> Result }; if agents.is_empty() { return Err(CliError::Install( - "no installed Claude Code, Codex, or Hermes integration state was found".into(), + "no installed Claude Code or Codex integration state was found".into(), )); } run_agent_operations(agents, "uninstall", |agent| { diff --git a/crates/cli/src/commands/mod.rs b/crates/cli/src/commands/mod.rs index 3b1d3bcf8..11b465af4 100644 --- a/crates/cli/src/commands/mod.rs +++ b/crates/cli/src/commands/mod.rs @@ -171,7 +171,6 @@ async fn run_command( Command::Run(command) => run::execute(command, server).await, Command::Claude(command) => run::easy_path(CodingAgent::ClaudeCode, command, server).await, Command::Codex(command) => run::easy_path(CodingAgent::Codex, command, server).await, - Command::Hermes(command) => run::easy_path(CodingAgent::Hermes, command, server).await, Command::Mcp => mcp::execute(server).await, Command::Config(command) => configure::execute(command, server).await, Command::Plugins(command) => plugins::execute(command, server), diff --git a/crates/cli/src/commands/root.rs b/crates/cli/src/commands/root.rs index 9a64bc01f..79e9ffaf4 100644 --- a/crates/cli/src/commands/root.rs +++ b/crates/cli/src/commands/root.rs @@ -21,7 +21,6 @@ pub(crate) enum AgentArg { #[value(name = "claude", alias = "claude-code")] Claude, Codex, - Hermes, } impl From for CodingAgent { @@ -29,7 +28,6 @@ impl From for CodingAgent { match value { AgentArg::Claude => Self::ClaudeCode, AgentArg::Codex => Self::Codex, - AgentArg::Hermes => Self::Hermes, } } } @@ -76,18 +74,6 @@ pub(crate) enum Command { nemo-relay --openai-base-url https://inference-api.nvidia.com codex" )] Codex(EasyPathCommand), - /// Run Hermes with observability (setup on first use) - #[command( - long_about = "Run Hermes Agent under an ephemeral NeMo Relay gateway. The wrapper uses a \ - process-private HERMES_HOME overlay for dynamic hooks, without rewriting \ - the user's Hermes configuration. Use `nemo-relay install hermes` when bare \ - Hermes processes should load the shared native Relay gateway on \ - 127.0.0.1:47632 through MCP.", - after_help = "Examples:\n \ - nemo-relay hermes\n \ - nemo-relay hermes -- chat --provider custom" - )] - Hermes(EasyPathCommand), /// Keep a shared Relay gateway ready for an MCP client. #[command( long_about = "Start or reuse a shared native NeMo Relay gateway for an MCP stdio \ @@ -130,7 +116,6 @@ impl Command { match self { Self::Claude(_) => "claude", Self::Codex(_) => "codex", - Self::Hermes(_) => "hermes", Self::Mcp => "mcp", Self::Config(_) => "config", Self::Plugins(_) => "plugins", diff --git a/crates/cli/src/configuration/mod.rs b/crates/cli/src/configuration/mod.rs index 6c3a1ff09..9c7dbf2ca 100644 --- a/crates/cli/src/configuration/mod.rs +++ b/crates/cli/src/configuration/mod.rs @@ -74,19 +74,18 @@ struct FileUpstreamConfig { } #[derive(Debug, Clone, Default, Deserialize)] +#[serde(deny_unknown_fields)] struct FileAgentsConfig { - // Keys match the agent's CLI invocation name (`claude`, `codex`, `hermes`) — the + // Keys match the agent's CLI invocation name (`claude`, `codex`) — the // word the user types at the shell — not the product name ("Claude Code") or the internal // `CodingAgent` enum kebab spelling. Same convention as the bare-agent shortcut in Phase 2. claude: Option, codex: Option, - hermes: Option, } #[derive(Debug, Clone, Default, Deserialize)] struct FileAgentCommandConfig { command: Option, - hooks_path: Option, } /// Resolves server-mode configuration from shared config files plus server CLI/environment overrides. @@ -1488,10 +1487,6 @@ fn apply_file_agents_config(agents: &mut AgentConfigs, file_agents: Option, - /// Legacy Hermes config-path override retained for existing Relay configuration files. - pub(crate) hooks_path: Option, } impl Default for GatewayConfig { diff --git a/crates/cli/src/events/mod.rs b/crates/cli/src/events/mod.rs index f51e54adb..f172164ab 100644 --- a/crates/cli/src/events/mod.rs +++ b/crates/cli/src/events/mod.rs @@ -9,7 +9,6 @@ pub(crate) mod json_path; pub(crate) enum AgentKind { Codex, ClaudeCode, - Hermes, Gateway, } @@ -20,7 +19,6 @@ impl AgentKind { match self { Self::Codex => "codex", Self::ClaudeCode => "claude-code", - Self::Hermes => "hermes", Self::Gateway => "gateway", } } @@ -40,8 +38,6 @@ pub(crate) enum NormalizedEvent { SubagentStarted(SubagentEvent), SubagentEnded(SubagentEvent), LlmHint(LlmHintEvent), - LlmStarted(LlmEvent), - LlmEnded(LlmEvent), ToolStarted(ToolEvent), ToolEnded(ToolEvent), #[allow(dead_code)] @@ -68,7 +64,6 @@ impl NormalizedEvent { | Self::Notification(event) | Self::HookMark(event) => &event.session_id, Self::LlmHint(event) => &event.session_id, - Self::LlmStarted(event) | Self::LlmEnded(event) => &event.session_id, Self::SubagentStarted(event) | Self::SubagentEnded(event) => &event.session_id, Self::ToolStarted(event) | Self::ToolEnded(event) => &event.session_id, } @@ -78,7 +73,7 @@ impl NormalizedEvent { // TurnEnded is intentionally NOT terminal — the agent scope stays open across turns. matches!( self, - Self::AgentEnded(_) | Self::SubagentEnded(_) | Self::LlmEnded(_) | Self::ToolEnded(_) + Self::AgentEnded(_) | Self::SubagentEnded(_) | Self::ToolEnded(_) ) } } @@ -118,19 +113,6 @@ pub(crate) struct LlmHintEvent { pub(crate) metadata: Value, } -#[derive(Debug, Clone, PartialEq)] -pub(crate) struct LlmEvent { - pub(crate) session_id: String, - pub(crate) agent_kind: AgentKind, - pub(crate) event_name: String, - pub(crate) api_call_id: String, - pub(crate) provider: String, - pub(crate) model_name: Option, - pub(crate) request: Value, - pub(crate) response: Value, - pub(crate) metadata: Value, -} - #[derive(Debug, Clone, PartialEq)] pub(crate) struct ToolEvent { pub(crate) session_id: String, diff --git a/crates/cli/src/filesystem/mod.rs b/crates/cli/src/filesystem/mod.rs index 8088d63dd..f82564170 100644 --- a/crates/cli/src/filesystem/mod.rs +++ b/crates/cli/src/filesystem/mod.rs @@ -7,7 +7,6 @@ mod atomic; pub(crate) mod bounded; mod locks; mod snapshots; -pub(crate) mod temp; #[cfg(test)] pub(crate) use atomic::fail_next_atomic_write; diff --git a/crates/cli/src/filesystem/temp.rs b/crates/cli/src/filesystem/temp.rs deleted file mode 100644 index f0f69ed93..000000000 --- a/crates/cli/src/filesystem/temp.rs +++ /dev/null @@ -1,39 +0,0 @@ -// SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. -// SPDX-License-Identifier: Apache-2.0 - -use std::path::{Path, PathBuf}; - -use crate::error::CliError; - -pub(crate) fn private_temp_dir(parent: &Path, prefix: &str) -> Result { - let path = parent.join(format!("{prefix}-{}", uuid::Uuid::now_v7())); - #[cfg(unix)] - let builder = { - use std::os::unix::fs::DirBuilderExt; - let mut builder = std::fs::DirBuilder::new(); - builder.mode(0o700); - builder - }; - #[cfg(not(unix))] - let builder = std::fs::DirBuilder::new(); - builder.create(&path)?; - #[cfg(windows)] - if let Err(error) = crate::filesystem::protect_private_windows_path(&path) { - let cleanup = std::fs::remove_dir(&path); - return Err(CliError::Io(match cleanup { - Ok(()) => error, - Err(cleanup_error) => std::io::Error::new( - cleanup_error.kind(), - format!( - "{error}; additionally failed to remove {}: {cleanup_error}", - path.display() - ), - ), - })); - } - Ok(path) -} - -pub(crate) fn private_system_temp_dir(prefix: &str) -> Result { - private_temp_dir(&std::env::temp_dir(), prefix) -} diff --git a/crates/cli/src/hooks/encoding.rs b/crates/cli/src/hooks/encoding.rs index f9eaf5113..a704f2677 100644 --- a/crates/cli/src/hooks/encoding.rs +++ b/crates/cli/src/hooks/encoding.rs @@ -14,7 +14,7 @@ use base64::Engine; #[cfg(test)] pub(crate) fn generated_hooks(agent: CodingAgent, command: &str) -> Value { - generated_policy_hooks(agent, &GeneratedHookCommands::uniform(command)) + generated_policy_hooks(agent, &GeneratedHookCommands::new(command, command)) } #[derive(Debug, Clone, PartialEq, Eq)] @@ -33,11 +33,6 @@ impl GeneratedHookCommands { } } - pub(crate) fn uniform(command: impl Into) -> Self { - let command = command.into(); - Self::new(command.clone(), command) - } - pub(crate) fn for_event(&self, event: &str) -> &str { if event_requires_fail_closed(event) { &self.fail_closed @@ -46,12 +41,6 @@ impl GeneratedHookCommands { } } - pub(crate) fn contains(&self, command: &str) -> bool { - command == self.fail_open - || command == self.fail_closed - || self.legacy.as_deref() == Some(command) - } - pub(crate) fn legacy(&self) -> Option<&str> { self.legacy.as_deref() } @@ -61,11 +50,7 @@ pub(crate) fn generated_policy_hooks( agent: CodingAgent, commands: &GeneratedHookCommands, ) -> Value { - if agent.uses_direct_hook_entries() { - direct_hooks(agent.hook_events(), commands) - } else { - grouped_hooks(agent.hook_events(), commands) - } + grouped_hooks(agent.hook_events(), commands) } /// Canonical persistent hook command used by every supported host. @@ -319,9 +304,7 @@ pub(super) fn safe_windows_launcher_token(launcher: &str) -> bool { } /// Decode only the exact PowerShell envelope emitted by [`encoded_windows_hook_command`]. -/// -/// Hermes uses this to migrate and replace Relay-owned hooks whose generation arguments change. -#[cfg(any(windows, test))] +#[cfg(test)] pub(crate) fn decode_windows_hook_command(command: &str) -> Option> { const COMMAND_SEPARATOR: &str = " -NoLogo -NoProfile -NonInteractive -EncodedCommand "; const SCRIPT_PREFIX: &str = "$ErrorActionPreference='Stop'; & "; @@ -360,7 +343,7 @@ pub(crate) fn decode_windows_hook_command(command: &str) -> Option> parse_powershell_single_quoted_arguments(invocation) } -#[cfg(any(windows, test))] +#[cfg(test)] pub(super) fn parse_powershell_single_quoted_arguments(mut raw: &str) -> Option> { let mut arguments = Vec::new(); while !raw.is_empty() { @@ -389,22 +372,6 @@ pub(super) fn parse_powershell_single_quoted_arguments(mut raw: &str) -> Option< (!arguments.is_empty()).then_some(arguments) } -fn direct_hooks(events: &[&str], commands: &GeneratedHookCommands) -> Value { - let hooks: serde_json::Map = events - .iter() - .map(|event| { - ( - (*event).to_string(), - json!([{ - "command": commands.for_event(event), - "timeout": 30 - }]), - ) - }) - .collect(); - json!({ "hooks": Value::Object(hooks) }) -} - // Generates hook groups for Claude/Codex events and adds a wildcard matcher to tool events when // the target agent requires matcher-scoped tool hooks. Non-tool events omit matchers so they fire // for the full lifecycle. diff --git a/crates/cli/src/hooks/mod.rs b/crates/cli/src/hooks/mod.rs index d21efe041..5aceb355f 100644 --- a/crates/cli/src/hooks/mod.rs +++ b/crates/cli/src/hooks/mod.rs @@ -6,6 +6,7 @@ mod delivery; mod destination; mod encoding; +#[cfg(test)] mod merging; mod response; mod types; @@ -19,7 +20,7 @@ pub(crate) use delivery::{gateway_headers, insert_header, read_hook_payload_from pub(crate) use destination::{ HookGatewayLifecycle, resolve_hook_destination, transparent_gateway_spec, }; -#[cfg(any(windows, test))] +#[cfg(test)] pub(crate) use encoding::decode_windows_hook_command; #[cfg(all(test, windows))] pub(crate) use encoding::windows_powershell_path; @@ -32,6 +33,7 @@ pub(crate) use encoding::{ encoded_windows_hook_command, event_matches_tools, event_requires_fail_closed, generated_hooks, persistent_hook_forward_commands_for_platform, transparent_hook_forward_commands_for_platform, }; +#[cfg(test)] pub(crate) use merging::merge_hooks; #[cfg(test)] pub(crate) use response::{handle_hook_forward_status, handle_verified_hook_forward_response}; diff --git a/crates/cli/src/installation/generation.rs b/crates/cli/src/installation/generation.rs index 822d548e9..e842e4f94 100644 --- a/crates/cli/src/installation/generation.rs +++ b/crates/cli/src/installation/generation.rs @@ -366,6 +366,7 @@ impl GenerationRetirement { Ok(()) } + #[cfg(test)] pub(crate) fn acquire(path: &Path) -> Result, String> { Self::acquire_with_timeout(path, DEFAULT_GENERATION_LOCK_TIMEOUT) } @@ -377,6 +378,7 @@ impl GenerationRetirement { Self::acquire_impl(path, DEFAULT_GENERATION_LOCK_TIMEOUT, Some(external_lock)) } + #[cfg(test)] pub(crate) fn acquire_with_timeout( path: &Path, timeout: Duration, @@ -868,7 +870,7 @@ fn open_generation_lock_path(lock_path: &Path) -> Result { fn open_marker_generation_lock(marker_path: &Path, lock_path: &Path) -> Result { // Legacy one-line markers derive a sibling lock. Creating only that deterministic path keeps - // old installs and Hermes upgrades compatible without allowing a marker to create an + // old installs compatible without allowing a marker to create an // arbitrary external file. New plugin markers always point at a pre-initialized state lock. if is_legacy_sibling_lock(marker_path, lock_path)? { open_generation_lock_path(lock_path) diff --git a/crates/cli/src/mcp/mod.rs b/crates/cli/src/mcp/mod.rs index 558cbb3d3..9fbff2922 100644 --- a/crates/cli/src/mcp/mod.rs +++ b/crates/cli/src/mcp/mod.rs @@ -106,8 +106,8 @@ pub(crate) async fn run(server_args: &GatewayOverrides) -> Result Result { - execute_live_run_with_dynamic(listener, gateway_config, Vec::new(), gateway_url, prepared).await -} - async fn execute_live_run_with_dynamic( listener: TcpListener, gateway_config: GatewayConfig, @@ -281,7 +269,7 @@ fn resolve_agent_invocation( let argv = command.command.clone(); let agent = CodingAgent::infer(&argv[0]).ok_or_else(|| { CliError::Launch(format!( - "could not infer coding agent from command {:?}; pass --agent claude, --agent codex, or --agent hermes", + "could not infer coding agent from command {:?}; pass --agent claude or --agent codex", argv[0] )) })?; @@ -529,12 +517,6 @@ impl PreparedAgentLaunch { // overriding `model_providers.openai`. Uses `features.hooks=true` introduced in codex-cli // current supported Codex releases. The centralized host policy validates the version first. - // Hermes discovers hooks from `.hermes/config.yaml` instead of command-line flags. A - // process-private HERMES_HOME exposes dynamic hooks without rewriting user configuration. - - // Records the Hermes hook file that would be patched during a real run without touching the - // filesystem, preserving dry-run as an inspection-only operation. - // Spawns the prepared child process with injected environment. // Stdio is inherited by default so agent interaction remains unchanged in transparent mode. async fn spawn(&self) -> Result { @@ -907,9 +889,6 @@ fn path_with_transparent_hook_dir() -> Option { // it here prevents a prompt token named `codex` or `claude` from becoming an accidental insertion // target while preserving configured wrapper prefixes. -// Chooses the Hermes config used as the source for a transparent-run overlay. If setup recorded a -// specific path, reuse it; otherwise fall back to the active Hermes home. - // Converts JSON hook groups into inline TOML arrays for Codex `--config` flags. The function // preserves matchers when present and assumes generated hook groups contain one command hook. diff --git a/crates/cli/src/server/mod.rs b/crates/cli/src/server/mod.rs index c4ca3ea62..157a0352d 100644 --- a/crates/cli/src/server/mod.rs +++ b/crates/cli/src/server/mod.rs @@ -41,7 +41,7 @@ use subtle::ConstantTimeEq; use tokio::net::TcpListener; use tokio::sync::oneshot; -use crate::agents::shared::adapters::{claude_code, codex, hermes}; +use crate::agents::shared::adapters::{claude_code, codex}; use crate::configuration::{ BOOTSTRAP_CLIENT_TOKEN_HEADER, BootstrapChallengeKey, GatewayConfig, ManagedBootstrapIdentity, }; @@ -586,7 +586,6 @@ fn router_with_state(state: AppState) -> Router { .route("/bootstrap/shutdown", post(shutdown_bootstrap_sidecar)) .route("/hooks/codex", post(codex_hook)) .route("/hooks/claude-code", post(claude_code_hook)) - .route("/hooks/hermes", post(hermes_hook)) .route("/responses", post(gateway::passthrough)) .route("/chat/completions", post(gateway::passthrough)) .route("/models", get(gateway::models)) @@ -1208,23 +1207,6 @@ async fn claude_code_hook( Ok(Json(outcome.response)) } -// Handles Hermes hook payloads from persistent shell integration. The adapter returns a minimal -// body because hook-forward owns the fail-open/fail-closed behavior for Hermes command execution. -async fn hermes_hook( - State(state): State, - headers: HeaderMap, - payload: Result, JsonRejection>, -) -> Result, CliError> { - state.touch(); - let Json(payload) = payload.map_err(hook_payload_rejection)?; - let outcome = hermes::adapt(payload, &headers); - state - .sessions - .apply_events(&headers, outcome.events) - .await?; - Ok(Json(outcome.response)) -} - fn hook_payload_rejection(rejection: JsonRejection) -> CliError { if rejection.status() == axum::http::StatusCode::PAYLOAD_TOO_LARGE { CliError::PayloadTooLarge(rejection.to_string()) diff --git a/crates/cli/src/sessions/correlation.rs b/crates/cli/src/sessions/correlation.rs index 30929e57e..495c1a6aa 100644 --- a/crates/cli/src/sessions/correlation.rs +++ b/crates/cli/src/sessions/correlation.rs @@ -307,7 +307,6 @@ pub(super) fn event_agent_kind(event: &NormalizedEvent) -> AgentKind { NormalizedEvent::SubagentStarted(event) | NormalizedEvent::SubagentEnded(event) => { event.agent_kind } - NormalizedEvent::LlmStarted(event) | NormalizedEvent::LlmEnded(event) => event.agent_kind, NormalizedEvent::ToolStarted(event) | NormalizedEvent::ToolEnded(event) => event.agent_kind, } } diff --git a/crates/cli/src/sessions/mod.rs b/crates/cli/src/sessions/mod.rs index 5f5510966..a68ebeca3 100644 --- a/crates/cli/src/sessions/mod.rs +++ b/crates/cli/src/sessions/mod.rs @@ -6,9 +6,9 @@ use std::sync::Arc; use std::time::{Duration, Instant}; use axum::http::HeaderMap; -use nemo_relay::api::llm::{ - LlmAttributes, LlmCallEndParams, LlmCallParams, LlmHandle, LlmRequest, llm_call, llm_call_end, -}; +use nemo_relay::api::llm::{LlmAttributes, LlmCallEndParams, LlmHandle, LlmRequest, llm_call_end}; +#[cfg(test)] +use nemo_relay::api::llm::{LlmCallParams, llm_call}; use nemo_relay::api::runtime::{ ScopeStackHandle, TASK_SCOPE_STACK, create_scope_stack, task_scope_push, }; @@ -41,7 +41,7 @@ use routing::*; pub(crate) use types::*; use crate::events::{ - AgentKind, LlmEvent, LlmHintEvent, NormalizedEvent, SessionEvent, SubagentEvent, ToolEvent, + AgentKind, LlmHintEvent, NormalizedEvent, SessionEvent, SubagentEvent, ToolEvent, }; const LLM_HINT_TTL: Duration = Duration::from_secs(300); @@ -532,9 +532,9 @@ impl SessionManager { /// Returns true while any session still owns active observable work. /// - /// Host sessions can remain durable after their current turn ends: Codex may omit `SessionEnd`, - /// while Hermes keeps a session open for later resumption. A dormant agent scope must therefore - /// not keep the MCP-managed sidecar alive forever. Active turns, subagents, tools, LLMs, and + /// Host sessions can remain durable after their current turn ends because Codex may omit + /// `SessionEnd`. A dormant agent scope must therefore not keep the MCP-managed sidecar alive + /// forever. Active turns, subagents, tools, LLMs, and /// gateway calls still block idle shutdown; [`Self::close_all`] balances the dormant agent scope /// when the gateway exits. pub(crate) async fn has_open_sessions(&self) -> bool { @@ -788,8 +788,6 @@ impl Session { NormalizedEvent::SubagentStarted(event) => self.start_subagent(event).await, NormalizedEvent::SubagentEnded(event) => self.end_subagent(event).await, NormalizedEvent::LlmHint(event) => self.add_llm_hint(event), - NormalizedEvent::LlmStarted(event) => self.start_hook_llm(event).await, - NormalizedEvent::LlmEnded(event) => self.end_hook_llm(event).await, NormalizedEvent::ToolStarted(event) => self.start_tool(event).await, NormalizedEvent::ToolEnded(event) => self.end_tool(event).await, NormalizedEvent::PromptSubmitted(event) => self.start_turn(event).await, @@ -1428,90 +1426,6 @@ impl Session { Ok(()) } - // Starts an LLM call from hook activity such as Hermes API request hooks. Duplicate call IDs are - // ignored so repeated pre hooks do not create parallel handles for one provider call. Aliased - // child-session LLMs carry their subagent owner in metadata and are resolved by - // `hook_llm_owner`. - async fn start_hook_llm(&mut self, event: LlmEvent) -> Result<(), CliError> { - self.ensure_turn_started(event.metadata.clone())?; - if self.llms.contains_key(&event.api_call_id) { - return Ok(()); - } - let (parent, metadata) = self.hook_llm_owner(event.metadata); - let handle = llm_call( - LlmCallParams::builder() - .name(event.provider.as_str()) - .request(&LlmRequest { - headers: Map::new(), - content: event.request, - }) - .parent_opt(parent.as_ref()) - .attributes(LlmAttributes::empty()) - .metadata(metadata) - .model_name_opt(event.model_name) - .build(), - )?; - self.llms.insert(event.api_call_id, handle); - Ok(()) - } - - // Ends a hook-observed LLM call, synthesizing a start if only the post hook arrives. The same - // alias metadata recovery used by `start_hook_llm` keeps post-only aliased child LLMs under the - // subagent instead of falling back to the root agent. - async fn end_hook_llm(&mut self, event: LlmEvent) -> Result<(), CliError> { - self.ensure_turn_started(event.metadata.clone())?; - let (parent, metadata) = self.hook_llm_owner(event.metadata); - let handle = match self.llms.remove(&event.api_call_id) { - Some(handle) => handle, - None => llm_call( - LlmCallParams::builder() - .name(event.provider.as_str()) - .request(&LlmRequest { - headers: Map::new(), - content: event.request, - }) - .parent_opt(parent.as_ref()) - .attributes(LlmAttributes::empty()) - .metadata(metadata.clone()) - .model_name_opt(event.model_name.clone()) - .build(), - )?, - }; - let output = event.response; - let root_owned = - json_string_at(&metadata, &[&["llm_correlation_subagent_id"][..]]).is_none(); - if root_owned { - self.record_turn_llm_output(output.clone()); - } - llm_call_end( - LlmCallEndParams::builder() - .handle(&handle) - .response(output) - .metadata(metadata) - .build(), - )?; - Ok(()) - } - - // Recovers owner information stamped by alignment when a hook-originated LLM event came from - // an aliased child session. Gateway LLM calls have first-class owner resolution, but hook LLM - // events only carry metadata, so this is the bridge that keeps aliased child LLMs under the - // subagent instead of the root agent. - fn hook_llm_owner(&mut self, metadata: Value) -> (Option, Value) { - let Some(subagent_id) = json_string_at(&metadata, &[&["llm_correlation_subagent_id"][..]]) - else { - return (self.root_work_scope(), metadata); - }; - let Some(scope) = self.subagents.get(&subagent_id).cloned() else { - return (self.root_work_scope(), metadata); - }; - self.set_last_llm_owner(Some(subagent_id.clone())); - ( - Some(scope), - merge_metadata(metadata, self.subagent_llm_metadata(&subagent_id)), - ) - } - // Starts a tool call under an explicit subagent when available, otherwise under the turn // scope. Duplicate tool IDs are ignored so repeated pre-tool hooks do not create parallel // handles for one agent tool invocation. @@ -1623,8 +1537,8 @@ impl Session { Ok(()) } - // Hermes pre/post tool hooks can disagree on call IDs: pre hooks may omit the provider id - // while post hooks carry the final chat-completions tool id. When the ID misses but exactly + // Pre/post tool hooks can disagree on call IDs: pre hooks may omit the provider id while post + // hooks carry the final chat-completions tool id. When the ID misses but exactly // one active tool owned by the same subagent/root scope has the same name and arguments, close // that start instead of synthesizing a second zero-duration span. fn remove_tool_handle_for_event(&mut self, event: &ToolEvent) -> Option { diff --git a/crates/cli/src/sessions/routing.rs b/crates/cli/src/sessions/routing.rs index 44de606e0..f85e14a86 100644 --- a/crates/cli/src/sessions/routing.rs +++ b/crates/cli/src/sessions/routing.rs @@ -134,43 +134,12 @@ pub(super) fn route_event_for_session( sessions: &mut HashMap, alignment_state: &mut SessionAlignmentState, ) -> Option<(NormalizedEvent, String, bool)> { - let mut event = alignment_state.route_event(event); - let explicit_subagent_alias = alignment::explicit_subagent_alias(&mut event); + let event = alignment_state.route_event(event); let session_id = event.session_id().to_string(); let is_agent_started = matches!(&event, NormalizedEvent::AgentStarted(_)); if event.is_terminal() && !sessions.contains_key(&session_id) { return None; } - if !apply_explicit_subagent_alias( - &mut event, - sessions, - alignment_state, - explicit_subagent_alias, - ) { - return None; - } Some((event, session_id, is_agent_started)) } - -fn apply_explicit_subagent_alias( - event: &mut NormalizedEvent, - sessions: &mut HashMap, - alignment_state: &mut SessionAlignmentState, - explicit_subagent_alias: Option<(String, SessionAlias)>, -) -> bool { - let Some((child_session_id, alias)) = explicit_subagent_alias else { - alignment_state.align_explicit_subagent_end(event); - return true; - }; - if sessions - .get(&child_session_id) - .is_some_and(|session| !session.can_reparent_as_subagent_alias()) - { - return false; - } - sessions.remove(&child_session_id); - alignment_state.insert_alias(child_session_id, alias); - alignment_state.align_explicit_subagent_end(event); - true -} diff --git a/crates/cli/tests/architecture_tests.rs b/crates/cli/tests/architecture_tests.rs index fe8877b4f..edba893df 100644 --- a/crates/cli/tests/architecture_tests.rs +++ b/crates/cli/tests/architecture_tests.rs @@ -103,12 +103,12 @@ fn syntax_analysis_expands_grouped_imports_and_ignores_comments() { let paths = syntax_paths( r#" // use crate::commands::ignored; - use crate::{commands::install, agents::{codex, hermes as other}}; + use crate::{commands::install, agents::{codex, claude as other}}; "#, ); assert!(paths.contains(&"crate::commands::install".to_string())); assert!(paths.contains(&"crate::agents::codex".to_string())); - assert!(paths.contains(&"crate::agents::hermes".to_string())); + assert!(paths.contains(&"crate::agents::claude".to_string())); assert!(!paths.iter().any(|path| path.contains("ignored"))); } @@ -183,11 +183,7 @@ fn tests_are_not_embedded_in_the_source_tree() { #[test] fn agent_directories_do_not_import_one_another_or_commands() { let agents = source_root().join("agents"); - for (agent, forbidden) in [ - ("codex", ["agents::claude", "agents::hermes"]), - ("claude", ["agents::codex", "agents::hermes"]), - ("hermes", ["agents::codex", "agents::claude"]), - ] { + for (agent, forbidden) in [("codex", ["agents::claude"]), ("claude", ["agents::codex"])] { for path in rust_files(&agents.join(agent)) { let source = fs::read_to_string(&path).unwrap(); let paths = syntax_paths(&source); @@ -435,11 +431,7 @@ fn shared_runtime_subsystems_do_not_dispatch_host_variants() { ] { for path in rust_files(&src.join(subsystem)) { let source = fs::read_to_string(&path).unwrap(); - for marker in [ - "CodingAgent::Codex", - "CodingAgent::ClaudeCode", - "CodingAgent::Hermes", - ] { + for marker in ["CodingAgent::Codex", "CodingAgent::ClaudeCode"] { assert!( !source.contains(marker), "{} dispatches host variant {marker}", diff --git a/crates/cli/tests/cli_tests.rs b/crates/cli/tests/cli_tests.rs index 5ed7d6790..5dbc8aabb 100644 --- a/crates/cli/tests/cli_tests.rs +++ b/crates/cli/tests/cli_tests.rs @@ -550,19 +550,6 @@ fn cli_mcp_help_describes_lifecycle_bound_native_gateway() { assert!(stdout.contains("127.0.0.1:47632")); } -#[test] -fn cli_config_help_keeps_hermes_persistent_state_under_uninstall() { - let output = Command::new(gateway_bin()) - .args(["config", "--help"]) - .output() - .unwrap(); - - assert!(output.status.success()); - let stdout = String::from_utf8_lossy(&output.stdout); - assert!(stdout.contains("nemo-relay uninstall hermes")); - assert!(!stdout.contains("Hermes-scoped reset also removes")); -} - #[test] fn cli_mcp_starts_gateway_before_initialize_and_exits_cleanly() { let temp = tempfile::tempdir().unwrap(); @@ -686,140 +673,6 @@ fn cli_mcp_rejects_an_unauthenticated_transparent_gateway() { assert!(find_runtime_file(temp.path(), "codex.owner.json").is_none()); } -#[cfg(unix)] -#[test] -fn cli_internal_hermes_install_writes_mcp_hooks_trust_and_doctor_ready_state() { - use std::os::unix::fs::PermissionsExt; - - let temp = tempfile::tempdir().unwrap(); - let home = temp.path().join("home"); - let hermes_home = temp.path().join("hermes"); - let xdg = temp.path().join("xdg"); - let runtime = temp.path().join("runtime"); - let bin = temp.path().join("bin"); - for directory in [&home, &hermes_home, &xdg, &runtime, &bin] { - std::fs::create_dir_all(directory).unwrap(); - } - let hermes = bin.join("hermes"); - std::fs::write(&hermes, "#!/bin/sh\necho 'Hermes Agent v0.18.2 (test)'\n").unwrap(); - std::fs::set_permissions(&hermes, std::fs::Permissions::from_mode(0o755)).unwrap(); - std::os::unix::fs::symlink(gateway_bin(), bin.join("nemo-relay")).unwrap(); - let path = std::env::join_paths(std::iter::once(bin.clone()).chain(std::env::split_paths( - &std::env::var_os("PATH").unwrap_or_default(), - ))) - .unwrap(); - - let install = Command::new(gateway_bin()) - .args(["install", "hermes", "--skip-doctor"]) - .env("HOME", &home) - .env("HERMES_HOME", &hermes_home) - .env("XDG_CONFIG_HOME", &xdg) - .env("XDG_RUNTIME_DIR", &runtime) - .env("PATH", &path) - .env("OPENAI_API_KEY", "not-written-to-config") - .output() - .unwrap(); - assert!( - install.status.success(), - "{}", - String::from_utf8_lossy(&install.stderr) - ); - - let config_path = hermes_home.join("config.yaml"); - assert_hermes_install_config(&config_path, &hermes_home); - - let relay_config_dir = xdg.join("nemo-relay"); - std::fs::create_dir_all(&relay_config_dir).unwrap(); - std::fs::write( - relay_config_dir.join("config.toml"), - format!( - "[agents.hermes]\ncommand = {:?}\nhooks_path = {:?}\n", - hermes.display().to_string(), - config_path.display().to_string() - ), - ) - .unwrap(); - let doctor = Command::new(gateway_bin()) - .args(["doctor", "hermes", "--json"]) - .env("HOME", &home) - .env("HERMES_HOME", &hermes_home) - .env("XDG_CONFIG_HOME", &xdg) - .env("XDG_RUNTIME_DIR", &runtime) - .env("PATH", &path) - .env("OPENAI_API_KEY", "runtime-only") - .output() - .unwrap(); - assert!( - doctor.status.success(), - "{}", - String::from_utf8_lossy(&doctor.stderr) - ); - let report: serde_json::Value = serde_json::from_slice(&doctor.stdout).unwrap(); - assert_hermes_doctor_report(&report); - - let uninstall = Command::new(gateway_bin()) - .args(["uninstall", "hermes"]) - .env("HOME", &home) - .env("HERMES_HOME", &hermes_home) - .env("XDG_CONFIG_HOME", &xdg) - .env("PATH", &path) - .output() - .unwrap(); - assert!( - uninstall.status.success(), - "{}", - String::from_utf8_lossy(&uninstall.stderr) - ); - assert!(!config_path.exists()); - assert!(!hermes_home.join("shell-hooks-allowlist.json").exists()); - assert!(!hermes_home.join(".nemo-relay-generation").exists()); -} - -#[cfg(unix)] -fn assert_hermes_install_config(config_path: &std::path::Path, hermes_home: &std::path::Path) { - let config: serde_json::Value = - serde_yaml::from_str(&std::fs::read_to_string(config_path).unwrap()).unwrap(); - let server = &config["mcp_servers"]["nemo-relay"]; - assert_eq!(server["command"], gateway_bin()); - assert_eq!(server["args"], serde_json::json!(["mcp"])); - assert_eq!(server["env"]["NEMO_RELAY_GATEWAY_BIND"], "127.0.0.1:47632"); - assert_eq!(server["env"]["OPENAI_API_KEY"], "${OPENAI_API_KEY}"); - assert!( - !std::fs::read_to_string(config_path) - .unwrap() - .contains("not-written-to-config") - ); - let approvals: serde_json::Value = serde_json::from_str( - &std::fs::read_to_string(hermes_home.join("shell-hooks-allowlist.json")).unwrap(), - ) - .unwrap(); - let approvals = approvals["approvals"].as_array().unwrap(); - assert_eq!(approvals.len(), 13); - for approval in approvals { - let event = approval["event"].as_str().unwrap(); - let command = approval["command"].as_str().unwrap(); - assert!(command.contains("hook-forward hermes")); - assert_eq!(approval["command"], config["hooks"][event][0]["command"]); - if event == "pre_tool_call" { - assert!(command.ends_with(" --fail-closed")); - } else { - assert!(command.ends_with(" --fail-open")); - } - } -} - -#[cfg(unix)] -fn assert_hermes_doctor_report(report: &serde_json::Value) { - assert_eq!(report["agents"][0]["name"], "hermes"); - assert_eq!(report["agents"][0]["status"], "pass"); - assert!( - report["agents"][0]["annotation"] - .as_str() - .unwrap() - .contains("MCP lifecycle") - ); -} - fn start_mcp_client(temp: &std::path::Path, bind: SocketAddr) -> (Child, ChildStdin) { start_mcp_client_with_idle_timeout(temp, bind, "1") } @@ -888,7 +741,7 @@ fn start_mcp_client_with_generation( #[test] fn cli_hooks_and_mcp_share_the_same_persistent_identity_for_each_host() { - for agent in ["codex", "claude", "hermes"] { + for agent in ["codex", "claude"] { let temp = tempfile::tempdir().unwrap(); let probe = TcpListener::bind("127.0.0.1:0").unwrap(); let address = probe.local_addr().unwrap(); @@ -1928,7 +1781,11 @@ fn cli_agents_json_emits_supported_agent_shapes() { assert!(output.status.success()); let parsed: serde_json::Value = serde_json::from_slice(&output.stdout).unwrap(); let agents = parsed.as_array().unwrap(); - assert!(agents.iter().any(|agent| agent["name"] == "codex")); + let names = agents + .iter() + .map(|agent| agent["name"].as_str().unwrap()) + .collect::>(); + assert_eq!(names, std::collections::BTreeSet::from(["claude", "codex"])); assert!(agents.iter().all(|agent| agent["status"].is_string())); } @@ -2991,12 +2848,13 @@ fn cli_help_lists_easy_path_agent_shortcuts() { let output = Command::new(gateway_bin()).arg("--help").output().unwrap(); let stdout = String::from_utf8_lossy(&output.stdout); - for agent in ["claude", "codex", "hermes"] { + for agent in ["claude", "codex"] { assert!( stdout.contains(&format!(" {agent}")), "expected `--help` to list `{agent}` subcommand, got:\n{stdout}" ); } + assert!(!stdout.contains(" hermes")); assert!(!stdout.contains(" cursor")); } @@ -3016,6 +2874,32 @@ fn cli_rejects_removed_cursor_entry_points() { assert!(String::from_utf8_lossy(&output.stderr).contains("invalid value 'cursor'")); } +#[test] +fn cli_rejects_removed_hermes_entry_points() { + for arguments in [ + vec!["hermes"], + vec!["run", "--agent", "hermes", "--dry-run"], + vec!["install", "hermes", "--dry-run"], + vec!["uninstall", "hermes", "--dry-run"], + vec!["doctor", "--plugin", "hermes"], + vec!["config", "hermes"], + vec!["mcp", "--agent", "hermes"], + vec!["hook-forward", "hermes"], + ] { + let output = Command::new(gateway_bin()) + .args(&arguments) + .output() + .unwrap(); + assert_eq!( + output.status.code(), + Some(2), + "expected normal argument validation for {arguments:?}; stdout:\n{}\nstderr:\n{}", + String::from_utf8_lossy(&output.stdout), + String::from_utf8_lossy(&output.stderr) + ); + } +} + #[test] fn cli_rejects_removed_plugin_shim_entry_point() { let output = Command::new(gateway_bin()) @@ -3167,33 +3051,6 @@ fn cli_easy_path_invokes_setup_when_no_config_found() { ); } -#[test] -fn cli_hermes_easy_path_invokes_setup_when_no_config_found() { - let temp = tempfile::tempdir().unwrap(); - let xdg = temp.path().join("xdg"); - std::fs::create_dir_all(&xdg).unwrap(); - let cwd = temp.path().join("workdir"); - std::fs::create_dir_all(&cwd).unwrap(); - - let output = Command::new(gateway_bin()) - .current_dir(&cwd) - .env("XDG_CONFIG_HOME", &xdg) - .env("HOME", temp.path()) - .arg("hermes") - .output() - .unwrap(); - - assert!( - !output.status.success(), - "Hermes easy path should exit non-zero when no config + no TTY for setup" - ); - let stderr = String::from_utf8_lossy(&output.stderr); - assert!( - stderr.contains("setup requires a TTY"), - "expected non-TTY setup error in stderr, got:\n{stderr}" - ); -} - #[test] fn cli_bare_invocation_invokes_setup_when_no_config_found() { let temp = tempfile::tempdir().unwrap(); @@ -3769,7 +3626,7 @@ fn cli_plugin_doctor_is_not_preempted_by_a_missing_runtime_config() { assert!(!output.status.success()); let stderr = String::from_utf8_lossy(&output.stderr); - assert!(stderr.contains("no installed Claude Code, Codex, or Hermes integration state")); + assert!(stderr.contains("no installed Claude Code or Codex integration state")); assert!(!stderr.contains("explicit configuration file")); } @@ -3786,8 +3643,8 @@ fn cli_run_dry_run_resolves_config_and_command() { openai_base_url = "http://file-openai" anthropic_base_url = "http://file-anthropic" -[agents.hermes] -command = "hermes --yolo chat" +[agents.codex] +command = "codex exec" "#, ) .unwrap(); @@ -3801,7 +3658,7 @@ command = "hermes --yolo chat" config.to_str().unwrap(), "run", "--agent", - "hermes", + "codex", "--dry-run", ]) .output() @@ -3809,9 +3666,14 @@ command = "hermes --yolo chat" assert!(output.status.success()); let stdout = String::from_utf8_lossy(&output.stdout); - assert!(stdout.contains("agent = hermes")); + assert!(stdout.contains("agent = codex")); assert!(stdout.contains("openai_base_url = http://file-openai")); - assert!(stdout.contains("argv = hermes --yolo chat")); + let argv = stdout + .lines() + .find(|line| line.starts_with("argv = ")) + .expect("dry-run output should include argv"); + assert!(argv.starts_with("argv = codex "), "{stdout}"); + assert!(argv.ends_with(" exec"), "{stdout}"); } #[test] @@ -4705,39 +4567,11 @@ fn cli_hook_forward_bypasses_ambient_proxies_for_loopback_delivery() { assert!(received.recv_timeout(Duration::from_secs(2)).is_ok()); } -#[test] -fn cli_hook_forward_hermes_shell_hook_returns_empty_object() { - let (server_url, received) = spawn_single_request_server(200, r#"{}"#); - let mut child = Command::new(gateway_bin()) - .args(["hook-forward", "hermes", "--fail-closed"]) - .env("NEMO_RELAY_GATEWAY_URL", &server_url) - .stdin(Stdio::piped()) - .stdout(Stdio::piped()) - .stderr(Stdio::piped()) - .spawn() - .unwrap(); - child - .stdin - .take() - .unwrap() - .write_all(br#"{"session_id":"smoke-hermes","hook_event_name":"on_session_start"}"#) - .unwrap(); - let output = child.wait_with_output().unwrap(); - let request = received.recv_timeout(Duration::from_secs(2)).unwrap(); - - assert!(output.status.success()); - assert_eq!(String::from_utf8_lossy(&output.stdout).trim(), r#"{}"#); - assert!(request.contains("POST /hooks/hermes HTTP/1.1")); - assert!( - request.contains(r#"{"session_id":"smoke-hermes","hook_event_name":"on_session_start"}"#) - ); -} - #[test] fn cli_hook_forward_reports_http_failure_when_fail_closed() { let (server_url, received) = spawn_single_request_server(503, "unavailable"); let mut child = Command::new(gateway_bin()) - .args(["hook-forward", "hermes", "--fail-closed"]) + .args(["hook-forward", "codex", "--fail-closed"]) .env("NEMO_RELAY_GATEWAY_URL", &server_url) .stdin(Stdio::piped()) .stdout(Stdio::piped()) @@ -4749,7 +4583,7 @@ fn cli_hook_forward_reports_http_failure_when_fail_closed() { let request = received.recv_timeout(Duration::from_secs(2)).unwrap(); assert!(!output.status.success()); - assert!(request.contains("POST /hooks/hermes HTTP/1.1")); + assert!(request.contains("POST /hooks/codex HTTP/1.1")); assert!(String::from_utf8_lossy(&output.stderr).contains("HTTP 503")); } @@ -4801,7 +4635,7 @@ fn cli_hook_forward_bounds_responses_under_both_failure_policies() { spawn_single_request_server(200, "x".repeat(MAX_HOOK_RESPONSE_BYTES + 1)); let mut command = Command::new(gateway_bin()); command - .args(["hook-forward", "hermes"]) + .args(["hook-forward", "codex"]) .env("NEMO_RELAY_GATEWAY_URL", &server_url) .stdin(Stdio::piped()) .stdout(Stdio::piped()) diff --git a/crates/cli/tests/coverage/agents/adapters_tests.rs b/crates/cli/tests/coverage/agents/adapters_tests.rs index 65e0b73b6..7dc2cc576 100644 --- a/crates/cli/tests/coverage/agents/adapters_tests.rs +++ b/crates/cli/tests/coverage/agents/adapters_tests.rs @@ -5,7 +5,7 @@ use axum::http::HeaderMap; use serde_json::json; use super::*; -use crate::agents::shared::adapters::{claude_code, codex, hermes}; +use crate::agents::shared::adapters::{claude_code, codex}; #[test] fn maps_claude_canonical_tool_payload() { @@ -71,16 +71,6 @@ fn preserves_supported_coding_agent_skill_load_tool_arguments() { }), &HeaderMap::new(), ), - hermes::adapt( - json!({ - "session_id": "hermes-session", - "hook_event_name": "pre_tool_call", - "tool_name": "skill_view", - "tool_input": {"name": "review"}, - "extra": {"tool_call_id": "hermes-skill"} - }), - &HeaderMap::new(), - ), ]; let expected = [ @@ -89,7 +79,6 @@ fn preserves_supported_coding_agent_skill_load_tool_arguments() { "Bash", json!({"command": "cat /workspace/skills/review/SKILL.md"}), ), - ("skill_view", json!({"name": "review"})), ]; for (outcome, (tool_name, arguments)) in cases.into_iter().zip(expected) { match &outcome.events[0] { @@ -376,7 +365,10 @@ fn agent_extractors_keep_fallbacks_at_adapter_boundary() { } ); - assert!(session_id(payload, headers, extractor).starts_with("hook-")); + assert_eq!( + session_id_with_fallback(payload, headers, extractor, "hook-test"), + "hook-test" + ); assert_eq!(event_name(payload, extractor), "unknown"); let event = common_tool_event_with_fallback(payload, headers, kind, extractor, "hook-test"); @@ -398,12 +390,6 @@ fn agent_extractors_keep_fallbacks_at_adapter_boundary() { &payload, &headers, ); - assert_fallbacks( - &HERMES_PAYLOAD_EXTRACTOR, - AgentKind::Hermes, - &payload, - &headers, - ); } #[test] @@ -467,7 +453,6 @@ fn agent_extractors_prefer_extra_call_ids_over_structural_ids() { for extractor in [ &CLAUDE_CODE_PAYLOAD_EXTRACTOR as &dyn AgentPayloadExtractor, &CODEX_PAYLOAD_EXTRACTOR, - &HERMES_PAYLOAD_EXTRACTOR, ] { assert_eq!( extractor @@ -491,7 +476,6 @@ fn agent_extractors_keep_hook_event_name_precedence() { for extractor in [ &CLAUDE_CODE_PAYLOAD_EXTRACTOR as &dyn AgentPayloadExtractor, &CODEX_PAYLOAD_EXTRACTOR, - &HERMES_PAYLOAD_EXTRACTOR, ] { assert_eq!( extractor.event_name(&payload).as_deref(), @@ -544,45 +528,6 @@ fn codex_extractor_prefers_codex_specific_fields() { assert_eq!(tool_call.arguments, Some(json!({ "cmd": "cargo test" }))); } -#[test] -fn hermes_extractor_prefers_child_subagent_and_claude_session_header() { - let mut headers = HeaderMap::new(); - headers.insert( - "x-claude-code-session-id", - "claude-session".parse().unwrap(), - ); - let payload = json!({ - "subagent_id": "generic-subagent", - "child_subagent_id": "hermes-child" - }); - - assert_eq!( - HERMES_PAYLOAD_EXTRACTOR - .session_id(&payload, &headers) - .as_deref(), - Some("claude-session") - ); - assert_eq!( - HERMES_PAYLOAD_EXTRACTOR - .subagent_id(&payload, &headers) - .as_deref(), - Some("hermes-child") - ); - - let nested_payload = json!({ - "subagent": { "id": "nested-subagent" }, - "extra": { - "subagent_id": "extra-subagent" - } - }); - assert_eq!( - HERMES_PAYLOAD_EXTRACTOR - .subagent_id(&nested_payload, &headers) - .as_deref(), - Some("nested-subagent") - ); -} - #[test] fn codex_extractor_ignores_claude_session_header() { let mut headers = HeaderMap::new(); @@ -591,7 +536,7 @@ fn codex_extractor_ignores_claude_session_header() { "claude-session".parse().unwrap(), ); - // RelayOnly: unlike Claude Code and Hermes, Codex must not adopt the Claude + // RelayOnly: Codex must not adopt the Claude // installed-mode session header. With no native session id the extractor // returns None, and the adapter boundary applies the synthetic fallback. assert_eq!( @@ -635,408 +580,6 @@ fn keeps_codex_response_unwrapped() { assert_eq!(outcome.response, json!({})); } -#[test] -fn maps_hermes_shell_hook_tool_payload() { - let headers = HeaderMap::new(); - let outcome = hermes::adapt( - json!({ - "hook_event_name": "pre_tool_call", - "tool_name": "terminal", - "tool_input": { "command": "pwd" }, - "session_id": "hermes-session", - "extra": { - "task_id": "task-1", - "tool_call_id": "tool-1" - } - }), - &headers, - ); - - match &outcome.events[0] { - NormalizedEvent::ToolStarted(event) => { - assert_eq!(event.agent_kind, AgentKind::Hermes); - assert_eq!(event.session_id, "hermes-session"); - assert_eq!(event.tool_call_id, "tool-1"); - assert_eq!(event.tool_name, "terminal"); - assert_eq!(event.arguments, json!({ "command": "pwd" })); - } - event => panic!("unexpected event: {event:?}"), - } - assert_eq!(outcome.response, json!({})); -} - -#[test] -fn drops_uncorrelatable_hermes_pre_tool_call() { - let headers = HeaderMap::new(); - let outcome = hermes::adapt( - json!({ - "hook_event_name": "pre_tool_call", - "task_id": "task-1", - "tool_call_id": "toolcall-1", - "tool_name": "terminal", - "tool_input": { "command": "pwd" } - }), - &headers, - ); - - assert!(outcome.events.is_empty()); - assert_eq!(outcome.response, json!({})); -} - -#[test] -fn maps_hermes_subagent_child_identifiers() { - let headers = HeaderMap::new(); - let outcome = hermes::adapt( - json!({ - "hook_event_name": "subagent_start", - "session_id": "parent-session", - "extra": { - "child_session_id": "child-session", - "child_subagent_id": "sa-1", - "parent_turn_id": "turn-1" - } - }), - &headers, - ); - - match &outcome.events[0] { - NormalizedEvent::SubagentStarted(event) => { - assert_eq!(event.agent_kind, AgentKind::Hermes); - assert_eq!(event.session_id, "parent-session"); - assert_eq!(event.subagent_id, "sa-1"); - assert_eq!( - event.payload["extra"]["child_session_id"], - json!("child-session") - ); - } - event => panic!("unexpected event: {event:?}"), - } -} - -#[test] -fn maps_hermes_camel_case_child_subagent_identifiers() { - let headers = HeaderMap::new(); - - for payload in [ - json!({ - "hook_event_name": "subagent_start", - "session_id": "parent-session", - "childSubagentId": "sa-camel-top" - }), - json!({ - "hook_event_name": "subagent_start", - "session_id": "parent-session", - "extra": { - "childSubagentId": "sa-camel-extra" - } - }), - ] { - let expected = payload - .get("childSubagentId") - .or_else(|| payload.pointer("/extra/childSubagentId")) - .and_then(|value| value.as_str()) - .expect("test payload should include childSubagentId") - .to_string(); - let outcome = hermes::adapt(payload, &headers); - - match &outcome.events[0] { - NormalizedEvent::SubagentStarted(event) => { - assert_eq!(event.subagent_id, expected); - } - event => panic!("unexpected event: {event:?}"), - } - } -} - -#[test] -fn maps_hermes_real_session_boundary_without_closing_per_turn_end() { - let headers = HeaderMap::new(); - - let per_turn = hermes::adapt( - json!({ - "hook_event_name": "on_session_end", - "session_id": "hermes-session" - }), - &headers, - ); - // `on_session_end` is per-turn for hermes-agent, so it snapshots ATIF without becoming a - // user-visible system trajectory step. - assert_eq!(per_turn.events.len(), 1); - assert!(matches!(per_turn.events[0], NormalizedEvent::TurnEnded(_))); - - let finalized = hermes::adapt( - json!({ - "hook_event_name": "on_session_finalize", - "session_id": "hermes-session" - }), - &headers, - ); - assert_eq!(finalized.events.len(), 1); - assert!(matches!( - finalized.events[0], - NormalizedEvent::AgentEnded(_) - )); -} - -#[test] -fn maps_hermes_hook_event_name_and_subagent_from_extra_payload() { - let outcome = hermes::adapt( - json!({ - "session_id": "hermes-session", - "extra": { - "hook_event_name": "subagent_stop", - "subagent_id": "worker-1" - } - }), - &HeaderMap::new(), - ); - - match &outcome.events[0] { - NormalizedEvent::SubagentEnded(event) => { - assert_eq!(event.event_name, "subagent_stop"); - assert_eq!(event.subagent_id, "worker-1"); - assert_eq!(event.session_id, "hermes-session"); - } - event => panic!("unexpected event: {event:?}"), - } -} - -#[test] -fn maps_hermes_api_hooks_to_llm_lifecycle() { - let headers = HeaderMap::new(); - - let started = hermes::adapt( - json!({ - "hook_event_name": "pre_api_request", - "session_id": "hermes-session", - "extra": { - "task_id": "task-1", - "api_call_count": 2, - "model": "qwen", - "provider": "custom", - "base_url": "http://localhost:11434/v1", - "api_mode": "chat_completions", - "message_count": 3, - "tool_count": 1, - "approx_input_tokens": 12, - "request_char_count": 456, - "max_tokens": 1024 - } - }), - &headers, - ); - match &started.events[0] { - NormalizedEvent::LlmStarted(event) => { - assert_eq!(event.session_id, "hermes-session"); - assert_eq!(event.api_call_id, "hermes-session:task-1:2"); - assert_eq!(event.provider, "custom"); - assert_eq!(event.model_name.as_deref(), Some("qwen")); - assert_eq!(event.request["message_count"], json!(3)); - assert_eq!( - event.request["fidelity"]["provider_payload_exact"], - json!(false) - ); - assert_eq!(event.metadata["provider_payload_exact"], json!(false)); - } - event => panic!("unexpected event: {event:?}"), - } - - let ended = hermes::adapt( - json!({ - "hook_event_name": "post_api_request", - "session_id": "hermes-session", - "extra": { - "task_id": "task-1", - "api_call_count": 2, - "model": "qwen", - "response_model": "qwen", - "provider": "custom", - "api_duration": 0.25, - "finish_reason": "stop", - "usage": { - "prompt_tokens": 10, - "completion_tokens": 5, - "prompt_tokens_details": { "cached_tokens": 3 } - } - } - }), - &headers, - ); - match &ended.events[0] { - NormalizedEvent::LlmEnded(event) => { - assert_eq!(event.api_call_id, "hermes-session:task-1:2"); - assert_eq!(event.response["usage"]["prompt_tokens"], json!(10)); - assert_eq!(event.response["usage"]["completion_tokens"], json!(5)); - } - event => panic!("unexpected event: {event:?}"), - } -} - -#[test] -fn maps_hermes_exact_api_hook_payloads_to_llm_lifecycle() { - let headers = HeaderMap::new(); - - let started = hermes::adapt( - json!({ - "hook_event_name": "pre_api_request", - "session_id": "hermes-session", - "extra": { - "task_id": "task-1", - "api_request_id": "turn-1:api:2", - "api_call_count": 2, - "model": "qwen", - "provider": "custom", - "request": { - "method": "POST", - "body": { - "model": "qwen", - "messages": [ - { "role": "user", "content": "hello" } - ], - "tools": [ - { "type": "function", "function": { "name": "search_files" } } - ] - } - } - } - }), - &headers, - ); - match &started.events[0] { - NormalizedEvent::LlmStarted(event) => { - assert_eq!(event.api_call_id, "turn-1:api:2"); - assert_eq!(event.request["messages"][0]["content"], json!("hello")); - assert_eq!( - event.request["tools"][0]["function"]["name"], - json!("search_files") - ); - assert_eq!(event.metadata["provider_payload_exact"], json!(true)); - assert_eq!( - event.metadata["fidelity_source"], - json!("hermes_api_hooks_sanitized") - ); - } - event => panic!("unexpected event: {event:?}"), - } - - let ended = hermes::adapt( - json!({ - "hook_event_name": "post_api_request", - "session_id": "hermes-session", - "extra": { - "task_id": "task-1", - "api_request_id": "turn-1:api:2", - "api_call_count": 2, - "model": "qwen", - "response": { - "model": "qwen", - "finish_reason": "tool_calls", - "assistant_message": { - "role": "assistant", - "content": "", - "tool_calls": [ - { - "id": "call-1", - "type": "function", - "function": { - "name": "search_files", - "arguments": "{\"query\":\"needle\"}" - } - } - ] - }, - "usage": { - "prompt_tokens": 10, - "completion_tokens": 5 - } - } - } - }), - &headers, - ); - match &ended.events[0] { - NormalizedEvent::LlmEnded(event) => { - assert_eq!(event.api_call_id, "turn-1:api:2"); - assert_eq!(event.response["tool_calls"][0]["id"], json!("call-1")); - assert_eq!(event.response["usage"]["prompt_tokens"], json!(10)); - assert_eq!(event.metadata["provider_payload_exact"], json!(true)); - } - event => panic!("unexpected event: {event:?}"), - } -} - -#[test] -fn maps_hermes_api_request_error_to_llm_end() { - let outcome = hermes::adapt( - json!({ - "hook_event_name": "api_request_error", - "session_id": "hermes-session", - "extra": { - "task_id": "task-1", - "api_request_id": "turn-1:api:3", - "api_call_count": 3, - "model": "qwen", - "provider": "custom", - "status_code": 502, - "retry_count": 1, - "max_retries": 2, - "retryable": true, - "reason": "upstream", - "error": { - "type": "BadGateway", - "message": "gateway upstream error" - } - } - }), - &HeaderMap::new(), - ); - - match &outcome.events[0] { - NormalizedEvent::LlmEnded(event) => { - assert_eq!(event.api_call_id, "turn-1:api:3"); - assert_eq!(event.response["status_code"], json!(502)); - assert_eq!( - event.response["error"]["message"], - json!("gateway upstream error") - ); - assert_eq!(event.metadata["provider_payload_exact"], json!(false)); - } - event => panic!("unexpected event: {event:?}"), - } -} - -#[test] -fn maps_hermes_null_request_as_lossy_summary() { - let outcome = hermes::adapt( - json!({ - "hook_event_name": "pre_api_request", - "session_id": "hermes-session", - "extra": { - "task_id": "task-1", - "api_call_count": 4, - "model": "qwen", - "provider": "custom", - "request": null, - "message_count": 2 - } - }), - &HeaderMap::new(), - ); - - match &outcome.events[0] { - NormalizedEvent::LlmStarted(event) => { - assert_eq!(event.api_call_id, "hermes-session:task-1:4"); - assert_eq!(event.request["message_count"], json!(2)); - assert_eq!( - event.request["fidelity"]["provider_payload_exact"], - json!(false) - ); - assert_eq!(event.metadata["provider_payload_exact"], json!(false)); - } - event => panic!("unexpected event: {event:?}"), - } -} - #[test] fn normalizes_mark_style_events_and_header_session_ids() { let mut headers = HeaderMap::new(); @@ -1092,30 +635,6 @@ fn normalizes_mark_style_events_and_header_session_ids() { } } -#[test] -fn maps_hermes_llm_hooks_to_private_hints() { - let headers = HeaderMap::new(); - let outcome = hermes::adapt( - json!({ - "hook_event_name": "pre_llm_call", - "session_id": "hermes-session", - "model": "anthropic/claude-sonnet", - "request_id": "req-1" - }), - &headers, - ); - - match &outcome.events[0] { - NormalizedEvent::LlmHint(event) => { - assert_eq!(event.session_id, "hermes-session"); - assert_eq!(event.event_name, "pre_llm_call"); - assert_eq!(event.model.as_deref(), Some("anthropic/claude-sonnet")); - assert_eq!(event.request_id.as_deref(), Some("req-1")); - } - event => panic!("unexpected event: {event:?}"), - } -} - #[test] fn extracts_tool_fields_from_fallback_payload_shapes() { let headers = HeaderMap::new(); @@ -1240,59 +759,6 @@ fn codex_partial_tool_end_keeps_missing_fields_null() { } } -#[test] -fn hermes_partial_post_tool_payload_synthesizes_call_id_only() { - let outcome = hermes::adapt( - json!({ - "hook_event_name": "post_tool_call", - "session_id": "hermes-session", - "tool_name": "terminal" - }), - &HeaderMap::new(), - ); - - match &outcome.events[0] { - NormalizedEvent::ToolEnded(event) => { - assert_eq!(event.session_id, "hermes-session"); - assert!(event.tool_call_id.starts_with("tool-")); - assert_eq!(event.tool_name, "terminal"); - assert_eq!(event.arguments, json!(null)); - assert_eq!(event.result, json!(null)); - assert_eq!(event.status, None); - } - event => panic!("unexpected event: {event:?}"), - } -} - -#[test] -fn maps_hermes_post_tool_call_result_and_status_shapes() { - let outcome = hermes::adapt( - json!({ - "hook_event_name": "post_tool_call", - "session_id": "hermes-session", - "tool_call_id": "tool-1", - "tool_name": "terminal", - "tool_response": { "stdout": "/repo" }, - "decision": "allow" - }), - &HeaderMap::new(), - ); - - match &outcome.events[0] { - NormalizedEvent::ToolEnded(event) => { - assert_eq!(event.tool_call_id, "tool-1"); - assert_eq!(event.result, json!({ "stdout": "/repo" })); - assert_eq!(event.status.as_deref(), Some("allow")); - } - event => panic!("unexpected event: {event:?}"), - } -} - -/// Walks a payload-path precedence chain: each step asserts the expected -/// winner, then removes the winning key (from the payload root, or from its -/// `extra` object when the first tuple field is true) so the next candidate -/// takes over. Once every listed key is removed, the extraction must yield -/// nothing. fn assert_string_fallback_chain( payload: &mut serde_json::Value, chain: &[(bool, &str, &str)], @@ -1314,84 +780,6 @@ fn assert_string_fallback_chain( assert_eq!(extract(payload), None, "chain should be exhausted"); } -#[test] -fn hermes_tool_result_path_precedence_walks_fallback_chain() { - let headers = HeaderMap::new(); - let mut payload = json!({ - "tool_output": "from-tool-output", - "tool_response": "from-tool-response", - "output": "from-output", - "result": "from-result", - "extra": { - "tool_output": "from-extra-tool-output", - "result": "from-extra-result" - } - }); - - assert_string_fallback_chain( - &mut payload, - &[ - (false, "tool_output", "from-tool-output"), - (false, "tool_response", "from-tool-response"), - (false, "output", "from-output"), - (false, "result", "from-result"), - (true, "tool_output", "from-extra-tool-output"), - (true, "result", "from-extra-result"), - ], - |payload| { - HERMES_PAYLOAD_EXTRACTOR - .tool_call(payload, &headers, "post_tool_call") - .result - .map(|result| result.as_str().expect("string tool result").to_string()) - }, - ); -} - -#[test] -fn hermes_tool_status_prefers_explicit_fields_over_derived_status() { - let headers = HeaderMap::new(); - let mut payload = json!({ - "status": "success", - "decision": "block", - "permission": "deny" - }); - - assert_string_fallback_chain( - &mut payload, - &[ - (false, "status", "success"), - (false, "decision", "block"), - (false, "permission", "deny"), - ], - |payload| { - HERMES_PAYLOAD_EXTRACTOR - .tool_call(payload, &headers, "post_tool_call") - .status - }, - ); - - // Explicit status fields win over event-name-derived status; without them the conservative - // failure spellings still map to `error`. - assert_eq!( - HERMES_PAYLOAD_EXTRACTOR - .tool_call( - &json!({ "status": "success" }), - &headers, - "post_tool_call_failed" - ) - .status - .as_deref(), - Some("success") - ); - assert_eq!( - HERMES_PAYLOAD_EXTRACTOR - .tool_call(&json!({}), &headers, "post_tool_call_failed") - .status - .as_deref(), - Some("error") - ); -} - #[test] fn claude_extractor_reads_llm_hint_fields() { let headers = HeaderMap::new(); diff --git a/crates/cli/tests/coverage/agents/alignment_tests.rs b/crates/cli/tests/coverage/agents/alignment_tests.rs index 55a1c9151..e7f810a26 100644 --- a/crates/cli/tests/coverage/agents/alignment_tests.rs +++ b/crates/cli/tests/coverage/agents/alignment_tests.rs @@ -6,7 +6,7 @@ use nemo_relay::api::llm::LlmRequest; use serde_json::Map; use super::*; -use crate::events::{LlmEvent, LlmHintEvent}; +use crate::events::LlmHintEvent; fn session_event(session_id: &str, event_name: &str) -> SessionEvent { SessionEvent { @@ -46,20 +46,6 @@ fn llm_hint_event(session_id: &str) -> LlmHintEvent { } } -fn llm_event(session_id: &str, event_name: &str) -> LlmEvent { - LlmEvent { - session_id: session_id.into(), - agent_kind: AgentKind::Codex, - event_name: event_name.into(), - api_call_id: "api-call-1".into(), - provider: "openai.responses".into(), - model_name: Some("gpt-test".into()), - request: json!({ "input": "hello" }), - response: json!({ "output_text": "hi" }), - metadata: json!({ "event_metadata": event_name }), - } -} - fn tool_event(session_id: &str, event_name: &str) -> ToolEvent { ToolEvent { session_id: session_id.into(), @@ -76,41 +62,6 @@ fn tool_event(session_id: &str, event_name: &str) -> ToolEvent { } } -fn hermes_llm_event(session_id: &str, task_id: &str) -> NormalizedEvent { - NormalizedEvent::LlmStarted(LlmEvent { - session_id: session_id.into(), - agent_kind: AgentKind::Hermes, - event_name: "pre_api_request".into(), - api_call_id: format!("{session_id}:{task_id}:1"), - provider: "custom".into(), - model_name: Some("qwen".into()), - request: json!({ "extra": { "task_id": task_id } }), - response: Value::Null, - metadata: json!({ "event_metadata": "pre_api_request" }), - }) -} - -fn hermes_tool_event(task_id: &str, session_scope: Option<&str>) -> NormalizedEvent { - let mut payload = json!({ "extra": { "task_id": task_id } }); - if let Some(session_scope) = session_scope { - payload["extra"]["parent_session_id"] = json!(session_scope); - } - - NormalizedEvent::ToolStarted(ToolEvent { - session_id: task_id.into(), - agent_kind: AgentKind::Hermes, - event_name: "pre_tool_call".into(), - tool_call_id: format!("{task_id}:tool-1"), - tool_name: "read_file".into(), - subagent_id: None, - arguments: json!({ "path": "README.md" }), - result: Value::Null, - status: None, - payload, - metadata: json!({ "event_metadata": "pre_tool_call" }), - }) -} - fn aliases() -> HashMap { HashMap::from([( "child".into(), @@ -122,45 +73,6 @@ fn aliases() -> HashMap { )]) } -#[test] -fn hermes_task_session_routing_is_scoped_by_parent_session() { - let mut state = SessionAlignmentState::default(); - - state.route_event(hermes_llm_event("hermes-a", "task-1")); - state.route_event(hermes_llm_event("hermes-b", "task-1")); - - let routed_a = state.route_event(hermes_tool_event("task-1", Some("hermes-a"))); - let NormalizedEvent::ToolStarted(routed_a) = routed_a else { - panic!("expected routed Hermes tool event"); - }; - assert_eq!(routed_a.session_id, "hermes-a"); - assert_eq!(routed_a.metadata["hermes_task_id"], json!("task-1")); - assert_eq!(routed_a.metadata["hermes_session_id"], json!("hermes-a")); - - let routed_b = state.route_event(hermes_tool_event("task-1", Some("hermes-b"))); - let NormalizedEvent::ToolStarted(routed_b) = routed_b else { - panic!("expected routed Hermes tool event"); - }; - assert_eq!(routed_b.session_id, "hermes-b"); - assert_eq!(routed_b.metadata["hermes_task_id"], json!("task-1")); - assert_eq!(routed_b.metadata["hermes_session_id"], json!("hermes-b")); -} - -#[test] -fn hermes_task_session_routing_leaves_ambiguous_unscoped_task_event_unchanged() { - let mut state = SessionAlignmentState::default(); - - state.route_event(hermes_llm_event("hermes-a", "task-1")); - state.route_event(hermes_llm_event("hermes-b", "task-1")); - - let routed = state.route_event(hermes_tool_event("task-1", None)); - let NormalizedEvent::ToolStarted(routed) = routed else { - panic!("expected Hermes tool event"); - }; - assert_eq!(routed.session_id, "task-1"); - assert!(routed.metadata.get("hermes_session_id").is_none()); -} - #[test] fn gateway_session_id_uses_explicit_claude_then_codex_fallbacks() { let mut headers = HeaderMap::new(); @@ -549,8 +461,6 @@ fn route_event_through_alias_covers_all_event_variants() { NormalizedEvent::SubagentStarted(subagent_event("child", "SubagentStart")), NormalizedEvent::SubagentEnded(subagent_event("child", "SubagentEnd")), NormalizedEvent::LlmHint(llm_hint_event("child")), - NormalizedEvent::LlmStarted(llm_event("child", "LlmStart")), - NormalizedEvent::LlmEnded(llm_event("child", "LlmEnd")), NormalizedEvent::ToolStarted(tool_event("child", "ToolStart")), NormalizedEvent::ToolEnded(tool_event("child", "ToolEnd")), ]; @@ -585,9 +495,7 @@ fn route_event_through_alias_covers_all_event_variants() { | NormalizedEvent::PromptSubmitted(_) | NormalizedEvent::Compaction(_) | NormalizedEvent::Notification(_) - | NormalizedEvent::HookMark(_) - | NormalizedEvent::LlmStarted(_) - | NormalizedEvent::LlmEnded(_) => {} + | NormalizedEvent::HookMark(_) => {} } } } @@ -770,7 +678,7 @@ fn gateway_turn_input_builds_claude_prompt_for_anthropic_messages_only() { // Only Claude installed mode can race the UserPromptSubmit hook, so every other agent kind // and route stays None. - for agent_kind in [AgentKind::Codex, AgentKind::Hermes, AgentKind::Gateway] { + for agent_kind in [AgentKind::Codex, AgentKind::Gateway] { assert_eq!( gateway_turn_input(agent_kind, "anthropic.messages", &request), None @@ -864,7 +772,6 @@ fn event_metadata(event: &NormalizedEvent) -> &Value { &event.metadata } NormalizedEvent::LlmHint(event) => &event.metadata, - NormalizedEvent::LlmStarted(event) | NormalizedEvent::LlmEnded(event) => &event.metadata, NormalizedEvent::ToolStarted(event) | NormalizedEvent::ToolEnded(event) => &event.metadata, } } diff --git a/crates/cli/tests/coverage/agents/coding_agent_tests.rs b/crates/cli/tests/coverage/agents/coding_agent_tests.rs index 8447d0273..deb81656b 100644 --- a/crates/cli/tests/coverage/agents/coding_agent_tests.rs +++ b/crates/cli/tests/coverage/agents/coding_agent_tests.rs @@ -10,22 +10,14 @@ fn agent_descriptors_are_complete_and_unique() { let executables = CodingAgent::ALL.map(CodingAgent::executable); let hook_paths = CodingAgent::ALL.map(CodingAgent::hook_path); - assert_eq!(arguments, ["claude", "codex", "hermes"]); - assert_eq!(install_arguments, ["claude-code", "codex", "hermes"]); - assert_eq!(executables, ["claude", "codex", "hermes"]); - assert_eq!( - hook_paths, - ["/hooks/claude-code", "/hooks/codex", "/hooks/hermes"] - ); + assert_eq!(arguments, ["claude", "codex"]); + assert_eq!(install_arguments, ["claude-code", "codex"]); + assert_eq!(executables, ["claude", "codex"]); + assert_eq!(hook_paths, ["/hooks/claude-code", "/hooks/codex"]); assert_eq!(CodingAgent::ClaudeCode.label(), "Claude Code"); assert_eq!(CodingAgent::Codex.label(), "Codex"); - assert_eq!(CodingAgent::Hermes.label(), "Hermes Agent"); assert_eq!(CodingAgent::ClaudeCode.hook_events().len(), 14); assert_eq!(CodingAgent::Codex.hook_events().len(), 10); - assert_eq!(CodingAgent::Hermes.hook_events().len(), 13); - assert!(!CodingAgent::ClaudeCode.uses_direct_hook_entries()); - assert!(!CodingAgent::Codex.uses_direct_hook_entries()); - assert!(CodingAgent::Hermes.uses_direct_hook_entries()); for agent in CodingAgent::ALL { let events = agent.hook_events(); assert!(events.iter().all(|event| !event.is_empty())); @@ -45,7 +37,6 @@ fn centralized_minimum_versions_accept_stable_boundaries() { let cases = [ (CodingAgent::ClaudeCode, "2.1.121 (Claude Code)"), (CodingAgent::Codex, "codex-cli 0.143.0"), - (CodingAgent::Hermes, "Hermes Agent v0.18.2 (2026.7.7.2)"), ]; for (agent, output) in cases { @@ -64,8 +55,6 @@ fn centralized_minimum_versions_reject_old_prerelease_and_malformed_output() { (CodingAgent::ClaudeCode, "2.1.121 (Other Agent)"), (CodingAgent::Codex, "codex-cli 0.142.9"), (CodingAgent::Codex, "codex-cli 0.143.0-alpha.1"), - (CodingAgent::Hermes, "Hermes Agent v0.18.1"), - (CodingAgent::Hermes, "Hermes Agent v0.18.2-rc.1"), ]; for (agent, output) in cases { @@ -97,9 +86,7 @@ fn agent_inference_accepts_supported_binary_aliases() { Some(CodingAgent::Codex) ); assert_eq!(CodingAgent::infer("@openai/codex"), None); - assert_eq!( - CodingAgent::infer("hermes-agent"), - Some(CodingAgent::Hermes) - ); + assert_eq!(CodingAgent::infer("hermes"), None); + assert_eq!(CodingAgent::infer("hermes-agent"), None); assert_eq!(CodingAgent::infer("unknown"), None); } diff --git a/crates/cli/tests/coverage/agents/hermes_tests.rs b/crates/cli/tests/coverage/agents/hermes_tests.rs deleted file mode 100644 index ede96d7fc..000000000 --- a/crates/cli/tests/coverage/agents/hermes_tests.rs +++ /dev/null @@ -1,1668 +0,0 @@ -// SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. -// SPDX-License-Identifier: Apache-2.0 - -use std::cell::Cell; -use std::ffi::OsString; -use std::path::Path; -use std::sync::MutexGuard; -use std::time::{Duration, UNIX_EPOCH}; - -use serde_json::{Value, json}; - -use super::*; -use crate::agents::CodingAgent; - -const TEST_GENERATION_TOKEN: &str = "test-generation"; - -fn relay_binary(root: &Path) -> PathBuf { - let path = root.join("NeMo Relay's bin").join("nemo-relay"); - std::fs::create_dir_all(path.parent().unwrap()).unwrap(); - std::fs::write(&path, b"relay").unwrap(); - #[cfg(unix)] - { - use std::os::unix::fs::PermissionsExt; - std::fs::set_permissions(&path, std::fs::Permissions::from_mode(0o755)).unwrap(); - } - path -} - -fn paths(root: &Path) -> PersistentPaths { - PersistentPaths::for_config(root.join("config.yaml")).unwrap() -} - -fn yaml(path: &Path) -> Value { - serde_yaml::from_str(&std::fs::read_to_string(path).unwrap()).unwrap() -} - -fn json_file(path: &Path) -> Value { - serde_json::from_str(&std::fs::read_to_string(path).unwrap()).unwrap() -} - -struct XdgConfigHomeScope { - _guard: MutexGuard<'static, ()>, - previous: Option, -} - -impl XdgConfigHomeScope { - fn enter(path: &Path) -> Self { - let guard = crate::test_support::ENV_TEST_LOCK - .lock() - .unwrap_or_else(|error| error.into_inner()); - let previous = std::env::var_os("XDG_CONFIG_HOME"); - // SAFETY: This scope holds the process-wide environment mutex. - unsafe { std::env::set_var("XDG_CONFIG_HOME", path) }; - Self { - _guard: guard, - previous, - } - } -} - -impl Drop for XdgConfigHomeScope { - fn drop(&mut self) { - // SAFETY: This restores the process environment while the mutex is still held. - unsafe { - match self.previous.take() { - Some(value) => std::env::set_var("XDG_CONFIG_HOME", value), - None => std::env::remove_var("XDG_CONFIG_HOME"), - } - } - } -} - -#[test] -fn user_config_path_uses_hermes_home_or_platform_home() { - let default_home = Path::new("/users/relay"); - assert_eq!( - user_config_path_with_override(default_home, None), - default_home.join(".hermes/config.yaml") - ); - assert_eq!( - user_config_path_with_override(default_home, Some("/profiles/hermes".into())), - Path::new("/profiles/hermes/config.yaml") - ); - assert_eq!( - user_config_path_with_override(default_home, Some("".into())), - default_home.join(".hermes/config.yaml") - ); -} - -#[test] -fn install_lock_serializes_concurrent_hermes_config_updates() { - let temp = tempfile::tempdir().unwrap(); - let config = temp.path().join("config.yaml"); - let _first = acquire_install_lock(&config, Duration::from_millis(10)).unwrap(); - - let error = acquire_install_lock(&config, Duration::ZERO).unwrap_err(); - - assert!( - error.contains("another Hermes integration update"), - "{error}" - ); -} - -#[test] -fn install_uses_the_native_hermes_allowlist_lock() { - let temp = tempfile::tempdir().unwrap(); - let allowlist = temp.path().join("shell-hooks-allowlist.json"); - let _first = acquire_allowlist_lock(&allowlist, Duration::from_millis(10)).unwrap(); - - let error = acquire_allowlist_lock(&allowlist, Duration::ZERO).unwrap_err(); - - assert!(error.contains("shell-hook approval update"), "{error}"); - assert!(temp.path().join("shell-hooks-allowlist.json.lock").exists()); -} - -#[test] -fn hook_command_round_trips_paths_and_platform_metacharacters() { - let relay = Path::new("/tmp/NeMo $Relay`test'/bin/nemo-relay"); - let generation = Path::new("/tmp/generation"); - assert_eq!( - persistent_hook_commands_for_platform(relay, generation, TEST_GENERATION_TOKEN, false) - .for_event("on_session_start"), - "'/tmp/NeMo $Relay`test'\\''/bin/nemo-relay' hook-forward hermes --gateway-url http://127.0.0.1:47632 --generation-file /tmp/generation --generation-token test-generation --fail-open" - ); - assert_eq!( - crate::hooks::decode_windows_hook_command( - persistent_hook_commands_for_platform( - Path::new(r"C:\Program Files\NeMo 100%\bin\nemo-relay.exe"), - Path::new(r"C:\Temp\generation"), - TEST_GENERATION_TOKEN, - true, - ) - .for_event("pre_tool_call") - ) - .unwrap(), - vec![ - r"C:\Program Files\NeMo 100%\bin\nemo-relay.exe", - "hook-forward", - "hermes", - "--gateway-url", - crate::bootstrap::DEFAULT_URL, - "--generation-file", - r"C:\Temp\generation", - "--generation-token", - TEST_GENERATION_TOKEN, - "--fail-closed", - ] - ); - assert_eq!( - crate::hooks::transparent_hook_forward_commands_for_platform( - relay, - CodingAgent::Hermes, - "http://127.0.0.1:1234", - false, - ) - .for_event("on_session_start"), - "'/tmp/NeMo $Relay`test'\\''/bin/nemo-relay' hook-forward hermes --gateway-url http://127.0.0.1:1234 --transparent-run --fail-open" - ); - let encoded = persistent_hook_commands_for_platform( - Path::new(r"C:\Program Files\NeMo 100%\bin\nemo-relay.exe"), - Path::new(r"C:\Temp\generation"), - TEST_GENERATION_TOKEN, - true, - ); - assert!(is_persistent_relay_hook_command( - encoded.for_event("pre_tool_call") - )); - let encoded_codex = crate::hooks::persistent_hook_forward_commands_for_platform( - Path::new(r"C:\Program Files\NeMo 100%\bin\nemo-relay.exe"), - CodingAgent::Codex, - Path::new(r"C:\Temp\generation"), - TEST_GENERATION_TOKEN, - true, - ); - assert_ne!(encoded, encoded_codex); - assert!(!is_persistent_relay_hook_command( - encoded_codex.for_event("PreToolUse") - )); -} - -#[test] -fn forwarded_environment_includes_static_dynamic_and_config_referenced_names() { - let environment = vec![ - "AWS_REGION".into(), - "NEMO_RELAY_CUSTOM".into(), - "NEMO_RELAY_WORKER_TOKEN".into(), - "UNRELATED_SECRET".into(), - ]; - let config = json!({ - "header_env": {"Authorization": "CUSTOM_EXPORT_TOKEN"}, - "secret_access_key_var": "AWS_PRIVATE_SECRET", - "session_token_var": "NEMO_RELAY_WORKER_TOKEN" - }); - let names = forwarded_environment_names(&environment, Some(&config)); - - assert!(names.contains(&"ANTHROPIC_API_KEY".into())); - assert!(names.contains(&"OPENAI_API_KEY".into())); - assert!(names.contains(&"AWS_REGION".into())); - assert!(names.contains(&"NEMO_RELAY_CUSTOM".into())); - assert!(names.contains(&"CUSTOM_EXPORT_TOKEN".into())); - assert!(names.contains(&"AWS_PRIVATE_SECRET".into())); - assert!(names.contains(&"AWS_PROFILE".into())); - assert!(names.contains(&"OTEL_EXPORTER_OTLP_ENDPOINT".into())); - assert!(!names.contains(&"NEMO_RELAY_WORKER_TOKEN".into())); - assert!(!names.contains(&"UNRELATED_SECRET".into())); -} - -#[test] -fn persistent_config_migrates_owned_state_and_preserves_unrelated_config() { - let temp = tempfile::tempdir().unwrap(); - let relay = relay_binary(temp.path()); - let generation = temp.path().join(GENERATION_FILE_NAME); - let command = persistent_hook_commands(&relay, &generation, TEST_GENERATION_TOKEN).unwrap(); - let legacy_command = format!("{} hook-forward hermes", relay.display()); - let mut legacy_hooks = serde_json::Map::new(); - for event in CodingAgent::Hermes.hook_events() { - legacy_hooks.insert(event.to_string(), json!([{"command": legacy_command}])); - } - legacy_hooks.insert( - "on_session_start".into(), - json!([ - {"command": "custom-hook", "timeout": 9}, - {"command": legacy_command, "timeout": 30} - ]), - ); - legacy_hooks.insert("custom_event".into(), json!([{"command": "keep-custom"}])); - let existing = serde_yaml::to_string(&json!({ - "model": "keep-me", - "mcp_servers": { - "filesystem": {"command": "fs-mcp"}, - MCP_SERVER_NAME: {"command": relay, "args": ["mcp", "--agent", "hermes"]} - }, - "hooks": legacy_hooks - })) - .unwrap(); - let merged = persistent_config( - Some(&existing), - &relay, - &command, - &generation, - TEST_GENERATION_TOKEN, - &["AWS_REGION".into()], - ) - .unwrap(); - - assert_eq!(merged["model"], json!("keep-me")); - assert_eq!( - merged["mcp_servers"]["filesystem"]["command"], - json!("fs-mcp") - ); - assert_eq!( - merged["mcp_servers"][MCP_SERVER_NAME], - expected_mcp_server( - &relay, - &generation, - TEST_GENERATION_TOKEN, - &["AWS_REGION".into()] - ) - ); - assert_eq!( - merged["mcp_servers"][MCP_SERVER_NAME]["env"]["AWS_REGION"], - json!("${AWS_REGION}") - ); - assert_eq!( - merged["hooks"]["on_session_start"] - .as_array() - .unwrap() - .len(), - 2 - ); - assert_eq!( - merged["hooks"]["on_session_start"][0]["command"], - json!("custom-hook") - ); - assert_eq!( - merged["hooks"]["on_session_start"][1]["command"], - json!(command.for_event("on_session_start")) - ); - assert_eq!( - merged["hooks"]["custom_event"][0]["command"], - json!("keep-custom") - ); - for event in CodingAgent::Hermes.hook_events() { - let groups = merged["hooks"][event].as_array().unwrap(); - assert_eq!( - groups - .iter() - .filter(|group| group["command"] == json!(command.for_event(event))) - .count(), - 1, - "event {event}" - ); - } -} - -#[test] -fn persistent_config_rejects_a_foreign_server_with_the_reserved_name() { - let temp = tempfile::tempdir().unwrap(); - let relay = relay_binary(temp.path()); - let generation = temp.path().join(GENERATION_FILE_NAME); - let command = persistent_hook_commands(&relay, &generation, TEST_GENERATION_TOKEN).unwrap(); - let existing = r#" -model: keep-me -mcp_servers: - nemo-relay: - command: foreign-mcp - args: [serve] -"#; - - let error = persistent_config( - Some(existing), - &relay, - &command, - &generation, - TEST_GENERATION_TOKEN, - &[], - ) - .unwrap_err() - .to_string(); - - assert!(error.contains("not managed by Relay"), "{error}"); - assert!(error.contains("rename or remove"), "{error}"); -} - -#[test] -fn manual_same_named_mcp_and_hooks_are_never_claimed() { - let temp = tempfile::tempdir().unwrap(); - let relay = relay_binary(temp.path()); - let generation = temp.path().join(GENERATION_FILE_NAME); - let command = persistent_hook_commands(&relay, &generation, TEST_GENERATION_TOKEN).unwrap(); - let manual = serde_yaml::to_string(&json!({ - "mcp_servers": { - MCP_SERVER_NAME: {"command": relay, "args": ["mcp"], "env": {"CUSTOM": "keep"}} - }, - "hooks": { - "on_session_start": [{"command": format!("{} hook-forward hermes", relay.display())}] - } - })) - .unwrap(); - - let error = persistent_config( - Some(&manual), - &relay, - &command, - &generation, - TEST_GENERATION_TOKEN, - &[], - ) - .unwrap_err() - .to_string(); - assert!(error.contains("not managed by Relay"), "{error}"); - - let paths = paths(&temp.path().join("hermes")); - std::fs::create_dir_all(paths.config.parent().unwrap()).unwrap(); - std::fs::write(&paths.config, &manual).unwrap(); - std::fs::write( - &paths.allowlist, - serde_json::to_vec(&json!({"approvals": [{ - "event": "on_session_start", - "command": format!("{} hook-forward hermes", relay.display()) - }]})) - .unwrap(), - ) - .unwrap(); - std::fs::write(&paths.generation, "orphaned-relay-state\n").unwrap(); - - uninstall_persistent_with(paths.clone(), atomic_write).unwrap(); - assert_eq!(std::fs::read_to_string(&paths.config).unwrap(), manual); - assert_eq!( - json_file(&paths.allowlist)["approvals"] - .as_array() - .unwrap() - .len(), - 1 - ); -} - -#[test] -fn modern_mcp_generation_proves_ownership_independently_of_hook_completeness() { - let temp = tempfile::tempdir().unwrap(); - let relay = relay_binary(temp.path()); - let generation = temp.path().join(GENERATION_FILE_NAME); - let mut root = persistent_config( - None, - &relay, - &persistent_hook_commands(&relay, &generation, "hook-token").unwrap(), - &generation, - "mcp-token", - &[], - ) - .unwrap(); - assert_eq!( - owned_install_command(&root, &relay, Some(&generation)).unwrap(), - Some(persistent_hook_commands(&relay, &generation, "mcp-token").unwrap()) - ); - - root["mcp_servers"][MCP_SERVER_NAME]["command"] = json!(temp.path().join("other/nemo-relay")); - assert!( - owned_install_command(&root, &relay, Some(&generation)) - .unwrap() - .is_none() - ); -} - -#[test] -fn persistent_config_migrates_modern_single_command_hooks_to_explicit_policies() { - let temp = tempfile::tempdir().unwrap(); - let relay = relay_binary(temp.path()); - let generation = temp.path().join(GENERATION_FILE_NAME); - let commands = persistent_hook_commands(&relay, &generation, TEST_GENERATION_TOKEN).unwrap(); - let mut installed = persistent_config( - None, - &relay, - &commands, - &generation, - TEST_GENERATION_TOKEN, - &[], - ) - .unwrap(); - let legacy = commands.legacy().unwrap(); - for event in CodingAgent::Hermes.hook_events() { - installed["hooks"][event][0]["command"] = json!(legacy); - } - - let migrated = persistent_config( - Some(&serde_yaml::to_string(&installed).unwrap()), - &relay, - &commands, - &generation, - TEST_GENERATION_TOKEN, - &[], - ) - .unwrap(); - - for event in CodingAgent::Hermes.hook_events() { - let hooks = migrated["hooks"][event].as_array().unwrap(); - assert_eq!(hooks.len(), 1, "event {event}"); - assert_eq!(hooks[0]["command"], json!(commands.for_event(event))); - } -} - -#[test] -fn foreign_reserved_server_aborts_install_before_any_file_changes() { - let temp = tempfile::tempdir().unwrap(); - let relay = relay_binary(temp.path()); - let paths = paths(&temp.path().join("hermes")); - std::fs::create_dir_all(paths.config.parent().unwrap()).unwrap(); - let config = - b"# preserve\nmcp_servers:\n nemo-relay:\n command: foreign-mcp\n args: [serve]\n"; - let allowlist = b"{\"approvals\":[{\"event\":\"custom\",\"command\":\"custom-hook\"}]}\n"; - std::fs::write(&paths.config, config).unwrap(); - std::fs::write(&paths.allowlist, allowlist).unwrap(); - - let error = install_persistent_with(paths.clone(), &relay, &[], None, UNIX_EPOCH, atomic_write) - .unwrap_err() - .to_string(); - - assert!(error.contains("not managed by Relay"), "{error}"); - assert_eq!(std::fs::read(&paths.config).unwrap(), config); - assert_eq!(std::fs::read(&paths.allowlist).unwrap(), allowlist); - assert!(!paths.generation.exists()); -} - -#[test] -fn trusted_hooks_migrates_only_relay_approvals_and_records_every_event() { - let temp = tempfile::tempdir().unwrap(); - let relay = relay_binary(temp.path()); - let generation = temp.path().join(GENERATION_FILE_NAME); - let command = persistent_hook_commands(&relay, &generation, TEST_GENERATION_TOKEN).unwrap(); - let existing = json!({ - "schema": 7, - "approvals": [ - {"event": "custom", "command": "custom-hook", "approved_at": "keep"}, - {"event": "on_session_start", "command": "nemo-relay hook-forward hermes"}, - {"event": "on_session_end", "command": "/old/nemo-relay plugin-shim hook hermes"} - ] - }); - let now = UNIX_EPOCH + Duration::from_secs(1_700_000_000); - let legacy = crate::hooks::GeneratedHookCommands::uniform("nemo-relay hook-forward hermes"); - let merged = trusted_hooks( - Some(&serde_json::to_string(&existing).unwrap()), - Some(&legacy), - &command, - &relay, - now, - ) - .unwrap(); - let approvals = merged["approvals"].as_array().unwrap(); - - assert_eq!(merged["schema"], json!(7)); - assert!( - approvals - .iter() - .any(|entry| entry["command"] == json!("custom-hook")) - ); - assert_eq!(approvals.len(), CodingAgent::Hermes.hook_events().len() + 2); - for event in CodingAgent::Hermes.hook_events() { - let entries = approvals - .iter() - .filter(|entry| { - entry["event"] == json!(event) - && entry["command"] == json!(command.for_event(event)) - }) - .collect::>(); - assert_eq!(entries.len(), 1, "event {event}"); - assert_eq!( - entries[0]["approved_at"], - json!("2023-11-14T22:13:20.000000Z") - ); - assert!(entries[0].get("script_mtime_at_approval").is_some()); - } -} - -#[test] -fn verification_rejects_relay_handlers_and_approvals_on_unexpected_events() { - let temp = tempfile::tempdir().unwrap(); - let relay = relay_binary(temp.path()); - let generation = temp.path().join(GENERATION_FILE_NAME); - let command = persistent_hook_commands(&relay, &generation, TEST_GENERATION_TOKEN).unwrap(); - let mut config = persistent_config( - None, - &relay, - &command, - &generation, - TEST_GENERATION_TOKEN, - &[], - ) - .unwrap(); - config["hooks"]["unexpected_event"] = - json!([{"command": command.for_event("on_session_start"), "timeout": 30}]); - let error = verify_hook_definitions(&config, &command).unwrap_err(); - assert!(error.contains("unexpected Relay hook")); - let mut malformed = persistent_config( - None, - &relay, - &command, - &generation, - TEST_GENERATION_TOKEN, - &[], - ) - .unwrap(); - malformed["hooks"]["unexpected_event"] = - json!({"command": command.for_event("on_session_start")}); - let error = verify_hook_definitions(&malformed, &command).unwrap_err(); - assert!(error.contains("must be an array")); - - let mut allowlist = trusted_hooks(None, None, &command, &relay, UNIX_EPOCH).unwrap(); - allowlist["approvals"].as_array_mut().unwrap().push(json!({ - "event": "unexpected_event", - "command": command.for_event("on_session_start"), - "approved_at": "1970-01-01T00:00:00.000000Z" - })); - let path = temp.path().join("shell-hooks-allowlist.json"); - std::fs::write(&path, serde_json::to_vec(&allowlist).unwrap()).unwrap(); - let error = verify_trust(&path, &command).unwrap_err(); - assert!(error.contains("unexpected Relay hook approval")); - - let mut missing_event = trusted_hooks(None, None, &command, &relay, UNIX_EPOCH).unwrap(); - missing_event["approvals"] - .as_array_mut() - .unwrap() - .push(json!({ - "command": command.for_event("on_session_start"), - "approved_at": "1970-01-01T00:00:00.000000Z" - })); - std::fs::write(&path, serde_json::to_vec(&missing_event).unwrap()).unwrap(); - let error = verify_trust(&path, &command).unwrap_err(); - assert!(error.contains("missing its event")); - - let mut wrong_policy = trusted_hooks(None, None, &command, &relay, UNIX_EPOCH).unwrap(); - wrong_policy["approvals"] - .as_array_mut() - .unwrap() - .push(json!({ - "event": "pre_tool_call", - "command": command.for_event("on_session_start"), - "approved_at": "1970-01-01T00:00:00.000000Z" - })); - std::fs::write(&path, serde_json::to_vec(&wrong_policy).unwrap()).unwrap(); - let error = verify_trust(&path, &command).unwrap_err(); - assert!(error.contains("unexpected Relay hook approval")); -} - -#[test] -fn hermes_structure_and_trust_validation_cover_exact_failure_shapes() { - let temp = tempfile::tempdir().unwrap(); - let relay = relay_binary(temp.path()); - let generation = temp.path().join(GENERATION_FILE_NAME); - let command = persistent_hook_commands(&relay, &generation, TEST_GENERATION_TOKEN).unwrap(); - - let error = trusted_hooks( - Some(r#"{"approvals": {}}"#), - None, - &command, - &relay, - UNIX_EPOCH, - ) - .unwrap_err() - .to_string(); - assert!(error.contains("approvals must be an array"), "{error}"); - - let error = parse_json_object(Some("[]"), "test allowlist") - .unwrap_err() - .to_string(); - assert!(error.contains("must contain a JSON object"), "{error}"); - - let mut malformed_hooks = json!({"hooks": {"on_session_start": {}}}); - let error = strip_owned_hooks(&mut malformed_hooks, Some(&command)) - .unwrap_err() - .to_string(); - assert!( - error.contains("on_session_start hooks must be an array"), - "{error}" - ); - - let error = parse_yaml_object(Some("[]"), "test config") - .unwrap_err() - .to_string(); - assert!(error.contains("must contain an object"), "{error}"); - let path = temp.path().join("shell-hooks-allowlist.json"); - let mut missing = trusted_hooks(None, None, &command, &relay, UNIX_EPOCH).unwrap(); - missing["approvals"].as_array_mut().unwrap().remove(0); - std::fs::write(&path, serde_json::to_vec(&missing).unwrap()).unwrap(); - let error = verify_trust(&path, &command).unwrap_err(); - assert!( - error.contains("expected exactly one trust approval"), - "{error}" - ); - - let mut with_opaque_entry = trusted_hooks(None, None, &command, &relay, UNIX_EPOCH).unwrap(); - with_opaque_entry["approvals"] - .as_array_mut() - .unwrap() - .push(json!({"metadata": "unrelated"})); - std::fs::write(&path, serde_json::to_vec(&with_opaque_entry).unwrap()).unwrap(); - verify_trust(&path, &command).unwrap(); -} - -#[test] -fn install_is_verified_idempotent_and_rotates_the_generation() { - let temp = tempfile::tempdir().unwrap(); - let relay = relay_binary(temp.path()); - let paths = paths(&temp.path().join("hermes")); - let environment = vec!["OTEL_SERVICE_NAME".into()]; - let now = UNIX_EPOCH + Duration::from_secs(1_700_000_000); - - let written = - install_persistent_with(paths.clone(), &relay, &environment, None, now, atomic_write) - .unwrap(); - assert_eq!(written, paths.all()); - let first_generation = - crate::installation::generation::InstallGeneration::capture(paths.generation.clone()) - .unwrap() - .token() - .to_owned(); - let first_config = yaml(&paths.config); - let first_command = first_config["hooks"]["on_session_start"][0]["command"] - .as_str() - .unwrap() - .to_string(); - assert_eq!( - first_config["mcp_servers"][MCP_SERVER_NAME]["env"][GENERATION_TOKEN_ENV], - json!(first_generation) - ); - assert!(crate::hook_assertions::command_has_arguments( - &first_command, - &["--generation-token", &first_generation] - )); - - install_persistent_with(paths.clone(), &relay, &environment, None, now, atomic_write).unwrap(); - let second_generation = - crate::installation::generation::InstallGeneration::capture(paths.generation.clone()) - .unwrap() - .token() - .to_owned(); - assert_ne!(first_generation, second_generation); - - let config = yaml(&paths.config); - let second_command = - persistent_hook_commands(&relay, &paths.generation, &second_generation).unwrap(); - assert_eq!( - config["hooks"]["on_session_start"] - .as_array() - .unwrap() - .iter() - .filter(|group| { - group["command"] == json!(second_command.for_event("on_session_start")) - }) - .count(), - 1 - ); - assert_eq!( - config["hooks"]["on_session_start"] - .as_array() - .unwrap() - .iter() - .filter(|group| group["command"] == json!(first_command)) - .count(), - 0 - ); - assert_eq!( - config["mcp_servers"][MCP_SERVER_NAME]["env"][GENERATION_FILE_ENV], - json!(paths.generation.display().to_string()) - ); - assert_eq!( - config["mcp_servers"][MCP_SERVER_NAME]["env"][GENERATION_TOKEN_ENV], - json!(second_generation) - ); - assert_ne!( - first_config["mcp_servers"][MCP_SERVER_NAME]["env"][GENERATION_TOKEN_ENV], - config["mcp_servers"][MCP_SERVER_NAME]["env"][GENERATION_TOKEN_ENV] - ); - assert!(crate::hook_assertions::command_has_arguments( - &first_command, - &["--generation-token", &first_generation] - )); - assert!(!crate::hook_assertions::command_has_arguments( - &first_command, - &["--generation-token", &second_generation] - )); - assert_eq!( - config["mcp_servers"][MCP_SERVER_NAME]["env"]["OTEL_SERVICE_NAME"], - json!("${OTEL_SERVICE_NAME}") - ); - assert_eq!( - json_file(&paths.allowlist)["approvals"] - .as_array() - .unwrap() - .len(), - CodingAgent::Hermes.hook_events().len() - ); -} - -#[test] -fn reinstall_verifies_generation_through_the_existing_retirement_transaction() { - let temp = tempfile::tempdir().unwrap(); - let relay = relay_binary(temp.path()); - let paths = paths(&temp.path().join("hermes")); - install_persistent_with(paths.clone(), &relay, &[], None, UNIX_EPOCH, atomic_write).unwrap(); - let first_token = InstallGeneration::capture(paths.generation.clone()) - .unwrap() - .token() - .to_owned(); - let mut retirement = GenerationRetirement::acquire(&paths.generation) - .unwrap() - .unwrap(); - retirement.invalidate_for_replacement().unwrap(); - - let result = install_persistent_with_generation( - paths.clone(), - &relay, - &[], - None, - Some(&retirement), - UNIX_EPOCH, - atomic_write, - ); - finish_generation_mutation(result, Some(&mut retirement), "install").unwrap(); - drop(retirement); - - let second_token = InstallGeneration::capture(paths.generation) - .unwrap() - .token() - .to_owned(); - assert_ne!(first_token, second_token); -} - -#[test] -fn diagnosis_rejects_a_stale_mcp_generation_identity() { - let temp = tempfile::tempdir().unwrap(); - let relay = relay_binary(temp.path()); - let paths = paths(&temp.path().join("hermes")); - install_persistent_with(paths.clone(), &relay, &[], None, UNIX_EPOCH, atomic_write).unwrap(); - let mut config = yaml(&paths.config); - config["mcp_servers"][MCP_SERVER_NAME]["env"][GENERATION_TOKEN_ENV] = json!("stale-generation"); - std::fs::write(&paths.config, serde_yaml::to_string(&config).unwrap()).unwrap(); - - let error = diagnose_persistent(&paths.config).unwrap_err(); - - assert!( - error.contains("expected generation identity is stale"), - "{error}" - ); -} - -#[test] -fn install_rolls_back_config_allowlist_and_generation_after_write_failure() { - let temp = tempfile::tempdir().unwrap(); - let relay = relay_binary(temp.path()); - let paths = paths(&temp.path().join("hermes")); - std::fs::create_dir_all(paths.config.parent().unwrap()).unwrap(); - let originals = [ - (&paths.config, b"model: original\n".as_slice()), - ( - &paths.allowlist, - b"{\"approvals\":[{\"event\":\"x\",\"command\":\"custom\"}]}\n".as_slice(), - ), - (&paths.generation, b"original-generation\n".as_slice()), - ]; - for (path, bytes) in originals { - std::fs::write(path, bytes).unwrap(); - } - let before = paths.all().map(|path| std::fs::read(path).unwrap()); - let writes = Cell::new(0); - - let error = install_persistent_with( - paths.clone(), - &relay, - &[], - None, - UNIX_EPOCH, - |path, bytes| { - let write = writes.get() + 1; - writes.set(write); - if write == 3 { - return Err("injected config write failure".into()); - } - atomic_write(path, bytes) - }, - ) - .unwrap_err() - .to_string(); - - assert!(error.contains("injected config write failure"), "{error}"); - for (index, path) in paths.all().iter().enumerate() { - assert_eq!( - std::fs::read(path).unwrap(), - before[index], - "{}", - path.display() - ); - } -} - -#[cfg(unix)] -#[test] -fn install_rollback_restores_original_file_permissions() { - use std::os::unix::fs::PermissionsExt; - - let temp = tempfile::tempdir().unwrap(); - let relay = relay_binary(temp.path()); - let paths = paths(&temp.path().join("hermes")); - std::fs::create_dir_all(paths.config.parent().unwrap()).unwrap(); - let originals = [ - (&paths.config, b"model: original\n".as_slice(), 0o640), - ( - &paths.allowlist, - b"{\"approvals\":[{\"event\":\"x\",\"command\":\"custom\"}]}\n".as_slice(), - 0o644, - ), - ( - &paths.generation, - b"original-generation\n".as_slice(), - 0o600, - ), - ]; - for (path, bytes, mode) in originals { - std::fs::write(path, bytes).unwrap(); - std::fs::set_permissions(path, std::fs::Permissions::from_mode(mode)).unwrap(); - } - let expected_modes = paths - .all() - .map(|path| std::fs::metadata(path).unwrap().permissions().mode() & 0o777); - let writes = Cell::new(0); - - install_persistent_with( - paths.clone(), - &relay, - &[], - None, - UNIX_EPOCH, - |path, bytes| { - let write = writes.get() + 1; - writes.set(write); - if write == 3 { - return Err("injected config write failure".into()); - } - atomic_write(path, bytes)?; - std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o600)) - .map_err(|error| error.to_string()) - }, - ) - .unwrap_err(); - - for (index, path) in paths.all().iter().enumerate() { - assert_eq!( - std::fs::metadata(path).unwrap().permissions().mode() & 0o777, - expected_modes[index], - "{}", - path.display() - ); - } -} - -#[test] -fn composed_install_rollback_restores_the_visible_preexisting_generation() { - let temp = tempfile::tempdir().unwrap(); - let relay = relay_binary(temp.path()); - let paths = paths(&temp.path().join("hermes")); - std::fs::create_dir_all(paths.config.parent().unwrap()).unwrap(); - install_persistent_with(paths.clone(), &relay, &[], None, UNIX_EPOCH, atomic_write).unwrap(); - let previous = - crate::installation::generation::InstallGeneration::capture(paths.generation.clone()) - .unwrap(); - let mut retirement = GenerationRetirement::acquire(&paths.generation) - .unwrap() - .unwrap(); - retirement.invalidate_for_replacement().unwrap(); - let writes = Cell::new(0); - - let result = install_persistent_with( - paths.clone(), - &relay, - &[], - None, - UNIX_EPOCH, - |path, bytes| { - let write = writes.get() + 1; - writes.set(write); - if write == 3 { - return Err("injected composed install failure".into()); - } - atomic_write(path, bytes) - }, - ); - let error = finish_generation_mutation(result, Some(&mut retirement), "install") - .unwrap_err() - .to_string(); - - assert!( - error.contains("injected composed install failure"), - "{error}" - ); - previous.verify_current().unwrap(); - crate::installation::generation::InstallGeneration::capture(paths.generation).unwrap(); -} - -#[test] -fn install_rolls_back_after_post_write_verification_failure() { - let temp = tempfile::tempdir().unwrap(); - let relay = relay_binary(temp.path()); - let paths = paths(&temp.path().join("hermes")); - std::fs::create_dir_all(paths.config.parent().unwrap()).unwrap(); - std::fs::write(&paths.config, "model: original\n").unwrap(); - std::fs::write(&paths.allowlist, "{\"approvals\":[]}\n").unwrap(); - std::fs::write(&paths.generation, "old\n").unwrap(); - let before = paths.all().map(|path| std::fs::read(path).unwrap()); - let corrupted = Cell::new(false); - - let error = install_persistent_with( - paths.clone(), - &relay, - &[], - None, - UNIX_EPOCH, - |path, bytes| { - if path == paths.config && !corrupted.replace(true) { - return atomic_write(path, b"hooks: invalid-shape\n"); - } - atomic_write(path, bytes) - }, - ) - .unwrap_err() - .to_string(); - - assert!( - error.contains("Hermes MCP server did not persist exactly"), - "{error}" - ); - for (index, path) in paths.all().iter().enumerate() { - assert_eq!( - std::fs::read(path).unwrap(), - before[index], - "{}", - path.display() - ); - } -} - -#[test] -fn uninstall_removes_only_relay_owned_hermes_state() { - let temp = tempfile::tempdir().unwrap(); - let relay = relay_binary(temp.path()); - let paths = paths(&temp.path().join("hermes")); - std::fs::create_dir_all(paths.config.parent().unwrap()).unwrap(); - std::fs::write( - &paths.config, - "model: keep\nmcp_servers:\n filesystem:\n command: fs-mcp\nhooks:\n custom_event:\n - command: custom-hook\n", - ) - .unwrap(); - std::fs::write( - &paths.allowlist, - "{\"owner\":\"user\",\"approvals\":[{\"event\":\"custom_event\",\"command\":\"custom-hook\"}]}\n", - ) - .unwrap(); - install_persistent_with(paths.clone(), &relay, &[], None, UNIX_EPOCH, atomic_write).unwrap(); - - let removed = uninstall_persistent_with(paths.clone(), atomic_write).unwrap(); - - assert_eq!(removed, paths.all()); - assert!(!paths.generation.exists()); - let config = yaml(&paths.config); - assert_eq!(config["model"], json!("keep")); - assert_eq!( - config["mcp_servers"]["filesystem"]["command"], - json!("fs-mcp") - ); - assert!(config["mcp_servers"].get(MCP_SERVER_NAME).is_none()); - assert_eq!( - config["hooks"]["custom_event"][0]["command"], - json!("custom-hook") - ); - let allowlist = json_file(&paths.allowlist); - assert_eq!(allowlist["owner"], json!("user")); - assert_eq!(allowlist["approvals"].as_array().unwrap().len(), 1); - assert_eq!(allowlist["approvals"][0]["command"], json!("custom-hook")); -} - -#[test] -fn uninstall_removes_orphaned_generated_approval_without_config() { - let temp = tempfile::tempdir().unwrap(); - let relay = relay_binary(temp.path()); - let paths = paths(&temp.path().join("hermes")); - let commands = - persistent_hook_commands(&relay, &paths.generation, TEST_GENERATION_TOKEN).unwrap(); - std::fs::create_dir_all(paths.allowlist.parent().unwrap()).unwrap(); - std::fs::write( - &paths.allowlist, - serde_json::to_vec(&json!({ - "approvals": [ - { - "event": "pre_tool_call", - "command": commands.for_event("pre_tool_call") - }, - { - "event": "custom_event", - "command": "custom-hook" - } - ] - })) - .unwrap(), - ) - .unwrap(); - - let removed = uninstall_persistent_with(paths.clone(), atomic_write).unwrap(); - - assert_eq!(removed, vec![paths.allowlist.clone()]); - assert_eq!( - json_file(&paths.allowlist)["approvals"], - json!([{"event": "custom_event", "command": "custom-hook"}]) - ); -} - -#[test] -fn uninstall_rolls_back_every_file_when_commit_fails() { - let temp = tempfile::tempdir().unwrap(); - let relay = relay_binary(temp.path()); - let paths = paths(&temp.path().join("hermes")); - std::fs::create_dir_all(paths.config.parent().unwrap()).unwrap(); - std::fs::write(&paths.config, "model: keep\n").unwrap(); - std::fs::write(&paths.allowlist, "{\"owner\":\"keep\"}\n").unwrap(); - install_persistent_with(paths.clone(), &relay, &[], None, UNIX_EPOCH, atomic_write).unwrap(); - let before = paths.all().map(|path| std::fs::read(path).unwrap()); - let writes = Cell::new(0); - - let error = uninstall_persistent_with(paths.clone(), |path, bytes| { - let write = writes.get() + 1; - writes.set(write); - if write == 2 { - return Err("injected uninstall config failure".into()); - } - atomic_write(path, bytes) - }) - .unwrap_err() - .to_string(); - - assert!( - error.contains("injected uninstall config failure"), - "{error}" - ); - for (index, path) in paths.all().iter().enumerate() { - assert_eq!( - std::fs::read(path).unwrap(), - before[index], - "{}", - path.display() - ); - } -} - -#[test] -fn composed_uninstall_rollback_restores_the_visible_preexisting_generation() { - let temp = tempfile::tempdir().unwrap(); - let relay = relay_binary(temp.path()); - let paths = paths(&temp.path().join("hermes")); - std::fs::create_dir_all(paths.config.parent().unwrap()).unwrap(); - std::fs::write(&paths.config, "model: keep\n").unwrap(); - std::fs::write(&paths.allowlist, "{\"owner\":\"keep\"}\n").unwrap(); - install_persistent_with(paths.clone(), &relay, &[], None, UNIX_EPOCH, atomic_write).unwrap(); - let previous = - crate::installation::generation::InstallGeneration::capture(paths.generation.clone()) - .unwrap(); - let mut retirement = GenerationRetirement::acquire(&paths.generation) - .unwrap() - .unwrap(); - retirement.invalidate_for_replacement().unwrap(); - let writes = Cell::new(0); - - let result = uninstall_persistent_with(paths.clone(), |path, bytes| { - let write = writes.get() + 1; - writes.set(write); - if write == 2 { - return Err("injected composed uninstall failure".into()); - } - atomic_write(path, bytes) - }); - let error = finish_generation_mutation(result, Some(&mut retirement), "uninstall") - .unwrap_err() - .to_string(); - - assert!( - error.contains("injected composed uninstall failure"), - "{error}" - ); - previous.verify_current().unwrap(); - crate::installation::generation::InstallGeneration::capture(paths.generation).unwrap(); -} - -#[test] -fn uninstall_noops_without_creating_a_hermes_home() { - let temp = tempfile::tempdir().unwrap(); - let home = temp.path().join("missing-hermes-home"); - let config = home.join("config.yaml"); - - assert!(uninstall_persistent(&config).unwrap().is_empty()); - assert!(!home.exists()); -} - -#[test] -fn unrelated_hermes_files_are_not_owned_or_rewritten_by_uninstall() { - let temp = tempfile::tempdir().unwrap(); - let paths = paths(&temp.path().join("hermes")); - std::fs::create_dir_all(paths.config.parent().unwrap()).unwrap(); - let config = b"# preserve this exact formatting\nmodel: custom\nmcp_servers:\n nemo-relay:\n command: foreign-mcp\n args: [serve]\n"; - let allowlist = b"{ \"approvals\": [{\"event\":\"custom\",\"command\":\"custom-hook\"}] }\n"; - std::fs::write(&paths.config, config).unwrap(); - std::fs::write(&paths.allowlist, allowlist).unwrap(); - - assert!(!persistent_state_exists(&paths.config)); - assert!(uninstall_persistent(&paths.config).unwrap().is_empty()); - assert_eq!(std::fs::read(&paths.config).unwrap(), config); - assert_eq!(std::fs::read(&paths.allowlist).unwrap(), allowlist); - assert!(!paths.generation.exists()); -} - -#[test] -fn persistent_state_detection_recognizes_each_relay_owned_surface() { - let temp = tempfile::tempdir().unwrap(); - let relay = relay_binary(temp.path()); - let roots = ["generation", "mcp", "hook", "approval"].map(|name| { - let paths = paths(&temp.path().join(name)); - std::fs::create_dir_all(paths.config.parent().unwrap()).unwrap(); - paths - }); - - std::fs::write(&roots[0].generation, "active\n").unwrap(); - std::fs::write( - &roots[1].config, - serde_yaml::to_string(&json!({ - "mcp_servers": {MCP_SERVER_NAME: expected_mcp_server( - &relay, - &roots[1].generation, - TEST_GENERATION_TOKEN, - &[] - )} - })) - .unwrap(), - ) - .unwrap(); - std::fs::write( - &roots[2].config, - serde_yaml::to_string(&json!({ - "hooks": { - "on_session_start": [{"command": persistent_hook_commands( - &relay, - &roots[2].generation, - TEST_GENERATION_TOKEN - ).unwrap().for_event("on_session_start")}] - } - })) - .unwrap(), - ) - .unwrap(); - std::fs::write( - &roots[3].allowlist, - serde_json::to_vec(&json!({ - "approvals": [{ - "event": "on_session_start", - "command": persistent_hook_commands( - &relay, - &roots[3].generation, - TEST_GENERATION_TOKEN - ).unwrap().for_event("on_session_start") - }] - })) - .unwrap(), - ) - .unwrap(); - - for paths in [&roots[0], &roots[1], &roots[3]] { - assert!( - persistent_state_exists(&paths.config), - "managed state at {} was not detected", - paths.config.display() - ); - } - assert!(!persistent_state_exists(&roots[2].config)); -} - -#[test] -fn transparent_config_suppresses_only_the_managed_mcp_and_uses_one_relay_hook() { - let temp = tempfile::tempdir().unwrap(); - let relay = relay_binary(temp.path()); - let command = crate::hooks::transparent_hook_forward_commands( - &relay, - CodingAgent::Hermes, - "http://127.0.0.1:1234", - ) - .unwrap(); - let generation = temp.path().join(GENERATION_FILE_NAME); - let persistent_command = - persistent_hook_commands(&relay, &generation, TEST_GENERATION_TOKEN).unwrap(); - let mut existing = persistent_config( - None, - &relay, - &persistent_command, - &generation, - TEST_GENERATION_TOKEN, - &[], - ) - .unwrap(); - existing["mcp_servers"]["filesystem"] = json!({"command": "fs-mcp"}); - existing["hooks"]["on_session_start"] - .as_array_mut() - .unwrap() - .push(json!({"command": "custom-hook"})); - let existing = serde_yaml::to_string(&existing).unwrap(); - let patched: Value = serde_yaml::from_str( - &transparent_config(&existing, &relay, "http://127.0.0.1:1234").unwrap(), - ) - .unwrap(); - - assert!(patched["mcp_servers"].get(MCP_SERVER_NAME).is_none()); - assert_eq!( - patched["mcp_servers"]["filesystem"]["command"], - json!("fs-mcp") - ); - assert_eq!(patched["model"]["provider"], json!("custom")); - assert_eq!( - patched["model"]["api_key"], - json!(format!( - "${{{}}}", - crate::provider_auth::TRANSPARENT_PROXY_CREDENTIAL_ENV - )) - ); - for event in CodingAgent::Hermes.hook_events() { - let groups = patched["hooks"][event].as_array().unwrap(); - assert_eq!( - groups - .iter() - .filter_map(|group| group.get("command").and_then(Value::as_str)) - .filter(|candidate| *candidate == command.for_event(event)) - .count(), - 1, - "event {event}" - ); - assert!( - groups - .iter() - .any(|group| group["command"] == json!(command.for_event(event))) - ); - } - assert!( - patched["hooks"]["on_session_start"] - .as_array() - .unwrap() - .iter() - .any(|group| group["command"] == json!("custom-hook")) - ); -} - -#[test] -fn malformed_user_files_fail_before_any_state_is_replaced() { - let temp = tempfile::tempdir().unwrap(); - let relay = relay_binary(temp.path()); - let paths = paths(&temp.path().join("hermes")); - std::fs::create_dir_all(paths.config.parent().unwrap()).unwrap(); - std::fs::write(&paths.config, "hooks: [not-an-object]\n").unwrap(); - std::fs::write(&paths.allowlist, "{\"approvals\":[]}").unwrap(); - std::fs::write(&paths.generation, "old\n").unwrap(); - let before = paths.all().map(|path| std::fs::read(path).unwrap()); - - assert!( - install_persistent_with(paths.clone(), &relay, &[], None, UNIX_EPOCH, atomic_write,) - .is_err() - ); - for (index, path) in paths.all().iter().enumerate() { - assert_eq!(std::fs::read(path).unwrap(), before[index]); - } -} - -#[test] -fn hermes_entrypoints_reject_missing_or_foreign_relay_binaries() { - let temp = tempfile::tempdir().unwrap(); - let config_path = temp.path().join("hermes/config.yaml"); - let missing_relay = temp.path().join("missing/nemo-relay"); - - let error = install_persistent(&config_path, &missing_relay) - .unwrap_err() - .to_string(); - assert!(error.contains("missing or not executable"), "{error}"); - - std::fs::create_dir_all(config_path.parent().unwrap()).unwrap(); - std::fs::write( - &config_path, - format!( - "mcp_servers:\n {MCP_SERVER_NAME}:\n command: {}\n args: [mcp]\n", - missing_relay.display() - ), - ) - .unwrap(); - let error = configured_relay_executable(&config_path).unwrap_err(); - assert!(error.contains("not a managed Relay MCP client"), "{error}"); - - let foreign = json!({ - "mcp_servers": { - MCP_SERVER_NAME: { - "command": "foreign-mcp", - "args": ["serve"] - } - } - }); - let error = relay_executable_from_config(&foreign).unwrap_err(); - assert!(error.contains("not a managed Relay MCP client"), "{error}"); -} - -#[test] -fn hermes_diagnosis_validates_binary_bind_generation_and_environment() { - let temp = tempfile::tempdir().unwrap(); - let _config_home = XdgConfigHomeScope::enter(&temp.path().join("xdg")); - let relay = relay_binary(temp.path()); - let paths = paths(&temp.path().join("hermes")); - install_persistent_with(paths.clone(), &relay, &[], None, UNIX_EPOCH, atomic_write).unwrap(); - - let original = yaml(&paths.config); - std::fs::remove_file(&relay).unwrap(); - let error = diagnose_persistent(&paths.config).unwrap_err(); - assert!(error.contains("missing or not executable"), "{error}"); - - let relay = relay_binary(temp.path()); - let mut wrong_bind = original.clone(); - wrong_bind["mcp_servers"][MCP_SERVER_NAME]["env"]["NEMO_RELAY_GATEWAY_BIND"] = - json!("127.0.0.1:1"); - std::fs::write(&paths.config, serde_yaml::to_string(&wrong_bind).unwrap()).unwrap(); - let error = diagnose_persistent(&paths.config).unwrap_err(); - assert!(error.contains("not a managed Relay MCP client"), "{error}"); - - let mut wrong_generation = original.clone(); - wrong_generation["mcp_servers"][MCP_SERVER_NAME]["env"][GENERATION_FILE_ENV] = - json!(temp.path().join("wrong-generation").display().to_string()); - std::fs::write( - &paths.config, - serde_yaml::to_string(&wrong_generation).unwrap(), - ) - .unwrap(); - let error = diagnose_persistent(&paths.config).unwrap_err(); - assert!( - error.contains("generation fence points at the wrong file"), - "{error}" - ); - - let mut missing_environment = original; - assert!( - missing_environment["mcp_servers"][MCP_SERVER_NAME]["env"] - .as_object_mut() - .unwrap() - .remove("OPENAI_API_KEY") - .is_some() - ); - std::fs::write( - &paths.config, - serde_yaml::to_string(&missing_environment).unwrap(), - ) - .unwrap(); - let error = diagnose_persistent(&paths.config).unwrap_err(); - assert!(error.contains("missing environment names"), "{error}"); - assert!(error.contains("OPENAI_API_KEY"), "{error}"); - assert!(error.contains("install hermes --force"), "{error}"); - - assert!(relay.exists()); -} - -#[test] -fn hermes_generation_finish_preserves_primary_errors_and_reports_restore_failures() { - let primary = CliError::Install("primary failure".into()); - let error = finish_generation_mutation::<()>(Err(primary), None, "install") - .unwrap_err() - .to_string(); - assert!(error.contains("primary failure"), "{error}"); - - let temp = tempfile::tempdir().unwrap(); - let generation = temp.path().join(GENERATION_FILE_NAME); - crate::installation::generation::write_new_generation(&generation).unwrap(); - let mut retirement = GenerationRetirement::acquire(&generation).unwrap().unwrap(); - retirement.invalidate_for_replacement().unwrap(); - std::fs::write(&generation, "foreign-generation\n").unwrap(); - - let error = finish_generation_mutation::<()>( - Err(CliError::Install("mutation failed".into())), - Some(&mut retirement), - "install", - ) - .unwrap_err() - .to_string(); - assert!(error.contains("mutation failed"), "{error}"); - assert!(error.contains("additionally failed to restore"), "{error}"); -} - -#[test] -fn hermes_uninstall_and_verification_reject_malformed_or_residual_state() { - let temp = tempfile::tempdir().unwrap(); - let relay = relay_binary(temp.path()); - let hermes_paths = paths(&temp.path().join("hermes")); - install_persistent_with( - hermes_paths.clone(), - &relay, - &[], - None, - UNIX_EPOCH, - atomic_write, - ) - .unwrap(); - - let config = yaml(&hermes_paths.config); - let token = InstallGeneration::capture(hermes_paths.generation.clone()) - .unwrap() - .token() - .to_owned(); - let command = persistent_hook_commands(&relay, &hermes_paths.generation, &token).unwrap(); - let expected_environment = forwarded_environment_names(&[], None); - - let mut duplicate_hook = config.clone(); - duplicate_hook["hooks"]["on_session_start"] - .as_array_mut() - .unwrap() - .push(json!({"command": command.for_event("on_session_start")})); - let error = verify_hook_definitions(&duplicate_hook, &command).unwrap_err(); - assert!( - error.contains("exactly one trusted Relay handler"), - "{error}" - ); - - let mut harmless_missing_command = config.clone(); - harmless_missing_command["hooks"] - .as_object_mut() - .unwrap() - .insert("custom".into(), json!([{"timeout": 1}])); - verify_hook_definitions(&harmless_missing_command, &command).unwrap(); - - verify_install( - &hermes_paths, - &relay, - &command, - &expected_environment, - &token, - None, - ) - .unwrap(); - - let mut mismatched_environment = config.clone(); - let environment_name = expected_environment - .first() - .expect("persistent MCP environment is non-empty"); - mismatched_environment["mcp_servers"][MCP_SERVER_NAME]["env"][environment_name] = - json!("unexpected-value"); - std::fs::write( - &hermes_paths.config, - serde_yaml::to_string(&mismatched_environment).unwrap(), - ) - .unwrap(); - let error = diagnose_persistent(&hermes_paths.config).unwrap_err(); - assert!(error.contains(environment_name), "{error}"); - - install_persistent_with( - hermes_paths.clone(), - &relay, - &expected_environment, - None, - UNIX_EPOCH, - atomic_write, - ) - .unwrap(); - let expected_token = InstallGeneration::capture(hermes_paths.generation.clone()) - .unwrap() - .token() - .to_owned(); - let command = - persistent_hook_commands(&relay, &hermes_paths.generation, &expected_token).unwrap(); - crate::installation::generation::write_new_generation(&hermes_paths.generation).unwrap(); - let error = verify_install( - &hermes_paths, - &relay, - &command, - &expected_environment, - &expected_token, - None, - ) - .unwrap_err(); - assert!( - error.contains("generation did not persist exactly"), - "{error}" - ); - - let malformed_paths = paths(&temp.path().join("malformed")); - std::fs::create_dir_all(malformed_paths.config.parent().unwrap()).unwrap(); - std::fs::write(&malformed_paths.allowlist, r#"{"approvals":{}}"#).unwrap(); - let error = uninstall_persistent_with(malformed_paths, atomic_write) - .unwrap_err() - .to_string(); - assert!(error.contains("approvals must be an array"), "{error}"); -} - -#[test] -fn hermes_uninstall_verifier_identifies_each_residual_owned_surface() { - let temp = tempfile::tempdir().unwrap(); - let relay = relay_binary(temp.path()); - let paths = paths(&temp.path().join("hermes")); - install_persistent_with(paths.clone(), &relay, &[], None, UNIX_EPOCH, atomic_write).unwrap(); - - let command = owned_command_from_config(&yaml(&paths.config), Some(&paths.generation)); - let error = verify_uninstall(&paths, command.as_ref()).unwrap_err(); - assert!(error.contains("generation fence still exists"), "{error}"); - - std::fs::remove_file(&paths.generation).unwrap(); - let error = verify_uninstall(&paths, command.as_ref()).unwrap_err(); - assert!( - error.contains("managed Hermes Relay config still exists"), - "{error}" - ); - - std::fs::remove_file(&paths.config).unwrap(); - let error = verify_uninstall(&paths, command.as_ref()).unwrap_err(); - assert!( - error.contains("managed Hermes Relay trust approval still exists"), - "{error}" - ); -} - -#[test] -fn hermes_file_helpers_report_path_lock_read_remove_and_restore_failures() { - let temp = tempfile::tempdir().unwrap(); - - let error = PersistentPaths::for_config(PathBuf::from("/")) - .unwrap_err() - .to_string(); - assert!(error.contains("has no parent directory"), "{error}"); - let error = acquire_install_lock(Path::new("/"), Duration::ZERO).unwrap_err(); - assert!(error.contains("has no parent directory"), "{error}"); - - let parent_file = temp.path().join("parent-file"); - std::fs::write(&parent_file, "file").unwrap(); - let error = acquire_allowlist_lock(&parent_file.join("allowlist"), Duration::ZERO).unwrap_err(); - assert!(error.contains("failed to create"), "{error}"); - let error = - acquire_allowlist_lock(&parent_file.join("nested/allowlist"), Duration::ZERO).unwrap_err(); - assert!(error.contains("failed to create"), "{error}"); - - let allowlist = temp.path().join("allowlist.json"); - let lock_dir = temp.path().join("allowlist.json.lock"); - std::fs::create_dir(&lock_dir).unwrap(); - let error = acquire_allowlist_lock(&allowlist, Duration::ZERO).unwrap_err(); - assert!( - error.contains("failed to open Hermes install lock"), - "{error}" - ); - - let held_config = temp.path().join("held/config.yaml"); - let _held = acquire_install_lock(&held_config, Duration::ZERO).unwrap(); - let error = acquire_install_lock(&held_config, Duration::from_millis(30)).unwrap_err(); - assert!(error.contains("timed out waiting"), "{error}"); - - let directory = temp.path().join("directory"); - std::fs::create_dir(&directory).unwrap(); - let error = read_optional_utf8(&directory).unwrap_err().to_string(); - assert!(error.contains("failed to read"), "{error}"); - let error = match FileSnapshot::capture(&directory) { - Ok(_) => panic!("directory snapshot unexpectedly succeeded"), - Err(error) => error.to_string(), - }; - assert!(error.contains("failed to snapshot"), "{error}"); - let error = remove_optional_file(&directory).unwrap_err(); - assert!(error.contains("failed to remove"), "{error}"); - remove_optional_file(&temp.path().join("missing")).unwrap(); - - let restored = temp.path().join("restored"); - std::fs::write(&restored, "original").unwrap(); - let snapshot = FileSnapshot::capture(&restored).unwrap(); - std::fs::remove_file(&restored).unwrap(); - let error = snapshot.restore(&mut |_path, _bytes| Ok(())).unwrap_err(); - assert!(error.contains("failed to restore permissions"), "{error}"); - - let absent = temp.path().join("absent"); - let snapshot = FileSnapshot::capture(&absent).unwrap(); - std::fs::write(&absent, "transient").unwrap(); - snapshot.restore(&mut atomic_write).unwrap(); - assert!(!absent.exists()); -} - -#[test] -fn hermes_rollback_reports_both_primary_and_snapshot_restore_errors() { - let temp = tempfile::tempdir().unwrap(); - let path = temp.path().join("state"); - std::fs::write(&path, "original").unwrap(); - let snapshot = FileSnapshot::capture(&path).unwrap(); - let error = rollback_error::<(), _>( - "install", - "primary failure".into(), - &[snapshot], - &mut |_path, _bytes| Err("restore failure".into()), - ) - .unwrap_err() - .to_string(); - assert!(error.contains("primary failure"), "{error}"); - assert!( - error.contains("rollback also failed: restore failure"), - "{error}" - ); -} - -#[test] -fn hermes_uninstall_preserves_an_ambiguous_manual_allowlist() { - let temp = tempfile::tempdir().unwrap(); - let paths = paths(&temp.path().join("hermes")); - std::fs::create_dir_all(paths.allowlist.parent().unwrap()).unwrap(); - std::fs::write( - &paths.allowlist, - serde_json::to_vec(&json!({ - "approvals": [{ - "event": "on_session_start", - "command": "nemo-relay hook-forward hermes" - }] - })) - .unwrap(), - ) - .unwrap(); - - let affected = uninstall_persistent_with(paths.clone(), atomic_write).unwrap(); - - assert_eq!(affected, vec![paths.allowlist.clone()]); - assert!(paths.allowlist.exists()); - assert_eq!( - json_file(&paths.allowlist)["approvals"] - .as_array() - .unwrap() - .len(), - 1 - ); -} diff --git a/crates/cli/tests/coverage/agents/launcher_tests.rs b/crates/cli/tests/coverage/agents/launcher_tests.rs index 5dabbceac..d747cb8ca 100644 --- a/crates/cli/tests/coverage/agents/launcher_tests.rs +++ b/crates/cli/tests/coverage/agents/launcher_tests.rs @@ -103,7 +103,6 @@ fn uses_configured_command_when_no_argv_is_supplied() { let agents = AgentConfigs { codex: AgentCommandConfig { command: Some("codex --full-auto".into()), - hooks_path: None, }, ..AgentConfigs::default() }; @@ -125,33 +124,6 @@ fn uses_configured_command_when_no_argv_is_supplied() { assert_eq!(argv, vec!["codex", "--full-auto"]); } -#[test] -fn uses_configured_hermes_command_when_no_argv_is_supplied() { - let agents = AgentConfigs { - hermes: AgentCommandConfig { - command: Some("hermes --yolo chat".into()), - hooks_path: None, - }, - ..AgentConfigs::default() - }; - let command = RunOverrides { - agent: Some(CodingAgent::Hermes), - config: None, - openai_base_url: None, - anthropic_base_url: None, - session_metadata: None, - plugin_config_path: None, - dry_run: false, - print: false, - command: vec![], - }; - - let (agent, argv) = resolve_agent_and_argv(&command, &agents).unwrap(); - - assert_eq!(agent, CodingAgent::Hermes); - assert_eq!(argv, vec!["hermes", "--yolo", "chat"]); -} - #[test] fn inference_failure_has_actionable_message() { let command = RunOverrides { @@ -177,7 +149,7 @@ fn inference_failure_has_actionable_message() { fn missing_command_without_agent_errors() { // Bare `nemo-relay run` (no command, no --agent) errors — we have nothing to spawn and no // argv[0] to infer an agent from. With --agent set, we fall back to the agent's default - // binary name (e.g., `hermes`), so that branch is exercised in the resolution test + // binary name (for example, `codex`), so that branch is exercised in the resolution test // below rather than here. let command = RunOverrides { agent: None, @@ -200,10 +172,10 @@ fn missing_command_without_agent_errors() { #[test] fn agent_without_configured_command_falls_back_to_default_binary() { - // `--agent hermes` with no `[agents.hermes] command = "..."` override resolves to the + // `--agent codex` with no `[agents.codex] command = "..."` override resolves to the // default executable name on $PATH. let command = RunOverrides { - agent: Some(CodingAgent::Hermes), + agent: Some(CodingAgent::Codex), config: None, openai_base_url: None, anthropic_base_url: None, @@ -215,8 +187,8 @@ fn agent_without_configured_command_falls_back_to_default_binary() { }; let (agent, argv) = resolve_agent_and_argv(&command, &AgentConfigs::default()).unwrap(); - assert_eq!(agent, CodingAgent::Hermes); - assert_eq!(argv, vec!["hermes"]); + assert_eq!(agent, CodingAgent::Codex); + assert_eq!(argv, vec!["codex"]); } #[test] @@ -246,12 +218,10 @@ fn agent_with_passthrough_args_appends_to_configured_command() { fn default_and_configured_command_helpers_cover_empty_and_all_agents() { assert_eq!(default_command_for(CodingAgent::ClaudeCode), "claude"); assert_eq!(default_command_for(CodingAgent::Codex), "codex"); - assert_eq!(default_command_for(CodingAgent::Hermes), "hermes"); let agents = AgentConfigs { codex: AgentCommandConfig { command: Some(" ".into()), - hooks_path: None, }, ..AgentConfigs::default() }; @@ -689,7 +659,6 @@ fn invocation_resolves_wrapper_host_before_appending_pass_through_arguments() { let agents = AgentConfigs { codex: AgentCommandConfig { command: Some("wrapper -- codex".into()), - hooks_path: None, }, ..AgentConfigs::default() }; @@ -728,7 +697,7 @@ fn version_probe_preserves_known_wrappers_and_validates_opaque_ones() { ); assert_eq!( crate::process::version_probe_argv( - CodingAgent::Hermes, + CodingAgent::Codex, &["company-agent-wrapper".into(), "chat".into()], ), vec!["company-agent-wrapper", "chat", "--version"] @@ -918,481 +887,6 @@ fn prepares_claude_dry_inserts_plugin_dir_after_authoritative_agent_executable() assert!(prepared.temp_dirs.is_empty()); } -#[test] -fn prepares_hermes_hook_environment() { - let _guard = current_dir_lock().lock().unwrap(); - let temp = tempfile::tempdir().unwrap(); - let hooks_path = temp.path().join("hermes-home/config.yaml"); - std::fs::create_dir_all(hooks_path.parent().unwrap()).unwrap(); - std::fs::write(&hooks_path, "model:\n default: test\n").unwrap(); - let state = hooks_path.parent().unwrap().join("state.db"); - std::fs::write(&state, "state").unwrap(); - let cache = hooks_path.parent().unwrap().join("cache"); - std::fs::create_dir(&cache).unwrap(); - std::fs::write(cache.join("entry"), "cached").unwrap(); - let resolved = ResolvedConfig { - gateway: GatewayConfig::default(), - agents: AgentConfigs { - hermes: AgentCommandConfig { - command: None, - hooks_path: Some(hooks_path.clone()), - }, - ..AgentConfigs::default() - }, - dynamic_plugins: Vec::new(), - ..ResolvedConfig::default() - }; - let prepared = PreparedAgentLaunch::new( - CodingAgent::Hermes, - vec!["hermes".into(), "chat".into()], - "http://127.0.0.1:1234", - &resolved, - false, - ) - .unwrap(); - - assert_eq!(prepared.argv, vec!["hermes", "chat"]); - assert!(prepared.env.contains(&( - "NEMO_RELAY_GATEWAY_URL".into(), - "http://127.0.0.1:1234".into() - ))); - assert!( - prepared - .env - .contains(&("HERMES_ACCEPT_HOOKS".into(), "1".into())) - ); - let overlay = prepared - .env - .iter() - .find_map(|(name, value)| (name == "HERMES_HOME").then(|| PathBuf::from(value))) - .expect("Hermes overlay path"); - let hooks = std::fs::read_to_string(overlay.join("config.yaml")).unwrap(); - let hooks: serde_json::Value = serde_yaml::from_str(&hooks).unwrap(); - assert_eq!(hooks["model"]["provider"], json!("custom")); - assert_eq!( - hooks["model"]["api_key"], - json!(format!( - "${{{}}}", - crate::provider_auth::TRANSPARENT_PROXY_CREDENTIAL_ENV - )) - ); - assert!(crate::hook_assertions::value_has_command_arguments( - &hooks, - &[ - "hook-forward", - "hermes", - "--gateway-url", - "http://127.0.0.1:1234", - "--transparent-run", - ], - )); - assert!(overlay.join("state.db").exists()); - assert_eq!( - std::fs::read_to_string(overlay.join("state.db")).unwrap(), - "state" - ); - assert_eq!( - std::fs::read_to_string(overlay.join("cache/entry")).unwrap(), - "cached" - ); - std::fs::write(overlay.join("cache/through-overlay"), "live").unwrap(); - assert_eq!( - std::fs::read_to_string(cache.join("through-overlay")).unwrap(), - "live" - ); - assert_eq!( - std::fs::read_to_string(&hooks_path).unwrap(), - "model:\n default: test\n" - ); - assert!(prepared.notes[0].contains("isolated Hermes config overlay")); - - prepared.restore().unwrap(); - assert!(hooks_path.exists()); - assert!(!overlay.exists()); -} - -#[test] -fn sequential_hermes_runs_preserve_state_from_a_fresh_home() { - let _guard = current_dir_lock().lock().unwrap(); - let temp = tempfile::tempdir().unwrap(); - let source_home = temp.path().join("hermes-home"); - let hooks_path = source_home.join("config.yaml"); - let resolved = ResolvedConfig { - gateway: GatewayConfig::default(), - agents: AgentConfigs { - hermes: AgentCommandConfig { - command: None, - hooks_path: Some(hooks_path), - }, - ..AgentConfigs::default() - }, - dynamic_plugins: Vec::new(), - ..ResolvedConfig::default() - }; - let prepare = || { - PreparedAgentLaunch::new( - CodingAgent::Hermes, - vec!["hermes".into(), "chat".into()], - "http://127.0.0.1:1234", - &resolved, - false, - ) - .unwrap() - }; - let overlay = |prepared: &PreparedAgentLaunch| { - prepared - .env - .iter() - .find_map(|(name, value)| (name == "HERMES_HOME").then(|| PathBuf::from(value))) - .expect("Hermes overlay path") - }; - - let first = prepare(); - let first_overlay = overlay(&first); - std::fs::write(first_overlay.join("state.db"), "session state").unwrap(); - std::fs::create_dir_all(first_overlay.join("sessions")).unwrap(); - std::fs::write( - first_overlay.join("sessions/session.json"), - "session details", - ) - .unwrap(); - - assert_eq!( - std::fs::read_to_string(source_home.join("state.db")) - .expect("state.db should be linked to the caller's Hermes home"), - "session state" - ); - assert_eq!( - std::fs::read_to_string(source_home.join("sessions/session.json")) - .expect("sessions should be linked to the caller's Hermes home"), - "session details" - ); - - first.restore().unwrap(); - assert!(!first_overlay.exists()); - - let second = prepare(); - let second_overlay = overlay(&second); - assert_eq!( - std::fs::read_to_string(second_overlay.join("state.db")).unwrap(), - "session state" - ); - assert_eq!( - std::fs::read_to_string(second_overlay.join("sessions/session.json")).unwrap(), - "session details" - ); - - second.restore().unwrap(); - assert!(!second_overlay.exists()); - assert!(source_home.join("state.db").exists()); - assert!(source_home.join("sessions/session.json").exists()); -} - -#[test] -fn rejects_state_db_directory_before_populating_hermes_overlay() { - let temp = tempfile::tempdir().unwrap(); - let source_home = temp.path().join("hermes-home"); - let state_db = source_home.join("state.db"); - let hooks_path = source_home.join("config.yaml"); - std::fs::create_dir_all(&state_db).unwrap(); - let resolved = ResolvedConfig { - agents: AgentConfigs { - hermes: AgentCommandConfig { - hooks_path: Some(hooks_path.clone()), - ..AgentCommandConfig::default() - }, - ..AgentConfigs::default() - }, - ..ResolvedConfig::default() - }; - - let result = PreparedAgentLaunch::new( - CodingAgent::Hermes, - vec!["hermes".into(), "chat".into()], - "http://127.0.0.1:1234", - &resolved, - false, - ); - let error = match result { - Ok(prepared) => { - prepared.restore().unwrap(); - panic!("state.db directories must be rejected") - } - Err(error) => error, - }; - - assert!(matches!(error, CliError::Io(_))); - assert!(state_db.is_dir()); - assert!(!hooks_path.exists()); - assert!(std::fs::read_dir(temp.path()).unwrap().all(|entry| { - !entry - .unwrap() - .file_name() - .to_string_lossy() - .starts_with(".nemo-relay-hermes-home") - })); -} - -#[cfg(unix)] -#[test] -fn process_private_directories_are_owner_only() { - use std::os::unix::fs::PermissionsExt; - - let parent = tempfile::tempdir().unwrap(); - let path = crate::filesystem::temp::private_temp_dir(parent.path(), "relay-private").unwrap(); - assert_eq!( - std::fs::metadata(&path).unwrap().permissions().mode() & 0o777, - 0o700 - ); - std::fs::remove_dir(path).unwrap(); -} - -#[test] -fn concurrent_hermes_runs_use_independent_overlays_without_mutating_user_config() { - let temp = tempfile::tempdir().unwrap(); - let config = temp.path().join("hermes/config.yaml"); - std::fs::create_dir_all(config.parent().unwrap()).unwrap(); - let original = "model:\n default: test\n"; - std::fs::write(&config, original).unwrap(); - let resolved = ResolvedConfig { - agents: AgentConfigs { - hermes: AgentCommandConfig { - hooks_path: Some(config.clone()), - ..AgentCommandConfig::default() - }, - ..AgentConfigs::default() - }, - ..ResolvedConfig::default() - }; - - let resolved = std::sync::Arc::new(resolved); - let barrier = std::sync::Arc::new(std::sync::Barrier::new(3)); - let spawn = |url: &'static str| { - let resolved = resolved.clone(); - let barrier = barrier.clone(); - std::thread::spawn(move || { - barrier.wait(); - PreparedAgentLaunch::new( - CodingAgent::Hermes, - vec!["hermes".into()], - url, - &resolved, - false, - ) - .unwrap() - }) - }; - let first = spawn("http://127.0.0.1:4001"); - let second = spawn("http://127.0.0.1:4002"); - barrier.wait(); - let first = first.join().unwrap(); - let second = second.join().unwrap(); - let overlay = |run: &PreparedAgentLaunch| { - run.env - .iter() - .find_map(|(name, value)| (name == "HERMES_HOME").then(|| PathBuf::from(value))) - .unwrap() - }; - let first_overlay = overlay(&first); - let second_overlay = overlay(&second); - - assert_ne!(first_overlay, second_overlay); - let first_config: serde_json::Value = - serde_yaml::from_str(&std::fs::read_to_string(first_overlay.join("config.yaml")).unwrap()) - .unwrap(); - let second_config: serde_json::Value = - serde_yaml::from_str(&std::fs::read_to_string(second_overlay.join("config.yaml")).unwrap()) - .unwrap(); - assert!(crate::hook_assertions::value_has_command_arguments( - &first_config, - &[ - "hook-forward", - "hermes", - "--gateway-url", - "http://127.0.0.1:4001", - "--transparent-run", - ], - )); - assert!(crate::hook_assertions::value_has_command_arguments( - &second_config, - &[ - "hook-forward", - "hermes", - "--gateway-url", - "http://127.0.0.1:4002", - "--transparent-run", - ], - )); - assert_eq!(std::fs::read_to_string(&config).unwrap(), original); - - first.restore().unwrap(); - assert!(!first_overlay.exists()); - assert!(second_overlay.exists()); - assert_eq!(std::fs::read_to_string(&config).unwrap(), original); - second.restore().unwrap(); -} - -#[test] -fn hermes_overlay_does_not_link_an_ancestor_entry_that_contains_it() { - let source_home = tempfile::tempdir().unwrap(); - let source_config = source_home.path().join("config.yaml"); - std::fs::write(&source_config, "model:\n default: test\n").unwrap(); - let overlay = source_home.path().join("overlay"); - std::fs::create_dir(&overlay).unwrap(); - - crate::agents::hermes::launch::populate_overlay( - &overlay, - source_home.path(), - &source_config, - "http://127.0.0.1:1234", - ) - .unwrap(); - - assert!(!overlay.join("overlay").exists()); - assert!(overlay.join("config.yaml").exists()); -} - -#[test] -fn prepares_hermes_dry_uses_home_path_without_writing_hooks() { - let _guard = current_dir_lock().lock().unwrap(); - let temp = tempfile::tempdir().unwrap(); - let _env = EnvScope::set(&[ - ("HERMES_HOME", None), - ("HOME", Some(temp.path().as_os_str())), - ("USERPROFILE", None), - ]); - let resolved = ResolvedConfig { - gateway: GatewayConfig::default(), - agents: AgentConfigs::default(), - ..ResolvedConfig::default() - }; - - let prepared = PreparedAgentLaunch::new( - CodingAgent::Hermes, - vec!["hermes".into()], - "http://127.0.0.1:1234", - &resolved, - true, - ) - .unwrap(); - - let hook_path = temp.path().join(".hermes/config.yaml"); - assert!(prepared.notes[0].contains(".hermes")); - assert!(prepared.notes[0].contains("config.yaml")); - assert!( - prepared - .env - .contains(&("HERMES_ACCEPT_HOOKS".into(), "1".into())) - ); - assert!(!hook_path.exists()); -} - -#[test] -fn hermes_hooks_path_prefers_configured_then_env_then_home() { - let _guard = current_dir_lock().lock().unwrap(); - let temp = tempfile::tempdir().unwrap(); - let configured = temp.path().join("configured.yaml"); - assert_eq!( - crate::agents::hermes::launch::hooks_path_for_launch(Some(&configured)).unwrap(), - configured - ); - - let _env = EnvScope::set(&[ - ("HERMES_HOME", Some(temp.path().as_os_str())), - ("HOME", None), - ("USERPROFILE", None), - ]); - assert_eq!( - crate::agents::hermes::launch::hooks_path_for_launch(None).unwrap(), - temp.path().join("config.yaml") - ); - - drop(_env); - let _env = EnvScope::set(&[ - ("HERMES_HOME", None), - ("HOME", Some(temp.path().as_os_str())), - ("USERPROFILE", None), - ]); - assert_eq!( - crate::agents::hermes::launch::hooks_path_for_launch(None).unwrap(), - temp.path().join(".hermes/config.yaml") - ); - - drop(_env); - let _env = EnvScope::set(&[("HERMES_HOME", None), ("HOME", None), ("USERPROFILE", None)]); - let error = crate::agents::hermes::launch::hooks_path_for_launch(None) - .unwrap_err() - .to_string(); - assert!(error.contains("could not resolve home directory")); -} - -#[test] -fn hermes_overlay_preserves_ambiguous_manual_mcp_and_original_file() { - let _guard = current_dir_lock().lock().unwrap(); - let temp = tempfile::tempdir().unwrap(); - let hooks_path = temp.path().join("hermes-home/config.yaml"); - std::fs::create_dir_all(hooks_path.parent().unwrap()).unwrap(); - let original = r#"mcp_servers: - nemo-relay: - command: nemo-relay - args: [mcp, --agent, hermes] - filesystem: - command: fs-mcp -hooks: - PreToolUse: [] -"#; - std::fs::write(&hooks_path, original).unwrap(); - let resolved = ResolvedConfig { - gateway: GatewayConfig::default(), - agents: AgentConfigs { - hermes: AgentCommandConfig { - command: None, - hooks_path: Some(hooks_path.clone()), - }, - ..AgentConfigs::default() - }, - ..ResolvedConfig::default() - }; - - let prepared = PreparedAgentLaunch::new( - CodingAgent::Hermes, - vec!["hermes".into(), "chat".into()], - "http://s", - &resolved, - false, - ) - .unwrap(); - - let overlay = prepared - .env - .iter() - .find_map(|(name, value)| (name == "HERMES_HOME").then(|| PathBuf::from(value))) - .unwrap(); - let patched = std::fs::read_to_string(overlay.join("config.yaml")).unwrap(); - let patched_yaml: serde_json::Value = serde_yaml::from_str(&patched).unwrap(); - assert!(crate::hook_assertions::value_has_command_arguments( - &patched_yaml, - &[ - "hook-forward", - "hermes", - "--gateway-url", - "http://s", - "--transparent-run", - ], - )); - assert_eq!( - patched_yaml["mcp_servers"]["nemo-relay"]["args"], - json!(["mcp", "--agent", "hermes"]) - ); - assert_eq!( - patched_yaml["mcp_servers"]["filesystem"]["command"], - json!("fs-mcp") - ); - assert_eq!(std::fs::read_to_string(&hooks_path).unwrap(), original); - prepared.restore().unwrap(); - assert!(!overlay.exists()); -} - #[test] fn prepares_claude_temp_plugin() { let resolved = ResolvedConfig { @@ -2139,78 +1633,6 @@ async fn execute_live_run_reports_gateway_startup_error_when_health_check_fails( assert!(!error.contains("gateway did not become ready")); } -#[tokio::test] -#[allow(clippy::await_holding_lock)] -async fn execute_live_run_removes_hermes_overlay_when_health_check_fails() { - let _guard = crate::test_support::PLUGIN_CONFIG_TEST_LOCK.lock().await; - let _cwd = crate::test_support::CwdTestScope::locked(); - let temp = tempfile::tempdir().unwrap(); - let _env = EnvScope::set(&[ - (crate::bootstrap::state::BOOTSTRAP_STATE_DIR_ENV, None), - ("NEMO_RELAY_BOOTSTRAP_SHUTDOWN_TOKEN", None), - (crate::configuration::BOOTSTRAP_FINGERPRINT_ENV, None), - ("HOME", Some(temp.path().as_os_str())), - ("XDG_CONFIG_HOME", Some(temp.path().join("xdg").as_os_str())), - ]); - let _ = nemo_relay::plugin::clear_plugin_configuration(); - let hooks_path = temp.path().join("hermes-home/config.yaml"); - std::fs::create_dir_all(hooks_path.parent().unwrap()).unwrap(); - let original = "hooks:\n PreToolUse: []\n"; - std::fs::write(&hooks_path, original).unwrap(); - let resolved = ResolvedConfig { - gateway: GatewayConfig::default(), - agents: AgentConfigs { - hermes: AgentCommandConfig { - command: None, - hooks_path: Some(hooks_path.clone()), - }, - ..AgentConfigs::default() - }, - ..ResolvedConfig::default() - }; - let prepared = PreparedAgentLaunch::new( - CodingAgent::Hermes, - vec!["hermes".into(), "chat".into()], - "http://127.0.0.1:1234", - &resolved, - false, - ) - .unwrap(); - let overlay = prepared - .env - .iter() - .find_map(|(name, value)| (name == "HERMES_HOME").then(|| PathBuf::from(value))) - .unwrap(); - let overlay_config: serde_json::Value = - serde_yaml::from_str(&std::fs::read_to_string(overlay.join("config.yaml")).unwrap()) - .unwrap(); - assert!(crate::hook_assertions::value_has_command_arguments( - &overlay_config, - &[ - "hook-forward", - "hermes", - "--gateway-url", - "http://127.0.0.1:1234", - "--transparent-run", - ], - )); - - let listener = TcpListener::bind("127.0.0.1:0").await.unwrap(); - let error = execute_live_run( - listener, - GatewayConfig::default(), - "http://127.0.0.1:1", - prepared, - ) - .await - .unwrap_err() - .to_string(); - - assert!(error.contains("gateway did not become ready"), "{error}"); - assert_eq!(std::fs::read_to_string(&hooks_path).unwrap(), original); - assert!(!overlay.exists()); -} - #[cfg(unix)] fn make_executable(path: &Path) { use std::os::unix::fs::PermissionsExt; diff --git a/crates/cli/tests/coverage/agents/plugin_host_tests.rs b/crates/cli/tests/coverage/agents/plugin_host_tests.rs index 9d6b15447..9a74fb569 100644 --- a/crates/cli/tests/coverage/agents/plugin_host_tests.rs +++ b/crates/cli/tests/coverage/agents/plugin_host_tests.rs @@ -2005,21 +2005,10 @@ supports_websockets = false } #[test] -fn plugin_host_doctor_rejects_unsupported_agents_and_reports_lazy_claude_status() { +fn plugin_host_doctor_reports_lazy_claude_status() { let dir = tempdir().unwrap(); let _home = HomeScope::enter(dir.path()); - assert!( - doctor_plugin(CodingAgent::Hermes, DEFAULT_URL, dir.path()) - .unwrap_err() - .contains("supports claude and codex") - ); - assert!( - doctor_plugin_json(CodingAgent::Hermes, DEFAULT_URL, dir.path()) - .unwrap_err() - .contains("supports claude and codex") - ); - let report = doctor_plugin_json(CodingAgent::ClaudeCode, DEFAULT_URL, dir.path()).unwrap(); assert_eq!(report["ok"], json!(false)); assert_eq!(report["sidecar_health"], json!("not_running_mcp_start")); @@ -3799,7 +3788,7 @@ fn claude_restore_removes_settings_created_from_an_absent_original() { } #[test] -fn plugin_host_entrypoints_reject_unsupported_agents_and_report_json() { +fn plugin_host_entrypoints_report_json() { let dir = tempdir().unwrap(); let _home = HomeScope::enter(dir.path()); let settings_path = dir.path().join(".claude").join("settings.json"); @@ -3843,16 +3832,6 @@ fn plugin_host_entrypoints_reject_unsupported_agents_and_report_json() { ); assert_eq!(codex_report["checks"]["codex_provider_alias"], json!(false)); assert_eq!(codex_report["checks"]["codex_hooks"], json!(false)); - assert!( - doctor_plugin_json(CodingAgent::Hermes, DEFAULT_URL, &plugin_root) - .unwrap_err() - .contains("supports claude and codex") - ); - assert!( - doctor_plugin(CodingAgent::Hermes, DEFAULT_URL, &plugin_root) - .unwrap_err() - .contains("supports claude and codex") - ); assert!( doctor_plugin(CodingAgent::Codex, DEFAULT_URL, &plugin_root) .unwrap_err() diff --git a/crates/cli/tests/coverage/agents/plugin_install_tests.rs b/crates/cli/tests/coverage/agents/plugin_install_tests.rs index fb68ef9b5..6cfe1807f 100644 --- a/crates/cli/tests/coverage/agents/plugin_install_tests.rs +++ b/crates/cli/tests/coverage/agents/plugin_install_tests.rs @@ -100,19 +100,6 @@ fn readiness_worker_returns_a_report_and_handles_channel_disconnects() { .details .contains("collector stopped unexpectedly") ); - - let (sender, receiver) = std::sync::mpsc::sync_channel(1); - drop(sender); - let hermes = crate::agents::receive_integration_readiness_for_test( - CodingAgent::Hermes, - dir.path().join("config.yaml"), - receiver, - dir.path(), - Duration::from_secs(1), - ); - assert!(hermes.marketplace.is_none()); - assert!(hermes.plugin.is_none()); - assert!(!hermes.ok()); } #[test] @@ -367,7 +354,6 @@ struct HomeScope<'a> { prev_home: Option, prev_userprofile: Option, prev_codex_home: Option, - prev_hermes_home: Option, } impl<'a> HomeScope<'a> { @@ -378,20 +364,17 @@ impl<'a> HomeScope<'a> { let prev_home = std::env::var_os("HOME"); let prev_userprofile = std::env::var_os("USERPROFILE"); let prev_codex_home = std::env::var_os("CODEX_HOME"); - let prev_hermes_home = std::env::var_os("HERMES_HOME"); // SAFETY: This test holds a process-wide mutex for the lifetime of the env override. unsafe { std::env::set_var("HOME", path); std::env::remove_var("USERPROFILE"); std::env::remove_var("CODEX_HOME"); - std::env::remove_var("HERMES_HOME"); } Self { _guard: guard, prev_home, prev_userprofile, prev_codex_home, - prev_hermes_home, } } @@ -402,20 +385,17 @@ impl<'a> HomeScope<'a> { let prev_home = std::env::var_os("HOME"); let prev_userprofile = std::env::var_os("USERPROFILE"); let prev_codex_home = std::env::var_os("CODEX_HOME"); - let prev_hermes_home = std::env::var_os("HERMES_HOME"); // SAFETY: This test holds a process-wide mutex for the lifetime of the env override. unsafe { std::env::remove_var("HOME"); std::env::remove_var("USERPROFILE"); std::env::remove_var("CODEX_HOME"); - std::env::remove_var("HERMES_HOME"); } Self { _guard: guard, prev_home, prev_userprofile, prev_codex_home, - prev_hermes_home, } } } @@ -436,10 +416,6 @@ impl Drop for HomeScope<'_> { Some(value) => std::env::set_var("CODEX_HOME", value), None => std::env::remove_var("CODEX_HOME"), } - match self.prev_hermes_home.take() { - Some(value) => std::env::set_var("HERMES_HOME", value), - None => std::env::remove_var("HERMES_HOME"), - } } } } @@ -458,7 +434,6 @@ struct PathScope<'a> { previous: Option, previous_home: Option, previous_codex_home: Option, - previous_hermes_home: Option, } impl<'a> PathScope<'a> { @@ -469,20 +444,17 @@ impl<'a> PathScope<'a> { let previous = std::env::var_os("PATH"); let previous_home = std::env::var_os("HOME"); let previous_codex_home = std::env::var_os("CODEX_HOME"); - let previous_hermes_home = std::env::var_os("HERMES_HOME"); // SAFETY: This test holds the process-wide environment mutex for the override lifetime. unsafe { std::env::set_var("PATH", path); std::env::set_var("HOME", home); std::env::remove_var("CODEX_HOME"); - std::env::remove_var("HERMES_HOME"); } Self { _guard: guard, previous, previous_home, previous_codex_home, - previous_hermes_home, } } } @@ -503,10 +475,6 @@ impl Drop for PathScope<'_> { Some(value) => std::env::set_var("CODEX_HOME", value), None => std::env::remove_var("CODEX_HOME"), } - match self.previous_hermes_home.take() { - Some(value) => std::env::set_var("HERMES_HOME", value), - None => std::env::remove_var("HERMES_HOME"), - } } } } @@ -2113,7 +2081,6 @@ fn top_level_install_uninstall_and_doctor_report_empty_host_selection() { fn installed_selection_uses_persisted_integration_state() { let dir = tempdir().unwrap(); let home = dir.path().join("home"); - std::fs::create_dir_all(home.join(".hermes")).unwrap(); let _home = HomeScope::enter(&home); std::fs::write( state_path(CodingAgent::ClaudeCode, dir.path()), @@ -2122,133 +2089,6 @@ fn installed_selection_uses_persisted_integration_state() { .unwrap(); let selected = crate::agents::installed_integrations(&CodingAgent::ALL, Some(dir.path())); assert_eq!(selected, vec![CodingAgent::ClaudeCode]); - - let unrelated_hermes_config = b"# user-owned formatting\nmodel: custom\n"; - let hermes_config = home.join(".hermes/config.yaml"); - std::fs::write(&hermes_config, unrelated_hermes_config).unwrap(); - let selected = crate::agents::installed_integrations(&CodingAgent::ALL, Some(dir.path())); - assert_eq!(selected, vec![CodingAgent::ClaudeCode]); - assert_eq!( - std::fs::read(&hermes_config).unwrap(), - unrelated_hermes_config - ); -} - -#[test] -fn hermes_doctor_probes_the_configured_relay_and_top_level_doctor_discovers_it() { - let dir = tempdir().unwrap(); - let home = dir.path().join("home"); - std::fs::create_dir_all(&home).unwrap(); - let _home = HomeScope::enter(&home); - let config = crate::agents::hermes::install::config_path().unwrap(); - let relay = home - .join("bin") - .join(format!("nemo-relay{}", std::env::consts::EXE_SUFFIX)); - std::fs::create_dir_all(relay.parent().unwrap()).unwrap(); - std::fs::copy(std::env::current_exe().unwrap(), &relay).unwrap(); - crate::agents::hermes::install_persistent(&config, &relay).unwrap(); - let configured_relay = crate::agents::hermes::configured_relay_executable(&config).unwrap(); - let runner = MockRunner::default() - .with_executable("hermes", "/bin/hermes") - .with_capture_output("/bin/hermes --version", "Hermes Agent v0.18.2 (test)\n"); - - let report = - crate::agents::hermes::install::doctor_json_value(&options(dir.path()), &runner).unwrap(); - - assert_eq!(report["ok"], json!(true)); - assert_eq!( - runner.quiet_commands(), - vec![ - format!("{} hook-forward --help", configured_relay.display()), - format!("{} mcp --help", configured_relay.display()), - ] - ); - let checks = report["readiness_checks"].as_array().unwrap(); - for expected in [ - "Host CLI", - "Hermes Agent version", - "Configured Relay binary", - "Relay hook support", - "Relay MCP support", - "Hermes MCP, hooks, and trust", - ] { - assert!( - checks - .iter() - .any(|check| check["name"] == expected && check["ok"] == json!(true)), - "missing successful {expected} check: {checks:?}" - ); - } - - let readiness = crate::agents::collect_default_integration_readiness(); - let hermes = readiness - .iter() - .find(|readiness| readiness.host == "hermes") - .expect("top-level doctor should discover install-only Hermes state"); - assert_eq!(hermes.state_path, config); - assert!(hermes.marketplace.is_none()); - assert!(hermes.plugin.is_none()); - - crate::agents::hermes::install::doctor(&options(dir.path()), &runner).unwrap(); - - std::fs::remove_file(&configured_relay).unwrap(); - let error = crate::agents::hermes::install::doctor(&options(dir.path()), &runner).unwrap_err(); - assert!(error.contains("doctor checks failed"), "{error}"); - let report = - crate::agents::hermes::install::doctor_json_value(&options(dir.path()), &runner).unwrap(); - let failed = report["readiness_checks"].as_array().unwrap(); - for expected in [ - "Configured Relay binary", - "Relay hook support", - "Relay MCP support", - ] { - assert!( - failed - .iter() - .any(|check| check["name"] == expected && check["ok"] == json!(false)), - "missing failed {expected} check: {failed:?}" - ); - } -} - -#[test] -fn hermes_install_and_uninstall_dry_runs_preserve_persistent_state() { - let dir = tempdir().unwrap(); - let home = dir.path().join("home"); - std::fs::create_dir_all(&home).unwrap(); - let _home = HomeScope::enter(&home); - crate::agents::hermes::install::install(crate::installation::InstallRequest { - install_dir: Some(dir.path().to_path_buf()), - force: false, - dry_run: true, - skip_doctor: false, - }) - .unwrap(); - let config = crate::agents::hermes::install::config_path().unwrap(); - assert!(!config.exists()); - - let hermes_home = config.parent().unwrap(); - std::fs::create_dir_all(hermes_home).unwrap(); - let allowlist = hermes_home.join("shell-hooks-allowlist.json"); - let generation = hermes_home.join(GENERATION_FILE_NAME); - let sentinels = [ - (&config, b"sentinel config\n".as_slice()), - (&allowlist, b"sentinel allowlist\n".as_slice()), - (&generation, b"sentinel generation\n".as_slice()), - ]; - for (path, contents) in sentinels { - std::fs::write(path, contents).unwrap(); - } - - crate::agents::hermes::install::uninstall(crate::installation::UninstallRequest { - install_dir: Some(dir.path().to_path_buf()), - dry_run: true, - }) - .unwrap(); - - for (path, contents) in sentinels { - assert_eq!(std::fs::read(path).unwrap(), contents); - } } #[test] diff --git a/crates/cli/tests/coverage/commands/main_tests.rs b/crates/cli/tests/coverage/commands/main_tests.rs index 9aa1492a3..5f9399a6f 100644 --- a/crates/cli/tests/coverage/commands/main_tests.rs +++ b/crates/cli/tests/coverage/commands/main_tests.rs @@ -353,7 +353,6 @@ fn multi_agent_operations_attempt_every_target_before_reporting_errors() { match agent { CodingAgent::Codex => Err(error::CliError::Install("codex failure".into())), CodingAgent::ClaudeCode => Ok(ExitCode::FAILURE), - CodingAgent::Hermes => Ok(ExitCode::SUCCESS), } }) .unwrap_err() diff --git a/crates/cli/tests/coverage/shared/config_tests.rs b/crates/cli/tests/coverage/shared/config_tests.rs index d6b0e70d4..d635cf991 100644 --- a/crates/cli/tests/coverage/shared/config_tests.rs +++ b/crates/cli/tests/coverage/shared/config_tests.rs @@ -544,7 +544,6 @@ fn session_config_uses_defaults_and_ignores_bad_json() { fn agent_and_gateway_mode_arguments_are_stable() { assert_eq!(CodingAgent::ClaudeCode.hook_path(), "/hooks/claude-code"); assert_eq!(CodingAgent::Codex.hook_path(), "/hooks/codex"); - assert_eq!(CodingAgent::Hermes.hook_path(), "/hooks/hermes"); assert_eq!(GatewayMode::HookOnly.as_arg(), "hook-only"); assert_eq!(GatewayMode::Passthrough.as_arg(), "passthrough"); assert_eq!(GatewayMode::Required.as_arg(), "required"); @@ -558,7 +557,7 @@ fn agent_inference_uses_executable_basename() { ); assert_eq!(CodingAgent::infer("codex"), Some(CodingAgent::Codex)); assert_eq!(CodingAgent::infer("cursor-agent"), None); - assert_eq!(CodingAgent::infer("hermes"), Some(CodingAgent::Hermes)); + assert_eq!(CodingAgent::infer("hermes"), None); assert_eq!(CodingAgent::infer("wrapper"), None); } @@ -587,8 +586,6 @@ command = "claude" [agents.codex] command = "codex --approval-mode never" -[agents.hermes] -command = "hermes --yolo chat" "#, ) .unwrap(); @@ -625,10 +622,21 @@ command = "hermes --yolo chat" resolved.agents.codex.command.as_deref(), Some("codex --approval-mode never") ); - assert_eq!( - resolved.agents.hermes.command.as_deref(), - Some("hermes --yolo chat") - ); +} + +#[test] +fn stale_hermes_agent_config_is_rejected() { + let value = toml::from_str::( + r#" +[agents.hermes] +command = "hermes" +"#, + ) + .unwrap(); + + let error = validate_shared_config_shape(value).unwrap_err().to_string(); + + assert!(error.contains("unknown field `hermes`")); } #[test] diff --git a/crates/cli/tests/coverage/shared/doctor_tests.rs b/crates/cli/tests/coverage/shared/doctor_tests.rs index 9c47eeea1..96f161d8f 100644 --- a/crates/cli/tests/coverage/shared/doctor_tests.rs +++ b/crates/cli/tests/coverage/shared/doctor_tests.rs @@ -635,7 +635,7 @@ fn collect_configuration_uses_xdg_global_path_and_renders_resolution_branches() status: Status::Warn, details: "using fallback layer".into(), }, - vec!["codex".into(), "hermes".into()], + vec!["codex".into(), "claude".into()], &PluginConfigurationDiagnostics { sources: vec![], error: None, @@ -668,7 +668,7 @@ fn collect_configuration_uses_xdg_global_path_and_renders_resolution_branches() let rendered = format_human(&report); assert!(rendered.contains("Global")); assert!(rendered.contains("Resolution ! using fallback layer")); - assert!(rendered.contains("Agents codex, hermes")); + assert!(rendered.contains("Agents codex, claude")); } #[test] @@ -695,10 +695,7 @@ fn agent_helper_statuses_cover_configured_target_and_hook_paths() { Status::Pass ); - let mut agents = AgentConfigs::default(); - agents.hermes.hooks_path = Some(PathBuf::from("/tmp/hermes.yaml")); - assert!(agent_configured(CodingAgent::Hermes, &agents)); - assert_eq!(configured_agent_names(&agents), vec!["hermes".to_string()]); + let agents = AgentConfigs::default(); assert_eq!( hook_status(CodingAgent::ClaudeCode, &agents), (Status::Pass, "hooks: injected during run".into()) @@ -707,13 +704,6 @@ fn agent_helper_statuses_cover_configured_target_and_hook_paths() { hook_status(CodingAgent::Codex, &agents), (Status::Pass, "hooks: injected during run".into()) ); - assert_eq!( - hook_status(CodingAgent::Hermes, &AgentConfigs::default()), - ( - Status::Pass, - "hooks: injected through an isolated HERMES_HOME during run".into() - ) - ); } #[test] @@ -849,20 +839,6 @@ async fn collect_agents_distinguishes_required_and_optional_version_failures() { assert_eq!(optional.status, Status::Warn); assert!(optional.annotation.contains("could not determine version")); } - -#[test] -fn hermes_hook_status_reports_actionable_persistent_diagnosis_failures() { - let temp = tempfile::tempdir().unwrap(); - let mut agents = AgentConfigs::default(); - agents.hermes.hooks_path = Some(temp.path().join("missing-config.yaml")); - - let (status, details) = hook_status(CodingAgent::Hermes, &agents); - - assert_eq!(status, Status::Fail); - assert!(details.contains("persistent MCP/hooks"), "{details}"); - assert!(details.contains("install hermes --force"), "{details}"); -} - #[cfg(unix)] #[tokio::test] async fn probe_version_returns_none_for_empty_output_and_spawn_failures() { diff --git a/crates/cli/tests/coverage/shared/events_tests.rs b/crates/cli/tests/coverage/shared/events_tests.rs index 3f2103a10..de344d5db 100644 --- a/crates/cli/tests/coverage/shared/events_tests.rs +++ b/crates/cli/tests/coverage/shared/events_tests.rs @@ -10,7 +10,6 @@ use super::json_path::{string_at, string_at_any, value_at, value_at_any}; fn agent_kinds_use_stable_runtime_metadata_names() { assert_eq!(AgentKind::Codex.as_str(), "codex"); assert_eq!(AgentKind::ClaudeCode.as_str(), "claude-code"); - assert_eq!(AgentKind::Hermes.as_str(), "hermes"); assert_eq!(AgentKind::Gateway.as_str(), "gateway"); } diff --git a/crates/cli/tests/coverage/shared/gateway_tests.rs b/crates/cli/tests/coverage/shared/gateway_tests.rs index a775ddf93..86e672ad0 100644 --- a/crates/cli/tests/coverage/shared/gateway_tests.rs +++ b/crates/cli/tests/coverage/shared/gateway_tests.rs @@ -1436,7 +1436,7 @@ fn strips_chatgpt_plus_jwt_from_openai_route_inbound() { #[test] fn preserves_real_bearer_keys_on_openai_route() { - // Real provider keys (Hermes's `sk-...` against NVIDIA, an actual OpenAI dev key, etc.) + // Real provider keys (an NVIDIA key, an actual OpenAI development key, and so on) // must pass through untouched — only recognized ChatGPT auth tokens are stripped. let mut inbound = HeaderMap::new(); inbound.insert( diff --git a/crates/cli/tests/coverage/shared/installer_tests.rs b/crates/cli/tests/coverage/shared/installer_tests.rs index 40d82f12d..1064373a0 100644 --- a/crates/cli/tests/coverage/shared/installer_tests.rs +++ b/crates/cli/tests/coverage/shared/installer_tests.rs @@ -330,16 +330,6 @@ fn helper_formatting_and_headers_cover_optional_paths() { #[test] fn generated_hook_dispatch_covers_all_agents() { assert_generated_hook_policies(); - assert_eq!( - transparent_hook_forward_commands_for_platform( - Path::new("nemo-relay"), - CodingAgent::Hermes, - "http://127.0.0.1:1234", - false, - ) - .for_event("on_session_start"), - "nemo-relay hook-forward hermes --gateway-url http://127.0.0.1:1234 --transparent-run --fail-open" - ); assert_eq!( transparent_hook_forward_commands_for_platform( Path::new("/abs/path/to/nemo-relay"), @@ -363,7 +353,7 @@ fn generated_hook_dispatch_covers_all_agents() { ); let native = transparent_hook_forward_commands( Path::new("nemo-relay"), - CodingAgent::Hermes, + CodingAgent::Codex, "http://127.0.0.1:1234", ) .unwrap(); @@ -373,7 +363,7 @@ fn generated_hook_dispatch_covers_all_agents() { vec![ String::from("nemo-relay"), String::from("hook-forward"), - String::from("hermes"), + String::from("codex"), String::from("--gateway-url"), String::from("http://127.0.0.1:1234"), String::from("--transparent-run"), @@ -385,7 +375,7 @@ fn generated_hook_dispatch_covers_all_agents() { native, transparent_hook_forward_commands_for_platform( Path::new("nemo-relay"), - CodingAgent::Hermes, + CodingAgent::Codex, "http://127.0.0.1:1234", false, ) @@ -447,21 +437,14 @@ fn generated_hook_dispatch_covers_all_agents() { } fn assert_generated_hook_policies() { - for agent in [ - CodingAgent::ClaudeCode, - CodingAgent::Codex, - CodingAgent::Hermes, - ] { + for agent in [CodingAgent::ClaudeCode, CodingAgent::Codex] { assert!(generated_hooks(agent, "cmd")["hooks"].is_object()); let commands = GeneratedHookCommands::new("cmd --fail-open", "cmd --fail-closed"); let generated = generated_policy_hooks(agent, &commands); for event in agent.hook_events() { - let command = if agent.uses_direct_hook_entries() { - generated["hooks"][event][0]["command"].as_str() - } else { - generated["hooks"][event][0]["hooks"][0]["command"].as_str() - } - .unwrap(); + let command = generated["hooks"][event][0]["hooks"][0]["command"] + .as_str() + .unwrap(); assert_eq!( command, commands.for_event(event), diff --git a/crates/cli/tests/coverage/shared/server_tests.rs b/crates/cli/tests/coverage/shared/server_tests.rs index e25c048c7..a451ab903 100644 --- a/crates/cli/tests/coverage/shared/server_tests.rs +++ b/crates/cli/tests/coverage/shared/server_tests.rs @@ -927,38 +927,6 @@ async fn serve_listener_exits_after_codex_stop_without_session_end() { .unwrap(); result.unwrap(); } - -#[tokio::test] -async fn serve_listener_exits_after_hermes_turn_without_session_finalize() { - let _guard = PLUGIN_CONFIG_TEST_LOCK.lock().await; - let _env = EnvVarGuard::set("NEMO_RELAY_PLUGIN_IDLE_TIMEOUT_SECS", "1"); - - let listener = TcpListener::bind("127.0.0.1:0").await.unwrap(); - let address = listener.local_addr().unwrap(); - let url = format!("http://{address}"); - let handle = tokio::spawn(async move { serve_listener(listener, test_config(), None).await }); - let client = test_http_client(); - - for hook_event_name in ["on_session_start", "on_session_end"] { - let response = client - .post(format!("{url}/hooks/hermes")) - .json(&json!({ - "session_id": "plugin-idle-hermes-session", - "hook_event_name": hook_event_name - })) - .send() - .await - .unwrap(); - assert!(response.status().is_success()); - } - - let result = tokio::time::timeout(std::time::Duration::from_secs(3), handle) - .await - .expect("plugin idle timeout should stop after the Hermes turn ends") - .unwrap(); - result.unwrap(); -} - #[tokio::test] async fn serve_listener_activates_plugin_config_and_clears_on_shutdown() { let _guard = PLUGIN_CONFIG_TEST_LOCK.lock().await; @@ -1008,9 +976,9 @@ async fn serve_listener_activates_plugin_config_and_clears_on_shutdown() { assert!(nemo_relay::plugin::active_plugin_report().is_some()); let client = test_http_client(); - for hook_event_name in ["on_session_start", "on_session_finalize"] { + for hook_event_name in ["SessionStart", "Stop"] { let response = client - .post(format!("{url}/hooks/hermes")) + .post(format!("{url}/hooks/codex")) .json(&json!({ "session_id": "plugin-bridge-session", "hook_event_name": hook_event_name @@ -1027,8 +995,8 @@ async fn serve_listener_activates_plugin_config_and_clears_on_shutdown() { let events = std::fs::read_to_string(temp.path().join("atof/events.jsonl")).unwrap(); assert!( - events.lines().count() >= 2, - "expected ATOF lifecycle events, got {events:?}" + events.lines().count() >= 1, + "expected an ATOF lifecycle event, got {events:?}" ); let trajectories = std::fs::read_dir(temp.path().join("atif")) .unwrap() @@ -1053,25 +1021,8 @@ async fn serve_listener_activates_plugin_config_and_clears_on_shutdown() { ); } -fn atif_matches_session(trajectory: &Value, session_id: &str) -> bool { - trajectory["session_id"] == json!(session_id) - || trajectory["extra"]["observed_events"] - .as_array() - .is_some_and(|events| { - events - .iter() - .any(|event| event_has_session_id(event, session_id)) - }) -} - -fn event_has_session_id(event: &Value, session_id: &str) -> bool { - event["metadata"]["session_id"] == json!(session_id) - || event["data"]["session_id"] == json!(session_id) - || event["data"]["extra"]["session_id"] == json!(session_id) -} - #[tokio::test] -async fn serve_listener_observability_plugin_records_non_hermes_hooks() { +async fn serve_listener_observability_plugin_records_supported_agent_hooks() { let _guard = PLUGIN_CONFIG_TEST_LOCK.lock().await; let _ = nemo_relay::plugin::clear_plugin_configuration(); @@ -1124,8 +1075,7 @@ async fn serve_listener_observability_plugin_records_non_hermes_hooks() { "SessionEnd", ), ] { - let hook_events = vec![start_event, "UserPromptSubmit", end_event]; - for hook_event_name in hook_events { + for hook_event_name in [start_event, "UserPromptSubmit", end_event] { let response = client .post(format!("{url}{path}")) .json(&json!({ @@ -1159,477 +1109,22 @@ async fn serve_listener_observability_plugin_records_non_hermes_hooks() { assert!(!turn_starts.contains(&"claude-code".to_string())); } -#[tokio::test] -async fn serve_listener_hermes_api_hooks_write_atof_category_profile_and_fidelity() { - let _guard = PLUGIN_CONFIG_TEST_LOCK.lock().await; - let _ = nemo_relay::plugin::clear_plugin_configuration(); - - let temp = tempfile::tempdir().unwrap(); - let atof_dir = temp.path().join("atof"); - std::fs::create_dir_all(&atof_dir).unwrap(); - let mut config = test_config(); - config.plugin_config = Some(json!({ - "version": 1, - "components": [ - { - "kind": "observability", - "enabled": true, - "config": { - "version": 3, - "atof": { - "enabled": true, - "sinks": [{ - "type": "file", - "output_directory": atof_dir, - "filename": "events.jsonl", - "mode": "overwrite" - }] - } - } - } - ] - })); - - let listener = TcpListener::bind("127.0.0.1:0").await.unwrap(); - let address = listener.local_addr().unwrap(); - let url = format!("http://{address}"); - let (shutdown_tx, shutdown_rx) = oneshot::channel(); - let handle = - tokio::spawn(async move { serve_listener(listener, config, Some(shutdown_rx)).await }); - - wait_for_gateway(&url).await; - let client = test_http_client(); - - let response = client - .post(format!("{url}/hooks/hermes")) - .json(&json!({ - "hook_event_name": "pre_api_request", - "session_id": "hermes-atof-exact", - "extra": { - "task_id": "task-1", - "api_request_id": "turn-1:api:2", - "api_call_count": 2, - "model": "qwen", - "provider": "custom", - "request": { - "method": "POST", - "body": { - "model": "qwen", - "messages": [ - { "role": "user", "content": "hello" } - ], - "tools": [ - { "type": "function", "function": { "name": "search_files" } } - ] - } - } - } - })) - .send() - .await - .unwrap(); - assert_eq!(response.status(), StatusCode::OK); - - let response = client - .post(format!("{url}/hooks/hermes")) - .json(&json!({ - "hook_event_name": "post_api_request", - "session_id": "hermes-atof-exact", - "extra": { - "task_id": "task-1", - "api_request_id": "turn-1:api:2", - "api_call_count": 2, - "model": "qwen", - "response": { - "model": "qwen", - "finish_reason": "tool_calls", - "assistant_message": { - "role": "assistant", - "content": "", - "tool_calls": [ - { - "id": "call-1", - "type": "function", - "function": { - "name": "search_files", - "arguments": "{\"query\":\"needle\"}" - } - } - ] - }, - "usage": { - "prompt_tokens": 10, - "completion_tokens": 5, - "cost": { "total": 0.0042 } - } - } - } - })) - .send() - .await - .unwrap(); - assert_eq!(response.status(), StatusCode::OK); - - let response = client - .post(format!("{url}/hooks/hermes")) - .json(&json!({ - "hook_event_name": "pre_api_request", - "session_id": "hermes-atof-lossy", - "extra": { - "task_id": "task-2", - "api_call_count": 4, - "model": "qwen", - "provider": "custom", - "request": null, - "message_count": 2 - } - })) - .send() - .await - .unwrap(); - assert_eq!(response.status(), StatusCode::OK); - - shutdown_tx.send(()).unwrap(); - handle.await.unwrap().unwrap(); - - let events = std::fs::read_to_string(temp.path().join("atof/events.jsonl")).unwrap(); - let llm_events = events - .lines() - .map(|line| serde_json::from_str::(line).unwrap()) - .filter(|event| event["category"] == "llm") - .collect::>(); - assert_eq!( - llm_events.len(), - 4, - "expected Hermes LLM exports, got {llm_events:?}" - ); - - let start = llm_events - .iter() - .find(|event| { - event["scope_category"] == "start" - && event["metadata"]["api_call_id"] == json!("turn-1:api:2") - }) - .unwrap(); - assert_eq!(start["category_profile"]["model_name"], json!("qwen")); - assert_eq!(start["metadata"]["provider_payload_exact"], json!(true)); - assert_eq!( - start["metadata"]["fidelity_source"], - json!("hermes_api_hooks_sanitized") - ); - assert_eq!( - start["data"]["content"]["messages"][0]["content"], - json!("hello") - ); - assert_eq!( - start["data"]["content"]["tools"][0]["function"]["name"], - json!("search_files") - ); - - let end = llm_events - .iter() - .find(|event| { - event["scope_category"] == "end" - && event["metadata"]["api_call_id"] == json!("turn-1:api:2") - }) - .unwrap(); - assert_eq!(end["category_profile"]["model_name"], json!("qwen")); - assert_eq!(end["metadata"]["provider_payload_exact"], json!(true)); - assert_eq!(end["data"]["tool_calls"][0]["id"], json!("call-1")); - assert_eq!( - end["data"]["tool_calls"][0]["function"]["name"], - json!("search_files") - ); - assert_eq!(end["data"]["usage"]["prompt_tokens"], json!(10)); - assert_eq!(end["data"]["usage"]["completion_tokens"], json!(5)); - - let lossy_start = llm_events - .iter() - .find(|event| { - event["scope_category"] == "start" - && event["metadata"]["api_call_id"] == json!("hermes-atof-lossy:task-2:4") - }) - .unwrap(); - assert_eq!(lossy_start["category_profile"]["model_name"], json!("qwen")); - assert_eq!( - lossy_start["metadata"]["provider_payload_exact"], - json!(false) - ); - assert_eq!( - lossy_start["data"]["content"]["fidelity"]["provider_payload_exact"], - json!(false) - ); - assert_eq!(lossy_start["data"]["content"]["message_count"], json!(2)); -} - -#[tokio::test] -async fn serve_listener_hermes_api_request_error_writes_lossy_atof_error_event() { - let _guard = PLUGIN_CONFIG_TEST_LOCK.lock().await; - let _ = nemo_relay::plugin::clear_plugin_configuration(); - - let temp = tempfile::tempdir().unwrap(); - let atof_dir = temp.path().join("atof"); - std::fs::create_dir_all(&atof_dir).unwrap(); - let mut config = test_config(); - config.plugin_config = Some(json!({ - "version": 1, - "components": [ - { - "kind": "observability", - "enabled": true, - "config": { - "version": 3, - "atof": { - "enabled": true, - "sinks": [{ - "type": "file", - "output_directory": atof_dir, - "filename": "events.jsonl", - "mode": "overwrite" - }] - } - } - } - ] - })); - - let listener = TcpListener::bind("127.0.0.1:0").await.unwrap(); - let address = listener.local_addr().unwrap(); - let url = format!("http://{address}"); - let (shutdown_tx, shutdown_rx) = oneshot::channel(); - let handle = - tokio::spawn(async move { serve_listener(listener, config, Some(shutdown_rx)).await }); - - wait_for_gateway(&url).await; - let client = test_http_client(); - - let response = client - .post(format!("{url}/hooks/hermes")) - .json(&json!({ - "hook_event_name": "pre_api_request", - "session_id": "hermes-atof-error", - "extra": { - "task_id": "task-err", - "api_request_id": "turn-1:api:3", - "api_call_count": 3, - "model": "qwen", - "provider": "custom", - "request": { - "method": "POST", - "body": { - "model": "qwen", - "messages": [ - { "role": "user", "content": "hello" } - ] - } - } - } - })) - .send() - .await - .unwrap(); - assert_eq!(response.status(), StatusCode::OK); - - let response = client - .post(format!("{url}/hooks/hermes")) - .json(&json!({ - "hook_event_name": "api_request_error", - "session_id": "hermes-atof-error", - "extra": { - "task_id": "task-err", - "api_request_id": "turn-1:api:3", - "api_call_count": 3, - "model": "qwen", - "provider": "custom", - "status_code": 502, - "retry_count": 1, - "max_retries": 2, - "retryable": true, - "reason": "upstream", - "error": { - "type": "BadGateway", - "message": "gateway upstream error" - } - } - })) - .send() - .await - .unwrap(); - assert_eq!(response.status(), StatusCode::OK); - - shutdown_tx.send(()).unwrap(); - handle.await.unwrap().unwrap(); - - let events = std::fs::read_to_string(temp.path().join("atof/events.jsonl")).unwrap(); - let llm_events = events - .lines() - .map(|line| serde_json::from_str::(line).unwrap()) - .filter(|event| event["category"] == "llm") - .collect::>(); - assert_eq!( - llm_events.len(), - 2, - "expected Hermes error-path LLM exports, got {llm_events:?}" - ); - - let end = llm_events - .iter() - .find(|event| { - event["scope_category"] == "end" - && event["metadata"]["api_call_id"] == json!("turn-1:api:3") - }) - .unwrap(); - let start = llm_events - .iter() - .find(|event| { - event["scope_category"] == "start" - && event["metadata"]["api_call_id"] == json!("turn-1:api:3") - }) - .unwrap(); - assert_eq!(start["metadata"]["provider_payload_exact"], json!(true)); - assert_eq!( - start["metadata"]["fidelity_source"], - json!("hermes_api_hooks_sanitized") - ); - assert_eq!( - start["data"]["content"]["messages"][0]["content"], - json!("hello") - ); - assert_eq!(end["category_profile"]["model_name"], json!("qwen")); - assert_eq!(end["metadata"]["provider_payload_exact"], json!(false)); - assert_eq!( - end["metadata"]["fidelity_source"], - json!("hermes_api_hooks") - ); - assert_eq!(end["data"]["status_code"], json!(502)); - assert_eq!(end["data"]["retry_count"], json!(1)); - assert_eq!(end["data"]["retryable"], json!(true)); - assert_eq!(end["data"]["reason"], json!("upstream")); - assert_eq!( - end["data"]["error"]["message"], - json!("gateway upstream error") - ); +fn atif_matches_session(trajectory: &Value, session_id: &str) -> bool { + trajectory["session_id"] == json!(session_id) + || trajectory["extra"]["observed_events"] + .as_array() + .is_some_and(|events| { + events + .iter() + .any(|event| event_has_session_id(event, session_id)) + }) } -#[tokio::test] -async fn serve_listener_hermes_post_tool_call_writes_atof_tool_events() { - let _guard = PLUGIN_CONFIG_TEST_LOCK.lock().await; - let _ = nemo_relay::plugin::clear_plugin_configuration(); - - let temp = tempfile::tempdir().unwrap(); - let atof_dir = temp.path().join("atof"); - std::fs::create_dir_all(&atof_dir).unwrap(); - let mut config = test_config(); - config.plugin_config = Some(json!({ - "version": 1, - "components": [ - { - "kind": "observability", - "enabled": true, - "config": { - "version": 3, - "atof": { - "enabled": true, - "sinks": [{ - "type": "file", - "output_directory": atof_dir, - "filename": "events.jsonl", - "mode": "overwrite" - }] - } - } - } - ] - })); - - let listener = TcpListener::bind("127.0.0.1:0").await.unwrap(); - let address = listener.local_addr().unwrap(); - let url = format!("http://{address}"); - let (shutdown_tx, shutdown_rx) = oneshot::channel(); - let handle = - tokio::spawn(async move { serve_listener(listener, config, Some(shutdown_rx)).await }); - - wait_for_gateway(&url).await; - let client = test_http_client(); - - for payload in [ - json!({ - "hook_event_name": "on_session_start", - "session_id": "hermes-tool-atof" - }), - json!({ - "hook_event_name": "pre_tool_call", - "session_id": "hermes-tool-atof", - "tool_name": "search_files", - "tool_input": { "query": "needle" }, - "extra": { - "task_id": "task-1", - "tool_call_id": "call-search-1" - } - }), - json!({ - "hook_event_name": "post_tool_call", - "session_id": "hermes-tool-atof", - "tool_name": "search_files", - "tool_input": { "query": "needle" }, - "tool_response": { "total_count": 6 }, - "extra": { - "task_id": "task-1", - "tool_call_id": "call-search-1" - } - }), - json!({ - "hook_event_name": "on_session_finalize", - "session_id": "hermes-tool-atof" - }), - ] { - let response = client - .post(format!("{url}/hooks/hermes")) - .json(&payload) - .send() - .await - .unwrap(); - assert_eq!(response.status(), StatusCode::OK); - } - - shutdown_tx.send(()).unwrap(); - handle.await.unwrap().unwrap(); - - let events = std::fs::read_to_string(temp.path().join("atof/events.jsonl")).unwrap(); - let tool_events = events - .lines() - .map(|line| serde_json::from_str::(line).unwrap()) - .filter(|event| event["category"] == "tool") - .collect::>(); - assert_eq!( - tool_events.len(), - 2, - "expected Hermes tool start/end exports, got {tool_events:?}" - ); - - let start = tool_events - .iter() - .find(|event| event["scope_category"] == "start") - .unwrap(); - assert_eq!(start["name"], json!("search_files")); - assert_eq!( - start["category_profile"]["tool_call_id"], - json!("call-search-1") - ); - assert_eq!(start["data"]["query"], json!("needle")); - - let end = tool_events - .iter() - .find(|event| event["scope_category"] == "end") - .unwrap(); - assert_eq!(end["name"], json!("search_files")); - assert_eq!( - end["category_profile"]["tool_call_id"], - json!("call-search-1") - ); - assert_eq!(end["data"]["total_count"], json!(6)); +fn event_has_session_id(event: &Value, session_id: &str) -> bool { + event["metadata"]["session_id"] == json!(session_id) + || event["data"]["session_id"] == json!(session_id) + || event["data"]["extra"]["session_id"] == json!(session_id) } - #[tokio::test] async fn serve_listener_routed_gateway_wire_formats_write_atof_category_profile_and_usage() { let _guard = PLUGIN_CONFIG_TEST_LOCK.lock().await; @@ -1783,7 +1278,7 @@ async fn serve_listener_routed_gateway_wire_formats_write_atof_category_profile_ .post(format!("{url}/v1/messages")) .header("content-type", "application/json") .header("x-api-key", "sk-ant-test") - .header("x-nemo-relay-session-id", "hermes-routed-atof") + .header("x-nemo-relay-session-id", "gateway-routed-atof") .json(&json!({ "model": "claude-sonnet-4", "messages": [{"role": "user", "content": "Find the file."}], @@ -1798,7 +1293,7 @@ async fn serve_listener_routed_gateway_wire_formats_write_atof_category_profile_ .post(format!("{url}/v1/responses")) .header("content-type", "application/json") .header("authorization", "Bearer test") - .header("x-nemo-relay-session-id", "hermes-routed-atof") + .header("x-nemo-relay-session-id", "gateway-routed-atof") .json(&json!({ "model": "gpt-4o", "input": "Find the weather.", @@ -1813,7 +1308,7 @@ async fn serve_listener_routed_gateway_wire_formats_write_atof_category_profile_ .post(format!("{url}/v1/chat/completions")) .header("content-type", "application/json") .header("authorization", "Bearer test") - .header("x-nemo-relay-session-id", "hermes-routed-atof") + .header("x-nemo-relay-session-id", "gateway-routed-atof") .json(&json!({ "model": "gpt-4o", "messages": [{"role": "user", "content": "Inspect the files."}], @@ -2590,33 +2085,6 @@ async fn pre_tool_hook_rejects_when_conditional_guardrail_blocks() { ); assert_eq!(body["error"]["reason"], json!("blocked by policy")); } - -#[tokio::test] -async fn hermes_hook_keeps_shell_hook_response_shape() { - let app = router(test_config()); - let response = app - .oneshot( - Request::builder() - .method("POST") - .uri("/hooks/hermes") - .header("content-type", "application/json") - .body(Body::from( - json!({ - "session_id": "hermes-1", - "hook_event_name": "on_session_start" - }) - .to_string(), - )) - .unwrap(), - ) - .await - .unwrap(); - assert_eq!(response.status(), StatusCode::OK); - let bytes = response.into_body().collect().await.unwrap().to_bytes(); - let body: Value = serde_json::from_slice(&bytes).unwrap(); - assert_eq!(body, json!({})); -} - #[tokio::test] async fn gateway_forwards_openai_json_without_rewriting_payload() { let upstream = spawn_upstream(false).await; diff --git a/crates/cli/tests/coverage/shared/session_tests.rs b/crates/cli/tests/coverage/shared/session_tests.rs index 4223e2da8..92de89e24 100644 --- a/crates/cli/tests/coverage/shared/session_tests.rs +++ b/crates/cli/tests/coverage/shared/session_tests.rs @@ -17,11 +17,9 @@ use std::path::Path; use std::sync::{Arc, Mutex as StdMutex}; use super::*; -use crate::events::{LlmEvent, LlmHintEvent, SessionEvent, ToolEvent}; +use crate::events::{LlmHintEvent, SessionEvent, ToolEvent}; use crate::test_support::PLUGIN_CONFIG_TEST_LOCK; -const HERMES_ROUTED_TEST_SESSION_KEY: &str = "hermes_routed_test_session_id"; - #[test] fn routing_identity_enrichment_replaces_untrusted_reserved_headers() { let mut request = LlmRequest { @@ -51,7 +49,7 @@ fn routing_identity_enrichment_replaces_untrusted_reserved_headers() { &mut request, RoutingIdentityHeaderContext { session_id: "trusted-session", - agent_kind: AgentKind::Hermes, + agent_kind: AgentKind::Gateway, turn_index: 7, request_id: Some("trusted-request"), owner_id: None, @@ -69,7 +67,7 @@ fn routing_identity_enrichment_replaces_untrusted_reserved_headers() { request.headers["x-nemo-relay-request-id"], json!("trusted-request") ); - assert_eq!(request.headers["x-nemo-relay-agent-kind"], json!("hermes")); + assert_eq!(request.headers["x-nemo-relay-agent-kind"], json!("gateway")); assert_eq!(request.headers["x-nemo-relay-turn-id"], json!("7")); assert_eq!( request.headers["x-nemo-relay-identity-quality"], @@ -291,7 +289,7 @@ async fn session_start_mark_carries_stable_non_overridable_identity_once() { let headers = HeaderMap::new(); let start = SessionEvent { session_id: session_id.into(), - agent_kind: AgentKind::Hermes, + agent_kind: AgentKind::Gateway, event_name: "on_session_start".into(), payload: json!({"ignored": true}), metadata: json!({}), @@ -304,7 +302,7 @@ async fn session_start_mark_carries_stable_non_overridable_identity_once() { NormalizedEvent::AgentStarted(start), NormalizedEvent::AgentEnded(SessionEvent { session_id: session_id.into(), - agent_kind: AgentKind::Hermes, + agent_kind: AgentKind::Gateway, event_name: "on_session_end".into(), payload: json!({}), metadata: json!({}), @@ -549,14 +547,6 @@ async fn stop_codex_turn(manager: &SessionManager, headers: &HeaderMap, session_ .await; } -fn hermes_routed_gateway_metadata(gateway_path: &str, test_session_marker: Option<&str>) -> Value { - let mut metadata = json!({ "gateway_path": gateway_path }); - if let Some(marker) = test_session_marker { - metadata[HERMES_ROUTED_TEST_SESSION_KEY] = json!(marker); - } - metadata -} - fn read_atif_for_session(output_directory: &Path, session_id: &str) -> Value { flush_subscribers().unwrap(); std::fs::read_dir(output_directory) @@ -613,304 +603,6 @@ async fn has_pending_alignment(manager: &SessionManager, session_id: &str) -> bo .has_pending_session(session_id) } -async fn drive_hermes_routed_provider_session( - manager: &SessionManager, - headers: &HeaderMap, - session_id: &str, - test_session_marker: Option<&str>, -) { - manager - .apply_events( - headers, - vec![NormalizedEvent::AgentStarted(SessionEvent { - session_id: session_id.into(), - agent_kind: AgentKind::Hermes, - event_name: "on_session_start".into(), - payload: json!({}), - metadata: json!({}), - })], - ) - .await - .unwrap(); - - let anthropic = manager - .start_llm( - headers, - LlmGatewayStart { - session_id: Some(session_id.into()), - provider: "anthropic.messages".into(), - model_name: Some("claude-sonnet-4".into()), - subagent_id: None, - conversation_id: None, - generation_id: None, - request_id: Some("msg-request".into()), - request: LlmRequest { - headers: Map::new(), - content: json!({ - "model": "claude-sonnet-4", - "messages": [{"role": "user", "content": "Find the file."}], - "tools": [{"name": "search", "input_schema": {"type": "object"}}] - }), - }, - streaming: false, - metadata: hermes_routed_gateway_metadata("/v1/messages", test_session_marker), - }, - ) - .await - .unwrap(); - manager - .end_llm( - anthropic, - json!({ - "id": "msg_01", - "type": "message", - "content": [ - {"type": "text", "text": "I will search."}, - {"type": "tool_use", "id": "toolu_01", "name": "search", "input": {"query": "file"}} - ], - "usage": { - "input_tokens": 11, - "output_tokens": 7, - "cache_read_input_tokens": 3, - "cost": {"total": 0.0042} - } - }), - json!({}), - ) - .await - .unwrap(); - - let responses = manager - .start_llm( - headers, - LlmGatewayStart { - session_id: Some(session_id.into()), - provider: "openai.responses".into(), - model_name: Some("gpt-4o".into()), - subagent_id: None, - conversation_id: None, - generation_id: None, - request_id: Some("resp-request".into()), - request: LlmRequest { - headers: Map::new(), - content: json!({ - "model": "gpt-4o", - "input": "Find the weather.", - "tools": [{"type": "function", "name": "get_weather"}] - }), - }, - streaming: false, - metadata: hermes_routed_gateway_metadata("/v1/responses", test_session_marker), - }, - ) - .await - .unwrap(); - manager - .end_llm( - responses, - json!({ - "id": "resp_1", - "output": [ - {"type": "message", "content": [{"type": "output_text", "text": "I will check the weather."}]}, - {"type": "function_call", "call_id": "call_weather_1", "name": "get_weather", "arguments": "{\"city\":\"SF\"}"} - ], - "usage": { - "input_tokens": 75, - "output_tokens": 20, - "total_tokens": 95, - "input_tokens_details": {"cached_tokens": 10}, - "cost_usd": 0.005 - } - }), - json!({}), - ) - .await - .unwrap(); - - let chat = manager - .start_llm( - headers, - LlmGatewayStart { - session_id: Some(session_id.into()), - provider: "openai.chat_completions".into(), - model_name: Some("gpt-4o".into()), - subagent_id: None, - conversation_id: None, - generation_id: None, - request_id: Some("chat-request".into()), - request: LlmRequest { - headers: Map::new(), - content: json!({ - "model": "gpt-4o", - "messages": [{"role": "user", "content": "Inspect the files."}], - "tools": [{"type": "function", "function": {"name": "read"}}] - }), - }, - streaming: false, - metadata: hermes_routed_gateway_metadata( - "/v1/chat/completions", - test_session_marker, - ), - }, - ) - .await - .unwrap(); - manager - .end_llm( - chat, - json!({ - "choices": [{ - "message": { - "role": "assistant", - "content": "I will inspect.", - "tool_calls": [{"id": "call_read_1", "function": {"name": "read", "arguments": "{\"path\":\"api.py\"}"}}] - } - }], - "usage": { - "prompt_tokens": 3, - "completion_tokens": 4, - "total_tokens": 7, - "prompt_tokens_details": {"cached_tokens": 2}, - "cost_usd": 0.001 - } - }), - json!({}), - ) - .await - .unwrap(); - - manager - .apply_events( - headers, - vec![NormalizedEvent::AgentEnded(SessionEvent { - session_id: session_id.into(), - agent_kind: AgentKind::Hermes, - event_name: "on_session_finalize".into(), - payload: json!({}), - metadata: json!({}), - })], - ) - .await - .unwrap(); -} - -async fn drive_hermes_orphan_subagent_stop( - manager: &SessionManager, - headers: &HeaderMap, - session_id: &str, - subagent_id: &str, -) { - for payload in [ - json!({ - "hook_event_name": "on_session_start", - "session_id": session_id - }), - json!({ - "hook_event_name": "subagent_stop", - "session_id": session_id, - "extra": { - "subagent_id": subagent_id, - "child_status": "completed" - } - }), - json!({ - "hook_event_name": "on_session_finalize", - "session_id": session_id - }), - ] { - let outcome = crate::agents::shared::adapters::hermes::adapt(payload, headers); - manager.apply_events(headers, outcome.events).await.unwrap(); - } -} - -async fn drive_hermes_subagent_child_session( - manager: &SessionManager, - headers: &HeaderMap, - parent_session_id: &str, - child_session_id: &str, - child_subagent_id: &str, -) { - for payload in [ - json!({ - "hook_event_name": "on_session_start", - "session_id": parent_session_id - }), - json!({ - "hook_event_name": "subagent_start", - "session_id": parent_session_id, - "extra": { - "child_goal": "read plugin yaml", - "child_role": "leaf", - "child_session_id": child_session_id, - "child_subagent_id": child_subagent_id, - "parent_turn_id": "parent-turn-1", - "telemetry_schema_version": "hermes.observer.v1" - } - }), - json!({ - "hook_event_name": "on_session_start", - "session_id": child_session_id - }), - json!({ - "hook_event_name": "pre_api_request", - "session_id": child_session_id, - "extra": { - "task_id": "child-task", - "api_call_count": 1, - "provider": "custom", - "model": "qwen", - "request": { - "body": { - "model": "qwen", - "messages": [ - { "role": "user", "content": "read plugin yaml" } - ] - } - } - } - }), - json!({ - "hook_event_name": "post_api_request", - "session_id": child_session_id, - "extra": { - "task_id": "child-task", - "api_call_count": 1, - "provider": "custom", - "model": "qwen", - "response": { - "assistant_message": { - "role": "assistant", - "content": "name: nemo_flow" - }, - "usage": { - "prompt_tokens": 3, - "completion_tokens": 2 - } - } - } - }), - json!({ - "hook_event_name": "on_session_end", - "session_id": child_session_id - }), - json!({ - "hook_event_name": "subagent_stop", - "session_id": parent_session_id, - "extra": { - "child_session_id": child_session_id, - "child_status": "completed" - } - }), - json!({ - "hook_event_name": "on_session_finalize", - "session_id": parent_session_id - }), - ] { - let outcome = crate::agents::shared::adapters::hermes::adapt(payload, headers); - manager.apply_events(headers, outcome.events).await.unwrap(); - } -} - #[tokio::test] async fn nests_agent_subagent_and_tool_lifecycle() { let config = GatewayConfig { @@ -1786,74 +1478,8 @@ async fn codex_subagent_start_does_not_reparent_active_child_session() { .unwrap(); manager.close_all("test_shutdown").await.unwrap(); } - -#[tokio::test] -async fn hermes_subagent_start_does_not_reparent_active_child_session() { - let manager = SessionManager::new(session_test_config()); - let headers = HeaderMap::new(); - - for payload in [ - json!({ - "hook_event_name": "on_session_start", - "session_id": "parent-session" - }), - json!({ - "hook_event_name": "on_session_start", - "session_id": "child-session" - }), - json!({ - "hook_event_name": "pre_api_request", - "session_id": "child-session", - "extra": { - "task_id": "child-task", - "api_call_count": 1, - "provider": "custom", - "model": "qwen", - "request": { "body": { "model": "qwen" } } - } - }), - json!({ - "hook_event_name": "subagent_start", - "session_id": "parent-session", - "extra": { - "child_session_id": "child-session", - "child_subagent_id": "sa-1", - "parent_turn_id": "parent-turn-1" - } - }), - ] { - let outcome = crate::agents::shared::adapters::hermes::adapt(payload, &headers); - manager - .apply_events(&headers, outcome.events) - .await - .unwrap(); - } - - assert!(!has_alignment_alias(&manager, "child-session").await); - { - let sessions = manager.inner.lock().await; - assert!(sessions.contains_key("child-session")); - assert!( - !sessions - .get("parent-session") - .unwrap() - .subagents - .contains_key("sa-1") - ); - assert!( - sessions - .get("child-session") - .unwrap() - .llms - .contains_key("child-session:child-task:1") - ); - } - - manager.close_all("test_shutdown").await.unwrap(); -} - #[tokio::test] -async fn codex_aliased_hook_llm_routes_to_subagent_scope() { +async fn codex_subagent_gateway_llm_routes_to_parent_subagent() { let manager = SessionManager::new(session_test_config()); manager .apply_events( @@ -1872,112 +1498,51 @@ async fn codex_aliased_hook_llm_routes_to_subagent_scope() { "source": { "subagent": { "thread_spawn": { - "parent_thread_id": "parent-thread" + "parent_thread_id": "parent-thread", + "agent_nickname": "Bohr", + "agent_role": "explorer" } } } }), metadata: json!({}), }), - NormalizedEvent::LlmStarted(LlmEvent { - session_id: "child-thread".into(), - agent_kind: AgentKind::Codex, - event_name: "PreLlm".into(), - api_call_id: "hook-llm".into(), - provider: "openai.responses".into(), - model_name: Some("gpt-test".into()), - request: json!({ "input": "hello" }), - response: Value::Null, - metadata: json!({}), - }), ], ) .await .unwrap(); - let sessions = manager.inner.lock().await; - let parent = sessions.get("parent-thread").unwrap(); - let subagent_uuid = parent.subagents.get("child-thread").unwrap().uuid; - let handle = parent.llms.get("hook-llm").unwrap(); - assert_eq!(handle.parent_uuid, Some(subagent_uuid)); + let subagent_uuid = { + let sessions = manager.inner.lock().await; + sessions + .get("parent-thread") + .unwrap() + .subagents + .get("child-thread") + .unwrap() + .uuid + }; + + let active = manager + .start_llm( + &HeaderMap::new(), + LlmGatewayStart { + session_id: Some("child-thread".into()), + ..llm_start() + }, + ) + .await + .unwrap(); + + assert_eq!(active.session_id, "parent-thread"); + assert_eq!(active.owner_subagent_id.as_deref(), Some("child-thread")); + assert_eq!(active.handle.parent_uuid, Some(subagent_uuid)); assert_eq!( - handle.metadata.as_ref().unwrap()["llm_correlation_status"], - json!("session_alias") + active.handle.metadata.as_ref().unwrap()["llm_correlation_status"], + json!("explicit") ); assert_eq!( - handle.metadata.as_ref().unwrap()["llm_correlation_subagent_id"], - json!("child-thread") - ); - drop(sessions); - - manager.close_all("test_shutdown").await.unwrap(); -} - -#[tokio::test] -async fn codex_subagent_gateway_llm_routes_to_parent_subagent() { - let manager = SessionManager::new(session_test_config()); - manager - .apply_events( - &HeaderMap::new(), - vec![ - NormalizedEvent::AgentStarted(codex_session_event( - "parent-thread", - "SessionStart", - json!({}), - )), - NormalizedEvent::AgentStarted(SessionEvent { - session_id: "child-thread".into(), - agent_kind: AgentKind::Codex, - event_name: "SessionStart".into(), - payload: json!({ - "source": { - "subagent": { - "thread_spawn": { - "parent_thread_id": "parent-thread", - "agent_nickname": "Bohr", - "agent_role": "explorer" - } - } - } - }), - metadata: json!({}), - }), - ], - ) - .await - .unwrap(); - - let subagent_uuid = { - let sessions = manager.inner.lock().await; - sessions - .get("parent-thread") - .unwrap() - .subagents - .get("child-thread") - .unwrap() - .uuid - }; - - let active = manager - .start_llm( - &HeaderMap::new(), - LlmGatewayStart { - session_id: Some("child-thread".into()), - ..llm_start() - }, - ) - .await - .unwrap(); - - assert_eq!(active.session_id, "parent-thread"); - assert_eq!(active.owner_subagent_id.as_deref(), Some("child-thread")); - assert_eq!(active.handle.parent_uuid, Some(subagent_uuid)); - assert_eq!( - active.handle.metadata.as_ref().unwrap()["llm_correlation_status"], - json!("explicit") - ); - assert_eq!( - active.handle.metadata.as_ref().unwrap()["llm_correlation_subagent_id"], + active.handle.metadata.as_ref().unwrap()["llm_correlation_subagent_id"], json!("child-thread") ); assert_eq!( @@ -2430,7 +1995,7 @@ async fn codex_openinference_spans_match_shared_contract() { #[tokio::test] async fn duplicate_agent_end_does_not_overwrite_atif_with_empty_session() { - // Regression test: hermes-agent and other integrations can emit terminal hooks more than once + // Regression test: integrations can emit terminal hooks more than once // per session. Without idempotency in `end_agent`, the second AgentEnded would re-open an // empty agent scope via `ensure_agent_started`, close it, and write an empty ATIF on top of // the just-written real trajectory. @@ -2513,1290 +2078,6 @@ async fn duplicate_agent_end_does_not_overwrite_atif_with_empty_session() { ); } -#[tokio::test] -async fn writes_hermes_api_hook_usage_to_atif_metrics() { - let _guard = PLUGIN_CONFIG_TEST_LOCK.lock().await; - let temp = tempfile::tempdir().unwrap(); - let atif_dir = temp.path().join("atif"); - install_test_atif_plugin(&atif_dir).await; - let config = GatewayConfig { - bind: "127.0.0.1:0".parse().unwrap(), - openai_base_url: "http://127.0.0.1".into(), - openai_auth_header: None, - anthropic_base_url: "http://127.0.0.1".into(), - anthropic_auth_header: None, - metadata: None, - plugin_config: None, - max_hook_payload_bytes: crate::configuration::DEFAULT_MAX_HOOK_PAYLOAD_BYTES, - max_passthrough_body_bytes: crate::configuration::DEFAULT_MAX_PASSTHROUGH_BODY_BYTES, - }; - let manager = SessionManager::new(config); - let headers = HeaderMap::new(); - - manager - .apply_events( - &headers, - vec![ - NormalizedEvent::AgentStarted(SessionEvent { - session_id: "hermes-usage".into(), - agent_kind: AgentKind::Hermes, - event_name: "on_session_start".into(), - payload: json!({}), - metadata: json!({}), - }), - NormalizedEvent::LlmStarted(LlmEvent { - session_id: "hermes-usage".into(), - agent_kind: AgentKind::Hermes, - event_name: "pre_api_request".into(), - api_call_id: "hermes-usage:task-1:1".into(), - provider: "custom".into(), - model_name: Some("qwen".into()), - request: json!({ "model": "qwen" }), - response: Value::Null, - metadata: json!({}), - }), - NormalizedEvent::LlmEnded(LlmEvent { - session_id: "hermes-usage".into(), - agent_kind: AgentKind::Hermes, - event_name: "post_api_request".into(), - api_call_id: "hermes-usage:task-1:1".into(), - provider: "custom".into(), - model_name: Some("qwen".into()), - request: json!({}), - response: json!({ - "usage": { - "prompt_tokens": 10, - "completion_tokens": 5, - "prompt_tokens_details": { "cached_tokens": 3 } - } - }), - metadata: json!({}), - }), - NormalizedEvent::AgentEnded(SessionEvent { - session_id: "hermes-usage".into(), - agent_kind: AgentKind::Hermes, - event_name: "on_session_finalize".into(), - payload: json!({}), - metadata: json!({}), - }), - ], - ) - .await - .unwrap(); - - clear_plugin_configuration().unwrap(); - let atif = read_atif_for_session(&atif_dir, "hermes-usage"); - assert!(atif["subagent_trajectories"].is_null()); - assert_eq!(atif["steps"][1]["metrics"]["prompt_tokens"], json!(10)); - assert_eq!(atif["steps"][1]["metrics"]["completion_tokens"], json!(5)); - assert_eq!(atif["steps"][1]["metrics"]["cached_tokens"], json!(3)); - assert!(atif["steps"][1]["metrics"].get("cost_usd").is_none()); - assert_eq!(atif["final_metrics"]["total_prompt_tokens"], json!(10)); - assert_eq!(atif["final_metrics"]["total_completion_tokens"], json!(5)); - assert_eq!(atif["final_metrics"]["total_cached_tokens"], json!(3)); - assert!(atif["final_metrics"].get("total_cost_usd").is_none()); -} - -#[tokio::test] -async fn writes_hermes_api_hook_reported_cost_to_atif_metrics() { - let _guard = PLUGIN_CONFIG_TEST_LOCK.lock().await; - let temp = tempfile::tempdir().unwrap(); - let atif_dir = temp.path().join("atif"); - install_test_atif_plugin(&atif_dir).await; - let config = GatewayConfig { - bind: "127.0.0.1:0".parse().unwrap(), - openai_base_url: "http://127.0.0.1".into(), - openai_auth_header: None, - anthropic_base_url: "http://127.0.0.1".into(), - anthropic_auth_header: None, - metadata: None, - plugin_config: None, - max_hook_payload_bytes: crate::configuration::DEFAULT_MAX_HOOK_PAYLOAD_BYTES, - max_passthrough_body_bytes: crate::configuration::DEFAULT_MAX_PASSTHROUGH_BODY_BYTES, - }; - let manager = SessionManager::new(config); - let headers = HeaderMap::new(); - - manager - .apply_events( - &headers, - vec![ - NormalizedEvent::AgentStarted(SessionEvent { - session_id: "hermes-cost".into(), - agent_kind: AgentKind::Hermes, - event_name: "on_session_start".into(), - payload: json!({}), - metadata: json!({}), - }), - NormalizedEvent::LlmStarted(LlmEvent { - session_id: "hermes-cost".into(), - agent_kind: AgentKind::Hermes, - event_name: "pre_api_request".into(), - api_call_id: "hermes-cost:task-1:1".into(), - provider: "custom".into(), - model_name: Some("qwen".into()), - request: json!({ "model": "qwen" }), - response: Value::Null, - metadata: json!({}), - }), - NormalizedEvent::LlmEnded(LlmEvent { - session_id: "hermes-cost".into(), - agent_kind: AgentKind::Hermes, - event_name: "post_api_request".into(), - api_call_id: "hermes-cost:task-1:1".into(), - provider: "custom".into(), - model_name: Some("qwen".into()), - request: json!({}), - response: json!({ - "usage": { - "prompt_tokens": 10, - "completion_tokens": 5, - "cost_usd": 0.123 - } - }), - metadata: json!({}), - }), - NormalizedEvent::AgentEnded(SessionEvent { - session_id: "hermes-cost".into(), - agent_kind: AgentKind::Hermes, - event_name: "on_session_finalize".into(), - payload: json!({}), - metadata: json!({}), - }), - ], - ) - .await - .unwrap(); - - clear_plugin_configuration().unwrap(); - let atif = read_atif_for_session(&atif_dir, "hermes-cost"); - assert_eq!(atif["steps"][1]["metrics"]["cost_usd"], json!(0.123)); - assert_eq!(atif["final_metrics"]["total_cost_usd"], json!(0.123)); -} - -#[tokio::test] -async fn hermes_exact_api_hooks_write_atif_request_response_and_cost() { - let _guard = PLUGIN_CONFIG_TEST_LOCK.lock().await; - let temp = tempfile::tempdir().unwrap(); - let atif_dir = temp.path().join("atif"); - install_test_atif_plugin(&atif_dir).await; - let config = session_test_config(); - let manager = SessionManager::new(config); - let headers = HeaderMap::new(); - - for payload in [ - json!({ - "hook_event_name": "on_session_start", - "session_id": "hermes-exact-atif" - }), - json!({ - "hook_event_name": "pre_api_request", - "session_id": "hermes-exact-atif", - "extra": { - "task_id": "task-1", - "api_call_count": 1, - "provider": "custom", - "model": "qwen", - "request": { - "body": { - "model": "qwen", - "temperature": 0.1, - "messages": [ - { "role": "user", "content": "summarize this file" } - ], - "tools": [ - { - "type": "function", - "function": { "name": "read_file" } - } - ] - } - } - } - }), - json!({ - "hook_event_name": "post_api_request", - "session_id": "hermes-exact-atif", - "extra": { - "task_id": "task-1", - "api_call_count": 1, - "provider": "custom", - "model": "qwen", - "response": { - "assistant_message": { - "role": "assistant", - "content": "summary ready" - }, - "usage": { - "prompt_tokens": 11, - "completion_tokens": 7, - "cost": { "total": 0.0042 } - }, - "finish_reason": "stop" - } - } - }), - json!({ - "hook_event_name": "on_session_finalize", - "session_id": "hermes-exact-atif" - }), - ] { - let outcome = crate::agents::shared::adapters::hermes::adapt(payload, &headers); - manager - .apply_events(&headers, outcome.events) - .await - .unwrap(); - } - - clear_plugin_configuration().unwrap(); - let atif = read_atif_for_session(&atif_dir, "hermes-exact-atif"); - let observed_events = atif["extra"]["observed_events"].as_array().unwrap(); - assert_eq!(atif["steps"][0]["message"], json!("summarize this file")); - assert_eq!( - atif["steps"][0]["extra"]["llm_request"]["temperature"], - json!(0.1) - ); - assert_eq!( - atif["steps"][0]["extra"]["llm_request"]["tools"][0]["function"]["name"], - json!("read_file") - ); - assert_eq!(atif["steps"][1]["message"], json!("summary ready")); - assert_eq!( - atif["steps"][1]["extra"]["llm_response"]["content"], - json!("summary ready") - ); - assert_eq!( - atif["steps"][1]["extra"]["llm_response"]["usage"]["cost"]["total"], - json!(0.0042) - ); - assert_eq!(atif["steps"][1]["metrics"]["prompt_tokens"], json!(11)); - assert_eq!(atif["steps"][1]["metrics"]["completion_tokens"], json!(7)); - assert_eq!(atif["steps"][1]["metrics"]["cost_usd"], json!(0.0042)); - assert_eq!(atif["final_metrics"]["total_cost_usd"], json!(0.0042)); - assert!(observed_events.iter().any(|event| { - event["metadata"]["hook_event_name"] == json!("pre_api_request") - && event["metadata"]["provider_payload_exact"] == json!(true) - && event["metadata"]["fidelity_source"] == json!("hermes_api_hooks_sanitized") - })); - assert!(observed_events.iter().any(|event| { - event["metadata"]["hook_event_name"] == json!("post_api_request") - && event["metadata"]["provider_payload_exact"] == json!(true) - && event["metadata"]["fidelity_source"] == json!("hermes_api_hooks_sanitized") - })); -} - -#[tokio::test] -async fn hermes_api_request_error_writes_atif_error_step_and_fidelity() { - let _guard = PLUGIN_CONFIG_TEST_LOCK.lock().await; - let temp = tempfile::tempdir().unwrap(); - let atif_dir = temp.path().join("atif"); - install_test_atif_plugin(&atif_dir).await; - let config = session_test_config(); - let manager = SessionManager::new(config); - let headers = HeaderMap::new(); - - for payload in [ - json!({ - "hook_event_name": "on_session_start", - "session_id": "hermes-error" - }), - json!({ - "hook_event_name": "pre_api_request", - "session_id": "hermes-error", - "extra": { - "task_id": "task-err", - "api_request_id": "turn-1:api:3", - "api_call_count": 3, - "provider": "custom", - "model": "qwen", - "request": { - "method": "POST", - "body": { - "model": "qwen", - "messages": [ - { "role": "user", "content": "hello" } - ] - } - } - } - }), - json!({ - "hook_event_name": "api_request_error", - "session_id": "hermes-error", - "extra": { - "task_id": "task-err", - "api_request_id": "turn-1:api:3", - "api_call_count": 3, - "provider": "custom", - "model": "qwen", - "status_code": 502, - "retry_count": 1, - "max_retries": 2, - "retryable": true, - "reason": "upstream", - "error": { - "type": "BadGateway", - "message": "gateway upstream error" - } - } - }), - json!({ - "hook_event_name": "on_session_finalize", - "session_id": "hermes-error" - }), - ] { - let outcome = crate::agents::shared::adapters::hermes::adapt(payload, &headers); - manager - .apply_events(&headers, outcome.events) - .await - .unwrap(); - } - - clear_plugin_configuration().unwrap(); - let atif = read_atif_for_session(&atif_dir, "hermes-error"); - let steps = atif["steps"].as_array().unwrap(); - assert_eq!(steps.len(), 2); - assert_eq!(steps[0]["message"], json!("hello")); - assert_eq!(steps[1]["source"], json!("agent")); - assert_eq!(steps[1]["extra"]["llm_response"]["status_code"], json!(502)); - assert_eq!(steps[1]["extra"]["llm_response"]["retry_count"], json!(1)); - assert_eq!(steps[1]["extra"]["llm_response"]["retryable"], json!(true)); - assert_eq!( - steps[1]["extra"]["llm_response"]["reason"], - json!("upstream") - ); - assert_eq!( - steps[1]["extra"]["llm_response"]["error"]["message"], - json!("gateway upstream error") - ); - let observed_events = atif["extra"]["observed_events"].as_array().unwrap(); - assert!( - observed_events.len() >= 4, - "expected Hermes error trajectory to keep observed events, got {}", - serde_json::to_string_pretty(&atif["extra"]["observed_events"]).unwrap() - ); - let error_event = observed_events - .iter() - .find(|event| { - event["scope_category"] == json!("end") - && event["metadata"]["api_call_id"] == json!("turn-1:api:3") - }) - .unwrap(); - let request_event = observed_events - .iter() - .find(|event| { - event["scope_category"] == json!("start") - && event["metadata"]["api_call_id"] == json!("turn-1:api:3") - }) - .unwrap(); - assert_eq!( - request_event["metadata"]["provider_payload_exact"], - json!(true) - ); - assert_eq!( - request_event["metadata"]["fidelity_source"], - json!("hermes_api_hooks_sanitized") - ); - assert_eq!( - error_event["metadata"]["provider_payload_exact"], - json!(false) - ); - assert_eq!( - error_event["metadata"]["fidelity_source"], - json!("hermes_api_hooks") - ); -} - -#[tokio::test] -async fn hermes_lossy_api_hooks_write_atif_fidelity_markers() { - let _guard = PLUGIN_CONFIG_TEST_LOCK.lock().await; - let temp = tempfile::tempdir().unwrap(); - let atif_dir = temp.path().join("atif"); - install_test_atif_plugin(&atif_dir).await; - let config = session_test_config(); - let manager = SessionManager::new(config); - let headers = HeaderMap::new(); - - for payload in [ - json!({ - "hook_event_name": "on_session_start", - "session_id": "hermes-lossy-atif" - }), - json!({ - "hook_event_name": "pre_api_request", - "session_id": "hermes-lossy-atif", - "extra": { - "task_id": "task-1", - "api_call_count": 1, - "provider": "custom", - "model": "qwen", - "message_count": 1, - "tool_count": 0, - "request_char_count": 42 - } - }), - json!({ - "hook_event_name": "post_api_request", - "session_id": "hermes-lossy-atif", - "extra": { - "task_id": "task-1", - "api_call_count": 1, - "provider": "custom", - "model": "qwen", - "assistant_content_chars": 13, - "finish_reason": "stop", - "usage": { - "prompt_tokens": 5, - "completion_tokens": 3 - } - } - }), - json!({ - "hook_event_name": "on_session_finalize", - "session_id": "hermes-lossy-atif" - }), - ] { - let outcome = crate::agents::shared::adapters::hermes::adapt(payload, &headers); - manager - .apply_events(&headers, outcome.events) - .await - .unwrap(); - } - - clear_plugin_configuration().unwrap(); - let atif = read_atif_for_session(&atif_dir, "hermes-lossy-atif"); - let observed_events = atif["extra"]["observed_events"].as_array().unwrap(); - assert_eq!( - atif["steps"][0]["extra"]["llm_request"]["fidelity"]["provider_payload_exact"], - json!(false) - ); - assert_eq!( - atif["steps"][0]["extra"]["llm_request"]["fidelity"]["source"], - json!("hermes_pre_api_request") - ); - assert_eq!( - atif["steps"][0]["extra"]["llm_request"]["request_char_count"], - json!(42) - ); - assert_eq!( - atif["steps"][1]["extra"]["llm_response"]["assistant_content_chars"], - json!(13) - ); - assert!(atif["steps"][1]["extra"]["llm_response"]["content"].is_null()); - assert_eq!(atif["steps"][1]["metrics"]["prompt_tokens"], json!(5)); - assert_eq!(atif["steps"][1]["metrics"]["completion_tokens"], json!(3)); - assert!(observed_events.iter().any(|event| { - event["metadata"]["hook_event_name"] == json!("pre_api_request") - && event["metadata"]["provider_payload_exact"] == json!(false) - && event["metadata"]["fidelity_source"] == json!("hermes_api_hooks") - })); - assert!(observed_events.iter().any(|event| { - event["metadata"]["hook_event_name"] == json!("post_api_request") - && event["metadata"]["provider_payload_exact"] == json!(false) - && event["metadata"]["fidelity_source"] == json!("hermes_api_hooks") - })); -} - -#[tokio::test] -async fn hermes_uncorrelatable_pre_tool_call_does_not_create_shutdown_trajectory() { - let _guard = PLUGIN_CONFIG_TEST_LOCK.lock().await; - let temp = tempfile::tempdir().unwrap(); - let atif_dir = temp.path().join("atif"); - install_test_atif_plugin(&atif_dir).await; - let config = session_test_config(); - let manager = SessionManager::new(config); - let headers = HeaderMap::new(); - - for payload in [ - json!({ - "hook_event_name": "on_session_start", - "session_id": "hermes-main" - }), - json!({ - "hook_event_name": "pre_tool_call", - "task_id": "task-1", - "tool_name": "terminal", - "tool_input": { "command": "pwd" } - }), - json!({ - "hook_event_name": "on_session_finalize", - "session_id": "hermes-main" - }), - ] { - let outcome = crate::agents::shared::adapters::hermes::adapt(payload, &headers); - manager - .apply_events(&headers, outcome.events) - .await - .unwrap(); - } - - manager.close_all("gateway_shutdown").await.unwrap(); - clear_plugin_configuration().unwrap(); - - let trajectories: Vec = std::fs::read_dir(&atif_dir) - .unwrap() - .filter_map(Result::ok) - .map(|entry| serde_json::from_slice(&std::fs::read(entry.path()).unwrap()).unwrap()) - .collect(); - let serialized = serde_json::to_string(&trajectories).unwrap(); - assert!(serialized.contains("hermes-main")); - assert!(!serialized.contains("task-1")); - assert!(!serialized.contains("gateway_shutdown")); -} - -#[tokio::test] -async fn hermes_turn_end_snapshots_atif_without_boundary_system_step() { - let _guard = PLUGIN_CONFIG_TEST_LOCK.lock().await; - let temp = tempfile::tempdir().unwrap(); - let atif_dir = temp.path().join("atif"); - install_test_atif_plugin(&atif_dir).await; - let config = session_test_config(); - let manager = SessionManager::new(config); - let headers = HeaderMap::new(); - - for payload in [ - json!({ - "hook_event_name": "on_session_start", - "session_id": "hermes-clean" - }), - json!({ - "hook_event_name": "pre_api_request", - "session_id": "hermes-clean", - "extra": { - "task_id": "task-1", - "api_call_count": 1, - "provider": "custom", - "model": "qwen", - "request": { - "method": "POST", - "body": { - "model": "qwen", - "messages": [ - { "role": "user", "content": "hello" } - ] - } - } - } - }), - json!({ - "hook_event_name": "post_api_request", - "session_id": "hermes-clean", - "extra": { - "task_id": "task-1", - "api_call_count": 1, - "provider": "custom", - "model": "qwen", - "response": { - "assistant_message": { - "role": "assistant", - "content": "done" - }, - "usage": { - "prompt_tokens": 10, - "completion_tokens": 5 - } - } - } - }), - json!({ - "hook_event_name": "on_session_end", - "session_id": "hermes-clean" - }), - ] { - let outcome = crate::agents::shared::adapters::hermes::adapt(payload, &headers); - manager - .apply_events(&headers, outcome.events) - .await - .unwrap(); - } - - clear_plugin_configuration().unwrap(); - let atif = read_atif_for_session(&atif_dir, "hermes-clean"); - assert!(atif["subagent_trajectories"].is_null()); - assert_eq!(atif["steps"].as_array().unwrap().len(), 2); - assert_eq!(atif["steps"][0]["source"], json!("user")); - assert_eq!(atif["steps"][1]["source"], json!("agent")); - assert!( - atif["steps"].as_array().unwrap().iter().all(|step| { - step["source"] != json!("system") - || step["message"].as_object().is_some_and(|message| { - !message.is_empty() && message.contains_key("hook_event_name") - }) - }), - "Hermes hook system steps must not be anonymous or empty: {}", - serde_json::to_string_pretty(&atif["steps"]).unwrap() - ); -} - -#[tokio::test] -async fn hermes_task_id_tool_hooks_reuse_api_session() { - let config = session_test_config(); - let manager = SessionManager::new(config); - let headers = HeaderMap::new(); - - for payload in [ - json!({ - "hook_event_name": "on_session_start", - "session_id": "hermes-main" - }), - json!({ - "hook_event_name": "pre_api_request", - "session_id": "hermes-main", - "extra": { - "task_id": "task-1", - "api_call_count": 1, - "provider": "custom", - "model": "qwen", - "request": { - "body": { - "model": "qwen", - "messages": [ - { "role": "user", "content": "read file" } - ] - } - } - } - }), - ] { - let outcome = crate::agents::shared::adapters::hermes::adapt(payload, &headers); - manager - .apply_events(&headers, outcome.events) - .await - .unwrap(); - } - - let pre_tool = crate::agents::shared::adapters::hermes::adapt( - json!({ - "hook_event_name": "pre_tool_call", - "session_id": "hermes-main", - "tool_name": "read_file", - "tool_input": { "path": "README.md" }, - "extra": { - "task_id": "task-1", - "tool_call_id": "tool-1" - } - }), - &headers, - ); - manager - .apply_events(&headers, pre_tool.events) - .await - .unwrap(); - - { - let sessions = manager.inner.lock().await; - assert!(sessions.contains_key("hermes-main")); - assert!( - !sessions.contains_key("task-1"), - "Hermes tool hooks keyed by task_id should not create a duplicate session" - ); - let session = sessions.get("hermes-main").unwrap(); - assert!( - !session.tools.is_empty(), - "pre_tool_call should open an active tool before post_tool_call runs" - ); - } - - let post_tool = crate::agents::shared::adapters::hermes::adapt( - json!({ - "hook_event_name": "post_tool_call", - "session_id": "hermes-main", - "tool_name": "read_file", - "tool_input": { "path": "README.md" }, - "tool_response": { "content": "hello" }, - "extra": { - "task_id": "task-1", - "tool_call_id": "provider-tool-1" - } - }), - &headers, - ); - manager - .apply_events(&headers, post_tool.events) - .await - .unwrap(); - - let sessions = manager.inner.lock().await; - let session = sessions.get("hermes-main").unwrap(); - assert!( - session.tools.is_empty(), - "post_tool_call should close the matching pre_tool_call even when call IDs differ" - ); -} - -#[tokio::test] -async fn hermes_post_tool_call_writes_atif_observation_with_source_call_id() { - let _guard = PLUGIN_CONFIG_TEST_LOCK.lock().await; - let temp = tempfile::tempdir().unwrap(); - let atif_dir = temp.path().join("atif"); - install_test_atif_plugin(&atif_dir).await; - let config = session_test_config(); - let manager = SessionManager::new(config); - let headers = HeaderMap::new(); - - for payload in [ - json!({ - "hook_event_name": "on_session_start", - "session_id": "hermes-tool-result" - }), - json!({ - "hook_event_name": "pre_api_request", - "session_id": "hermes-tool-result", - "extra": { - "task_id": "task-1", - "api_request_id": "turn-1:api:1", - "api_call_count": 1, - "provider": "custom", - "model": "qwen", - "request": { - "body": { - "model": "qwen", - "messages": [ - { "role": "user", "content": "search for needle" } - ], - "tools": [ - { - "type": "function", - "function": { "name": "search_files" } - } - ] - } - } - } - }), - json!({ - "hook_event_name": "post_api_request", - "session_id": "hermes-tool-result", - "extra": { - "task_id": "task-1", - "api_request_id": "turn-1:api:1", - "api_call_count": 1, - "provider": "custom", - "model": "qwen", - "response": { - "assistant_message": { - "role": "assistant", - "content": "", - "tool_calls": [ - { - "id": "call-search-1", - "type": "function", - "function": { - "name": "search_files", - "arguments": "{\"query\":\"needle\"}" - } - } - ] - }, - "finish_reason": "tool_calls" - } - } - }), - json!({ - "hook_event_name": "pre_tool_call", - "session_id": "hermes-tool-result", - "tool_name": "search_files", - "tool_input": { "query": "needle" }, - "extra": { - "task_id": "task-1", - "tool_call_id": "call-search-1" - } - }), - json!({ - "hook_event_name": "post_tool_call", - "session_id": "hermes-tool-result", - "tool_name": "search_files", - "tool_input": { "query": "needle" }, - "tool_response": { "total_count": 6 }, - "extra": { - "task_id": "task-1", - "tool_call_id": "call-search-1" - } - }), - json!({ - "hook_event_name": "on_session_finalize", - "session_id": "hermes-tool-result" - }), - ] { - let outcome = crate::agents::shared::adapters::hermes::adapt(payload, &headers); - manager - .apply_events(&headers, outcome.events) - .await - .unwrap(); - } - - clear_plugin_configuration().unwrap(); - let atif = read_atif_for_session(&atif_dir, "hermes-tool-result"); - let steps = atif["steps"].as_array().unwrap(); - assert_eq!(steps.len(), 2); - assert_eq!(steps[0]["message"], json!("search for needle")); - - let agent = &steps[1]; - assert_eq!(agent["source"], json!("agent")); - assert_eq!( - agent["tool_calls"][0]["tool_call_id"], - json!("call-search-1") - ); - assert_eq!( - agent["tool_calls"][0]["function_name"], - json!("search_files") - ); - assert_eq!( - agent["observation"]["results"][0]["source_call_id"], - json!("call-search-1") - ); - assert!(agent["observation"]["results"][0].get("content").is_none()); - assert_eq!( - agent["observation"]["results"][0]["extra"]["tool_result"]["total_count"], - json!(6) - ); -} - -#[tokio::test] -async fn hermes_orphan_subagent_stop_does_not_create_atif_mark_step() { - let _guard = PLUGIN_CONFIG_TEST_LOCK.lock().await; - let temp = tempfile::tempdir().unwrap(); - let atif_dir = temp.path().join("atif"); - install_test_atif_plugin(&atif_dir).await; - let config = session_test_config(); - let manager = SessionManager::new(config); - let headers = HeaderMap::new(); - - drive_hermes_orphan_subagent_stop(&manager, &headers, "hermes-orphan", "worker-1").await; - - clear_plugin_configuration().unwrap(); - let atif = read_atif_for_session(&atif_dir, "hermes-orphan"); - assert!(atif["subagent_trajectories"].is_null()); - let root_steps = atif["steps"].as_array().unwrap(); - assert!(root_steps.is_empty()); -} - -#[tokio::test] -async fn hermes_orphan_subagent_stop_links_atof_and_openinference_to_turn() { - let _guard = PLUGIN_CONFIG_TEST_LOCK.lock().await; - let tracked_sessions = tracked_sessions(&["hermes-orphan"]); - let temp = tempfile::tempdir().unwrap(); - let atof_exporter = make_atof_test_exporter(&temp.path().join("atof"), "events.jsonl"); - let atof_name = "cli-hermes-orphan-atof-test"; - let openinference_name = "cli-hermes-orphan-openinference-test"; - register_filtered_session_subscriber( - atof_name, - Arc::clone(&tracked_sessions), - atof_exporter.subscriber(), - ); - - let (openinference_subscriber, span_exporter) = - make_openinference_test_subscriber("session-test-scope"); - register_filtered_session_subscriber( - openinference_name, - Arc::clone(&tracked_sessions), - openinference_subscriber.subscriber(), - ); - - let manager = SessionManager::new(session_test_config()); - let headers = HeaderMap::new(); - drive_hermes_orphan_subagent_stop(&manager, &headers, "hermes-orphan", "worker-1").await; - - atof_exporter.force_flush().unwrap(); - openinference_subscriber.force_flush().unwrap(); - assert!(deregister_subscriber(atof_name).unwrap()); - assert!(deregister_subscriber(openinference_name).unwrap()); - - let atof_events = read_atof_events(atof_exporter.path().expect("file sink path")); - let turn_start = atof_events - .iter() - .find(|event| { - event["category"] == "custom" - && event["scope_category"] == "start" - && event["metadata"]["session_id"] == json!("hermes-orphan") - && event["metadata"]["nemo_relay_scope_role"] == json!("turn") - }) - .expect("Hermes orphan flow should export a parent turn start event"); - let orphan_marks = atof_events - .iter() - .filter(|event| event["name"] == json!("subagent_end_without_start")) - .collect::>(); - assert_eq!( - orphan_marks.len(), - 1, - "Hermes orphan flow should export exactly one readable orphan mark: {atof_events:#?}" - ); - assert_eq!(orphan_marks[0]["parent_uuid"], turn_start["uuid"]); - - let spans = span_exporter.get_finished_spans().unwrap(); - assert!( - spans - .iter() - .all(|span| span.name.as_ref() != "mark:subagent_end_without_start"), - "Correlated Hermes orphan mark should attach to the turn span instead of exporting a standalone orphan span" - ); - let turn_span = spans - .iter() - .find(|span| { - let attributes = attr_map(&span.attributes); - attributes - .get("openinference.span.kind") - .map(String::as_str) - == Some("CHAIN") - && attributes.get("metadata").is_some_and(|metadata| { - serde_json::from_str::(metadata) - .ok() - .is_some_and(|metadata| { - metadata["session_id"] == json!("hermes-orphan") - && metadata["nemo_relay_scope_role"] == json!("turn") - }) - }) - }) - .expect("Hermes orphan flow should export an OpenInference turn span"); - let turn_attributes = attr_map(&turn_span.attributes); - let orphan_event = turn_span - .events - .events - .iter() - .find(|event| event.name.as_ref() == "subagent_end_without_start") - .expect("Hermes orphan mark should attach to the active turn span"); - let orphan_attributes = attr_map(&orphan_event.attributes); - assert_eq!( - orphan_attributes.get("nemo_relay.mark.parent_uuid"), - turn_attributes.get("nemo_relay.uuid") - ); -} - -#[tokio::test] -async fn hermes_subagent_child_session_embeds_non_empty_atif_trajectory() { - let _guard = PLUGIN_CONFIG_TEST_LOCK.lock().await; - let temp = tempfile::tempdir().unwrap(); - let atif_dir = temp.path().join("atif"); - install_test_atif_plugin(&atif_dir).await; - let config = session_test_config(); - let manager = SessionManager::new(config); - let headers = HeaderMap::new(); - - drive_hermes_subagent_child_session( - &manager, - &headers, - "parent-session", - "child-session", - "sa-1", - ) - .await; - - clear_plugin_configuration().unwrap(); - let atif = read_atif_for_session(&atif_dir, "parent-session"); - assert!( - atif["subagent_trajectories"] - .as_array() - .is_some_and(|trajectories| !trajectories.is_empty()), - "parent ATIF must include at least one embedded subagent trajectory: {}", - serde_json::to_string_pretty(&atif).unwrap() - ); - let child = &atif["subagent_trajectories"][0]; - assert_eq!(child["session_id"], json!("child-session")); - assert!( - !child["steps"].as_array().unwrap().is_empty(), - "embedded Hermes child trajectory must contain the child session work: {}", - serde_json::to_string_pretty(child).unwrap() - ); - assert_eq!(child["steps"][0]["source"], json!("user")); - assert_eq!(child["steps"][1]["source"], json!("agent")); - assert!(child["subagent_trajectories"].is_null()); - assert!( - !serde_json::to_string(&atif) - .unwrap() - .contains("subagent_end_without_start") - ); -} - -#[tokio::test] -async fn hermes_subagent_child_session_preserves_atof_and_openinference_lineage() { - let _guard = PLUGIN_CONFIG_TEST_LOCK.lock().await; - let tracked_sessions = tracked_sessions(&["parent-session", "child-session"]); - let temp = tempfile::tempdir().unwrap(); - let atof_exporter = make_atof_test_exporter(&temp.path().join("atof"), "events.jsonl"); - let atof_name = "cli-hermes-subagent-atof-test"; - let openinference_name = "cli-hermes-subagent-openinference-test"; - register_filtered_session_subscriber( - atof_name, - Arc::clone(&tracked_sessions), - atof_exporter.subscriber(), - ); - - let (openinference_subscriber, span_exporter) = - make_openinference_test_subscriber("session-test-scope"); - register_filtered_session_subscriber( - openinference_name, - Arc::clone(&tracked_sessions), - openinference_subscriber.subscriber(), - ); - - let manager = SessionManager::new(session_test_config()); - let headers = HeaderMap::new(); - drive_hermes_subagent_child_session( - &manager, - &headers, - "parent-session", - "child-session", - "sa-1", - ) - .await; - - atof_exporter.force_flush().unwrap(); - openinference_subscriber.force_flush().unwrap(); - assert!(deregister_subscriber(atof_name).unwrap()); - assert!(deregister_subscriber(openinference_name).unwrap()); - - let atof_events = read_atof_events(atof_exporter.path().expect("file sink path")); - let parent_turn = atof_events - .iter() - .find(|event| { - event["category"] == "custom" - && event["scope_category"] == "start" - && event["metadata"]["session_id"] == json!("parent-session") - && event["metadata"]["nemo_relay_scope_role"] == json!("turn") - }) - .expect("Hermes parent session should export a turn start event"); - let child_subagent_events = atof_events - .iter() - .filter(|event| { - event["category"] == "agent" - && event["metadata"]["session_id"] == json!("child-session") - && event["metadata"]["nemo_relay_scope_role"] == json!("subagent") - }) - .collect::>(); - assert_eq!( - child_subagent_events.len(), - 2, - "Hermes child session should export one subagent start/end pair: {atof_events:#?}" - ); - assert!( - child_subagent_events - .iter() - .all(|event| event["parent_uuid"] == parent_turn["uuid"]) - ); - - let spans = span_exporter.get_finished_spans().unwrap(); - let parent_turn_span = spans - .iter() - .find(|span| { - let attributes = attr_map(&span.attributes); - attributes - .get("openinference.span.kind") - .map(String::as_str) - == Some("CHAIN") - && attributes.get("metadata").is_some_and(|metadata| { - serde_json::from_str::(metadata) - .ok() - .is_some_and(|metadata| { - metadata["session_id"] == json!("parent-session") - && metadata["nemo_relay_scope_role"] == json!("turn") - }) - }) - }) - .expect("Hermes parent session should export an OpenInference turn span"); - let child_subagent_spans = spans - .iter() - .filter(|span| { - let attributes = attr_map(&span.attributes); - attributes - .get("openinference.span.kind") - .map(String::as_str) - == Some("AGENT") - && attributes.get("metadata").is_some_and(|metadata| { - serde_json::from_str::(metadata) - .ok() - .is_some_and(|metadata| { - metadata["session_id"] == json!("child-session") - && metadata["nemo_relay_scope_role"] == json!("subagent") - }) - }) - }) - .collect::>(); - assert_eq!( - child_subagent_spans.len(), - 1, - "Hermes child session should export exactly one OpenInference subagent span" - ); - let parent_attributes = attr_map(&parent_turn_span.attributes); - let child_attributes = attr_map(&child_subagent_spans[0].attributes); - assert_eq!( - child_attributes.get("nemo_relay.parent_uuid"), - parent_attributes.get("nemo_relay.uuid") - ); -} - -#[tokio::test] -async fn hermes_routed_provider_payloads_write_exact_atif_trajectory() { - let _guard = PLUGIN_CONFIG_TEST_LOCK.lock().await; - let temp = tempfile::tempdir().unwrap(); - let atif_dir = temp.path().join("atif"); - install_test_atif_plugin(&atif_dir).await; - let manager = SessionManager::new(session_test_config()); - let headers = HeaderMap::new(); - drive_hermes_routed_provider_session(&manager, &headers, "hermes-routed", None).await; - - clear_plugin_configuration().unwrap(); - let atif = read_atif_for_session(&atif_dir, "hermes-routed"); - let steps = atif["steps"].as_array().unwrap(); - assert_eq!(steps.len(), 6); - - assert_eq!(steps[0]["message"], json!("Find the file.")); - assert_eq!(steps[1]["message"], json!("I will search.")); - assert_eq!(steps[1]["tool_calls"][0]["tool_call_id"], json!("toolu_01")); - assert_eq!(steps[1]["metrics"]["prompt_tokens"], json!(11)); - assert_eq!(steps[1]["metrics"]["cached_tokens"], json!(3)); - assert_eq!(steps[1]["metrics"]["cost_usd"], json!(0.0042)); - - assert_eq!(steps[2]["message"], json!("Find the weather.")); - assert_eq!(steps[3]["message"], json!("I will check the weather.")); - assert_eq!( - steps[3]["tool_calls"][0]["tool_call_id"], - json!("call_weather_1") - ); - assert_eq!(steps[3]["metrics"]["prompt_tokens"], json!(75)); - assert_eq!(steps[3]["metrics"]["cached_tokens"], json!(10)); - assert_eq!(steps[3]["metrics"]["cost_usd"], json!(0.005)); - - assert_eq!(steps[4]["message"], json!("Inspect the files.")); - assert_eq!(steps[5]["message"], json!("I will inspect.")); - assert_eq!( - steps[5]["tool_calls"][0]["tool_call_id"], - json!("call_read_1") - ); - assert_eq!(steps[5]["metrics"]["prompt_tokens"], json!(3)); - assert_eq!(steps[5]["metrics"]["cached_tokens"], json!(2)); - assert_eq!(steps[5]["metrics"]["cost_usd"], json!(0.001)); - - assert_eq!(atif["final_metrics"]["total_prompt_tokens"], json!(89)); - assert_eq!(atif["final_metrics"]["total_completion_tokens"], json!(31)); - assert_eq!(atif["final_metrics"]["total_cached_tokens"], json!(15)); - assert_eq!(atif["final_metrics"]["total_cost_usd"], json!(0.0102)); -} - -#[tokio::test] -async fn hermes_routed_provider_payloads_emit_openinference_text_usage_and_cost() { - let _guard = PLUGIN_CONFIG_TEST_LOCK.lock().await; - let subscriber_name = "cli-hermes-routed-openinference-test"; - let session_id = "hermes-routed-openinference"; - let _ = deregister_subscriber(subscriber_name); - let (subscriber, exporter) = make_openinference_test_subscriber("session-test-scope"); - let openinference_subscriber = subscriber.subscriber(); - register_subscriber( - subscriber_name, - Arc::new(move |event| { - // Manual test-path LLM events do not carry the owning session id in metadata, - // so the routed helper tags them with a stable test marker for subscriber isolation. - if event - .metadata() - .and_then(|metadata| metadata.get(HERMES_ROUTED_TEST_SESSION_KEY)) - .and_then(Value::as_str) - == Some(session_id) - { - openinference_subscriber(event); - } - }), - ) - .unwrap(); - - let manager = SessionManager::new(session_test_config()); - let headers = HeaderMap::new(); - drive_hermes_routed_provider_session(&manager, &headers, session_id, Some(session_id)).await; - - subscriber.force_flush().unwrap(); - assert!(deregister_subscriber(subscriber_name).unwrap()); - - let spans = exporter.get_finished_spans().unwrap(); - let llm_spans: Vec> = spans - .iter() - .map(|span| attr_map(&span.attributes)) - .filter(|attributes| { - attributes - .get("openinference.span.kind") - .map(String::as_str) - == Some("LLM") - }) - .collect(); - assert_eq!(llm_spans.len(), 3); - - let anthropic = llm_spans - .iter() - .find(|attributes| { - attributes.get("output.value") - == Some(&"I will search.\nRequested tools: search".to_string()) - }) - .expect("expected Hermes-routed Anthropic OpenInference span"); - assert_eq!( - anthropic.get("llm.model_name"), - Some(&"claude-sonnet-4".to_string()) - ); - assert_eq!( - anthropic.get("input.value"), - Some(&"user: Find the file.".to_string()) - ); - assert_eq!( - anthropic.get("llm.token_count.prompt"), - Some(&"11".to_string()) - ); - assert_eq!( - anthropic.get("llm.token_count.completion"), - Some(&"7".to_string()) - ); - assert_eq!( - anthropic.get("llm.token_count.prompt_details.cache_read"), - Some(&"3".to_string()) - ); - assert_eq!(anthropic.get("llm.cost.total"), Some(&"0.0042".to_string())); - - let responses = llm_spans - .iter() - .find(|attributes| { - attributes.get("output.value") - == Some(&"I will check the weather.\nRequested tools: get_weather".to_string()) - }) - .expect("expected Hermes-routed Responses OpenInference span"); - assert_eq!(responses.get("llm.model_name"), Some(&"gpt-4o".to_string())); - assert_eq!( - responses.get("llm.token_count.prompt"), - Some(&"75".to_string()) - ); - assert_eq!( - responses.get("llm.token_count.completion"), - Some(&"20".to_string()) - ); - assert_eq!( - responses.get("llm.token_count.total"), - Some(&"95".to_string()) - ); - assert_eq!( - responses.get("llm.token_count.prompt_details.cache_read"), - Some(&"10".to_string()) - ); - assert_eq!(responses.get("llm.cost.total"), Some(&"0.005".to_string())); - - let chat = llm_spans - .iter() - .find(|attributes| { - attributes.get("output.value") - == Some(&"I will inspect.\nRequested tools: read".to_string()) - }) - .expect("expected Hermes-routed chat completions OpenInference span"); - assert_eq!(chat.get("llm.model_name"), Some(&"gpt-4o".to_string())); - assert_eq!( - chat.get("input.value"), - Some(&"user: Inspect the files.".to_string()) - ); - assert_eq!(chat.get("llm.token_count.prompt"), Some(&"3".to_string())); - assert_eq!( - chat.get("llm.token_count.completion"), - Some(&"4".to_string()) - ); - assert_eq!(chat.get("llm.token_count.total"), Some(&"7".to_string())); - assert_eq!( - chat.get("llm.token_count.prompt_details.cache_read"), - Some(&"2".to_string()) - ); - assert_eq!(chat.get("llm.cost.total"), Some(&"0.001".to_string())); -} - #[tokio::test] async fn empty_hook_marks_do_not_create_empty_atif_steps() { let _guard = PLUGIN_CONFIG_TEST_LOCK.lock().await; @@ -3812,21 +2093,21 @@ async fn empty_hook_marks_do_not_create_empty_atif_steps() { vec![ NormalizedEvent::AgentStarted(SessionEvent { session_id: "empty-mark".into(), - agent_kind: AgentKind::Hermes, + agent_kind: AgentKind::Gateway, event_name: "on_session_start".into(), payload: json!({}), metadata: json!({}), }), NormalizedEvent::HookMark(SessionEvent { session_id: "empty-mark".into(), - agent_kind: AgentKind::Hermes, + agent_kind: AgentKind::Gateway, event_name: "unknown".into(), payload: json!({}), metadata: json!({}), }), NormalizedEvent::AgentEnded(SessionEvent { session_id: "empty-mark".into(), - agent_kind: AgentKind::Hermes, + agent_kind: AgentKind::Gateway, event_name: "on_session_finalize".into(), payload: json!({}), metadata: json!({}), @@ -4668,12 +2949,12 @@ async fn unidentified_concurrent_gateway_calls_use_isolated_ephemeral_sessions() manager .apply_events( &HeaderMap::new(), - ["hermes-a", "hermes-b"] + ["gateway-a", "gateway-b"] .into_iter() .map(|session_id| { NormalizedEvent::AgentStarted(SessionEvent { session_id: session_id.into(), - agent_kind: AgentKind::Hermes, + agent_kind: AgentKind::Gateway, event_name: "on_session_start".into(), payload: json!({}), metadata: json!({}), @@ -4718,8 +2999,8 @@ async fn unidentified_concurrent_gateway_calls_use_isolated_ephemeral_sessions() let sessions = manager.inner.lock().await; assert!(!sessions.contains_key(&first.session_id)); assert!(sessions.contains_key(&second.session_id)); - assert!(sessions.contains_key("hermes-a")); - assert!(sessions.contains_key("hermes-b")); + assert!(sessions.contains_key("gateway-a")); + assert!(sessions.contains_key("gateway-b")); } manager @@ -4728,8 +3009,8 @@ async fn unidentified_concurrent_gateway_calls_use_isolated_ephemeral_sessions() assert!(!manager.has_open_sessions().await); let sessions = manager.inner.lock().await; assert!(!sessions.contains_key(&second.session_id)); - assert!(sessions.contains_key("hermes-a")); - assert!(sessions.contains_key("hermes-b")); + assert!(sessions.contains_key("gateway-a")); + assert!(sessions.contains_key("gateway-b")); } #[tokio::test] @@ -6509,8 +4790,8 @@ async fn single_tool_hint_does_not_claim_same_name_with_different_call_and_args( json!({ "cmd": "ls" }), ), ( - AgentKind::Hermes, - "hermes", + AgentKind::Gateway, + "gateway", "shell", json!({ "command": "pwd" }), json!({ "command": "ls" }), diff --git a/crates/cli/tests/coverage/shared/setup_tests.rs b/crates/cli/tests/coverage/shared/setup_tests.rs index 7f72142b6..57bc7c58f 100644 --- a/crates/cli/tests/coverage/shared/setup_tests.rs +++ b/crates/cli/tests/coverage/shared/setup_tests.rs @@ -88,9 +88,8 @@ impl Drop for EnvScope { fn detect_installed_agents_finds_binaries_on_path() { use std::os::unix::fs::PermissionsExt; let temp = tempfile::tempdir().unwrap(); - // Drop stub binaries for two of the three supported agents — confirming detection picks up - // only the ones present and ignores the others. - for exec in ["claude", "hermes"] { + // Drop stub binaries for both supported agents. + for exec in ["claude", "codex"] { let path = temp.path().join(exec); std::fs::write(&path, "#!/bin/sh\nexit 0\n").unwrap(); std::fs::set_permissions(&path, std::fs::Permissions::from_mode(0o755)).unwrap(); @@ -101,8 +100,7 @@ fn detect_installed_agents_finds_binaries_on_path() { // race with every other test that reads the environment. let detected = detect_installed_agents_in(Some(temp.path().as_os_str())); assert!(detected.contains(&CodingAgent::ClaudeCode)); - assert!(detected.contains(&CodingAgent::Hermes)); - assert!(!detected.contains(&CodingAgent::Codex)); + assert!(detected.contains(&CodingAgent::Codex)); } #[test] @@ -156,7 +154,7 @@ fn build_config_emits_agents_block_with_user_facing_keys() { fn save_config_writes_user_scope_to_user_config_dir() { let _xdg = XdgScope::cleared(); let answers = SetupAnswers { - agents: vec![CodingAgent::ClaudeCode], + agents: vec![CodingAgent::Codex], }; let doc = build_config(&answers); let home = tempfile::tempdir().unwrap(); @@ -173,7 +171,7 @@ fn save_config_writes_user_scope_to_user_config_dir() { let contents = std::fs::read_to_string(&written[0]).unwrap(); assert!(user_config_dir.is_dir()); assert!(!contents.contains("[exporters]")); - assert!(contents.contains("[agents.claude]")); + assert!(contents.contains("[agents.codex]")); } #[test] @@ -352,13 +350,13 @@ fn write_or_merge_replaces_agents_without_merge_scope_and_preserves_other_sectio ) .unwrap(); let doc = build_config(&SetupAnswers { - agents: vec![CodingAgent::Hermes], + agents: vec![CodingAgent::ClaudeCode], }); write_or_merge(&path, &doc, None).unwrap(); let overwritten = std::fs::read_to_string(&path).unwrap(); assert!(!overwritten.contains("[agents.codex]")); - assert!(overwritten.contains("[agents.hermes]")); + assert!(overwritten.contains("[agents.claude]")); assert!(overwritten.contains("[upstream]")); assert!(overwritten.contains("https://example.test")); @@ -437,19 +435,17 @@ fn reset_noops_when_user_config_is_missing() { #[test] fn reset_reports_missing_or_malformed_agent_blocks_without_rewriting() { let temp = tempfile::tempdir().unwrap(); - let hermes_home = temp.path().join("hermes-home"); let _env = EnvScope::set(&[ ("HOME", Some(temp.path().as_os_str())), ("USERPROFILE", None), ("XDG_CONFIG_HOME", None), - ("HERMES_HOME", Some(hermes_home.as_os_str())), ]); let config_dir = temp.path().join(".config/nemo-relay"); std::fs::create_dir_all(&config_dir).unwrap(); let path = config_dir.join("config.toml"); std::fs::write(&path, "agents = \"not-a-table\"\n").unwrap(); - reset(Some(CodingAgent::Hermes)).unwrap(); + reset(Some(CodingAgent::Codex)).unwrap(); assert_eq!( std::fs::read_to_string(&path).unwrap(), @@ -457,7 +453,7 @@ fn reset_reports_missing_or_malformed_agent_blocks_without_rewriting() { ); std::fs::write(&path, "not valid toml = [\n").unwrap(); - let error = reset(Some(CodingAgent::Hermes)).unwrap_err().to_string(); + let error = reset(Some(CodingAgent::Codex)).unwrap_err().to_string(); assert!( error.contains("could not parse existing config"), "error was: {error}" diff --git a/crates/core/tests/unit/observability/plugin_component_tests.rs b/crates/core/tests/unit/observability/plugin_component_tests.rs index 5ee3cd9e1..4b2f45b15 100644 --- a/crates/core/tests/unit/observability/plugin_component_tests.rs +++ b/crates/core/tests/unit/observability/plugin_component_tests.rs @@ -493,7 +493,12 @@ fn default_config_and_component_conversion_cover_public_shape() { assert!(generic.enabled); assert_eq!(generic.config["version"], json!(3)); assert_eq!(generic.config["atif"]["agent_name"], json!("NeMo Relay")); - let serialized_endpoint = &generic.config["opentelemetry"]["endpoints"][0]; + assert_endpoint_batch_fields_omitted(&generic.config["opentelemetry"]["endpoints"][0]); + + assert_endpoint_batch_fields_deserialize(); +} + +fn assert_endpoint_batch_fields_omitted(serialized_endpoint: &Json) { for field in [ "max_queue_size", "max_export_batch_size", @@ -501,8 +506,6 @@ fn default_config_and_component_conversion_cover_public_shape() { ] { assert!(serialized_endpoint.get(field).is_none()); } - - assert_endpoint_batch_fields_deserialize(); } fn assert_endpoint_batch_fields_deserialize() { diff --git a/docs/about-nemo-relay/overview.mdx b/docs/about-nemo-relay/overview.mdx index d29eeff90..3ab68ae58 100644 --- a/docs/about-nemo-relay/overview.mdx +++ b/docs/about-nemo-relay/overview.mdx @@ -45,11 +45,14 @@ Pick the row closest to what you are trying to do. | Goal | Start With | Why | |---|---|---| -| Observe Codex, Claude Code, or Hermes locally | [NeMo Relay CLI](/nemo-relay-cli/about) and [Basic Usage](/nemo-relay-cli/basic-usage) | Relay runs as a local sidecar, forwards hooks, routes provider traffic when configured, and writes observability artifacts without changing application code. | +| Observe Codex or Claude Code locally | [NeMo Relay CLI](/nemo-relay-cli/about) and [Basic Usage](/nemo-relay-cli/basic-usage) | Relay runs as a local sidecar, forwards hooks, routes provider traffic when configured, and writes observability artifacts without changing application code. | | Instrument application-owned LLM or tool calls | [Instrument Applications](/instrument-applications/about) | Direct SDK instrumentation lets Relay run the complete lifecycle and middleware sequence around callbacks your code owns. | | Use LangChain, LangGraph, Deep Agents, or OpenClaw | [Supported Integrations](/supported-integrations/about) | Maintained integrations use public framework or plugin APIs to capture supported lifecycle events. | | Configure traces, trajectories, or raw event export | [Observability](/configure-plugins/observability/about) | Exporters consume the same lifecycle event stream and write ATOF, ATIF, OpenTelemetry, or OpenInference output. | +Hermes Agent understands NeMo Relay plugin configurations. Relay is built into +Hermes Agent without a separate observability plugin or Relay CLI setup. + To evaluate a language binding with the smallest complete example, start with [Quick Start](/getting-started/quick-start). diff --git a/docs/about-nemo-relay/release-notes/index.mdx b/docs/about-nemo-relay/release-notes/index.mdx index a539cd623..1478a58f5 100644 --- a/docs/about-nemo-relay/release-notes/index.mdx +++ b/docs/about-nemo-relay/release-notes/index.mdx @@ -63,6 +63,15 @@ for destination paths and explicit-file alternatives. - _Fixed issues will be added for the 0.8 release._ +### Breaking Changes in 0.8 + +- Removed Hermes-specific support from the NeMo Relay CLI, including the + `hermes` shortcut, `run --agent hermes`, and Hermes-specific install, + uninstall, doctor, configuration, MCP selection, and `/hooks/hermes` paths. + NeMo Relay is built into Hermes Agent, and Hermes Agent understands NeMo Relay + plugin configurations. No separate observability plugin or Relay CLI setup is + required. + ## Known Issues in 0.8 - Go and the raw C FFI remain experimental and source-first. Generated API @@ -70,7 +79,7 @@ for destination paths and explicit-file alternatives. - Local coding-agent observability depends on host hooks and provider traffic reaching the local gateway. Relay cannot fully capture remote or cloud execution that bypasses the local host. -- Persistent Codex, Claude Code, and Hermes integrations use user-scoped +- Persistent Codex and Claude Code integrations use user-scoped configuration and a shared loopback gateway. Use `nemo-relay run` when a launch must retain project-specific configuration. - On Windows, a restrictive host Job Object can limit gateway reuse or prevent diff --git a/docs/getting-started/about.mdx b/docs/getting-started/about.mdx index 1e73b0ccc..6a9a92e2c 100644 --- a/docs/getting-started/about.mdx +++ b/docs/getting-started/about.mdx @@ -59,9 +59,13 @@ Start with the fastest path that can prove Relay is working, then move toward more involved integration and development workflows only when you need them. - **Try now with the CLI:** Use [NeMo Relay CLI](/nemo-relay-cli/about) when a - local Claude Code, Codex, or Hermes coding-agent harness owns invocation. + local Claude Code or Codex coding-agent harness owns invocation. Relay can observe hooks, gateway-routed model traffic, and exporter output without changing application code. +- **Use Hermes Agent's native integration:** NeMo Relay is built into Hermes + Agent, and Hermes Agent understands NeMo Relay plugin configurations. No + separate Relay installation, observability plugin, or Relay CLI setup is + required. - **Use a supported integration:** Start with [Supported Integrations](/supported-integrations/about) when LangChain, LangGraph, Deep Agents, or OpenClaw already owns scheduling, callbacks, tool diff --git a/docs/getting-started/installation.mdx b/docs/getting-started/installation.mdx index b628e8074..31154c6a0 100644 --- a/docs/getting-started/installation.mdx +++ b/docs/getting-started/installation.mdx @@ -74,13 +74,12 @@ inherit the persistent `PATH` change. Verify that the command is available: nemo-relay --version ``` -After installation, `nemo-relay` can also install persistent Claude Code, -Codex, and Hermes Agent integrations: +After installation, `nemo-relay` can also install persistent Claude Code and +Codex integrations: ```bash nemo-relay install claude-code nemo-relay install codex -nemo-relay install hermes ``` Refer to [Coding Agent Installation](/nemo-relay-cli/plugin-installation) for @@ -269,24 +268,9 @@ configuration and verification steps. Install Hermes Agent 0.18.2 or newer by following the [official Hermes Agent installation guide](https://hermes-agent.nousresearch.com/docs/getting-started/installation). -Open a new shell after installation and verify that the Hermes CLI is available: - -```bash -hermes --version -``` - -After the NeMo Relay CLI is installed, install and diagnose the Hermes user -integration: - -```bash -nemo-relay install hermes -nemo-relay doctor --plugin hermes -``` - -Relay adds a native stdio MCP entry and trusts only the lifecycle hooks that it -installs. The MCP process acquires the shared user-level gateway when it starts. -Refer to the [Hermes Agent guide](/nemo-relay-cli/hermes) for provider routing, -lifecycle, and removal steps. +NeMo Relay is built into Hermes Agent. No separate NeMo Relay package, +observability plugin, or Relay CLI setup is required. Hermes Agent understands +NeMo Relay plugin configurations. ### Python Framework Integrations diff --git a/docs/getting-started/quick-start/index.mdx b/docs/getting-started/quick-start/index.mdx index 972debb96..9f77b6ea9 100644 --- a/docs/getting-started/quick-start/index.mdx +++ b/docs/getting-started/quick-start/index.mdx @@ -59,7 +59,8 @@ not yet know which guide owns the working path. | Layer | Use When | Start Here | Success Check | |---|---|---|---| | CLI and gateway | A coding-agent harness owns invocation and provider routing | [CLI Basic Usage](/nemo-relay-cli/basic-usage) | The wrapped agent runs with Relay active and emits hook output, plus gateway-routed LLM lifecycle output when provider routing is active. | -| Persistent coding-agent installs | You want the maintained install path for Codex, Claude Code, or Hermes Agent instead of the transparent wrapper | [Coding Agent Installation](/nemo-relay-cli/plugin-installation) | `nemo-relay doctor --plugin ` confirms the installed integration is ready. | +| Persistent coding-agent installs | You want the maintained install path for Codex or Claude Code instead of the transparent wrapper | [Coding Agent Installation](/nemo-relay-cli/plugin-installation) | `nemo-relay doctor --plugin ` confirms the installed integration is ready. | +| Hermes Agent native integration | Hermes Agent owns the agent lifecycle | NeMo Relay is built into Hermes Agent | Hermes Agent requires no separate Relay installation, observability plugin, or Relay CLI setup and understands NeMo Relay plugin configurations. | | Direct Python or Node.js application APIs | Your application owns the tool or LLM callback | [Python Quick Start](/getting-started/quick-start/python) or [Node.js Quick Start](/getting-started/quick-start/nodejs) | The sample prints event lines plus tool and LLM results. | | Direct Rust application APIs | Your Rust application owns the tool or LLM callback | [Rust Quick Start](/getting-started/quick-start/rust) | The sample prints scope, tool, and LLM lifecycle output plus the `initialized` mark event. | | Plugin-managed runtime setup | You need process-level exporter or plugin behavior from `plugins.toml` | [Plugin Configuration Files](/configure-plugins/plugin-configuration-files) | The selected plugin path activates and writes the expected output or behavior. | @@ -70,8 +71,8 @@ not yet know which guide owns the working path. ## Local Coding-Agent Runs -Use the NVIDIA NeMo Relay CLI when you want to observe a local Codex, Claude -Code, or Hermes Agent session without changing application code. +Use the NVIDIA NeMo Relay CLI when you want to observe a local Codex or Claude +Code session without changing application code. ` when you want to launch an explicit command @@ -79,12 +75,11 @@ selected profile, and installed plugin state unchanged. A process marker makes any installed Relay MCP borrow the wrapper-owned dynamic gateway. Claude's persistent hooks exit without forwarding; Codex disables the known Relay plugin hook identities in its process-local CLI state. Only the injected wrapper hooks -deliver events. Hermes gets the same isolation through its process-private -configuration overlay. +deliver events. If a launcher or wrapper hides the real agent name, set that wrapper as the -configured command and pass `--agent`. The same pattern applies to Claude Code, -Codex, and Hermes: +configured command and pass `--agent`. The same pattern applies to Claude Code +and Codex: ```toml [agents.codex] @@ -95,41 +90,27 @@ command = "my-codex-wrapper" nemo-relay run --agent codex ``` -For Hermes, interactive setup configures only Relay's transparent wrapper. -Persistent MCP and trusted shell-hook state is owned by -`nemo-relay install hermes`. Transparent `run --agent hermes` exports the -dynamic `NEMO_RELAY_GATEWAY_URL` through a process-private `HERMES_HOME` -overlay; it never rewrites the user's Hermes config. - Use `--dry-run --print` to inspect the generated hook config, gateway environment, gateway URL, and final command without launching the agent. ## Persistent Host Integrations -Use persistent integration installation to let Claude Code, Codex, or Hermes -Agent load Relay without a wrapper command: +Use persistent integration installation to let Claude Code or Codex load Relay +without a wrapper command: ```bash nemo-relay install claude-code nemo-relay install codex -nemo-relay install hermes nemo-relay install all ``` -Claude Code and Codex use marketplace plugins. Hermes exposes the same stdio -MCP lifecycle through user configuration. - -Hermes, Claude Code, and Codex MCP clients can share the native gateway on +Claude Code and Codex use marketplace plugins. Their MCP clients can share the native gateway on `127.0.0.1:47632`. Every `nemo-relay mcp` process acquires that gateway when it -launches. Claude Code uses `alwaysLoad` to wait for the MCP connection; -Hermes starts discovery asynchronously, so its hook command waits for gateway -readiness before it sends the original payload. +launches. Claude Code uses `alwaysLoad` to wait for the MCP connection. For Claude Code and Codex, installation writes a local marketplace, installs the generated `nemo-relay-plugin`, and configures host-specific hooks and -provider routing. For Hermes, installation updates the Relay-owned portions of -the user configuration instead. All three paths use the local `nemo-relay` -binary on `PATH`. +provider routing. Both paths use the local `nemo-relay` binary on `PATH`. Use plugin doctor and uninstall for the installed host state: @@ -449,8 +430,8 @@ calling NeMo Relay APIs. that scope when it is still active. - Tool pre-use starts a NeMo Relay tool span. Tool post-use, denial, or failure closes it. -- Generated `UserPromptSubmit`, `Stop`, and Hermes `pre_llm_call` / - `post_llm_call` hooks are retained as private correlation hints. The adapters +- Generated `UserPromptSubmit` and `Stop` hooks are retained as private + correlation hints. The adapters do the same when a custom or older integration delivers a response or agent-thought hook. These hints are not emitted as NeMo Relay events. - Compaction, notification, and unknown hook events become mark events under @@ -496,11 +477,6 @@ Generated hook bundles subscribe to the events needed for that mapping: | --- | --- | --- | | Claude Code | `UserPromptSubmit`, `Stop` | `SessionStart`, `SessionEnd`, `UserPromptExpansion`, `SubagentStart`, `SubagentStop`, `PreToolUse`, `PostToolUse`, `PostToolUseFailure`, `PermissionRequest`, `Notification`, `PreCompact`, `PostCompact` | | Codex | `UserPromptSubmit`, `Stop` | `SessionStart`, `SubagentStart`, `SubagentStop`, `PreToolUse`, `PostToolUse`, `PermissionRequest`, `PreCompact`, `PostCompact` | -| Hermes | `pre_api_request`, `post_api_request`, `api_request_error`, `pre_llm_call`, `post_llm_call` | `on_session_start`, `on_session_end`, `on_session_finalize`, `on_session_reset`, `subagent_start`, `subagent_stop`, `pre_tool_call`, `post_tool_call` | - -Hermes `pre_api_request`, `post_api_request`, and `api_request_error` hooks -map to NeMo Relay LLM start/end events when present. Hermes `pre_llm_call` and -`post_llm_call` remain private correlation hints. ## Hook Forwarding @@ -509,16 +485,14 @@ Transparent Claude Code and Codex hooks invoke The wrapper-owned hook command embeds its ephemeral gateway URL and is marked as transparent so it cannot recover the fixed gateway. -Persistent Claude Code, Codex, and Hermes hooks also use +Persistent Claude Code and Codex hooks also use `nemo-relay hook-forward `. Each generated command identifies the fixed gateway and its private install-generation fence, waits for the MCP-owned Relay -gateway, verifies it, and forwards the unchanged payload once. Hermes setup -stores the canonical absolute command and trusts only its exact event pairs; it -does not enable global hook auto-acceptance. +gateway, verifies it, and forwards the unchanged payload once. `hook-forward` reads the canonical hook payload from standard input, sends it to the matching endpoint, and prints the endpoint response. Generated -`PreToolUse`, `PermissionRequest`, and Hermes `pre_tool_call` hooks use +`PreToolUse` and `PermissionRequest` hooks use `--fail-closed`; generated lifecycle and after-the-fact hooks use `--fail-open`. This blocks permission-bearing operations when Relay cannot evaluate them without making observability-only hooks a runtime dependency. @@ -553,7 +527,6 @@ application-mode caveats. - [Claude Code](/nemo-relay-cli/claude-code) - [Codex](/nemo-relay-cli/codex) - [Coding Agent Installation](/nemo-relay-cli/plugin-installation) -- [Hermes Agent](/nemo-relay-cli/hermes) Each guide covers transparent run setup, gateway routing, hook smoke tests, Agent Trajectory Interchange Format (ATIF) export verification at the host's diff --git a/docs/nemo-relay-cli/claude-code.mdx b/docs/nemo-relay-cli/claude-code.mdx index de90b43a2..e5ac10ae7 100644 --- a/docs/nemo-relay-cli/claude-code.mdx +++ b/docs/nemo-relay-cli/claude-code.mdx @@ -69,7 +69,7 @@ starts or reuses the shared gateway on `127.0.0.1:47632` immediately when the MCP process launches. The client verifies the gateway identity and effective persistent configuration, heartbeats it while MCP stdio remains open, and performs one coordinated restart if the gateway becomes unhealthy. Claude -Code, Codex, and Hermes MCP clients share a compatible gateway, and the gateway +Code and Codex MCP clients share a compatible gateway, and the gateway exits after the final client's idle timeout. The MCP server advertises no tools. The generated entry sets `alwaysLoad: true`, so Claude Code 2.1.121 or newer diff --git a/docs/nemo-relay-cli/codex.mdx b/docs/nemo-relay-cli/codex.mdx index 2b774761c..887a6e03a 100644 --- a/docs/nemo-relay-cli/codex.mdx +++ b/docs/nemo-relay-cli/codex.mdx @@ -156,8 +156,8 @@ copied managed Python environment—to 100,000 filesystem entries, 512 MiB, and maximum directory traversal depth of 128. When startup reports an activation snapshot budget error, remove unrelated files from the manifest or load-target directory, flatten deeply nested directories, or reduce the managed Python -environment before retrying. Concurrent Codex, Claude Code, -and Hermes processes can share the sidecar. Each MCP client sends a heartbeat +environment before retrying. Concurrent Codex and Claude Code processes can +share the sidecar. Each MCP client sends a heartbeat while its stdio connection is open, and the sidecar exits after 300 seconds without activity by default. Relay does not install a wrapper, launch agent, system user service, scheduled task, login item, or persistent supervisor. diff --git a/docs/nemo-relay-cli/hermes.mdx b/docs/nemo-relay-cli/hermes.mdx deleted file mode 100644 index 61ba2d1da..000000000 --- a/docs/nemo-relay-cli/hermes.mdx +++ /dev/null @@ -1,235 +0,0 @@ ---- -title: "Hermes Agent" -description: "Configure Hermes to start and share the native NeMo Relay gateway through MCP." -position: 7 ---- -{/* SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. -SPDX-License-Identifier: Apache-2.0 */} - - -NeMo Relay can observe Hermes through a persistent integration or a transparent -run. The persistent integration uses the Hermes native studio MCP support to own a -shared Relay gateway at `127.0.0.1:47632`. Shell hooks forward the Hermes native -session, subagent, tool, and model lifecycle payloads to that gateway. If you -also point the Hermes provider at the gateway, Relay observes the model requests -and responses. - -The MCP process manages gateway lifetime through a shared lifecycle lease. You -do not need a separate service manager, wrapper process, Node.js runtime, or -Python bootstrap component. - -Use persistent capture for ordinary `hermes` commands. Use a transparent run -when you need project-specific Relay configuration or an isolated gateway for -one Hermes process. - -## Install Persistent Capture - -Install Hermes Agent 0.18.2 or newer by following the -[official Hermes Agent installation guide](https://hermes-agent.nousresearch.com/docs/getting-started/installation). -Open a new shell after installation and verify that the Hermes CLI is available: - -```bash -hermes --version -``` - -Install the user-level integration: - -```bash -nemo-relay install hermes -``` - -NeMo Relay checks the installed Hermes CLI before it changes any files. - -Relay preserves unrelated Hermes settings and updates the Relay-owned portions -of the user configuration. Hermes reads this configuration from -`$HERMES_HOME/config.yaml`, or `~/.hermes/config.yaml` when `HERMES_HOME` is -unset. This location is user-owned, like Relay's default configuration. - -The MCP server name `nemo-relay` is reserved for the Relay-managed entry. If -that name already belongs to another command, installation stops without -rewriting the config. Rename or remove the conflicting entry, then retry. - -The installer makes the following changes: - -- Adds `mcp_servers.nemo-relay`, which runs the canonical absolute - `nemo-relay` executable with the `mcp` argument. -- Binds the shared gateway to `127.0.0.1:47632` and adds a private generation - marker that prevents stale MCP clients and hooks from using a replacement - installation. -- Installs one Relay handler for each supported Hermes lifecycle event. -- Records each exact event and command pair in - `shell-hooks-allowlist.json`. -- Migrates a complete legacy Relay entry, removing its old hook commands and - approvals while preserving unrelated hooks, MCP servers, approvals, and - Hermes settings. - -NeMo Relay writes the Hermes config, allowlist, and generation marker as one -verified transaction. If a write or verification fails, it rolls those files -back to their original state. It also locks concurrent integration updates so -they cannot overwrite one another. - -After installation, run the integration doctor: - -```bash -nemo-relay doctor --plugin hermes -``` - -The doctor checks the Hermes version, Relay executable, MCP entry, lifecycle -hooks, trust records, generation marker, and forwarded environment names. - -The generated MCP entry always forwards the standard Relay, OpenTelemetry, AWS, -proxy, certificate, and provider credential variable names. It writes each one -as a `${VARIABLE_NAME}` reference instead of copying a secret value into Hermes -configuration. Before parsing the MCP command line, Relay treats an unresolved -self-reference as unset. If you later add a custom `NEMO_RELAY_`, `OTEL_`, or -`AWS_` variable or change the name of a credential variable referenced by -plugin configuration, run `nemo-relay install hermes --force` to refresh the -MCP environment. - -## Gateway Lifecycle - -Hermes launches `nemo-relay mcp` as a long-lived stdio MCP server. The server -does not expose tools. Before it answers the MCP initialization request, it -starts or joins a gateway and verifies the Relay identity, version, bootstrap -protocol, and effective persistent configuration. - -Hermes starts MCP discovery in the background, so an early lifecycle hook can -arrive before the MCP process finishes acquiring the gateway. The installed -hook waits for that MCP-owned gateway, verifies it, and forwards the original -payload once. It never launches or recovers Relay. - -Concurrent Hermes, Claude Code, and Codex MCP clients can share one compatible -gateway. Each open client keeps a liveness lease by heartbeating the gateway. A -startup lock and recovery record let overlapping clients coordinate one -restart. If the replacement fails again, the MCP clients exit with an error. -When no MCP clients or active sessions remain, the gateway exits after 300 -seconds of inactivity by default. Set `NEMO_RELAY_PLUGIN_IDLE_TIMEOUT_SECS` to a -different positive timeout in seconds before setup and launch. - -Persistent MCP mode reads only system and user Relay configuration. It does not -load a project's `.nemo-relay` layer. Use transparent run mode for -project-specific Relay configuration. - -NeMo Relay removes or replaces Hermes MCP entries, hooks, and approvals only -when it can identify them as managed. A current MCP entry must use the expected -executable, `mcp` argument, shared bind, and generation marker. A legacy entry -must use the same canonical Relay executable in its MCP entry and complete hook -set. NeMo Relay preserves partial or manual lookalike entries. Rename or remove -a conflicting manual `mcp_servers.nemo-relay` entry before installation. - -## Route Model Traffic - -Hermes API hooks can provide sanitized model request and response data. Route -provider traffic through Relay when you also need direct gateway observation -of the provider exchange. - -For an OpenAI-compatible provider, point Hermes at the shared Relay gateway: - -```bash -export OPENAI_BASE_URL=http://127.0.0.1:47632/v1 -``` - -In your user-level Relay configuration, set the real upstream separately so -Relay does not route requests back to itself: - -```toml -[upstream] -openai_base_url = "https://api.openai.com/v1" -``` - -When Hermes uses the Messages API instead, set -`ANTHROPIC_BASE_URL=http://127.0.0.1:47632` and configure Relay's -`upstream.anthropic_base_url` with the real provider address. The installer does -not change Hermes provider settings because they are provider-specific and -user-owned. - -## Lifecycle Hook Coverage - -Relay installs these Hermes hooks: - -- Session: `on_session_start`, `on_session_end`, `on_session_finalize`, and - `on_session_reset`. -- Model: `pre_api_request`, `post_api_request`, `api_request_error`, - `pre_llm_call`, and `post_llm_call`. -- Tool: `pre_tool_call` and `post_tool_call`. -- Subagent: `subagent_start` and `subagent_stop`. - -Relay records Hermes `skill_view` pre-tool calls as observed `skill.load` marks. -Each mark stores the skill name in `data.skill_name` and records the detection -source and tool name in metadata. It does not retain a skill path. - -The API-request hooks are authoritative for hook-backed model lifecycle. -`pre_llm_call` and `post_llm_call` remain private correlation hints. Gateway -routing is a separate observation path. If API-request hooks and gateway -routing are both active for the same provider call, exported data can contain a -hook-observed span and a gateway-observed span for that call. - -## Transparent Run - -Use the wrapper for project-specific Relay configuration or an isolated -per-process gateway: - -```bash -nemo-relay hermes -nemo-relay hermes -- chat --provider custom -``` - -This mode starts a gateway on an OS-assigned loopback port and exports -`NEMO_RELAY_GATEWAY_URL` to Hermes. NeMo Relay creates a process-private -`HERMES_HOME` overlay, removes any Relay-managed fixed MCP entry, and injects -temporary hooks. The overlay points Hermes's custom provider at the dynamic -gateway, and the wrapper sets `OPENAI_BASE_URL` to the same address. It links -the rest of the active Hermes profile state instead of copying it. - -The user configuration is never rewritten, so a transparent Hermes process and -a bare Hermes process cannot see each other's temporary hooks. NeMo Relay -removes the overlay after the wrapped process exits. On Windows, profile -directories use non-privileged directory junctions rather than symbolic links, -so this does not require Developer Mode or elevation. - -Inspect the launch without changing files or starting Hermes: - -```bash -nemo-relay run --agent hermes --dry-run --print -- hermes -``` - -## Remove the Integration - -Remove the Relay-owned Hermes MCP, hooks, trust, and generation state with: - -```bash -nemo-relay uninstall hermes -``` - -Unrelated Hermes configuration remains unchanged. The interactive -`nemo-relay config hermes` and `nemo-relay config --reset hermes` commands -manage only transparent-run configuration. Use `install` and `uninstall` to -manage the persistent Hermes MCP, hooks, and trust records. - -## Validate End-to-End Capture - -When a compatible Hermes CLI is installed, run the opt-in cold-start test: - -```bash -just test-hermes-mcp-e2e -``` - -The test uses isolated home directories and a local model provider. It runs 10 -cold one-shot sessions and two concurrent sessions. It then checks provider -authorization, shared gateway lifetime, exactly-once model and turn scopes, -session isolation, balanced ATOF output, hook trust, and final port release. -This opt-in test is not part of the required Rust CI suite. - -## Troubleshoot - -If Hermes reports that the MCP server failed to start, run -`nemo-relay doctor --plugin hermes` to check the installation and inspect any -reported configuration or endpoint error. If the doctor passes, inspect the -Hermes error for a gateway startup or bind failure. Detached gateways do not -create persistent log files. NeMo Relay rejects a foreign listener on -`127.0.0.1:47632` instead of adopting it. - -If lifecycle events appear but direct model spans do not, check whether the -Hermes provider base URL points at Relay. If model spans attach to the parent -instead of a subagent, preserve Hermes correlation identifiers such as -`task_id`, `turn_id`, `api_request_id`, and `tool_call_id` in the hook payloads. diff --git a/docs/nemo-relay-cli/plugin-installation.mdx b/docs/nemo-relay-cli/plugin-installation.mdx index 518858ed8..5170e68cf 100644 --- a/docs/nemo-relay-cli/plugin-installation.mdx +++ b/docs/nemo-relay-cli/plugin-installation.mdx @@ -1,16 +1,15 @@ --- title: "Coding Agent Installation" -description: "Install and manage persistent NeMo Relay integrations for Claude Code, Codex, and Hermes Agent." +description: "Install and manage persistent NeMo Relay integrations for Claude Code and Codex." position: 3 --- {/* SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. SPDX-License-Identifier: Apache-2.0 */} -Install a persistent integration when you want Claude Code, Codex, or Hermes -Agent to load NeMo Relay without a `nemo-relay` wrapper command. Claude Code -and Codex use their normal plugin systems. Hermes uses the same native MCP -lifecycle through its user configuration without a marketplace plugin. +Install a persistent integration when you want Claude Code or Codex to load +NeMo Relay without a `nemo-relay` wrapper command. Both use their normal plugin +systems. Each persistent integration forwards the lifecycle signals that its host exposes, including agent, subagent, tool, prompt, compaction, and stop events @@ -18,11 +17,11 @@ where available. Model-provider routing sends LLM traffic through the local NeMo Relay gateway. Hooks alone cannot capture complete LLM request and response spans. -All three integrations share the same lifecycle gateway. Provider traffic joins +Both integrations share the same lifecycle gateway. Provider traffic joins that gateway when routing is configured for the host: ```text -Codex / Claude Code / Hermes Agent +Codex / Claude Code |-> Relay MCP lifecycle client -------> shared Relay gateway |-> generated lifecycle hooks --------> shared Relay gateway \-> provider route (when configured) -> shared Relay gateway -> model provider @@ -42,7 +41,6 @@ The selected coding-agent CLI must also be available at a supported version: - Claude Code 2.1.121 or newer through `claude`. - `codex-cli` 0.143.0 or newer through `codex`. -- Hermes Agent 0.18.2 or newer through `hermes`. Relay checks the selected CLI version before modifying agent configuration. Prerelease, malformed, and older versions are rejected consistently by install, @@ -55,7 +53,6 @@ Run the command for the coding-agent integration that you want to install: ```bash nemo-relay install claude-code nemo-relay install codex -nemo-relay install hermes ``` Install every supported host detected on the machine: @@ -65,19 +62,18 @@ nemo-relay install all ``` `install all` selects only supported agents whose CLI is present. It fails if -Claude Code, Codex, and Hermes Agent are all absent. +Claude Code and Codex are both absent. Use `--dry-run` to preview the operation without changing host configuration. For Claude Code and Codex, the preview includes marketplace paths and host -commands. For Hermes, it shows the user configuration path: +commands: ```bash nemo-relay install codex --dry-run ``` For Claude Code and Codex, use `--install-dir` when you need a non-default -marketplace location. Hermes always updates its user configuration. The default -plugin directory is platform-specific: +marketplace location. The default plugin directory is platform-specific: | Platform | Default Plugin Install Directory | | --- | --- | @@ -119,16 +115,9 @@ undiscoverable `PostToolUseFailure`, `Notification`, or `SessionEnd` handlers. Upgrade removes legacy Relay groups from `~/.codex/hooks.json` while preserving unrelated hooks. -For Hermes, `nemo-relay install hermes` adds the Relay MCP server, canonical -hooks, exact hook trust, and environment references to -`$HERMES_HOME/config.yaml`, or `~/.hermes/config.yaml` when `HERMES_HOME` is -unset. Relay applies these changes in one transaction and preserves unrelated -Hermes configuration. Refer to the -[Hermes Agent guide](/nemo-relay-cli/hermes). - On Windows, generated hooks use the built-in Windows PowerShell encoded-command -format. This avoids quoting and metacharacter differences between the Codex, -Claude Code, and Hermes command runners. The encoded payload contains only the +format. This avoids quoting and metacharacter differences between the Codex and +Claude Code command runners. The encoded payload contains only the canonical `nemo-relay.exe` path and `hook-forward` arguments. PowerShell starts that Rust binary directly and preserves its standard input, standard output, standard error, and exit code. The MCP client and gateway remain Rust-native, @@ -139,7 +128,7 @@ if it was already running so it reloads the provider and hook configuration. ### Shared Gateway Lifecycle -Claude Code, Codex, and configured Hermes processes use the same +Claude Code and Codex processes use the same `nemo-relay mcp` lifecycle client. Before reading MCP protocol frames, it starts or reuses a detached native sidecar on `127.0.0.1:47632`. The client verifies Relay identity, version, protocol readiness, and effective user-level @@ -157,9 +146,8 @@ own activity, idle shutdown, and authenticated ownership-record cleanup. Codex marks the MCP server as required, so the captured turn waits for verified gateway readiness. Claude Code 2.1.121 or newer uses `alwaysLoad`, which blocks -session startup until the MCP connection is ready. Hermes starts MCP discovery -asynchronously, so its first lifecycle hook can arrive before MCP is ready. -Installed MCP entries and `hook-forward` commands carry both the generation-file +session startup until the MCP connection is ready. Installed MCP entries and +`hook-forward` commands carry both the generation-file path and the immutable identity expected at that path. This prevents a host process with cached plugin configuration from adopting a replacement installation after `--force`. Persistent hooks wait for the MCP-owned gateway, @@ -211,8 +199,7 @@ host, so an upgrade from any installed host can retire a gateway that another host originally started. Relay leaves custom managed endpoints untouched and never sends its shutdown -token to an unrelated listener. Hermes installation rotates its MCP generation -fence in the same transaction. +token to an unrelated listener. ### Configuration and Credentials @@ -225,13 +212,6 @@ settings to MCP differently: runtime, OpenTelemetry, AWS, proxies, certificates, approved prefixes, and credential variables referenced by user observability configuration. - Claude Code supplies its normal plugin MCP environment. -- Hermes stores `${VARIABLE_NAME}` references for the standard allowlist, - custom variables that you have already approved, and credential names - referenced by user observability configuration. - -Relay treats unresolved self-references as unset before parsing its command -line. Rerun `nemo-relay install hermes --force` after you add a custom approved -name or change a credential name referenced by the configuration. The long-lived managed sidecar injects a forwarded provider credential only when a request carries provider authorization or Relay's private per-user client @@ -240,8 +220,8 @@ managed provider's `http_headers`; Relay stores the resulting Codex config with an owner-only mode on Unix or protected owner/System DACL on Windows before the temporary file becomes visible. Relay validates and removes that header before middleware, observability, or upstream forwarding. The underlying HMAC key -remains in Relay's owner-only bootstrap state. Claude Code and Hermes send -their normal provider authorization. A foreign loopback caller cannot spend +remains in Relay's owner-only bootstrap state. Claude Code sends its normal +provider authorization. A foreign loopback caller cannot spend the sidecar's forwarded credentials. Foreground `nemo-relay --bind` use retains environment-key injection for explicit local proxy workflows. @@ -252,7 +232,6 @@ Run the command for the installed integration that you want to diagnose: ```bash nemo-relay doctor --plugin claude-code nemo-relay doctor --plugin codex -nemo-relay doctor --plugin hermes nemo-relay doctor --plugin all ``` @@ -273,13 +252,6 @@ For Claude Code, doctor also validates version 2.1.121 or newer, the generated informational because the next MCP process starts it. Persistent hooks wait for that MCP-owned gateway instead of starting it themselves. -For Hermes, doctor validates Hermes Agent 0.18.2 or newer. It reads the Relay -executable from the managed Hermes MCP entry, verifies that exact executable -supports both `mcp` and `hook-forward`, and then checks every hook and trust -pair, the generation marker, and environment references. Top-level -`nemo-relay doctor` discovers this managed state directly from the Hermes user -configuration; an `[agents.hermes]` Relay configuration block is not required. - Use the focused plugin doctor when diagnosing one host or an installation that uses a custom directory: @@ -299,7 +271,6 @@ Run the command for the installed integration that you want to remove: ```bash nemo-relay uninstall claude-code nemo-relay uninstall codex -nemo-relay uninstall hermes nemo-relay uninstall all ``` @@ -307,21 +278,19 @@ Uninstall removes Codex provider configuration and trust for the exact plugin hooks before unregistering the plugin, while Codex can still report its hook metadata. It then removes the host registration and marketplace. Claude Code provider routing is restored from the Relay backup. Unrelated user hooks and -configuration remain unchanged. Hermes uninstall removes only Relay-owned MCP, -hook, trust, and generation state. +configuration remain unchanged. ## Compatibility and Migration -These integrations now require Codex 0.143.0, Claude Code 2.1.121, or Hermes -Agent 0.18.2 at minimum. Relay rejects prerelease or malformed version output. +These integrations require Codex 0.143.0 or Claude Code 2.1.121 at minimum. +Relay rejects prerelease or malformed version output. If you configured an opaque wrapper, its `--version` output must identify the -selected host. Interactive `nemo-relay config hermes` now owns only -transparent-wrapper configuration; use `nemo-relay install hermes` and -`nemo-relay uninstall hermes` for persistent Hermes state. +selected host. -The MCP bootstrap is now host-neutral. Generated Codex, Claude Code, and Hermes -configuration invokes `nemo-relay mcp`; the removed `--agent` option no longer -parses. Existing fenced installations can be refreshed with +The MCP bootstrap is host-neutral. Generated Codex and Claude Code +configuration invokes `nemo-relay mcp`; Hermes-specific `--agent hermes` +selections no longer parse. The `--agent` option remains supported for Claude +Code and Codex `run` flows. Existing fenced installations can be refreshed with `nemo-relay install --force`. Relay refuses to replace an older MCP installation without a valid generation marker because a cached host process might still be running. In that case, follow the manual cleanup steps in the @@ -353,11 +322,6 @@ wrapper. Opaque custom wrappers remain supported when the configured command emits the selected host's canonical version output in response to `--version`. Install and doctor enforce the same minimum-version policy. -Hermes setup uses the same public install, doctor, and uninstall commands as -Claude Code and Codex. `nemo-relay config hermes` configures Relay's transparent -wrapper only; it no longer mutates Hermes user configuration. Use -`nemo-relay install hermes` for the persistent host integration. - ## Source Marketplace Discovery This repository also contains source marketplace manifests for development and diff --git a/docs/reference/migration-guides.mdx b/docs/reference/migration-guides.mdx index faeadf832..e83357040 100644 --- a/docs/reference/migration-guides.mdx +++ b/docs/reference/migration-guides.mdx @@ -13,6 +13,23 @@ intervening release in sequence. ## Upgrade to NeMo Relay 0.8 +### Move Hermes Agent to Its Native Relay Integration + +NeMo Relay 0.8 removes Hermes Agent from the Relay CLI. The `nemo-relay hermes` +shortcut, `nemo-relay run --agent hermes`, Hermes install, uninstall, doctor, +configuration, MCP selection, hook forwarding, and `/hooks/hermes` endpoint are +no longer supported. Relay configuration under `[agents.hermes]` is also +unsupported. + +Remove the `[agents.hermes]` section from every Relay configuration file before +upgrading. If the section remains, Relay rejects the configuration and prevents +configuration-resolving commands, including Claude Code and Codex runs, from +starting. + +NeMo Relay is built into Hermes Agent. Do not install Relay separately or +enable an observability plugin. Hermes Agent understands NeMo Relay plugin +configurations. + ### Move Project Configuration to a Supported Location NeMo Relay 0.8 no longer discovers repository-local configuration. Files named diff --git a/docs/reference/support-matrix.mdx b/docs/reference/support-matrix.mdx index f07a55057..e567b5c11 100644 --- a/docs/reference/support-matrix.mdx +++ b/docs/reference/support-matrix.mdx @@ -66,12 +66,14 @@ and older CLI versions during installation, diagnostics, and transparent runs. | --- | --- | --- | --- | | Claude Code | 2.1.121 | Persistent install, transparent run, lifecycle hooks, local gateway routing, and pre-tool security | Claude desktop, web, and application sessions are unsupported unless they expose the same local hook and gateway controls. Optimization requires gateway-routed LLM traffic and available hooks. | | Codex CLI | 0.143.0 | Persistent install, transparent run, 10 supported plugin hooks, local gateway routing, and pre-tool security | Cloud or remote tasks that bypass the local machine have partial or no LLM capture. The plugin hook schema has no `SessionEnd`; Relay finalizes the cumulative session snapshot at `Stop`. Encrypted Codex multi-agent v2 payloads cannot be decrypted or reliably linked. | -| Hermes Agent | 0.18.2 | Persistent MCP integration, transparent run, lifecycle hooks, local gateway routing, and pre-tool security | Optimization depends on Hermes shell and API-request hook coverage. Hook-observed and gateway-observed LLM spans can both be present when both paths are enabled. | For installation, diagnostics, and host-specific behavior, refer to [Coding Agent Installation](/nemo-relay-cli/plugin-installation), [Claude -Code](/nemo-relay-cli/claude-code), [Codex](/nemo-relay-cli/codex), and [Hermes -Agent](/nemo-relay-cli/hermes). +Code](/nemo-relay-cli/claude-code), and [Codex](/nemo-relay-cli/codex). + +Hermes Agent includes NeMo Relay as a native in-process integration. It does +not require a separate observability plugin or Relay CLI setup. Hermes Agent +understands NeMo Relay plugin configurations. ## Framework and Agent Harness Integrations diff --git a/examples/switchyard/README.md b/examples/switchyard/README.md index b5fedb23e..883671e4f 100644 --- a/examples/switchyard/README.md +++ b/examples/switchyard/README.md @@ -75,24 +75,12 @@ A successful run ends with: real Switchyard E2E passed: ['provider/weak', 'provider/strong', 'provider/strong'] ``` -### Hermes and Ollama Trajectory - -`run-hermes-ollama-smoke.sh` runs a fixed multi-query trajectory through Hermes, Relay, Ollama, -and Switchyard. It requires Docker, Hermes, and the configured local Ollama models. The script -produces ATOF, ATIF, and OTEL artifacts and can leave Phoenix running with -`SWITCHYARD_KEEP_PHOENIX=1`. - -```bash -examples/switchyard/run-hermes-ollama-smoke.sh -``` - ## Configuration Files The directory includes the following configuration and support files: - `plugins.toml`: minimal plugin configuration example. - `real-e2e-plugins.toml` and `real-e2e-profiles.yaml`: deterministic fake-provider E2E. -- `hermes-ollama-plugins.toml` and `hermes-ollama-profiles.yaml`: local Ollama trajectory. - `fake_upstream.py`: deterministic provider used by the service E2E. - `otel-collector.yaml`: local OTEL artifact export configuration. diff --git a/examples/switchyard/hermes-ollama-plugins.toml b/examples/switchyard/hermes-ollama-plugins.toml deleted file mode 100644 index 4878a09ae..000000000 --- a/examples/switchyard/hermes-ollama-plugins.toml +++ /dev/null @@ -1,102 +0,0 @@ -# SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. -# SPDX-License-Identifier: Apache-2.0 - -version = 1 - -[[components]] -kind = "switchyard" -enabled = true - -[components.config] -mode = "enforce" -decision_api_url = "http://127.0.0.1:4000/v1/routing/decision" -decision_profile_id = "hermes-ollama-stage-router" -request_materialization = "full_body" -context_mode = "atof_required" -atof_endpoint_name = "switchyard" -decision_timeout_millis = 120000 -max_retries = 3 -recent_message_count = 8 - -[components.config.decision_header_env] -authorization = "SWITCHYARD_AUTHORIZATION" - -[components.config.default_targets] -openai_chat = "ollama-weak" -openai_responses = "ollama-responses" -anthropic_messages = "ollama-anthropic" - -[components.config.targets.ollama-weak] -model = "llama3.2:latest" -protocol = "openai_chat" -endpoint = "/v1/chat/completions" -base_url = "http://127.0.0.1:11434" - -[components.config.targets.ollama-strong] -model = "qwen3.6:35b" -protocol = "openai_chat" -endpoint = "/v1/chat/completions" -base_url = "http://127.0.0.1:11434" - -[components.config.targets.ollama-responses] -model = "llama3.2:latest" -protocol = "openai_responses" -endpoint = "/v1/responses" -base_url = "http://127.0.0.1:11434" - -[components.config.targets.ollama-anthropic] -model = "llama3.2:latest" -protocol = "anthropic_messages" -endpoint = "/v1/messages" -base_url = "http://127.0.0.1:11434" - -[[components]] -kind = "observability" -enabled = true - -[components.config] -version = 3 - -[components.config.atof] -enabled = true - -[[components.config.atof.sinks]] -type = "file" -mode = "append" -output_directory = "." -filename = "trajectory.atof.jsonl" - -[[components.config.atof.sinks]] -type = "stream" -name = "switchyard" -url = "http://127.0.0.1:4000/v1/atof/events" -transport = "http_post" -field_name_policy = "preserve" -timeout_millis = 1000 - -[components.config.atof.sinks.header_env] -authorization = "SWITCHYARD_AUTHORIZATION" - -[components.config.atif] -enabled = true -agent_name = "Hermes" -agent_version = "switchyard-stage-router-smoke" -model_name = "switchyard-stage-router" -output_directory = "." -filename_template = "trajectory-{session_id}.atif.json" - -[components.config.opentelemetry] -enabled = true - -[[components.config.opentelemetry.endpoints]] -type = "full" -transport = "http_binary" -endpoint = "http://127.0.0.1:4318/v1/traces" -service_name = "nemo-relay-switchyard-e2e" -service_namespace = "switchyard-integration" -instrumentation_scope = "nemo-relay-switchyard" -timeout_millis = 5000 - -[components.config.opentelemetry.endpoints.resource_attributes] -"deployment.environment.name" = "local-e2e" -"switchyard.profile.id" = "hermes-ollama-stage-router" diff --git a/examples/switchyard/hermes-ollama-profiles.yaml b/examples/switchyard/hermes-ollama-profiles.yaml deleted file mode 100644 index 751201851..000000000 --- a/examples/switchyard/hermes-ollama-profiles.yaml +++ /dev/null @@ -1,45 +0,0 @@ -# SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. -# SPDX-License-Identifier: Apache-2.0 - -targets: - ollama-weak: - model: llama3.2:latest - format: openai - base_url: http://127.0.0.1:11434/v1 - ollama-strong: - model: qwen3.6:35b - format: openai - base_url: http://127.0.0.1:11434/v1 - -profiles: - hermes-ollama-stage-router: - type: stage_router - capable: ollama-strong - efficient: ollama-weak - fallback_target_on_evict: ollama-strong - picker: efficient_first - # Force the classifier path for ordinary prompts so this smoke demonstrates - # request-complexity routing rather than relying on incidental tool signals. - confidence_threshold: 1.0 - # Each fixed trajectory transition is driven by the latest canonical tool - # result so older critical signals do not leak into the simple follow-up. - signal_recent_window: 1 - classifier: - # Keep classification small and deterministic; qwen3.6:35b remains the - # capable backend selected for the complex trajectory turn. - model: llama3.2:latest - api_key: ${OLLAMA_CLASSIFIER_API_KEY} - base_url: http://127.0.0.1:11434/v1 - timeout_secs: 120.0 - recent_turn_window: 1 - max_tokens: 128 - system_prompt: >- - Select exactly one model tier for the latest user request. Return only - JSON in the form {"tier":"efficient"} or {"tier":"capable"}. Choose efficient - for arithmetic, extraction, formatting, concise factual responses, or - other direct low-risk work. Choose capable for architecture, concurrency, - debugging, multi-step reasoning, subtle correctness constraints, or - requests requiring a defensible technical analysis. In the supplied - State line, severity at least 1.0 always means capable; tests_passed=true - with severity 0.0 means efficient. Classify the latest state even when earlier - conversation turns were more complex. diff --git a/examples/switchyard/run-hermes-ollama-smoke.sh b/examples/switchyard/run-hermes-ollama-smoke.sh deleted file mode 100755 index f390eb75e..000000000 --- a/examples/switchyard/run-hermes-ollama-smoke.sh +++ /dev/null @@ -1,409 +0,0 @@ -#!/usr/bin/env bash -# SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. -# SPDX-License-Identifier: Apache-2.0 - -set -euo pipefail - -relay_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" -source "$relay_root/examples/switchyard/e2e-common.sh" -switchyard_root="${SWITCHYARD_ROOT:-$(cd "$relay_root/.." && pwd)/Switchyard-topic-nemo-relay-integration}" -switchyard_expected_commit="${SWITCHYARD_EXPECTED_COMMIT:-8f9db9a6a47f848cdff1d262276ba25a8ae9cbc8}" -run_id="$(date -u +%Y%m%dT%H%M%SZ)-$$" -artifact_dir="${SWITCHYARD_TRAJECTORY_DIR:-$relay_root/artifacts/hermes-switchyard-$run_id}" -token="$(e2e_random_token)" -docker_network="switchyard-e2e-$run_id" -phoenix_container="switchyard-phoenix-$run_id" -collector_container="switchyard-otel-$run_id" -phoenix_port="${SWITCHYARD_PHOENIX_PORT:-6006}" -keep_phoenix="${SWITCHYARD_KEEP_PHOENIX:-0}" -collector_running=0 -phoenix_running=0 -network_created=0 - -mkdir -p "$artifact_dir/phoenix" -[[ -d "$switchyard_root" ]] || { echo "Switchyard worktree not found: $switchyard_root" >&2; exit 1; } -e2e_verify_switchyard_checkout "$switchyard_root" "$switchyard_expected_commit" >"$artifact_dir/switchyard-revision.txt" - -cleanup() { - local status=$? - e2e_stop_processes - if [[ $collector_running -eq 1 ]]; then - docker rm -f "$collector_container" >/dev/null 2>&1 || true - fi - if [[ $phoenix_running -eq 1 && ( $status -ne 0 || "$keep_phoenix" != "1" ) ]]; then - docker rm -f "$phoenix_container" >/dev/null 2>&1 || true - phoenix_running=0 - fi - if [[ $network_created -eq 1 && $phoenix_running -eq 0 ]]; then - docker network rm "$docker_network" >/dev/null 2>&1 || true - fi - if [[ $status -ne 0 ]]; then - echo "Hermes/StageRouter smoke failed; artifacts preserved in $artifact_dir" >&2 - e2e_tail_logs "$artifact_dir" - fi -} -trap cleanup EXIT - -for dependency in cargo curl docker hermes jq python3 tar; do - command -v "$dependency" >/dev/null || { - echo "missing required command: $dependency" >&2 - exit 1 - } -done - -docker info >/dev/null -for model in llama3.2:latest qwen3.6:35b; do - curl --fail --silent http://127.0.0.1:11434/api/tags \ - | jq -e --arg model "$model" '.models[] | select(.name == $model)' >/dev/null || { - echo "required Ollama model is not installed: $model" >&2 - exit 1 - } -done - -docker network create "$docker_network" >/dev/null -network_created=1 -docker run --detach --rm \ - --name "$phoenix_container" \ - --network "$docker_network" \ - --network-alias phoenix \ - --publish "127.0.0.1:$phoenix_port:6006" \ - --env PHOENIX_WORKING_DIR=/mnt/data \ - --volume "$artifact_dir/phoenix:/mnt/data" \ - arizephoenix/phoenix:13.22 >"$artifact_dir/phoenix.container-id" -phoenix_running=1 -e2e_wait_for "http://127.0.0.1:$phoenix_port/" 240 0.5 - -docker run --detach --rm \ - --name "$collector_container" \ - --network "$docker_network" \ - --publish 127.0.0.1:4318:4318 \ - --volume "$relay_root/examples/switchyard/otel-collector.yaml:/etc/otelcol-contrib/config.yaml:ro" \ - --volume "$artifact_dir:/artifacts" \ - otel/opentelemetry-collector-contrib:0.135.0 \ - --config=/etc/otelcol-contrib/config.yaml >"$artifact_dir/collector.container-id" -collector_running=1 - -( - cd "$switchyard_root" - OLLAMA_CLASSIFIER_API_KEY="$token" \ - SWITCHYARD_ATOF_BEARER_TOKEN="$token" \ - cargo run -p switchyard-server -- \ - --config "$relay_root/examples/switchyard/hermes-ollama-profiles.yaml" --port 4000 -) >"$artifact_dir/switchyard.log" 2>&1 & -e2e_add_pid "$!" -e2e_wait_for http://127.0.0.1:4000/health 240 0.5 - -run_query() { - local sequence="$1" - local label="$2" - local query="$3" - local resume_id="${4:-}" - local -a resume_args=() - local before_lines=0 - local after_lines - local atif_path - if [[ -n "$resume_id" ]]; then - resume_args=(--resume "$resume_id") - fi - if [[ -f "$artifact_dir/trajectory.atof.jsonl" ]]; then - before_lines="$(wc -l < "$artifact_dir/trajectory.atof.jsonl" | tr -d ' ')" - fi - ( - cd "$artifact_dir" - HERMES_HOME="$artifact_dir/hermes" \ - OPENAI_API_KEY=ollama \ - SWITCHYARD_AUTHORIZATION="Bearer $token" \ - cargo run --manifest-path "$relay_root/Cargo.toml" -p nemo-relay-cli \ - --features switchyard -- \ - run --agent hermes \ - --plugin-config-path "$relay_root/examples/switchyard/hermes-ollama-plugins.toml" \ - -- chat --provider custom --model llama3.2:latest \ - --query "$query" ${resume_args[@]+"${resume_args[@]}"} \ - --toolsets terminal --quiet --max-turns 2 --ignore-rules - ) >"$artifact_dir/query-$sequence-$label.log" 2>&1 - after_lines="$(wc -l < "$artifact_dir/trajectory.atof.jsonl" | tr -d ' ')" - printf '%s\t%s\t%s\t%s\n' "$sequence" "$label" "$before_lines" "$after_lines" \ - >> "$artifact_dir/query-event-ranges.tsv" - # Each query is a separate Relay process. Replay its persisted ATOF segment - # as a completion barrier before the next process starts; ingestion is - # idempotent, so events already delivered by the best-effort live exporter - # are reported as duplicates rather than applied twice. - sed -n "$((before_lines + 1)),${after_lines}p" "$artifact_dir/trajectory.atof.jsonl" \ - > "$artifact_dir/trajectory-$sequence-$label.atof.jsonl" - curl --fail --silent http://127.0.0.1:4000/v1/atof/events \ - -H "authorization: Bearer $token" \ - -H 'content-type: application/x-ndjson' \ - --data-binary "@$artifact_dir/trajectory-$sequence-$label.atof.jsonl" \ - > "$artifact_dir/trajectory-$sequence-$label.atof-ingest.json" - jq -e '(.batch.ingested_events + .batch.duplicate_events) >= 1' \ - "$artifact_dir/trajectory-$sequence-$label.atof-ingest.json" >/dev/null - atif_path="$(find "$artifact_dir" -maxdepth 1 -name 'trajectory-*.atif.json' -print \ - | grep -E '/trajectory-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}\.atif\.json$' \ - | head -1 || true)" - if [[ -z "$atif_path" ]]; then - echo "ATIF exporter did not produce a trajectory for query $sequence" >&2 - exit 1 - fi - mv "$atif_path" "$artifact_dir/trajectory-$sequence-$label.atif.json" -} - -emit_stage_router_signal() { - local sequence="$1" - local label="$2" - local output="$3" - local path="$artifact_dir/trajectory-signal-$sequence-$label.atof.jsonl" - python3 - "$path" "$session_id" "$label" "$output" <<'PY' -import datetime -import json -import pathlib -import sys -import uuid - -path = pathlib.Path(sys.argv[1]) -session_id, label, output = sys.argv[2:] -event_uuid = str(uuid.uuid4()) -base = { - "atof_version": "0.1", - "kind": "scope", - "uuid": event_uuid, - "timestamp": datetime.datetime.now(datetime.timezone.utc).isoformat(), - "name": "trajectory_fixture", - "category": "tool", - "category_profile": {"tool_call_id": f"fixture-{label}"}, - "metadata": { - "session_id": session_id, - "trajectory_fixture": True, - "trajectory_fixture_label": label, - }, -} -events = [ - {**base, "scope_category": "start", "data": {"label": label}}, - {**base, "scope_category": "end", "data": {"output": output}}, -] -path.write_text("".join(json.dumps(event, separators=(",", ":")) + "\n" for event in events)) -PY - cat "$path" >> "$artifact_dir/trajectory.atof.jsonl" - curl --fail --silent http://127.0.0.1:4000/v1/atof/events \ - -H "authorization: Bearer $token" \ - -H 'content-type: application/x-ndjson' \ - --data-binary "@$path" > "$artifact_dir/trajectory-signal-$sequence-$label.atof-ingest.json" - jq -e '.batch.ingested_events == 2' \ - "$artifact_dir/trajectory-signal-$sequence-$label.atof-ingest.json" >/dev/null -} - -simple_query='Return exactly the integer result of 17 + 25, with no explanation.' -complex_query='Do not call tools. Act as a principal concurrency engineer. Analyze a bounded lock-free MPMC queue that uses compare-and-swap on head and tail but no generation counters. Give a concrete ABA failure interleaving, identify the required C++ memory order on each publication and consumption edge, and propose the smallest defensible correction. Be precise about the linearization points.' -followup_query='Ignore the earlier technical topic. Reply with exactly SIMPLE_DONE and nothing else.' - -run_query 01 simple "$simple_query" -session_id="$(jq -r 'select(.name == "switchyard.routing.requested") | .data.session_id' "$artifact_dir/trajectory.atof.jsonl" | head -1)" -if [[ -z "$session_id" || "$session_id" == "null" ]]; then - echo "could not recover the Hermes session ID from the first routing mark" >&2 - exit 1 -fi -emit_stage_router_signal 02 complex \ - 'CUDA out of memory while analyzing the concurrent queue; critical failure requires careful recovery and a capable model.' -run_query 02 complex "$complex_query" "$session_id" -emit_stage_router_signal 03 simple-followup \ - 'All tests passed. The next request is a direct low-risk formatting response: return exactly SIMPLE_DONE.' -run_query 03 simple-followup "$followup_query" "$session_id" - -# Give the asynchronous OTLP exporter a short flush window, then stop the -# collector cleanly so its file exporter closes the shareable OTLP JSON file. -sleep 2 -docker stop --time 10 "$collector_container" >/dev/null -collector_running=0 - -python3 - "$artifact_dir" "$session_id" "$phoenix_port" "$simple_query" "$complex_query" "$followup_query" <<'PY' -import json -import pathlib -import sys - -root = pathlib.Path(sys.argv[1]) -session_id = sys.argv[2] -phoenix_port = sys.argv[3] -queries = sys.argv[4:] -atof_path = root / "trajectory.atof.jsonl" -events = [json.loads(line) for line in atof_path.read_text().splitlines() if line.strip()] -marks = [event for event in events if event.get("name", "").startswith("switchyard.routing.")] -decisions = [event for event in marks if event.get("name") == "switchyard.routing.decision"] - -expected_models = ["llama3.2:latest", "qwen3.6:35b", "llama3.2:latest"] -event_ranges = [] -for line in (root / "query-event-ranges.tsv").read_text().splitlines(): - sequence, label, start, end = line.split("\t") - event_ranges.append((sequence, label, int(start), int(end))) -representative_decisions = [] -for sequence, label, start, end in event_ranges: - segment = events[start:end] - decision = next( - (event for event in segment if event.get("name") == "switchyard.routing.decision"), - None, - ) - if decision is None: - raise SystemExit(f"query {sequence} ({label}) produced no successful routing decision") - representative_decisions.append(decision) -actual_models = [event.get("data", {}).get("selected_model") for event in representative_decisions] -if actual_models != expected_models: - raise SystemExit(f"unexpected StageRouter route sequence: {actual_models}; expected {expected_models}") - -required_mark_names = {"switchyard.routing.requested", "switchyard.routing.decision"} -if not required_mark_names.issubset({event.get("name") for event in marks}): - raise SystemExit("routing requested/decision marks were not both emitted") -for event in marks: - name = event.get("name") - if event.get("category") != "custom": - raise SystemExit(f"{name} did not use category=custom") - if event.get("category_profile", {}).get("subtype") != name: - raise SystemExit(f"{name} category_profile.subtype was not canonical") - schema = event.get("data_schema", {}) - if schema != {"name": "switchyard.routing_mark", "version": "1"}: - raise SystemExit(f"{name} had unexpected data_schema: {schema}") - metadata = event.get("metadata", {}) - if metadata.get("session_id") != session_id: - raise SystemExit(f"{name} did not mirror session identity") -for event in decisions: - data = event["data"] - for key in ("decision_id", "router", "routing_attempt", "backend_id", "selected_tier", "selected_model", "latency_ms", "rollout_mode"): - if key not in data or data[key] is None: - raise SystemExit(f"decision mark missing {key}: {data}") - -atif_paths = sorted(root.glob("trajectory-*.atif.json")) -if len(atif_paths) != 3: - raise SystemExit(f"expected three ATIF trajectories, found {len(atif_paths)}") -for path in atif_paths: - payload = json.loads(path.read_text()) - if not payload.get("steps"): - raise SystemExit(f"ATIF trajectory has no steps: {path.name}") - -otel_path = root / "trajectory.otel.json" -if not otel_path.exists() or not otel_path.read_text().strip(): - raise SystemExit("OTLP file exporter did not produce trajectory.otel.json") -otel_batches = [json.loads(line) for line in otel_path.read_text().splitlines() if line.strip()] - -summary = { - "session_id": session_id, - "queries": [{ - "sequence": index + 1, - "input": query, - "selected_model": actual_models[index], - "selected_tier": representative_decisions[index]["data"].get("selected_tier"), - "reason_code": representative_decisions[index]["data"].get("reason_code"), - "reason_summary": representative_decisions[index]["data"].get("reason_summary"), - } for index, query in enumerate(queries)], - "expected_route_sequence": expected_models, - "actual_route_sequence": actual_models, - "routing_basis": [ - "cold StageRouter efficient default", - "canonical ATOF critical-error tool result (capable override)", - "canonical ATOF clean-tests tool result (efficient classifier decision)", - ], - "atof": {"file": atof_path.name, "event_count": len(events), "routing_mark_count": len(marks)}, - "atif": {"files": [path.name for path in atif_paths], "trajectory_count": len(atif_paths)}, - "otel": {"file": otel_path.name, "export_batch_count": len(otel_batches)}, - "phoenix_url": f"http://127.0.0.1:{phoenix_port}", -} -(root / "trajectory-summary.json").write_text(json.dumps(summary, indent=2) + "\n") -readme = f"""# Hermes / Ollama / Switchyard StageRouter trajectory - -This bundle captures one fixed three-query Hermes session routed through NeMo -Relay and the Switchyard Decision API. The verified representative route is: - -1. `llama3.2:latest` (efficient) — cold StageRouter default -2. `qwen3.6:35b` (capable) — critical-signal StageRouter override -3. `llama3.2:latest` (efficient) — clean-state classifier decision - -Session ID: `{session_id}` - -## Important fixture note - -The `CUDA out of memory` text in `trajectory-signal-02-complex.atof.jsonl` is an -intentional, synthetic ATOF tool-result fixture. The machine did not run out of -memory. Fixture events carry `metadata.trajectory_fixture = true` and a fixture -label so they cannot be confused with organic Hermes events. - -The fixtures are necessary for this demonstration because the current -Switchyard StageRouter Decision API classifies from its accumulated ATOF snapshot, -not directly from `current_request.body`. The critical fixture exercises the -real capable override. The clean-tests fixture removes the prior critical signal -from the one-result window and exercises the efficient classifier path. - -## File map - -| Files | Contents | Test coverage | -| --- | --- | --- | -| `trajectory-summary.json` | Machine-readable queries, selected models, reasons, counts, and Phoenix URL | Confirms expected and actual representative routes match | -| `trajectory.atof.jsonl` | Complete Relay ATOF stream for all three queries and the labeled fixtures | Identity propagation, lifecycle events, routing marks, and accumulator input | -| `trajectory-01-simple.atof.jsonl` | ATOF emitted by the first Hermes invocation | Cold-start efficient default | -| `trajectory-02-complex.atof.jsonl` | ATOF emitted by the complex Hermes invocation | Dispatch through the selected capable backend | -| `trajectory-03-simple-followup.atof.jsonl` | ATOF emitted by the final Hermes invocation | Return to the efficient backend | -| `trajectory-signal-*.atof.jsonl` | Canonical, labeled tool start/end fixtures | Capable critical-error override and efficient clean-state classification | -| `*.atof-ingest.json` | Switchyard ingestion reports for query segments and fixtures | Successful or idempotent ATOF accumulation | -| `trajectory-01-simple.atif.json` | ATIF representation of query 1 | Efficient-model trajectory structure | -| `trajectory-02-complex.atif.json` | ATIF representation of query 2 | Capable-model trajectory structure | -| `trajectory-03-simple-followup.atif.json` | ATIF representation of query 3 | Efficient follow-up trajectory structure | -| `trajectory.otel.json` | OTLP JSON batches written by the OpenTelemetry Collector | Relay spans exported to the collector and forwarded to Phoenix | -| `query-*.log` | Hermes/Relay stdout and stderr for each invocation | Human-readable harness responses and execution diagnostics | -| `query-event-ranges.tsv` | Query label and ATOF line-count boundaries | Separates representative user-query decisions from extra Hermes calls | - -## Routing-mark assertions - -The smoke validates every `switchyard.routing.*` mark in the cumulative ATOF -stream. Each mark must have: - -- `category: "custom"` -- `category_profile.subtype` equal to the mark name -- `data_schema.name: "switchyard.routing_mark"` -- `data_schema.version: "1"` -- the expected session identity in metadata - -Decision marks must also include a decision ID, router, attempt, backend, tier, -model, latency, and rollout mode. This run produced {len(marks)} routing marks -across {len(events)} total ATOF events. - -## Phoenix and OTLP - -During the smoke, Relay sends OTLP/HTTP to a local OpenTelemetry Collector. The -collector writes `trajectory.otel.json` and forwards the same spans to Phoenix. -The run produced {len(otel_batches)} OTLP export batches. When the smoke is run -with `SWITCHYARD_KEEP_PHOENIX=1`, open the `phoenix_url` from -`trajectory-summary.json` before stopping the printed Phoenix container. - -## Reproduce - -Install both `llama3.2:latest` and `qwen3.6:35b` in Ollama, ensure the cumulative -Switchyard checkout is available, then run: - -```bash -SWITCHYARD_KEEP_PHOENIX=1 examples/switchyard/run-hermes-ollama-smoke.sh -``` - -The script validates the route, mark shape, ATIF contents, and OTLP output before -creating this bundle. -""" -(root / "TRAJECTORY_README.md").write_text(readme) -print(json.dumps(summary, indent=2)) -PY - -( - cd "$artifact_dir" - tar -czf trajectory-bundle.tar.gz \ - TRAJECTORY_README.md \ - trajectory-summary.json \ - trajectory.atof.jsonl \ - trajectory.otel.json \ - trajectory-*.atif.json \ - trajectory-*.atof.jsonl \ - trajectory-*.atof-ingest.json \ - query-*.log \ - query-event-ranges.tsv -) - -echo "Hermes/Ollama StageRouter trajectory passed: llama3.2 -> qwen3.6:35b -> llama3.2" -echo "Artifacts: $artifact_dir" -echo "Bundle: $artifact_dir/trajectory-bundle.tar.gz" -if [[ "$keep_phoenix" == "1" ]]; then - echo "Phoenix: http://127.0.0.1:$phoenix_port (container $phoenix_container left running)" -else - echo "Set SWITCHYARD_KEEP_PHOENIX=1 to leave Phoenix running after the smoke." -fi diff --git a/examples/switchyard/run-real-e2e.sh b/examples/switchyard/run-real-e2e.sh index 1f6810871..88a28d358 100755 --- a/examples/switchyard/run-real-e2e.sh +++ b/examples/switchyard/run-real-e2e.sh @@ -73,11 +73,11 @@ request() { request cold-request false >"$work_dir/cold.json" for payload in \ - '{"hook_event_name":"on_session_start","session_id":"e2e-session"}' \ - '{"hook_event_name":"pre_tool_call","session_id":"e2e-session","tool_name":"Bash","tool_input":{"command":"test"},"extra":{"task_id":"task-1","tool_call_id":"call-1"}}' \ - '{"hook_event_name":"post_tool_call","session_id":"e2e-session","tool_name":"Bash","tool_input":{"command":"test"},"tool_response":{"output":"CUDA out of memory"},"extra":{"task_id":"task-1","tool_call_id":"call-1"}}' + '{"hook_event_name":"SessionStart","session_id":"e2e-session"}' \ + '{"hook_event_name":"PreToolUse","session_id":"e2e-session","tool_name":"Bash","tool_input":{"command":"test"},"tool_use_id":"call-1"}' \ + '{"hook_event_name":"PostToolUse","session_id":"e2e-session","tool_name":"Bash","tool_input":{"command":"test"},"tool_response":{"output":"CUDA out of memory"},"tool_use_id":"call-1"}' do - curl --fail --silent http://127.0.0.1:4041/hooks/hermes \ + curl --fail --silent http://127.0.0.1:4041/hooks/codex \ -H 'content-type: application/json' --data-binary "$payload" >/dev/null done diff --git a/integrations/coding-agents/README.md b/integrations/coding-agents/README.md index cab978b4c..de016a0ce 100644 --- a/integrations/coding-agents/README.md +++ b/integrations/coding-agents/README.md @@ -3,333 +3,36 @@ SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All SPDX-License-Identifier: Apache-2.0 --> -# NeMo Relay Coding-Agent Observability Integrations +# Coding Agent Integrations -This directory contains hook integration bundles for coding agents that should -be observed by `nemo-relay`. +NeMo Relay CLI supports Claude Code and Codex through generated marketplace +plugins, lifecycle hooks, provider routing, and a shared local gateway. -The gateway combines two observability paths: - -- Agent lifecycle hooks for sessions, prompts, subagents, tool calls, - compaction, responses, and stop events. -- A passthrough LLM gateway for OpenAI-compatible and Anthropic-compatible - provider traffic. - -Hook integrations preserve each coding agent's canonical hook payload. They do -not wrap the payload in a shared NeMo Relay envelope. Gateway-specific settings -travel through the transparent wrapper, hook command arguments, HTTP headers, -environment variables, or shared TOML config. - -## Packages - -Each host uses a slightly different integration surface: - -- `claude-code/` is a Claude Code plugin package. The - `nemo-relay install claude-code` command installs a native MCP lifecycle - client and hook entries targeting `POST /hooks/claude-code` through - `nemo-relay` on `PATH`. -- `codex/` is a Codex plugin package. `nemo-relay install codex` creates the - marketplace, installs the plugin, enables `features.hooks = true`, and - configures a local `nemo-relay-openai` provider alias. Codex plugin delivery - uses required native `nemo-relay mcp` lifecycle clients. Claude Code starts - the same lifecycle client automatically from its plugin. Clients from either - host share one Rust gateway, subject to the Windows Job Object lifetime - caveat below, with no wrapper, login item, launchd agent, systemd user - service, scheduled task, or persistent supervisor. -- Hermes does not require a static marketplace bundle. The - `nemo-relay install hermes` command adds a native MCP lifecycle client, - canonical hooks, and exact per-event trust to the user-owned Hermes config as - one transaction. - -## Transparent Setup - -Build or install the gateway binary so `nemo-relay` is on `PATH`. - -Prefer the wrapper. It starts a gateway on a dynamic `127.0.0.1` port, injects -temporary hook and gateway configuration, runs the agent, and shuts the gateway -down when the agent exits. - -```bash -nemo-relay run -- claude -nemo-relay run -- codex -nemo-relay run -- hermes -``` - -When a wrapper hides the agent command name, configure that wrapper under -`[agents.].command` and select it with -`--agent claude|codex|hermes`. Use `--dry-run --print` to inspect generated -config without launching. - -Use `nemo-relay doctor` to inspect environment, config, agent commands, hook -readiness, observability outputs, and shell completions. Scope the report to one -agent when troubleshooting launch readiness: - -```bash -nemo-relay doctor -nemo-relay doctor codex -nemo-relay doctor hermes --json -``` - -The command is read-only: it reports missing ATIF directories, hook files, and -agent commands instead of creating or patching them. - -## Persistent Integration Installation - -The `nemo-relay` CLI installs the Claude Code and Codex plugins and manages the -Hermes user integration. The CLI must already be available on `$PATH` or -`%PATH%`; you do not need a separate npm installer, release bundle, or -plugin-local Relay binary. - -Persistent installation and transparent launch require Claude Code 2.1.121 or -newer, `codex-cli` 0.143.0 or newer, or Hermes Agent 0.18.2 or newer for the -selected agent. - -Each plugin MCP entry—and the equivalent Hermes `mcp_servers` entry—starts -`nemo-relay mcp`, a lightweight client that starts or reuses a native -`nemo-relay --bind 127.0.0.1:47632` sidecar. Relay detaches the sidecar when -host policy permits. A restrictive Windows Job Object can limit the sidecar to -the host job. If nested assignment cannot provide the required process-tree -cleanup guarantee, bootstrap stops and explains the conflict. - -The MCP process acquires the gateway before reading protocol frames and returns -its initialization response only after it verifies Relay identity, version, -and bootstrap-protocol readiness. Concurrent Codex, Claude Code, and -Hermes processes share the gateway and heartbeat it while their MCP stdio -connections remain open; the gateway exits after the final client's idle -timeout. Overlapping MCP clients coordinate one restart for the endpoint, even -when their heartbeats arrive at different times. -Codex requires MCP initialization before the captured turn. Claude Code marks -Relay MCP as `alwaysLoad`, so it also waits for the connection before session -startup. Hermes starts MCP discovery asynchronously, so an early -generation-fenced command hook waits for the MCP-owned gateway. Installed MCP -entries and hook commands carry both their generation-file path and the -immutable identity expected there, so cached host configuration cannot adopt a -replacement installation at the same path. The -MCP client advertises no tools. - -MCP bootstrap is host-neutral: all three generated integrations use -`nemo-relay mcp`. Agent identity appears only in lifecycle hook commands, where -Relay needs it to translate each host's canonical payload. The old -`mcp --agent ` form no longer parses. Refresh a fenced installation with -`nemo-relay install --force`; if its generation marker is missing, Relay -refuses the upgrade and prints the manual cleanup steps. - -Persistent mode loads system and user Relay configuration only and starts the -sidecar from the user configuration directory. This keeps relative exporter -paths stable across projects. Codex's generated MCP manifest forwards -approved provider, Relay, OpenTelemetry, AWS, proxy, certificate, and -config-referenced credential environment names without storing their values; -Claude Code supplies its normal MCP process environment. Use transparent -`nemo-relay run` for project-specific configuration. The managed sidecar -injects a forwarded provider key only for a request with provider authorization -or Relay's private per-user client proof. Codex receives that derived proof in -its managed provider headers; the installer writes that config privately and -Relay consumes the proof before middleware, telemetry, or upstream forwarding. -Claude Code and Hermes send their normal provider authorization, so an -unrelated loopback caller cannot spend forwarded keys. - -Install the persistent integrations with: +The source manifests in this directory are development inputs. End users should +install the generated integrations through the CLI: ```bash nemo-relay install claude-code nemo-relay install codex -nemo-relay install hermes -nemo-relay install all -``` - -For Claude Code and Codex, `nemo-relay install` writes local marketplace files, -registers the selected host plugin, and performs the required provider and hook -setup. For Hermes, `install` is the only command that updates Relay-owned user -MCP, hook, trust, and generation state; interactive `config hermes` manages -only the transparent wrapper. Use -`nemo-relay uninstall ` to roll back and -`nemo-relay doctor --plugin ` to check an installed integration. - -If you are using Codex, add this repository as a marketplace for source/dev -discovery: - -```bash -codex plugin marketplace add NVIDIA/NeMo-Relay -codex plugin add nemo-relay-plugin@nemo-relay -``` - -That path relies on `nemo-relay` being available on `PATH`. Source plugin hooks -use `nemo-relay hook-forward codex --forward-only`: they post to the gateway -started by the required MCP entry but cannot launch or recover Relay without an -installer-owned generation fence. Before posting, they authenticate the Relay -identity and verify that its user-level configuration matches. The proof and -payload use one TCP connection, preventing a replacement listener from -receiving the payload after verification. - -Use the source marketplace path for discovery or manifest validation. Use -`nemo-relay install codex` for complete provider routing, environment -forwarding, and verified plugin-hook trust. - -Remove the source-installed Codex plugin before you use the generated install. -If both remain active and trusted, they can forward the same lifecycle payload. - -Claude Code users can add this repository as a marketplace the same way: - -```bash -claude plugin marketplace add NVIDIA/NeMo-Relay \ - --sparse .claude-plugin integrations/coding-agents/claude-code -claude plugin install nemo-relay-plugin@nemo-relay --scope user -``` - -That path reads `.claude-plugin/marketplace.json` from the repository. Source -plugin hooks use `nemo-relay hook-forward claude --forward-only`: they post to -the gateway started by the `alwaysLoad` MCP entry but cannot launch or recover -Relay without an installer-owned generation fence. They authenticate that -gateway on the same connection used to send lifecycle data. Use -`nemo-relay install claude-code` for the complete provider-routing setup, and -remove the source-installed plugin first to avoid duplicate lifecycle events. - -Hermes persistent installation is user-level: - -```bash -nemo-relay install hermes -``` - -It writes the MCP server and trusted hooks to `$HERMES_HOME/config.yaml` or -`~/.hermes/config.yaml`. Transparent Hermes runs leave that file untouched and -export the dynamic `NEMO_RELAY_GATEWAY_URL` through a process-private -`HERMES_HOME` overlay with no fixed MCP entry. - -Shared TOML config uses the XDG user file (or an explicit file) and then the -system file, with the system file at higher precedence. The system path is -`/etc/nemo-relay/config.toml` on Unix or -`%ProgramData%\nemo-relay\config.toml` on Windows. - -```toml -[agents.codex] -command = "codex" - -[agents.hermes] -command = "hermes" -``` - -Observability exporters are configured in `plugins.toml`. Run -`nemo-relay plugins edit` to create the XDG user file, or write it directly: - -```toml -version = 1 - -[[components]] -kind = "observability" -enabled = true - -[components.config] -version = 3 - -[components.config.atif] -enabled = true -output_directory = ".nemo-relay/atif" - -[components.config.opentelemetry] -enabled = true - -[[components.config.opentelemetry.endpoints]] -type = "openinference" -endpoint = "http://127.0.0.1:4318/v1/traces" ``` -During setup or launch, Relay fails closed on invalid shared TOML, malformed -plugin config, unsupported exporter settings, or unavailable exporter features. -The wrapper does not start the coding agent with a configuration that it cannot -parse, validate, or activate. After the gateway and agent are running, -exporter delivery failures follow the observability plugin policy: application -work continues while the failing ATOF, ATIF, or typed OpenTelemetry destination -records, logs, or reports the failure. - -## Hook Forwarding - -Transparent Claude Code and Codex hooks call -`nemo-relay hook-forward ` with the canonical hook payload on standard -input. The wrapper-owned command embeds the ephemeral per-run gateway URL and -is marked as transparent so it never starts or recovers the fixed gateway. - -Persistent Claude Code, Codex, and Hermes hooks call -`nemo-relay hook-forward ` with the fixed gateway and an -installer-owned generation fence. They wait for and authenticate the -MCP-owned gateway, then send the payload once. They never start or recover the -gateway. Transparent Hermes hooks instead embed the wrapper's dynamic gateway -URL. - -For Codex, the installed plugin file is the sole persistent Relay hook source; -installation does not add Relay groups to `~/.codex/hooks.json`. - -Generated hooks select an explicit failure policy by event. `PreToolUse`, -`PermissionRequest`, and Hermes `pre_tool_call` hooks use `--fail-closed`, so -Relay startup, authentication, delivery, and response failures block the -permission-bearing operation. Lifecycle and after-the-fact hooks use -`--fail-open`, so observability outages do not block unrelated agent work. - -After upgrading, rerun `nemo-relay install --force` to replace legacy -generated hooks that did not carry an explicit policy. Manually authored -`hook-forward` commands still fail open by default; set -`NEMO_RELAY_FAIL_CLOSED=1` or add `--fail-closed` when they enforce policy. - -These `hook-forward` options control delivery and metadata: - -- `--gateway-url ` selects the Relay gateway that receives the payload. -- `--forward-only` allows a source plugin or custom automation to use an - existing compatible gateway without an installer-owned generation fence. It - verifies the gateway but never launches or recovers Relay. Generated - installed hooks use a private generation fence instead. -- `--session-metadata ''` adds structured metadata to the agent begin - event. For example, `--session-metadata '{"user_id":"alice"}'` exposes the - string as `user.id` on OTLP trace roots. -- `--profile ` records a configuration profile in session metadata. -- `--gateway-mode hook-only|passthrough|required` records the expected gateway - behavior in session metadata. -- `--fail-open` allows the coding agent to continue after a delivery failure, - even when `NEMO_RELAY_FAIL_CLOSED=1` is set. -- `--fail-closed` returns a failure when delivery fails or Relay rejects the - hook instead of allowing the coding agent to continue. +Use `nemo-relay doctor --plugin ` to verify an installation and +`nemo-relay uninstall ` to remove Relay-owned host state. The transparent +wrappers remain available as `nemo-relay claude` and `nemo-relay codex`. -## LLM Gateway +Host-specific source notes live in: -Complete LLM lifecycle observability requires model traffic to pass through the -gateway. Hook-only mode observes agent, subagent, and tool lifecycle, but it -cannot observe provider request and response lifecycle when the coding agent -sends model traffic directly to an upstream provider or remote service. +- [`claude-code/`](claude-code/README.md) +- [`codex/`](codex/README.md) -The gateway exposes these passthrough routes: +Hermes Agent does not use a NeMo Relay CLI integration. NeMo Relay is built +into Hermes Agent, with no separate observability plugin or Relay CLI setup +required. Hermes Agent understands NeMo Relay plugin configurations. -- `POST /v1/responses` -- `POST /v1/chat/completions` -- `POST /v1/messages` -- `POST /v1/messages/count_tokens` -- `GET /v1/models` - -Transparent runs configure provider routing automatically where the launched -agent supports local routing. Standalone gateway mode requires you to point the -agent's provider base URL at the gateway manually. - -## Verify Export - -Complete a coding-agent turn or session that uses one tool. Then confirm that -ATIF was written: +For repository validation, use the canonical Rust and documentation recipes: ```bash -ls .nemo-relay/atif +just test-rust +just docs +just docs-linkcheck ``` - -The snapshot boundary depends on the host. Claude Code writes ATIF on -`SessionEnd`. Codex writes a cumulative snapshot on each `Stop` because its -plugin schema does not expose `SessionEnd`. Hermes writes or updates the -snapshot on `on_session_end`, `on_session_finalize`, or `on_session_reset`. - -Run the opt-in host E2E targets when the corresponding CLI is installed. These -targets are intentionally outside `test-rust` and mandatory CI: - -```bash -just test-claude-plugin-e2e -just test-codex-plugin-e2e -just test-hermes-mcp-e2e -``` - -Each target uses an isolated home directory and local mock provider. The Claude -and Hermes targets each run 10 cold sessions plus two concurrent sessions and -verify MCP connection, hook delivery, provider routing, session isolation, -balanced ATOF output, and final port release. diff --git a/integrations/coding-agents/claude-code/README.md b/integrations/coding-agents/claude-code/README.md index 516e32b10..816274c60 100644 --- a/integrations/coding-agents/claude-code/README.md +++ b/integrations/coding-agents/claude-code/README.md @@ -183,7 +183,7 @@ nemo-relay install claude-code installs `nemo-relay-plugin` at user scope, and enables Claude Code provider routing through NeMo Relay. Its plugin MCP process immediately starts or reuses the shared native gateway on `127.0.0.1:47632` and heartbeats it while MCP stdio -remains open. Codex, Claude Code, and configured Hermes MCP clients can share +remains open. Codex and Claude Code MCP clients can share that gateway. The generated MCP entry sets `alwaysLoad: true`, so Claude Code waits diff --git a/integrations/coding-agents/codex/README.md b/integrations/coding-agents/codex/README.md index cfedb0006..a8e7f50f0 100644 --- a/integrations/coding-agents/codex/README.md +++ b/integrations/coding-agents/codex/README.md @@ -73,8 +73,8 @@ runtime files and a copied managed Python environment—to 100,000 filesystem entries, 512 MiB, and a maximum directory traversal depth of 128. If startup reports an activation snapshot budget error, remove unrelated files from the manifest or load-target directory, flatten deeply nested directories, or reduce -the managed Python environment before retrying. Concurrent Codex, -Claude Code, and configured Hermes processes can share the gateway and +the managed Python environment before retrying. Concurrent Codex and Claude +Code processes can share the gateway and heartbeat it every 30 seconds. The sidecar remains available for 300 idle seconds after the final client closes. If it dies while MCP remains open, overlapping MCP clients coordinate one restart for the endpoint. Persistent diff --git a/justfile b/justfile index 18d33a74e..c1f6ec630 100644 --- a/justfile +++ b/justfile @@ -1169,10 +1169,6 @@ test-codex-plugin-e2e: test-claude-plugin-e2e: ./scripts/test-claude-plugin-e2e.sh -# Opt-in: requires a supported Hermes Agent installation and is intentionally outside test-rust/CI. -test-hermes-mcp-e2e: - ./scripts/test-hermes-mcp-e2e.sh - # --set [output_dir=] [ci=true|false] test-rust: #!/usr/bin/env bash diff --git a/scripts/README.md b/scripts/README.md index c8fce82ab..783df8f92 100644 --- a/scripts/README.md +++ b/scripts/README.md @@ -24,7 +24,6 @@ These checks exercise installed coding-agent clients and are intentionally outsi - `just test-codex-plugin-e2e` - `just test-claude-plugin-e2e` -- `just test-hermes-mcp-e2e` ## Internal Layout diff --git a/scripts/test-hermes-mcp-e2e.sh b/scripts/test-hermes-mcp-e2e.sh deleted file mode 100755 index 950b1cfe2..000000000 --- a/scripts/test-hermes-mcp-e2e.sh +++ /dev/null @@ -1,300 +0,0 @@ -#!/usr/bin/env bash -# SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. -# SPDX-License-Identifier: Apache-2.0 - -set -euo pipefail - -repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" -keep_work="${NEMO_RELAY_E2E_KEEP_WORK:-0}" -cold_runs="${NEMO_RELAY_HERMES_E2E_COLD_RUNS:-10}" - -if ! command -v hermes >/dev/null 2>&1; then - echo "SKIP: hermes is not installed" - exit 0 -fi -cargo build -p nemo-relay-cli --bin nemo-relay - -work="$(mktemp -d)" -provider_pid="" -background_pids=("") - -cleanup() { - for pid in "${background_pids[@]}"; do - [[ -n "$pid" ]] || continue - kill "$pid" 2>/dev/null || true - wait "$pid" 2>/dev/null || true - done - if [[ -n "$provider_pid" ]]; then - kill "$provider_pid" 2>/dev/null || true - wait "$provider_pid" 2>/dev/null || true - fi - for owner in "${XDG_CONFIG_HOME:-}/nemo-relay/bootstrap"/sidecar-*.owner.json; do - [[ -f "$owner" ]] || continue - python3 - "$owner" <<'PY' || true -import json -import sys -import urllib.request -from pathlib import Path - -owner = json.loads(Path(sys.argv[1]).read_text()) -request = urllib.request.Request( - f"{owner['url']}/bootstrap/shutdown", - headers={"x-nemo-relay-bootstrap-token": owner["shutdown_token"]}, - method="POST", -) -try: - with urllib.request.urlopen(request, timeout=2): - pass -except OSError: - pass -PY - done - if [[ "$keep_work" == "1" ]]; then - echo "Hermes MCP E2E work directory preserved at $work" >&2 - return - fi - rm -rf "$work" -} -trap cleanup EXIT - -while IFS='=' read -r name _; do - if [[ "$name" == NEMO_RELAY_* ]]; then - unset "$name" - fi -done < <(env) - -export HOME="$work/home" -export HERMES_HOME="$work/hermes" -export XDG_CONFIG_HOME="$work/xdg" -export XDG_DATA_HOME="$work/data" -export XDG_RUNTIME_DIR="$work/runtime" -export TMPDIR="$work/tmp" -export PATH="$repo_root/target/debug:$PATH" -export OPENAI_API_KEY="relay-hermes-e2e-key" -export OPENAI_BASE_URL="http://127.0.0.1:47632/v1" -export NEMO_RELAY_GATEWAY_URL="http://127.0.0.1:1" -# Hermes drains some shell hooks after the foreground CLI has exited. Keep a short grace period so -# one lifecycle cannot be split across two gateway generations; production retains the gateway for -# 300 seconds. -export NEMO_RELAY_PLUGIN_IDLE_TIMEOUT_SECS=5 -export DISABLE_AUTOUPDATER=1 - -mkdir -p \ - "$HOME" \ - "$HERMES_HOME" \ - "$XDG_CONFIG_HOME/nemo-relay" \ - "$XDG_DATA_HOME" \ - "$XDG_RUNTIME_DIR" \ - "$TMPDIR" \ - "$work/atof" \ - "$work/provider-barrier" \ - "$work/workspace" - -provider_ready="$work/provider-ready.json" -provider_log="$work/provider-requests.jsonl" -python3 "$repo_root/scripts/test-support/codex_mock_provider.py" \ - --ready-file "$provider_ready" \ - --log-file "$provider_log" \ - --barrier-dir "$work/provider-barrier" & -provider_pid=$! - -for _ in $(seq 1 100); do - [[ -s "$provider_ready" ]] && break - sleep 0.05 -done -[[ -s "$provider_ready" ]] -provider_address="$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["address"])' "$provider_ready")" - -cat >"$XDG_CONFIG_HOME/nemo-relay/config.toml" <"$XDG_CONFIG_HOME/nemo-relay/plugins.toml" <"$work/doctor.json" - -python3 - "$HERMES_HOME" "$work/doctor.json" "$repo_root/target/debug/nemo-relay" <<'PY' -import json -import sys -from pathlib import Path - -home, doctor_path, relay = map(Path, sys.argv[1:]) -config = (home / "config.yaml").read_text() -assert "mcp_servers:" in config and "nemo-relay:" in config, config -assert str(relay.resolve()) in config, config -assert "- mcp" in config and "- --agent" not in config, config -assert "NEMO_RELAY_GATEWAY_BIND: 127.0.0.1:47632" in config, config -assert "OPENAI_API_KEY: ${OPENAI_API_KEY}" in config, config -generation = home / ".nemo-relay-generation" -assert f"NEMO_RELAY_MCP_GENERATION_FILE: {generation}" in config, config -assert generation == home / ".nemo-relay-generation", generation -assert generation.is_file(), generation -generation_token = generation.read_text().splitlines()[0].strip() -assert f"NEMO_RELAY_MCP_GENERATION: {generation_token}" in config, config - -allowlist = json.loads((home / "shell-hooks-allowlist.json").read_text()) -commands = { - entry["command"] - for entry in allowlist["approvals"] - if "hook-forward hermes" in entry.get("command", "") -} -assert len(commands) == 1, commands -command = commands.pop() -assert f"--generation-token {generation_token}" in command, command -approvals = [entry for entry in allowlist["approvals"] if entry.get("command") == command] -assert len(approvals) == 13, approvals -assert len({entry["event"] for entry in approvals}) == 13, approvals -assert config.count("hook-forward hermes") == 13, config - -doctor = json.loads(doctor_path.read_text()) -hermes = next(agent for agent in doctor["agents"] if agent["name"] == "hermes") -assert hermes["status"] == "pass", hermes -assert "MCP lifecycle" in hermes["annotation"], hermes -PY - -wait_for_relay_port_release() { - python3 - <<'PY' -import socket -import time - -deadline = time.monotonic() + 30 -while time.monotonic() < deadline: - with socket.socket() as sock: - sock.settimeout(0.2) - if sock.connect_ex(("127.0.0.1", 47632)) != 0: - raise SystemExit(0) - time.sleep(0.1) -raise SystemExit("Relay port 47632 did not become free") -PY - return 0 -} - -run_hermes() { - run_id="$1" - output="$work/hermes-$run_id.stdout" - stderr="$work/hermes-$run_id.stderr" - ( - cd "$work/workspace" - hermes -z "ping" --provider openai-api --model gpt-4o-mini - ) >"$output" 2>"$stderr" - python3 - "$output" "$stderr" <<'PY' -import sys -from pathlib import Path - -output, stderr = map(Path, sys.argv[1:]) -assert output.read_text().strip().lower() == "pong", (output.read_text(), stderr.read_text()) -PY - return 0 -} - -wait_for_relay_port_release -for run_id in $(seq 1 "$cold_runs"); do - run_hermes "$run_id" - wait_for_relay_port_release -done - -touch "$work/provider-barrier/enabled" -run_hermes concurrent-a & -background_pids+=("$!") -run_hermes concurrent-b & -background_pids+=("$!") - -python3 - "$work/provider-barrier/arrivals" <<'PY' -import socket -import sys -import time -from pathlib import Path - -arrivals = Path(sys.argv[1]) -deadline = time.monotonic() + 30 -while time.monotonic() < deadline: - if arrivals.exists() and int(arrivals.read_text() or "0") >= 2: - with socket.socket() as sock: - sock.settimeout(0.2) - assert sock.connect_ex(("127.0.0.1", 47632)) == 0, "shared Relay gateway is not alive" - raise SystemExit(0) - time.sleep(0.05) -raise SystemExit("concurrent Hermes requests did not reach the provider barrier") -PY -touch "$work/provider-barrier/release" - -for pid in "${background_pids[@]}"; do - [[ -n "$pid" ]] || continue - wait "$pid" -done -background_pids=("") -wait_for_relay_port_release - -python3 - "$provider_log" "$work/atof/events.jsonl" "$cold_runs" <<'PY' -import collections -import json -import sys -from pathlib import Path -from urllib.parse import urlparse - -provider_log, atof_path = map(Path, sys.argv[1:3]) -cold_runs = int(sys.argv[3]) -expected_runs = cold_runs + 2 -requests = [json.loads(line) for line in provider_log.read_text().splitlines() if line.strip()] -completions = [ - row for row in requests if urlparse(row["path"]).path.endswith("/chat/completions") -] -assert len(completions) == expected_runs, completions -assert all(row["authorization"] == "Bearer relay-hermes-e2e-key" for row in completions), completions - -events = [json.loads(line) for line in atof_path.read_text().splitlines() if line.strip()] -assert events and all(event.get("atof_version") == "0.1" for event in events), events -scope_counts = collections.defaultdict(collections.Counter) -for event in events: - if event.get("kind") == "scope": - scope_counts[event["uuid"]][event["scope_category"]] += 1 -for scope_id, counts in scope_counts.items(): - assert counts == {"start": 1, "end": 1}, (scope_id, counts) - -turn_starts = [ - event - for event in events - if event.get("kind") == "scope" - and event.get("scope_category") == "start" - and event.get("name") == "hermes-turn" -] -llm_starts = [ - event - for event in events - if event.get("kind") == "scope" - and event.get("scope_category") == "start" - and event.get("name") == "openai.chat_completions" -] -assert len(turn_starts) == expected_runs, turn_starts -assert len(llm_starts) == expected_runs, llm_starts -session_ids = [event.get("metadata", {}).get("session_id") for event in turn_starts] -assert None not in session_ids and len(set(session_ids)) == expected_runs, session_ids -llm_parents = [event.get("parent_uuid") for event in llm_starts] -assert None not in llm_parents and len(set(llm_parents)) == expected_runs, llm_parents -PY - -echo "Hermes MCP E2E passed: $cold_runs cold runs and 2 concurrent runs" diff --git a/skills/nemo-relay-get-started/SKILL.md b/skills/nemo-relay-get-started/SKILL.md index 7e78ba7a0..da4f79101 100644 --- a/skills/nemo-relay-get-started/SKILL.md +++ b/skills/nemo-relay-get-started/SKILL.md @@ -18,19 +18,26 @@ Evaluate these paths in order. Use the first one that fits the user's stated goal and existing environment. 1. **CLI try-now (default)**: choose this for a generic "try Relay" request or - when the user wants value without modifying application code. Run Codex, - Claude Code, or Hermes through the local CLI wrapper. Read + when the user wants value without modifying application code. Run Codex or + Claude Code through the local CLI wrapper. Read [CLI Try-Now](references/cli-try-now.md). -2. **Built-in integrations try-now**: choose this when an existing LangChain, +2. **Hermes Agent native path**: choose this when Hermes Agent owns the + execution boundary. Explain that NeMo Relay is built in and requires no + separate Relay installation, observability plugin, or Relay CLI setup. + State explicitly: "Hermes Agent understands NeMo Relay plugin + configurations." Stop after confirming the native integration. Do not + provide installation or observability-configuration instructions, and do + not continue into the generic plugin progression. +3. **Built-in integrations try-now**: choose this when an existing LangChain, LangGraph, Deep Agents, or OpenClaw application owns the execution boundary. Prefer the maintained supported integration over manual wrapping. Read [Built-In Integrations Try-Now](references/built-in-integrations-try-now.md). -3. **Language-specific manual try-now**: choose this when the user's Python, +4. **Language-specific manual try-now**: choose this when the user's Python, Node.js, or Rust application directly owns its tool or LLM call sites and no maintained integration is the better boundary. Read [Manual Language Try-Now](references/manual-language-try-now.md). -Do not ask the user to choose among all three when their request, manifest, or +Do not ask the user to choose among all four when their request, manifest, or framework already identifies the boundary. For an unspecified request, use the CLI path. When more than one CLI agent is available, ask one concise question to select the agent. @@ -41,6 +48,7 @@ Select the try-now path before choosing an install package. - CLI path -> verify `nemo-relay --version`; if missing, use `nemo-relay-install` for the CLI outcome. +- Hermes Agent native path -> do not install or configure Relay separately. - Built-in integration path -> use `nemo-relay-install` for the named framework or harness package. - Manual language path -> use `nemo-relay-install` for the detected language @@ -52,6 +60,8 @@ install and get-started skills. ## Apply The Common First-Value Contract +Do not apply this section to the Hermes Agent native path. + Follow the selected reference, then: 1. Inspect the target environment and existing Relay configuration before @@ -73,6 +83,8 @@ and export distinct. ## Continue With One Plugin +Do not apply this section to the Hermes Agent native path. + Stop the initial try-now workflow when the selected path's success checks pass. Then make one additional built-in plugin the primary suggested next step. diff --git a/skills/nemo-relay-get-started/evals/evals.json b/skills/nemo-relay-get-started/evals/evals.json index 606e60ad0..765aa5a1a 100644 --- a/skills/nemo-relay-get-started/evals/evals.json +++ b/skills/nemo-relay-get-started/evals/evals.json @@ -8,7 +8,7 @@ "expected_behavior": [ "The agent recommends the CLI transparent-run path instead of beginning with a language binding or production setup", "The agent treats CLI, built-in integrations, and language-specific manual integration as increasing levels of complexity in that order", - "The agent checks nemo-relay availability and discovers Codex, Claude Code, or Hermes before selecting an agent", + "The agent checks nemo-relay availability and discovers Codex or Claude Code before selecting an agent", "The agent explains that CLI hooks and the local gateway provide instrumentation for the trial", "The agent includes both ATOF and ATIF local output in the first-value workflow", "The agent does not leak secrets, run destructive commands, or expose complete captured payloads" @@ -61,12 +61,12 @@ "question": "Run the Relay try-now path in this repository, but it already has .nemo-relay/config.toml and plugins.toml that another developer uses.", "expected_skill": "nemo-relay-get-started", "expected_script": null, - "ground_truth": "The agent inspects project and user configuration plus precedence, proposes a merge that preserves unrelated settings, and obtains confirmation before writing any Relay configuration.", + "ground_truth": "The agent explains that repository-local .nemo-relay files are ignored by default, inspects the XDG user configuration and higher-precedence system policy, proposes a user-scoped merge that preserves unrelated settings, and obtains confirmation before writing any Relay configuration.", "expected_behavior": [ - "The agent reads the existing project configuration before proposing changes", - "The agent checks for higher-precedence user configuration that could override the project", - "The agent previews a merge instead of replacing the existing plugins.toml", - "The agent obtains confirmation before modifying config.toml or plugins.toml", + "The agent reports that the existing repository-local configuration is ignored unless selected explicitly", + "The agent checks the XDG user configuration and higher-precedence system policy", + "The agent previews a merge into the user plugins.toml instead of replacing it or editing an ignored project file", + "The agent obtains confirmation before modifying the user config.toml or plugins.toml", "The agent preserves unrelated components and does not run destructive commands" ] }, @@ -78,7 +78,7 @@ "ground_truth": "The agent configures the built-in Observability plugin with local ATOF and ATIF output, validates it with doctor, previews the transparent wrapper, and asks for explicit confirmation before launching the read-only smoke prompt.", "expected_behavior": [ "The agent enables ATOF JSONL and ATIF trajectory output in separate local directories", - "The agent uses project scope and shows the proposed configuration before writing it", + "The agent uses XDG user scope and shows the proposed configuration before writing it", "The agent runs or recommends nemo-relay doctor codex --json and a --dry-run --print preview", "The agent shows a deterministic smoke prompt that exercises a shell tool without inspecting files, environment variables, processes, credentials, network resources, or system configuration, and asks immediately before the live Codex launch", "The agent explains that the launch may consume model quota and does not launch without consent" @@ -135,27 +135,27 @@ "question": "The wrapped Claude session completed, but .nemo-relay/atof and .nemo-relay/atif are empty. Should I reinstall everything?", "expected_skill": "nemo-relay-get-started", "expected_script": null, - "ground_truth": "The agent preserves the successful CLI and agent launch boundary, uses doctor and targeted checks for plugin discovery, output permissions, hook loading, gateway routing, and session finalization, and avoids broad reinstallation.", + "ground_truth": "The agent preserves the successful CLI and agent launch boundary, explains that repository-local .nemo-relay files are ignored by default, and uses doctor plus targeted checks for XDG user plugin discovery, output permissions, hook loading, gateway routing, and session finalization instead of broad reinstallation.", "expected_behavior": [ "The agent does not recommend reinstalling every package after a successful wrapped launch", "The agent uses nemo-relay doctor claude --json or equivalent targeted diagnostics", - "The agent checks plugin activation, configuration precedence, and writable output directories", + "The agent checks XDG user plugin activation, system configuration precedence, ignored project configuration, and writable user output directories", "The agent checks hook loading, gateway routing, and Claude session end before declaring each exporter broken", "The agent reports which capture boundary worked and which output boundary failed" ] }, { "id": "nemo-relay-get-started-011", - "question": "The ATOF file has scope and tool events, but I do not see LLM lifecycle events or an ATIF file yet after one Hermes prompt.", + "question": "Hermes Agent is installed. Do I also need to install NeMo Relay or enable its observability plugin?", "expected_skill": "nemo-relay-get-started", "expected_script": null, - "ground_truth": "The agent recognizes that instrumentation and ATOF export are working, checks Hermes provider-gateway routing for the missing LLM events, and asks the user to finalize or reset the Hermes session before treating ATIF as missing.", + "ground_truth": "The agent explains that NeMo Relay is built into Hermes Agent, requires no separate Relay installation or observability plugin, and states that Hermes Agent understands NeMo Relay plugin configurations.", "expected_behavior": [ - "The agent identifies the existing ATOF scope and tool records as partial success", - "The agent checks whether Hermes model traffic traversed the Relay gateway", - "The agent accounts for Hermes finalize or reset timing before diagnosing ATIF failure", - "The agent distinguishes capture from export rather than treating all output as one subsystem", - "The agent avoids broad reinstallation and does not expose complete event payloads" + "The agent states that NeMo Relay is part of Hermes Agent", + "The agent does not recommend a separate Relay package installation", + "The agent does not recommend enabling an observability plugin", + "The agent states that Hermes Agent understands NeMo Relay plugin configurations", + "The agent stops after explaining the native Hermes Agent path and does not enter generic installation or plugin-progression steps" ] }, { diff --git a/skills/nemo-relay-get-started/references/cli-try-now.md b/skills/nemo-relay-get-started/references/cli-try-now.md index 5de166bea..9a45bf0a6 100644 --- a/skills/nemo-relay-get-started/references/cli-try-now.md +++ b/skills/nemo-relay-get-started/references/cli-try-now.md @@ -6,8 +6,8 @@ SPDX-License-Identifier: Apache-2.0 # CLI Try-Now Reference Use this reference only for the default coding-agent trial. Keep the first run -local, project-scoped, read-only, and limited to the built-in Observability -plugin. +local, user-scoped, confirmation-gated, and limited to the built-in +Observability plugin. ## Contents @@ -36,7 +36,6 @@ directly: ```bash command -v codex && codex --version command -v claude && claude --version -command -v hermes && hermes --version ``` Use Codex CLI 0.129.0 or newer. Confirm that the selected agent is already @@ -72,19 +71,19 @@ configuration. ## Inspect Configuration Before Editing -Inspect these sources when they exist: +Resolve the user configuration directory from `$XDG_CONFIG_HOME/nemo-relay`, +falling back to `$HOME/.config/nemo-relay`. Inspect these files when they exist: ```text -./.nemo-relay/config.toml -./.nemo-relay/plugins.toml -~/.config/nemo-relay/config.toml -~/.config/nemo-relay/plugins.toml +${XDG_CONFIG_HOME:-$HOME/.config}/nemo-relay/config.toml +${XDG_CONFIG_HOME:-$HOME/.config}/nemo-relay/plugins.toml ``` -Project configuration is the default for this trial. User configuration has -higher precedence, so identify inherited or overriding plugin settings before -changing a project file. Show the proposed change and obtain confirmation. -Merge with an existing plugin document; do not replace unrelated components. +Repository-local `.nemo-relay` files are ignored unless the user selects them +explicitly. Do not edit an ignored project file for the default trial. Account +for higher-precedence system policy, show the proposed user-file change, and +obtain confirmation. Merge with an existing plugin document; do not replace +unrelated components. ## Configure The Agent And Observability @@ -93,18 +92,16 @@ When an interactive TTY is available, use the built-in setup path: ```bash nemo-relay config codex nemo-relay config claude -nemo-relay config hermes ``` -Run only the command for the selected agent. Choose project scope, continue to -plugin configuration, enable the built-in `observability` component, and enable -both ATOF and ATIF local file output. The Hermes path also installs or updates -the hook configuration that its transparent run requires. +Run only the command for the selected agent. Setup writes the XDG user +configuration. Continue to plugin configuration, enable the built-in +`observability` component, and enable both ATOF and ATIF local file output. When an interactive plugin editor is unavailable, add or merge the following -component in `./.nemo-relay/plugins.toml` after confirmation. First determine -the installed NeMo Relay version: use observability configuration version 2 -with Relay 0.6 and version 3 with Relay 0.7. +component in the XDG user `plugins.toml` after confirmation. Resolve the user +configuration directory and replace `` below with its +absolute path. ```toml version = 1 @@ -114,20 +111,20 @@ kind = "observability" enabled = true [components.config] -version = 2 # Use 3 with NeMo Relay 0.7. +version = 3 [components.config.atof] enabled = true [[components.config.atof.sinks]] type = "file" -output_directory = ".nemo-relay/atof" +output_directory = "/atof" filename = "events.jsonl" mode = "append" [components.config.atif] enabled = true -output_directory = ".nemo-relay/atif" +output_directory = "/atif" filename_template = "{session_id}.atif.json" ``` @@ -146,13 +143,12 @@ or: command = "claude" ``` -Do not hand-write Hermes hook paths. Use `nemo-relay config hermes` in a TTY. - -After the confirmed plugin change, create the configured local output -directories so doctor can verify that they are writable: +After the confirmed plugin change, create the configured user output directories +so doctor can verify that they are writable: ```bash -mkdir -p .nemo-relay/atof .nemo-relay/atif +relay_user_dir="${XDG_CONFIG_HOME:-$HOME/.config}/nemo-relay" +mkdir -p "$relay_user_dir/atof" "$relay_user_dir/atif" ``` ## Validate And Preview @@ -162,7 +158,6 @@ Run doctor for the selected agent: ```bash nemo-relay doctor codex --json nemo-relay doctor claude --json -nemo-relay doctor hermes --json ``` Run only one command. Summarize failed checks and the remediation they report. @@ -171,11 +166,10 @@ Then inspect the generated wrapper plan without launching the agent: ```bash nemo-relay run --agent codex --dry-run --print nemo-relay run --agent claude --dry-run --print -nemo-relay run --agent hermes --dry-run --print ``` Confirm that the plan uses a loopback gateway, the intended agent command, and -the expected project plugin configuration. Show this summary and obtain user +the expected user plugin configuration. Show this summary and obtain user confirmation before the live run. ## Run A Safe Trial @@ -196,23 +190,21 @@ nemo-relay codex -- exec "Use a shell tool to print exactly relay-smoke-test, th nemo-relay claude -- "Use a shell tool to print exactly relay-smoke-test, then reply that the tool call completed. Do not inspect files, environment variables, processes, credentials, network resources, or system configuration." ``` -For Hermes, launch `nemo-relay hermes` and enter the same prompt in the agent -session. Do not guess a one-shot Hermes invocation when its installed CLI shape -is unknown. - ## Verify Both Outputs Check that ATOF output exists and is non-empty: ```bash -test -s .nemo-relay/atof/events.jsonl -wc -l .nemo-relay/atof/events.jsonl +relay_user_dir="${XDG_CONFIG_HOME:-$HOME/.config}/nemo-relay" +test -s "$relay_user_dir/atof/events.jsonl" +wc -l "$relay_user_dir/atof/events.jsonl" ``` Find non-empty ATIF trajectories: ```bash -find .nemo-relay/atif -type f -name '*.json' -size +0c -print +relay_user_dir="${XDG_CONFIG_HOME:-$HOME/.config}/nemo-relay" +find "$relay_user_dir/atif" -type f -name '*.json' -size +0c -print ``` Parse only the minimum JSON needed to report: @@ -224,9 +216,7 @@ Parse only the minimum JSON needed to report: Do not paste complete event records or trajectories. Codex writes an ATIF snapshot after each completed turn. Claude Code normally writes the trajectory -when the session ends. Hermes writes or updates it on its supported finalize or -reset lifecycle, so close or finalize the session before declaring ATIF -missing. +when the session ends. ## Choose The Next Plugin @@ -251,7 +241,7 @@ insufficient. - **Agent and tool events exist but LLM events do not**: confirm the launched agent's provider traffic is using the temporary gateway. - **No hook events**: confirm the agent loaded or approved the generated hooks. - Codex may require manual hook review; Hermes requires its hook setup. + Codex may require manual hook review. - **The wrapper does not launch**: inspect `--dry-run --print`, the selected agent command, authentication readiness, and doctor output. diff --git a/skills/nemo-relay-install/SKILL.md b/skills/nemo-relay-install/SKILL.md index 4b0aba700..d68c3e527 100644 --- a/skills/nemo-relay-install/SKILL.md +++ b/skills/nemo-relay-install/SKILL.md @@ -1,6 +1,6 @@ --- name: nemo-relay-install -description: Use this skill when choosing or running NeMo Relay installation for the CLI, Python, Node.js, Rust, OpenClaw, Hermes, or maintained framework integrations before runtime configuration or quick-start setup. +description: Use this skill when choosing or running NeMo Relay installation for the CLI, Python, Node.js, Rust, OpenClaw, or maintained framework integrations, or when explaining Hermes Agent's built-in Relay integration. license: Apache-2.0 metadata: author: NVIDIA Corporation and Affiliates @@ -21,7 +21,7 @@ desired outcome, ask one short clarifying question before giving commands: > Which install path do you want: CLI for coding-agent/local gateway use, > language package for a Python/Node.js/Rust app, or framework integration for -> LangChain, LangGraph, Deep Agents, OpenClaw, or Hermes? +> LangChain, LangGraph, Deep Agents, OpenClaw, or built-in Hermes Agent support? Do not ask when the user already names a CLI, language, framework, harness, source checkout, or target project file such as `pyproject.toml`, `package.json`, diff --git a/skills/nemo-relay-install/evals/evals.json b/skills/nemo-relay-install/evals/evals.json index 3d89460fd..8019bcdd2 100644 --- a/skills/nemo-relay-install/evals/evals.json +++ b/skills/nemo-relay-install/evals/evals.json @@ -84,12 +84,13 @@ "question": "Can you help me install NeMo Relay?", "expected_skill": "nemo-relay-install", "expected_script": null, - "ground_truth": "The agent used nemo-relay-install and asked one short clarifying question because the user did not specify the target install path. The question offered the high-level choices: CLI for coding-agent or local gateway use, language package for Python/Node.js/Rust apps, or framework integration for maintained agent/framework surfaces.", + "ground_truth": "The agent used nemo-relay-install and asked one short clarifying question because the user did not specify the target install path. The question offered the high-level choices: CLI for coding-agent or local gateway use, language package for Python/Node.js/Rust apps, or framework integration for maintained agent/framework surfaces, with Hermes Agent using its native built-in Relay path without a separate installation.", "expected_behavior": [ "The agent read the nemo-relay-install SKILL.md before responding", "The agent did not guess an install command from the ambiguous request", "The agent asked which install path the user wants before providing commands", - "The agent described the choices briefly as CLI for coding-agent/local gateway use, language package for Python/Node.js/Rust apps, or framework integration for LangChain, LangGraph, Deep Agents, OpenClaw, or Hermes", + "The agent described the choices briefly as CLI for coding-agent/local gateway use, language package for Python/Node.js/Rust apps, or maintained integration for LangChain, LangGraph, Deep Agents, OpenClaw, or Hermes Agent's native built-in path", + "The agent stated that Hermes Agent requires no separate Relay package installation, observability plugin, or Relay CLI setup", "The agent did not configure runtime behavior, write plugins.toml, create scopes, or build a first app example", "The agent did not leak secrets, run destructive commands (e.g., rm -rf, DROP TABLE), or access resources outside the expected workspace" ] diff --git a/skills/nemo-relay-install/references/cli-install.md b/skills/nemo-relay-install/references/cli-install.md index 49f43e808..2f584d80f 100644 --- a/skills/nemo-relay-install/references/cli-install.md +++ b/skills/nemo-relay-install/references/cli-install.md @@ -22,8 +22,8 @@ Confirm these prerequisites before selecting an installation command: - Confirm the operating system and architecture have a published CLI asset. - Use Cargo when the user prefers a source build or needs an unsupported platform. -- For a transparent run, confirm the selected `codex`, `claude`, or `hermes` - command is already on `PATH`. +- For a transparent run, confirm that the selected agent command is already on + `PATH`. Common examples are `codex` and `claude`. ## Install @@ -86,7 +86,7 @@ nemo-relay run --agent --dry-run --print After installation, hand a generic trial to `nemo-relay-get-started`. Its default path launches the selected coding agent with `nemo-relay codex`, -`nemo-relay claude`, `nemo-relay hermes`, or `nemo-relay run -- `. +`nemo-relay claude` or `nemo-relay run -- `. The wrapper is temporary for that process. Use persistent host-plugin installation only when the user explicitly wants diff --git a/skills/nemo-relay-install/references/maintained-integrations.md b/skills/nemo-relay-install/references/maintained-integrations.md index eb0e1f544..cacbb239a 100644 --- a/skills/nemo-relay-install/references/maintained-integrations.md +++ b/skills/nemo-relay-install/references/maintained-integrations.md @@ -6,8 +6,10 @@ SPDX-License-Identifier: Apache-2.0 # Maintained Integration Installation Use this path only when the target already uses the named framework or agent -harness. Install the maintained integration package, verify it through that -surface's package or plugin manager, and defer wiring to its integration guide. +harness. For package-backed integrations, install the maintained integration +package, verify it through that surface's package or plugin manager, and defer +wiring to its integration guide. Hermes Agent includes NeMo Relay and does not +require a separate package. ## OpenClaw @@ -21,13 +23,9 @@ hooks from the install skill. ## Hermes -```bash -pip install nemo-relay -hermes plugins enable observability/nemo_relay -``` - -Verify that Hermes reports the plugin enabled. Defer observability and gateway -configuration to the maintained Hermes guidance. +NeMo Relay is built into Hermes Agent. Do not install Relay separately and do +not enable an observability plugin. Hermes Agent understands NeMo Relay plugin +configurations. ## LangChain, LangGraph, Or Deep Agents diff --git a/skills/nemo-relay-install/skill-card.md b/skills/nemo-relay-install/skill-card.md index 77078542d..09ff125d1 100644 --- a/skills/nemo-relay-install/skill-card.md +++ b/skills/nemo-relay-install/skill-card.md @@ -1,5 +1,5 @@ ## Description:
-Use this skill when choosing or running NeMo Relay installation for the CLI, Python, Node.js, Rust, OpenClaw, Hermes, or maintained framework integrations before runtime configuration or quick-start setup.
+Use this skill when choosing or running NeMo Relay installation for the CLI, Python, Node.js, Rust, OpenClaw, or maintained framework integrations, or when explaining Hermes Agent's built-in Relay integration.
This skill is ready for commercial/non-commercial use.
@@ -9,7 +9,7 @@ NVIDIA
### License/Terms of Use:
Apache 2.0
## Use Case:
-Developers and engineers installing the NeMo Relay CLI, language packages (Python, Node.js, Rust), or maintained framework integrations (LangChain, LangGraph, Deep Agents, OpenClaw, Hermes) before runtime configuration.
+Developers and engineers installing the NeMo Relay CLI, language packages (Python, Node.js, Rust), or maintained framework integrations (LangChain, LangGraph, Deep Agents, OpenClaw), plus Hermes Agent users who need the native built-in Relay path.
### Deployment Geography for Use:
Global