perf(*): let raven agents share memory with their host and with each other

CONTEXT.md

@@ -3392,6 +3392,67 @@ and every tool result.
 _Avoid_: calling it a transcript - a reader drawing a delegated run wants the Instance Log
 or `transcript.jsonl`, not this.
 
+**Idle close** (`AcpConnectionPool.close_idle`, `subagents.runtime.idleCloseS`):
+The pool ending a pooled ACP agent's process once the connection has been quiet for the
+configured budget: no request of the host's unanswered (a held wake is one), none of the agent's
+being answered, no frame from the agent and no lease for the whole budget. Only an agent that
+advertises `loadSession`, or one that never opened a session, is closed, so a stateful
+instance's next turn reopens its session in a fresh process with `session/load`. An agent that
+declares `raven.residentWork` is asked first and kept while it holds Resident work. Each event
+loop closes only the connections made on it, whose pipes are its own.
+_Avoid_: calling it a timeout - nothing in flight is cut short, nor anything held for later.
+
+**Resident work** (`_raven/residentWork`, `CronService.armed_jobs`, `running_plugin_services`):
+What a Raven agent holds that runs with nobody asking: an enabled cron job of its own partition
+while its scheduler runs, and a running plugin Service. Neither puts anything on the wire, and
+closing the process loses both: no other partition claims the job, and the Service lives only as
+long as the process. The agent declares `raven.residentWork` in `agentCapabilities._meta` and
+answers `_raven/residentWork` as things stand at the moment of asking, so a one-shot wake that
+has fired no longer counts. The host decides on it, so it is a protocol extension the host asks,
+not Response Meta.
+_Avoid_: a held wake (`raven.holdTurn`) for it - that keeps a prompt open and the agent busy;
+resident work is what an agent holds between prompts.
+
+**LLM relay** (`raven/providers/relay.py`, `relay_provider.py`, `relay_server.py`, `subagents.runtime.llmRelay`):
+The host making a Raven agent's LiteLLM calls for it. The host offers a local socket and a token
+in the launch environment (`RAVEN_LLM_RELAY`), and an agent that reads them builds a
+`RelayProvider` where it would have built a `LiteLLMProvider`: the same construction and the
+same answers (both stand on `LiteLLMShape`), with only the single `chat` / `chat_stream` attempt
+and the `relayed` catalogue lookups crossing to the host. Retry, classification, usage records
+and traces stay in the agent; the host's calls run under `recorded_by_caller` so a call is
+billed once. The relay is served by the event loop that started it: a later loop starts another,
+and a loop running beside it in another thread is offered the same socket. A connection presents
+the token first, alone and under a deadline, so a peer without it is dropped before a frame is
+read. The host builds an agent's provider without writing the agent's key into its own
+environment, and makes each call in a context of its own under the window of the agent's turn,
+so the output ceiling on the wire is the one the agent's loop reserved. A call is made with the
+host's environment, so an agent whose own configuration sets something LiteLLM reads from it (an
+endpoint, a cloud credential, a proxy or CA bundle) is launched without the relay; and a refusal
+of prompt-cache marks is learned on the host too, which builds the agent's resend. An agent that
+cannot reach the host loads LiteLLM and works as before.
+
+**Fork template** (`raven/acp_client/fork_template.py`, `raven/cli/_fork_template.py`, `subagents.runtime.forkTemplate`):
+On Linux, one process per host that has imported what Raven agents import, frozen out of the
+collector's reach, and forks each Raven agent the host launches, so the agents share those pages.
+Two halves: the host's side in `acp_client` decides when to ask, starts and ends the template and
+holds the endpoint a forked agent runs behind; the template's own side is a launcher beside the
+CLI, because it loads the surfaces by name and runs `python -m raven`, which the client shelf may
+not. The client starts it through Raven's entry point, which settles the certificate store first,
+and never imports it. It starts through the parent-death
+wrapper like any agent, its stdin is a lifeline only the host holds, and it belongs to the event
+loop that started it: its pipes and the watcher that reaps it are that loop's, so it is ended with
+that loop and a later loop starts another; a loop running beside it in another thread launches the
+ordinary way rather than end the agents it still has. Each forked agent leads a session of its own
+and arms the parent-death signal against the template. A product launcher's `os.execv` of
+`python -m raven` runs Raven in the same process. Agents write the modules they imported beside
+the template, per product, and the next template loads the modules at least two products import
+(one product's own while only one has been seen): a module one product alone imports is shared by
+no other process, so it is left for that product to import after the fork. Only the host's own
+interpreter starting `-m raven` or a product launcher is forked; anything else launches the
+ordinary way. A template that has not reported ready within a minute is ended, and launches go
+the ordinary way for ten minutes before a new one is started; one that reports it cannot serve,
+or fails to start any other way, is not used again by that loop.
+
 **Leftover agent** (`raven/acp_client/leftovers.py`):
 An ACP agent process group whose launching host is no longer running. An ACP agent runs in a
 session of its own, so it gets none of the signals sent to its host's process group or

docs-site/docs/agent-integrations.md

@@ -73,6 +73,28 @@ and Raven-Code's CLI hosting start a native product launcher explicitly. A
 successful readiness probe or delegation receipt is not evidence that the task
 completed; inspect the result, files, and tests or report artifacts.
 
+### Sharing memory with the host { #sharing-memory-with-the-host }
+
+Each native agent runs as a Raven process of its own. Three settings under
+`subagents.runtime` in the host's `config.json` keep that from costing a whole
+Raven per agent. All three are on by default and are read live, so a change
+applies to the next launch.
+
+| Key | Default | What it does |
+| --- | --- | --- |
+| `idleCloseS` | `600` | Closes an agent's process once it has been idle this many seconds: no request in flight either way and nothing said. An agent that still holds an armed reminder or a running plugin service is kept. Only an agent that can reload its sessions is closed, so the next turn resumes the conversation in a fresh process. `0` or `null` keeps agents until the host exits. |
+| `llmRelay` | `true` | The host makes a native agent's LiteLLM calls, so the agent never loads LiteLLM. The agent still builds each request, retries and records usage itself. An agent whose own `env` sets an LLM endpoint, a cloud credential, a proxy or a CA bundle makes its calls itself. |
+| `forkTemplate` | `true` | On Linux, native agents are forked from one process that has already loaded Raven, and share those pages instead of each loading its own copy. Other systems launch agents as before. |
+
+The template loads ahead only the modules at least two native agents share --
+one agent's own modules are imported by that agent after the fork -- so it
+grows with what the agents have in common, not with how many kinds there are.
+It is a process too, so it saves memory once two or more native agents run at
+the same time; with a single agent it costs somewhat more than it saves. Set
+`forkTemplate` to `false` on a device that only ever runs one. An agent the
+relay or the template cannot serve -- the host cannot be reached, or the agent
+runs on another interpreter -- starts the ordinary way.
+
 ### Minimal ACP invocation { #minimal-acp-invocation }
 
 Each shipped entry is started by its launcher. Use the corresponding command:

docs-site/docs/agent-integrations.zh.md

@@ -62,6 +62,24 @@ roster。不要假定修改显示名就创建了新身份。可通过 `raven onb
 launcher。readiness probe 成功或收到委派回执，都不代表任务已完成；仍需检查结果、文件、
 测试或报告产物。
 
+### 与宿主共享内存 { #sharing-memory-with-the-host }
+
+每个原生 Agent 都是一个独立的 Raven 进程。宿主 `config.json` 中 `subagents.runtime`
+下的三个设置让它不必每个 Agent 都占用一整份 Raven 的内存。三者默认开启，并实时读取，
+修改后从下一次启动起生效。
+
+| 键 | 默认值 | 作用 |
+| --- | --- | --- |
+| `idleCloseS` | `600` | Agent 空闲达到该秒数（双方都没有在途请求，也没有任何消息）后关闭其进程。仍持有已设定的提醒或正在运行的插件服务的 Agent 会被保留。只关闭能重新加载会话的 Agent，因此下一轮会在新进程中续上原对话。`0` 或 `null` 表示一直保留到宿主退出。 |
+| `llmRelay` | `true` | 由宿主代为发出原生 Agent 的 LiteLLM 调用，Agent 自身不再加载 LiteLLM。请求组装、重试和用量记录仍由 Agent 自己完成。若 Agent 自己的 `env` 设置了 LLM 端点、云凭据、代理或 CA 证书，它会自己发起调用。 |
+| `forkTemplate` | `true` | 在 Linux 上，原生 Agent 从一个已加载 Raven 的进程 fork 出来，共享这些内存页，而不是各自加载一份。其他系统仍按原方式启动。 |
+
+模板只预加载至少两个原生 Agent 共用的模块，某个 Agent 独有的模块由它在 fork 之后自行
+导入，因此模板的大小取决于各 Agent 的共同部分，而不是 Agent 的种类数。模板本身也是一个
+进程，因此同时运行两个及以上原生 Agent 时才会省内存；只有一个 Agent 时，它的占用会略多于
+它省下的部分。只会运行单个 Agent 的设备可将 `forkTemplate` 设为 `false`。中转或模板无法服务的 Agent（宿主不可达，或 Agent 使用另一个解释器）
+会按原方式启动。
+
 ### 最小 ACP 调用 { #minimal-acp-invocation }
 
 随 Raven 发布的每个 Agent 都由自己的 launcher 启动。使用对应命令：

raven/acp/capabilities.py

@@ -81,7 +81,14 @@ def agent_capabilities() -> dict[str, Any]:
         # turn already in flight -- a person speaking to an agent whose prompt
         # is held open while it waits on its own scheduled work. Advertised by
         # presence, which is how raven's own client reads it.
-        "_meta": {protocol.SESSION_MCP_CAPABILITY: {}, protocol.STEER_CAPABILITY: True},
+        # And ``_raven/residentWork``: whether this process holds work nobody is
+        # asking for yet (an armed cron job, a running plugin service), asked
+        # by raven's client before it closes the agent for idleness.
+        "_meta": {
+            protocol.SESSION_MCP_CAPABILITY: {},
+            protocol.STEER_CAPABILITY: True,
+            protocol.RESIDENT_WORK_CAPABILITY: True,
+        },
         # ``list``, ``resume``, ``close`` and ``delete``, and an empty object is
         # how the schema spells "supported". Each one is declared because the
         # method behind it exists and is honoured: ``_session_resume`` reopens a

raven/acp/methods.py

@@ -284,12 +284,28 @@ async def _route(self, method: str, params: dict[str, Any], *, is_request: bool)
             return await self._session_cancel(params)
         if method == protocol.STEER_METHOD:
             return await self._session_steer(params)
+        if method == protocol.RESIDENT_WORK_METHOD:
+            return self._resident_work()
         if method in UNIMPLEMENTED_METHODS:
             raise AcpMethodError(protocol.METHOD_NOT_FOUND, f"{method} is not implemented")
         raise AcpMethodError(protocol.METHOD_NOT_FOUND, f"unknown method {method}")
 
     # -- handlers ---------------------------------------------------------
 
+    def _resident_work(self) -> dict[str, Any]:
+        """What this agent holds that runs with nobody asking (``_raven/residentWork``).
+
+        An armed cron job of this agent's partition fires only from this
+        process, and a plugin service lives only as long as it does, so a client
+        about to close the agent for idleness asks first. Read at the moment of
+        asking: a one-shot wake that has fired no longer counts.
+        """
+        loop = self._agent_loop
+        cron = getattr(loop, "cron_service", None)
+        jobs = len(cron.armed_jobs()) if cron is not None else 0
+        services = len(getattr(loop, "running_plugin_services", ()) or ())
+        return {"resident": bool(jobs or services), "cronJobs": jobs, "services": services}
+
     def _initialize(self, params: dict[str, Any]) -> dict[str, Any]:
         """Answer the handshake and keep what the client declared.
 

raven/acp/protocol.py

@@ -78,6 +78,14 @@
 capability is advertised by presence under ``agentCapabilities._meta``.
 """
 
+RESIDENT_WORK_METHOD = "_raven/residentWork"
+RESIDENT_WORK_CAPABILITY = "raven.residentWork"
+"""raven's resident-work extension: what this agent holds that runs with nobody asking.
+
+Asked by raven's own client before it closes the agent for idleness; same names
+there, advertised the same way as the steer extension.
+"""
+
 STOP_REASONS = frozenset(
     {
         "end_turn",

raven/acp_client/capabilities.py

@@ -39,6 +39,7 @@
 from raven.acp_client.permissions import auto_approver
 from raven.acp_client.protocol import (
     MENU_UNREAD_META,
+    RESIDENT_WORK_CAPABILITY,
     SESSION_MCP_CAPABILITY,
     STEER_CAPABILITY,
     AcpConnectionError,
@@ -511,6 +512,8 @@ class _Handshake:
     can_load: bool = False
     can_delete: bool = False
     can_steer: bool = False
+    reports_resident_work: bool = False
+    """The agent answers ``_raven/residentWork``, so it can be asked before it is closed for idleness."""
     mcp_http: bool = False
     mcp_sse: bool = False
     session_mcp: bool = False
@@ -556,6 +559,7 @@ def _read_initialize(result: Any) -> _Handshake:
         # The one extension raven itself serves, announced in the schema's
         # extension carrier; a client that did not read it must not call it.
         can_steer=STEER_CAPABILITY in _dict(caps.get("_meta")),
+        reports_resident_work=RESIDENT_WORK_CAPABILITY in _dict(caps.get("_meta")),
         mcp_http=bool(mcp_caps.get("http")),
         mcp_sse=bool(mcp_caps.get("sse")),
         # The same carrier as the steer flag, and read the same way: presence,

raven/acp_client/client.py

@@ -142,6 +142,92 @@ async def _end(pgid: int, proc: Any) -> None:
         pass
 
 
+#: What LiteLLM reads from the environment at call time beside the key and base
+#: a provider passes explicitly: other endpoints and API versions, cloud
+#: credentials, proxies and CA bundles. A relayed call is made in the host, with
+#: the host's environment, so an agent configured with its own value for one of
+#: these is launched without the relay and makes its calls itself.
+_LLM_ENV_SUFFIXES = ("_API_BASE", "_BASE_URL", "_API_VERSION")
+_LLM_ENV_PREFIXES = ("AWS_", "AZURE_", "VERTEX", "LITELLM_")
+_LLM_ENV_NAMES = frozenset(
+    {
+        "GOOGLE_APPLICATION_CREDENTIALS",
+        "HTTP_PROXY",
+        "HTTPS_PROXY",
+        "ALL_PROXY",
+        "NO_PROXY",
+        "SSL_CERT_FILE",
+        "SSL_CERT_DIR",
+        "REQUESTS_CA_BUNDLE",
+        "CURL_CA_BUNDLE",
+    }
+)
+
+
+def _own_llm_env(agent_env: dict[str, str]) -> list[str]:
+    """The variables LiteLLM would read that this agent's own config sets differently from the host.
+
+    Only the agent's configured map: the login shell's environment differs from a
+    host's in ways nobody chose for one agent (a proxy list's casing), and
+    reading those would turn the relay off for every agent.
+    """
+    own = []
+    for key, value in agent_env.items():
+        upper = key.upper()
+        if upper.startswith("RAVEN_"):
+            continue
+        if upper in _LLM_ENV_NAMES or upper.endswith(_LLM_ENV_SUFFIXES) or upper.startswith(_LLM_ENV_PREFIXES):
+            if os.environ.get(key) != value:
+                own.append(key)
+    return sorted(own)
+
+
+async def _share_with_raven_agent(
+    name: str,
+    argv: list[str],
+    env: dict[str, str],
+    cwd: str | None,
+    base_env: dict[str, str],
+    agent_env: dict[str, str] | None = None,
+) -> Any:
+    """What this host shares with a Raven agent it launches: its model calls, and its loaded code.
+
+    Returns the environment to launch with -- the relay's address added when
+    the host's ``subagents.runtime.llmRelay`` is on -- or, when the fork
+    template started the agent, the endpoint it is already running behind.
+    Anything that is not a Raven agent on this interpreter is returned its
+    environment unchanged, and every failure here falls back to that.
+    """
+    from raven.acp_client import fork_template, leftovers
+    from raven.agent.subagent.backends.env import host_identity_env
+    from raven.agent.subagent.role import subagent_role_env
+    from raven.config.live import default_live, subagent_runtime
+
+    if not fork_template.raven_agent_argv(argv):
+        return env
+    runtime = subagent_runtime(default_live())
+    relayed = False
+    own = _own_llm_env(agent_env or {})
+    if runtime.llm_relay and own:
+        logger.info("acp agent {!r}: its own {} would not reach relayed calls; it makes them itself", name, own)
+    elif runtime.llm_relay:
+        from raven.providers.relay_server import relay_env
+
+        try:
+            env = {**env, **(await relay_env())}
+            relayed = True
+        except Exception as exc:  # noqa: BLE001 - the agent loads LiteLLM itself, as before
+            logger.warning("acp agent {!r}: no llm relay for it: {}", name, exc)
+    if runtime.fork_template:
+        template_env = {**base_env, **host_identity_env(), **subagent_role_env(), **leftovers.launch_env()}
+        endpoint = await fork_template.fork_launch(
+            argv, env, cwd, base_env=template_env, skip=("litellm",) if relayed else ()
+        )
+        if endpoint is not None:
+            return endpoint
+    return env
+
+
 def _retrieved(task: asyncio.Task) -> None:
     """Read the outcome of a task nobody waits on, so a failure is not reported as never retrieved."""
     if not task.cancelled():
@@ -494,6 +580,16 @@ async def launch(
         # The agent's own argv, for what this logs and reports: the spawned one
         # may lead with the wrapper that ties the agent to this host.
         program_argv = argv
+        shared = await _share_with_raven_agent(name, argv, child_env, cwd, base_env, env or {})
+        if isinstance(shared, dict):
+            child_env = shared
+        else:
+            client = cls(
+                name=name, endpoint=shared, on_request=on_request, on_notification=on_notification, journal=journal
+            )
+            client._start_loops()
+            logger.info("acp agent {!r}: forked {} from the template (pid {})", name, program_argv[:2], shared.pid)
+            return client
         argv = leftovers.bind_to_host(argv, child_env, cwd)
         job = None
         try:
@@ -594,7 +690,7 @@ async def close(self) -> None:
         if self._closed:
             return
         self._closed = True
-        if isinstance(self._endpoint, _ProcessEndpoint):
+        if hasattr(self._endpoint, "grace"):
             self._endpoint.grace = self._granted_grace()
         try:
             await self._endpoint.end()
@@ -853,6 +949,12 @@ def prompting(self) -> bool:
         """
         return any(method == "session/prompt" for method in self._request_methods.values())
 
+    @property
+    def busy(self) -> bool:
+        """Whether either side is waiting on the other: a request of raven's own
+        unanswered (a turn, a held wake included) or one of the agent's being answered."""
+        return bool(self._pending) or bool(self._answer_tasks)
+
     @property
     def idle_seconds(self) -> float:
         """Seconds since the agent last put any frame on the wire.