Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions docs/architecture/rfcs/typescript-control-plane-migration-v0.md
Original file line number Diff line number Diff line change
Expand Up @@ -2090,6 +2090,16 @@ daemon command. CLI and App surfaces consume the same lifecycle projection
(`running`, `stopped`, or `unavailable`) and stable diagnostic code. Raw stderr,
tokens, local paths, and private runtime metadata are not projected.

Node launch observation remains in the existing Python transport adapter; it is
not a second control-plane decision owner. Startup and doctor share its bounded
probe and distinguish unknown compatibility after a timeout or launch failure
from a parsed unsupported version. A later cold-start failure during deep doctor
must retain the same diagnosis and recovery, even after its initial probe
succeeded; recovery is verified by retrying the actual request. The
[host diagnostic contract](../../reference/protocols/host-integration-surface-v0.md#managed-node-startup-diagnostics)
defines the budgets, stable codes and recovery. This repairs readiness diagnosis;
sustained runtime and whole-task performance require separate evidence.

The runtime fingerprint includes every executed TS module and contract. An
upgrade starts a runtime for the new fingerprint; an old process can finish
in-flight work and exits on idle. Requests carry stable effect identities, so a
Expand Down
33 changes: 33 additions & 0 deletions docs/reference/protocols/host-integration-surface-v0.md
Original file line number Diff line number Diff line change
Expand Up @@ -209,6 +209,39 @@ Optional projections such as `task_graph_projection_v0`,
inputs to a host integration. They do not add graph write authority, launch
workers, change quota gates, or create a new source of truth.

### Managed Node startup diagnostics

The Effect transport's version probe and process readiness each have a bounded
15-second budget; the version probe previously allowed only 2 seconds. This
changes cold startup tolerance and the worst-case failed-probe wait, not work
Turn deadlines or the Node.js 22.22.3 compatibility floor. Warm requests reuse
the serving runtime without a new version probe. `loopx doctor` still observes
the current launcher independently, so replacing PATH is visible immediately.

Startup, doctor and guided start-goal share the same probe diagnosis:

A successful initial doctor check does not guarantee a later cold-start probe
will succeed. Deep doctor preserves the later probe's diagnostic and recovery
below, with host-permission recovery taking precedence. Other runtime startup
failures retain their existing recovery advice.

| Diagnostic code | Observation | Recovery |
| --- | --- | --- |
| `node_unavailable` | Node missing or a parsed version below the floor | Install or activate a qualified Node on PATH |
| `node_probe_timeout` | No completed version observation within the budget | Check host load and the launcher; compatibility remains unknown |
| `node_probe_launch_failed` | The launcher could not be executed | Repair the launcher on PATH |
| `node_probe_exit_failed` | The probe exited unsuccessfully | Repair the launcher on PATH |
| `node_probe_invalid_version` | Successful exit without a valid version | Verify the launcher's version output |
| `runtime_host_permission_denied` | Host denied the probe | Retry through host-approved runtime access |

After recovery, run `loopx doctor --deep` and retry the original request. Failed
probes do not dispatch semantic work, grant capability authority, rewrite Goal
state or publish raw output, stderr or executable paths. Timeout and cancellation
kill and reap the probe child; cancellation propagates to the caller.
The request transport does not automatically repeat a failed Node probe before
the caller can recover. Existing retries for other startup and safe transport
failures remain in place.

## Controlled Writes

Writes must be CLI-equivalent, idempotent where possible, and fail closed when
Expand Down
9 changes: 4 additions & 5 deletions loopx/cli_commands/start_goal.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
EffectRuntimeStartupError,
MINIMUM_NODE_VERSION_TEXT,
)
from ..control_plane.runtime.node_probe import node_probe_remediation
from ._host_thread import current_host_thread_id

PrintPayload = Callable[
Expand All @@ -32,11 +33,6 @@
_FINE_GRAINED_PREFIX = re.compile(r"\A--fine-grained(?P<remainder>(?:\s[\s\S]*)?)\Z")

_EFFECT_RUNTIME_STARTUP_REMEDIATION_BY_CODE = {
"node_unavailable": (
f"Install or activate Node.js {MINIMUM_NODE_VERSION_TEXT} or newer "
"on PATH, then run `loopx doctor --deep` and retry "
"`loopx start-goal --guided`."
),
"startup_lock_timeout": (
"Another LoopX TypeScript control-plane runtime may be starting. Run "
"`loopx doctor --deep`, wait for the active startup to settle, and retry "
Expand Down Expand Up @@ -69,6 +65,9 @@ def _effect_runtime_startup_recommended_action(
diagnostic_code: str,
message: str,
) -> str:
node_action = node_probe_remediation(diagnostic_code)
if node_action is not None:
return f"{node_action} Then retry `loopx start-goal --guided`."
if diagnostic_code == "invalid_idle_timeout":
# The TypeScript runtime owns the idle-timeout validation rules and
# publishes them as the typed startup message, so the projection must
Expand Down
83 changes: 31 additions & 52 deletions loopx/control_plane/effect_runtime.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,7 @@
import hashlib
import json
import os
import re
import secrets
import shutil
import socket
import subprocess
import tempfile
Expand All @@ -22,6 +20,14 @@

from ..file_lock import process_is_alive
from .runtime.file_reads import iter_binary_file_reads
from .runtime.node_probe import (
HOST_PERMISSION_RECOMMENDATION,
MINIMUM_NODE_VERSION as MINIMUM_NODE_VERSION,
MINIMUM_NODE_VERSION_TEXT as MINIMUM_NODE_VERSION_TEXT,
STARTUP_READY_TIMEOUT_SECONDS as STARTUP_READY_TIMEOUT_SECONDS,
node_probe_remediation,
probe_node as _probe_node,
)
from .content_digest import BARE_SHA256_PATTERN

EFFECT_RUNTIME_REQUEST_SCHEMA_VERSION = "loopx_effect_runtime_request_v0"
Expand All @@ -31,8 +37,6 @@
EFFECT_RUNTIME_STARTUP_ERROR_SCHEMA_VERSION = (
"loopx_effect_runtime_startup_error_v0"
)
MINIMUM_NODE_VERSION = (22, 22, 3)
MINIMUM_NODE_VERSION_TEXT = ".".join(str(part) for part in MINIMUM_NODE_VERSION)
MAX_RESPONSE_BYTES = 2 * 1024 * 1024
MAX_REQUEST_BYTES = 2 * 1024 * 1024
MAX_LOCAL_SNAPSHOT_BYTES = 64 * 1024 * 1024
Expand All @@ -47,7 +51,6 @@
})
MAX_STARTUP_DIAGNOSTIC_BYTES = 8 * 1024
STARTUP_LOCK_TIMEOUT_SECONDS = 15.0
STARTUP_READY_TIMEOUT_SECONDS = 15.0
STARTUP_POLL_SECONDS = 0.025
RUNTIME_LOCATOR_PERMISSION_RETRIES = 3
RUNTIME_RETRY_SETTLE_SECONDS = 0.25
Expand All @@ -58,7 +61,6 @@
# turns an in-flight write into an avoidable ambiguous response.
CANONICAL_AUTHORITY_WRITE_TIMEOUT_SECONDS = 45.0
CANONICAL_AUTHORITY_READ_TIMEOUT_SECONDS = 15.0
_NODE_VERSION_RE = re.compile(r"^v?(\d+)\.(\d+)\.(\d+)(?:[-+].*)?$")
_RUNTIME_SOURCE_SUFFIXES = frozenset({".json", ".ts"})
_RuntimeSourceSnapshot = tuple[tuple[str, int, int, int], ...]

Expand Down Expand Up @@ -227,11 +229,7 @@ def __init__(self, message: str, *, diagnostic_code: str) -> None:
class EffectRuntimeHostPermissionError(EffectRuntimeStartupError):
"""The host denied local runtime access before any request was dispatched."""

recommended_action = (
"retry the same registry, Goal, Agent and Turn through host-approved "
"local runtime access; do not enable optional capabilities, replace "
"authority or spend until the guard succeeds"
)
recommended_action = HOST_PERMISSION_RECOMMENDATION

def __init__(self) -> None:
super().__init__(
Expand All @@ -241,6 +239,10 @@ def __init__(self) -> None:
)


class EffectRuntimeNodeProbeError(EffectRuntimeStartupError):
"""A pre-dispatch toolchain observation requires caller recovery."""


class EffectRuntimeResponseAmbiguous(EffectRuntimeStartupError):
"""The request may have executed even though its response was lost."""

Expand Down Expand Up @@ -397,38 +399,16 @@ def _runtime_server_path() -> Path:


def _node_executable() -> str:
status, executable, _version = _probe_node()
if status != "ready" or executable is None:
raise EffectRuntimeStartupError(
f"LoopX Effect runtime requires Node.js {MINIMUM_NODE_VERSION_TEXT} "
"or newer",
diagnostic_code="node_unavailable",
probe = _probe_node(timeout=STARTUP_READY_TIMEOUT_SECONDS)
if not probe.ready:
if probe.diagnostic_code == "runtime_host_permission_denied":
raise EffectRuntimeHostPermissionError()
raise EffectRuntimeNodeProbeError(
probe.failure_message,
diagnostic_code=str(probe.diagnostic_code),
)
return executable


def _probe_node() -> tuple[str, str | None, str | None]:
executable = shutil.which("node")
if executable is None:
return "missing", None, None
try:
completed = subprocess.run(
[executable, "--version"],
check=False,
capture_output=True,
text=True, encoding="utf-8", errors="replace",
timeout=2,
)
except (OSError, subprocess.TimeoutExpired):
return "probe_failed", executable, None
match = _NODE_VERSION_RE.fullmatch(completed.stdout.strip())
version = tuple(int(part) for part in match.groups()) if match else None
if completed.returncode != 0 or version is None:
return "probe_failed", executable, None
version_text = ".".join(str(part) for part in version)
if version < MINIMUM_NODE_VERSION:
return "unsupported", executable, version_text
return "ready", executable, version_text
assert probe.executable is not None
return probe.executable


def _pid_is_alive(value: object) -> bool:
Expand Down Expand Up @@ -1044,6 +1024,7 @@ def effect_runtime_request(
EffectRuntimeRemoteError,
EffectRuntimeResponseAmbiguous,
EffectRuntimeHostPermissionError,
EffectRuntimeNodeProbeError,
):
raise
except EffectRuntimeStartupError as exc:
Expand Down Expand Up @@ -1137,10 +1118,11 @@ def _sqlite_restart_recommendation(identity: Mapping[str, Any] | None) -> str |
def collect_effect_runtime_readiness(*, deep: bool = False) -> dict[str, object]:
"""Report whether the managed TS Effect runtime can serve control-plane work."""

status, _executable, version = _probe_node()
ready = status == "ready"
probe = _probe_node(timeout=STARTUP_READY_TIMEOUT_SECONDS)
status, version = probe.status, probe.version
ready = probe.ready
runtime_state = "unavailable"
runtime_diagnostic_code: str | None = None
runtime_diagnostic_code: str | None = probe.diagnostic_code
runtime_identity: dict[str, Any] | None = None
host_permission_error: EffectRuntimeHostPermissionError | None = None
if ready:
Expand Down Expand Up @@ -1190,12 +1172,8 @@ def collect_effect_runtime_readiness(*, deep: bool = False) -> dict[str, object]
if host_permission_error is not None
else None
if ready
else (
f"Install Node.js {MINIMUM_NODE_VERSION_TEXT} or newer, then "
"rerun `loopx doctor --deep`."
if status in {"missing", "unsupported"}
else "Repair Node.js on PATH, then rerun `loopx doctor --deep`."
)
else probe.recommended_action
or "Repair the packaged LoopX runtime, then rerun `loopx doctor --deep`."
),
}
if ready:
Expand Down Expand Up @@ -1231,7 +1209,8 @@ def collect_effect_runtime_readiness(*, deep: bool = False) -> dict[str, object]
"recommended_action": (
exc.recommended_action
if isinstance(exc, EffectRuntimeHostPermissionError)
else "Run `loopx doctor --deep` again after any concurrent startup "
else node_probe_remediation(diagnostic_code)
or "Run `loopx doctor --deep` again after any concurrent startup "
"finishes. If the same diagnostic code remains, reinstall LoopX "
"and verify Node.js before retrying."
),
Expand Down
130 changes: 130 additions & 0 deletions loopx/control_plane/runtime/node_probe.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
"""Physical Node launch observation for the existing Effect transport.

This adapter does not select capabilities or authorize work. Startup and doctor
share its observations and public-safe remediation; neither interprets a failed
version probe as an unsupported version.
"""
from __future__ import annotations

import re
import shutil
import subprocess
from dataclasses import dataclass
from enum import StrEnum

MINIMUM_NODE_VERSION = (22, 22, 3)
MINIMUM_NODE_VERSION_TEXT = ".".join(str(part) for part in MINIMUM_NODE_VERSION)
STARTUP_READY_TIMEOUT_SECONDS = 15.0
HOST_PERMISSION_RECOMMENDATION = (
"retry the same registry, Goal, Agent and Turn through host-approved "
"local runtime access; do not enable optional capabilities, replace "
"authority or spend until the guard succeeds"
)
_VERSION_RE = re.compile(r"^v?(\d+)\.(\d+)\.(\d+)(?:[-+].*)?$")


class NodeProbeOutcome(StrEnum):
"""Local process facts, projected onto the existing readiness statuses."""

READY = "ready"
MISSING = "missing"
UNSUPPORTED = "unsupported"
TIMED_OUT = "timed_out"
PERMISSION_DENIED = "permission_denied"
LAUNCH_FAILED = "launch_failed"
EXIT_FAILED = "exit_failed"
INVALID_VERSION = "invalid_version"


_FAILURES = {
NodeProbeOutcome.MISSING: (
"node_unavailable", f"LoopX Effect runtime requires Node.js {MINIMUM_NODE_VERSION_TEXT} or newer on PATH",
),
NodeProbeOutcome.UNSUPPORTED: (
"node_unavailable", f"LoopX Effect runtime requires Node.js {MINIMUM_NODE_VERSION_TEXT} or newer",
),
NodeProbeOutcome.TIMED_OUT: (
"node_probe_timeout", "Node.js version probe exceeded the bounded startup budget; compatibility is unknown",
),
NodeProbeOutcome.PERMISSION_DENIED: (
"runtime_host_permission_denied", "Host permission denied the Node.js version probe before request dispatch",
),
NodeProbeOutcome.LAUNCH_FAILED: (
"node_probe_launch_failed", "Node.js version probe could not be launched; compatibility is unknown",
),
NodeProbeOutcome.EXIT_FAILED: (
"node_probe_exit_failed", "Node.js version probe exited unsuccessfully; compatibility is unknown",
),
NodeProbeOutcome.INVALID_VERSION: (
"node_probe_invalid_version", "Node.js version probe returned no valid version; compatibility is unknown",
),
}
_REMEDIATION = {
"node_unavailable": f"Install or activate Node.js {MINIMUM_NODE_VERSION_TEXT} or newer on PATH, then run `loopx doctor --deep`.",
"node_probe_timeout": "Check host load and the Node.js launcher on PATH, then rerun `loopx doctor --deep`. A timed-out probe does not establish version incompatibility.",
"node_probe_launch_failed": "Repair the Node.js launcher on PATH, then rerun `loopx doctor --deep`.",
"node_probe_exit_failed": "Repair the Node.js launcher on PATH, then rerun `loopx doctor --deep`.",
"node_probe_invalid_version": "Verify that the Node.js launcher on PATH returns a valid version, then rerun `loopx doctor --deep`.",
"runtime_host_permission_denied": HOST_PERMISSION_RECOMMENDATION,
}


def node_probe_remediation(diagnostic_code: str) -> str | None:
return _REMEDIATION.get(diagnostic_code)


@dataclass(frozen=True)
class NodeProbe:
outcome: NodeProbeOutcome
executable: str | None = None
version: str | None = None

@property
def ready(self) -> bool:
return self.outcome is NodeProbeOutcome.READY

@property
def status(self) -> str:
if self.outcome in {NodeProbeOutcome.READY, NodeProbeOutcome.MISSING, NodeProbeOutcome.UNSUPPORTED}:
return self.outcome.value
return "probe_failed"

@property
def diagnostic_code(self) -> str | None:
failure = _FAILURES.get(self.outcome)
return failure[0] if failure else None

@property
def failure_message(self) -> str:
return _FAILURES[self.outcome][1]

@property
def recommended_action(self) -> str | None:
return node_probe_remediation(self.diagnostic_code or "")


def probe_node(*, timeout: float = STARTUP_READY_TIMEOUT_SECONDS) -> NodeProbe:
executable = shutil.which("node")
if executable is None:
return NodeProbe(NodeProbeOutcome.MISSING)
try:
# run kills and waits for the probe child on timeout/cancellation. No
# retry, version cache, shell or semantic request is involved.
completed = subprocess.run(
[executable, "--version"], check=False, capture_output=True,
text=True, encoding="utf-8", errors="replace", timeout=timeout,
)
except subprocess.TimeoutExpired:
return NodeProbe(NodeProbeOutcome.TIMED_OUT, executable)
except PermissionError:
return NodeProbe(NodeProbeOutcome.PERMISSION_DENIED, executable)
except OSError:
return NodeProbe(NodeProbeOutcome.LAUNCH_FAILED, executable)
if completed.returncode != 0:
return NodeProbe(NodeProbeOutcome.EXIT_FAILED, executable)
match = _VERSION_RE.fullmatch(completed.stdout.strip())
if match is None:
return NodeProbe(NodeProbeOutcome.INVALID_VERSION, executable)
version = tuple(int(part) for part in match.groups())
outcome = NodeProbeOutcome.UNSUPPORTED if version < MINIMUM_NODE_VERSION else NodeProbeOutcome.READY
return NodeProbe(outcome, executable, ".".join(str(part) for part in version))
Loading
Loading