diff --git a/.orca/reviewers/orca.md b/.orca/reviewers/orca.md index d57dbec84..bbd36cb29 100644 --- a/.orca/reviewers/orca.md +++ b/.orca/reviewers/orca.md @@ -14,12 +14,14 @@ performance, tests — belongs to the shipped reviewers running beside you. - **Forge neutrality**: interfaces and persisted documents name git concepts, never a forge; a forge-specific type stays in the package that talks to it. - **New mutable state is a design question**: a new `var` field, mutable - collection or `AtomicReference` needs the task description or a comment at + collection, `AtomicReference` or `orca.concurrency.Ref` needs the task description or a comment at the declaration to name the alternatives (actor, method-local state, an `AtomicReference` over an immutable value) and why each was rejected — report the missing rationale, not the state itself. Actor-held state and an - `AtomicReference` over an immutable value updated by a pure function are the - sanctioned forms. + `AtomicReference` (`orca.concurrency.Ref` in safe modules) over an immutable value + updated by a pure function are the sanctioned forms; never + `caps.SharedCapability` + `caps.Stateful`. Safe mode still compiles + `@volatile` and `synchronized`, so flag them as before. - **No back-compat machinery**, and no default values on domain or persisted fields. - **Failure types**: a recoverable `Either[E, T]` has `E <: OrcaFlowException`; @@ -42,10 +44,12 @@ performance, tests — belongs to the shipped reviewers running beside you. WorkspaceWrite)` to make code compile; `WorkspaceWrite` never crosses a fork; a new gated write calls `WorkspaceWrite.check` first. - **Writes under `.orca/`**: a whole-file write replaces an `OrcaDir.OrcaFile`; - any other write goes inside an `OrcaDir.ensure*` directory, with `os.write` - over `os.write.over`. A new file gets a row in AGENTS.md's "What a run writes + any other write goes inside an `OrcaDir.ensure*` directory, with + `fs.writeNew` over `fs.writeOver` (plain modules: `os.write` over + `os.write.over`). A new file gets a row in AGENTS.md's "What a run writes to disk". -- **Subprocesses capture stderr** — `QuietProc.call` or a `CliRunner`; a +- **Subprocesses capture stderr** — `Processes`, a `CliRunner`, or + `QuietProc` in plain modules (runner/shell); a `spawnPiped` caller drains `stderrLines` as well as `stdoutLines`. Raw `os.proc` with `os.Inherit` only in the `shell/` terminal handoffs and `TtyProbe`. diff --git a/.scalafmt.conf b/.scalafmt.conf index a6d656a48..750fede41 100644 --- a/.scalafmt.conf +++ b/.scalafmt.conf @@ -1,9 +1,2 @@ -version = 3.8.3 -runner.dialect = scala3 - -# scalafmt 3.8.3's scala3 dialect parser cannot tokenize capture-checking -# syntax (the `C^` capture-set type parameter and `->{C}` capture arrows in -# CheckedPar.scala), failing before any `// format: off` region is consulted. -# Exclude that one file until scalafmt gains capture-checking support; it is -# hand-formatted to match the surrounding style. -project.excludeFilters = ["tools/src/main/scala/orca/CheckedPar.scala"] +version = 3.11.5 +runner.dialect = scala3future diff --git a/AGENTS.md b/AGENTS.md index 90f6c9527..4d93c2837 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -6,38 +6,22 @@ orca live in [CONTRIBUTING.md](CONTRIBUTING.md); end-user documentation at [orca.virtuslab.com](https://orca.virtuslab.com) (source in `docs/`). Orca is implemented in Scala 3 on top of [Ox](https://ox.softwaremill.com/) -for structured concurrency, [tapir](https://tapir.softwaremill.com/) for -JSON Schema derivation, and +for structured concurrency, [tapir](https://tapir.softwaremill.com/)'s +schema model to describe and render JSON Schema (derived by +`orca.json.SchemaDerivation`, in `core`), and [jsoniter-scala](https://github.com/plokhotnyuk/jsoniter-scala) for codec generation. ## Project layout -``` -orca/ -├── build.sbt / project/ -├── tools/ # tool traits + os-backed impls (git/gh/fs), internal LLM SPI + session durability, InStage, events, subprocess -├── flow/ # stage/display/fail + FlowContext/FlowControl; orca.{plan,review,pr,progress} -├── claude/ codex/ gemini/ opencode/ pi/ # one module per coding-agent backend -├── runner/ # flow() entry, DefaultFlowContext/DefaultFlowControl, FlowLifecycle, terminal UI -└── shell/ # `orca-shell`, the `orca` CLI's interactive front-end (ADR 0021) -``` - -Dependency graph: - -``` -tools (standalone) - ├── flow → tools - ├── claude / codex / gemini / - │ opencode / pi → tools - ├── runner → tools + flow + all five backends - └── shell → runner (published `orca`) -``` +The module tree and dependency graph are in +[docs/development.md](docs/development.md#layout). The runner module owns the `flow` entry point (`package orca`) and wires defaults via `DefaultFlowContext` and `DefaultFlowControl` (`package orca.runner`); the stage -setup/teardown/recovery state machine is `orca.runner.FlowLifecycle`. The -terminal UI lives in `orca.runner.terminal`, behind an `Interaction`, so +setup/teardown/recovery state machine is `orca.lifecycle.FlowLifecycle`, in the +`lifecycle` module; what runner and shell use from it is `private[orca]`. +The terminal UI lives in `orca.runner.terminal`, behind an `Interaction`, so swapping it for a Slack or HTTP equivalent is one substitution at the call site rather than rewiring modules. @@ -48,14 +32,77 @@ backend-agnostic role accessors (ADR 0020) — `stage`/`display`/`fail`, Implementations live in focused subpackages: `orca.tools` (os-backed git/gh/fs impls + their traits), `orca.gitref` (validated branch names and commit hashes, and `Head`), -`orca.json` (`JsonData`, `RawJson`, whole-file `JsonFile` documents), +`orca.json` (`JsonData` and `JsonCodec` in core; `RawJson`, whole-file +`JsonFile` documents), `orca.agents` + `orca.backend` (LLM SPI, `SessionSupport`, per-backend decoders; the SPI is `private[orca]` or `private[backend]`, so a new harness is added in orca itself), `orca.subprocess` (subprocess shim), `orca.events` -(event bus), one `orca.backend.` per coding agent, and `orca.runner` / -`orca.runner.terminal` (wiring + terminal UI). The flow module adds +(event bus), one `orca.backend.` per coding agent, `orca.lifecycle` +(run setup, recovery and teardown), and `orca.runner` / `orca.runner.terminal` +(wiring + terminal UI). The flow module adds `orca.{plan,review,pr,progress}`. +### Safe mode + +Modules are moving, bottom-up, onto Scala's safe mode, where code may only +reference safe-mode or `@assumeSafe` definitions. The main sources of `tools`, +`flow`, `lifecycle` and the five backends compile with +`-language:experimental.safe` (their tests do not); `runner` and `shell` are +plain (wiring, terminal, logging, file locks). `core` is the trusted base, and the only module where +`@assumeSafe` appears (`AssumeSafeScopeTest`): + +- Capabilities (`orca.effect`): `FileSystem`, `Processes`, `Network`. Each + is a `caps.SharedCapability` whose members perform the effect; its + `ambient` mint is `private[orca]` and not `@assumeSafe`, so only wiring and + tests create one. +- Facades over libraries safe code can't reference: `orca.safe.{prelude, + Logger}`, `orca.concurrency` (with `Ref`), `orca.mcp`, `orca.json.{JsonData, + JsonCodec, JsonSchemas}`. + Safe code uses `derives JsonCodec` where plain code uses `derives + ConfiguredJsonValueCodec`. +- Effect implementations (`QuietProc`, `OsPipedSubProcess`, `LocalServer`, + `JdkHttpClient`, `CookieScan`) are not `@assumeSafe`; safe code reaches them + through a capability only (`SafeModeExposureTest`). A handle a capability + returns (`PipedCliProcess`, `HttpClient`) is untracked: code holding one + performs its effects without naming the capability. + +In safe modules, effectful classes (`OsGitTool`, `Worktrees`, `OrcaDir`, +`JsonFile`, `OsFsTool`, the backends) take capabilities as constructor +parameters; wiring (`flowEntry`, the `*Agents` objects, the shell) builds them +with `X.ambient`. `lifecycle` takes the terminal (`tty`, `ask`), the PATH +probe and the worktree file lock (`WorktreeLock`) as values from `flowEntry`. Safe code can't turn a class holding a capability into a +plain trait such as `AgentBackend` or `CliRunner`, so each safe backend's +`*Agents` object, and its `*AgentsTest`, lives in `runner`, in the backend's +package (`orca.backend.`). A session probe that reads the disk is +built there too and passed to the backend (`CodexBackend.rolloutProbe`), and +so is a factory for the `GitTool` the backend reads through: a backend starts +processes only through its `CliRunner`. A probe or factory that needs a +directory takes it as an argument, and the backend passes its own `workDir`. + +Safe mode does not check member calls on values, so it catches ambient +effects reached by accident; it is not a sandbox. `JsonData` derives schemas +with `SchemaDerivation`, because tapir's Magnolia `Schema.derived` fails +capture checking. It ignores tapir's schema annotations, and the +`*SchemaParityTest` suites pin its output to tapir's; `SchemaParityCoverageTest` +fails when a derivation site has no parity entry. + +The prelude's `pipe` and `tap` are inline, and Scala 3.10.0 types an inline +argument's proxy without its captures: on a value that captures a capability +they can fail capture checking ("Found `X^{fs}`, Required `X`"). Use a named +`val` there instead. + +In 3.10.0, `List.maxByOption` takes a pure function, so a key that uses a +capability fails ("capability fs cannot flow into capture set {}"): `map` to +`(value, key)` pairs first, then `maxByOption(_._2)`. + +Scala 3.10.0's `SafeRefs.checker` shares a mutable flag across compiler +instances in one JVM, so concurrent safe-mode compiles report false "Cannot +refer to ..." errors; `build.sbt` compiles one module at a time. The flag can +stay stuck in a long-lived sbt server: if those errors appear, `sbt shutdown`. + +Migrating a module: sbt reports only 100 errors; `set ThisBuild / maxErrors +:= 2000` shows them all. + ## The stage-bound runtime The flow runtime is specified in [ADR 0018](adr/0018-stage-bound-flow-runtime.md) — @@ -64,7 +111,8 @@ most easily broken: - **Capability gating.** `FlowContext` (reads + emit; thread-safe) is not a capability, so forks receive it freely. Three compile-time capabilities gate - side effects: `FlowControl` (authority to start a stage; thread-affine; + flow-level side effects (starting stages, LLM runs, workspace writes), + separate from core's `FileSystem`/`Processes`/`Network`: `FlowControl` (authority to start a stage; thread-affine; unrelated to `FlowContext` — `flow` provides both, and a stage-starting helper takes `(using FlowContext, FlowControl)`), and a SPLIT pair of stage-bound capability tokens (both in `tools`, `package orca`) — `InStage`, @@ -385,11 +433,16 @@ with a test pinning the observed wire shape. ### Scala style - Braceless syntax; explicit return types on every public member. -- No class-level `var`s outside an Ox actor's state. Mutable state whose - updates also write to the terminal or files lives in an actor; other - mutable state stays in method bodies, or in an `AtomicReference` over an - immutable value updated by a pure function (`CostTracker`, - `DeniedToolTracker`). +- Remove mutable state first: use a fork's return value, a fold, or state a + channel or scope already holds. No class-level `var`s outside an Ox actor's + state. Mutable state whose updates also write to the terminal or files lives + in an actor; other mutable state stays in method bodies, or, when shared + across threads, in an `AtomicReference` over an immutable value updated by a + pure function (`CostTracker`, `DeniedToolTracker`) — `orca.concurrency.Ref` in safe + modules. Safe mode rejects `var` fields, so there an actor's state, and + state confined to one thread (`StageFrames`), is a `Ref` too. Never `caps.SharedCapability` + `caps.Stateful` for it: that tracks + nothing. `@volatile` and `synchronized` still compile in safe mode, so review + is the gate. - Opaque-type aliases for domain string labels (e.g. `Title`, `SessionId`). - Recoverable failures return `Either[E, T]` where `E <: OrcaFlowException`; system failures throw. Use Ox's `.orThrow` at the call site when the @@ -558,8 +611,8 @@ number, change it only additively (ADR 0025). [ADR 0010](adr/0010-prompts-and-helpers-convention.md): sibling `XxxPrompts` object + `instructions: String = …` parameter. - Subprocesses launched from a tool **must** capture stderr — go through - [`subprocess.QuietProc.call`](tools/src/main/scala/orca/subprocess/QuietProc.scala) - or a `CliRunner`. os-lib defaults `os.proc(...).call(...)`'s `stderr` to + [`effect.Processes`](core/src/main/scala/orca/effect/Processes.scala), + a `CliRunner`, or `QuietProc` in plain modules (runner/shell). os-lib defaults `os.proc(...).call(...)`'s `stderr` to `Inherit`, which lets subprocess output bypass the renderer's StatusBar and tear the spinner row. `CliRunner.spawnPiped` always pipes stderr, so its caller must drain `stderrLines` as well as `stdoutLines`. Raw `os.proc` @@ -567,8 +620,9 @@ number, change it only additively (ADR 0025). run, agent session) and `TtyProbe`, which inherits the fd it probes. - Every `spawnPiped` child carries a unique `ORCA_TURN_COOKIE` (`orca.subprocess.EnvCookie`). `SubprocessSpawn.open` registers `EnvCookieSweep` - with the turn scope, so at turn end it scans `/proc/*/environ` for the - agent's cookie and REPORTS what is still running — the backstop for work an + with the turn scope, so at turn end it asks the process's `survivors()` (a + `/proc/*/environ` scan) for what carries the agent's cookie and REPORTS what is still + running — the backstop for work an agent detached from orca's process tree, which no parent-link teardown can reach. opencode's turns spawn nothing (they run on the per-run server), so they are not swept; the server's cookie is swept when the server stops. @@ -582,8 +636,9 @@ number, change it only additively (ADR 0025). `OrcaDir.OrcaFile` (only `OrcaDir` creates one) and its `replace`, whose rename replaces a leaf symlink instead of following it. Other writes (appends, lock files) go inside a directory an `OrcaDir.ensure*` accessor returned, with - `os.write` (`CREATE_NEW`, refuses a leaf symlink) over `os.write.over`; a lock - file opens with `NOFOLLOW_LINKS`. The + `fs.writeNew` (`CREATE_NEW`, refuses a leaf symlink) over `fs.writeOver` + (plain modules: `os.write` over `os.write.over`); a lock file opens with + `NOFOLLOW_LINKS`. The check runs at the earliest `.orca` touch (`FlowLock` → `ensureCache`), ahead of any mutation. diff --git a/CLAUDE.md b/CLAUDE.md index b6589da0d..927c75448 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -9,7 +9,7 @@ invoke the `direct-style-scala` skill (via the Skill tool). This applies to subagents dispatched to implement or review a task: load the skill first, then start the work. -When a change would add mutable state — a `var`, a mutable collection or an -`AtomicReference` field — research the alternatives in a separate agent and +When a change would add mutable state — a `var`, a mutable collection, an +`AtomicReference` or an `orca.concurrency.Ref` field — research the alternatives in a separate agent and have a second agent critique the recommendation before implementing either. `.orca/reviewers/orca.md` says what the result must record. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 14aaee75b..8ad9cd030 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -72,10 +72,11 @@ rewrite (the flows resolve from Maven Central), so it skips itself. sbt publishLocal ``` -Installs `org.virtuslab::orca:0.0.17` plus its transitive modules -(`orca-tools`, `orca-flow`, and the five backends -`orca-{claude,codex,gemini,opencode,pi}`) into `~/.ivy2/local` so a flow -script with `//> using repository ivy2Local` can resolve them. For an +Installs every module into `~/.ivy2/local`: `org.virtuslab::orca` (the runner), +the modules it depends on (`orca-core`, `orca-tools`, `orca-flow`, +`orca-lifecycle`, and the five backends `orca-{claude,codex,gemini,opencode,pi}`), +and `orca-shell`. A flow script with `//> using repository ivy2Local` can then +resolve them. For an iteration loop, `sbt "~publishLocal"` rebuilds and republishes on every save. diff --git a/README.md b/README.md index b1e87d32a..60814130a 100644 --- a/README.md +++ b/README.md @@ -95,7 +95,7 @@ Flows are ordinary Scala scripts. For example, the below one plans, implements each task with a review, and opens a PR: ```scala -//> using scala 3.9.0 +//> using scala 3.10.0 //> using dep "org.virtuslab::orca:0.2.1" //> using jvm 21 diff --git a/adr/0025-run-event-log.md b/adr/0025-run-event-log.md index 85f0d8637..e8f43bf21 100644 --- a/adr/0025-run-event-log.md +++ b/adr/0025-run-event-log.md @@ -42,7 +42,8 @@ stay as log files; the event log points to them. - **One writer per attempt, asynchronous.** The attempt keeps an in-memory projection of the log, loaded from the file at start (empty if absent). The projection is an immutable value held by an actor; the run reads it through - that actor. A second actor appends events to the file in the background. + that actor. The actor queues each event on a channel, and one writer fork + drains it, appending to the file in the background. Writes lost in a crash are acceptable: this is a cache. - **Run boundary in the log.** A run of the same prompt reuses the key and the file. Success appends `RunSucceeded`; session lookup ignores everything before @@ -85,7 +86,7 @@ stay as log files; the event log points to them. - The session store's writes are no longer a read-modify-write of one file, so `upsert` drops `WorkspaceWrite`; the actor serialises updates. - Queued appends are lost when the attempt's scope ends, so finishing an - attempt waits for the appender's queue to empty. + attempt closes the queue and waits for the writer to drain it. - No compatibility with existing cache files is owed; old manifests stop being listed. - Supersedes ADR 0021 §8's file layout (manifest and cost log) and the location diff --git a/build.sbt b/build.sbt index 631c53c51..f3cd1daf4 100644 --- a/build.sbt +++ b/build.sbt @@ -18,6 +18,16 @@ scalacOptions ++= Seq( javacOptions ++= Seq("--release", "21") +// Scala 3.10.0's safe-mode checker (`SafeRefs.checker`) shares a mutable flag +// across compiler instances in one JVM, so concurrent safe-mode compiles report +// false "Cannot refer to ..." errors. Remove once fixed upstream. +Global / concurrentRestrictions += Tags.limit(Tags.Compile, 1) + +// tapir's Magnolia `Schema.derived`, which the schema-parity tests compare +// against, inlines `Predef.ArrowAssoc`, deprecated since Scala 3.10; `@nowarn` +// can't reach a warning positioned in inlined code. +Test / scalacOptions += "-Wconf:cat=deprecation&msg=ArrowAssoc:s" + // Makes sbt-git answer its read-only queries with the `git` CLI instead of the // bundled JGit, which reads a linked worktree's `.git` *file* as a bare repo and // aborts project load with NoWorkTreeException — without this, sbt does not @@ -34,6 +44,10 @@ useReadableConsoleGit lazy val buildVersionProperty = Def.setting("-Dorca.build.version=" + version.value) +// The Scala version flow scripts pin, for tests that check or use the pin. +lazy val scalaVersionProperty = + Def.setting("-Dorca.build.scalaVersion=" + scalaVersion.value) + // sbt 2 runs forked tests through its own worker, so `java.class.path` in the // test JVM holds only that worker's jars. Tests that start a compiler or a // child JVM on the Test classpath read it from this property instead. @@ -86,40 +100,71 @@ developers := List( ) ) -lazy val tools = (project in file("tools")) +// Main sources only: tests mint capabilities and use os-lib directly. +lazy val safeMode = Seq( + Compile / scalacOptions += "-language:experimental.safe", + Test / scalacOptions -= "-language:experimental.safe" +) + +// The trusted base: effect capabilities, their implementations and facades +// over libraries safe code cannot reference. The only module where +// `@assumeSafe` may appear (`AssumeSafeScopeTest`). +lazy val core = (project in file("core")) .settings( - name := "orca-tools", + name := "orca-core", + // NetworkTest asserts no Netty threads outlive its server, so no other + // suite may run one at the same time. + Test / parallelExecution := false, + Test / javaOptions += testClasspathProperty.value, libraryDependencies ++= Seq( osLib, jsoniter, jsoniterMacros, + ox, + tapirNettySync, + // `JsonSchemas` renders tapir schemas as JSON Schema. tapirApispec, sttpApispecCirce, + // The `mcp` facade. + chimp, + slf4j, + scala3Compiler + ) + ) + +lazy val tools = (project in file("tools")) + .dependsOn(core, core % "test->test") + .settings(safeMode*) + .settings( + name := "orca-tools", + libraryDependencies ++= Seq( + osLib, + jsoniter, + jsoniterMacros, ox, jsonSchemaValidator, - // The shared MCP server (orca.backend.mcp.AskUserMcpServer) is consumed - // by the claude, codex and gemini backends, so chimp-server + the netty - // backend live here — tools already hosts the cross-backend - // orca.backend package — rather than in any single backend module. - chimp, - tapirNettySync + // `McpHostTest` calls a chimp tool's logic directly. + chimp % Test ) ) lazy val claude = (project in file("claude")) .dependsOn(tools, tools % "test->test") + .settings(safeMode*) .settings( name := "orca-claude", libraryDependencies ++= Seq( osLib, jsoniter, jsoniterMacros, - logback + // Binds chimp/tapir/netty's SLF4J calls in the MCP-server tests. + logback % Test ) ) lazy val codex = (project in file("codex")) .dependsOn(tools, tools % "test->test") + .settings(safeMode*) .settings( name := "orca-codex", libraryDependencies ++= Seq(osLib, jsoniter, jsoniterMacros) @@ -127,16 +172,17 @@ lazy val codex = (project in file("codex")) lazy val pi = (project in file("pi")) .dependsOn(tools, tools % "test->test") + .settings(safeMode*) .settings( name := "orca-pi", libraryDependencies ++= Seq(osLib, jsoniter, jsoniterMacros) ) // The OpenCode backend talks HTTP+SSE to a headless `opencode serve` (ADR -// 0014); it uses the JDK's java.net.http client (no extra dependency) behind a -// small testable trait, plus jsoniter for the wire DTOs. +// 0014), through `Network.httpClient`. lazy val opencode = (project in file("opencode")) .dependsOn(tools, tools % "test->test") + .settings(safeMode*) .settings( name := "orca-opencode", libraryDependencies ++= Seq(osLib, jsoniter, jsoniterMacros) @@ -144,6 +190,7 @@ lazy val opencode = (project in file("opencode")) lazy val gemini = (project in file("gemini")) .dependsOn(tools, tools % "test->test") + .settings(safeMode*) .settings( name := "orca-gemini", libraryDependencies ++= Seq(osLib, jsoniter, jsoniterMacros) @@ -151,20 +198,36 @@ lazy val gemini = (project in file("gemini")) lazy val flow = (project in file("flow")) .dependsOn(tools, tools % "test->test") + .settings(safeMode*) .settings( name := "orca-flow", Test / javaOptions += testClasspathProperty.value, + // Not `logback-test.xml`: through `flow % "test->test"` that would + // override runner's own logback.xml in runner's tests. + Test / javaOptions += "-Dlogback.configurationFile=flow-logback-test.xml", libraryDependencies ++= Seq( ox, jsoniterMacros, jsonSchemaValidator, scala3Compiler, - munitScalacheck + munitScalacheck, + // Lets tests capture what orca logs. + logback % Test ) ) +// A run's setup, recovery and teardown; `runner` wires it. +lazy val lifecycle = (project in file("lifecycle")) + .dependsOn(tools, tools % "test->test", flow) + .settings(safeMode*) + .settings( + name := "orca-lifecycle", + libraryDependencies ++= Seq(osLib, jsoniter, jsoniterMacros, ox) + ) + lazy val runner = (project in file("runner")) .dependsOn( + lifecycle, tools, tools % "test->test", flow, @@ -193,6 +256,7 @@ lazy val runner = (project in file("runner")) // semantics the guard exists to enforce. Test / parallelExecution := false, Test / javaOptions += buildVersionProperty.value, + Test / javaOptions += scalaVersionProperty.value, Test / javaOptions += testClasspathProperty.value, // `flow(...)` resolves the global settings file and reviewer directory from // the config home, so the tests here that drive it would otherwise read — @@ -211,6 +275,7 @@ lazy val runner = (project in file("runner")) ox, mainargs, jline, + logback, fansi, jsoniterMacros, munitScalacheck @@ -218,11 +283,12 @@ lazy val runner = (project in file("runner")) ) lazy val shell = (project in file("shell")) - // `pi` and `gemini` are already on the classpath transitively via `runner`, - // but declared explicitly since shell reads pi's session store + // `lifecycle`, `pi` and `gemini` are already on the classpath transitively + // via `runner`, but declared explicitly since shell uses them directly: + // lifecycle's `RoleAgents` and `RunTarget`, and pi's session store // (`PiSessionStore`) and gemini's session listing (`GeminiSessionList`) to // build the resume argv for a recorded chat (ADR 0021 §8). - .dependsOn(runner, pi, gemini, tools % "test->test") + .dependsOn(runner, lifecycle, pi, gemini, tools % "test->test") .settings( name := "orca-shell", Compile / mainClass := Some("orca.shell.Main"), @@ -245,15 +311,17 @@ lazy val shell = (project in file("shell")) // concurrently running suite observes the temporarily-ignored handler — // the same isolation runner uses for its lock and logger state. Test / parallelExecution := false, + Test / javaOptions += scalaVersionProperty.value, // The shell's own version (`OrcaBuild.current`), as a resource rather than // the jar manifest, so a class-directory run (tests, `sbt shell/run`) knows // it too — the same version a `publishLocal` in the same sbt session - // publishes. + // publishes. Next to it, the Scala version flows pin + // (`OrcaBuild.scalaVersion`). Compile / resourceGenerators += Def.task { - val file = - (Compile / resourceManaged).value / "orca" / "shell" / "version" - IO.write(file, version.value) - Seq(file) + val dir = (Compile / resourceManaged).value / "orca" / "shell" + IO.write(dir / "version", version.value) + IO.write(dir / "scala-version", scalaVersion.value) + Seq(dir / "version", dir / "scala-version") }.taskValue, // Bundles the top-level flows/*.sc scripts as jar resources under // orca/shell/flows/ (ADR 0021 §7), so `BuiltInFlows` can extract them to a @@ -357,4 +425,4 @@ lazy val orcaRoot = rootProject ) } ) - .aggregate(tools, flow, claude, codex, opencode, pi, gemini, runner, shell) + .aggregate(core, tools, flow, lifecycle, claude, codex, opencode, pi, gemini, runner, shell) diff --git a/claude/src/main/scala/orca/backend/claude/ClaudeBackend.scala b/claude/src/main/scala/orca/backend/claude/ClaudeBackend.scala index f411bd142..4f5e224d8 100644 --- a/claude/src/main/scala/orca/backend/claude/ClaudeBackend.scala +++ b/claude/src/main/scala/orca/backend/claude/ClaudeBackend.scala @@ -18,8 +18,7 @@ import orca.backend.{ IdScheme, SessionSupport, SubprocessSpawn, - SystemPromptComposer, - TurnResources + SystemPromptComposer } import orca.subprocess.CliRunner import orca.backend.mcp.{ @@ -29,8 +28,11 @@ import orca.backend.mcp.{ McpHost, RepoMcpServer } -import orca.tools.{GitTool, OsGitHubTool, OsGitTool} +import orca.effect.{FileSystem, Network} +import orca.tools.{GitTool, OsGitHubTool} import orca.backend.claude.streamjson.OutboundMessage +import orca.concurrency.defaultBufferCapacity +import orca.safe.prelude.given import ox.{Ox, ResourceScope} import scala.concurrent.duration.FiniteDuration @@ -54,15 +56,21 @@ private[claude] final case class TurnMcp( * the backend, so a long flow doesn't accumulate them. */ private[orca] class ClaudeBackend( + net: Network, + fs: FileSystem, cli: CliRunner, - private[claude] val projectsDir: os.Path = os.home / ".claude" / "projects", + /** Builds the `GitTool` behind the repo-read MCP tools, given [[workDir]]: + * they read the repository the agent is looking at. + */ + gitAt: os.Path -> GitTool, + /** [[ClaudeBackend.transcriptProbe]] in production. */ + transcriptExists: (os.Path, String) -> Boolean, /** Shared between the spawn path ([[open]]) and the existence probe (see - * [[sessions]]): claude writes a session's transcript under - * `//.jsonl`, so both sides reading the - * SAME field keeps the probe honest. The `os.pwd` default serves only - * bare/test construction; the runtime passes the flow's real `workDir`. + * [[sessions]]): claude writes a session's transcript under a directory + * derived from its cwd, so both sides reading the SAME field keeps the + * probe honest. */ - override val workDir: os.Path = os.pwd + override val workDir: os.Path ) extends AgentBackend[BackendTag.ClaudeCode.type]: /** Claude's sessions live on disk (`~/.claude/projects/.../.jsonl`) and @@ -75,14 +83,10 @@ private[orca] class ClaudeBackend( */ val tag: BackendTag.ClaudeCode.type = BackendTag.ClaudeCode - /** Backs the repo-read MCP tools. Derived from `workDir` rather than injected - * so it cannot disagree with the directory claude is spawned in — the whole - * point of the tools is to read the repository the agent is looking at. - */ - private val git: GitTool = new OsGitTool(workDir) - export ClaudeArgs.enforcementCell + private val git: GitTool = gitAt(workDir) + override def structuredOutputMode: StructuredOutputMode = StructuredOutputMode.Tool @@ -96,10 +100,7 @@ private[orca] class ClaudeBackend( val sessions: SessionSupport[BackendTag.ClaudeCode.type] = SessionSupport.durable( IdScheme.ClientClaimed, - id => - os.exists( - projectsDir / ClaudeBackend.cwdSlug(workDir) / s"$id.jsonl" - ) + id => transcriptExists(workDir, id) ) /** Spawn `claude` in stream-json mode, write the opening user message, close @@ -122,7 +123,7 @@ private[orca] class ClaudeBackend( )(using Ox): LiveTurn[BackendTag.ClaudeCode.type] = import turn.* val askUser: Option[AskUserSession] = - Option.when(mode.isInteractive)(AskUserSession.allocate()) + Option.when(mode.isInteractive)(AskUserSession.allocate(net)) val servers = turnServers(config.tools) val mcpConfig = Option.when(askUser.isDefined || servers.nonEmpty): writeMcpConfig(askUser.map(_.server), servers) @@ -180,7 +181,7 @@ private[orca] class ClaudeBackend( slugs = RepoMcpServer.ToolSlugs, timeout = RepoMcpServer.ToolTimeout, hint = RepoMcpServer.Hint, - host = RepoMcpServer.start(git) + host = RepoMcpServer.start(net, git) ) // `NetworkOnly` only, so reviewers stay network-free. val githubReads = Option.when(tools.hasScopedNetwork): @@ -189,7 +190,7 @@ private[orca] class ClaudeBackend( slugs = GitHubMcpServer.ToolSlugs, timeout = GitHubMcpServer.ToolTimeout, hint = GitHubMcpServer.Hint, - host = GitHubMcpServer.start(new OsGitHubTool(cli, workDir)) + host = GitHubMcpServer.start(net, new OsGitHubTool(cli, workDir)) ) List(repoReads, githubReads).flatten @@ -219,7 +220,7 @@ private[orca] class ClaudeBackend( McpConfig.HttpServer(host.url, AskUserMcpServer.ToolTimeout) ) ++ servers.map(s => s.name -> McpConfig.HttpServer(s.host.url, s.timeout)) - TurnResources.tempFile( + fs.tempFile( McpConfig.render(entries.toMap), prefix = "orca-mcp-", suffix = ".json" @@ -234,7 +235,7 @@ private[orca] class ClaudeBackend( config: AgentConfig, hints: List[String] )(using ResourceScope): os.Path = - TurnResources.tempFile( + fs.tempFile( SystemPromptComposer.combine(config, hints.reduceOption(_ + "\n\n" + _)), prefix = "orca-system-prompt-", suffix = ".md" @@ -242,6 +243,15 @@ private[orca] class ClaudeBackend( object ClaudeBackend: + /** Whether claude has a transcript for session `id` started in `workDir`; + * `projectsDir` is claude's `~/.claude/projects`. + */ + def transcriptProbe( + fs: FileSystem, + projectsDir: os.Path + ): (os.Path, String) ->{fs} Boolean = + (workDir, id) => fs.exists(projectsDir / cwdSlug(workDir) / s"$id.jsonl") + /** Derives the project-directory slug that claude uses under * `~/.claude/projects/`. E.g. `/home/foo/.bar` → `-home-foo--bar`. */ diff --git a/claude/src/main/scala/orca/backend/claude/ClaudeDecoder.scala b/claude/src/main/scala/orca/backend/claude/ClaudeDecoder.scala index 8b9b42a94..cc54e5fc9 100644 --- a/claude/src/main/scala/orca/backend/claude/ClaudeDecoder.scala +++ b/claude/src/main/scala/orca/backend/claude/ClaudeDecoder.scala @@ -54,7 +54,7 @@ private[claude] final class ClaudeDecoder(outputSchema: Option[String]) state.copy(responseIds = state.responseIds ++ messageId), content ) - case InboundMessage.User(content) => userMessage(state, content) + case InboundMessage.User(content) => userMessage(state, content) case result: InboundMessage.Result => if result.isError then resultError(state, result) else resultSuccess(state, result) @@ -143,7 +143,7 @@ private[claude] final class ClaudeDecoder(outputSchema: Option[String]) ): TurnEvent = PermissionRefusal.toolName(body) match case Some(tool) if isError => TurnEvent.ToolDenied(tool) - case _ => + case _ => TurnEvent.ToolResult( // claude's tool_result block carries only a tool_use_id, not the // name — the grammar legalizes None here (see TurnEvent). diff --git a/claude/src/main/scala/orca/backend/claude/McpConfig.scala b/claude/src/main/scala/orca/backend/claude/McpConfig.scala index ad195feec..9b6964462 100644 --- a/claude/src/main/scala/orca/backend/claude/McpConfig.scala +++ b/claude/src/main/scala/orca/backend/claude/McpConfig.scala @@ -1,10 +1,7 @@ package orca.backend.claude -import com.github.plokhotnyuk.jsoniter_scala.core.{ - JsonValueCodec, - writeToString -} -import com.github.plokhotnyuk.jsoniter_scala.macros.JsonCodecMaker +import com.github.plokhotnyuk.jsoniter_scala.core.JsonValueCodec +import orca.json.JsonCodec import scala.concurrent.duration.FiniteDuration @@ -17,10 +14,10 @@ private[claude] object McpConfig: /** One `mcpServers.` entry; `timeout` is in milliseconds. */ private case class Entry(`type`: String, url: String, timeout: Long) private case class File(mcpServers: Map[String, Entry]) - private given JsonValueCodec[File] = JsonCodecMaker.make + private given JsonValueCodec[File] = JsonCodec.make def render(servers: Map[String, HttpServer]): String = - writeToString( + JsonCodec.writeString( File( servers.map((name, s) => name -> Entry("http", s.url, s.timeout.toMillis) diff --git a/claude/src/main/scala/orca/backend/claude/streamjson/ContentBlock.scala b/claude/src/main/scala/orca/backend/claude/streamjson/ContentBlock.scala index ee87bd370..ee051f040 100644 --- a/claude/src/main/scala/orca/backend/claude/streamjson/ContentBlock.scala +++ b/claude/src/main/scala/orca/backend/claude/streamjson/ContentBlock.scala @@ -1,15 +1,11 @@ package orca.backend.claude.streamjson -import com.github.plokhotnyuk.jsoniter_scala.core.readFromString -import com.github.plokhotnyuk.jsoniter_scala.macros.{ - ConfiguredJsonValueCodec, - JsonCodecMaker -} -import orca.json.RawJson +import com.github.plokhotnyuk.jsoniter_scala.core.JsonValueCodec +import orca.json.{JsonCodec, RawJson} +import orca.safe.Logger +import orca.safe.prelude.NonFatal import orca.util.OrcaDebug -import scala.util.control.NonFatal - /** A single block inside an assistant or user message's `content` array. * * Claude Code emits several block shapes; we model the ones the decoder @@ -26,31 +22,30 @@ private[claude] enum ContentBlock: private[claude] object ContentBlock: def parse(rawJson: String): ContentBlock = - val envelope = readFromString[BlockEnvelope](rawJson) + val envelope = JsonCodec.readString[BlockEnvelope](rawJson) envelope.`type` match - case "text" => readFromString[TextWire](rawJson).toBlock - case "thinking" => readFromString[ThinkingWire](rawJson).toBlock - case "tool_use" => readFromString[ToolUseWire](rawJson).toBlock - case "tool_result" => readFromString[ToolResultWire](rawJson).toBlock - case other => Unknown(other) + case "text" => JsonCodec.readString[TextWire](rawJson).toBlock + case "thinking" => JsonCodec.readString[ThinkingWire](rawJson).toBlock + case "tool_use" => JsonCodec.readString[ToolUseWire](rawJson).toBlock + case "tool_result" => + JsonCodec.readString[ToolResultWire](rawJson).toBlock + case other => Unknown(other) // --- Wire shapes --- - private case class BlockEnvelope(`type`: String) - derives ConfiguredJsonValueCodec + private case class BlockEnvelope(`type`: String) derives JsonCodec - private case class TextWire(text: String) derives ConfiguredJsonValueCodec: + private case class TextWire(text: String) derives JsonCodec: def toBlock: ContentBlock = Text(text) - private case class ThinkingWire(thinking: String) - derives ConfiguredJsonValueCodec: + private case class ThinkingWire(thinking: String) derives JsonCodec: def toBlock: ContentBlock = Thinking(thinking) private case class ToolUseWire( id: String, name: String, input: RawJson - ) derives ConfiguredJsonValueCodec: + ) derives JsonCodec: def toBlock: ContentBlock = ToolUse(id = id, name = name, rawInput = input.value) @@ -62,7 +57,7 @@ private[claude] object ContentBlock: tool_use_id: String, content: RawJson = RawJson("\"\""), is_error: Option[Boolean] = None - ) derives ConfiguredJsonValueCodec: + ) derives JsonCodec: def toBlock: ContentBlock = ToolResult( toolUseId = tool_use_id, @@ -82,7 +77,7 @@ private[claude] object ContentBlock: else trimmed private def decodeStringLiteral(raw: String): String = - try readFromString[String](raw)(using stringCodec) + try JsonCodec.readString[String](raw)(using stringCodec) catch case NonFatal(t) => // Claude occasionally sends malformed string literals; fall back to the @@ -94,7 +89,7 @@ private[claude] object ContentBlock: private def decodeNestedBlocks(raw: String): String = try val flat = flattenBlocks( - readFromString[List[NestedBlock]](raw)(using nestedBlocksCodec) + JsonCodec.readString[List[NestedBlock]](raw)(using nestedBlocksCodec) ) if flat.nonEmpty then flat else raw catch @@ -110,7 +105,7 @@ private[claude] object ContentBlock: if OrcaDebug.streamTrace || OrcaDebug.enabled then val snippet = if raw.length > 200 then s"${raw.take(200)}…" else raw - System.err.println( + Logger.stderr( s"[orca-debug claude.$where] swallowed ${t.getClass.getName}: " + s"${Option(t.getMessage).getOrElse("")} | raw=$snippet" ) @@ -134,12 +129,7 @@ private[claude] object ContentBlock: def asToolName: Option[String] = tool_name.filter(_ => `type` == "tool_reference") - private given stringCodec - : com.github.plokhotnyuk.jsoniter_scala.core.JsonValueCodec[String] = - JsonCodecMaker.make + private given stringCodec: JsonValueCodec[String] = JsonCodec.make - private given nestedBlocksCodec - : com.github.plokhotnyuk.jsoniter_scala.core.JsonValueCodec[List[ - NestedBlock - ]] = - JsonCodecMaker.make + private given nestedBlocksCodec: JsonValueCodec[List[NestedBlock]] = + JsonCodec.make diff --git a/claude/src/main/scala/orca/backend/claude/streamjson/InboundMessage.scala b/claude/src/main/scala/orca/backend/claude/streamjson/InboundMessage.scala index 1eccb521b..b5b386b89 100644 --- a/claude/src/main/scala/orca/backend/claude/streamjson/InboundMessage.scala +++ b/claude/src/main/scala/orca/backend/claude/streamjson/InboundMessage.scala @@ -1,10 +1,7 @@ package orca.backend.claude.streamjson import orca.events.{Usage} -import orca.json.RawJson - -import com.github.plokhotnyuk.jsoniter_scala.core.readFromString -import com.github.plokhotnyuk.jsoniter_scala.macros.ConfiguredJsonValueCodec +import orca.json.{JsonCodec, RawJson} /** One message parsed off of claude's stdout when running with `--output-format * stream-json --verbose --include-partial-messages`. Each variant carries only @@ -58,7 +55,7 @@ private[claude] object InboundMessage: * callers decide whether to skip or fail. */ def parse(line: String): InboundMessage = - val envelope = readFromString[TopEnvelope](line) + val envelope = JsonCodec.readString[TopEnvelope](line) envelope.`type` match case "system" => parseSystem(line) case "assistant" => parseAssistant(line) @@ -69,21 +66,21 @@ private[claude] object InboundMessage: case other => Unknown(other) private def parseSystem(line: String): InboundMessage = - val wire = readFromString[SystemWire](line) + val wire = JsonCodec.readString[SystemWire](line) if wire.subtype == "init" then SystemInit(wire.session_id.getOrElse(""), wire.model) else Unknown(s"system.${wire.subtype}") private def parseAssistant(line: String): InboundMessage = - val wire = readFromString[MessageWire](line) + val wire = JsonCodec.readString[MessageWire](line) Assistant(wire.message.toBlocks, wire.message.id) private def parseUser(line: String): InboundMessage = - val wire = readFromString[MessageWire](line) + val wire = JsonCodec.readString[MessageWire](line) User(wire.message.toBlocks) private def parseResult(line: String): InboundMessage = - val wire = readFromString[ResultWire](line) + val wire = JsonCodec.readString[ResultWire](line) // The result message's own `num_turns` looks like a call count and is not // one: measured against claude 2.1.220 it is (tool calls + 1), so a // response issuing three tool calls at once reports three turns where one @@ -110,39 +107,39 @@ private[claude] object InboundMessage: ) private def parseControlRequest(line: String): InboundMessage = - ControlRequest(readFromString[ControlRequestWire](line).request.subtype) + ControlRequest( + JsonCodec.readString[ControlRequestWire](line).request.subtype + ) private def parseStreamEvent(line: String): InboundMessage = - val wire = readFromString[StreamEventWire](line) + val wire = JsonCodec.readString[StreamEventWire](line) StreamEvent(StreamEventPayload.parse(wire.event.value)) // --- Wire shapes --- - private case class TopEnvelope(`type`: String) - derives ConfiguredJsonValueCodec + private case class TopEnvelope(`type`: String) derives JsonCodec private case class SystemWire( subtype: String, session_id: Option[String] = None, model: Option[String] = None - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec private case class InnerMessage( content: List[RawJson] = Nil, id: Option[String] = None - ) derives ConfiguredJsonValueCodec: + ) derives JsonCodec: def toBlocks: List[ContentBlock] = content.map(b => ContentBlock.parse(b.value)) - private case class MessageWire(message: InnerMessage) - derives ConfiguredJsonValueCodec + private case class MessageWire(message: InnerMessage) derives JsonCodec private case class UsageWire( input_tokens: Option[Long] = None, output_tokens: Option[Long] = None, cache_creation_input_tokens: Option[Long] = None, cache_read_input_tokens: Option[Long] = None - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec private case class ResultWire( subtype: String, @@ -153,13 +150,11 @@ private[claude] object InboundMessage: total_cost_usd: Option[BigDecimal] = None, is_error: Option[Boolean] = None, model: Option[String] = None - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec - private case class ControlRequestInner(subtype: String) - derives ConfiguredJsonValueCodec + private case class ControlRequestInner(subtype: String) derives JsonCodec private case class ControlRequestWire(request: ControlRequestInner) - derives ConfiguredJsonValueCodec + derives JsonCodec - private case class StreamEventWire(event: RawJson) - derives ConfiguredJsonValueCodec + private case class StreamEventWire(event: RawJson) derives JsonCodec diff --git a/claude/src/main/scala/orca/backend/claude/streamjson/OutboundMessage.scala b/claude/src/main/scala/orca/backend/claude/streamjson/OutboundMessage.scala index 64ade7b4e..90d10f092 100644 --- a/claude/src/main/scala/orca/backend/claude/streamjson/OutboundMessage.scala +++ b/claude/src/main/scala/orca/backend/claude/streamjson/OutboundMessage.scala @@ -1,7 +1,6 @@ package orca.backend.claude.streamjson -import com.github.plokhotnyuk.jsoniter_scala.core.writeToString -import com.github.plokhotnyuk.jsoniter_scala.macros.ConfiguredJsonValueCodec +import orca.json.JsonCodec /** Messages orca writes to claude's stdin. * @@ -15,7 +14,7 @@ private[claude] object OutboundMessage: /** A user message carrying `text`, as a single NDJSON line (no trailing * newline — the caller appends one). */ - def userText(text: String): String = writeToString( + def userText(text: String): String = JsonCodec.writeString( UserTextWire( `type` = "user", message = UserTextInner( @@ -28,14 +27,14 @@ private[claude] object OutboundMessage: // --- Wire shapes --- private case class UserTextContent(`type`: String, text: String) - derives ConfiguredJsonValueCodec + derives JsonCodec private case class UserTextInner( role: String, content: List[UserTextContent] - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec private case class UserTextWire( `type`: String, message: UserTextInner - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec diff --git a/claude/src/main/scala/orca/backend/claude/streamjson/StreamEventPayload.scala b/claude/src/main/scala/orca/backend/claude/streamjson/StreamEventPayload.scala index 593c06738..706897cc2 100644 --- a/claude/src/main/scala/orca/backend/claude/streamjson/StreamEventPayload.scala +++ b/claude/src/main/scala/orca/backend/claude/streamjson/StreamEventPayload.scala @@ -1,9 +1,6 @@ package orca.backend.claude.streamjson -import orca.json.RawJson - -import com.github.plokhotnyuk.jsoniter_scala.core.readFromString -import com.github.plokhotnyuk.jsoniter_scala.macros.ConfiguredJsonValueCodec +import orca.json.{JsonCodec, RawJson} /** The inner `event` of a `stream_event` message: one chunk of a partial * message. We route on the block-delta events (text, thinking, tool-input @@ -23,38 +20,40 @@ private[claude] object StreamEventPayload: /** `rawJson` is the serialized value of the outer message's `event` field. */ def parse(rawJson: String): StreamEventPayload = - val envelope = readFromString[EventEnvelope](rawJson) + val envelope = JsonCodec.readString[EventEnvelope](rawJson) envelope.`type` match case "content_block_delta" => parseBlockDelta(rawJson) case "content_block_start" => parseBlockStart(rawJson) - case "content_block_stop" => - ContentBlockStop(readFromString[BlockStopWire](rawJson).index) + case "content_block_stop" => + ContentBlockStop(JsonCodec.readString[BlockStopWire](rawJson).index) case other => Unhandled(other) private def parseBlockDelta(rawJson: String): StreamEventPayload = - val outer = readFromString[BlockDeltaOuter](rawJson) - val delta = readFromString[DeltaEnvelope](outer.delta.value) + val outer = JsonCodec.readString[BlockDeltaOuter](rawJson) + val delta = JsonCodec.readString[DeltaEnvelope](outer.delta.value) delta.`type` match case "text_delta" => TextDelta( outer.index, - readFromString[TextDeltaWire](outer.delta.value).text + JsonCodec.readString[TextDeltaWire](outer.delta.value).text ) case "thinking_delta" => ThinkingDelta( outer.index, - readFromString[ThinkingDeltaWire](outer.delta.value).thinking + JsonCodec.readString[ThinkingDeltaWire](outer.delta.value).thinking ) case "input_json_delta" => InputJsonDelta( outer.index, - readFromString[InputJsonDeltaWire](outer.delta.value).partial_json + JsonCodec + .readString[InputJsonDeltaWire](outer.delta.value) + .partial_json ) case other => Unhandled(s"content_block_delta.$other") private def parseBlockStart(rawJson: String): StreamEventPayload = - val outer = readFromString[BlockStartWire](rawJson) + val outer = JsonCodec.readString[BlockStartWire](rawJson) ContentBlockStart( outer.index, ContentBlock.parse(outer.content_block.value) @@ -62,25 +61,20 @@ private[claude] object StreamEventPayload: // --- Wire shapes --- - private case class EventEnvelope(`type`: String) - derives ConfiguredJsonValueCodec + private case class EventEnvelope(`type`: String) derives JsonCodec private case class BlockDeltaOuter(index: Int, delta: RawJson) - derives ConfiguredJsonValueCodec + derives JsonCodec - private case class DeltaEnvelope(`type`: String) - derives ConfiguredJsonValueCodec + private case class DeltaEnvelope(`type`: String) derives JsonCodec - private case class TextDeltaWire(text: String) - derives ConfiguredJsonValueCodec + private case class TextDeltaWire(text: String) derives JsonCodec - private case class ThinkingDeltaWire(thinking: String) - derives ConfiguredJsonValueCodec + private case class ThinkingDeltaWire(thinking: String) derives JsonCodec - private case class InputJsonDeltaWire(partial_json: String) - derives ConfiguredJsonValueCodec + private case class InputJsonDeltaWire(partial_json: String) derives JsonCodec private case class BlockStartWire(index: Int, content_block: RawJson) - derives ConfiguredJsonValueCodec + derives JsonCodec - private case class BlockStopWire(index: Int) derives ConfiguredJsonValueCodec + private case class BlockStopWire(index: Int) derives JsonCodec diff --git a/claude/src/test/scala/orca/backend/claude/ClaudeBackendTest.scala b/claude/src/test/scala/orca/backend/claude/ClaudeBackendTest.scala index 4ce1e0a1c..1cb4ec174 100644 --- a/claude/src/test/scala/orca/backend/claude/ClaudeBackendTest.scala +++ b/claude/src/test/scala/orca/backend/claude/ClaudeBackendTest.scala @@ -1,6 +1,7 @@ package orca.backend.claude -import orca.testkit.OpenTurn +import orca.effect.Network +import orca.testkit.{Effects, OpenTurn} import orca.backend.{ Dispatch, ResumeOrigin, @@ -18,7 +19,8 @@ import orca.agents.{ onWire } import orca.{OrcaFlowException} -import orca.subprocess.{FakePipedCliProcess, SpawnStubCliRunner} +import orca.subprocess.{CliRunner, FakePipedCliProcess, SpawnStubCliRunner} +import orca.tools.OsGitTool import orca.testkit.TempDirs import com.github.plokhotnyuk.jsoniter_scala.core.readFromString @@ -56,9 +58,28 @@ class ClaudeBackendTest extends munit.FunSuite: p.sendSigInt() p + /** `projectsDir` defaults to an empty temp dir, so no test reads the + * developer's real transcripts. + */ + private def claudeBackend( + cli: CliRunner, + projectsDir: os.Path = TempDirs.dir(), + workDir: os.Path = os.pwd + ): ClaudeBackend = + val fs = Effects.fs + new ClaudeBackend( + Network.ambient, + fs, + cli, + new OsGitTool(_, Effects.procs, fs), + ClaudeBackend.transcriptProbe(fs, projectsDir), + workDir + ) + private def withBackend[T](runner: SpawnStubCliRunner)( body: ox.Ox ?=> ClaudeBackend => T - ): T = SupervisedBackend.using(new ClaudeBackend(runner))(body) + ): T = + SupervisedBackend.using(claudeBackend(runner))(body) private def freshSid: SessionId[BackendTag.ClaudeCode.type] = SessionId[BackendTag.ClaudeCode.type]( @@ -280,7 +301,7 @@ class ClaudeBackendTest extends munit.FunSuite: // The declaration behind the prompt's delivery instruction: --json-schema // (asserted below) makes the CLI inject a StructuredOutput tool, so the // instruction must ask for that tool call, not raw reply text. - val backend = new ClaudeBackend(new SpawnStubCliRunner(Nil)) + val backend = claudeBackend(new SpawnStubCliRunner(Nil)) assertEquals( backend.structuredOutputMode, orca.agents.StructuredOutputMode.Tool @@ -552,7 +573,7 @@ class ClaudeBackendTest extends munit.FunSuite: os.write(tmpProjects / slug / s"${SessionId.value(freshSid)}.jsonl", "") val runner = new SpawnStubCliRunner(List(successfulProcess())) SupervisedBackend.using( - new ClaudeBackend(runner, projectsDir = tmpProjects, workDir = cwd) + claudeBackend(runner, projectsDir = tmpProjects, workDir = cwd) ): backend => val _ = backend.runAutonomous("continue", freshSid, AgentConfig()) val args = runner.calls.head @@ -571,7 +592,7 @@ class ClaudeBackendTest extends munit.FunSuite: os.makeDir.all(tmpProjects / slug) os.write(tmpProjects / slug / s"${SessionId.value(freshSid)}.jsonl", "") SupervisedBackend.using( - new ClaudeBackend( + claudeBackend( new SpawnStubCliRunner(Nil), projectsDir = tmpProjects, workDir = cwd @@ -593,23 +614,14 @@ class ClaudeBackendTest extends munit.FunSuite: // with a worktree-style `workDir` (!= the process cwd) must probe AND spawn // under that same directory. val tmpProjects = TempDirs.dir() - val flowWorkDir = - TempDirs.dir() // stands in for a worktree checkout, != os.pwd - assert( - flowWorkDir != os.pwd, - "test setup requires a workDir distinct from the process cwd" - ) + val flowWorkDir = TempDirs.dir() // stands in for a worktree checkout val slug = ClaudeBackend.cwdSlug(flowWorkDir) os.makeDir.all(tmpProjects / slug) os.write(tmpProjects / slug / s"${SessionId.value(freshSid)}.jsonl", "") val runner = new SpawnStubCliRunner(List(successfulProcess())) SupervisedBackend.using( - new ClaudeBackend( - runner, - projectsDir = tmpProjects, - workDir = flowWorkDir - ) + claudeBackend(runner, projectsDir = tmpProjects, workDir = flowWorkDir) ): backend => assertEquals(backend.workDir, flowWorkDir) backend.sessions.rehydrate(freshSid, freshSid.onWire) @@ -625,19 +637,6 @@ class ClaudeBackendTest extends munit.FunSuite: "spawn must use the SAME workDir the probe just read" ) - // The bare-construction default (`workDir = os.pwd`) probes the WRONG - // directory when the process cwd differs from the flow's workDir — - // exactly the bug `ClaudeAgents.default` must avoid by passing - // `workDir = wiring.workDir` explicitly. - SupervisedBackend.using( - new ClaudeBackend(new SpawnStubCliRunner(Nil), projectsDir = tmpProjects) - ): backend => - backend.sessions.rehydrate(freshSid, freshSid.onWire) - assertEquals( - backend.sessions.dispatchFor(freshSid).asTurnDispatch, - TurnDispatch.Fresh - ) - test( "an unrecorded id resumes as an earlier attempt's when its transcript is present" ): @@ -651,7 +650,7 @@ class ClaudeBackendTest extends munit.FunSuite: os.makeDir.all(tmpProjects / slug) os.write(tmpProjects / slug / s"${SessionId.value(freshSid)}.jsonl", "") SupervisedBackend.using( - new ClaudeBackend( + claudeBackend( new SpawnStubCliRunner(Nil), projectsDir = tmpProjects, workDir = cwd @@ -668,7 +667,7 @@ class ClaudeBackendTest extends munit.FunSuite: val tmpProjects = TempDirs.dir() val cwd = TempDirs.dir() SupervisedBackend.using( - new ClaudeBackend( + claudeBackend( new SpawnStubCliRunner(Nil), projectsDir = tmpProjects, workDir = cwd @@ -684,7 +683,7 @@ class ClaudeBackendTest extends munit.FunSuite: val missing = TempDirs.dir() / "no-such-dir" val cwd = TempDirs.dir() SupervisedBackend.using( - new ClaudeBackend( + claudeBackend( new SpawnStubCliRunner(Nil), projectsDir = missing, workDir = cwd diff --git a/claude/src/test/scala/orca/backend/claude/ClaudeIntegrationTest.scala b/claude/src/test/scala/orca/backend/claude/ClaudeIntegrationTest.scala index 21aa3efe8..3a8204f31 100644 --- a/claude/src/test/scala/orca/backend/claude/ClaudeIntegrationTest.scala +++ b/claude/src/test/scala/orca/backend/claude/ClaudeIntegrationTest.scala @@ -1,5 +1,6 @@ package orca.backend.claude +import orca.effect.Network import orca.testkit.OpenTurn import com.github.plokhotnyuk.jsoniter_scala.core.readFromString import com.github.plokhotnyuk.jsoniter_scala.macros.ConfiguredJsonValueCodec @@ -13,7 +14,8 @@ import orca.agents.{ } import orca.backend.{TurnEvent, Dispatch, SupervisedBackend} import orca.subprocess.OsProcCliRunner -import orca.testkit.{GitRepo, TempDirs} +import orca.testkit.{Effects, GitRepo, TempDirs} +import orca.tools.OsGitTool import orca.backend.claude.streamjson.OutboundMessage /** End-to-end tests against the real `claude` CLI. Gated on the @@ -31,9 +33,23 @@ class ClaudeIntegrationTest extends munit.FunSuite: import scala.concurrent.duration.DurationInt 2.minutes + /** Over the real CLI and its real transcript directory. */ + private def realBackend(workDir: os.Path): ClaudeBackend = + val fs = Effects.fs + val procs = Effects.procs + new ClaudeBackend( + Network.ambient, + fs, + OsProcCliRunner(procs), + new OsGitTool(_, procs, fs), + // Same path as ClaudeAgents.default. + ClaudeBackend.transcriptProbe(fs, os.home / ".claude" / "projects"), + workDir + ) + private def withBackend(body: ox.Ox ?=> ClaudeBackend => Unit): Unit = SupervisedBackend.using( - new ClaudeBackend(OsProcCliRunner, workDir = TempDirs.dir()) + realBackend(TempDirs.dir()) )(body) private def fresh = SessionId.fresh[BackendTag.ClaudeCode.type] @@ -158,7 +174,7 @@ class ClaudeIntegrationTest extends munit.FunSuite: val _ = os.proc("git", "commit", "-m", "add secret").call(cwd = repo) os.write.over(repo / "secret.txt", "OVERWRITTEN") SupervisedBackend.using( - new ClaudeBackend(OsProcCliRunner, workDir = repo) + realBackend(repo) ): backend => val result = backend.runAutonomous( prompt = "Use git_file_at to read secret.txt at HEAD. Reply with " + @@ -195,7 +211,7 @@ class ClaudeIntegrationTest extends munit.FunSuite: val _ = os.proc("git", "add", "-A").call(cwd = repo) val _ = os.proc("git", "commit", "-m", "PINEAPPLE commit").call(cwd = repo) SupervisedBackend.using( - new ClaudeBackend(OsProcCliRunner, workDir = repo) + realBackend(repo) ): backend => val result = backend.runAutonomous( prompt = "Call git_show for HEAD, passing only the rev. Reply with " + diff --git a/claude/src/test/scala/orca/backend/claude/ClaudeTurnTest.scala b/claude/src/test/scala/orca/backend/claude/ClaudeTurnTest.scala index 00886bd4c..55225ec49 100644 --- a/claude/src/test/scala/orca/backend/claude/ClaudeTurnTest.scala +++ b/claude/src/test/scala/orca/backend/claude/ClaudeTurnTest.scala @@ -1,6 +1,7 @@ package orca.backend.claude import orca.agents.Model +import orca.effect.Network import orca.events.{TurnDebit, Usage} import orca.testkit.Usages.usage import orca.{AgentTurnFailed, OrcaFlowException, OrcaInteractiveCancelled} @@ -277,7 +278,7 @@ class ClaudeTurnTest extends munit.FunSuite: live.cancel() live.awaitResult() match case Left(_: OrcaInteractiveCancelled) => () - case other => + case other => fail(s"expected Left(OrcaInteractiveCancelled), got: $other") assertEquals(process.sigIntCount, 1) @@ -440,7 +441,7 @@ class ClaudeTurnTest extends munit.FunSuite: supervised: given BufferCapacity = BufferCapacity(8) val process = new FakePipedCliProcess() - val askUser = AskUserSession.allocate() + val askUser = AskUserSession.allocate(Network.ambient) val live = ClaudeTurn(process, askUser = AskUserChannel.Mcp(askUser)) val bridge = askUser.bridge diff --git a/codex/src/main/scala/orca/backend/codex/CodexBackend.scala b/codex/src/main/scala/orca/backend/codex/CodexBackend.scala index 348379a62..e24ef83a4 100644 --- a/codex/src/main/scala/orca/backend/codex/CodexBackend.scala +++ b/codex/src/main/scala/orca/backend/codex/CodexBackend.scala @@ -18,11 +18,12 @@ import orca.backend.{ IdScheme, SessionSupport, SubprocessSpawn, - TurnResources, SystemPromptComposer } import orca.backend.mcp.{AskUserMcpServer, AskUserSession} +import orca.effect.{FileSystem, Network} import orca.subprocess.CliRunner +import orca.concurrency.defaultBufferCapacity import ox.{Ox, ResourceScope} /** Codex backend. Both autonomous and interactive paths drive `codex exec @@ -44,28 +45,15 @@ import ox.{Ox, ResourceScope} * calls skip the bridge entirely. */ private[orca] class CodexBackend( + net: Network, + fs: FileSystem, cli: CliRunner, - private[codex] val sessionsDir: os.Path = os.home / ".codex" / "sessions", - /** Fixed at construction; every spawn (`open`) runs in this directory. The - * `os.pwd` default serves bare/test construction; the runtime - * (`CodexAgents.default`) passes the flow's real `workDir`. - */ - override val workDir: os.Path = os.pwd + /** [[CodexBackend.rolloutProbe]] in production. */ + sessionProbe: String -> Boolean, + /** Fixed at construction; every spawn (`open`) runs in this directory. */ + override val workDir: os.Path ) extends AgentBackend[BackendTag.Codex.type]: - /** Codex's threads are server-side and durable: the probe walks - * [[sessionsDir]] for a file matching `rollout-*-.jsonl` (the - * caller's stable id never appears in a rollout filename). - * - * `false` results whenever no server id is mapped — including one that - * failed the [[orca.agents.SessionId.isSafe]] guard (blocks regex injection; - * e.g. `.*` would match every rollout file), the map not yet rehydrated, or - * the sessions dir missing. - * - * Some machines' codex uses SQLite (`~/.codex/state_5.sqlite`) instead of - * `rollout-*.jsonl`; with no matching files the probe returns `false` → - * re-seed, always safe. - */ val tag: BackendTag.Codex.type = BackendTag.Codex export CodexArgs.enforcementCell @@ -90,15 +78,7 @@ private[orca] class CodexBackend( * through `codex exec resume `. */ val sessions: SessionSupport[BackendTag.Codex.type] = - SessionSupport.durable( - IdScheme.ServerMinted, - id => - os.exists(sessionsDir) && os.walk - .stream(sessionsDir) - .exists(p => - p.last.startsWith("rollout-") && p.last.endsWith(s"-$id.jsonl") - ) - ) + SessionSupport.durable(IdScheme.ServerMinted, sessionProbe) /** Spawn `codex exec --json` (fresh) or `codex exec resume ` * (continuation), per `dispatch`, and wrap the process in a live @@ -118,7 +98,7 @@ private[orca] class CodexBackend( import turn.* val schemaFile = writeSchemaIfPresent(outputSchema) val askUser: Option[AskUserSession] = - Option.when(mode.isInteractive)(AskUserSession.allocate()) + Option.when(mode.isInteractive)(AskUserSession.allocate(net)) SubprocessSpawn.open("codex", events) { // codex `exec` has no `--system-prompt` flag (it picks up `AGENTS.md` // files for static instructions), so fold the composed system prompt into @@ -169,8 +149,35 @@ private[orca] class CodexBackend( ResourceScope ): Option[os.Path] = schema.map: body => - TurnResources.tempFile( + fs.tempFile( body, prefix = "orca-codex-schema-", suffix = ".json" ) + +private[orca] object CodexBackend: + + /** Codex's threads are server-side and durable: the probe looks under + * `sessionsDir` for a file matching `rollout-*-.jsonl` (the + * caller's stable id never appears in a rollout filename). + * + * `false` results whenever no server id is mapped — including one that + * failed the [[orca.agents.SessionId.isSafe]] guard (blocks regex injection; + * e.g. `.*` would match every rollout file), the map not yet rehydrated, or + * the sessions dir missing. + * + * Some machines' codex uses SQLite (`~/.codex/state_5.sqlite`) instead of + * `rollout-*.jsonl`; with no matching files the probe returns `false` → + * re-seed, always safe. + */ + def rolloutProbe( + fs: FileSystem, + sessionsDir: os.Path + ): String ->{fs} Boolean = + id => + fs.exists(sessionsDir) && fs + .findFile( + sessionsDir, + p => p.last.startsWith("rollout-") && p.last.endsWith(s"-$id.jsonl") + ) + .nonEmpty diff --git a/codex/src/main/scala/orca/backend/codex/CodexDecoder.scala b/codex/src/main/scala/orca/backend/codex/CodexDecoder.scala index d747d6d47..9522ab652 100644 --- a/codex/src/main/scala/orca/backend/codex/CodexDecoder.scala +++ b/codex/src/main/scala/orca/backend/codex/CodexDecoder.scala @@ -18,9 +18,8 @@ import orca.backend.{ import orca.backend.mcp.AskUserMcpServer import orca.subprocess.PipedCliProcess import orca.backend.codex.jsonl.{FileChangeDetail, InboundEvent, Item} +import orca.json.JsonCodec -import com.github.plokhotnyuk.jsoniter_scala.core.writeToString -import com.github.plokhotnyuk.jsoniter_scala.macros.ConfiguredJsonValueCodec import ox.Ox /** Decodes one `codex exec --json` turn: JSONL → [[InboundEvent]] → @@ -113,7 +112,7 @@ private[codex] final class CodexDecoder( state, TurnEvent.AssistantToolCall( toolName = "bash", - rawInput = writeToString(BashInput(command)) + rawInput = JsonCodec.writeString(BashInput(command)) ) ) case Item.FileChange(_, changes, _) => @@ -121,7 +120,7 @@ private[codex] final class CodexDecoder( state, TurnEvent.AssistantToolCall( toolName = "file_change", - rawInput = writeToString(FileChangeInput(changes.map(toWire))) + rawInput = JsonCodec.writeString(FileChangeInput(changes.map(toWire))) ) ) case Item.McpToolCall(id, server, tool, _, _, _) @@ -195,7 +194,7 @@ private[codex] final class CodexDecoder( // Matched a suppressed ask_user call started above; drop the mirrored // completion. case Some(rest) => Step.continue(state.copy(echoes = rest)) - case None => + case None => Step.continue( state, TurnEvent.ToolResult( @@ -295,13 +294,13 @@ private[codex] object CodexDecoder: * input, so we wrap the command string in a one-key object the renderer can * introspect. */ - private case class BashInput(command: String) derives ConfiguredJsonValueCodec + private case class BashInput(command: String) derives JsonCodec private case class FileChangeWire(path: String, kind: String) - derives ConfiguredJsonValueCodec + derives JsonCodec private case class FileChangeInput(changes: List[FileChangeWire]) - derives ConfiguredJsonValueCodec + derives JsonCodec private[codex] object CodexTurn: diff --git a/codex/src/main/scala/orca/backend/codex/jsonl/InboundEvent.scala b/codex/src/main/scala/orca/backend/codex/jsonl/InboundEvent.scala index 1c8d356d7..3a1f7b8ec 100644 --- a/codex/src/main/scala/orca/backend/codex/jsonl/InboundEvent.scala +++ b/codex/src/main/scala/orca/backend/codex/jsonl/InboundEvent.scala @@ -1,12 +1,8 @@ package orca.backend.codex.jsonl import orca.events.{Usage} -import orca.json.RawJson - -import com.github.plokhotnyuk.jsoniter_scala.core.readFromString -import com.github.plokhotnyuk.jsoniter_scala.macros.ConfiguredJsonValueCodec - -import scala.util.control.NonFatal +import orca.json.{JsonCodec, RawJson} +import orca.safe.prelude.NonFatal /** One event parsed off codex's stdout when it runs with `exec --json`. * @@ -102,7 +98,7 @@ private[codex] object InboundEvent: * callers decide whether to skip or fail. */ def parse(line: String): InboundEvent = - val envelope = readFromString[TopEnvelope](line) + val envelope = JsonCodec.readString[TopEnvelope](line) envelope.`type` match case "thread.started" => parseThreadStarted(line) case "turn.started" => TurnStarted @@ -114,11 +110,11 @@ private[codex] object InboundEvent: case other => Unknown(other) private def parseThreadStarted(line: String): InboundEvent = - val wire = readFromString[ThreadStartedWire](line) + val wire = JsonCodec.readString[ThreadStartedWire](line) ThreadStarted(wire.thread_id, wire.model) private def parseTurnCompleted(line: String): InboundEvent = - val wire = readFromString[TurnCompletedWire](line) + val wire = JsonCodec.readString[TurnCompletedWire](line) val u = wire.usage.getOrElse(UsageWire()) TurnCompleted( // Every axis is worth decoding here: codex emits no cost, so the price @@ -138,15 +134,15 @@ private[codex] object InboundEvent: ) private def parseError(line: String): InboundEvent = - val wire = readFromString[ErrorWire](line) + val wire = JsonCodec.readString[ErrorWire](line) Error(wire.message.getOrElse("")) private def parseTurnFailed(line: String): InboundEvent = - val wire = readFromString[TurnFailedWire](line) + val wire = JsonCodec.readString[TurnFailedWire](line) TurnFailed(wire.error.flatMap(_.message).getOrElse("")) private def parseItem(line: String): Item = - val wire = readFromString[ItemWire](line) + val wire = JsonCodec.readString[ItemWire](line) val item = wire.item item.`type` match case "agent_message" => @@ -199,7 +195,7 @@ private[codex] object InboundEvent: if trimmed == "null" || trimmed.isEmpty then None else try - val wire = readFromString[McpResultWire](trimmed) + val wire = JsonCodec.readString[McpResultWire](trimmed) val joined = wire.content .flatMap(c => c.text.filter(_ => c.`type` == "text")) .mkString @@ -212,13 +208,12 @@ private[codex] object InboundEvent: // --- Wire shapes --- - private case class TopEnvelope(`type`: String) - derives ConfiguredJsonValueCodec + private case class TopEnvelope(`type`: String) derives JsonCodec private case class ThreadStartedWire( thread_id: String, model: Option[String] = None - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec private case class UsageWire( input_tokens: Option[Long] = None, @@ -226,22 +221,21 @@ private[codex] object InboundEvent: cache_write_input_tokens: Option[Long] = None, output_tokens: Option[Long] = None, reasoning_output_tokens: Option[Long] = None - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec private case class TurnCompletedWire(usage: Option[UsageWire] = None) - derives ConfiguredJsonValueCodec + derives JsonCodec - private case class ErrorWire(message: Option[String] = None) - derives ConfiguredJsonValueCodec + private case class ErrorWire(message: Option[String] = None) derives JsonCodec private case class TurnFailedErrorWire(message: Option[String] = None) - derives ConfiguredJsonValueCodec + derives JsonCodec private case class TurnFailedWire(error: Option[TurnFailedErrorWire] = None) - derives ConfiguredJsonValueCodec + derives JsonCodec private case class FileChangeWire(path: String, kind: String) - derives ConfiguredJsonValueCodec + derives JsonCodec private case class ItemBody( id: String, @@ -256,9 +250,9 @@ private[codex] object InboundEvent: tool: Option[String] = None, arguments: Option[RawJson] = None, result: Option[RawJson] = None - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec - private case class ItemWire(item: ItemBody) derives ConfiguredJsonValueCodec + private case class ItemWire(item: ItemBody) derives JsonCodec /** Subset of the MCP `tools/call` result we extract for display — just the * `content` array. `isError` is read off the item's top-level `status` field @@ -266,9 +260,9 @@ private[codex] object InboundEvent: */ private case class McpResultWire( content: List[McpContentWire] = Nil - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec private case class McpContentWire( `type`: String, text: Option[String] = None - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec diff --git a/codex/src/test/scala/orca/backend/codex/CodexBackendTest.scala b/codex/src/test/scala/orca/backend/codex/CodexBackendTest.scala index afcdb9d11..b95c35f10 100644 --- a/codex/src/test/scala/orca/backend/codex/CodexBackendTest.scala +++ b/codex/src/test/scala/orca/backend/codex/CodexBackendTest.scala @@ -1,6 +1,7 @@ package orca.backend.codex -import orca.testkit.OpenTurn +import orca.effect.Network +import orca.testkit.{Effects, OpenTurn} import orca.backend.{SupervisedBackend, SystemPromptComposer} import orca.agents.{ TurnDispatch, @@ -25,7 +26,15 @@ class CodexBackendTest extends munit.FunSuite: workDir: os.Path = os.pwd )( body: ox.Ox ?=> CodexBackend => T - ): T = SupervisedBackend.using(new CodexBackend(runner, workDir = workDir))( + ): T = SupervisedBackend.using( + new CodexBackend( + Network.ambient, + Effects.fs, + runner, + sessionProbe = _ => false, + workDir = workDir + ) + )( body ) @@ -472,7 +481,13 @@ class CodexBackendTest extends munit.FunSuite: val tmpSessions = TempDirs.dir() os.write(tmpSessions / s"rollout-2024-01-01-$serverId.jsonl", "") SupervisedBackend.using( - new CodexBackend(new SpawnStubCliRunner(Nil), tmpSessions) + new CodexBackend( + Network.ambient, + Effects.fs, + new SpawnStubCliRunner(Nil), + CodexBackend.rolloutProbe(Effects.fs, tmpSessions), + workDir = os.pwd + ) ): backend => backend.sessions.rehydrate( clientSid, @@ -492,7 +507,13 @@ class CodexBackendTest extends munit.FunSuite: "" ) SupervisedBackend.using( - new CodexBackend(new SpawnStubCliRunner(Nil), tmpSessions) + new CodexBackend( + Network.ambient, + Effects.fs, + new SpawnStubCliRunner(Nil), + CodexBackend.rolloutProbe(Effects.fs, tmpSessions), + workDir = os.pwd + ) ): backend => assertEquals( backend.sessions.dispatchFor(clientSid).asTurnDispatch, @@ -502,7 +523,13 @@ class CodexBackendTest extends munit.FunSuite: test("a rehydrated id opens fresh when no matching file exists"): val tmpSessions = TempDirs.dir() SupervisedBackend.using( - new CodexBackend(new SpawnStubCliRunner(Nil), tmpSessions) + new CodexBackend( + Network.ambient, + Effects.fs, + new SpawnStubCliRunner(Nil), + CodexBackend.rolloutProbe(Effects.fs, tmpSessions), + workDir = os.pwd + ) ): backend => backend.sessions.rehydrate( clientSid, @@ -516,7 +543,13 @@ class CodexBackendTest extends munit.FunSuite: test("a rehydrated id opens fresh when the sessions dir is absent"): val missing = TempDirs.dir() / "no-such-sessions" SupervisedBackend.using( - new CodexBackend(new SpawnStubCliRunner(Nil), missing) + new CodexBackend( + Network.ambient, + Effects.fs, + new SpawnStubCliRunner(Nil), + CodexBackend.rolloutProbe(Effects.fs, missing), + workDir = os.pwd + ) ): backend => backend.sessions.rehydrate( clientSid, diff --git a/codex/src/test/scala/orca/backend/codex/CodexIntegrationTest.scala b/codex/src/test/scala/orca/backend/codex/CodexIntegrationTest.scala index ef6c5452b..b833721a6 100644 --- a/codex/src/test/scala/orca/backend/codex/CodexIntegrationTest.scala +++ b/codex/src/test/scala/orca/backend/codex/CodexIntegrationTest.scala @@ -1,6 +1,7 @@ package orca.backend.codex -import orca.testkit.OpenTurn +import orca.effect.Network +import orca.testkit.{Effects, OpenTurn} import orca.AgentTurnFailed import orca.agents.{ AutoApprove, @@ -39,7 +40,16 @@ class CodexIntegrationTest extends munit.FunSuite: workDir: os.Path = TempDirs.dir() )(body: ox.Ox ?=> CodexBackend => Unit): Unit = SupervisedBackend.using( - new CodexBackend(OsProcCliRunner, workDir = workDir) + new CodexBackend( + Network.ambient, + Effects.fs, + OsProcCliRunner(Effects.procs), + CodexBackend.rolloutProbe( + Effects.fs, + os.home / ".codex" / "sessions" + ), + workDir = workDir + ) )(body) private val unsandboxed: AgentConfig = diff --git a/codex/src/test/scala/orca/backend/codex/CodexTurnTest.scala b/codex/src/test/scala/orca/backend/codex/CodexTurnTest.scala index 27ee157ec..9a1dce5d2 100644 --- a/codex/src/test/scala/orca/backend/codex/CodexTurnTest.scala +++ b/codex/src/test/scala/orca/backend/codex/CodexTurnTest.scala @@ -1,6 +1,7 @@ package orca.backend.codex import orca.agents.{Model, WireSessionId} +import orca.effect.Network import orca.events.{TurnDebit, Usage} import orca.testkit.Usages.usage import orca.{OrcaFlowException, OrcaInteractiveCancelled} @@ -413,7 +414,7 @@ class CodexTurnTest extends munit.FunSuite: live.cancel() live.awaitResult() match case Left(_: OrcaInteractiveCancelled) => () - case other => + case other => fail(s"expected Left(OrcaInteractiveCancelled), got: $other") assertEquals(process.sigIntCount, 1) @@ -744,7 +745,7 @@ class CodexTurnTest extends munit.FunSuite: val process = new FakePipedCliProcess() val live = CodexTurn( process, - askUser = AskUserChannel.Mcp(AskUserSession.allocate()) + askUser = AskUserChannel.Mcp(AskUserSession.allocate(Network.ambient)) ) process.enqueueStdout( @@ -795,7 +796,7 @@ class CodexTurnTest extends munit.FunSuite: supervised: given BufferCapacity = BufferCapacity(8) val process = new FakePipedCliProcess() - val askUser = AskUserSession.allocate() + val askUser = AskUserSession.allocate(Network.ambient) val live = CodexTurn( process, askUser = AskUserChannel.Mcp(askUser) diff --git a/core/src/main/scala/orca/concurrency/Ref.scala b/core/src/main/scala/orca/concurrency/Ref.scala new file mode 100644 index 000000000..26235c02f --- /dev/null +++ b/core/src/main/scala/orca/concurrency/Ref.scala @@ -0,0 +1,36 @@ +package orca.concurrency + +import language.experimental.captureChecking + +import caps.assumeSafe + +import java.util.concurrent.atomic.AtomicReference +import scala.annotation.tailrec + +/** A value shared across threads, replaced by pure functions of it. `S` must be + * immutable. + * + * For `update` and `modify`: `f` may run more than once; the compiler rejects + * an `f` that captures a capability; don't log, emit events or touch other + * `Ref`s inside it. + */ +@assumeSafe +final class Ref[S] private (initial: S): + private val cell: AtomicReference[S] = AtomicReference(initial) + + def get: S = cell.get() + + def set(s: S): Unit = cell.set(s) + + def update(f: S -> S): Unit = modify(s => (f(s), ())) + + /** Replaces the value with the first of `f`'s pair, returning the second. */ + @tailrec + def modify[R](f: S -> (S, R)): R = + val current = cell.get() + val (next, result) = f(current) + if cell.compareAndSet(current, next) then result else modify(f) + +@assumeSafe +object Ref: + def apply[S](initial: S): Ref[S] = new Ref(initial) diff --git a/core/src/main/scala/orca/concurrency/concurrency.scala b/core/src/main/scala/orca/concurrency/concurrency.scala new file mode 100644 index 000000000..ffe82d54b --- /dev/null +++ b/core/src/main/scala/orca/concurrency/concurrency.scala @@ -0,0 +1,125 @@ +package orca.concurrency + +import language.experimental.captureChecking + +import caps.assumeSafe +import ox.{Fork, Ox, OxUnsupervised, ResourceScope, UnsupervisedFork} +import ox.channels.{BufferCapacity, Channel, ChannelClosed, Source} +import ox.flow.Flow +import ox.resilience.{ResultPolicy, RetryConfig} +import ox.scheduling.Schedule + +import scala.concurrent.duration.FiniteDuration +import scala.util.NotGiven + +// The Ox operations, and the few JVM concurrency primitives, that safe code +// uses. Ox's types (`Ox`, `Fork`, `Channel`, …) and their members need no +// forwarder. + +@assumeSafe +inline def supervised[T](inline body: Ox ?=> T): T = ox.supervised(body) +@assumeSafe +inline def fork[T](inline body: => T)(using Ox): Fork[T] = ox.fork(body) +@assumeSafe +inline def forkDiscard[T](inline body: => T)(using Ox): Unit = + ox.forkDiscard(body) +@assumeSafe +inline def forkUnsupervised[T](inline body: => T)(using + OxUnsupervised +): UnsupervisedFork[T] = ox.forkUnsupervised(body) + +@assumeSafe +def useInScope[T](acquire: => T)(release: T => Unit)(using + ResourceScope +): T = ox.useInScope(acquire)(release) + +/** A scope for cleanup only, for code without a concurrency scope. */ +@assumeSafe +def resourceScope[T](body: ResourceScope ?=> T)(using + NotGiven[OxUnsupervised] +): T = ox.resourceScope(body) +@assumeSafe +def releaseAfterScope(release: => Unit)(using ResourceScope): Unit = + ox.releaseAfterScope(release) + +@assumeSafe +def sleep(howLong: FiniteDuration): Unit = ox.sleep(howLong) +@assumeSafe +inline def forever(inline f: Unit): Nothing = ox.forever(f) + +/** `System.nanoTime`: monotonic, for measuring elapsed time. */ +@assumeSafe +def nanoTime(): Long = System.nanoTime() + +/** A thread's identity: equal only to the identity of the same thread. */ +@assumeSafe +opaque type ThreadIdentity = Thread + +@assumeSafe +def currentThread(): ThreadIdentity = Thread.currentThread() + +/** Runs `f` over `items`, at most `parallelism` at a time, calling `onResult` + * on each result as it completes; results are in completion order. The first + * failure interrupts the rest and is rethrown. + */ +@assumeSafe +def mapParUnordered[A, R](parallelism: Int)(items: Seq[A])(f: A => R)( + onResult: R => Unit +): List[R] = + Flow + .fromIterable(items) + .mapParUnordered(parallelism)(f) + .tap(onResult) + .runToList() + +@assumeSafe +def bufferedChannel[T](capacity: Int): Channel[T] = + Channel.buffered[T](capacity) +@assumeSafe +def bufferedDefaultChannel[T](using BufferCapacity): Channel[T] = + Channel.bufferedDefault[T] + +/** ox's implicit-scope default, which safe code can't reference. */ +@assumeSafe +given defaultBufferCapacity: BufferCapacity = BufferCapacity.default + +@assumeSafe +def rendezvousChannel[T]: Channel[T] = Channel.rendezvous[T] + +/** An actor running `logic` in a fork of the enclosing scope, one call at a + * time; its mailbox holds `BufferCapacity` calls. Call it with `ask`: a `tell` + * that throws ends the enclosing scope. + */ +@assumeSafe +def actor[T](logic: T)(using Ox, BufferCapacity): ox.channels.ActorRef[T] = + ox.channels.Actor.create(logic) + +/** What `source` delivers, read lazily until it is done or errored. Each `next` + * blocks like `receive`. + */ +@assumeSafe +def receiveUntilClosed[T](source: Source[T]): Iterator[T] = + Iterator + .continually(source.receiveOrClosed()) + .takeWhile(!_.isInstanceOf[ChannelClosed]) + .map(_.asInstanceOf[T]) + +@assumeSafe +def retry[T](config: RetryConfig[Throwable, T])(operation: => T): T = + ox.resilience.retry(config)(operation) +@assumeSafe +def retryConfig[E, T]( + schedule: Schedule, + policy: ResultPolicy[E, T] +): RetryConfig[E, T] = RetryConfig(schedule, policy) +@assumeSafe +def retryWhen[E, T](isWorthRetrying: E => Boolean): ResultPolicy[E, T] = + ResultPolicy.retryWhen[E, T](isWorthRetrying) +@assumeSafe +def resultPolicy[E, T]( + isSuccess: T => Boolean, + isWorthRetrying: E => Boolean +): ResultPolicy[E, T] = ResultPolicy(isSuccess, isWorthRetrying) +@assumeSafe +def exponentialBackoff(first: FiniteDuration): Schedule = + Schedule.exponentialBackoff(first) diff --git a/core/src/main/scala/orca/effect/FileSystem.scala b/core/src/main/scala/orca/effect/FileSystem.scala new file mode 100644 index 000000000..c6cd64f6d --- /dev/null +++ b/core/src/main/scala/orca/effect/FileSystem.scala @@ -0,0 +1,155 @@ +package orca.effect + +import language.experimental.captureChecking + +import caps.assumeSafe +import ox.ResourceScope + +import java.nio.ByteBuffer +import java.nio.channels.FileChannel +import java.nio.file.StandardOpenOption +import java.time.Instant +import scala.util.Using +import scala.util.control.NonFatal + +/** Authority to read and write the local filesystem. */ +@assumeSafe +final class FileSystem private () extends caps.SharedCapability: + def exists(p: os.Path): Boolean = os.exists(p) + + /** [[exists]] without following a final symlink: a dangling link counts. */ + def entryExists(p: os.Path): Boolean = os.exists(p, followLinks = false) + def isFile(p: os.Path): Boolean = os.isFile(p) + def isDir(p: os.Path): Boolean = os.isDir(p) + def isLink(p: os.Path): Boolean = os.isLink(p) + def read(p: os.Path): String = os.read(p) + + /** Throws `java.nio.file.NoSuchFileException` when `p` is absent. */ + def readBytes(p: os.Path): IArray[Byte] = + // Nothing else holds the array, so the immutable view captures nothing. + caps.unsafe.unsafeAssumePure(IArray.unsafeFromArray(os.read.bytes(p))) + + /** At most the first `maxBytes` bytes of `p`. */ + def readPrefix(p: os.Path, maxBytes: Int): IArray[Byte] = + caps.unsafe.unsafeAssumePure( + IArray.unsafeFromArray(os.read.bytes(p, offset = 0, count = maxBytes)) + ) + + def modifiedAt(p: os.Path): Instant = Instant.ofEpochMilli(os.mtime(p)) + + /** Replaces `p`'s content, creating missing parent directories. */ + def writeOver(p: os.Path, content: String): Unit = + os.write.over(p, content, createFolders = true) + + /** Creates `p` with `content`; throws + * `java.nio.file.FileAlreadyExistsException` when `p` exists. + */ + def writeNew(p: os.Path, content: String): Unit = os.write(p, content) + + /** [[writeNew]] for bytes, flushed to disk before returning. The file gets + * the umask's permissions. + */ + def writeNewDurably(p: os.Path, bytes: IArray[Byte]): Unit = + Using.resource( + FileChannel.open( + p.toNIO, + StandardOpenOption.CREATE_NEW, + StandardOpenOption.WRITE + ) + ): channel => + val buffer = ByteBuffer.wrap(IArray.genericWrapArray(bytes).toArray) + while buffer.hasRemaining do channel.write(buffer): Unit + channel.force(true) + + /** Renames `from` over `to` atomically, replacing a symlink at `to` rather + * than writing through it. Throws + * `java.nio.file.AtomicMoveNotSupportedException` where the filesystem + * cannot. + */ + def moveAtomically(from: os.Path, to: os.Path): Unit = + os.move(from, to, replaceExisting = true, atomicMove = true) + + /** Moves `from` over `to`, replacing it. */ + def moveReplacing(from: os.Path, to: os.Path): Unit = + os.move(from, to, replaceExisting = true) + + def remove(p: os.Path): Unit = os.remove(p): Unit + def append(p: os.Path, text: String): Unit = os.write.append(p, text) + + /** At most the last `maxBytes` bytes of `p`. */ + def readSuffix(p: os.Path, maxBytes: Int): IArray[Byte] = + val size = os.size(p) + val count = math.min(size, maxBytes.toLong).toInt + caps.unsafe.unsafeAssumePure( + IArray.unsafeFromArray(os.read.bytes(p, offset = size - count, count)) + ) + + /** `f`'s defined results over the lines of `p`, read without holding the + * whole file. + */ + def collectLines[A](p: os.Path)(f: String => Option[A]): List[A] = + os.read.lines.stream(p).flatMap(f(_)).toList + + /** Removes `p` with everything under it; a symlink is removed, not followed. + */ + def removeAll(p: os.Path): Unit = os.remove.all(p) + + /** Creates `p` and its missing parents; a no-op for an existing directory. + */ + def makeDirs(p: os.Path): Unit = os.makeDir.all(p) + + /** Every file under `root`, recursively, that `keep` accepts. */ + def files(root: os.Path, keep: os.Path => Boolean): List[os.Path] = + os.walk.stream(root).filter(p => keep(p) && os.isFile(p)).toList + + /** Some file under `root`, recursively, that `keep` accepts; stops at the + * first. + */ + def findFile(root: os.Path, keep: os.Path => Boolean): Option[os.Path] = + os.walk.stream(root).find(p => keep(p) && os.isFile(p)) + + /** The entries directly inside `dir`. */ + def list(dir: os.Path): IndexedSeq[os.Path] = os.list(dir) + + /** `p` with symlinks resolved; `p` must exist. */ + def realPath(p: os.Path): os.Path = os.Path(p.toNIO.toRealPath()) + + /** A temp file holding `contents`, in `dir` or else the system temp + * directory, removed when the enclosing scope ends. + */ + def tempFile( + contents: String, + prefix: String, + suffix: String, + dir: Option[os.Path] = None + )(using ResourceScope): os.Path = + scoped( + os.temp( + contents, + dir = dir.orNull, + prefix = prefix, + suffix = suffix, + deleteOnExit = false + ) + )(os.remove(_): Unit) + + /** A temp directory, removed with its contents when the enclosing scope ends. + */ + def tempDir(prefix: String)(using ResourceScope): os.Path = + scoped(os.temp.dir(prefix = prefix, deleteOnExit = false))(os.remove.all(_)) + + // Temp paths skip `deleteOnExit`: each registration would stay in the JVM's + // exit-hook list for the rest of a flow that runs hundreds of turns. A + // failing removal is logged: by then the scope's work has its outcome. + private def scoped(path: os.Path)(remove: os.Path => Unit)(using + ResourceScope + ): os.Path = + ox.useInScope(path): p => + try remove(p) + catch case NonFatal(e) => FileSystem.log.warn(s"removing $p failed", e) + +object FileSystem: + private val log = org.slf4j.LoggerFactory.getLogger("orca.effect") + + /** For wiring and tests only: safe code cannot call this. */ + private[orca] def ambient: FileSystem = new FileSystem() diff --git a/core/src/main/scala/orca/effect/HttpClient.scala b/core/src/main/scala/orca/effect/HttpClient.scala new file mode 100644 index 000000000..eada0fa77 --- /dev/null +++ b/core/src/main/scala/orca/effect/HttpClient.scala @@ -0,0 +1,35 @@ +package orca.effect + +import language.experimental.captureChecking + +import scala.concurrent.duration.FiniteDuration + +/** A status and the body, decoded with the Content-Type charset (UTF-8 when + * absent or unsupported). + */ +final case class HttpReply(status: Int, body: String): + def isSuccess: Boolean = status / 100 == 2 + +/** An open response body read as lines, e.g. a server-sent-event stream. */ +trait HttpLines: + /** Ends at the body's end; throws `java.io.IOException` once [[close]]d or + * once the reading thread is interrupted. + */ + def lines: Iterator[String] + + /** Unblocks a thread reading [[lines]]. Callable from any thread, more than + * once. + */ + def close(): Unit + +/** Requests to one server, from [[Network.httpClient]]. `path` is appended to + * the base URL. Transport failures and timeouts throw `java.io.IOException`. + */ +trait HttpClient: + def get(path: String, timeout: FiniteDuration): HttpReply + def postJson(path: String, json: String, timeout: FiniteDuration): HttpReply + + /** Returns once headers arrive, whatever the status; no timeout bounds the + * body. + */ + def openLines(path: String): HttpLines diff --git a/core/src/main/scala/orca/effect/JdkHttpClient.scala b/core/src/main/scala/orca/effect/JdkHttpClient.scala new file mode 100644 index 000000000..dc002bd6e --- /dev/null +++ b/core/src/main/scala/orca/effect/JdkHttpClient.scala @@ -0,0 +1,69 @@ +package orca.effect + +import java.io.{BufferedReader, InputStreamReader} +import java.net.URI +import java.net.http.{HttpClient as JHttpClient, HttpRequest, HttpResponse} +import java.nio.charset.StandardCharsets +import java.util.Base64 +import scala.concurrent.duration.FiniteDuration +import scala.jdk.DurationConverters.* + +/** The `java.net.http` binding behind [[Network.httpClient]]. */ +private[effect] final class JdkHttpClient( + baseUrl: String, + user: String, + password: String +) extends HttpClient: + + // HTTP/1.1: a cleartext h2c upgrade hangs some servers (opencode) on a POST + // with a body. + private val client: JHttpClient = + JHttpClient.newBuilder().version(JHttpClient.Version.HTTP_1_1).build() + + // Preemptive: java.net.http's `Authenticator` only answers a challenge. + private val authHeader: String = + "Basic " + Base64.getEncoder.encodeToString( + s"$user:$password".getBytes(StandardCharsets.UTF_8) + ) + + def get(path: String, timeout: FiniteDuration): HttpReply = + send(request(path).timeout(timeout.toJava).GET().build()) + + def postJson(path: String, json: String, timeout: FiniteDuration): HttpReply = + send( + request(path) + .timeout(timeout.toJava) + .header("Content-Type", "application/json") + .POST(HttpRequest.BodyPublishers.ofString(json)) + .build() + ) + + def openLines(path: String): HttpLines = + // A request timeout would also bound reading the body on newer JDKs. + val body = client + .send( + request(path).GET().build(), + HttpResponse.BodyHandlers.ofInputStream() + ) + .body() + val reader = + new BufferedReader(new InputStreamReader(body, StandardCharsets.UTF_8)) + new HttpLines: + def lines: Iterator[String] = + Iterator.continually(reader.readLine()).takeWhile(_ != null) + // Closing the body stream unblocks a parked `readLine`; closing an + // `ofLines` stream does not. + def close(): Unit = body.close() + + /** Abandons in-flight requests: an open line stream may outlive its reader. + */ + def shutdown(): Unit = client.shutdownNow() + + private def send(req: HttpRequest): HttpReply = + val resp = client.send(req, HttpResponse.BodyHandlers.ofString()) + HttpReply(resp.statusCode(), resp.body()) + + private def request(path: String): HttpRequest.Builder = + HttpRequest + .newBuilder(URI.create(baseUrl + path)) + .header("Authorization", authHeader) diff --git a/core/src/main/scala/orca/effect/LocalServer.scala b/core/src/main/scala/orca/effect/LocalServer.scala new file mode 100644 index 000000000..f243d51b7 --- /dev/null +++ b/core/src/main/scala/orca/effect/LocalServer.scala @@ -0,0 +1,59 @@ +package orca.effect + +import io.netty.channel.MultiThreadIoEventLoopGroup +import io.netty.channel.nio.NioIoHandler +import io.netty.channel.socket.nio.NioServerSocketChannel +import org.slf4j.LoggerFactory +import ox.{Ox, useInScope} +import sttp.shared.Identity +import sttp.tapir.server.ServerEndpoint +import sttp.tapir.server.netty.NettyConfig.EventLoopConfig +import sttp.tapir.server.netty.sync.NettySyncServer + +import java.util.concurrent.TimeUnit +import scala.concurrent.duration.{DurationInt, FiniteDuration} +import scala.util.control.NonFatal + +/** The Netty binding behind [[Network.serveLocal]]. */ +private[effect] object LocalServer: + + private val log = LoggerFactory.getLogger("orca.effect") + + def serve( + endpoints: List[ServerEndpoint[Any, Identity]], + requestTimeout: FiniteDuration + )(using Ox): Int = + // tapir's own group waits out a 2s shutdown quiet period, twice per stop. + val group = useInScope( + MultiThreadIoEventLoopGroup(1, NioIoHandler.newFactory()) + )(shutdown) + // tapir's `start()` leaves teardown to the caller, and a leaked binding + // strands its event-loop threads for the life of the JVM. + val binding = useInScope( + NettySyncServer() + .host("127.0.0.1") + .port(0) + .modifyConfig( + // Netty requires the idle timeout to exceed the request timeout. + _.requestTimeout(requestTimeout) + .idleTimeout(requestTimeout + 1.minute) + .eventLoopConfig( + EventLoopConfig(() => group, classOf[NioServerSocketChannel]) + ) + .withDontShutdownEventLoopGroupOnClose + ) + .addEndpoints(endpoints) + .start() + ): b => + try b.stop() + catch case NonFatal(e) => log.warn("local server stop failed", e) + binding.port + + // Uninterruptible: the wait is bounded by the 2s timeout, and an interrupted + // scope teardown would otherwise leave the event-loop thread running. + private def shutdown(group: MultiThreadIoEventLoopGroup): Unit = + try + group + .shutdownGracefully(0, 2, TimeUnit.SECONDS) + .syncUninterruptibly(): Unit + catch case NonFatal(e) => log.warn("local server shutdown failed", e) diff --git a/core/src/main/scala/orca/effect/Network.scala b/core/src/main/scala/orca/effect/Network.scala new file mode 100644 index 000000000..32ff90441 --- /dev/null +++ b/core/src/main/scala/orca/effect/Network.scala @@ -0,0 +1,37 @@ +package orca.effect + +import language.experimental.captureChecking + +import caps.assumeSafe +import ox.{Ox, ResourceScope} +import sttp.shared.Identity +import sttp.tapir.server.ServerEndpoint + +import scala.concurrent.duration.FiniteDuration + +/** Authority to open network connections and bind local servers. */ +@assumeSafe +final class Network private () extends caps.SharedCapability: + + /** Serves `endpoints` on `127.0.0.1` at an ephemeral port until the enclosing + * scope ends, and returns the port. Requests are handled in forks of that + * scope. A failure to stop the server is logged, not thrown. + */ + def serveLocal( + endpoints: List[ServerEndpoint[Any, Identity]], + requestTimeout: FiniteDuration + )(using Ox): Int = + LocalServer.serve(endpoints, requestTimeout) + + /** An HTTP/1.1 client for the server at `baseUrl`, sending Basic auth + * preemptively on every request. Shut down when the enclosing scope ends, + * abandoning in-flight requests and open line streams. + */ + def httpClient(baseUrl: String, user: String, password: String)(using + ResourceScope + ): HttpClient = + ox.useInScope(JdkHttpClient(baseUrl, user, password))(_.shutdown()) + +object Network: + /** For wiring and tests only: safe code cannot call this. */ + private[orca] def ambient: Network = new Network() diff --git a/core/src/main/scala/orca/effect/Processes.scala b/core/src/main/scala/orca/effect/Processes.scala new file mode 100644 index 000000000..8c83e7294 --- /dev/null +++ b/core/src/main/scala/orca/effect/Processes.scala @@ -0,0 +1,61 @@ +package orca.effect + +import language.experimental.captureChecking + +import caps.assumeSafe +import orca.subprocess.{ + CappedResult, + CliResult, + OsPipedSubProcess, + PipedCliProcess, + QuietProc +} + +/** Authority to start child processes. Every child's stderr is captured, never + * inherited. + */ +@assumeSafe +final class Processes private () extends caps.SharedCapability: + + /** Runs `args` to completion; a non-zero exit is returned, not thrown. `env` + * is added to the inherited environment. + */ + def call( + args: Seq[String], + cwd: os.Path, + env: Map[String, String] = Map.empty, + stdin: String = "" + ): CliResult = + val result = QuietProc.call(args, cwd = cwd, env = env, stdin = stdin) + Processes.log.debug( + "exit {}: {}", + result.exitCode, + args.headOption.getOrElse("?") + ) + CliResult(result.exitCode, result.out.text(), result.err.text()) + + /** [[call]], keeping at most `maxBytes` of each of stdout and stderr. */ + def callCapped( + args: Seq[String], + maxBytes: Int, + cwd: os.Path, + env: Map[String, String] = Map.empty + ): CappedResult = + QuietProc.callCapped(args, maxBytes, cwd = cwd, env = env) + + /** Starts `args` with piped stdin, stdout and stderr. The caller must drain + * both `stdoutLines` and `stderrLines` until EOF: a child blocks once either + * pipe's buffer is full. + */ + def spawnPiped( + args: Seq[String], + cwd: os.Path, + env: Map[String, String] = Map.empty + ): PipedCliProcess = + OsPipedSubProcess.spawn(args, env = env, cwd = cwd) + +object Processes: + private val log = org.slf4j.LoggerFactory.getLogger("orca.proc") + + /** For wiring and tests only: safe code cannot call this. */ + private[orca] def ambient: Processes = new Processes() diff --git a/core/src/main/scala/orca/json/JsonCodec.scala b/core/src/main/scala/orca/json/JsonCodec.scala new file mode 100644 index 000000000..10f340e89 --- /dev/null +++ b/core/src/main/scala/orca/json/JsonCodec.scala @@ -0,0 +1,62 @@ +package orca.json + +import language.experimental.captureChecking + +import caps.assumeSafe +import com.github.plokhotnyuk.jsoniter_scala.core.{ + JsonReader, + JsonValueCodec, + JsonWriter, + readFromArray, + readFromString, + writeToArray, + writeToString +} +import com.github.plokhotnyuk.jsoniter_scala.macros.{ + CodecMakerConfig, + ConfiguredJsonValueCodec, + JsonCodecMaker +} + +/** A jsoniter-scala codec that safe code can derive: `derives JsonCodec`, or + * `JsonCodec.derived(using JsonCodec.config.with…)` for a non-default config. + */ +@assumeSafe +trait JsonCodec[A] extends ConfiguredJsonValueCodec[A] + +@assumeSafe +object JsonCodec: + + /** jsoniter's default config, to refine with its `with…` methods. */ + inline def config: CodecMakerConfig = CodecMakerConfig + + inline def derived[A](using + inline config: CodecMakerConfig = CodecMakerConfig + ): JsonCodec[A] = + wrap(JsonCodecMaker.make[A](config)) + + /** `JsonCodecMaker.make`: a plain codec, e.g. for a collection type. */ + inline def make[A](using + inline config: CodecMakerConfig = CodecMakerConfig + ): JsonValueCodec[A] = + JsonCodecMaker.make[A](config) + + /** Throws `JsonReaderException` when `s` is not an `A`. */ + def readString[A](s: String)(using JsonValueCodec[A]): A = + readFromString[A](s) + def writeString[A](a: A)(using JsonValueCodec[A]): String = writeToString(a) + + /** Throws `JsonReaderException` when `bytes` are not an `A`. */ + def readBytes[A](bytes: IArray[Byte])(using JsonValueCodec[A]): A = + readFromArray[A](IArray.genericWrapArray(bytes).toArray) + def writeBytes[A](a: A)(using JsonValueCodec[A]): IArray[Byte] = + // Nothing else holds the array, so the immutable view captures nothing. + caps.unsafe.unsafeAssumePure(IArray.unsafeFromArray(writeToArray(a))) + + /** `codec` as a [[JsonCodec]]. */ + private[json] def wrap[A](codec: JsonValueCodec[A]): JsonCodec[A] = + new JsonCodec[A]: + def decodeValue(in: JsonReader, default: A): A = + codec.decodeValue(in, default) + def encodeValue(x: A, out: JsonWriter): Unit = codec.encodeValue(x, out) + def nullValue: A = codec.nullValue diff --git a/tools/src/main/scala/orca/json/JsonData.scala b/core/src/main/scala/orca/json/JsonData.scala similarity index 64% rename from tools/src/main/scala/orca/json/JsonData.scala rename to core/src/main/scala/orca/json/JsonData.scala index 6a4e9d184..6f57bdb51 100644 --- a/tools/src/main/scala/orca/json/JsonData.scala +++ b/core/src/main/scala/orca/json/JsonData.scala @@ -1,23 +1,22 @@ package orca.json +import language.experimental.captureChecking + +import caps.assumeSafe import com.github.plokhotnyuk.jsoniter_scala.core.{ JsonReader, JsonValueCodec, JsonWriter } -import com.github.plokhotnyuk.jsoniter_scala.macros.{ - CodecMakerConfig, - ConfiguredJsonValueCodec, - JsonCodecMaker -} +import com.github.plokhotnyuk.jsoniter_scala.macros.CodecMakerConfig import sttp.tapir.Schema -import scala.compiletime.{constValueTuple, erasedValue, summonFrom} +import scala.compiletime.{erasedValue, summonFrom} import scala.deriving.Mirror -/** Bundles a tapir `Schema` and a jsoniter-scala `ConfiguredJsonValueCodec` for - * a type. Flow scripts use `derives JsonData` on case classes that travel in - * and out of LLM calls as structured JSON. +/** Bundles a tapir `Schema` and a [[JsonCodec]] for a type. Flow scripts use + * `derives JsonData` on case classes that travel in and out of LLM calls as + * structured JSON. * * Scripts must import via `import orca.{*, given}` — `derives JsonData` on a * case class with nested case-class fields needs the forwarder givens below in @@ -29,10 +28,12 @@ import scala.deriving.Mirror * gets a schema its codec does not match (the codec adds a `"type"` field), so * it can be a stage result but not a `resultAs` output. */ +@assumeSafe trait JsonData[A]: def schema: Schema[A] - def codec: ConfiguredJsonValueCodec[A] + def codec: JsonCodec[A] +@assumeSafe object JsonData: /** Stricter-than-default jsoniter config: missing `List` / `Set` / `Map` @@ -47,22 +48,22 @@ object JsonData: def apply[A]( schemaInstance: Schema[A], - codecInstance: ConfiguredJsonValueCodec[A] + codecInstance: JsonCodec[A] ): JsonData[A] = new JsonData[A]: val schema: Schema[A] = schemaInstance - val codec: ConfiguredJsonValueCodec[A] = codecInstance + val codec: JsonCodec[A] = codecInstance /** A type that travels as a JSON string: written with `render`, read back * through `parse`, so a persisted value `parse` refuses fails to decode. */ def fromString[A]( - parse: String => Either[String, A], - render: A => String + parse: String -> Either[String, A], + render: A -> String ): JsonData[A] = apply( Schema.schemaForString.as[A], - new ConfiguredJsonValueCodec[A]: + new JsonCodec[A]: def decodeValue(in: JsonReader, default: A): A = in.readString(null) match case null => in.decodeError("expected a string") @@ -71,6 +72,25 @@ object JsonData: def nullValue: A = null.asInstanceOf[A] ) + /** A schema derived like `derives JsonData` derives one. */ + inline def derivedSchema[A](using Mirror.Of[A]): Schema[A] = + SchemaDerivation.derived[A] + + /** `Schema.string`: the schema of a type that travels as a JSON string. */ + def stringSchema[A]: Schema[A] = Schema.string[A] + + /** `Validator.nonEmpty`: rejects an empty list. */ + def nonEmptyValidator[A]: sttp.tapir.Validator[List[A]] = + sttp.tapir.Validator.nonEmpty[A, List] + + /** The schema of a list of `A`s. `schemaForIterable` returns a + * `Schema[Iterable[A]]`; at runtime both are the same array schema. + */ + def listSchema[A](using element: Schema[A]): Schema[List[A]] = + Schema + .schemaForIterable[A, List](using element) + .asInstanceOf[Schema[List[A]]] + inline def derived[A](using m: Mirror.Of[A]): JsonData[A] = inline m match case s: Mirror.SumOf[A] => @@ -80,18 +100,17 @@ object JsonData: private inline def general[A](using Mirror.Of[A]): JsonData[A] = apply( - Schema.derived[A], - ConfiguredJsonValueCodec.derived[A](using strictCodecConfig) + SchemaDerivation.derived[A], + JsonCodec.derived[A](using strictCodecConfig) ) // jsoniter writes each case as its Mirror label, so the schema lists the // labels too rather than relying on `toString`. private inline def stringEnum[A](s: Mirror.SumOf[A]): JsonData[A] = - val labels = - constValueTuple[s.MirroredElemLabels].toList.map(_.toString).toVector + val labels = SchemaDerivation.labels[s.MirroredElemLabels].toVector apply( Schema.derivedEnumeration[A](encode = Some(a => labels(s.ordinal(a)))), - ConfiguredJsonValueCodec.derived[A](using + JsonCodec.derived[A](using strictCodecConfig.withDiscriminatorFieldName(None) ) ) @@ -99,37 +118,26 @@ object JsonData: private inline def allSingletons[T <: Tuple]: Boolean = inline erasedValue[T] match case _: EmptyTuple => true - case _: (h *: t) => + case _: (h *: t) => summonFrom: case _: ValueOf[`h`] => allSingletons[t] case _ => false - /** Wraps a plain `JsonValueCodec` as a `ConfiguredJsonValueCodec` (a marker - * interface adding no methods) by delegating all calls. Used by the - * hand-written primitive/generic givens below. - */ - private def wrap[A](c: JsonValueCodec[A]): ConfiguredJsonValueCodec[A] = - new ConfiguredJsonValueCodec[A]: - def decodeValue(in: JsonReader, default: A): A = - c.decodeValue(in, default) - def encodeValue(x: A, out: JsonWriter): Unit = c.encodeValue(x, out) - def nullValue: A = c.nullValue - // ── Primitive givens ─────────────────────────────────────────────────────── // Use the tapir Schema companion methods directly (not `summon`) to avoid // triggering the package-level `schemaFromJsonData` given, which would // reference the very instance being initialised (causing an infinite loop). given stringJsonData: JsonData[String] = - apply(Schema.schemaForString, wrap(JsonCodecMaker.make)) + apply(Schema.schemaForString, JsonCodec.derived) given intJsonData: JsonData[Int] = - apply(Schema.schemaForInt, wrap(JsonCodecMaker.make)) + apply(Schema.schemaForInt, JsonCodec.derived) given longJsonData: JsonData[Long] = - apply(Schema.schemaForLong, wrap(JsonCodecMaker.make)) + apply(Schema.schemaForLong, JsonCodec.derived) given booleanJsonData: JsonData[Boolean] = - apply(Schema.schemaForBoolean, wrap(JsonCodecMaker.make)) + apply(Schema.schemaForBoolean, JsonCodec.derived) given doubleJsonData: JsonData[Double] = - apply(Schema.schemaForDouble, wrap(JsonCodecMaker.make)) + apply(Schema.schemaForDouble, JsonCodec.derived) /** Unit serialises as `{}`, a valid round-trippable "no meaningful payload" * value. `JsonCodecMaker` doesn't support `Unit`, so the codec is hand- @@ -139,7 +147,7 @@ object JsonData: */ given unitJsonData: JsonData[Unit] = apply( Schema.schemaForUnit, - new ConfiguredJsonValueCodec[Unit]: + new JsonCodec[Unit]: def decodeValue(in: JsonReader, default: Unit): Unit = if in.isNextToken('{') then if !in.isNextToken('}') then in.objectEndOrCommaError() @@ -154,25 +162,40 @@ object JsonData: given [A](using jd: JsonData[A]): JsonData[Option[A]] = given JsonValueCodec[A] = jd.codec - apply(Schema.schemaForOption(using jd.schema), wrap(JsonCodecMaker.make)) + apply( + Schema.schemaForOption(using jd.schema), + JsonCodec.derived + ) given [A](using jd: JsonData[A]): JsonData[List[A]] = given JsonValueCodec[A] = jd.codec - // schemaForIterable returns Schema[Iterable[A]]; the cast to Schema[List[A]] - // is safe because at runtime both are the same array schema with A elements. - apply( - Schema - .schemaForIterable[A, List](using jd.schema) - .asInstanceOf[Schema[List[A]]], - wrap(JsonCodecMaker.make) - ) + apply(listSchema(using jd.schema), JsonCodec.derived) + + // Inline so the schema is named after the concrete element types. + inline given [A, B](using + jdA: JsonData[A], + jdB: JsonData[B] + ): JsonData[(A, B)] = + tuple2(SchemaDerivation.derived[(A, B)], jdA, jdB) + + inline given [A, B, C](using + jdA: JsonData[A], + jdB: JsonData[B], + jdC: JsonData[C] + ): JsonData[(A, B, C)] = + tuple3(SchemaDerivation.derived[(A, B, C)], jdA, jdB, jdC) - given [A, B](using jdA: JsonData[A], jdB: JsonData[B]): JsonData[(A, B)] = + private def tuple2[A, B]( + schema: Schema[(A, B)], + jdA: JsonData[A], + jdB: JsonData[B] + ): JsonData[(A, B)] = given JsonValueCodec[A] = jdA.codec given JsonValueCodec[B] = jdB.codec - apply(Schema.derived[(A, B)], wrap(JsonCodecMaker.make)) + apply(schema, JsonCodec.derived) - given [A, B, C](using + private def tuple3[A, B, C]( + schema: Schema[(A, B, C)], jdA: JsonData[A], jdB: JsonData[B], jdC: JsonData[C] @@ -180,9 +203,10 @@ object JsonData: given JsonValueCodec[A] = jdA.codec given JsonValueCodec[B] = jdB.codec given JsonValueCodec[C] = jdC.codec - apply(Schema.derived[(A, B, C)], wrap(JsonCodecMaker.make)) + apply(schema, JsonCodec.derived) +@assumeSafe given schemaFromJsonData[A](using jd: JsonData[A]): Schema[A] = jd.schema -given codecFromJsonData[A](using jd: JsonData[A]): ConfiguredJsonValueCodec[A] = - jd.codec +@assumeSafe +given codecFromJsonData[A](using jd: JsonData[A]): JsonCodec[A] = jd.codec diff --git a/core/src/main/scala/orca/json/JsonSchemas.scala b/core/src/main/scala/orca/json/JsonSchemas.scala new file mode 100644 index 000000000..e66fcb557 --- /dev/null +++ b/core/src/main/scala/orca/json/JsonSchemas.scala @@ -0,0 +1,108 @@ +package orca.json + +import language.experimental.captureChecking + +import caps.assumeSafe +import io.circe.{Json, JsonObject} +import io.circe.syntax.EncoderOps +import sttp.apispec.circe.encoderSchema +import sttp.tapir.Schema +import sttp.tapir.docs.apispec.schema.TapirSchemaToJsonSchema + +import scala.util.boundary +import scala.util.boundary.break + +/** JSON-Schema strings in OpenAI's "strict" structured-output dialect (used by + * both `codex exec --output-schema` and `claude --json-schema`): + * + * - every object node carries `additionalProperties: false`; + * - every object's `required` array lists every key in `properties`. + * + * Optional fields and fields with Scala-side defaults are marked nullable, so + * requiring them is safe — the agent emits `null` or an empty list rather than + * omitting. + * + * A `Map[String, _]` field has no fixed key set, so it can't be expressed in + * the strict dialect; such a schema is refused, with a message naming the fix. + */ +@assumeSafe +private[orca] object JsonSchemas: + + /** `schema` in the strict dialect, or why it has no strict form. */ + def strict[O](using schema: Schema[O]): Either[String, String] = + val jsonSchema = + TapirSchemaToJsonSchema(schema, markOptionsAsNullable = true) + // Tapir stamps `$schema: .../draft/2020-12/schema` on its output; the + // claude CLI (observed on 2.1.207) has no 2020-12 meta-schema registered + // and rejects any schema declaring that dialect ("no schema with key or ref + // ...") before the turn starts. No consumer needs the declaration, so strip + // it. + toOpenAiStrict(jsonSchema.asJson).map( + _.mapObject(_.remove("$schema")).noSpaces + ) + + /** Every object subtree of `json` with the two strict-mode constraints + * injected, or why one cannot carry them. + */ + def toOpenAiStrict(json: Json): Either[String, Json] = + boundary(Right(strictly(json))) + + private def strictly(json: Json)(using + boundary.Label[Either[String, Json]] + ): Json = + json.fold( + jsonNull = json, + jsonBoolean = _ => json, + jsonNumber = _ => json, + jsonString = _ => json, + jsonArray = arr => Json.fromValues(arr.map(strictly)), + jsonObject = obj => Json.fromJsonObject(transformObject(obj)) + ) + + private def transformObject(obj: JsonObject)(using + boundary.Label[Either[String, Json]] + ): JsonObject = + val recursed = JsonObject.fromIterable( + obj.toIterable.map((k, v) => k -> strictly(v)) + ) + if isMapShaped(recursed) then break(Left(MapShapedMessage)) + if isObjectSchemaNode(recursed) then addStrictConstraints(recursed) + else recursed + + /** Tapir renders a `Map[String, T]` field as an object node whose + * `additionalProperties` is `T`'s own sub-schema (no `properties` key — the + * keys are unbounded). Overwriting it with `false` would silently discard + * the value-type constraint. + */ + private def isMapShaped(obj: JsonObject): Boolean = + obj("additionalProperties").exists(!_.isBoolean) + + private val MapShapedMessage: String = + "resultAs[O]: output schema has a Map[String, _]-shaped field " + + "(an object with a value-type `additionalProperties` schema), " + + "which OpenAI's strict structured-output mode doesn't support. " + + "Model it as a List of key/value case classes instead." + + /** A node is an "object schema" — eligible for the strict-mode constraints — + * when it declares `"type": "object"` AND carries a non-empty `"properties"` + * object. The properties check rules out empty/marker objects like + * `{"type":"object"}`. + */ + private def isObjectSchemaNode(obj: JsonObject): Boolean = + obj("type").flatMap(_.asString).contains("object") && + obj("properties").flatMap(_.asObject).exists(_.nonEmpty) + + private def addStrictConstraints(obj: JsonObject): JsonObject = + val props = + obj("properties").flatMap(_.asObject).getOrElse(JsonObject.empty) + val allKeys = props.keys.toList + // The Map[String, T] shape was already rejected above, so + // `additionalProperties` here is absent or already boolean — don't clobber + // an explicit boolean, only fill in the missing default. + val withAdditional = + if obj.contains("additionalProperties") then obj + else obj.add("additionalProperties", Json.False) + withAdditional.add( + "required", + Json.fromValues(allKeys.map(Json.fromString)) + ) diff --git a/core/src/main/scala/orca/json/SchemaDerivation.scala b/core/src/main/scala/orca/json/SchemaDerivation.scala new file mode 100644 index 000000000..60190293c --- /dev/null +++ b/core/src/main/scala/orca/json/SchemaDerivation.scala @@ -0,0 +1,85 @@ +package orca.json + +import language.experimental.captureChecking + +import sttp.tapir.{FieldName, Schema, SchemaType} +import sttp.tapir.Schema.SName + +import scala.compiletime.{constValue, erasedValue, summonFrom, summonInline} +import scala.deriving.Mirror + +/** Schema derivation standing in for tapir's `Schema.derived` (Magnolia), whose + * expansion fails capture checking. For products, tuples and sealed sums it + * renders to the same JSON Schema, up to key order; tapir's annotations + * (`@description` etc.) are not read. A field's type needs a `Schema` in + * scope; a sum's case is derived here when it has none. Recursive types are + * not supported. + */ +private[orca] object SchemaDerivation: + + inline def derived[A](using m: Mirror.Of[A]): Schema[A] = + inline m match + case p: Mirror.ProductOf[A] => product[A](p) + case s: Mirror.SumOf[A] => sum[A](s) + + private inline def product[A](p: Mirror.ProductOf[A]): Schema[A] = + val fields = labels[p.MirroredElemLabels] + .zip(elemSchemas[p.MirroredElemTypes]) + .zipWithIndex + .map: + case ((label, s), i) => + SchemaType.SProductField[A, Any]( + FieldName(label), + s.asInstanceOf[Schema[Any]], + a => Some(a.asInstanceOf[Product].productElement(i)) + ) + val schema = Schema[A](SchemaType.SProduct(fields), Some(name[A])) + if name[A].fullName.startsWith("scala.Tuple") then schema.tuple else schema + + private inline def sum[A](s: Mirror.SumOf[A]): Schema[A] = + val (leaves, choose) = cases[A](s) + Schema[A]( + SchemaType.SCoproduct[A](leaves.distinct, None)(a => + Some(SchemaType.SchemaWithValue(choose(a).asInstanceOf[Schema[Any]], a)) + ), + Some(name[A]) + ) + + /** The schemas of `s`'s cases, a nested sum's flattened into them as Magnolia + * does, and the one a value takes. + */ + private inline def cases[A]( + s: Mirror.SumOf[A] + ): (List[Schema[?]], A -> Schema[?]) = + val each = caseList[s.MirroredElemTypes] + (each.flatMap(_._1), a => each(s.ordinal(a))._2(a)) + + private inline def caseList[T <: Tuple] + : List[(List[Schema[?]], Any -> Schema[?])] = + inline erasedValue[T] match + case _: EmptyTuple => Nil + case _: (h *: t) => + val head: (List[Schema[?]], Any -> Schema[?]) = summonFrom: + case m: Mirror.SumOf[`h`] => + val (leaves, choose) = cases[h](m) + (leaves, a => choose(a.asInstanceOf[h])) + case s: Schema[`h`] => (List(s), _ => s) + case m: Mirror.ProductOf[`h`] => + val s = product[h](m) + (List(s), _ => s) + head :: caseList[t] + + private inline def name[A]: SName = + val (full, args) = TypeNames.schemaName[A] + SName(full, args) + + // `constValueTuple(...).toList` doesn't type-check under capture checking. + inline def labels[T <: Tuple]: List[String] = + inline erasedValue[T] match + case _: EmptyTuple => Nil + case _: (h *: t) => constValue[h].toString :: labels[t] + + private inline def elemSchemas[T <: Tuple]: List[Schema[?]] = + inline erasedValue[T] match + case _: EmptyTuple => Nil + case _: (h *: t) => summonInline[Schema[h]] :: elemSchemas[t] diff --git a/core/src/main/scala/orca/json/TypeNames.scala b/core/src/main/scala/orca/json/TypeNames.scala new file mode 100644 index 000000000..a3b4737ad --- /dev/null +++ b/core/src/main/scala/orca/json/TypeNames.scala @@ -0,0 +1,25 @@ +package orca.json + +import scala.quoted.* + +/** A type's schema name: its fully qualified name plus those of its type + * arguments, recursively. A parameterless enum case is `.`; + * Magnolia names it `.`, which JSON Schema keys don't show. + */ +private[json] object TypeNames: + inline def schemaName[A]: (String, List[String]) = ${ schemaNameImpl[A] } + + private def schemaNameImpl[A: Type](using + Quotes + ): Expr[(String, List[String])] = + import quotes.reflect.* + // A parameterless enum case's type symbol is the enum itself. + def symbolOf(t: TypeRepr): Symbol = t.dealias match + case ref: TermRef => ref.termSymbol + case other => other.typeSymbol + def full(t: TypeRepr): String = symbolOf(t).fullName.replace("$", "") + def args(t: TypeRepr): List[TypeRepr] = t.dealias match + case AppliedType(_, as) => as.flatMap(a => a :: args(a)) + case _ => Nil + val t = TypeRepr.of[A] + Expr((full(t), args(t).map(full))) diff --git a/core/src/main/scala/orca/mcp/mcp.scala b/core/src/main/scala/orca/mcp/mcp.scala new file mode 100644 index 000000000..aca35a5ed --- /dev/null +++ b/core/src/main/scala/orca/mcp/mcp.scala @@ -0,0 +1,84 @@ +package orca.mcp + +import language.experimental.captureChecking + +import caps.assumeSafe +import chimp.protocol.ToolContent +import chimp.server.{ + McpServer, + NoStructuredOutput, + ServerContext, + ServerTool, + ToolResult +} +import com.github.plokhotnyuk.jsoniter_scala.core.{ + JsonReaderException, + readFromString +} +import io.circe.Decoder +import orca.json.JsonData +import sttp.shared.Identity +import ox.either.catching +import sttp.tapir.server.ServerEndpoint + +// The chimp MCP-server operations safe code uses. + +@assumeSafe +type Tool[I] = + ServerTool[I, NoStructuredOutput, Identity, ServerContext[Identity]] + +/** What a [[Tool]] call returns: text contents, flagged as an error or not. */ +@assumeSafe +type Result = ToolResult[NoStructuredOutput] + +/** A tool whose input is decoded through its [[orca.json.JsonData]] codec. */ +@assumeSafe +def tool[I](name: String, description: String)( + handle: I => Result +)(using jd: JsonData[I]): Tool[I] = + chimp.server + .tool(name) + .description(description) + .input[I](using jd.schema, decoder(jd)) + .handle(handle) + +@assumeSafe +def text(text: String): Result = + ToolResult.text(text) +@assumeSafe +def error(message: String): Result = + ToolResult.error(message) +@assumeSafe +def fromEither(result: Either[String, String]): Result = + ToolResult.fromEither(result) + +/** `result` with `f` applied to each of its text contents. */ +@assumeSafe +def mapText(result: Result)(f: String => String): Result = + result.copy(content = result.content.map: + case t: ToolContent.Text => t.copy(text = f(t.text)) + case other => other) + +/** The path [[endpoint]] serves at. */ +@assumeSafe +val endpointPath: String = "mcp" + +/** An endpoint serving `tools` over MCP's HTTP transport at + * `/`[[endpointPath]]. + */ +@assumeSafe +def endpoint(tools: List[Tool[?]]): ServerEndpoint[Any, Identity] = + McpServer(tools = tools).endpoint(List(endpointPath)) + +// jsoniter appends a multi-line buffer dump to its messages; the agent gets +// the first line. +private def decoder[I](jd: JsonData[I]): Decoder[I] = + Decoder.decodeJson.emap: json => + readFromString[I](json.noSpaces)(using jd.codec) + .catching[JsonReaderException] + .left + .map(e => + Option(e.getMessage) + .flatMap(_.linesIterator.nextOption()) + .getOrElse(e.toString) + ) diff --git a/core/src/main/scala/orca/safe/Logger.scala b/core/src/main/scala/orca/safe/Logger.scala new file mode 100644 index 000000000..a46b4e4c0 --- /dev/null +++ b/core/src/main/scala/orca/safe/Logger.scala @@ -0,0 +1,29 @@ +package orca.safe + +import language.experimental.captureChecking + +import caps.assumeSafe + +/** An slf4j logger safe code may create without a capability: logging is + * deliberately left untracked. + */ +@assumeSafe +final class Logger(name: String): + private val underlying = org.slf4j.LoggerFactory.getLogger(name) + + def debug(msg: => String): Unit = + if underlying.isDebugEnabled then underlying.debug(msg) + def debug(msg: => String, t: Throwable): Unit = + if underlying.isDebugEnabled then underlying.debug(msg, t) + def warn(msg: String): Unit = underlying.warn(msg) + def warn(msg: String, t: Throwable): Unit = underlying.warn(msg, t) + def error(msg: String): Unit = underlying.error(msg) + def error(msg: String, t: Throwable): Unit = underlying.error(msg, t) + +@assumeSafe +object Logger: + /** Writes `line` to the JVM's stderr, bypassing the logging configuration: + * for opt-in debug dumps and for messages the user must see whatever the log + * level. + */ + def stderr(line: String): Unit = System.err.println(line) diff --git a/core/src/main/scala/orca/safe/prelude.scala b/core/src/main/scala/orca/safe/prelude.scala new file mode 100644 index 000000000..4c9950e04 --- /dev/null +++ b/core/src/main/scala/orca/safe/prelude.scala @@ -0,0 +1,92 @@ +package orca.safe + +import language.experimental.captureChecking + +import caps.assumeSafe + +import java.nio.file.FileSystems +import scala.language.implicitConversions + +/** Pure helpers over libraries safe code can't reference directly. Import with + * `import orca.safe.prelude.{*, given}`. + */ +@assumeSafe +object prelude: + extension [A](a: A) + // Not inline: Scala 3.10.0 types an inline argument's proxy without its + // captures, so `.discard` on a capturing value fails capture checking. + def discard: Unit = () + inline def pipe[B](inline f: A => B): B = f(a) + inline def tap(inline f: A => Unit): A = + f(a) + a + + /** `ox.either`, without its compile-time check against nesting. */ + inline def either[E, A]( + inline body: scala.util.boundary.Label[Either[E, A]] ?=> A + ): Either[E, A] = + scala.util.boundary(Right(body)) + export ox.either.{catching, ok, orThrow} + + /** `scala.util.control.NonFatal`. */ + object NonFatal: + def unapply(t: Throwable): Option[Throwable] = + scala.util.control.NonFatal.unapply(t) + + export scala.concurrent.duration.DurationInt + export java.nio.charset.StandardCharsets.UTF_8 + + // Imported givens take precedence over os-lib's implicit-scope conversions, + // which safe code can't reference. + given Conversion[String, os.PathChunk] = os.PathChunk.StringPathChunk(_) + given Conversion[os.RelPath, os.PathChunk] = os.PathChunk.RelPathChunk(_) + given Conversion[os.SubPath, os.PathChunk] = os.PathChunk.SubPathChunk(_) + val up: os.RelPath = os.up + val sub: os.SubPath = os.sub + + /** `path` resolved against `base` unless absolute. */ + def resolvePath(path: String, base: os.Path): os.Path = os.Path(path, base) + + /** `path` as a relative path, if it is one. */ + def relPath(path: String): Option[os.RelPath] = parsed(os.RelPath(path)) + + /** `path` as a path below a root, if it is one. */ + def subPath(path: String): Option[os.SubPath] = parsed(os.SubPath(path)) + + /** `path` as an absolute path, if it is one. */ + def absolutePath(path: String): Option[os.Path] = parsed(os.Path(path)) + + private def parsed[P](parse: => P): Option[P] = + try Some(parse) + catch case _: IllegalArgumentException => None + + /** `text` encoded as UTF-8. */ + def utf8Bytes(text: String): IArray[Byte] = + // Nothing else holds the array, so the immutable view captures nothing. + caps.unsafe.unsafeAssumePure(IArray.unsafeFromArray(text.getBytes(UTF_8))) + + /** `bytes` decoded as UTF-8, malformed input replaced. */ + def utf8String(bytes: IArray[Byte]): String = + String(IArray.genericWrapArray(bytes).toArray, UTF_8) + + /** SHA-256 of `bytes`. */ + def sha256(bytes: Array[Byte]): Array[Byte] = + java.security.MessageDigest.getInstance("SHA-256").digest(bytes) + + /** A random UUID's string form. Randomness is untracked, like + * `scala.util.Random`, which safe code reaches directly. + */ + def randomUuid(): String = java.util.UUID.randomUUID().toString + + /** The classpath resource at absolute `path` as UTF-8 text, `None` when there + * is none. Resources are part of the program, not effects. + */ + def resourceText(path: String): Option[String] = + Option(getClass.getResourceAsStream(path)).map: stream => + try scala.io.Source.fromInputStream(stream, "UTF-8").mkString + finally stream.close() + + /** A matcher for the JVM's default glob syntax. */ + def globMatcher(glob: String): os.RelPath => Boolean = + val m = FileSystems.getDefault.getPathMatcher(s"glob:$glob") + p => m.matches(p.toNIO) diff --git a/core/src/main/scala/orca/subprocess/CappedResult.scala b/core/src/main/scala/orca/subprocess/CappedResult.scala new file mode 100644 index 000000000..d10423bd1 --- /dev/null +++ b/core/src/main/scala/orca/subprocess/CappedResult.scala @@ -0,0 +1,13 @@ +package orca.subprocess + +/** Outcome of [[orca.effect.Processes.callCapped]]. `out` is a prefix of what + * the child wrote whenever `truncated` — and, being a prefix of bytes rather + * than of characters, can end in a replacement character where the cut split + * one. + */ +case class CappedResult( + exitCode: Int, + out: String, + truncated: Boolean, + err: String +) diff --git a/tools/src/main/scala/orca/subprocess/CliProcess.scala b/core/src/main/scala/orca/subprocess/CliProcess.scala similarity index 86% rename from tools/src/main/scala/orca/subprocess/CliProcess.scala rename to core/src/main/scala/orca/subprocess/CliProcess.scala index 850d80e9e..c1d0c5806 100644 --- a/tools/src/main/scala/orca/subprocess/CliProcess.scala +++ b/core/src/main/scala/orca/subprocess/CliProcess.scala @@ -61,11 +61,11 @@ trait PipedCliProcess extends CliProcess: */ def tryExitCode: Option[Int] - /** Cookie put in this process's environment at spawn; the backend's - * environment-cookie sweep looks for it. `None` only where there is no OS - * process to carry one (test fakes) — anything that really spawns must - * answer `Some`, or its turn's leaked work goes unreported with nothing to - * show that it was never looked for. Abstract for that reason: a default - * would make "didn't think about it" and "no process" the same answer. + /** Live processes, other than this JVM, carrying the [[EnvCookie]] put in + * this process's environment at spawn — including ones that detached from + * its tree. Empty where the platform cannot be scanned (anything but Linux), + * and where there is no OS process (test fakes). Abstract so that anything + * really spawning answers it: a default would make "didn't think about it" + * and "no process" the same answer. */ - def envCookie: Option[EnvCookie] + def survivors(): List[ProcessHandle] diff --git a/tools/src/main/scala/orca/subprocess/CliResult.scala b/core/src/main/scala/orca/subprocess/CliResult.scala similarity index 57% rename from tools/src/main/scala/orca/subprocess/CliResult.scala rename to core/src/main/scala/orca/subprocess/CliResult.scala index ddcf272a7..ecb80d1ff 100644 --- a/tools/src/main/scala/orca/subprocess/CliResult.scala +++ b/core/src/main/scala/orca/subprocess/CliResult.scala @@ -1,3 +1,4 @@ package orca.subprocess +/** A finished child process: its exit code and captured output. */ case class CliResult(exitCode: Int, stdout: String, stderr: String) diff --git a/core/src/main/scala/orca/subprocess/CookieScan.scala b/core/src/main/scala/orca/subprocess/CookieScan.scala new file mode 100644 index 000000000..645a3127f --- /dev/null +++ b/core/src/main/scala/orca/subprocess/CookieScan.scala @@ -0,0 +1,43 @@ +package orca.subprocess + +import java.nio.charset.StandardCharsets.UTF_8 +import java.nio.file.{Files, Path} +import scala.jdk.StreamConverters.* +import scala.util.control.NonFatal + +/** The `/proc` scan behind [[PipedCliProcess.survivors]]. Linux only: no + * equivalent has been verified on macOS, so elsewhere it finds nothing. + */ +private[orca] object CookieScan: + + private val ProcFs: Path = Path.of("/proc") + + /** `environ`'s entry separator. */ + private val Nul: Char = 0 + + /** Whether this machine exposes readable per-process environments. */ + val supported: Boolean = Files.isDirectory(ProcFs) + + /** Live processes other than this JVM whose environment carries `cookie`. */ + def carrying(cookie: EnvCookie): List[ProcessHandle] = + if !supported then Nil + else + val entry = s"${EnvCookie.VarName}=${cookie.value}" + val self = ProcessHandle.current().pid + ProcessHandle + .allProcesses() + .toScala(List) + .filter(h => h.pid != self && carriesEntry(h.pid, entry)) + + /** Compares whole NUL-delimited entries, so a variable whose name merely ends + * with ours cannot match. A process that exited mid-scan and one owned by + * another user both read as `false` — the file is gone or unreadable, which + * is all this needs to decide. + */ + private def carriesEntry(pid: Long, entry: String): Boolean = + try + new String( + Files.readAllBytes(ProcFs.resolve(pid.toString).resolve("environ")), + UTF_8 + ).split(Nul).contains(entry) + catch case NonFatal(_) => false diff --git a/tools/src/main/scala/orca/subprocess/EnvCookie.scala b/core/src/main/scala/orca/subprocess/EnvCookie.scala similarity index 88% rename from tools/src/main/scala/orca/subprocess/EnvCookie.scala rename to core/src/main/scala/orca/subprocess/EnvCookie.scala index c36457ba7..94d942da8 100644 --- a/tools/src/main/scala/orca/subprocess/EnvCookie.scala +++ b/core/src/main/scala/orca/subprocess/EnvCookie.scala @@ -12,8 +12,8 @@ import java.util.UUID * `setsid`, a double fork and reparenting to init — exactly the moves that * take a process out of `ProcessHandle.descendants`. * - * One cookie per spawn, minted in [[CliRunner.spawnPiped]]; every agent turn - * spawns a fresh CLI process. + * One cookie per spawn, minted in [[OsPipedSubProcess.spawn]]; every agent + * turn spawns a fresh CLI process. */ private[orca] opaque type EnvCookie = String diff --git a/core/src/main/scala/orca/subprocess/OsPipedSubProcess.scala b/core/src/main/scala/orca/subprocess/OsPipedSubProcess.scala new file mode 100644 index 000000000..027285cb9 --- /dev/null +++ b/core/src/main/scala/orca/subprocess/OsPipedSubProcess.scala @@ -0,0 +1,168 @@ +package orca.subprocess + +import org.slf4j.LoggerFactory +import ox.{abandonOnInterruptReads, discard, raceResult, sleep} + +import java.io.{BufferedReader, InputStream, InputStreamReader} +import java.nio.charset.StandardCharsets.UTF_8 +import java.util.concurrent.atomic.AtomicReference +import scala.concurrent.duration.* +import scala.jdk.CollectionConverters.given + +/** Takes the cookie unwrapped: a process orca actually spawned always carries + * one, so it cannot be constructed without it. + */ +private[subprocess] final class OsPipedSubProcess private ( + sub: os.SubProcess, + cookie: EnvCookie +) extends PipedCliProcess: + + def survivors(): List[ProcessHandle] = CookieScan.carrying(cookie) + + // Memoised so repeated calls return the same iterator, avoiding a second + // `BufferedReader` leak against the pipe. + private lazy val stdoutIterator: Iterator[String] = + linesOf(sub.wrapped.getInputStream) + private lazy val stderrIterator: Iterator[String] = + linesOf(sub.wrapped.getErrorStream) + + /** Lines of `pipe`, ending at EOF or once the process has exited and no line + * arrives within [[OsPipedSubProcess.OrphanedPipeGrace]]. + * + * Everything the process wrote is in the pipe buffer by the time it exits, + * so a read still blocked after the grace waits only on a descendant that + * inherited the pipe and outlived the process. That descendant keeps + * running, and so does the pipe's detached reader thread: blocked in the + * read until the descendant exits or writes, and parked for good if it + * writes. An unterminated last line is dropped in that case. + */ + private def linesOf(pipe: InputStream): Iterator[String] = + val reader = BufferedReader( + InputStreamReader(abandonOnInterruptReads(pipe), UTF_8) + ) + // `raceResult`, so a failed read surfaces at once. Not `reader.lines()`: its + // iterator reads again on each `hasNext` after the end. + Iterator + .continually(raceResult(reader.readLine(), endOfOutput())) + .takeWhile(_ != null) + + /** `null`, as `readLine` at EOF, once the process has exited and the grace + * has passed. + */ + private def endOfOutput(): String = + sub.wrapped.waitFor().discard + sleep(OsPipedSubProcess.OrphanedPipeGrace) + null + + /** Descendants that were alive when the root was signalled. Signalling the + * root can make it exit, at which point its children are reparented to init + * and drop out of the root's `descendants()` walk — so a later tree kill + * would silently miss exactly the processes it exists to reap. Killing a + * remembered `ProcessHandle` cannot hit an unrelated process that recycled + * the pid: the JDK signals only if the pid still has the start time the + * handle was taken with. + * + * The snapshot only ever fills when orca signals a root that is still alive + * — a turn that settles, or a cancel. A root that exited on its own (an + * agent crash, a clean exit with no terminal message) was never signalled, + * so nothing was recorded and its descendants survive. + */ + private val signalledDescendants: AtomicReference[List[ProcessHandle]] = + new AtomicReference(Nil) + + // Signals the root PID only, deliberately: a spawned child inherits the JVM's + // process group (`ProcessBuilder` does not setsid), so `kill -INT -` + // would signal orca itself. Descendants are covered by `destroyForciblyTree`, + // which walks parent→child links instead of the group. + def sendSigInt(): Unit = + // Walked outside the CAS: `updateAndGet` may retry its function, and an OS + // process walk is neither cheap nor idempotent enough to re-run. + val toRemember = liveDescendants() + signalledDescendants + .updateAndGet(prev => (prev ++ toRemember).distinct) + .discard + // Narrows (does not close) the window where the pid has been recycled by an + // unrelated process — unlike the ProcessHandle path, a shelled-out `kill` + // has no start-time check to protect it. SIGINT rather than + // `ProcessHandle.destroy`'s SIGTERM because that is the signal agent CLIs + // treat as "user interrupted me". + if sub.isAlive() then + QuietProc.call(Seq("kill", "-INT", sub.wrapped.pid.toString)).discard + + def isAlive: Boolean = sub.isAlive() + + def destroyForcibly(): Unit = + val _ = sub.wrapped.destroyForcibly() + + override def destroyForciblyTree(): Unit = + // Remembered handles are re-expanded rather than killed flat: by now the + // root is usually gone, so they are the only reachable branch of the tree, + // and each may have spawned children since the snapshot was taken. + // Reachability is best-effort throughout — a process that detached itself + // (`nohup … &`, `setsid`) appears in neither the walk nor the snapshot. A + // process group established here would not extend to it either: every agent + // CLI runs its tool calls in a session of their own, so orca's group never + // contains them (measured: a claude bash tool call's session id is its own + // pid). Reaching a detached process needs a marker it carries — e.g. a + // per-turn cookie injected into the environment and swept at teardown — or + // an OS-level container such as a cgroup scope. + val remembered = + signalledDescendants.get().flatMap(h => h +: descendantsOf(h)) + (liveDescendants() ++ remembered).foreach(_.destroyForcibly().discard) + sub.wrapped.toHandle.destroyForcibly().discard + + private def liveDescendants(): List[ProcessHandle] = + descendantsOf(sub.wrapped.toHandle) + + private def descendantsOf(handle: ProcessHandle): List[ProcessHandle] = + handle.descendants().iterator().asScala.toList + + def waitForExit(): Int = + val _ = sub.join() + sub.exitCode() + + def writeLine(line: String): Unit = + sub.stdin.writeLine(line) + sub.stdin.flush() + + def closeStdin(): Unit = sub.stdin.close() + + def stdoutLines: Iterator[String] = stdoutIterator + + def stderrLines: Iterator[String] = stderrIterator + + def tryExitCode: Option[Int] = + if sub.isAlive() then None else Some(sub.exitCode()) + +private[orca] object OsPipedSubProcess: + + private val log = LoggerFactory.getLogger("orca.proc") + + /** Starts `args` with all three stdio streams piped and a fresh [[EnvCookie]] + * added to its environment. + */ + def spawn( + args: Seq[String], + env: Map[String, String], + cwd: os.Path + ): PipedCliProcess = + log.debug("spawn: {} (cwd={})", args.mkString(" "), cwd) + // os-lib applies `env` on top of the inherited environment (replacing it + // only with `propagateEnv = false`), so this adds one variable and changes + // nothing else the CLI sees. + val cookie = EnvCookie.mint() + val sub = os + .proc(args) + .spawn( + cwd = cwd, + env = env + (EnvCookie.VarName -> cookie.value), + stdin = os.Pipe, + stdout = os.Pipe, + stderr = os.Pipe + ) + new OsPipedSubProcess(sub, cookie) + + /** How long a pipe may stay silent after the process exited before its stream + * ends. + */ + val OrphanedPipeGrace: FiniteDuration = 1.second diff --git a/tools/src/main/scala/orca/subprocess/QuietProc.scala b/core/src/main/scala/orca/subprocess/QuietProc.scala similarity index 80% rename from tools/src/main/scala/orca/subprocess/QuietProc.scala rename to core/src/main/scala/orca/subprocess/QuietProc.scala index 27282d422..a0f8a31b3 100644 --- a/tools/src/main/scala/orca/subprocess/QuietProc.scala +++ b/core/src/main/scala/orca/subprocess/QuietProc.scala @@ -4,22 +4,22 @@ package orca.subprocess * * os-lib defaults `os.proc(...).call(...)`'s `stderr` to `os.Inherit`, which * lets the child write straight to the parent's terminal — bypassing the - * renderer's [[orca.runner.terminal.StatusBar]] and producing torn frames - * (e.g. a stray "Switched to a new branch" on the spinner's row mid-redraw). - * So any tool that shells out from a flow goes through `QuietProc.call` (or a - * `CliRunner`, which delegates here via [[OsProcCliRunner]]); a direct - * `os.proc(...).call(...)` in production tool code is a leak. + * renderer's `StatusBar` and producing torn frames (e.g. a stray "Switched to + * a new branch" on the spinner's row mid-redraw). So any tool that shells out + * from a flow goes through `QuietProc.call` (or [[orca.effect.Processes]], + * which delegates here); a direct `os.proc(...).call(...)` in production tool + * code is a leak. * * Captured stderr lands in `result.err`, which the caller surfaces in error - * messages (see [[orca.tools.OsGitTool]]'s `git` helper). + * messages. */ private[orca] object QuietProc: private val log = org.slf4j.LoggerFactory.getLogger("orca.proc") /** Run `args` to completion. stdout + stderr are captured into the returned - * [[os.CommandResult]]; `check = false` means non-zero exits don't throw — - * the caller inspects `exitCode` / `err.text()` and decides how to react. + * `os.CommandResult`; `check = false` means non-zero exits don't throw — the + * caller inspects `exitCode` / `err.text()` and decides how to react. * * `cwd` and `env` default to concrete values (current directory / an empty * map merged onto the inherited environment) rather than os-lib's @@ -105,14 +105,3 @@ private[orca] object QuietProc: private def utf8(bytes: Array[Byte]): String = String(bytes, java.nio.charset.StandardCharsets.UTF_8) - -/** Outcome of [[QuietProc.callCapped]]. `out` is a prefix of what the child - * wrote whenever `truncated` — and, being a prefix of bytes rather than of - * characters, can end in a replacement character where the cut split one. - */ -private[orca] case class CappedResult( - exitCode: Int, - out: String, - truncated: Boolean, - err: String -) diff --git a/core/src/test/scala/orca/AssumeSafeScopeTest.scala b/core/src/test/scala/orca/AssumeSafeScopeTest.scala new file mode 100644 index 000000000..356649f76 --- /dev/null +++ b/core/src/test/scala/orca/AssumeSafeScopeTest.scala @@ -0,0 +1,57 @@ +package orca + +import orca.testkit.RepoRoot + +/** `@assumeSafe` exempts code from safe-mode checking, so it stays in a fixed + * set of `core` files, reviewed as the trusted base. + */ +class AssumeSafeScopeTest extends munit.FunSuite: + + // The annotation, possibly after others on its line, or its import; not a + // mention in a comment. + private val use = + raw"(?m)^\s*(@[\w.]+(\([^)\n]*\))?\s+)*@(scala\.)?(caps\.)?assumeSafe\b|\bimport\s+(scala\.)?caps\.(assumeSafe\b|\{[^}]*\bassumeSafe\b)".r + + /** Capabilities and pure facades: the only files that may use it. */ + private val allowed: Set[String] = Set( + "core/src/main/scala/orca/concurrency/concurrency.scala", + "core/src/main/scala/orca/concurrency/Ref.scala", + "core/src/main/scala/orca/effect/FileSystem.scala", + "core/src/main/scala/orca/effect/Network.scala", + "core/src/main/scala/orca/effect/Processes.scala", + "core/src/main/scala/orca/json/JsonCodec.scala", + "core/src/main/scala/orca/json/JsonData.scala", + "core/src/main/scala/orca/json/JsonSchemas.scala", + "core/src/main/scala/orca/mcp/mcp.scala", + "core/src/main/scala/orca/safe/Logger.scala", + "core/src/main/scala/orca/safe/prelude.scala" + ) + + /** Every module's main sources, plus the flow scripts and examples. */ + private def sourceRoots(root: os.Path): List[os.Path] = + os.list(root) + .map(_ / "src" / "main") + .filter(os.isDir) + .toList ++ List(root / "flows", root / "examples") + + test("the pattern matches every spelling of the annotation and its import"): + val spellings = List( + "@assumeSafe object X", + " @caps.assumeSafe final class X", + "@scala.caps.assumeSafe object X", + "@deprecated(\"x\") @caps.assumeSafe class X", + "import caps.assumeSafe", + "import scala.caps.assumeSafe", + "import caps.{SharedCapability, assumeSafe as trusted}" + ) + assertEquals(spellings.filterNot(use.findFirstIn(_).isDefined), Nil) + + test("@assumeSafe appears only in the allowed core files"): + val root = RepoRoot.dir + val users = sourceRoots(root) + .flatMap(os.walk(_)) + .filter(p => os.isFile(p) && (p.ext == "scala" || p.ext == "sc")) + .filter(p => use.findFirstIn(os.read(p)).isDefined) + .map(_.relativeTo(root).toString) + .toSet + assertEquals(users, allowed) diff --git a/core/src/test/scala/orca/SafeModeExposureTest.scala b/core/src/test/scala/orca/SafeModeExposureTest.scala new file mode 100644 index 000000000..43f10bb7c --- /dev/null +++ b/core/src/test/scala/orca/SafeModeExposureTest.scala @@ -0,0 +1,184 @@ +package orca + +import orca.testkit.Dotc + +/** What safe-mode code may reach: capabilities and the `@assumeSafe` facades, + * but neither an effect implementation nor the mint of a capability. + * + * Fixtures sit in `package orca.fixture`, so a `private[orca]` symbol fails + * for being unsafe, not for being inaccessible. + */ +class SafeModeExposureTest extends munit.FunSuite: + + private def safeFixture(body: String): String = + fixtureIn("orca.fixture", body) + + private def fixtureIn(pkg: String, body: String): String = + s"""package $pkg + |import language.experimental.safe + |$body + |""".stripMargin + + /** What each fixture reaches, the symbol its error must name, the package it + * sits in, and the code. A `private[effect]` symbol is reached from + * `orca.effect`, so it too fails for being unsafe. + */ + private val unreachable: List[(String, String, String, String)] = List( + ( + "an effect implementation", + "QuietProc", + "orca.fixture", + "def f(cwd: os.Path): Int = orca.subprocess.QuietProc.call(Seq(\"true\"), cwd).exitCode" + ), + ( + "the piped subprocess implementation", + "OsPipedSubProcess", + "orca.fixture", + "def f(cwd: os.Path): orca.subprocess.PipedCliProcess = orca.subprocess.OsPipedSubProcess.spawn(Seq(\"true\"), Map.empty, cwd)" + ), + ( + "the process scan behind a spawned process", + "CookieScan", + "orca.fixture", + "def f(c: orca.subprocess.EnvCookie): Int = orca.subprocess.CookieScan.carrying(c).size" + ), + ( + "the HTTP client implementation", + "JdkHttpClient", + "orca.effect", + "def f(): HttpClient = JdkHttpClient(\"http://127.0.0.1:1\", \"u\", \"p\")" + ), + ( + "the local HTTP server implementation", + "LocalServer", + "orca.effect", + "def f()(using ox.Ox): Int = LocalServer.serve(Nil, scala.concurrent.duration.Duration(1, \"s\"))" + ), + ( + "os-lib directly", + "read", + "orca.fixture", + "def f(p: os.Path): String = os.read(p)" + ), + ( + "FileSystem.ambient", + "ambient", + "orca.fixture", + "def f(): orca.effect.FileSystem = orca.effect.FileSystem.ambient" + ), + ( + "Processes.ambient", + "ambient", + "orca.fixture", + "def f(): orca.effect.Processes = orca.effect.Processes.ambient" + ), + ( + "Network.ambient", + "ambient", + "orca.fixture", + "def f(): orca.effect.Network = orca.effect.Network.ambient" + ), + ( + "an ambient effect passed through an inline facade", + "read", + "orca.fixture", + "def f(p: os.Path): String = orca.concurrency.supervised(os.read(p))" + ) + ) + + unreachable.foreach: (what, symbol, pkg, code) => + test(s"NEGATIVE: safe code cannot reference $what"): + val errors = Dotc.errorsOf(fixtureIn(pkg, code)) + assert( + errors.exists(e => e.contains("from safe code") && e.contains(symbol)), + s"expected a safe-mode rejection naming $symbol, got: $errors" + ) + + test("NEGATIVE: a Ref update cannot capture a capability"): + val errors = Dotc.errorsOf( + safeFixture( + """import orca.concurrency.Ref + |import orca.effect.FileSystem + |def f(r: Ref[Int], p: os.Path)(using fs: FileSystem): Unit = + | r.update(n => if fs.isFile(p) then n + 1 else n) + |""".stripMargin + ) + ) + assert( + errors.exists( + _.contains("capability `fs` cannot flow into capture set {}") + ), + s"expected the capture of fs rejected, got: $errors" + ) + + test("POSITIVE: safe code uses effects through capabilities"): + val errors = Dotc.errorsOf( + safeFixture( + """import orca.effect.{FileSystem, Network, Processes} + |import orca.concurrency.supervised + |import orca.safe.prelude.{*, given} + |def head(cwd: os.Path)(using procs: Processes): String = + | procs.call(Seq("git", "rev-parse", "HEAD"), cwd).stdout + |def log(cwd: os.Path)(using procs: Processes): String = + | procs.callCapped(Seq("git", "log"), 1024, cwd).out + |def firstLine(cwd: os.Path)(using procs: Processes): Option[String] = + | val p = procs.spawnPiped(Seq("echo", "hi"), cwd) + | try p.stdoutLines.nextOption() + | finally p.destroyForciblyTree() + |def size(p: os.Path)(using fs: FileSystem): Int = + | if fs.isFile(p) then fs.read(p).length else 0 + |def rewrite(dir: os.Path)(using fs: FileSystem): List[os.Path] = + | fs.writeOver(dir / "a.txt", "x") + | fs.files(dir, _ => true).map(fs.realPath) + |def serve()(using net: Network): Int = supervised: + | net.serveLocal(Nil, 1.second) + |def status(base: String)(using net: Network): Int = supervised: + | net.httpClient(base, "u", "p").get("/", 1.second).status + |""".stripMargin + ) + ) + assert(errors.isEmpty, s"expected no errors, got: $errors") + + test("POSITIVE: safe code uses the facades"): + val errors = Dotc.errorsOf( + safeFixture( + """import orca.json.{JsonCodec, JsonData, given} + |import orca.safe.Logger + |import orca.concurrency.* + |import orca.safe.prelude.{*, given} + | + |case class Wire(n: Int, tags: List[String]) derives JsonCodec + |enum Shape: + | case Dot + | case Line(length: Int) + |given JsonCodec[Shape] = + | JsonCodec.derived(using JsonCodec.config.withDiscriminatorFieldName(None)) + | + |case class Inner(name: String) derives JsonData + |case class Outer(inner: Inner, kind: Kind) derives JsonData + |enum Kind derives JsonData: + | case A, B + |enum Event derives JsonData: + | case Start + | case Stop(code: Int) + | + |private val log = Logger("fixture") + | + |def child(base: os.Path): os.Path = base / "a" / up / "b" + |def parsed(s: String): Either[String, Int] = + | either: + | val n = s.toIntOption.toRight("not a number").ok() + | n.pipe(_ + 1).tap(v => log.debug(s"parsed $v")) + |def guarded(f: () -> Int): Int = + | try f() + | catch case NonFatal(_) => 0 + |def concurrent(): Int = supervised: + | val a = fork(1) + | forkDiscard(sleep(1.millis)) + | a.join() + |def caught(s: String): Either[Throwable, Int] = s.toInt.catching[Throwable] + |def poll(): Nothing = forever(sleep(1.millis)) + |""".stripMargin + ) + ) + assert(errors.isEmpty, s"expected no errors, got: $errors") diff --git a/core/src/test/scala/orca/concurrency/RefTest.scala b/core/src/test/scala/orca/concurrency/RefTest.scala new file mode 100644 index 000000000..b39b71126 --- /dev/null +++ b/core/src/test/scala/orca/concurrency/RefTest.scala @@ -0,0 +1,17 @@ +package orca.concurrency + +class RefTest extends munit.FunSuite: + + test("modify reruns f on the value another writer set meanwhile"): + val ref = Ref(0) + var calls = 0 + val result = ref.modify: s => + calls += 1 + if calls == 1 then ref.set(10) + (s + 1, s) + assertEquals((result, ref.get, calls), (10, 11, 2)) + + test("update replaces the value with f of it"): + val ref = Ref(1) + ref.update(_ + 1) + assertEquals(ref.get, 2) diff --git a/core/src/test/scala/orca/effect/FileSystemTest.scala b/core/src/test/scala/orca/effect/FileSystemTest.scala new file mode 100644 index 000000000..8991b1601 --- /dev/null +++ b/core/src/test/scala/orca/effect/FileSystemTest.scala @@ -0,0 +1,48 @@ +package orca.effect + +class FileSystemTest extends munit.FunSuite: + + private val dir = FunFixture[os.Path]( + setup = _ => os.temp.dir(prefix = "fs-test"), + teardown = os.remove.all(_) + ) + + private val fs = FileSystem.ambient + + dir.test("writeOver creates missing parent directories"): d => + fs.writeOver(d / "a" / "b" / "f.txt", "x") + assertEquals(os.read(d / "a" / "b" / "f.txt"), "x") + + dir.test("files lists the accepted files, recursively"): d => + os.write(d / "top.txt", "", createFolders = true) + os.write(d / "sub" / "nested.txt", "", createFolders = true) + os.write(d / "sub" / "skipped.md", "", createFolders = true) + os.makeDir(d / "empty") + assertEquals( + fs.files(d, _.ext == "txt").map(_.relativeTo(d).toString).toSet, + Set("top.txt", "sub/nested.txt") + ) + + dir.test("findFile skips accepted directories"): d => + os.makeDir(d / "a.txt") + os.write(d / "sub" / "b.txt", "", createFolders = true) + assertEquals(fs.findFile(d, _.ext == "txt"), Some(d / "sub" / "b.txt")) + + dir.test("readPrefix stops at the end of a shorter file"): d => + os.write(d / "f.txt", "abc") + assertEquals( + fs.readPrefix(d / "f.txt", maxBytes = 512).toSeq, + "abc".getBytes.toSeq + ) + + dir.test("removeAll removes a symlink, not its target"): d => + os.write(d / "target" / "f.txt", "", createFolders = true) + os.symlink(d / "link", d / "target") + fs.removeAll(d / "link") + assert(!os.exists(d / "link", followLinks = false)) + assert(os.exists(d / "target" / "f.txt")) + + dir.test("realPath resolves a symlink"): d => + os.write(d / "target.txt", "") + os.symlink(d / "link.txt", d / "target.txt") + assertEquals(fs.realPath(d / "link.txt"), fs.realPath(d / "target.txt")) diff --git a/core/src/test/scala/orca/effect/HttpClientTest.scala b/core/src/test/scala/orca/effect/HttpClientTest.scala new file mode 100644 index 000000000..f4305cb58 --- /dev/null +++ b/core/src/test/scala/orca/effect/HttpClientTest.scala @@ -0,0 +1,132 @@ +package orca.effect + +import ox.{CancellableFork, Ox, forkCancellable, supervised, timeout} +import sttp.model.StatusCode +import sttp.shared.Identity +import sttp.tapir.* +import sttp.tapir.server.ServerEndpoint + +import java.io.{IOException, InputStream} +import java.nio.charset.StandardCharsets.UTF_8 +import java.util.concurrent.{CompletableFuture, CountDownLatch, TimeUnit} +import scala.concurrent.duration.DurationInt + +class HttpClientTest extends munit.FunSuite: + + private val whoami = + endpoint.get + .in("whoami") + .in(header[String]("Authorization")) + .out(stringBody) + .handleSuccess(identity) + + private val echo = + endpoint.post + .in("echo") + .in(header[String]("Content-Type")) + .in(stringBody) + .out(stringBody) + .handleSuccess((contentType, body) => s"$contentType $body") + + private val down = + endpoint.get + .in("down") + .errorOut(statusCode(StatusCode.ServiceUnavailable).and(stringBody)) + .handle(_ => Left("down")) + + /** A body that sends one line and the start of a second, then blocks until + * `release`. + */ + private final class HangingBody extends InputStream: + private val released = new CountDownLatch(1) + private var sent = false + def release(): Unit = released.countDown() + + // tapir's netty server reads with `readNBytes` and ends the body at a short + // read, so the first chunk is filled. + override def readNBytes(len: Int): Array[Byte] = + if !sent then + sent = true + ("ready\n" + "x" * (len - 6)).getBytes(UTF_8) + else + released.await() + Array.emptyByteArray + + def read(): Int = throw UnsupportedOperationException() + + private def withClient[T](endpoints: List[ServerEndpoint[Any, Identity]])( + body: Ox ?=> HttpClient => T + ): T = + supervised: + val net = Network.ambient + val port = net.serveLocal(endpoints, 5.seconds) + body(net.httpClient(s"http://127.0.0.1:$port", "user", "secret")) + + test("get returns the status and body, sending Basic auth"): + val reply = withClient(List(whoami))(_.get("/whoami", 5.seconds)) + assertEquals(reply, HttpReply(200, "Basic dXNlcjpzZWNyZXQ=")) + + test("postJson sends the body as JSON"): + val reply = + withClient(List(echo))(_.postJson("/echo", """{"a":1}""", 5.seconds)) + assertEquals(reply, HttpReply(200, """application/json {"a":1}""")) + + test("a non-2xx status is returned, not thrown"): + val reply = withClient(List(down))(_.get("/down", 5.seconds)) + assertEquals(reply, HttpReply(503, "down")) + assert(!reply.isSuccess) + + /** Opens a hanging stream, reads its first line in a fork, waits until that + * fork is parked in the next read, then runs `unblock` on it and returns how + * the read ended. + */ + private def unblockParkedReader( + unblock: (HttpLines, CancellableFork[Boolean]) => Either[ + Throwable, + Boolean + ] + ): Either[Throwable, Boolean] = + val hanging = new HangingBody + val stream = + endpoint.get.in("stream").out(inputStreamBody).handleSuccess(_ => hanging) + withClient(List(stream)): client => + val lines = client.openLines("/stream") + try + // Completing never parks the reader, so its next park is the read. + val readerThread = new CompletableFuture[Thread] + val reader = forkCancellable: + val it = lines.lines + assertEquals(it.next(), "ready") + readerThread.complete(Thread.currentThread()): Unit + it.hasNext + awaitParked(readerThread.get(5, TimeUnit.SECONDS)) + timeout(5.seconds)(unblock(lines, reader)) + finally + lines.close() + // Before the scope ends: the server's stop waits for the body. + hanging.release() + + /** A parked virtual thread reports `WAITING`; bounded, so a reader that never + * parks fails the test instead of hanging it. + */ + private def awaitParked(thread: Thread): Unit = + val deadline = System.nanoTime() + 5.seconds.toNanos + while thread.getState != Thread.State.WAITING do + assert( + System.nanoTime() < deadline, + s"reader never parked: ${thread.getState}" + ) + Thread.onSpinWait() + + test("openLines: close unblocks a reader parked in the body read"): + val outcome = unblockParkedReader: (lines, reader) => + lines.close() + scala.util.Try(reader.join()).toEither + assert(outcome.left.exists(_.isInstanceOf[IOException]), outcome) + + test("openLines: interrupting a reader parked in the body read unblocks it"): + val outcome = unblockParkedReader((_, reader) => reader.cancel()) + assert( + outcome.left.exists(_.getCause.isInstanceOf[InterruptedException]), + outcome + ) diff --git a/core/src/test/scala/orca/effect/NetworkTest.scala b/core/src/test/scala/orca/effect/NetworkTest.scala new file mode 100644 index 000000000..8b07b23b7 --- /dev/null +++ b/core/src/test/scala/orca/effect/NetworkTest.scala @@ -0,0 +1,48 @@ +package orca.effect + +import ox.supervised +import sttp.tapir.* + +import java.net.{ConnectException, InetSocketAddress, Socket, URI} +import java.net.http.{HttpClient, HttpRequest, HttpResponse} +import scala.concurrent.duration.DurationInt +import scala.jdk.CollectionConverters.* +import scala.util.Using + +class NetworkTest extends munit.FunSuite: + + private val ping = + endpoint.get.in("ping").out(stringBody).handleSuccess(_ => "pong") + + // Closed before the scope ends: an idle keep-alive connection makes the + // server's stop wait out its graceful-shutdown timeout. + private def get(port: Int): String = + val request = + HttpRequest.newBuilder(URI.create(s"http://127.0.0.1:$port/ping")).build() + Using.resource(HttpClient.newHttpClient()): + _.send(request, HttpResponse.BodyHandlers.ofString()).body() + + /** Thread-name prefixes of the server's event loop and tapir's executor. */ + private val nettyThreadPrefixes = + List("multiThreadIoEventLoopGroup", "defaultEventExecutor") + + test("serveLocal serves the endpoints on a loopback port"): + supervised: + val port = Network.ambient.serveLocal(List(ping), 5.seconds) + assertEquals(get(port), "pong") + + test("serveLocal stops serving when the scope ends"): + val port = supervised(Network.ambient.serveLocal(List(ping), 5.seconds)) + Using.resource(new Socket()): socket => + intercept[ConnectException]( + socket.connect(new InetSocketAddress("127.0.0.1", port), 1000) + ): Unit + + test("serveLocal leaves no Netty threads running when the scope ends"): + supervised: + get(Network.ambient.serveLocal(List(ping), 5.seconds)): Unit + val netty = Thread.getAllStackTraces.keySet.asScala.toList + .filter(t => nettyThreadPrefixes.exists(t.getName.startsWith)) + // A terminated executor's thread may still be finishing its exit. + netty.foreach(_.join(1000)) + assertEquals(netty.filter(_.isAlive).map(_.getName), Nil) diff --git a/core/src/test/scala/orca/json/JsonDataTest.scala b/core/src/test/scala/orca/json/JsonDataTest.scala new file mode 100644 index 000000000..363f4423a --- /dev/null +++ b/core/src/test/scala/orca/json/JsonDataTest.scala @@ -0,0 +1,13 @@ +package orca.json + +import com.github.plokhotnyuk.jsoniter_scala.core.{ + JsonReaderException, + readFromString +} + +case class WithList(xs: List[Int]) derives JsonData + +class JsonDataTest extends munit.FunSuite: + + test("a missing collection field fails to decode"): + intercept[JsonReaderException](readFromString[WithList]("{}")): Unit diff --git a/core/src/test/scala/orca/json/JsonSchemasTest.scala b/core/src/test/scala/orca/json/JsonSchemasTest.scala new file mode 100644 index 000000000..6a01e3292 --- /dev/null +++ b/core/src/test/scala/orca/json/JsonSchemasTest.scala @@ -0,0 +1,73 @@ +package orca.json + +import io.circe.parser.parse +import io.circe.{Json, JsonObject} + +class JsonSchemasTest extends munit.FunSuite: + + private def strictOrRefusal(input: String): Either[String, Json] = + JsonSchemas.toOpenAiStrict(parse(input).toOption.get) + + private def strict(input: String): Json = strictOrRefusal(input).toOption.get + + test("object node gains additionalProperties:false and full required list"): + // OpenAI strict mode demands every object declare additionalProperties + // false AND list every property in required (nullability is how + // optionality is expressed). Tapir's output doesn't supply either. + val out = strict( + """{"type":"object","properties":{"a":{"type":"string"},"b":{"type":"integer"}}}""" + ) + val obj = out.asObject.get + assertEquals(obj("additionalProperties"), Some(Json.False)) + val required = + obj("required").get.asArray.get.map(_.asString.get).toSet + assertEquals(required, Set("a", "b")) + + test("object without properties is left alone"): + // A marker `{"type":"object"}` (no properties) carries no constraints + // to enforce — don't graft additionalProperties onto an empty shell. + val out = strict("""{"type":"object"}""") + assertEquals(out.asObject.get("additionalProperties"), None) + assertEquals(out.asObject.get("required"), None) + + test("nested object schemas inside $defs are transformed too"): + // `$defs` is where Tapir parks the case class definitions referenced by + // `$ref` — they're the meaningful object schemas and must be strict too. + val out = strict( + """{ + | "type":"object", + | "properties":{"t":{"$ref":"#/$defs/Task"}}, + | "$defs":{ + | "Task":{"type":"object","properties":{"title":{"type":"string"}}} + | } + |}""".stripMargin + ) + val task = + out.hcursor + .downField("$defs") + .downField("Task") + .as[JsonObject] + .toOption + .get + assertEquals(task("additionalProperties"), Some(Json.False)) + assertEquals( + task("required").get.asArray.get.map(_.asString.get).toList, + List("title") + ) + + test( + "a Map-shaped additionalProperties field (Tapir's real Map shape) fails fast with an actionable message" + ): + // The realistic shape Tapir emits for a Map[String,T] field: an object + // node with additionalProperties but no `properties` key at all. + // The Map-shape check runs before the `properties` + // check, so this fails fast here rather than letting the non-strict + // schema reach codex/claude and bounce back as an opaque + // `invalid_json_schema`. + val refusal = strictOrRefusal( + """{"type":"object","additionalProperties":{"type":"integer"}}""" + ) + assert( + refusal.left.exists(_.contains("List of key/value case classes")), + s"expected actionable Map-field message, got: $refusal" + ) diff --git a/core/src/test/scala/orca/json/SchemaDerivationTest.scala b/core/src/test/scala/orca/json/SchemaDerivationTest.scala new file mode 100644 index 000000000..08da5bdb3 --- /dev/null +++ b/core/src/test/scala/orca/json/SchemaDerivationTest.scala @@ -0,0 +1,30 @@ +package orca.json + +case class Leaf(name: String, size: Option[Int]) derives JsonData +case class Box[A](value: A, all: List[A]) +object Box: + given JsonData[Box[Leaf]] = JsonData.derived + +sealed trait Node derives JsonData +case object Empty extends Node +case class Branch(left: Leaf, pair: (Leaf, Int)) extends Node + +sealed trait Shape derives JsonData +sealed trait Round extends Shape +case class Circle(radius: Int) extends Round +case object Square extends Shape + +enum Mixed derives JsonData: + case Zero + case One(n: Int) + case Another + +// Shapes the codebase's own `JsonData` types don't all exercise. +class SchemaDerivationTest extends SchemaParitySuite: + parity[Leaf] + parity[Box[Leaf]] + parity[Node] + parity[Shape] + parity[Mixed] + parity[(Leaf, Int)] + parity[(Leaf, Int, Boolean)] diff --git a/core/src/test/scala/orca/json/SchemaParityCoverageTest.scala b/core/src/test/scala/orca/json/SchemaParityCoverageTest.scala new file mode 100644 index 000000000..1cb54a4dd --- /dev/null +++ b/core/src/test/scala/orca/json/SchemaParityCoverageTest.scala @@ -0,0 +1,111 @@ +package orca.json + +import orca.testkit.RepoRoot + +import scala.util.matching.Regex + +/** Every type whose schema [[SchemaDerivation]] builds in main sources has a + * `parity[...]` entry in some `SchemaParitySuite`, so the parity tests grow + * with the codebase. + * + * Types are matched by simple name, as found in the source text. A + * parameterless enum is exempt: it takes the string-enum path, whose schema + * tapir's `Schema.derivedEnumeration` builds directly. + */ +class SchemaParityCoverageTest extends munit.FunSuite: + + /** Sites pinned elsewhere, and where. */ + private val exempt: Map[String, String] = Map( + // Its `Segment` is private. + "Stage" -> "StagePathTest pins the rendered schema" + ) + + private val declaration: Regex = + raw"\b(case class|class|enum|trait|object)\s+(\w+)".r + private val derivesClause: Regex = raw"\bderives\b[^\n]*\bJsonData\b".r + private val givenInstance: Regex = + raw"\bgiven\s+(?:\w+\s*:\s*)?(JsonData|Schema)\[(\w+(?:\.\w+)*)\]\s*=".r + private val derivedCall: Regex = raw"\bJsonData\.derived\[(\w+(?:\.\w+)*)\]".r + private val parityEntry: Regex = raw"\bparity\[(\w+(?:\.\w+)*)\]".r + private val comment: Regex = raw"(?s)/\*.*?\*/|(?m)(^|\s)//[^\n]*".r + + private def scalaFiles(dirs: List[os.Path]): List[os.Path] = + dirs + .filter(os.isDir) + .flatMap(os.walk(_)) + .filter(p => os.isFile(p) && p.ext == "scala") + + private def moduleDirs(sub: String): List[os.Path] = + os.list(RepoRoot.dir).map(_ / "src" / sub).toList + + private def simpleName(name: String): String = name.split('.').last + + /** The derivation sites in `src`, by type name. */ + private def sites(src: String): Set[String] = + val derived = derivesClause + .findAllMatchIn(src) + .flatMap(m => owner(src, m.start)) + .toSet + val givens = givenInstance + .findAllMatchIn(src) + .filter(m => derivesSchema(m.group(1), statement(src, m.end))) + .map(_.group(2)) + .toSet + (derived ++ givens ++ derivedCall.findAllMatchIn(src).map(_.group(1))) + .map(simpleName) + + /** The type a `derives` clause at `at` belongs to, unless it is exempt. */ + private def owner(src: String, at: Int): Option[String] = + declaration + .findAllMatchIn(src.substring(0, at)) + .toList + .lastOption + .filterNot(d => d.group(1) == "enum" && parameterless(src, d.start)) + .map(_.group(2)) + + /** Whether no case of the enum declared at `at` has fields. */ + private def parameterless(src: String, at: Int): Boolean = + val lineStart = src.lastIndexOf('\n', at) + 1 + val indent = at - lineStart + src + .substring(lineStart) + .linesIterator + .drop(1) + .takeWhile(l => l.isBlank || l.segmentLength(_ == ' ') > indent) + .filter(_.trim.startsWith("case ")) + .forall(!_.contains('(')) + + /** The text from `at` to the next blank line. */ + private def statement(src: String, at: Int): String = + val end = src.indexOf("\n\n", at) + src.substring(at, if end < 0 then src.length else end) + + // A `Schema` given counts only when derived here; a `JsonData` one unless + // it travels as a string. + private def derivesSchema(kind: String, rhs: String): Boolean = + if kind == "Schema" then + rhs.contains("SchemaDerivation.derived") || + rhs.contains("JsonData.derivedSchema") + else !rhs.contains("JsonData.fromString") + + // The derivation's own generic givens; SchemaDerivationTest covers them. + private def isDerivation(p: os.Path): Boolean = + p.startsWith( + RepoRoot.dir / "core" / "src" / "main" / "scala" / "orca" / "json" + ) + + private lazy val derivationSites: Set[String] = + scalaFiles(moduleDirs("main")) + .filterNot(isDerivation) + .flatMap(p => sites(comment.replaceAllIn(os.read(p), ""))) + .toSet + + test("every derivation site has a parity entry"): + val covered = scalaFiles(moduleDirs("test")) + .flatMap(p => parityEntry.findAllMatchIn(os.read(p)).map(_.group(1))) + .map(simpleName) + .toSet + assertEquals(derivationSites -- covered -- exempt.keySet, Set.empty[String]) + + test("every exemption names a derivation site"): + assertEquals(exempt.keySet -- derivationSites, Set.empty[String]) diff --git a/core/src/test/scala/orca/json/SchemaParitySuite.scala b/core/src/test/scala/orca/json/SchemaParitySuite.scala new file mode 100644 index 000000000..23680e116 --- /dev/null +++ b/core/src/test/scala/orca/json/SchemaParitySuite.scala @@ -0,0 +1,54 @@ +package orca.json + +import io.circe.Json +import io.circe.syntax.EncoderOps +import sttp.apispec.circe.encoderSchema +import sttp.tapir.Schema +import sttp.tapir.docs.apispec.schema.TapirSchemaToJsonSchema + +import scala.compiletime.{constValueTuple, erasedValue, summonFrom} +import scala.deriving.Mirror + +/** Pins schemas derived by [[SchemaDerivation]] (directly or through + * `JsonData`) to what tapir's own derivation produces, rendered as JSON + * Schema: a `parity[A]` test per type, comparing the `Schema[A]` in scope. + * + * Key order is not compared: `$defs` come out in a different order, as + * Magnolia names a parameterless enum case `.`, dropping the + * enum, and orders cases by that name. + */ +trait SchemaParitySuite extends munit.FunSuite: + + inline def parity[A](using schema: Schema[A], m: Mirror.Of[A]): Unit = + val name = schema.name.fold("?")(_.fullName) + val actual = SchemaParitySuite.render(schema) + val expected = SchemaParitySuite.render(SchemaParitySuite.tapirSchema[A]) + test(s"$name: schema matches tapir's derivation"): + assertEquals(actual, expected) + +object SchemaParitySuite: + + def render[A](schema: Schema[A]): Json = + TapirSchemaToJsonSchema(schema, markOptionsAsNullable = true).asJson + + /** The schema tapir derives as `JsonData` would ask it to: a string enum for + * a sum of parameterless cases, `Schema.derived` otherwise. + */ + inline def tapirSchema[A](using m: Mirror.Of[A]): Schema[A] = + inline m match + case s: Mirror.SumOf[A] => + inline if allSingletons[s.MirroredElemTypes] then + val labels = constValueTuple[s.MirroredElemLabels].toList + .map(_.toString) + .toVector + Schema.derivedEnumeration[A](encode = Some(a => labels(s.ordinal(a)))) + else Schema.derived[A] + case _ => Schema.derived[A] + + private inline def allSingletons[T <: Tuple]: Boolean = + inline erasedValue[T] match + case _: EmptyTuple => true + case _: (h *: t) => + summonFrom: + case _: ValueOf[`h`] => allSingletons[t] + case _ => false diff --git a/tools/src/test/scala/orca/subprocess/QuietProcTest.scala b/core/src/test/scala/orca/subprocess/QuietProcTest.scala similarity index 100% rename from tools/src/test/scala/orca/subprocess/QuietProcTest.scala rename to core/src/test/scala/orca/subprocess/QuietProcTest.scala diff --git a/core/src/test/scala/orca/testkit/Dotc.scala b/core/src/test/scala/orca/testkit/Dotc.scala new file mode 100644 index 000000000..7fcaf2a33 --- /dev/null +++ b/core/src/test/scala/orca/testkit/Dotc.scala @@ -0,0 +1,63 @@ +package orca.testkit + +import java.nio.file.Files + +import scala.collection.mutable.ListBuffer + +import dotty.tools.dotc.Main +import dotty.tools.dotc.core.Contexts.Context +import dotty.tools.dotc.interfaces.Diagnostic as IDiagnostic +import dotty.tools.dotc.reporting.{Diagnostic, Reporter} + +/** Compiles a fixture source with the real `dotc` against the calling module's + * Test classpath, for tests that pin what does and doesn't compile. + * + * munit's `compileErrors` runs only the typer, so it misses capture checking, + * separation checking and safe mode, which run after it. + */ +object Dotc: + + private class CollectingReporter extends Reporter: + val diagnostics: ListBuffer[Diagnostic] = ListBuffer.empty + def doReport(dia: Diagnostic)(using Context): Unit = + val _ = diagnostics += dia + + /** sbt 2 starts forked tests through its own worker, so `java.class.path` + * holds only the worker's jars; the build passes the real classpath as + * `orca.test.classpath`. + */ + private def classpath: String = + sys.props.getOrElse( + "orca.test.classpath", + System.getProperty("java.class.path") + ) + + /** The error messages of compiling `source` as one file. */ + def errorsOf(source: String): List[String] = + val srcDir = Files.createTempDirectory("dotc-src") + val outDir = Files.createTempDirectory("dotc-out") + try + val srcFile = srcDir.resolve("Fixture.scala") + Files.writeString(srcFile, source) + val reporter = new CollectingReporter + val args = Array( + "-classpath", + classpath, + "-d", + outDir.toString, + srcFile.toString + ) + val _ = Main.process(args, reporter) + reporter.diagnostics.toList + .filter(_.level >= IDiagnostic.ERROR) + .map(_.message) + finally + deleteRecursively(srcDir) + deleteRecursively(outDir) + + private def deleteRecursively(dir: java.nio.file.Path): Unit = + if Files.exists(dir) then + Files + .walk(dir) + .sorted(java.util.Comparator.reverseOrder()) + .forEach(Files.delete) diff --git a/tools/src/test/scala/orca/testkit/RepoRoot.scala b/core/src/test/scala/orca/testkit/RepoRoot.scala similarity index 100% rename from tools/src/test/scala/orca/testkit/RepoRoot.scala rename to core/src/test/scala/orca/testkit/RepoRoot.scala diff --git a/docs/authoring/tutorial.md b/docs/authoring/tutorial.md index 1e2fa8a4e..57ab9a807 100644 --- a/docs/authoring/tutorial.md +++ b/docs/authoring/tutorial.md @@ -10,7 +10,7 @@ Save the file as `implement.sc`. Every flow starts with the same header, which pins the Scala version, the Orca dependency and the JVM: ```scala -//> using scala 3.9.0 +//> using scala 3.10.0 //> using dep "org.virtuslab::orca:0.2.1" //> using jvm 21 diff --git a/docs/development.md b/docs/development.md index 897a7ca85..824bea560 100644 --- a/docs/development.md +++ b/docs/development.md @@ -12,32 +12,38 @@ architecture decision records. ## Layout Orca is written in Scala 3. It uses [Ox](https://ox.softwaremill.com/) for -structured concurrency, [tapir](https://tapir.softwaremill.com/) for JSON -Schema derivation and +structured concurrency, [tapir](https://tapir.softwaremill.com/)'s schema +model to describe and render JSON Schema (derived by orca's own +`SchemaDerivation`, in `core`) and [jsoniter-scala](https://github.com/plokhotnyuk/jsoniter-scala) for codecs. The repository is organised into these modules: ``` orca/ -├── tools/ # tool traits + os-lib-backed impls (git/gh/fs), LLM SPI, session durability, events +├── core/ # effect capabilities and their implementations, facades for safe-mode code, JsonData +├── tools/ # tool traits + os-lib-backed impls (git/gh/fs), LLM SPI, session durability, InStage, events, subprocess runner ├── flow/ # stage/display/fail, FlowContext/FlowControl; orca.{plan,review,pr,progress} ├── claude/ codex/ gemini/ opencode/ pi/ # one module per coding-agent backend -├── runner/ # flow() entry, default wiring, the run lifecycle, terminal UI -├── shell/ # orca-shell: the `orca` CLI +├── lifecycle/ # a run's setup, recovery and teardown; --worktree resolution +├── runner/ # flow() entry, DefaultFlowContext/DefaultFlowControl, wiring, terminal UI +├── shell/ # the `orca` CLI's interactive front-end (ADR 0021) ├── flows/ # the built-in flow scripts, bundled into the shell └── docs/ # this site ``` -The modules depend on each other as follows: +The modules depend on each other as follows. Each is published as +`orca-`, except `runner`, published as `orca`. ``` -tools (standalone) +core (standalone) +tools → core ├── flow → tools + │ └── lifecycle → tools + flow ├── claude / codex / gemini / │ opencode / pi → tools - ├── runner → tools + flow + all five backends (published as `orca`) - └── shell → runner (published as `orca-shell`) + ├── runner → tools + flow + lifecycle + all five backends + └── shell → runner + lifecycle ``` The user-facing surface is `package orca`: `flow`, the tool and role-agent diff --git a/flow/src/main/scala/orca/AgentSet.scala b/flow/src/main/scala/orca/AgentSet.scala index 79341218e..eb7d58d36 100644 --- a/flow/src/main/scala/orca/AgentSet.scala +++ b/flow/src/main/scala/orca/AgentSet.scala @@ -43,3 +43,10 @@ trait AgentSet: val agent = agentFor(spec.backend) val pinned = spec.model.fold(agent)(m => agent.withModel(Model(m))) spec.effort.fold(pinned)(pinned.withCheckedEffort) + + /** True when `a` runs on one of the five backends — one of this set's agents + * itself or a builder-derived sibling (`_.claude.opus`, `.withReadOnly`, …). + * Used only for the foreign-lead warning. + */ + private[orca] def isWiredBackend(a: Agent[?]): Boolean = + BackendTag.values.exists(t => agentFor(t).sharesBackendWith(a)) diff --git a/flow/src/main/scala/orca/BranchNamingStrategy.scala b/flow/src/main/scala/orca/BranchNamingStrategy.scala index e3ebb4d69..6092085e7 100644 --- a/flow/src/main/scala/orca/BranchNamingStrategy.scala +++ b/flow/src/main/scala/orca/BranchNamingStrategy.scala @@ -3,8 +3,7 @@ package orca import orca.agents.Agent import orca.tools.IssueHandle -import java.nio.charset.StandardCharsets -import java.security.MessageDigest +import orca.safe.prelude.{UTF_8, sha256} /** Strategy that produces a git-ref-safe feature-branch name. Resolved once per * flow run, inside a stage (the `InStage` token gates the call). @@ -72,7 +71,7 @@ object BranchNamingStrategy: * ignores `userPrompt` and `agent`; `text` is evaluated once per `resolve` * call (by-name for callers that want late binding). */ - def fromText(text: => String): BranchNamingStrategy = + def fromText(text: => String): BranchNamingStrategy^{text} = new BranchNamingStrategy: def resolve(userPrompt: String, agent: Agent[?])(using InStage): String = slug(text) @@ -123,8 +122,6 @@ object BranchNamingStrategy: * blank) so the name is stable across calls with the same input. */ private def fallback(text: String): String = - val bytes = MessageDigest - .getInstance("SHA-256") - .digest(text.getBytes(StandardCharsets.UTF_8)) + val bytes = sha256(text.getBytes(UTF_8)) val hex = bytes.map("%02x".format(_)).mkString.take(8) s"flow-$hex" diff --git a/flow/src/main/scala/orca/FlowContext.scala b/flow/src/main/scala/orca/FlowContext.scala index f7086a0a1..702fac80c 100644 --- a/flow/src/main/scala/orca/FlowContext.scala +++ b/flow/src/main/scala/orca/FlowContext.scala @@ -5,7 +5,7 @@ import orca.tools.FsTool import orca.tools.{GitTool, RuntimeGit} import orca.tools.GitHubTool import orca.agents.Agent -import orca.review.ReviewerCatalog +import orca.review.{GateShell, ReviewerCatalog} import scala.annotation.implicitNotFound @@ -71,5 +71,8 @@ trait FlowContext extends AgentSet: def userPrompt: String + /** Where the format and lint gates run their commands. */ + private[orca] def gateShell: GateShell + /** The library's event sink. Flows use `display`, `fail` and `.announce`. */ private[orca] def emit(event: OrcaEvent): Unit diff --git a/flow/src/main/scala/orca/FlowControl.scala b/flow/src/main/scala/orca/FlowControl.scala index a83b8347c..bb91a5f4a 100644 --- a/flow/src/main/scala/orca/FlowControl.scala +++ b/flow/src/main/scala/orca/FlowControl.scala @@ -23,7 +23,7 @@ import scala.annotation.implicitNotFound * `caps.ExclusiveCapability` encodes that affinity: separation checking * forbids two concurrent closures from both capturing this exclusive * capability, so a `fork` cannot smuggle the authority to start a stage onto - * another thread. (That marker is `@experimental` on 3.9.0, hence this file's + * another thread. (That marker is `@experimental` on 3.10.0, hence this file's * `captureChecking` import; the taint stays local to this compilation unit — * see ADR 0018 §6.) At runtime, [[StageFrames]]'s owner-thread assert enforces * it for [[withStage]] and [[claimSessionKey]]. diff --git a/flow/src/main/scala/orca/ReportedFailure.scala b/flow/src/main/scala/orca/ReportedFailure.scala index 07c4d6076..bf3de7714 100644 --- a/flow/src/main/scala/orca/ReportedFailure.scala +++ b/flow/src/main/scala/orca/ReportedFailure.scala @@ -22,6 +22,6 @@ private[orca] object ReportedFailure: def reportOnce(e: Throwable)(report: Throwable => Unit): ReportedFailure = e match case reported: ReportedFailure => reported - case _ => + case _ => report(e) ReportedFailure(e) diff --git a/flow/src/main/scala/orca/Session.scala b/flow/src/main/scala/orca/Session.scala index f4a80b318..056a4b8d9 100644 --- a/flow/src/main/scala/orca/Session.scala +++ b/flow/src/main/scala/orca/Session.scala @@ -246,7 +246,7 @@ private def rehydrate[B <: BackendTag]( )(using ctx: FlowContext): Unit = WireSessionId.parse[B](wire) match case Some(wireId) => agent.rehydrateResumeWireId(id, wireId) - case None => + case None => ctx.emit( OrcaEvent.Step( s"warning: session ${key.describe} has an invalid recorded wire " + diff --git a/flow/src/main/scala/orca/StageFrames.scala b/flow/src/main/scala/orca/StageFrames.scala index 5ea4173a1..ed12161dd 100644 --- a/flow/src/main/scala/orca/StageFrames.scala +++ b/flow/src/main/scala/orca/StageFrames.scala @@ -2,6 +2,7 @@ package orca import orca.agents.SessionKey import orca.gitref.CommitHash +import orca.concurrency.Ref /** Where a turn sits in this attempt's use of one durable conversation. Minted * by [[StageFrames.claimTurn]]. @@ -49,15 +50,19 @@ private[orca] enum SessionTurn: * and misattribute a stale or wrong-typed record. * * Thread-affine: reached only through [[FlowControl]], single-threaded per - * top-level `flow(...)` (R12, ADR 0018 §2.2), so plain vars state the real - * invariant. `ownerThread` (captured at construction) is asserted on every - * door that touches them, so a stray call from an `ox.fork` — always a fresh - * thread on the pinned ox 1.0.8 — throws instead of silently corrupting the - * frame stack / counters. Ox runs a `supervised:` block's own body on a fresh - * fork too, so `stage(...)` from the direct body of a user-opened nested scope - * is rejected just like an explicit `fork`. Production is unaffected: - * `runFlow` constructs the context inside the same `supervised:` body that - * runs the flow, so owner and body thread coincide. + * top-level `flow(...)` (R12, ADR 0018 §2.2). The state lives in [[Ref]]s only + * because safe mode rejects `var` fields; the owner-thread assert, not the + * `Ref`, is what keeps it consistent. An actor would move every stage call + * onto another thread, and `caps.Stateful` would make `FlowControl^` reach + * dozens of signatures for no check the assert doesn't already make. + * `ownerThread` (captured at construction) is asserted on every door that + * touches the state, so a stray call from an `ox.fork` — always a fresh thread + * on the pinned ox 1.0.8 — throws instead of silently corrupting the frame + * stack / counters. Ox runs a `supervised:` block's own body on a fresh fork + * too, so `stage(...)` from the direct body of a user-opened nested scope is + * rejected just like an explicit `fork`. Production is unaffected: `runFlow` + * constructs the context inside the same `supervised:` body that runs the + * flow, so owner and body thread coincide. * * '''This is the only enforcement of R12 for user flow scripts.''' The * capture/separation checking enforcement (ADR 0018 §6) catches a @@ -69,11 +74,12 @@ private[orca] enum SessionTurn: * assert can. */ private[orca] trait StageFrames: - private val ownerThread: Thread = Thread.currentThread() + private val ownerThread: concurrency.ThreadIdentity = + concurrency.currentThread() /** Throws when called off `ownerThread` (R12 — see the trait scaladoc). */ private def assertOwnerThread(what: String): Unit = - if Thread.currentThread() ne ownerThread then + if concurrency.currentThread() != ownerThread then throw new OrcaFlowException( s"$what called from a fork — forks get FlowContext only (ADR 0018 R12)" ) @@ -82,19 +88,19 @@ private[orca] trait StageFrames: * commit the stage started from, and the per-name occurrence counters for * stages nested directly beneath it. */ - private final class Frame( - val path: StagePath, - val baseCommit: Option[CommitHash] + private final case class Frame( + path: StagePath, + baseCommit: Option[CommitHash], + counts: Map[String, Int] = Map.empty ): - private var counts: Map[String, Int] = Map.empty - def next(name: String): Int = + /** This frame with `name`'s counter bumped, and its value before. */ + def next(name: String): (Frame, Int) = val n = counts.getOrElse(name, 0) - counts = counts.updated(name, n + 1) - n + (copy(counts = counts.updated(name, n + 1)), n) // The flow body's frame; it is never popped. - private var frames: List[Frame] = - List(new Frame(path = StagePath.FlowBody, baseCommit = None)) + private val frames: Ref[List[Frame]] = + Ref(List(Frame(path = StagePath.FlowBody, baseCommit = None))) /** Run `f` with a stage named `name` open, passing its path: bump the current * frame's occurrence counter for `name` and push a child frame recording @@ -107,21 +113,24 @@ private[orca] trait StageFrames: baseCommit: StagePath.Stage => Option[CommitHash] )(f: StagePath.Stage => R): R = assertOwnerThread("stage(...)") - val enclosing = frames - val path = enclosing.head.path.child(name, enclosing.head.next(name)) - frames = new Frame(path = path, baseCommit = baseCommit(path)) :: enclosing + val (bumped, n) = frames.get.head.next(name) + val enclosing = bumped :: frames.get.tail + // Stored before `baseCommit` runs, so the bump survives it throwing. + frames.set(enclosing) + val path = bumped.path.child(name, n) + frames.set(Frame(path = path, baseCommit = baseCommit(path)) :: enclosing) try f(path) - finally frames = enclosing + finally frames.set(enclosing) - def stageBaseCommit: Option[CommitHash] = frames.head.baseCommit + def stageBaseCommit: Option[CommitHash] = frames.get.head.baseCommit /** Throws unless no stage is open: `what` runs stages of its own and belongs * at the flow body's top level, whatever helper it is called through. */ private[orca] def assertAtFlowBody(what: String): Unit = assertOwnerThread(what) - frames.head.path match - case StagePath.FlowBody => () + frames.get.head.path match + case StagePath.FlowBody => () case stage: StagePath.Stage => throw new OrcaFlowException( s"$what called inside stage '${stage.display}' — call it at the flow " + @@ -130,7 +139,7 @@ private[orca] trait StageFrames: // The stage half of a key already scopes it, so one flat set covers the // attempt. - private var claimedSessionKeys: Set[SessionKey] = Set.empty + private val claimedSessionKeys: Ref[Set[SessionKey]] = Ref(Set.empty) /** Key a session named `name` to the stage currently open — the flow body * when there is none — and record the mint, rejecting a second mint of the @@ -154,9 +163,9 @@ private[orca] trait StageFrames: */ private[orca] def claimSessionKey(name: String): SessionKey = assertOwnerThread("agent.session(...)") - val frame = frames.head + val frame = frames.get.head val key = SessionKey(name = name, stage = frame.path) - if claimedSessionKeys.contains(key) then + if claimedSessionKeys.get.contains(key) then val where = frame.path match case StagePath.FlowBody => "the flow body" case stage: StagePath.Stage => s"stage '${stage.display}'" @@ -165,12 +174,12 @@ private[orca] trait StageFrames: "would drive one conversation. Give each its own `stage(...)`, or " + "rename one of them." ) - claimedSessionKeys = claimedSessionKeys + key + claimedSessionKeys.update(_ + key) key // A session id is unique across the run, so one flat set covers it as the // key set above covers keys. - private var drivenSessions: Set[String] = Set.empty + private val drivenSessions: Ref[Set[String]] = Ref(Set.empty) /** Claim `sessionId`'s next turn: [[SessionTurn.First]] exactly once per * conversation per attempt, [[SessionTurn.Later]] after that. @@ -181,7 +190,8 @@ private[orca] trait StageFrames: */ private[orca] def claimTurn(sessionId: String): SessionTurn = assertOwnerThread("session.run(...)") - if drivenSessions.contains(sessionId) then SessionTurn.Later - else - drivenSessions = drivenSessions + sessionId - SessionTurn.First + drivenSessions.modify: driven => + val turn = + if driven.contains(sessionId) then SessionTurn.Later + else SessionTurn.First + (driven + sessionId, turn) diff --git a/flow/src/main/scala/orca/events/CostTracker.scala b/flow/src/main/scala/orca/events/CostTracker.scala index 52ac35e6d..261211e7a 100644 --- a/flow/src/main/scala/orca/events/CostTracker.scala +++ b/flow/src/main/scala/orca/events/CostTracker.scala @@ -2,11 +2,11 @@ package orca.events import orca.agents.Model -import java.util.concurrent.atomic.AtomicReference +import orca.concurrency.Ref /** Listener that accumulates `TokensUsed` events along three independent axes — - * by `agent`, by `model` and by `role`. State is held in an `AtomicReference` - * so the tracker is safe to register across concurrent LLM calls. + * by `agent`, by `model` and by `role`. Safe to register across concurrent LLM + * calls. * * Cost comes off the event, resolved once for the whole attempt by * [[CostResolvingDispatcher]] — the tracker prices nothing, so it cannot @@ -59,53 +59,53 @@ class CostTracker extends OrcaListener: ): Map[K, Tally] = buckets.updatedWith(key)(prev => Some(prev.fold(tally)(_ + tally))) - private val state: AtomicReference[State] = AtomicReference(State()) + // Updated from concurrent turns by a pure fold. An actor would make every + // emitter wait on one thread and need a scope this listener doesn't have. + private val state: Ref[State] = Ref(State()) def onEvent(event: OrcaEvent): Unit = event match // `turn` is ignored: a retry's tokens count toward the attempt's spend like // any other turn's. case OrcaEvent.TokensUsed(t, cost) => - val _ = state.updateAndGet( - _.record(t.agent, t.model, t.usage, cost, t.role) - ) + state.update(_.record(t.agent, t.model, t.usage, cost, t.role)) case _ => () /** Usage accumulated across every call, regardless of axis. */ def total: Usage = - state.get().byAgent.values.foldLeft(Usage.empty)(_ + _.usage) + state.get.byAgent.values.foldLeft(Usage.empty)(_ + _.usage) /** Total cost across every call. `None` when no call surfaced a cost * (reported or estimable). */ - def totalCost: Option[Cost] = totalCostOf(state.get()) + def totalCost: Option[Cost] = totalCostOf(state.get) private def totalCostOf(s: State): Option[Cost] = s.byAgent.values.flatMap(_.cost).reduceOption(_ + _) /** Per-agent usage breakdown — keyed by `Agent.name`. */ def perAgent: Map[String, Usage] = - state.get().byAgent.view.mapValues(_.usage).toMap + state.get.byAgent.view.mapValues(_.usage).toMap /** Per-agent cost breakdown. Missing entry means that agent's calls had * neither reported nor estimable cost. */ - def perAgentCost: Map[String, Cost] = costsOf(state.get().byAgent) + def perAgentCost: Map[String, Cost] = costsOf(state.get.byAgent) /** Per-model usage breakdown. `None` collects calls whose model the backend * didn't report and the caller didn't pin in `AgentConfig`. */ def perModel: Map[Option[Model], Usage] = - state.get().byModel.view.mapValues(_.usage).toMap + state.get.byModel.view.mapValues(_.usage).toMap /** Per-model cost breakdown. Same key semantics as [[perModel]]. */ - def perModelCost: Map[Option[Model], Cost] = costsOf(state.get().byModel) + def perModelCost: Map[Option[Model], Cost] = costsOf(state.get.byModel) /** Per-role usage breakdown ([[orca.agents.Agent.role]], e.g. * `Some("reviewer")`). `None` collects calls from every agent with no role * tag — the common case. */ def perRole: Map[Option[String], Usage] = - state.get().byRole.view.mapValues(_.usage).toMap + state.get.byRole.view.mapValues(_.usage).toMap private def costsOf[K](buckets: Map[K, Tally]): Map[K, Cost] = buckets.collect { case (key, Tally(_, Some(cost))) => key -> cost } @@ -134,7 +134,7 @@ class CostTracker extends OrcaListener: * Empty string when no `TokensUsed` events have been observed. */ def summary: String = - val s = state.get() + val s = state.get if s.byAgent.isEmpty then "" else val roleLines = s.byRole.toList diff --git a/flow/src/main/scala/orca/events/DeniedToolTracker.scala b/flow/src/main/scala/orca/events/DeniedToolTracker.scala index 66907c489..13aecf590 100644 --- a/flow/src/main/scala/orca/events/DeniedToolTracker.scala +++ b/flow/src/main/scala/orca/events/DeniedToolTracker.scala @@ -1,22 +1,22 @@ package orca.events -import java.util.concurrent.atomic.AtomicReference +import orca.concurrency.Ref /** Listener that counts `ToolDenied` events per tool, with the names of the * agents that were denied it, for an end-of-run summary. */ class DeniedToolTracker extends OrcaListener: - private val tools: AtomicReference[DeniedTools] = - AtomicReference(DeniedTools.empty) + // Same shape as `CostTracker.state`: concurrent emitters, a pure fold. + private val tools: Ref[DeniedTools] = Ref(DeniedTools.empty) def onEvent(event: OrcaEvent): Unit = event match case d: OrcaEvent.ToolDenied => - val _ = tools.updateAndGet(_.add(d)) + tools.update(_.add(d)) case _ => () /** [[DeniedTools.summary]] of the events seen so far. */ - def summary: String = tools.get().summary + def summary: String = tools.get.summary /** Denied tool calls counted per tool, with the names of the agents that were * denied it. diff --git a/flow/src/main/scala/orca/events/EventDispatcher.scala b/flow/src/main/scala/orca/events/EventDispatcher.scala index cf17c35a6..63999d12e 100644 --- a/flow/src/main/scala/orca/events/EventDispatcher.scala +++ b/flow/src/main/scala/orca/events/EventDispatcher.scala @@ -1,8 +1,9 @@ package orca.events -import org.slf4j.LoggerFactory +import orca.safe.Logger +import orca.concurrency.Ref -import scala.util.control.NonFatal +import orca.safe.prelude.NonFatal /** Synchronously forwards every `OrcaEvent` to a fixed list of listeners in * registration order. Implements `OrcaListener` itself, so it can be passed @@ -18,22 +19,22 @@ import scala.util.control.NonFatal * thread-safe (see [[OrcaListener]]). */ class EventDispatcher(listeners: List[OrcaListener]) extends OrcaListener: - private val log = LoggerFactory.getLogger(classOf[EventDispatcher]) + private val log = Logger(classOf[EventDispatcher].getName) - // Listeners that threw, disabled for the rest of the run. Cross-thread - // (emitters include reviewer forks), hence a concurrent set. - private val quarantined = - java.util.concurrent.ConcurrentHashMap.newKeySet[OrcaListener]() + // Listeners that threw, disabled for the rest of the run. Emitters include + // reviewer forks, so it is shared across threads; an actor would serialise + // every event through one thread to guard a set that rarely changes. + private val quarantined: Ref[Set[OrcaListener]] = Ref(Set.empty) def onEvent(event: OrcaEvent): Unit = listeners.foreach: l => - if !quarantined.contains(l) then + if !quarantined.get.contains(l) then try l.onEvent(event) catch case NonFatal(e) => - // `add` returns false if another thread already quarantined `l`; - // gating on it makes the announcement fire exactly once. - if quarantined.add(l) then + // False when another thread already quarantined `l`; gating on it + // makes the announcement fire exactly once. + if quarantined.modify(q => (q + l, !q.contains(l))) then log.error( s"listener ${l.getClass.getName} failed on ${event.getClass.getSimpleName}" + " — listener disabled for the rest of the run", @@ -49,7 +50,7 @@ class EventDispatcher(listeners: List[OrcaListener]) extends OrcaListener: val payload = event match case OrcaEvent.Error(message, _) => s" (event error: $message)" case _ => "" - System.err.println( + Logger.stderr( s"[orca] listener ${l.getClass.getName} failed on " + s"${event.getClass.getSimpleName}: ${e.getMessage}$payload (listener disabled)" ) diff --git a/flow/src/main/scala/orca/plan/Plan.scala b/flow/src/main/scala/orca/plan/Plan.scala index 3b37df469..ba1b8b54f 100644 --- a/flow/src/main/scala/orca/plan/Plan.scala +++ b/flow/src/main/scala/orca/plan/Plan.scala @@ -213,7 +213,7 @@ object Plan: def reviewed( critiqueInstructions: String = PlanPrompts.Critique, reviseInstructions: String = PlanPrompts.Revise, - variant: Agent[?] => Agent[?] = identity + variant: Agent[?] => Agent[?] = a => a )(using FlowContext, InStage): WithChat[Plan] = critiqueThenRevise( planned, diff --git a/flow/src/main/scala/orca/plan/Roadmap.scala b/flow/src/main/scala/orca/plan/Roadmap.scala index 4d43f9fc5..8bbb08ed3 100644 --- a/flow/src/main/scala/orca/plan/Roadmap.scala +++ b/flow/src/main/scala/orca/plan/Roadmap.scala @@ -48,7 +48,7 @@ object Roadmap: def reviewed( critiqueInstructions: String = PlanPrompts.RoadmapCritique, reviseInstructions: String = PlanPrompts.RoadmapRevise, - variant: Agent[?] => Agent[?] = identity + variant: Agent[?] => Agent[?] = a => a )(using FlowContext, InStage): WithChat[Roadmap] = Plan.critiqueThenRevise( planned, diff --git a/flow/src/main/scala/orca/plan/Title.scala b/flow/src/main/scala/orca/plan/Title.scala index 227ece554..bc434d11d 100644 --- a/flow/src/main/scala/orca/plan/Title.scala +++ b/flow/src/main/scala/orca/plan/Title.scala @@ -23,6 +23,6 @@ object Title: def decodeValue(in: JsonReader, default: Title): Title = in.readString(default) def encodeValue(value: Title, out: JsonWriter): Unit = out.writeVal(value) - def nullValue: Title = null.asInstanceOf[Title] + def nullValue: Title = null - given Schema[Title] = Schema.string[Title] + given Schema[Title] = orca.json.JsonData.stringSchema[Title] diff --git a/flow/src/main/scala/orca/plan/Triage.scala b/flow/src/main/scala/orca/plan/Triage.scala index 12f701936..17cab9ea0 100644 --- a/flow/src/main/scala/orca/plan/Triage.scala +++ b/flow/src/main/scala/orca/plan/Triage.scala @@ -25,7 +25,7 @@ object Triage: case Change given Announce[Triage] = Announce.from: - case Reject(_) => "Triage: rejected" + case Reject(_) => "Triage: rejected" case Accept(summary, _, Kind.TestableBug(path)) => s"Triage: bug — $summary; failing test at $path" case Accept(summary, _, Kind.UntestableBug(_)) => diff --git a/flow/src/main/scala/orca/plan/reproduceBug.scala b/flow/src/main/scala/orca/plan/reproduceBug.scala index 90bb240f2..27fff5481 100644 --- a/flow/src/main/scala/orca/plan/reproduceBug.scala +++ b/flow/src/main/scala/orca/plan/reproduceBug.scala @@ -45,7 +45,7 @@ private def checkReproduction( agent: Agent[?] )(using ctx: FlowContext, ev: InStage): BugReportMatch = val testHint = ctx.stackSettings.test match - case Nil => "" + case Nil => "" case commands => s"\nThe project's test command: ${commands.mkString(" && ")}" agent diff --git a/flow/src/main/scala/orca/pr/PrStages.scala b/flow/src/main/scala/orca/pr/PrStages.scala index e06e7b6ea..3e44debaa 100644 --- a/flow/src/main/scala/orca/pr/PrStages.scala +++ b/flow/src/main/scala/orca/pr/PrStages.scala @@ -46,7 +46,7 @@ private[pr] enum CreateResult derives JsonData: */ private[pr] def refusalLine(reason: String, from: Staged[?]): String = from match - case Staged.Fresh(_) => reason + case Staged.Fresh(_) => reason case Staged.Replayed(_) => s"$reason (recorded from the earlier attempt; orca will not retry)" @@ -61,7 +61,7 @@ private[pr] def summarise( )(using FlowContext, FlowControl): Either[NoDefaultBase, PrSummary] = val (summaryContext, summaryInstructions) = context match case Some(c) => (c, instructions) - case None => + case None => ( s"User prompt: $userPrompt", s"$instructions\n\n${PrPrompts.ClosingRefs}" diff --git a/flow/src/main/scala/orca/pr/openPrFromBranch.scala b/flow/src/main/scala/orca/pr/openPrFromBranch.scala index fbe3fa838..93be43010 100644 --- a/flow/src/main/scala/orca/pr/openPrFromBranch.scala +++ b/flow/src/main/scala/orca/pr/openPrFromBranch.scala @@ -5,7 +5,7 @@ import orca.agents.Agent import orca.review.OpenFindings import orca.tools.{PrHandle, PrReadiness} -import ox.either.orThrow +import orca.safe.prelude.orThrow /** Push the current feature branch and open a PR for it, as three separate * [[stage]]s: push → summarise → create. Split because a stage commits only on diff --git a/flow/src/main/scala/orca/pr/openPrIfGitHub.scala b/flow/src/main/scala/orca/pr/openPrIfGitHub.scala index cc9b82d72..4a0d954c6 100644 --- a/flow/src/main/scala/orca/pr/openPrIfGitHub.scala +++ b/flow/src/main/scala/orca/pr/openPrIfGitHub.scala @@ -18,7 +18,7 @@ import orca.review.OpenFindings import orca.tools.{GitHubAvailability, NoDefaultBase, PrHandle, PrReadiness} import orca.util.TextUtil -import scala.util.control.NonFatal +import orca.safe.prelude.NonFatal /** [[openPrFromBranch]] where a PR can be opened, one reported line where it * can't — for a flow that should finish its work either way. Returns the @@ -126,7 +126,7 @@ private def probe(using ): Either[String, GitHubAvailability.Available] = gh.availability() match case available: GitHubAvailability.Available => Right(available) - case GitHubAvailability.Unavailable(why) => + case GitHubAvailability.Unavailable(why) => Left( s"${why.explanation}, no PR opened — push the branch and open the " + "PR yourself" diff --git a/flow/src/main/scala/orca/progress/ProgressScan.scala b/flow/src/main/scala/orca/progress/ProgressScan.scala index 01f42c85e..705591d43 100644 --- a/flow/src/main/scala/orca/progress/ProgressScan.scala +++ b/flow/src/main/scala/orca/progress/ProgressScan.scala @@ -1,6 +1,7 @@ package orca.progress import orca.OrcaDir +import orca.effect.FileSystem import orca.json.JsonFile /** A progress log found by [[ProgressScan.progressLogs]]: its path plus the @@ -19,7 +20,7 @@ case class ScannedProgressLog(path: os.Path, header: ProgressHeader) * committed symlink could otherwise redirect the read outside the working * tree. */ -object ProgressScan: +private[orca] final class ProgressScan(fs: FileSystem): /** Every `.progress.json` file under `/.orca/runs/`, in * `os.list` order — callers impose their own ordering (e.g. newest by @@ -28,12 +29,12 @@ object ProgressScan: def progressLogPaths(workDir: os.Path): List[os.Path] = val root = OrcaDir.rootPath(workDir) val runs = OrcaDir.runsPath(workDir) - if os.isLink(root) || !os.isDir(root) || os.isLink(runs) || !os.isDir(runs) + if fs.isLink(root) || !fs.isDir(root) || fs.isLink(runs) || !fs.isDir(runs) then Nil else - os.list(runs) + fs.list(runs) .iterator - .filter(p => !os.isLink(p) && os.isFile(p) && OrcaDir.isProgressLog(p)) + .filter(p => !fs.isLink(p) && fs.isFile(p) && OrcaDir.isProgressLog(p)) .toList /** [[progressLogPaths]] with each log's header parsed. A log that does not @@ -44,7 +45,7 @@ object ProgressScan: */ def progressLogs(workDir: os.Path): List[ScannedProgressLog] = progressLogPaths(workDir).flatMap: path => - JsonFile.read[ProgressLog](path) match + JsonFile(fs).read[ProgressLog](path) match case JsonFile.Read.Loaded(log) => Some(ScannedProgressLog(path, log.header)) case _ => None diff --git a/flow/src/main/scala/orca/progress/ProgressStore.scala b/flow/src/main/scala/orca/progress/ProgressStore.scala index 8a7348f28..135aab5b5 100644 --- a/flow/src/main/scala/orca/progress/ProgressStore.scala +++ b/flow/src/main/scala/orca/progress/ProgressStore.scala @@ -1,6 +1,7 @@ package orca.progress import orca.{OrcaDir, RunKey, WorkspaceWrite} +import orca.effect.FileSystem import orca.json.JsonFile /** Persistent store for a single flow run's [[ProgressLog]]. One @@ -77,12 +78,19 @@ object ProgressStore: * same repo produce different files; rerunning the same prompt resumes the * same log. */ - def default(workDir: os.Path, key: RunKey): ProgressStore = - OsProgressStore(workDir, key) - -private class OsProgressStore(workDir: os.Path, key: RunKey) + def default( + fs: FileSystem, + workDir: os.Path, + key: RunKey + ): ProgressStore^{fs} = + OsProgressStore(fs, workDir, key) + +private class OsProgressStore(fs: FileSystem, workDir: os.Path, key: RunKey) extends ProgressStore: + private val orcaDir = OrcaDir(fs) + private val jsonFile = JsonFile(fs) + val path: os.Path = OrcaDir.progressPath(workDir, key) def load(): Option[ProgressLog] = @@ -90,13 +98,13 @@ private class OsProgressStore(workDir: os.Path, key: RunKey) case JsonFile.Read.Loaded(log) => Some(log) case _ => None - def loadDetailed(): JsonFile.Read[ProgressLog] = JsonFile.read(path) + def loadDetailed(): JsonFile.Read[ProgressLog] = jsonFile.read(path) private[orca] def peek(): Either[String, PeekedLog] = - JsonFile + jsonFile .readBytes(path) .map: - case None => PeekedLog.Absent + case None => PeekedLog.Absent case Some(bytes) => JsonFile.decode[ProgressLog](bytes) match case Right(_) => PeekedLog.Parseable(bytes) @@ -107,21 +115,21 @@ private class OsProgressStore(workDir: os.Path, key: RunKey) ): Unit = ws.check("progressStore.restoreIfRemoved") peeked match - case PeekedLog.Absent => () + case PeekedLog.Absent => () case PeekedLog.Parseable(bytes) => restoreBytesIfRemoved(bytes) case PeekedLog.Unparseable(bytes) => restoreBytesIfRemoved(bytes) private def restoreBytesIfRemoved(bytes: IArray[Byte]): Unit = - if !os.exists(path) then - OrcaDir + if !fs.exists(path) then + orcaDir .progressFile(workDir, key) - .replace(IArray.genericWrapArray(bytes).toArray) + .replace(bytes) private[orca] def remove()(using ws: WorkspaceWrite): Unit = ws.check("progressStore.remove") - val _ = os.remove(path) + fs.remove(path) def writeHeader(header: ProgressHeader)(using ws: WorkspaceWrite): Unit = ws.check("progressStore.writeHeader") @@ -153,7 +161,7 @@ private class OsProgressStore(workDir: os.Path, key: RunKey) private def currentLogOrThrow(callerName: String): ProgressLog = loadDetailed() match case JsonFile.Read.Loaded(log) => log - case JsonFile.Read.Absent => + case JsonFile.Read.Absent => throw IllegalStateException( s"$callerName called before writeHeader: no log at $path" ) @@ -181,4 +189,4 @@ private class OsProgressStore(workDir: os.Path, key: RunKey) // which an append-only log can't express, and it's small and bounded so a // full rewrite is negligible. private def writeLog(log: ProgressLog): Unit = - JsonFile.write(OrcaDir.progressFile(workDir, key), log) + JsonFile.write(orcaDir.progressFile(workDir, key), log) diff --git a/flow/src/main/scala/orca/review/FindingId.scala b/flow/src/main/scala/orca/review/FindingId.scala index 2c2d16993..5dce5f616 100644 --- a/flow/src/main/scala/orca/review/FindingId.scala +++ b/flow/src/main/scala/orca/review/FindingId.scala @@ -43,6 +43,6 @@ object FindingId: in.readString(default) def encodeValue(value: FindingId, out: JsonWriter): Unit = out.writeVal(value) - def nullValue: FindingId = null.asInstanceOf[FindingId] + def nullValue: FindingId = null - given Schema[FindingId] = Schema.string[FindingId] + given Schema[FindingId] = orca.json.JsonData.stringSchema[FindingId] diff --git a/flow/src/main/scala/orca/review/FixRequest.scala b/flow/src/main/scala/orca/review/FixRequest.scala index b30b40da0..d69fd00ed 100644 --- a/flow/src/main/scala/orca/review/FixRequest.scala +++ b/flow/src/main/scala/orca/review/FixRequest.scala @@ -4,11 +4,6 @@ import orca.agents.AgentInput import orca.json.{JsonData, given} /** The fix instruction plus the findings handed to the coding agent each round. - * - * Lives in its own compilation unit (not `ReviewLoop.scala`) because its - * `derives JsonData` expands the tapir `Schema` macro, whose generated code - * does not type-check under `ReviewLoop.scala`'s `captureChecking` mode - * (needed for the reviewer fan-out — ADR 0018 §6). */ private[review] case class FixRequest( instructions: String, diff --git a/flow/src/main/scala/orca/review/GateShell.scala b/flow/src/main/scala/orca/review/GateShell.scala new file mode 100644 index 000000000..8165f1e4d --- /dev/null +++ b/flow/src/main/scala/orca/review/GateShell.scala @@ -0,0 +1,33 @@ +package orca.review + +import orca.OrcaDir +import orca.effect.{FileSystem, Processes} +import orca.concurrency.resourceScope +import orca.subprocess.CliResult + +/** Runs the format and lint gates' commands, and spills lint output where the + * summariser can read it. + */ +private[orca] trait GateShell: + /** `command` through `bash -c` in `workDir`, stderr merged into stdout so + * neither stream reaches the terminal. A nonzero exit is returned, not + * thrown. + */ + def run(command: String, workDir: os.Path): CliResult + + /** `body` over a file under `workDir`'s `.orca/cache/` holding `text`, + * removed after. + */ + def withSpillFile[T](text: String, workDir: os.Path)(body: os.Path => T): T + +private[orca] final class OsGateShell(procs: Processes, fs: FileSystem) + extends GateShell: + def run(command: String, workDir: os.Path): CliResult = + procs.call(Seq("bash", "-c", "exec 2>&1\n" + command), workDir) + + def withSpillFile[T](text: String, workDir: os.Path)( + body: os.Path => T + ): T = + val cacheDir = OrcaDir(fs).ensureCache(workDir) + resourceScope: + body(fs.tempFile(text, "lint-", ".txt", dir = Some(cacheDir))) diff --git a/flow/src/main/scala/orca/review/Lint.scala b/flow/src/main/scala/orca/review/Lint.scala index 9de1a82e1..d31c5d5a4 100644 --- a/flow/src/main/scala/orca/review/Lint.scala +++ b/flow/src/main/scala/orca/review/Lint.scala @@ -5,7 +5,7 @@ package orca.review import language.experimental.captureChecking import language.experimental.separationChecking -import orca.{FlowContext, InStage, OrcaDir} +import orca.{FlowContext, InStage} import orca.agents.{Agent, Chat, PromptEvent} /** The lint gate run alongside the reviewers each round: `commands` (each run @@ -22,16 +22,14 @@ import orca.agents.{Agent, Chat, PromptEvent} */ case class Lint(commands: List[String], agent: Agent[?]) -/** Run `cmd` through `bash -c` in `ctx.workDir`. A nonzero exit is returned, - * not thrown — the caller decides what a failure means. `mergeErrIntoOut` - * folds stderr into the captured stdout so neither stream reaches the terminal - * and tears the renderer's status row. +/** Run `cmd` through `bash -c` in `ctx.workDir`, stderr merged into stdout (so + * neither tears the renderer's status row). A nonzero exit is returned, not + * thrown — the caller decides what a failure means. */ private[review] def runShell(cmd: String)(using ctx: FlowContext -): os.CommandResult = - os.proc("bash", "-c", cmd) - .call(cwd = ctx.workDir, check = false, mergeErrIntoOut = true) +): orca.subprocess.CliResult = + ctx.gateShell.run(cmd, ctx.workDir) /** One executed lint command, rendered for the summariser as a labelled block * headed @@ -85,7 +83,7 @@ def lint( )(using ctx: FlowContext, ev: InStage): LintReport = val runs = commands.map: command => val proc = runShell(command) - LintRun(command, proc.exitCode, proc.out.text().trim) + LintRun(command, proc.exitCode, proc.stdout.trim) // The summariser is skipped only when every run is both silent AND successful. val allClean = runs.forall(r => r.exitCode == 0 && r.output.isEmpty) val result = @@ -141,24 +139,12 @@ private def summariseRuns( // agent that denies reads outside its worktree can reach it). // `.orca/cache/` self-ignores via the `.gitignore` `ensureCache` writes // first, so a stage's `git add -A` can never sweep the spill file, even - // after a crash mid-lint. `deleteOnExit = false`: the `finally` owns - // cleanup, avoiding a JVM-exit hook per lint call. - val cacheDir = OrcaDir.ensureCache(ctx.workDir) - val outputFile = - os.temp( - combined, - dir = cacheDir, - prefix = "lint-", - suffix = ".txt", - deleteOnExit = false - ) - try + // after a crash mid-lint. + ctx.gateShell.withSpillFile(combined, ctx.workDir): outputFile => summarise( promptHead + s"The blocks are in `$outputFile`\n" + "(the file may be large — read it in parts if needed)." ) - finally - val _ = os.remove(outputFile) // Public: restricting the companion would restrict `Lint(...)` too. object Lint: diff --git a/flow/src/main/scala/orca/review/OpenFinding.scala b/flow/src/main/scala/orca/review/OpenFinding.scala index 6dfd5a5e7..d99b72e43 100644 --- a/flow/src/main/scala/orca/review/OpenFinding.scala +++ b/flow/src/main/scala/orca/review/OpenFinding.scala @@ -40,10 +40,10 @@ enum OpenReason derives JsonData: case Custom(text: String) def describe: String = this match - case Declined(text) => text - case NoFixes => "fixer reported no fixes" - case Unaccounted => "fixer did not report on it" - case CapReached(max) => s"max fix turns ($max) reached" + case Declined(text) => text + case NoFixes => "fixer reported no fixes" + case Unaccounted => "fixer did not report on it" + case CapReached(max) => s"max fix turns ($max) reached" case StillFailing(sources) => s"${sources.mkString(", ")} still failing after the fix turn" case Custom(text) => text diff --git a/flow/src/main/scala/orca/review/ParsedPrompt.scala b/flow/src/main/scala/orca/review/ParsedPrompt.scala index 268efaf44..3c1d6b5e2 100644 --- a/flow/src/main/scala/orca/review/ParsedPrompt.scala +++ b/flow/src/main/scala/orca/review/ParsedPrompt.scala @@ -115,11 +115,11 @@ private[review] object ParsedPrompt: if c == '\\' && i + 1 < s.length then val next = s.charAt(i + 1) val toAppend = next match - case 'n' => "\n" - case 't' => "\t" - case 'r' => "\r" - case '"' => "\"" - case '\\' => "\\" + case 'n' => "\n" + case 't' => "\t" + case 'r' => "\r" + case '"' => "\"" + case '\\' => "\\" case other => "\\" + other // preserve unknown sequences verbatim val _ = sb.append(toAppend) diff --git a/flow/src/main/scala/orca/review/ReviewLogging.scala b/flow/src/main/scala/orca/review/ReviewLogging.scala index 2379640d6..7fd987fef 100644 --- a/flow/src/main/scala/orca/review/ReviewLogging.scala +++ b/flow/src/main/scala/orca/review/ReviewLogging.scala @@ -2,7 +2,7 @@ package orca.review import orca.agents.AgentInput import orca.review.diff.{DiffMessage, LastSent, ReReviewChanges} -import org.slf4j.LoggerFactory +import orca.safe.Logger /** Records what each review turn was actually sent, at DEBUG. * @@ -18,7 +18,7 @@ import org.slf4j.LoggerFactory * needs no capture checking. */ private[review] object ReviewLogging: - private val log = LoggerFactory.getLogger("orca.flow") + private val log = Logger("orca.flow") /** A reviewer's turn. The payload's shape is what a post-mortem needs first: * it says whether the reviewer was sent the whole change set, part of it, a @@ -42,13 +42,7 @@ private[review] object ReviewLogging: case DiffMessage.ReReview(ReReviewChanges.AlreadySeen(last)) => (s"already-seen/${lastSentShape(last)}", 0, Nil) log.debug( - "review prompt: reviewer={} round={} payload={} chars={} files={}\n{}", - reviewer, - round, - shape, - chars, - joined(files), - prompt + s"review prompt: reviewer=$reviewer round=$round payload=$shape chars=$chars files=${joined(files)}\n$prompt" ) /** The fix turn: which findings the coder was handed, then the prompt. @@ -57,24 +51,17 @@ private[review] object ReviewLogging: * second time, which is wasted whenever DEBUG is off. */ def fix(request: FixRequest): Unit = - if log.isDebugEnabled then - log.debug( - "fix prompt: findings={} keys={}\n{}", - request.findings.size, - joined(request.findings.map(_.key)), - render(request) - ) + log.debug( + s"fix prompt: findings=${request.findings.size} keys=${joined(request.findings.map(_.key))}\n${render(request)}" + ) /** The reviewer picker's one call, which happens before any round. Guarded * for the same reason as [[fix]]. */ def reviewerPick(request: ReviewerSelectionRequest): Unit = - if log.isDebugEnabled then - log.debug( - "reviewer picker prompt: candidates={}\n{}", - joined(request.availableReviewers.map(_.name)), - render(request) - ) + log.debug( + s"reviewer picker prompt: candidates=${joined(request.availableReviewers.map(_.name))}\n${render(request)}" + ) private def render[A: AgentInput](input: A): String = summon[AgentInput[A]].serialize(input) diff --git a/flow/src/main/scala/orca/review/ReviewLoop.scala b/flow/src/main/scala/orca/review/ReviewLoop.scala index 71f6e94da..d0d30e265 100644 --- a/flow/src/main/scala/orca/review/ReviewLoop.scala +++ b/flow/src/main/scala/orca/review/ReviewLoop.scala @@ -932,7 +932,7 @@ private[review] class ReviewFixLoop( private def recheck(lintChat: Option[Lint.Summariser]): Recheck = val checked = runChecks(firstAgentIndex = lintGate.size) lintGate match - case None => Recheck(checked, lintChat = None) + case None => Recheck(checked, lintChat = None) case Some(gate) => val report = lint( gate.commands, diff --git a/flow/src/main/scala/orca/review/ReviewOnce.scala b/flow/src/main/scala/orca/review/ReviewOnce.scala index 0ac7ab9e5..624bd022d 100644 --- a/flow/src/main/scala/orca/review/ReviewOnce.scala +++ b/flow/src/main/scala/orca/review/ReviewOnce.scala @@ -26,7 +26,7 @@ def reviewOnce( reviewerSelection: ReviewerSelector = ReviewerSelector.agentDriven )(using ctx: FlowContext, ev: InStage, fc: FlowControl): ReviewReport = val source = ReviewDiff.resolve(diff) match - case Right(s) => s + case Right(s) => s case Left(skipped) => throw new OrcaFlowException(s"cannot review: ${skipped.describe}") val sample = source.sample() diff --git a/flow/src/main/scala/orca/review/ReviewReport.scala b/flow/src/main/scala/orca/review/ReviewReport.scala index 6eb223ef1..4bffdca89 100644 --- a/flow/src/main/scala/orca/review/ReviewReport.scala +++ b/flow/src/main/scala/orca/review/ReviewReport.scala @@ -3,9 +3,6 @@ package orca.review import orca.json.{JsonData, given} import orca.plan.Task -// Not under capture checking: `derives JsonData` expands tapir's Schema macro, -// which does not type-check there (same split as FixRequest.scala). - /** One reviewer's findings, named so a report can attribute each one. */ case class ReviewerFindings(reviewer: String, findings: List[ReviewFinding]) derives JsonData diff --git a/flow/src/main/scala/orca/review/ReviewerCatalog.scala b/flow/src/main/scala/orca/review/ReviewerCatalog.scala index 2535f54b2..eb97317db 100644 --- a/flow/src/main/scala/orca/review/ReviewerCatalog.scala +++ b/flow/src/main/scala/orca/review/ReviewerCatalog.scala @@ -1,6 +1,7 @@ package orca.review import orca.OrcaFlowException +import orca.effect.FileSystem import orca.discovery.{Origin, TierPrecedence, TierWinner} import orca.util.TextUtil @@ -89,12 +90,17 @@ object ReviewerCatalog: * does not cover a symlinked tier directory. */ private[orca] def discover( + fs: FileSystem, projectDir: os.Path, globalDir: os.Path ): ReviewerCatalog = val scans = List( - ReviewerFileTier.Project -> scan(projectDir, ReviewerFileTier.Project), - ReviewerFileTier.Global -> scan(globalDir, ReviewerFileTier.Global) + ReviewerFileTier.Project -> scan( + fs, + projectDir, + ReviewerFileTier.Project + ), + ReviewerFileTier.Global -> scan(fs, globalDir, ReviewerFileTier.Global) ) val (parseFailures, discovered) = TierPrecedence .resolve(scans.map((tier, s) => tier -> s.files)) @@ -171,22 +177,26 @@ object ReviewerCatalog: * legitimate is a directory named `*.md`, and skipping one would leave the * slug missing from the roster with nothing said. */ - private def scan(dir: os.Path, tier: ReviewerFileTier): ReviewerScan = - if !os.isDir(dir) then ReviewerScan(Nil, Map.empty) + private def scan( + fs: FileSystem, + dir: os.Path, + tier: ReviewerFileTier + ): ReviewerScan = + if !fs.isDir(dir) then ReviewerScan(Nil, Map.empty) else val candidates = - os.list(dir).filter(_.last.endsWith(".md")).filterNot(isDocument) + fs.list(dir).filter(_.last.endsWith(".md")).filterNot(isDocument) // `os.isLink` is lstat/no-follow, so this catches a dangling link too. val (linked, plain) = - if tier == ReviewerFileTier.Project then candidates.partition(os.isLink) + if tier == ReviewerFileTier.Project then candidates.partition(fs.isLink) else (IndexedSeq.empty, candidates) // After the symlink split, so a project-tier link to a directory keeps // the more specific `Symlinked` message. `os.isDir` follows links, which // is what catches a global-tier link to a directory; it is false for a // dangling link, so that one still surfaces as unreadable. - val (directories, files) = plain.partition(os.isDir) + val (directories, files) = plain.partition(fs.isDir) val (readFailures, parsed) = - files.map(read).toList.partitionMap(identity) + files.map(read(fs, _)).toList.partitionMap(identity) val (collisions, unique) = parsed .groupBy(f => stemOf(f.path)) .partitionMap: @@ -214,9 +224,10 @@ object ReviewerCatalog: * with no target, a permission error — is a failure, not an absence. */ private def read( + fs: FileSystem, path: os.Path ): Either[ReviewerPromptFailure, ReviewerFile] = - try Right(ReviewerFile(path, ParsedPrompt.parse(os.read(path)))) + try Right(ReviewerFile(path, ParsedPrompt.parse(fs.read(path)))) catch case e: java.io.IOException => Left( diff --git a/flow/src/main/scala/orca/review/ReviewerSelectionRequest.scala b/flow/src/main/scala/orca/review/ReviewerSelectionRequest.scala index 7d6bc0db0..57336b4ff 100644 --- a/flow/src/main/scala/orca/review/ReviewerSelectionRequest.scala +++ b/flow/src/main/scala/orca/review/ReviewerSelectionRequest.scala @@ -4,19 +4,12 @@ import orca.agents.AgentInput import orca.json.{JsonData, given} import orca.plan.Title -/** The `(name, description)` pair the picker LLM sees for one reviewer. - * - * In its own compilation unit because `derives JsonData` expands the tapir - * `Schema` macro, whose generated code doesn't type-check under - * `ReviewerSelector.scala`'s `captureChecking` mode (same split as - * `FixRequest.scala`). - */ +/** The `(name, description)` pair the picker LLM sees for one reviewer. */ private[review] case class ReviewerInfo(name: String, description: String) derives JsonData /** The request handed to [[ReviewerSelector.agentDriven]]'s picker LLM: the * task, the changed files, the eligible reviewers, and the selection brief. - * Non-CC for the same reason as [[ReviewerInfo]]. */ private[review] case class ReviewerSelectionRequest( taskTitle: Title, diff --git a/flow/src/main/scala/orca/review/ReviewerSelector.scala b/flow/src/main/scala/orca/review/ReviewerSelector.scala index 1cb1077ed..018d5b9ad 100644 --- a/flow/src/main/scala/orca/review/ReviewerSelector.scala +++ b/flow/src/main/scala/orca/review/ReviewerSelector.scala @@ -214,7 +214,7 @@ object ReviewerSelector: history => val active = basePick(history) history.headOption match - case None => active + case None => active case Some(previous) => val reported = previous.reviewersWithFindings val narrowed = active.filter(e => reported.exists(_.id == e.id)) diff --git a/flow/src/main/scala/orca/review/ReviewerTurn.scala b/flow/src/main/scala/orca/review/ReviewerTurn.scala index e3f236427..c370f5653 100644 --- a/flow/src/main/scala/orca/review/ReviewerTurn.scala +++ b/flow/src/main/scala/orca/review/ReviewerTurn.scala @@ -49,7 +49,7 @@ private[review] object ReviewerTurn: )(using InStage): (ReviewResult, SessionEntry) = val (chat, delivery) = stored match case Some(se) => (se.chat, DiffDelivery.next(se.lastSent, current)) - case None => + case None => ( e.agent.withRole(ReviewerPrompts.Role).chat(), DiffDelivery.first(current) diff --git a/flow/src/main/scala/orca/review/Reviewers.scala b/flow/src/main/scala/orca/review/Reviewers.scala index 4aa96cd08..306e8197c 100644 --- a/flow/src/main/scala/orca/review/Reviewers.scala +++ b/flow/src/main/scala/orca/review/Reviewers.scala @@ -3,8 +3,7 @@ package orca.review import orca.{FlowContext, OrcaFlowException} import orca.agents.{BackendTag, Agent} -import ox.either -import ox.either.ok +import orca.safe.prelude.{either, ok} import scala.util.matching.Regex diff --git a/flow/src/main/scala/orca/review/SelectedReviewers.scala b/flow/src/main/scala/orca/review/SelectedReviewers.scala index f065f7f4e..84df4ba92 100644 --- a/flow/src/main/scala/orca/review/SelectedReviewers.scala +++ b/flow/src/main/scala/orca/review/SelectedReviewers.scala @@ -4,8 +4,6 @@ import orca.agents.Announce import orca.json.JsonData import orca.util.PromptResource -import sttp.tapir.Validator - /** What [[SelectedReviewers.pick]] resolved: the roster entries the picker * named, in roster order, and the names that matched none of them. A caller * cannot consume a partially-failed pick without naming the residue — a @@ -47,7 +45,7 @@ object SelectedReviewers: given JsonData[SelectedReviewers] = val plain = JsonData.derived[SelectedReviewers] JsonData( - plain.schema.modify(_.names)(_.validate(Validator.nonEmpty)), + plain.schema.modify(_.names)(_.validate(JsonData.nonEmptyValidator)), plain.codec ) diff --git a/flow/src/main/scala/orca/runlog/AttemptOutcome.scala b/flow/src/main/scala/orca/runlog/AttemptOutcome.scala index fb77bc059..81c1910a2 100644 --- a/flow/src/main/scala/orca/runlog/AttemptOutcome.scala +++ b/flow/src/main/scala/orca/runlog/AttemptOutcome.scala @@ -1,16 +1,13 @@ package orca.runlog -import com.github.plokhotnyuk.jsoniter_scala.macros.{ - CodecMakerConfig, - ConfiguredJsonValueCodec -} +import orca.json.JsonCodec /** How an attempt ended, as [[RunEvent.AttemptFinished]] records it. */ private[orca] enum AttemptOutcome: case Succeeded, Failed private[orca] object AttemptOutcome: - given codec: ConfiguredJsonValueCodec[AttemptOutcome] = - ConfiguredJsonValueCodec.derived[AttemptOutcome](using - CodecMakerConfig.withDiscriminatorFieldName(None) + given codec: JsonCodec[AttemptOutcome] = + JsonCodec.derived[AttemptOutcome](using + JsonCodec.config.withDiscriminatorFieldName(None) ) diff --git a/flow/src/main/scala/orca/runlog/RunEvent.scala b/flow/src/main/scala/orca/runlog/RunEvent.scala index 6ec6aa543..f903ef53a 100644 --- a/flow/src/main/scala/orca/runlog/RunEvent.scala +++ b/flow/src/main/scala/orca/runlog/RunEvent.scala @@ -2,14 +2,9 @@ package orca.runlog import com.github.plokhotnyuk.jsoniter_scala.core.{ JsonReaderException, - JsonValueCodec, - readFromString, - writeToString -} -import com.github.plokhotnyuk.jsoniter_scala.macros.{ - CodecMakerConfig, - ConfiguredJsonValueCodec + JsonValueCodec } +import orca.json.JsonCodec import orca.{AttemptId, StagePath} import orca.agents.{BackendTag, SessionKey} import orca.events.{Cost, StageOutcome} @@ -134,19 +129,19 @@ private[orca] object RunEvent: summon[JsonData[BranchName]].codec private given JsonValueCodec[StageOutcome] = - ConfiguredJsonValueCodec.derived[StageOutcome](using - CodecMakerConfig.withDiscriminatorFieldName(None) + JsonCodec.derived[StageOutcome](using + JsonCodec.config.withDiscriminatorFieldName(None) ) given codec: JsonValueCodec[RunEvent] = - ConfiguredJsonValueCodec.derived[RunEvent](using - CodecMakerConfig.withDiscriminatorFieldName(Some("type")) + JsonCodec.derived[RunEvent](using + JsonCodec.config.withDiscriminatorFieldName(Some("type")) ) /** The event as one JSON line, without the newline. */ - def encodeLine(event: RunEvent): String = writeToString(event) + def encodeLine(event: RunEvent): String = JsonCodec.writeString(event) /** The event on `line`; `None` when it does not decode. */ def decodeLine(line: String): Option[RunEvent] = - try Some(readFromString[RunEvent](line)) + try Some(JsonCodec.readString[RunEvent](line)) catch case _: JsonReaderException => None diff --git a/flow/src/main/scala/orca/runlog/RunEventLog.scala b/flow/src/main/scala/orca/runlog/RunEventLog.scala index 72fe2af3e..834016588 100644 --- a/flow/src/main/scala/orca/runlog/RunEventLog.scala +++ b/flow/src/main/scala/orca/runlog/RunEventLog.scala @@ -1,15 +1,19 @@ package orca.runlog import orca.{AttemptId, OrcaDir, RunKey, StagePath} +import orca.effect.FileSystem import orca.events.{OrcaEvent, OrcaListener} import orca.gitref.BranchName import orca.sessions.{SessionRecord, SessionStore} -import org.slf4j.LoggerFactory +import orca.safe.Logger +import orca.concurrency +import orca.concurrency.Ref import ox.Ox -import ox.channels.{Actor, ActorRef, BufferCapacity} +import ox.Fork +import ox.channels.{ActorRef, BufferCapacity, Channel} import java.time.Instant -import scala.util.control.NonFatal +import orca.safe.prelude.NonFatal /** The attempt's writer of its run's event log, * `.orca/cache/runs//events.jsonl` (ADR 0025), and the run's @@ -17,7 +21,7 @@ import scala.util.control.NonFatal * * Every call waits until the in-memory projection is updated; appending to the * file happens in the background, in call order. No call throws: a failed - * append is logged and dropped. + * append is logged and dropped, and so is anything recorded after [[finish]]. */ private[orca] trait RunEventLog extends OrcaListener, SessionStore: /** Records the run's success on `branch`, with the published PR/MR reference; @@ -34,28 +38,34 @@ private[orca] object RunEventLog: /** Prunes the run cache ([[RunPruning]]), loads the session records from the * run's event log and appends the attempt's [[RunEvent.AttemptStarted]]. The - * actors live as long as the enclosing scope, which must span every call up - * to [[RunEventLog.finish]]. + * projection's actor and the file's writer fork live in the enclosing scope, + * which must span every call up to [[RunEventLog.finish]]. * * `flowName` is the launched script's filename; `tracePath` the attempt's * trace log, if it has one. */ def start( + fs: FileSystem, workDir: os.Path, runKey: RunKey, attemptId: AttemptId, orcaVersion: String, flowName: Option[String], tracePath: Option[os.Path], - clock: () => Instant + clock: () -> Instant )(using Ox, BufferCapacity): RunEventLog = - val eventLog = OrcaDir.ensureEventLog(workDir, runKey) - prune(workDir, runKey) - val loaded = SessionProjection.records(RunEventReader.read(eventLog)) - endTornLine(eventLog) - val appender = Actor.create(EventAppender(eventLog)) - val owner = Actor.create( - ProjectionOwner(Projection(loaded, Nil), attemptId, clock, appender) + val eventLog = OrcaDir(fs).ensureEventLog(workDir, runKey) + prune(fs, workDir, runKey) + val loaded = SessionProjection.records(RunEventReader(fs).read(eventLog)) + endTornLine(fs, eventLog) + val queue = concurrency.bufferedDefaultChannel[RunEvent] + val writer = concurrency.fork( + concurrency + .receiveUntilClosed(queue) + .foreach(appendEvent(fs, eventLog, _)) + ) + val owner = concurrency.actor( + ProjectionOwner(Projection(loaded, Nil), attemptId, clock, queue) ) owner.ask( _.started( @@ -71,31 +81,31 @@ private[orca] object RunEventLog: ) ) ) - ActorRunEventLog(owner, appender) + ActorRunEventLog(owner, writer) // A crash mid-append leaves a last line without its newline; the next // append would join it, and neither line would decode. - private def endTornLine(eventLog: os.Path): Unit = + private def endTornLine(fs: FileSystem, eventLog: os.Path): Unit = try - if os.isFile(eventLog) then - val size = os.size(eventLog) - if size > 0 && os.read.bytes(eventLog, size - 1, 1).head != '\n' then - os.write.append(eventLog, "\n") + if fs.isFile(eventLog) then + if fs.readSuffix(eventLog, 1).headOption.exists(_ != '\n') then + fs.append(eventLog, "\n") catch case NonFatal(e) => log.warn(s"ending the torn last line of $eventLog failed", e) - private val log = LoggerFactory.getLogger("orca.flow") + private[runlog] val log = Logger("orca.flow") - private def prune(workDir: os.Path, runKey: RunKey): Unit = + private def prune(fs: FileSystem, workDir: os.Path, runKey: RunKey): Unit = val cacheRuns = OrcaDir.cacheRunsPath(workDir) - RunPruning.removeLegacy(OrcaDir.cachePath(workDir)) - RunPruning.pruneRuns(cacheRuns, runKey) - RunPruning.pruneTraces(cacheRuns) + val pruning = RunPruning(fs) + pruning.removeLegacy(OrcaDir.cachePath(workDir)) + pruning.pruneRuns(cacheRuns, runKey) + pruning.pruneTraces(cacheRuns) private class ActorRunEventLog( owner: ActorRef[ProjectionOwner], - appender: ActorRef[EventAppender] + writer: Fork[Unit] ) extends RunEventLog: def onEvent(event: OrcaEvent): Unit = owner.ask(_.observed(event)) def records(): List[SessionRecord] = owner.ask(_.records) @@ -106,9 +116,9 @@ private class ActorRunEventLog( def finish(outcome: AttemptOutcome): Unit = owner.ask(_.finished(outcome)) - // The appender handles its mailbox in order, so this returns after every + // The writer drains the queue in order, so this returns after every // queued append. - appender.ask(_ => ()) + writer.join() /** What the attempt knows of its run: the session records, and the open stages * innermost first. Each transition returns the next projection with the events @@ -235,38 +245,40 @@ private case class Projection( conversationKey = t.spend.conversationKey ) -/** Owns the [[Projection]]; ADR 0025 records why it is a `var` in an actor. - * Every method runs on the actor's thread, never throws, and hands the events - * it records to `appender`. +/** Owns the [[Projection]]. Every method runs on the actor's thread, never + * throws, and queues the events it records for the writer fork. [[finished]] + * closes the queue; a call after it is logged and dropped. */ private class ProjectionOwner( initial: Projection, attempt: AttemptId, - clock: () => Instant, - appender: ActorRef[EventAppender] + clock: () -> Instant, + queue: Channel[RunEvent] ): - private val log = LoggerFactory.getLogger("orca.flow") + import RunEventLog.log - private var state = initial + // The actor serialises every read and replacement, so plain get/set + // suffice: a `var` would do, but safe mode rejects `var` fields. + private val state: Ref[Projection] = Ref(initial) - def records: List[SessionRecord] = state.records + def records: List[SessionRecord] = state.get.records def started(event: RunEvent.AttemptStarted): Unit = guarded("recording the attempt's start"): - advance(state.recording(List(event))) + advance(state.get.recording(List(event))) def observed(event: OrcaEvent): Unit = guarded(s"recording ${event.getClass.getSimpleName}"): - advance(state.observe(event, clock(), attempt)) + advance(state.get.observe(event, clock(), attempt)) def upserted(record: SessionRecord): Unit = guarded("recording a session"): - advance(state.upsert(record, clock(), attempt)) + advance(state.get.upsert(record, clock(), attempt)) def succeeded(branch: BranchName, published: Option[String]): Unit = guarded("recording the run's success"): advance( - state.recording( + state.get.recording( List(RunEvent.RunSucceeded(clock(), attempt, branch, published)) ) ) @@ -274,32 +286,34 @@ private class ProjectionOwner( def finished(outcome: AttemptOutcome): Unit = guarded("recording the attempt's end"): advance( - state.recording( + state.get.recording( List(RunEvent.AttemptFinished(clock(), attempt, outcome)) ) ) + val _ = queue.doneOrClosed() /** Moves to the next projection, then queues its events for the file. */ private def advance(transition: (Projection, List[RunEvent])): Unit = val (next, events) = transition - state = next - events.foreach(e => appender.tell(_.append(e))) + state.set(next) + events.foreach(queue.send) // A throw here would quarantine the listener (from `observed`) or reach the // flow's teardown (from the rest). private def guarded(what: String)(op: => Unit): Unit = - try op - catch case NonFatal(e) => log.warn(s"$what failed (best-effort)", e) + if queue.isClosedForSend then + log.warn(s"$what after the attempt finished (dropped)") + else + try op + catch case NonFatal(e) => log.warn(s"$what failed (best-effort)", e) -/** Appends events to the event log at `path`. Runs on its own actor, so the +/** Appends `event` to the event log at `path`. Runs on the writer fork, so the * attempt does not wait on disk; a failed append is logged and the event - * dropped, since a throw would end the actor's scope. + * dropped, since a throw would end the writer's scope. */ -private class EventAppender(path: os.Path): - private val log = LoggerFactory.getLogger("orca.flow") - - def append(event: RunEvent): Unit = - try os.write.append(path, RunEvent.encodeLine(event) + "\n") - catch - case NonFatal(e) => - log.warn(s"appending to the run event log $path failed (dropped)", e) +private def appendEvent(fs: FileSystem, path: os.Path, event: RunEvent): Unit = + try fs.append(path, RunEvent.encodeLine(event) + "\n") + catch + case NonFatal(e) => + RunEventLog.log + .warn(s"appending to the run event log $path failed (dropped)", e) diff --git a/flow/src/main/scala/orca/runlog/RunEventReader.scala b/flow/src/main/scala/orca/runlog/RunEventReader.scala index 60f655550..6f59bb81c 100644 --- a/flow/src/main/scala/orca/runlog/RunEventReader.scala +++ b/flow/src/main/scala/orca/runlog/RunEventReader.scala @@ -2,10 +2,10 @@ package orca.runlog import com.github.plokhotnyuk.jsoniter_scala.core.{ JsonReaderException, - JsonValueCodec, - readFromString + JsonValueCodec } -import com.github.plokhotnyuk.jsoniter_scala.macros.JsonCodecMaker +import orca.effect.FileSystem +import orca.json.JsonCodec import java.io.{IOException, UncheckedIOException} import java.nio.file.NoSuchFileException @@ -14,7 +14,7 @@ import java.nio.file.NoSuchFileException * crash, an unknown `type` — are skipped. An absent or unreadable file reads * as no events. */ -private[orca] object RunEventReader: +private[orca] final class RunEventReader(fs: FileSystem): /** Every event in `path`, in file order. */ def read(path: os.Path): List[RunEvent] = readOrError(path).getOrElse(Nil) @@ -45,15 +45,19 @@ private[orca] object RunEventReader: private def decodedLines(path: os.Path)( decode: String => Option[RunEvent] ): Either[IOException | UncheckedIOException, List[RunEvent]] = - try Right(os.read.lines.stream(path).flatMap(decode(_)).toList) + try Right(fs.collectLines(path)(decode)) catch case _: NoSuchFileException => Right(Nil) case e: (IOException | UncheckedIOException) => Left(e) - private case class TypeOnly(`type`: String) - - private val typeOnlyCodec: JsonValueCodec[TypeOnly] = JsonCodecMaker.make - private def typeOf(line: String): Option[String] = - try Some(readFromString(line)(using typeOnlyCodec).`type`) + try + Some( + JsonCodec.readString(line)(using RunEventReader.typeOnlyCodec).`type` + ) catch case _: JsonReaderException => None + +private object RunEventReader: + case class TypeOnly(`type`: String) + + val typeOnlyCodec: JsonValueCodec[TypeOnly] = JsonCodec.make diff --git a/flow/src/main/scala/orca/runlog/RunPruning.scala b/flow/src/main/scala/orca/runlog/RunPruning.scala index c0d9c1ce1..c62062125 100644 --- a/flow/src/main/scala/orca/runlog/RunPruning.scala +++ b/flow/src/main/scala/orca/runlog/RunPruning.scala @@ -1,26 +1,19 @@ package orca.runlog import orca.{OrcaDir, RunKey} +import orca.effect.FileSystem +import orca.safe.prelude.given -import scala.util.control.NonFatal +import orca.safe.prelude.NonFatal /** Bounds `.orca/cache/runs/` by run count and by trace count, and removes the * cache files older versions wrote (ADR 0025). Runs at attempt start. Each * delete is guarded on its own, so a file a concurrent cleanup got to first * does not stop the rest. */ -private[runlog] object RunPruning: - - /** How many of the newest runs, and of the newest runs that recorded a - * session, are kept. Resumable runs are kept regardless. - */ - val MaxKept: Int = 20 - - /** How many attempts' trace logs are kept, across all runs. Counted apart - * from runs, since resumable runs are kept without limit and traces are the - * bulk of the cache. - */ - val MaxKeptTraces: Int = 40 +private[runlog] final class RunPruning(fs: FileSystem): + import RunPruning.* + private val reader = RunEventReader(fs) /** Deletes the run directories under `cacheRuns` outside the kept sets. Kept: * the `current` run's, every run with a session minted after its last @@ -42,9 +35,9 @@ private[runlog] object RunPruning: */ def pruneTraces(cacheRuns: os.Path): Unit = val byAttempt = listed(cacheRuns) - .filter(os.isDir(_, followLinks = false)) + .filter(p => fs.isDir(p) && !fs.isLink(p)) .flatMap(listed) - .filter(os.isFile(_, followLinks = false)) + .filter(p => fs.isFile(p) && !fs.isLink(p)) .groupBy(OrcaDir.attemptIdOf) .collect: case (Some(id), files) => id -> files @@ -63,7 +56,7 @@ private[runlog] object RunPruning: removeGuarded(cache / "attempts") listed(cache / "runs") .filter(f => - f.last.endsWith(".sessions.json") && os.isFile(f, followLinks = false) + f.last.endsWith(".sessions.json") && fs.isFile(f) && !fs.isLink(f) ) .foreach(removeGuarded) @@ -79,7 +72,7 @@ private[runlog] object RunPruning: private def runsNewestFirst(cacheRuns: os.Path, current: RunKey): List[Run] = listed(cacheRuns) - .filter(d => os.isDir(d, followLinks = false) && d.last != current.value) + .filter(d => fs.isDir(d) && !fs.isLink(d) && d.last != current.value) .map(runOf) .sortBy(_.modifiedAt)(using Ordering[Option[Long]].reverse) @@ -90,7 +83,7 @@ private[runlog] object RunPruning: private def runOf(dir: os.Path): Run = val log = dir / OrcaDir.EventLogName - val events = RunEventReader.readOnly( + val events = reader.readOnly( log, Set(classOf[RunEvent.SessionMinted], classOf[RunEvent.RunSucceeded]) ) @@ -99,12 +92,29 @@ private[runlog] object RunPruning: else if events.exists(_.isInstanceOf[RunEvent.SessionMinted]) then Sessions.Recorded else Sessions.NoneRecorded - Run(dir, Option.when(os.isFile(log))(os.mtime(log)), sessions) + Run( + dir, + Option.when(fs.isFile(log))(fs.modifiedAt(log).toEpochMilli), + sessions + ) private def listed(dir: os.Path): List[os.Path] = - try if os.isDir(dir) then os.list(dir).toList else Nil + try if fs.isDir(dir) then fs.list(dir).toList else Nil catch case NonFatal(_) => Nil private def removeGuarded(path: os.Path): Unit = - try os.remove.all(path) + try fs.removeAll(path) catch case NonFatal(_) => () + +private[runlog] object RunPruning: + + /** How many of the newest runs, and of the newest runs that recorded a + * session, are kept. Resumable runs are kept regardless. + */ + val MaxKept: Int = 20 + + /** How many attempts' trace logs are kept, across all runs. Counted apart + * from runs, since resumable runs are kept without limit and traces are the + * bulk of the cache. + */ + val MaxKeptTraces: Int = 40 diff --git a/flow/src/main/scala/orca/runlog/SessionProjection.scala b/flow/src/main/scala/orca/runlog/SessionProjection.scala index 3ff6b9136..ffdb3983a 100644 --- a/flow/src/main/scala/orca/runlog/SessionProjection.scala +++ b/flow/src/main/scala/orca/runlog/SessionProjection.scala @@ -20,7 +20,7 @@ private[runlog] object SessionProjection: event: RunEvent ): List[SessionRecord] = event match - case _: RunEvent.RunSucceeded => Nil + case _: RunEvent.RunSucceeded => Nil case m: RunEvent.SessionMinted => val record = SessionRecord( name = m.name, diff --git a/flow/src/main/scala/orca/settings/AgentSpec.scala b/flow/src/main/scala/orca/settings/AgentSpec.scala index 4b51ac6f2..197f49efb 100644 --- a/flow/src/main/scala/orca/settings/AgentSpec.scala +++ b/flow/src/main/scala/orca/settings/AgentSpec.scala @@ -62,7 +62,7 @@ private[orca] object AgentSpec: private def splitModel(pinned: String): (String, Option[String]) = pinned.indexOf(':') match case -1 => (pinned.trim, None) - case i => + case i => (pinned.take(i).trim, Some(pinned.drop(i + 1).trim).filter(_.nonEmpty)) /** `effort`, if the harness accepts it. */ @@ -86,7 +86,7 @@ private[orca] object AgentSpec: ): Either[String, (String, Option[Effort])] = value.lastIndexOf('@') match case -1 => Right((value, None)) - case i => + case i => val suffix = value.drop(i + 1).trim if !suffix.forall(_.isLetter) then Right((value, None)) else diff --git a/flow/src/main/scala/orca/settings/SettingKey.scala b/flow/src/main/scala/orca/settings/SettingKey.scala index 4e1a0b86f..4dda28915 100644 --- a/flow/src/main/scala/orca/settings/SettingKey.scala +++ b/flow/src/main/scala/orca/settings/SettingKey.scala @@ -42,5 +42,6 @@ private[settings] enum AgentKey(val raw: String) extends SettingKey: case ReviewAgent extends AgentKey("reviewAgent") private[settings] object SettingKey: - val values: Array[SettingKey] = StackKey.values ++ AgentKey.values + val values: List[SettingKey] = + StackKey.values.toList ++ AgentKey.values.toList def fromRaw(s: String): Option[SettingKey] = values.find(_.raw == s) diff --git a/flow/src/main/scala/orca/settings/SettingsFile.scala b/flow/src/main/scala/orca/settings/SettingsFile.scala index f2a119b60..a975abba8 100644 --- a/flow/src/main/scala/orca/settings/SettingsFile.scala +++ b/flow/src/main/scala/orca/settings/SettingsFile.scala @@ -92,7 +92,7 @@ private[orca] object SettingsFile: scope: SettingsScope ): Either[SettingsError, ParsedSettings] = Line.of(line) match - case Line.Inert => Right(acc) + case Line.Inert => Right(acc) case Line.Malformed(text) => Left(SettingsError.NoAssignment(number, text)) case Line.Assignment(rawKey, value) => @@ -115,8 +115,8 @@ private[orca] object SettingsFile: StackValue.parse(value) match case StackValue.Run(command) => Right(acc.copy(stack = Some(key.appendTo(stackSoFar, command)))) - case StackValue.Off => Right(acc.copy(stack = Some(stackSoFar))) - case StackValue.Empty => Right(acc) + case StackValue.Off => Right(acc.copy(stack = Some(stackSoFar))) + case StackValue.Empty => Right(acc) case StackValue.CommentedOut => Left(SettingsError.CommentedValue(number, key.raw)) @@ -260,14 +260,14 @@ private[orca] object SettingsFile: index: Int ): Set[Int] = def bare(line: String): String = line.stripLineEnd - val evidence = scala.collection.mutable.Set.empty[Int] + var evidence = Set.empty[Int] var j = index - 1 while j >= 0 && bare(lines(j)).trim.startsWith("#") && !headerLines(bare(lines(j))) do evidence += j j -= 1 - evidence.toSet + evidence /** `content` with every LIVE `format`/`lint`/`test` line removed, plus each * one's evidence comment block (see [[evidenceAbove]]) and every diff --git a/flow/src/main/scala/orca/stage.scala b/flow/src/main/scala/orca/stage.scala index 29afa8c8b..637ec455b 100644 --- a/flow/src/main/scala/orca/stage.scala +++ b/flow/src/main/scala/orca/stage.scala @@ -1,19 +1,16 @@ package orca -import com.github.plokhotnyuk.jsoniter_scala.core.{ - readFromString, - writeToString -} +import orca.json.JsonCodec import orca.events.{OrcaEvent, StageOutcome} import orca.gitref.CommitHash import orca.json.{JsonData, RawJson} import orca.progress.{StageEntry, StageStart} import orca.util.TextUtil -import org.slf4j.LoggerFactory +import orca.safe.Logger -import scala.util.control.NonFatal +import orca.safe.prelude.NonFatal -private val log = LoggerFactory.getLogger("orca.flow") +private val log = Logger("orca.flow") /** Run `body` as a named, resumable, committing stage (ADR 0018 §2.1). * @@ -41,7 +38,7 @@ private val log = LoggerFactory.getLogger("orca.flow") */ def stage[T: JsonData]( name: String, - commitMessage: Option[T => String] = None + commitMessage: Option[T -> String] = None )(body: (InStage, WorkspaceWrite) ?=> T)(using FlowContext, FlowControl): T = inStageFrame(name): id => resumeFrom(id).getOrElse(runStage(id, commitMessage)(body)) @@ -114,7 +111,7 @@ private[orca] def gatedStage[G, A, T: JsonData](name: String)( inStageFrame(name): id => resumeFrom[T](id) match case Some(value) => Right(Staged.Replayed(value)) - case None => + case None => gate.map(a => Staged.Fresh(runStage(id, None)(body(a)))) /** [[stage]] reporting where its result came from. */ @@ -141,7 +138,7 @@ private def resumeFrom[T: JsonData](id: StagePath.Stage)(using val decoded = try Some( - readFromString[T](entry.resultJson.value)(using + JsonCodec.readString[T](entry.resultJson.value)(using summon[JsonData[T]].codec ) ) @@ -156,7 +153,7 @@ private def resumeFrom[T: JsonData](id: StagePath.Stage)(using /** Run the body fresh, then record its result and commit (steps 3–4 above). */ private def runStage[T: JsonData]( id: StagePath.Stage, - commitMessage: Option[T => String] + commitMessage: Option[T -> String] )(body: (InStage, WorkspaceWrite) ?=> T)(using ctx: FlowContext, fc: FlowControl @@ -197,9 +194,10 @@ private def runStage[T: JsonData]( private def recordAndCommit[T: JsonData]( id: StagePath.Stage, result: T, - commitMessage: Option[T => String] + commitMessage: Option[T -> String] )(using ctx: FlowContext, fc: FlowControl): Unit = - val resultJson = writeToString(result)(using summon[JsonData[T]].codec) + val resultJson = + JsonCodec.writeString(result)(using summon[JsonData[T]].codec) // Fresh runtime tokens rather than the body's: recording + committing is the // runtime's own privileged step, not part of the user body. `InStage` is for // the cheap-model commit-message fallback, `WorkspaceWrite` for the @@ -216,8 +214,8 @@ private def recordAndCommit[T: JsonData]( // may already have committed this entry): log at DEBUG, never fail the stage. ctx.runtimeGit.commit(message) match case Right(()) => () - case Left(_) => - log.debug("stage {} commit was empty (already recorded?)", id.name) + case Left(_) => + log.debug(s"stage ${id.name} commit was empty (already recorded?)") /** Generate a commit message from the current working-tree changes via the * coding-role agent's cheap model (`codingAgent.cheapOneShot`), which is sent diff --git a/flow/src/test/resources/flow-logback-test.xml b/flow/src/test/resources/flow-logback-test.xml new file mode 100644 index 000000000..037c11dc6 --- /dev/null +++ b/flow/src/test/resources/flow-logback-test.xml @@ -0,0 +1,4 @@ + + + + diff --git a/flow/src/test/scala/orca/BoundedDiffTest.scala b/flow/src/test/scala/orca/BoundedDiffTest.scala index 0bcf5bb5c..7edecd0ef 100644 --- a/flow/src/test/scala/orca/BoundedDiffTest.scala +++ b/flow/src/test/scala/orca/BoundedDiffTest.scala @@ -260,7 +260,7 @@ class BoundedDiffTest extends munit.FunSuite: ): String = BoundedDiff.sectionsPayload(sections, paths, maxChars) match case BoundedDiff.SectionsCut.Rendered(payload) => payload - case BoundedDiff.SectionsCut.NothingFits => + case BoundedDiff.SectionsCut.NothingFits => fail(s"expected sections for $paths") test("the sections payload carries the requested files and nothing else"): diff --git a/flow/src/test/scala/orca/CcNegativeCompileTest.scala b/flow/src/test/scala/orca/CcNegativeCompileTest.scala index 4de3dc41a..cd7f867c7 100644 --- a/flow/src/test/scala/orca/CcNegativeCompileTest.scala +++ b/flow/src/test/scala/orca/CcNegativeCompileTest.scala @@ -1,13 +1,6 @@ package orca -import java.nio.file.Files - -import scala.collection.mutable.ListBuffer - -import dotty.tools.dotc.Main -import dotty.tools.dotc.core.Contexts.Context -import dotty.tools.dotc.interfaces.Diagnostic as IDiagnostic -import dotty.tools.dotc.reporting.{Diagnostic, Reporter} +import orca.testkit.Dotc /** Capture-checking enforcement pinned as automated negative-compile tests, so * a compiler upgrade or a wrapper edit can't silently drop it. The checked @@ -17,13 +10,6 @@ import dotty.tools.dotc.reporting.{Diagnostic, Reporter} * widening that "hides" the exclusive token), not at the wrapper boundary — * see [[orca.CheckedPar]]'s object doc. * - * An in-process `dotc` is used, not munit's `compileErrors`: that macro runs - * only the typer, but capture/separation checking runs post-typer, so a - * `compileErrors` version of the exclusive-capture cases would report ZERO - * errors and pass vacuously. Invoking the real `dotc` - * ([[dotty.tools.dotc.Main]]) against this module's own Test classpath runs - * the full checked-compilation pipeline. - * * Fixtures declare `package orca` because [[orca.CheckedPar]] is * `private[orca]`. They pass the capability token in as a method parameter * rather than minting it via `InStage.unsafe`: passing an exclusive parameter @@ -32,59 +18,6 @@ import dotty.tools.dotc.reporting.{Diagnostic, Reporter} */ class CcNegativeCompileTest extends munit.FunSuite: - /** Collects error/warning diagnostics from a `dotc` run instead of parsing - * stdout — a custom [[Reporter]] is the driver API's supported inspection - * point. - */ - private class CollectingReporter extends Reporter: - val diagnostics: ListBuffer[Diagnostic] = ListBuffer.empty - def doReport(dia: Diagnostic)(using Context): Unit = - val _ = diagnostics += dia - - /** The Test classpath. Tests are forked, but sbt 2 starts the fork through - * its own worker, so `java.class.path` holds only the worker's jars; the - * build passes the real classpath as `orca.test.classpath`. Falls back to - * `java.class.path` when run outside sbt. - */ - private def classpath: String = - sys.props.getOrElse( - "orca.test.classpath", - System.getProperty("java.class.path") - ) - - /** Compile one fixture source with `dotc` and return its error messages. Each - * call spins a fresh reporter and output dir; the classpath (the expensive - * part to assemble) is shared. - */ - private def compileErrorsOf(source: String): List[String] = - val srcDir = Files.createTempDirectory("cc-src") - val outDir = Files.createTempDirectory("cc-out") - try - val srcFile = srcDir.resolve("Fixture.scala") - Files.writeString(srcFile, source) - val reporter = new CollectingReporter - val args = Array( - "-classpath", - classpath, - "-d", - outDir.toString, - srcFile.toString - ) - val _ = Main.process(args, reporter) - reporter.diagnostics.toList - .filter(_.level >= IDiagnostic.ERROR) - .map(_.message) - finally - deleteRecursively(srcDir) - deleteRecursively(outDir) - - private def deleteRecursively(dir: java.nio.file.Path): Unit = - if Files.exists(dir) then - Files - .walk(dir) - .sorted(java.util.Comparator.reverseOrder()) - .forEach(Files.delete) - /** Fan-out fixture whose closures capture a `tok` of the given token type, * routed through the real `orca.CheckedPar.mapParUnordered`. Identical shape * across cases — only the token type varies — so the positive and negative @@ -106,7 +39,7 @@ class CcNegativeCompileTest extends munit.FunSuite: // compile errors at all — otherwise every negative case below would pass // vacuously. A plain type error must surface as a non-empty error list. test("harness detects a plain compile error"): - val errors = compileErrorsOf( + val errors = Dotc.errorsOf( "object Broken:\n val n: Int = \"not an int\"\n" ) assert( @@ -115,18 +48,18 @@ class CcNegativeCompileTest extends munit.FunSuite: ) test("(a) POSITIVE: capturing InStage (shared) compiles"): - val errors = compileErrorsOf(fixture("orca.InStage")) + val errors = Dotc.errorsOf(fixture("orca.InStage")) assert( errors.isEmpty, s"the load-bearing shared InStage fan-out capture must compile, got: $errors" ) test("(b) NEGATIVE: capturing WorkspaceWrite (exclusive) fails to compile"): - val errors = compileErrorsOf(fixture("orca.WorkspaceWrite")) + val errors = Dotc.errorsOf(fixture("orca.WorkspaceWrite")) assertSeparationFailure("WorkspaceWrite", errors) test("(c) NEGATIVE: capturing FlowControl (exclusive) fails to compile"): - val errors = compileErrorsOf(fixture("orca.FlowControl")) + val errors = Dotc.errorsOf(fixture("orca.FlowControl")) assertSeparationFailure("FlowControl", errors) /** Assert the fan-out was rejected specifically by separation checking — not @@ -179,11 +112,11 @@ class CcNegativeCompileTest extends munit.FunSuite: "(d) NEGATIVE: a ReviewerSelector whose returned arrow captures `ev` (InStage) fails to compile" ): val errors = - compileErrorsOf(selectorFixture("history => { val _ = ev; all }")) + Dotc.errorsOf(selectorFixture("history => { val _ = ev; all }")) assertCaptureEscape(errors) test("(e) POSITIVE: a pure history-narrowing ReviewerSelector compiles"): - val errors = compileErrorsOf( + val errors = Dotc.errorsOf( selectorFixture( "history => history.headOption match { case None => all; " + "case Some(batch) => batch.reviewersWithFindings }" @@ -236,7 +169,7 @@ class CcNegativeCompileTest extends munit.FunSuite: test( "(f) POSITIVE: a fan-out capturing FlowContext beside a FlowControl compiles" ): - val errors = compileErrorsOf(contextFixture) + val errors = Dotc.errorsOf(contextFixture) assert( errors.isEmpty, s"the fan-out must capture only the FlowContext, got: $errors" diff --git a/flow/src/test/scala/orca/CommitMessageTest.scala b/flow/src/test/scala/orca/CommitMessageTest.scala index dfe796e10..be87dafc8 100644 --- a/flow/src/test/scala/orca/CommitMessageTest.scala +++ b/flow/src/test/scala/orca/CommitMessageTest.scala @@ -2,7 +2,7 @@ package orca import orca.events.EventDispatcher import orca.agents.{Agent, BackendTag} -import orca.testkit.{ScriptedBackend, TestAgent, TextReplyingAgent} +import orca.testkit.{Effects, ScriptedBackend, TestAgent, TextReplyingAgent} import ox.discard import java.util.concurrent.ConcurrentLinkedQueue @@ -141,7 +141,7 @@ class CommitMessageTest extends munit.FunSuite: // A mid-stage event-log append, to pin that the stage diff carries // nothing from `.orca/` — the log the first stage committed included. val key = RunKey.of("p") - OrcaDir.ensureRunDir(run.dir, key).discard + Effects.orcaDir.ensureRunDir(run.dir, key).discard os.write.append(OrcaDir.eventLogPath(run.dir, key), "{}\n") "done" val _ = nextPrompt(prompts) diff --git a/flow/src/test/scala/orca/CostTrackerTest.scala b/flow/src/test/scala/orca/CostTrackerTest.scala index 4e07898e9..ef169cb46 100644 --- a/flow/src/test/scala/orca/CostTrackerTest.scala +++ b/flow/src/test/scala/orca/CostTrackerTest.scala @@ -667,9 +667,8 @@ class CostTrackerTest extends munit.FunSuite: assertEquals((new CostTracker).summary, "") test("concurrent onEvent calls from two threads keep every tally"): - // The reviewer fan-out emits from parallel forks. `AtomicReference`'s - // compare-and-set retry is what makes that safe; a get-then-set refactor - // would drop tallies. + // The reviewer fan-out emits from parallel forks. `Ref.update`'s retry + // is what makes that safe; a get-then-set refactor would drop tallies. val tracker = new CostTracker supervised: val emitters = (0 until 2).map: t => diff --git a/flow/src/test/scala/orca/FlowSessionTest.scala b/flow/src/test/scala/orca/FlowSessionTest.scala index 659500e7d..d6ba05d23 100644 --- a/flow/src/test/scala/orca/FlowSessionTest.scala +++ b/flow/src/test/scala/orca/FlowSessionTest.scala @@ -23,6 +23,7 @@ import orca.json.{JsonData, RawJson} import orca.progress.{BranchMode, ProgressHeader, ProgressStore, StageEntry} import orca.sessions.{SessionRecord, TestSessionStore} import orca.testkit.{ + Effects, GitRepo, PassthroughPrompts, ScriptedBackend, @@ -224,7 +225,7 @@ class FlowSessionTest extends FunSuite: listeners: List[orca.events.OrcaListener] = Nil ): TestRun = val dir = TempDirs.dir() - val store = ProgressStore.default(dir, RunKey.of("p")) + val store = ProgressStore.default(Effects.fs, dir, RunKey.of("p")) val sessionStore = new TestSessionStore given WorkspaceWrite = WorkspaceWrite.unsafe store.writeHeader( @@ -246,7 +247,7 @@ class FlowSessionTest extends FunSuite: resultJson = RawJson("null") ) ) - val git = new orca.tools.OsGitTool(dir) + val git = orca.testkit.Effects.gitTool(dir) TestRun( new TestFlowControl(store, sessionStore), new TestFlowContext( @@ -748,7 +749,7 @@ class FlowSessionTest extends FunSuite: // A real repo, unlike `makeRun`'s bare temp dir, so `headCommit()` has // something to report. val dir = GitRepo.seeded() - val store = ProgressStore.default(dir, RunKey.of("p")) + val store = ProgressStore.default(Effects.fs, dir, RunKey.of("p")) val sessionStore = new TestSessionStore store.writeHeader( ProgressHeader( @@ -777,7 +778,7 @@ class FlowSessionTest extends FunSuite: resultJson = RawJson("null") ) ) - val git = new orca.tools.OsGitTool(dir) + val git = orca.testkit.Effects.gitTool(dir) val run = TestRun( new TestFlowControl(store, sessionStore), new TestFlowContext( diff --git a/flow/src/test/scala/orca/NoTaintCanaryTest.scala b/flow/src/test/scala/orca/NoTaintCanaryTest.scala index 6abe0b769..cdcdf6581 100644 --- a/flow/src/test/scala/orca/NoTaintCanaryTest.scala +++ b/flow/src/test/scala/orca/NoTaintCanaryTest.scala @@ -7,15 +7,15 @@ import orca.progress.ProgressStore * * [[InStage]] extends `caps.SharedCapability` (non-experimental), while * [[WorkspaceWrite]] and [[FlowControl]] extend `caps.ExclusiveCapability` — - * which IS `@experimental` on 3.9.0 — behind a `captureChecking` language + * which IS `@experimental` on 3.10.0 — behind a `captureChecking` language * import in their OWN files. This file carries no such import and is not * `@experimental`, yet it freely mints, summons, and passes all three tokens. * * If a future compiler propagated `@experimental` from the capture-checking * parents into these definitions, referencing them from this plain, non-CC * compilation unit would fail with "... is marked @experimental", turning this - * test RED. That taint cannot happen on stable 3.9.0; this is the explicit, - * named insurance for it. + * test RED. That taint cannot happen on 3.10.0; this is the explicit, named + * insurance for it. */ class NoTaintCanaryTest extends munit.FunSuite: diff --git a/flow/src/test/scala/orca/SessionTest.scala b/flow/src/test/scala/orca/SessionTest.scala index 4f467a6f6..77f715368 100644 --- a/flow/src/test/scala/orca/SessionTest.scala +++ b/flow/src/test/scala/orca/SessionTest.scala @@ -6,8 +6,7 @@ import orca.agents.{Agent, BackendTag, SessionId, WireSessionId} import orca.backend.{Dispatch, IdScheme, ResumeOrigin, SessionSupport} import orca.runlog.TestRunLog import orca.sessions.SessionRecord -import orca.tools.OsGitTool -import orca.testkit.{ScriptedBackend, TempDirs, TestAgent} +import orca.testkit.{Effects, ScriptedBackend, TempDirs, TestAgent} /** Tests for `agent.session(name, seed)` get-or-create, keyed by the name and * the stage that minted it (ADR 0018 §2.6). @@ -62,10 +61,11 @@ class SessionTest extends FunSuite: new TestFlowContext( new EventDispatcher(listeners), "p", - wiredGit = Some(new OsGitTool(dir)) + wiredGit = Some(Effects.gitTool(dir)) ), new TestFlowControl( - orca.progress.ProgressStore.default(dir, RunKey.of("p")), + orca.progress.ProgressStore + .default(Effects.fs, dir, RunKey.of("p")), log ) ) @@ -495,9 +495,11 @@ class SessionTest extends FunSuite: agent.session("planner", seed = "brief").chat.id.value stage[String]("Implement", commitMessage): throw new RuntimeException(mint()) - new OsGitTool(run.dir).discardUncommitted(orca.tools.UntrackedFiles.Remove)( - using WorkspaceWrite.unsafe - ) + Effects + .gitTool(run.dir) + .discardUncommitted(orca.tools.UntrackedFiles.Remove)(using + WorkspaceWrite.unsafe + ) val reMinted = inRun(run.dir): stage("Implement", commitMessage)(mint()) @@ -514,9 +516,11 @@ class SessionTest extends FunSuite: val minted = inRun(run.dir)(agent.session("implementer", seed = "brief").chat.id.value) os.write.over(run.dir / "dirty.txt", "uncommitted") - new OsGitTool(run.dir).ensureClean("orca: pre-resume")(using - WorkspaceWrite.unsafe - ) + Effects + .gitTool(run.dir) + .ensureClean("orca: pre-resume")(using + WorkspaceWrite.unsafe + ) assertEquals(records(run.dir).map(_.id), List(minted)) private def uncommitted(dir: os.Path): String = diff --git a/flow/src/test/scala/orca/StageFramesTest.scala b/flow/src/test/scala/orca/StageFramesTest.scala index 7c1046183..bd32683e8 100644 --- a/flow/src/test/scala/orca/StageFramesTest.scala +++ b/flow/src/test/scala/orca/StageFramesTest.scala @@ -50,3 +50,13 @@ class StageFramesTest extends munit.FunSuite: import run.given val result = stage("same-thread")("ok") assertEquals(result, "ok") + + test("a stage whose base commit throws still consumes its occurrence"): + val run = TestRun.create(new EventDispatcher(Nil)) + val failing = intercept[RuntimeException]: + run.control.withStage("s", _ => throw new RuntimeException("no head"))( + _ => () + ) + assertEquals(failing.getMessage, "no head") + val next = run.control.withStage("s", _ => None)(identity) + assertEquals(next.occurrence, 1) diff --git a/flow/src/test/scala/orca/StageRuntimeTest.scala b/flow/src/test/scala/orca/StageRuntimeTest.scala index 4c4216546..b042e40ff 100644 --- a/flow/src/test/scala/orca/StageRuntimeTest.scala +++ b/flow/src/test/scala/orca/StageRuntimeTest.scala @@ -1,6 +1,6 @@ package orca -import orca.testkit.TextReplyingAgent +import orca.testkit.{Effects, TextReplyingAgent} import orca.json.RawJson import orca.events.{EventDispatcher, OrcaEvent, OrcaListener, StageOutcome} @@ -472,8 +472,12 @@ class StageRuntimeTest extends munit.FunSuite: dir: os.Path, listener: OrcaListener ): TestRun = - val git = new orca.tools.OsGitTool(dir) - val store = orca.progress.ProgressStore.default(dir, RunKey.of("p")) + val git = orca.testkit.Effects.gitTool(dir) + val store = orca.progress.ProgressStore.default( + Effects.fs, + dir, + RunKey.of("p") + ) TestRun( new TestFlowControl(store, new orca.sessions.TestSessionStore), new TestFlowContext( diff --git a/flow/src/test/scala/orca/TestFlowContext.scala b/flow/src/test/scala/orca/TestFlowContext.scala index a55a774f1..4a2a6ff93 100644 --- a/flow/src/test/scala/orca/TestFlowContext.scala +++ b/flow/src/test/scala/orca/TestFlowContext.scala @@ -1,5 +1,7 @@ package orca +import orca.testkit.Effects + import orca.events.{EventDispatcher, OrcaEvent} import orca.agents.{ Agent, @@ -45,5 +47,10 @@ class TestFlowContext( wiredGit.getOrElse(stub("git")) lazy val gh: GitHubTool = wiredGh.getOrElse(stub("gh")) lazy val fs: FsTool = stub("fs") + private[orca] lazy val gateShell: orca.review.GateShell = + orca.review.OsGateShell( + Effects.procs, + Effects.fs + ) private[orca] def emit(event: OrcaEvent): Unit = dispatcher.onEvent(event) diff --git a/flow/src/test/scala/orca/TestRun.scala b/flow/src/test/scala/orca/TestRun.scala index 990f60365..ddb5554d5 100644 --- a/flow/src/test/scala/orca/TestRun.scala +++ b/flow/src/test/scala/orca/TestRun.scala @@ -7,8 +7,7 @@ import orca.events.EventDispatcher import orca.progress.{BranchMode, ProgressHeader, ProgressStore} import orca.review.ReviewerCatalog import orca.sessions.TestSessionStore -import orca.testkit.GitRepo -import orca.tools.OsGitTool +import orca.testkit.{Effects, GitRepo} /** A test run's control and context, provided as givens by `import run.given`, * over the repo at `dir`. @@ -36,9 +35,9 @@ object TestRun: startingCommitUsable: Boolean = true ): TestRun = val dir = GitRepo.seeded() - val git = new OsGitTool(dir) + val git = Effects.gitTool(dir) val runKey = RunKey.of(userPrompt) - val store = ProgressStore.default(dir, runKey) + val store = ProgressStore.default(Effects.fs, dir, runKey) given WorkspaceWrite = WorkspaceWrite.unsafe // The seed commit stands in for the commit a real run binds at. val headCommit = git.headCommit().get diff --git a/flow/src/test/scala/orca/plan/CannedResult.scala b/flow/src/test/scala/orca/plan/CannedResult.scala index 9ae30d1ed..2ed767109 100644 --- a/flow/src/test/scala/orca/plan/CannedResult.scala +++ b/flow/src/test/scala/orca/plan/CannedResult.scala @@ -38,5 +38,4 @@ private[plan] class CannedResult[T: JsonData](value: T): tools = turn.config.tools ) ) - ScriptedBackend.json(value) - ) + ScriptedBackend.json(value)) diff --git a/flow/src/test/scala/orca/plan/PlanSchemaParityTest.scala b/flow/src/test/scala/orca/plan/PlanSchemaParityTest.scala new file mode 100644 index 000000000..58619dd26 --- /dev/null +++ b/flow/src/test/scala/orca/plan/PlanSchemaParityTest.scala @@ -0,0 +1,14 @@ +package orca.plan + +import orca.json.given + +@scala.annotation.experimental +class PlanSchemaParityTest extends orca.json.SchemaParitySuite: + parity[BugReportMatch] + parity[Plan] + parity[Epic] + parity[Roadmap] + parity[Task] + parity[TriageReply] + parity[Triage] + parity[Triage.Kind] diff --git a/flow/src/test/scala/orca/pr/PrFakes.scala b/flow/src/test/scala/orca/pr/PrFakes.scala index e7780c87b..e24d62324 100644 --- a/flow/src/test/scala/orca/pr/PrFakes.scala +++ b/flow/src/test/scala/orca/pr/PrFakes.scala @@ -7,7 +7,6 @@ import orca.agents.{Agent, BackendTag} import orca.tools.{ GitHubAvailability, NoDefaultBase, - OsGitTool, PrCreateFailed, PrHandle, PrReadiness, @@ -17,6 +16,7 @@ import orca.tools.{ import orca.progress.{BranchMode, ProgressHeader, ProgressStore} import orca.sessions.TestSessionStore import orca.testkit.{ + Effects, GitRepo, PassthroughPrompts, PushlessGit, @@ -128,7 +128,7 @@ private[pr] def seededPrRepo( os.write(dir / "code.txt", "real code") val _ = os.proc("git", "add", "code.txt").call(cwd = dir) val _ = os.proc("git", "commit", "-m", "work").call(cwd = dir) - val store = ProgressStore.default(dir, RunKey.of("p")) + val store = ProgressStore.default(Effects.fs, dir, RunKey.of("p")) given WorkspaceWrite = WorkspaceWrite.unsafe val startingBranch = branchMode match case BranchMode.Created => startBranch @@ -176,7 +176,7 @@ private[pr] def prRun( new EventDispatcher(List(listener)), "p", wiredGit = Some( - new RecordingGit(new OsGitTool(dir), calls, branchDiff, push, base) + new RecordingGit(Effects.gitTool(dir), calls, branchDiff, push, base) ), wiredGh = Some( new RecordingGh(calls, availability, createPr, prBodies, prReadinesses) diff --git a/flow/src/test/scala/orca/pr/PrSchemaParityTest.scala b/flow/src/test/scala/orca/pr/PrSchemaParityTest.scala new file mode 100644 index 000000000..5992d8c47 --- /dev/null +++ b/flow/src/test/scala/orca/pr/PrSchemaParityTest.scala @@ -0,0 +1,9 @@ +package orca.pr + +import orca.json.given + +@scala.annotation.experimental +class PrSchemaParityTest extends orca.json.SchemaParitySuite: + parity[PushResult] + parity[CreateResult] + parity[PrSummary] diff --git a/flow/src/test/scala/orca/progress/ProgressLogTest.scala b/flow/src/test/scala/orca/progress/ProgressLogTest.scala index 71128e950..4e8493b1b 100644 --- a/flow/src/test/scala/orca/progress/ProgressLogTest.scala +++ b/flow/src/test/scala/orca/progress/ProgressLogTest.scala @@ -87,9 +87,10 @@ class ProgressLogTest extends FunSuite: test("a branch that isn't a valid branch name fails to decode"): // Hand-editable like the commit below; only a valid name may reach git. val codec = summon[JsonData[ProgressLog]].codec - val json = writeToString(ProgressLog(header("feat/x"), Nil, None, Nil))( - using codec - ).replace("\"feat/x\"", "\"HEAD\"") + val json = + writeToString(ProgressLog(header("feat/x"), Nil, None, Nil))(using + codec + ).replace("\"feat/x\"", "\"HEAD\"") intercept[JsonReaderException]( readFromString[ProgressLog](json)(using codec @@ -100,9 +101,10 @@ class ProgressLogTest extends FunSuite: // The header is committed, hand-editable content; only a hash may reach // git as a diff base, so anything else fails the whole log. val codec = summon[JsonData[ProgressLog]].codec - val json = writeToString(ProgressLog(header("feat/x"), Nil, None, Nil))( - using codec - ).replace("0badc0ffee0ddf00d1234567890abcdef1234567", "--output=/etc/pw") + val json = + writeToString(ProgressLog(header("feat/x"), Nil, None, Nil))(using + codec + ).replace("0badc0ffee0ddf00d1234567890abcdef1234567", "--output=/etc/pw") intercept[JsonReaderException]( readFromString[ProgressLog](json)(using codec @@ -111,9 +113,10 @@ class ProgressLogTest extends FunSuite: test("a header missing userPrompt fails to decode"): val codec = summon[JsonData[ProgressLog]].codec - val json = writeToString(ProgressLog(header("feat/x"), Nil, None, Nil))( - using codec - ).replace(""""userPrompt":"fix the flaky test",""", "") + val json = + writeToString(ProgressLog(header("feat/x"), Nil, None, Nil))(using + codec + ).replace(""""userPrompt":"fix the flaky test",""", "") intercept[JsonReaderException]( readFromString[ProgressLog](json)(using codec diff --git a/flow/src/test/scala/orca/progress/ProgressScanTest.scala b/flow/src/test/scala/orca/progress/ProgressScanTest.scala index 4f142feaa..4a60f5744 100644 --- a/flow/src/test/scala/orca/progress/ProgressScanTest.scala +++ b/flow/src/test/scala/orca/progress/ProgressScanTest.scala @@ -5,7 +5,7 @@ import orca.tools.PrReadiness import munit.FunSuite import orca.{RunKey, WorkspaceWrite} import orca.gitref.CommitHash -import orca.testkit.{TempDirs, branchName} +import orca.testkit.{Effects, TempDirs, branchName} class ProgressScanTest extends FunSuite: @@ -24,11 +24,16 @@ class ProgressScanTest extends FunSuite: ) test("progressLogPaths is empty when .orca doesn't exist"): - assertEquals(ProgressScan.progressLogPaths(TempDirs.dir()), Nil) + assertEquals( + ProgressScan(Effects.fs) + .progressLogPaths(TempDirs.dir()), + Nil + ) test("progressLogPaths lists only *.progress.json files under .orca/runs"): val workDir = TempDirs.dir() - val store = ProgressStore.default(workDir, RunKey.of("my prompt")) + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of("my prompt")) store.writeHeader(header) List( "abc123def456.other.json", // another document @@ -36,62 +41,92 @@ class ProgressScanTest extends FunSuite: ".abc123def456.progress.json.1.tmp" // an in-flight temp file ).foreach(name => os.write(workDir / ".orca" / "runs" / name, "{}")) os.write(workDir / ".orca" / "settings.properties", "") - assertEquals(ProgressScan.progressLogPaths(workDir), List(store.path)) + assertEquals( + ProgressScan(Effects.fs).progressLogPaths(workDir), + List(store.path) + ) test("progressLogPaths excludes a symlinked log file"): val workDir = TempDirs.dir() - val store = ProgressStore.default(workDir, RunKey.of("my prompt")) + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of("my prompt")) store.writeHeader(header) val outside = TempDirs.dir() / "elsewhere.json" os.write(outside, "{}") os.remove(store.path): Unit os.symlink(store.path, outside) - assertEquals(ProgressScan.progressLogPaths(workDir), Nil) + assertEquals( + ProgressScan(Effects.fs).progressLogPaths(workDir), + Nil + ) test("progressLogPaths is empty when .orca itself is a symlink"): val workDir = TempDirs.dir() val real = TempDirs.dir() - ProgressStore.default(real, RunKey.of("my prompt")).writeHeader(header) + ProgressStore + .default(Effects.fs, real, RunKey.of("my prompt")) + .writeHeader(header) os.symlink(workDir / ".orca", real / ".orca") - assertEquals(ProgressScan.progressLogPaths(workDir), Nil) + assertEquals( + ProgressScan(Effects.fs).progressLogPaths(workDir), + Nil + ) test("progressLogPaths is empty when .orca/runs is a symlink"): val workDir = TempDirs.dir() val real = TempDirs.dir() - ProgressStore.default(real, RunKey.of("my prompt")).writeHeader(header) + ProgressStore + .default(Effects.fs, real, RunKey.of("my prompt")) + .writeHeader(header) os.makeDir.all(workDir / ".orca") os.symlink(workDir / ".orca" / "runs", real / ".orca" / "runs") - assertEquals(ProgressScan.progressLogPaths(workDir), Nil) + assertEquals( + ProgressScan(Effects.fs).progressLogPaths(workDir), + Nil + ) test("progressLogPaths skips a directory named like a log"): // Only files are candidates; a same-named directory must cost itself, not // the readable log beside it (an empty scan reads as "no runs in flight"). val workDir = TempDirs.dir() - val store = ProgressStore.default(workDir, RunKey.of("my prompt")) + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of("my prompt")) store.writeHeader(header) os.makeDir.all(workDir / ".orca" / "runs" / "000000000000.progress.json") - assertEquals(ProgressScan.progressLogPaths(workDir), List(store.path)) + assertEquals( + ProgressScan(Effects.fs).progressLogPaths(workDir), + List(store.path) + ) test("progressLogs pairs every readable log with its header"): val workDir = TempDirs.dir() - val store = ProgressStore.default(workDir, RunKey.of("my prompt")) + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of("my prompt")) store.writeHeader(header) - val corrupt = ProgressStore.default(workDir, RunKey.of("corrupt prompt")) + val corrupt = ProgressStore.default( + Effects.fs, + workDir, + RunKey.of("corrupt prompt") + ) corrupt.writeHeader(header) os.write.over(corrupt.path, "not json {{{") assertEquals( - ProgressScan.progressLogs(workDir), + ProgressScan(Effects.fs).progressLogs(workDir), List(ScannedProgressLog(store.path, header)) ) test("progressLogPaths lists every log, not just one"): val workDir = TempDirs.dir() - val one = ProgressStore.default(workDir, RunKey.of("one")) + val one = + ProgressStore.default(Effects.fs, workDir, RunKey.of("one")) one.writeHeader(header) - val two = ProgressStore.default(workDir, RunKey.of("two")) + val two = + ProgressStore.default(Effects.fs, workDir, RunKey.of("two")) two.writeHeader(header) // As a Set: `os.list` order is unspecified, and the scan promises none. assertEquals( - ProgressScan.progressLogPaths(workDir).toSet, + ProgressScan(Effects.fs) + .progressLogPaths(workDir) + .toSet, Set(one.path, two.path) ) diff --git a/flow/src/test/scala/orca/progress/ProgressSchemaParityTest.scala b/flow/src/test/scala/orca/progress/ProgressSchemaParityTest.scala new file mode 100644 index 000000000..8dfae4848 --- /dev/null +++ b/flow/src/test/scala/orca/progress/ProgressSchemaParityTest.scala @@ -0,0 +1,12 @@ +package orca.progress + +import orca.json.given + +@scala.annotation.experimental +class ProgressSchemaParityTest extends orca.json.SchemaParitySuite: + parity[ProgressHeader] + parity[StageEntry] + parity[StageStart] + parity[ProgressLog] + parity[FlowSource] + parity[PublishedWork] diff --git a/flow/src/test/scala/orca/progress/ProgressStoreTest.scala b/flow/src/test/scala/orca/progress/ProgressStoreTest.scala index c4a973214..50add7b29 100644 --- a/flow/src/test/scala/orca/progress/ProgressStoreTest.scala +++ b/flow/src/test/scala/orca/progress/ProgressStoreTest.scala @@ -6,7 +6,7 @@ import munit.FunSuite import orca.{RunKey, StagePath, WorkspaceWrite} import orca.gitref.CommitHash import orca.json.RawJson -import orca.testkit.{TempDirs, branchName} +import orca.testkit.{Effects, TempDirs, branchName} class ProgressStoreTest extends FunSuite: @@ -25,7 +25,8 @@ class ProgressStoreTest extends FunSuite: test("writeHeader then load returns the header with empty entries"): val workDir = TempDirs.dir() - val store = ProgressStore.default(workDir, RunKey.of("my prompt")) + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of("my prompt")) store.writeHeader(header) val loaded = store.load() assertEquals(loaded, Some(ProgressLog(header, Nil, None, Nil))) @@ -34,7 +35,8 @@ class ProgressStoreTest extends FunSuite: "upsertEntry with same id replaces (last write wins), different id appends" ): val workDir = TempDirs.dir() - val store = ProgressStore.default(workDir, RunKey.of("my prompt")) + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of("my prompt")) store.writeHeader(header) val a = @@ -62,7 +64,8 @@ class ProgressStoreTest extends FunSuite: test("recordStageStart replaces the base recorded for a stage"): val workDir = TempDirs.dir() - val store = ProgressStore.default(workDir, RunKey.of("my prompt")) + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of("my prompt")) store.writeHeader(header) val id = StagePath.FlowBody.child("s", 0) val commitA = CommitHash.from("a" * 40).get @@ -76,7 +79,8 @@ class ProgressStoreTest extends FunSuite: test("upsertEntry drops the completed stage's recorded start"): val workDir = TempDirs.dir() - val store = ProgressStore.default(workDir, RunKey.of("my prompt")) + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of("my prompt")) store.writeHeader(header) val done = StagePath.FlowBody.child("done", 0) val open = StageStart( @@ -93,7 +97,8 @@ class ProgressStoreTest extends FunSuite: // subtree (`"resultJson":{…}`), not an escaped string blob — keeping the // on-disk log directly readable when debugging. val workDir = TempDirs.dir() - val store = ProgressStore.default(workDir, RunKey.of("my prompt")) + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of("my prompt")) store.writeHeader(header) store.upsertEntry( StageEntry( @@ -106,14 +111,16 @@ class ProgressStoreTest extends FunSuite: test("load collapses a corrupt file to None"): val workDir = TempDirs.dir() - val store = ProgressStore.default(workDir, RunKey.of("my prompt")) + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of("my prompt")) store.writeHeader(header) os.write.over(store.path, "not json {{{") assertEquals(store.load(), None: Option[ProgressLog]) test("upsertEntry before writeHeader throws the absent-log message"): val workDir = TempDirs.dir() - val store = ProgressStore.default(workDir, RunKey.of("my prompt")) + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of("my prompt")) val ex = intercept[IllegalStateException]: store.upsertEntry( StageEntry( @@ -133,7 +140,8 @@ class ProgressStoreTest extends FunSuite: // as "upsertEntry called before writeHeader" — that message is wrong when // writeHeader plainly did run (the file exists). val workDir = TempDirs.dir() - val store = ProgressStore.default(workDir, RunKey.of("my prompt")) + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of("my prompt")) store.writeHeader(header) os.write.over(store.path, "not json {{{") val ex = intercept[IllegalStateException]: @@ -156,7 +164,8 @@ class ProgressStoreTest extends FunSuite: test("restoreIfRemoved puts back a peeked log whose directory is gone"): // A stash of the only file in `.orca/runs/` takes the directory with it. val workDir = TempDirs.dir() - val store = ProgressStore.default(workDir, RunKey.of("my prompt")) + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of("my prompt")) store.writeHeader(header) val peeked = store.peek().fold(fail(_), identity) os.remove.all(store.path / os.up) @@ -165,7 +174,8 @@ class ProgressStoreTest extends FunSuite: test("restoreIfRemoved leaves a log that is still there untouched"): val workDir = TempDirs.dir() - val store = ProgressStore.default(workDir, RunKey.of("my prompt")) + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of("my prompt")) store.writeHeader(header) val peeked = store.peek().fold(fail(_), identity) os.write.over(store.path, "edited since") @@ -174,7 +184,8 @@ class ProgressStoreTest extends FunSuite: test("restoreIfRemoved puts back an unparseable log too"): val workDir = TempDirs.dir() - val store = ProgressStore.default(workDir, RunKey.of("my prompt")) + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of("my prompt")) os.write(store.path, "not json", createFolders = true) val peeked = store.peek().fold(fail(_), identity) assert(peeked.isInstanceOf[PeekedLog.Unparseable], peeked) diff --git a/flow/src/test/scala/orca/review/GateShellTest.scala b/flow/src/test/scala/orca/review/GateShellTest.scala new file mode 100644 index 000000000..6ad1e1f8c --- /dev/null +++ b/flow/src/test/scala/orca/review/GateShellTest.scala @@ -0,0 +1,10 @@ +package orca.review + +import orca.testkit.{Effects, TempDirs} + +class GateShellTest extends munit.FunSuite: + + test("run merges stderr into stdout"): + val result = OsGateShell(Effects.procs, Effects.fs) + .run("echo out; echo err >&2", TempDirs.dir()) + assertEquals((result.stdout, result.stderr), ("out\nerr\n", "")) diff --git a/flow/src/test/scala/orca/review/ReviewLoopFixture.scala b/flow/src/test/scala/orca/review/ReviewLoopFixture.scala index b07de1aab..15b91323c 100644 --- a/flow/src/test/scala/orca/review/ReviewLoopFixture.scala +++ b/flow/src/test/scala/orca/review/ReviewLoopFixture.scala @@ -141,7 +141,7 @@ private[review] class FakeAgent( onRun() Option(remaining.poll()) match case Some(r) => ScriptedBackend.result(r.json) - case None => + case None => throw new AgentTurnFailed( s"$name: no reply scripted", TurnDebit.Unobserved diff --git a/flow/src/test/scala/orca/review/ReviewSchemaParityTest.scala b/flow/src/test/scala/orca/review/ReviewSchemaParityTest.scala new file mode 100644 index 000000000..93c761fc4 --- /dev/null +++ b/flow/src/test/scala/orca/review/ReviewSchemaParityTest.scala @@ -0,0 +1,25 @@ +package orca.review + +import orca.json.{JsonData, given} + +@scala.annotation.experimental +class ReviewSchemaParityTest extends orca.json.SchemaParitySuite: + parity[Location] + parity[ReviewFinding] + parity[OpenReason] + parity[OpenFinding] + parity[OpenFindings] + parity[ReviewerFindings] + parity[ReviewReport] + parity[ReviewResult] + parity[FixOutcome] + parity[DeclinedFinding] + parity[ReviewerInfo] + parity[ReviewerSelectionRequest] + // The given adds a validator on top of the derivation. + parity[SelectedReviewers](using + JsonData.derived[SelectedReviewers].schema, + summon[scala.deriving.Mirror.Of[SelectedReviewers]] + ) + parity[FixRequest] + parity[KeyedFinding] diff --git a/flow/src/test/scala/orca/review/ReviewerCatalogTest.scala b/flow/src/test/scala/orca/review/ReviewerCatalogTest.scala index b5b8cdeb7..e9e41d7d3 100644 --- a/flow/src/test/scala/orca/review/ReviewerCatalogTest.scala +++ b/flow/src/test/scala/orca/review/ReviewerCatalogTest.scala @@ -2,7 +2,7 @@ package orca.review import orca.{OrcaDir, OrcaFlowException} import orca.discovery.Origin -import orca.testkit.{RepoRoot, TempDirs} +import orca.testkit.{Effects, RepoRoot, TempDirs} class ReviewerCatalogTest extends munit.FunSuite: @@ -34,6 +34,9 @@ class ReviewerCatalogTest extends munit.FunSuite: assume(!os.exists(dir / "PROBE"), s"$dir is case-insensitive") val _ = os.remove(dir / "probe") + private def discover(project: os.Path, global: os.Path): ReviewerCatalog = + ReviewerCatalog.discover(Effects.fs, project, global) + private def named(reviewers: List[Reviewer]): List[String] = reviewers.map(_.name.value) @@ -42,7 +45,7 @@ class ReviewerCatalogTest extends munit.FunSuite: // directory fails here rather than quietly finding nothing. val projectDir = OrcaDir.reviewersPath(RepoRoot.dir) val catalog = - ReviewerCatalog.discover(projectDir, TempDirs.dir() / "absent") + discover(projectDir, TempDirs.dir() / "absent") val reviewer = catalog.all .find(_.name.value == "orca") .getOrElse(fail(s"no reviewer named 'orca' in $projectDir")) @@ -65,7 +68,7 @@ class ReviewerCatalogTest extends munit.FunSuite: val (project, global) = dirs() val _ = writeReviewer(project, "zeta") val _ = writeReviewer(project, "alpha") - val catalog = ReviewerCatalog.discover(project, global) + val catalog = discover(project, global) assertEquals( named(catalog.all), named(ReviewerPrompts.all) ++ List("alpha", "zeta") @@ -83,7 +86,7 @@ class ReviewerCatalogTest extends munit.FunSuite: description = "our own scala rules", body = "## Our scope" ) - val catalog = ReviewerCatalog.discover(project, global) + val catalog = discover(project, global) assertEquals(named(catalog.all), named(ReviewerPrompts.all)) // `scala-fp` is outside `minimal`, so a shadow of it adds nothing there. assertEquals(named(catalog.minimal), named(ReviewerPrompts.minimal)) @@ -101,7 +104,7 @@ class ReviewerCatalogTest extends munit.FunSuite: // stayed additive would run beside the reviewer it meant to replace. val (project, global) = dirs() val _ = writeReviewer(project, "Scala-FP", description = "ours") - val catalog = ReviewerCatalog.discover(project, global) + val catalog = discover(project, global) assertEquals(named(catalog.all), named(ReviewerPrompts.all)) assertEquals( catalog.all.find(_.name.value == "scala-fp").get.description, @@ -120,7 +123,7 @@ class ReviewerCatalogTest extends munit.FunSuite: files = Some("""\.scala$"""), body = "## Our scope" ) - val orca = ReviewerCatalog.discover(project, global).all.last + val orca = discover(project, global).all.last assertEquals(orca.systemPrompt, "## Our scope\n") assertEquals(orca.filePattern.map(_.regex), Some("""\.scala$""")) @@ -128,7 +131,7 @@ class ReviewerCatalogTest extends munit.FunSuite: val (project, global) = dirs() val _ = writeReviewer(project, "orca", description = "the project's") val _ = writeReviewer(global, "orca", description = "the user's") - val catalog = ReviewerCatalog.discover(project, global) + val catalog = discover(project, global) assertEquals( catalog.all.find(_.name.value == "orca").get.description, "the project's" @@ -141,33 +144,39 @@ class ReviewerCatalogTest extends munit.FunSuite: test("a global reviewer is picked up when the project has none"): val (project, global) = dirs() val _ = writeReviewer(global, "orca", description = "the user's") - val catalog = ReviewerCatalog.discover(project, global) + val catalog = discover(project, global) assertEquals(named(catalog.all), named(ReviewerPrompts.all) :+ "orca") assertEquals(catalog.discovered.map(_.tier), List(ReviewerFileTier.Global)) test("a CRLF reviewer file is read like any other"): val (project, global) = dirs() val _ = writeReviewer(project, "orca", newline = "\r\n") - val catalog = ReviewerCatalog.discover(project, global) + val catalog = discover(project, global) assertEquals(named(catalog.all), named(ReviewerPrompts.all) :+ "orca") test("a missing tier directory contributes nothing"): val (project, global) = dirs() - val catalog = ReviewerCatalog.discover(project, global) + val catalog = discover(project, global) assertEquals(catalog.all, ReviewerPrompts.all) assertEquals(catalog.discovered, Nil) test("a file that is not .md is ignored"): val (project, global) = dirs() os.write(project / "notes.txt", "not a reviewer", createFolders = true) - assertEquals(ReviewerCatalog.discover(project, global).discovered, Nil) + assertEquals( + discover(project, global).discovered, + Nil + ) test("README.md and _-prefixed files sit in the directory as documents"): val (project, global) = dirs() val doc = "# Reviewers\n\nDrop a reviewer prompt here.\n" os.write(project / "README.md", doc, createFolders = true) os.write(project / "_draft.md", doc, createFolders = true) - assertEquals(ReviewerCatalog.discover(project, global).discovered, Nil) + assertEquals( + discover(project, global).discovered, + Nil + ) test("any other .md with no frontmatter aborts, not silently skipped"): // Forgetting the frontmatter block is the likely authoring mistake; the @@ -179,7 +188,9 @@ class ReviewerCatalogTest extends munit.FunSuite: createFolders = true ) val e = - intercept[OrcaFlowException](ReviewerCatalog.discover(project, global)) + intercept[OrcaFlowException]( + discover(project, global) + ) assert(e.getMessage.contains("has no frontmatter block"), e.getMessage) test("a symlink is reported alongside a malformed file"): @@ -187,7 +198,9 @@ class ReviewerCatalogTest extends munit.FunSuite: val _ = writeReviewer(project, "alpha", description = "") os.symlink(project / "linked.md", writeReviewer(global, "elsewhere")) val e = - intercept[OrcaFlowException](ReviewerCatalog.discover(project, global)) + intercept[OrcaFlowException]( + discover(project, global) + ) assert(e.getMessage.contains("alpha"), e.getMessage) assert(e.getMessage.contains("linked.md is a symlink"), e.getMessage) @@ -196,7 +209,9 @@ class ReviewerCatalogTest extends munit.FunSuite: os.makeDir.all(global) os.symlink(global / "orca.md", global / "gone.md") val e = - intercept[OrcaFlowException](ReviewerCatalog.discover(project, global)) + intercept[OrcaFlowException]( + discover(project, global) + ) assert(e.getMessage.contains("cannot be read"), e.getMessage) test("a symlinked .md in the project tier aborts, naming the link"): @@ -206,7 +221,9 @@ class ReviewerCatalogTest extends munit.FunSuite: os.symlink(project / "orca.md", outside) assert(os.isLink(project / "orca.md"), "the fixture must be a symlink") val e = - intercept[OrcaFlowException](ReviewerCatalog.discover(project, global)) + intercept[OrcaFlowException]( + discover(project, global) + ) assert(e.getMessage.contains((project / "orca.md").toString), e.getMessage) test("a symlinked .md in the global tier is followed"): @@ -216,21 +233,25 @@ class ReviewerCatalogTest extends munit.FunSuite: os.makeDir.all(global) os.symlink(global / "orca.md", outside) assert(os.isLink(global / "orca.md"), "the fixture must be a symlink") - val catalog = ReviewerCatalog.discover(project, global) + val catalog = discover(project, global) assertEquals(named(catalog.all), named(ReviewerPrompts.all) :+ "orca") test("a directory named like a reviewer aborts in the project tier"): val (project, global) = dirs() os.makeDir.all(project / "orca.md") val e = - intercept[OrcaFlowException](ReviewerCatalog.discover(project, global)) + intercept[OrcaFlowException]( + discover(project, global) + ) assert(e.getMessage.contains("is a directory"), e.getMessage) test("a directory named like a reviewer aborts in the global tier"): val (project, global) = dirs() os.makeDir.all(global / "orca.md") val e = - intercept[OrcaFlowException](ReviewerCatalog.discover(project, global)) + intercept[OrcaFlowException]( + discover(project, global) + ) assert(e.getMessage.contains("is a directory"), e.getMessage) test("a global-tier symlink to a directory aborts"): @@ -240,7 +261,9 @@ class ReviewerCatalogTest extends munit.FunSuite: os.makeDir.all(global) os.symlink(global / "orca.md", TempDirs.dir("orca-outside-")) val e = - intercept[OrcaFlowException](ReviewerCatalog.discover(project, global)) + intercept[OrcaFlowException]( + discover(project, global) + ) assert(e.getMessage.contains("is a directory"), e.getMessage) test("a project-tier symlink to a directory is reported as a symlink"): @@ -250,7 +273,9 @@ class ReviewerCatalogTest extends munit.FunSuite: os.makeDir.all(project) os.symlink(project / "orca.md", TempDirs.dir("orca-outside-")) val e = - intercept[OrcaFlowException](ReviewerCatalog.discover(project, global)) + intercept[OrcaFlowException]( + discover(project, global) + ) assert(e.getMessage.contains("is a symlink"), e.getMessage) test("an unterminated frontmatter block aborts, not silently skipped"): @@ -261,7 +286,9 @@ class ReviewerCatalogTest extends munit.FunSuite: createFolders = true ) val e = - intercept[OrcaFlowException](ReviewerCatalog.discover(project, global)) + intercept[OrcaFlowException]( + discover(project, global) + ) assert( e.getMessage.contains("block the parser could not read"), e.getMessage @@ -273,7 +300,9 @@ class ReviewerCatalogTest extends munit.FunSuite: val _ = writeReviewer(project, "orca") val _ = writeReviewer(project, "Orca") val e = - intercept[OrcaFlowException](ReviewerCatalog.discover(project, global)) + intercept[OrcaFlowException]( + discover(project, global) + ) assert(e.getMessage.contains("Orca.md"), e.getMessage) assert(e.getMessage.contains("orca.md"), e.getMessage) @@ -285,7 +314,9 @@ class ReviewerCatalogTest extends munit.FunSuite: createFolders = true ) val e = - intercept[OrcaFlowException](ReviewerCatalog.discover(project, global)) + intercept[OrcaFlowException]( + discover(project, global) + ) assert(e.getMessage.contains((project / "orca.md").toString), e.getMessage) assert(e.getMessage.contains("description:"), e.getMessage) @@ -294,14 +325,18 @@ class ReviewerCatalogTest extends munit.FunSuite: val (project, global) = dirs() val _ = writeReviewer(project, "orca", description = "\" \"") val e = - intercept[OrcaFlowException](ReviewerCatalog.discover(project, global)) + intercept[OrcaFlowException]( + discover(project, global) + ) assert(e.getMessage.contains("description:"), e.getMessage) test("a reviewer file with no body aborts"): val (project, global) = dirs() val _ = writeReviewer(project, "orca", body = " ") val e = - intercept[OrcaFlowException](ReviewerCatalog.discover(project, global)) + intercept[OrcaFlowException]( + discover(project, global) + ) assert(e.getMessage.contains("system prompt"), e.getMessage) test("a block-scalar description aborts, naming the key"): @@ -313,7 +348,9 @@ class ReviewerCatalogTest extends munit.FunSuite: description = "> # folded\n checks the thing" ) val e = - intercept[OrcaFlowException](ReviewerCatalog.discover(project, global)) + intercept[OrcaFlowException]( + discover(project, global) + ) assert( e.getMessage.contains("'description:' as a YAML block"), e.getMessage @@ -323,7 +360,9 @@ class ReviewerCatalogTest extends munit.FunSuite: val (project, global) = dirs() val _ = writeReviewer(project, "orca", files = Some("|")) val e = - intercept[OrcaFlowException](ReviewerCatalog.discover(project, global)) + intercept[OrcaFlowException]( + discover(project, global) + ) assert(e.getMessage.contains("'files:' as a YAML block"), e.getMessage) private def assertWrapped( @@ -334,7 +373,9 @@ class ReviewerCatalogTest extends munit.FunSuite: val (project, global) = dirs() val _ = writeReviewer(project, "orca", description, files) val e = - intercept[OrcaFlowException](ReviewerCatalog.discover(project, global)) + intercept[OrcaFlowException]( + discover(project, global) + ) assert(e.getMessage.contains(s"wraps its '$key:' value"), e.getMessage) test("a description wrapped onto the next line aborts, naming the key"): @@ -356,14 +397,16 @@ class ReviewerCatalogTest extends munit.FunSuite: "---\ndescription: checks\ntools:\n - Read\n---\n\n## Scope\n", createFolders = true ) - val catalog = ReviewerCatalog.discover(project, global) + val catalog = discover(project, global) assert(named(catalog.all).contains("orca")) test("a reviewer file with an unparseable files: regex aborts"): val (project, global) = dirs() val _ = writeReviewer(project, "orca", files = Some("""\.scala($""")) val e = - intercept[OrcaFlowException](ReviewerCatalog.discover(project, global)) + intercept[OrcaFlowException]( + discover(project, global) + ) assert(e.getMessage.contains("files:"), e.getMessage) test("builtIn is the shipped set with nothing discovered"): @@ -376,7 +419,7 @@ class ReviewerCatalogTest extends munit.FunSuite: val _ = writeReviewer(project, "orca") val _ = writeReviewer(project, "scala-fp") assertEquals( - ReviewerCatalog.discover(project, global).describe, + discover(project, global).describe, Some( "discovered reviewers: orca (project); " + "scala-fp (project, shadows built-in)" diff --git a/flow/src/test/scala/orca/review/ReviewerSelectorTest.scala b/flow/src/test/scala/orca/review/ReviewerSelectorTest.scala index 333c853a7..8245d81c6 100644 --- a/flow/src/test/scala/orca/review/ReviewerSelectorTest.scala +++ b/flow/src/test/scala/orca/review/ReviewerSelectorTest.scala @@ -251,11 +251,11 @@ class ReviewerSelectorTest extends munit.FunSuite: val capture = new SelectorSteps val selector = ReviewerSelector.narrowingAcrossRounds(ReviewerSelector.allEveryRound) - val selectRound = selector.prepare(all, Title("any"), List("src/lib.rs"))( - using - capture.ctx, - summon[orca.InStage] - ) + val selectRound = + selector.prepare(all, Title("any"), List("src/lib.rs"))(using + capture.ctx, + summon[orca.InStage] + ) // Nobody reported. A lint gate can keep the fix loop iterating through // that, so the round must not run zero reviewers. assertEquals( diff --git a/flow/src/test/scala/orca/runlog/RunEventLogTest.scala b/flow/src/test/scala/orca/runlog/RunEventLogTest.scala index 16428997d..4436ecb56 100644 --- a/flow/src/test/scala/orca/runlog/RunEventLogTest.scala +++ b/flow/src/test/scala/orca/runlog/RunEventLogTest.scala @@ -5,11 +5,17 @@ import orca.agents.{BackendTag, Model, SessionKey} import orca.events.{Cost, CostBasis, OrcaEvent, StageOutcome, Usage} import orca.gitref.BranchName import orca.sessions.SessionRecord -import orca.testkit.TempDirs +import orca.testkit.{Effects, TempDirs} import ox.supervised import ox.channels.BufferCapacity +import ch.qos.logback.classic.{Level, Logger as LogbackLogger} +import ch.qos.logback.classic.spi.ILoggingEvent +import ch.qos.logback.core.read.ListAppender +import org.slf4j.LoggerFactory + import java.time.Instant +import scala.jdk.CollectionConverters.* class RunEventLogTest extends munit.FunSuite: @@ -70,6 +76,7 @@ class RunEventLogTest extends munit.FunSuite: private def started(dir: os.Path)(using ox.Ox): RunEventLog = given BufferCapacity = BufferCapacity(8) RunEventLog.start( + Effects.fs, dir, key, attempt, @@ -87,7 +94,21 @@ class RunEventLogTest extends munit.FunSuite: fileEvents(dir).drop(1).dropRight(1) private def fileEvents(dir: os.Path): List[RunEvent] = - RunEventReader.read(OrcaDir.eventLogPath(dir, key)) + RunEventReader(Effects.fs).read(OrcaDir.eventLogPath(dir, key)) + + /** What `orca.flow` logs at WARN while `body` runs, other suites included. */ + private def warnings(body: => Unit): List[String] = + val logger = + LoggerFactory.getLogger("orca.flow").asInstanceOf[LogbackLogger] + val appender = ListAppender[ILoggingEvent]() + appender.start() + logger.addAppender(appender) + try body + finally + val _ = logger.detachAppender(appender) + appender.list.asScala.toList + .filter(_.getLevel == Level.WARN) + .map(_.getFormattedMessage) test("the log starts with AttemptStarted carrying the schema"): val dir = TempDirs.dir() @@ -300,6 +321,24 @@ class RunEventLogTest extends munit.FunSuite: Some(RunEvent.AttemptFinished(at, attempt, AttemptOutcome.Succeeded)) ) + test("an event after finish is dropped with a warning"): + val dir = TempDirs.dir() + val warned = warnings: + supervised: + val log = started(dir) + log.finish(AttemptOutcome.Succeeded) + log.onEvent(OrcaEvent.StageStarted(outer)) + assertEquals( + fileEvents(dir).lastOption, + Some(RunEvent.AttemptFinished(at, attempt, AttemptOutcome.Succeeded)) + ) + assert( + warned.contains( + "recording StageStarted after the attempt finished (dropped)" + ), + warned + ) + test("a torn last line does not swallow the next AttemptStarted"): val dir = TempDirs.dir() os.write( diff --git a/flow/src/test/scala/orca/runlog/RunEventReaderTest.scala b/flow/src/test/scala/orca/runlog/RunEventReaderTest.scala index 9704173c7..72b6afba4 100644 --- a/flow/src/test/scala/orca/runlog/RunEventReaderTest.scala +++ b/flow/src/test/scala/orca/runlog/RunEventReaderTest.scala @@ -4,7 +4,7 @@ import orca.{AttemptId, StagePath} import orca.agents.BackendTag import orca.events.StageOutcome import orca.gitref.BranchName -import orca.testkit.TempDirs +import orca.testkit.{Effects, TempDirs} import ox.tap import java.time.Instant @@ -29,21 +29,34 @@ class RunEventReaderTest extends munit.FunSuite: RunEvent.encodeLine(finished), RunEvent.encodeLine(bound).take(20) ) - assertEquals(RunEventReader.read(log), List(bound, finished)) + assertEquals( + new RunEventReader(Effects.fs).read(log), + List(bound, finished) + ) test("ignores an unknown field"): val log = logOf( """{"type":"BranchBound","at":"2026-10-08T10:00:00Z","attempt":"1000-42","branch":"b","extra":{"x":1}}""" ) - assertEquals(RunEventReader.read(log), List(bound)) + assertEquals( + new RunEventReader(Effects.fs).read(log), + List(bound) + ) test("an absent file has no events"): - assertEquals(RunEventReader.read(TempDirs.dir() / "events.jsonl"), Nil) + assertEquals( + new RunEventReader(Effects.fs) + .read(TempDirs.dir() / "events.jsonl"), + Nil + ) test("an unreadable file has no events"): val dir = TempDirs.dir() / "events.jsonl" os.makeDir(dir) - assertEquals(RunEventReader.read(dir), Nil) + assertEquals( + new RunEventReader(Effects.fs).read(dir), + Nil + ) test("readOnly keeps only the events of the given types"): val log = logOf( @@ -52,7 +65,8 @@ class RunEventReaderTest extends munit.FunSuite: RunEvent.encodeLine(bound) ) assertEquals( - RunEventReader.readOnly(log, Set(classOf[RunEvent.AttemptFinished])), + new RunEventReader(Effects.fs) + .readOnly(log, Set(classOf[RunEvent.AttemptFinished])), List(finished) ) @@ -104,5 +118,9 @@ class RunEventReaderTest extends munit.FunSuite: ) val log = logOf(events.map(RunEvent.encodeLine)*) events.foreach(e => - assertEquals(RunEventReader.readOnly(log, Set(e.getClass)), List(e)) + assertEquals( + new RunEventReader(Effects.fs) + .readOnly(log, Set(e.getClass)), + List(e) + ) ) diff --git a/flow/src/test/scala/orca/runlog/RunPruningTest.scala b/flow/src/test/scala/orca/runlog/RunPruningTest.scala index 0080da12f..4f64ce6c5 100644 --- a/flow/src/test/scala/orca/runlog/RunPruningTest.scala +++ b/flow/src/test/scala/orca/runlog/RunPruningTest.scala @@ -3,7 +3,7 @@ package orca.runlog import orca.{AttemptId, OrcaDir, RunKey, StagePath} import orca.agents.BackendTag import orca.gitref.BranchName -import orca.testkit.TempDirs +import orca.testkit.{Effects, TempDirs} import ox.tap import java.time.Instant @@ -69,7 +69,7 @@ class RunPruningTest extends munit.FunSuite: runs(cacheRuns, "new", 21, newerThan = 100_000, started): Unit val currentDir = cacheRuns / current.value os.makeDir(currentDir) - RunPruning.pruneRuns(cacheRuns, current) + RunPruning(Effects.fs).pruneRuns(cacheRuns, current) assert(os.exists(currentDir)) test("keeps a run with a session minted after its last success"): @@ -77,7 +77,7 @@ class RunPruningTest extends munit.FunSuite: runs(cacheRuns, "done", 21, newerThan = 100_000, minted, succeeded): Unit val resumed = run(cacheRuns, "resumed", 1000, succeeded, minted) val neverSucceeded = run(cacheRuns, "neverSucceeded", 1000, minted) - RunPruning.pruneRuns(cacheRuns, current) + RunPruning(Effects.fs).pruneRuns(cacheRuns, current) assert(os.exists(resumed)) assert(os.exists(neverSucceeded)) assert(!os.exists(cacheRuns / "done20")) @@ -87,7 +87,7 @@ class RunPruningTest extends munit.FunSuite: val plain = runs(cacheRuns, "plain", 25, newerThan = 1_000_000, started) val sessions = runs(cacheRuns, "sessions", 25, newerThan = 100_000, minted, succeeded) - RunPruning.pruneRuns(cacheRuns, current) + RunPruning(Effects.fs).pruneRuns(cacheRuns, current) assertEquals( os.list(cacheRuns).toSet, (plain.take(20) ++ sessions.take(20)).toSet @@ -99,7 +99,7 @@ class RunPruningTest extends munit.FunSuite: val corrupt = cacheRuns / "corrupt" os.write(corrupt / OrcaDir.EventLogName, "not json\n", createFolders = true) os.mtime.set(corrupt / OrcaDir.EventLogName, 1000): Unit - RunPruning.pruneRuns(cacheRuns, current) + RunPruning(Effects.fs).pruneRuns(cacheRuns, current) assert(!os.exists(corrupt)) test("a run without an event log is the oldest"): @@ -107,7 +107,7 @@ class RunPruningTest extends munit.FunSuite: runs(cacheRuns, "old", 20, newerThan = 0, started): Unit val empty = cacheRuns / "empty" os.makeDir(empty) - RunPruning.pruneRuns(cacheRuns, current) + RunPruning(Effects.fs).pruneRuns(cacheRuns, current) assert(!os.exists(empty)) test( @@ -123,7 +123,7 @@ class RunPruningTest extends munit.FunSuite: val id = AttemptId(Instant.ofEpochMilli(1_000_000L + i), pid = 7) List(OrcaDir.TraceLogSuffix, OrcaDir.RolledTraceLogSuffix).map: suffix => (runDirs(i % 2) / s"${id.value}$suffix").tap(os.write(_, "")) - RunPruning.pruneTraces(cacheRuns) + RunPruning(Effects.fs).pruneTraces(cacheRuns) assertEquals( runDirs.flatMap(os.list(_)).toSet, traces.drop(1).flatten.toSet ++ logs @@ -138,7 +138,7 @@ class RunPruningTest extends munit.FunSuite: ) os.write(cache / "runs" / "abc.sessions.json", "[]", createFolders = true) val runDir = run(cache / "runs", "abc", 1000, started) - RunPruning.removeLegacy(cache) + RunPruning(Effects.fs).removeLegacy(cache) assertEquals(os.list(cache).toList, List(cache / "runs")) assertEquals(os.list(cache / "runs").toList, List(runDir)) assert(os.exists(runDir / OrcaDir.EventLogName)) diff --git a/flow/src/test/scala/orca/runlog/TestRunLog.scala b/flow/src/test/scala/orca/runlog/TestRunLog.scala index 157920798..536492184 100644 --- a/flow/src/test/scala/orca/runlog/TestRunLog.scala +++ b/flow/src/test/scala/orca/runlog/TestRunLog.scala @@ -1,5 +1,7 @@ package orca.runlog +import orca.testkit.Effects + import orca.{AttemptId, OrcaDir, RunKey} import orca.sessions.SessionRecord import ox.{Ox, supervised} @@ -25,6 +27,7 @@ object TestRunLog: def start(dir: os.Path, key: RunKey)(using Ox): RunEventLog = val clock = () => Instant.now() RunEventLog.start( + Effects.fs, dir, key, AttemptId(clock(), pid = ProcessHandle.current().pid()), @@ -39,5 +42,6 @@ object TestRunLog: */ def records(dir: os.Path, key: RunKey): List[SessionRecord] = SessionProjection.records( - RunEventReader.read(OrcaDir.eventLogPath(dir, key)) + new RunEventReader(Effects.fs) + .read(OrcaDir.eventLogPath(dir, key)) ) diff --git a/flows/epics.sc b/flows/epics.sc index 6ae782eca..4a8deee9e 100644 --- a/flows/epics.sc +++ b/flows/epics.sc @@ -1,5 +1,5 @@ // Plan a large change as epics, then plan, build and review each epic in turn. -//> using scala 3.9.0 +//> using scala 3.10.0 //> using dep "org.virtuslab::orca:0.2.1" //> using jvm 21 diff --git a/flows/implement.sc b/flows/implement.sc index ecea6f697..22b411612 100644 --- a/flows/implement.sc +++ b/flows/implement.sc @@ -1,5 +1,5 @@ // Plan a prompt into tasks, review each once, loop a review, then open a PR. -//> using scala 3.9.0 +//> using scala 3.10.0 //> using dep "org.virtuslab::orca:0.2.1" //> using jvm 21 diff --git a/flows/quick.sc b/flows/quick.sc index 3c7f0bded..faa346845 100644 --- a/flows/quick.sc +++ b/flows/quick.sc @@ -1,5 +1,5 @@ // Implement a task directly — no planning stage — review it, then open a PR. -//> using scala 3.9.0 +//> using scala 3.10.0 //> using dep "org.virtuslab::orca:0.2.1" //> using jvm 21 diff --git a/flows/resolve.sc b/flows/resolve.sc index 3bccc690c..c3a733ee7 100644 --- a/flows/resolve.sc +++ b/flows/resolve.sc @@ -1,5 +1,5 @@ // Resolve a request or GitHub issue: triage, reproduce a bug, fix or build it, PR. -//> using scala 3.9.0 +//> using scala 3.10.0 //> using dep "org.virtuslab::orca:0.2.1" //> using jvm 21 diff --git a/flows/review.sc b/flows/review.sc index ef1094515..923ca74ea 100644 --- a/flows/review.sc +++ b/flows/review.sc @@ -1,5 +1,5 @@ // Review a PR, a branch, or local changes — a list of findings, no fixes. -//> using scala 3.9.0 +//> using scala 3.10.0 //> using dep "org.virtuslab::orca:0.2.1" //> using jvm 21 diff --git a/gemini/src/main/scala/orca/backend/gemini/GeminiBackend.scala b/gemini/src/main/scala/orca/backend/gemini/GeminiBackend.scala index 12e477de7..087639c0f 100644 --- a/gemini/src/main/scala/orca/backend/gemini/GeminiBackend.scala +++ b/gemini/src/main/scala/orca/backend/gemini/GeminiBackend.scala @@ -9,7 +9,6 @@ import orca.agents.{ ToolSet, TurnDispatch } -import orca.subprocess.CliResult import orca.backend.{ AskUserChannel, LiveTurn, @@ -23,8 +22,11 @@ import orca.backend.{ TurnResources } import orca.backend.mcp.{AskUserMcpServer, AskUserSession} +import orca.effect.{FileSystem, Network} import orca.subprocess.CliRunner -import ox.{Ox, discard} +import orca.concurrency.defaultBufferCapacity +import orca.safe.prelude.discard +import ox.Ox /** Gemini backend. Both autonomous and interactive paths drive `gemini -p * --output-format stream-json` over stdio: stdout JSONL is parsed @@ -45,12 +47,11 @@ import ox.{Ox, discard} * Autonomous calls skip the bridge. */ private[orca] class GeminiBackend( + net: Network, + fs: FileSystem, cli: CliRunner, - /** Fixed at construction; every spawn runs in this directory. The `os.pwd` - * default serves bare/test construction; the runtime passes the flow's - * real `workDir`. - */ - override val workDir: os.Path = os.pwd + /** Fixed at construction; every spawn runs in this directory. */ + override val workDir: os.Path ) extends AgentBackend[BackendTag.Gemini.type]: /** Gemini's sessions are server-side and durable: the client→server map is @@ -83,7 +84,9 @@ private[orca] class GeminiBackend( SessionSupport.durable( IdScheme.ServerMinted, id => - val result = listSessionsOutput() + // gemini keeps sessions per project directory, so this lists only + // those the spawns in `workDir` can resume. + val result = cli.run(GeminiSessionList.argv, cwd = workDir) result.exitCode == 0 && GeminiSessionList.mentions(result.stdout, id) ) @@ -103,9 +106,11 @@ private[orca] class GeminiBackend( import turn.* val askUser: Option[AskUserSession] = Option.when(mode.isInteractive): - val session = AskUserSession.allocate() + val session = AskUserSession.allocate(net) TurnResources - .useCloseable(GeminiSettings.register(workDir, session.server.url)) + .useCloseable( + GeminiSettings.register(fs, workDir, session.server.url) + ) .discard session SubprocessSpawn.open("gemini", events) { @@ -133,9 +138,3 @@ private[orca] class GeminiBackend( askUser.fold(AskUserChannel.Unavailable)(AskUserChannel.Mcp(_)) ) } - - /** `gemini --list-sessions` in [[workDir]]: gemini keeps sessions per project - * directory, so it lists only those the spawns in [[workDir]] can resume. - */ - private[gemini] def listSessionsOutput(): CliResult = - cli.run(GeminiSessionList.argv, cwd = workDir) diff --git a/gemini/src/main/scala/orca/backend/gemini/GeminiDecoder.scala b/gemini/src/main/scala/orca/backend/gemini/GeminiDecoder.scala index f256ee037..1699e39b0 100644 --- a/gemini/src/main/scala/orca/backend/gemini/GeminiDecoder.scala +++ b/gemini/src/main/scala/orca/backend/gemini/GeminiDecoder.scala @@ -132,7 +132,7 @@ private[gemini] object GeminiDecoder role match case Role.User => Step.continue(state) case Role.Assistant if content.isEmpty => Step.continue(state) - case Role.Assistant => + case Role.Assistant => Step.continue( state.copy(answer = state.answer :+ content), TurnEvent.AssistantTextDelta(content) @@ -167,7 +167,7 @@ private[gemini] object GeminiDecoder ): Out = state.echoes.consume(id) match case Some(rest) => Step.continue(state.copy(echoes = rest)) - case None => + case None => Step.continue( state, TurnEvent.ToolResult( diff --git a/gemini/src/main/scala/orca/backend/gemini/GeminiSettings.scala b/gemini/src/main/scala/orca/backend/gemini/GeminiSettings.scala index 407635b88..0605f3517 100644 --- a/gemini/src/main/scala/orca/backend/gemini/GeminiSettings.scala +++ b/gemini/src/main/scala/orca/backend/gemini/GeminiSettings.scala @@ -2,17 +2,11 @@ package orca.backend.gemini import orca.OrcaFlowException import orca.backend.mcp.{AskUserMcpServer, McpHost} -import orca.json.RawJson +import orca.effect.FileSystem +import orca.json.{JsonCodec, RawJson} +import orca.safe.prelude.given -import com.github.plokhotnyuk.jsoniter_scala.core.{ - JsonValueCodec, - readFromString, - writeToString -} -import com.github.plokhotnyuk.jsoniter_scala.macros.{ - CodecMakerConfig, - JsonCodecMaker -} +import com.github.plokhotnyuk.jsoniter_scala.core.JsonValueCodec import scala.util.Try @@ -38,7 +32,7 @@ import scala.util.Try private[gemini] object GeminiSettings: private given objCodec: JsonValueCodec[Map[String, RawJson]] = - JsonCodecMaker.make + JsonCodec.make /** One `mcpServers.` entry. jsoniter emits fields in declaration order, * so this serialises as `{"httpUrl":…,"timeout":…}`. Decoding rejects any @@ -46,7 +40,7 @@ private[gemini] object GeminiSettings: */ private case class OrcaServerEntry(httpUrl: String, timeout: Long) private given entryCodec: JsonValueCodec[OrcaServerEntry] = - JsonCodecMaker.make(CodecMakerConfig.withSkipUnexpectedFields(false)) + JsonCodec.make(using JsonCodec.config.withSkipUnexpectedFields(false)) /** `timeout` (ms) mirrors [[orca.backend.mcp.AskUserMcpServer.ToolTimeout]] * (rendered as ms/sec across backends; keep in sync). Without it gemini @@ -65,34 +59,34 @@ private[gemini] object GeminiSettings: * Throws [[OrcaFlowException]] when `.gemini` or `settings.json` is a * symlink, before touching either. */ - def register(workDir: os.Path, mcpUrl: String): AutoCloseable = + def register( + fs: FileSystem, + workDir: os.Path, + mcpUrl: String + ): AutoCloseable^{fs} = val dir = workDir / ".gemini" val file = dir / "settings.json" - refuseSymlink(dir) - refuseSymlink(file) - val dirExisted = os.exists(dir) - val fileExisted = os.exists(file) - val prior = if fileExisted then withoutStaleOrca(os.read(file)) else None + refuseSymlink(fs, dir) + refuseSymlink(fs, file) + val dirExisted = fs.exists(dir) + val fileExisted = fs.exists(file) + val prior = if fileExisted then withoutStaleOrca(fs.read(file)) else None // A settings file holding only a stale entry means orca created `.gemini`. val orcaCreatedDir = !dirExisted || (fileExisted && prior.isEmpty) - os.write.over( - file, - withOrca(prior.getOrElse("{}"), mcpUrl), - createFolders = true - ) + fs.writeOver(file, withOrca(prior.getOrElse("{}"), mcpUrl)) () => prior match - case Some(content) => os.write.over(file, content) - case None => os.remove(file): Unit - if orcaCreatedDir && os.exists(dir) && os.list(dir).isEmpty then - os.remove(dir): Unit + case Some(content) => fs.writeOver(file, content) + case None => fs.remove(file) + if orcaCreatedDir && fs.exists(dir) && fs.list(dir).isEmpty then + fs.remove(dir) /** A committed `.gemini` symlink would redirect the write outside the working - * tree, e.g. into the user's global `~/.gemini`. `os.isLink` does not follow + * tree, e.g. into the user's global `~/.gemini`. `isLink` does not follow * links, so each path component is checked on its own. */ - private def refuseSymlink(path: os.Path): Unit = - if os.isLink(path) then + private def refuseSymlink(fs: FileSystem, path: os.Path): Unit = + if fs.isLink(path) then throw new OrcaFlowException( s"$path is a symlink — refusing to register orca's ask_user MCP " + "server through it. Replace it with a regular file or directory, or " + @@ -105,10 +99,14 @@ private[gemini] object GeminiSettings: private[gemini] def withOrca(content: String, mcpUrl: String): String = // Serialized through a typed codec, not interpolated, so a URL containing // `"` or `\` stays valid JSON. - val entry = RawJson(writeToString(OrcaServerEntry(mcpUrl, TimeoutMillis))) - val top = readFromString[Map[String, RawJson]](content) + val entry = RawJson( + JsonCodec.writeString(OrcaServerEntry(mcpUrl, TimeoutMillis)) + ) + val top = JsonCodec.readString[Map[String, RawJson]](content) val servers = mcpServers(top) + (AskUserMcpServer.ServerName -> entry) - writeToString(top + ("mcpServers" -> RawJson(writeToString(servers)))) + JsonCodec.writeString( + top + ("mcpServers" -> RawJson(JsonCodec.writeString(servers))) + ) /** The settings without an `orca` entry that [[withOrca]] wrote; `None` when * nothing else is left. The content is returned verbatim when there is no @@ -116,7 +114,7 @@ private[gemini] object GeminiSettings: * other shape is the user's own and is kept. */ private[gemini] def withoutStaleOrca(content: String): Option[String] = - val top = readFromString[Map[String, RawJson]](content) + val top = JsonCodec.readString[Map[String, RawJson]](content) val servers = mcpServers(top) if !servers.get(AskUserMcpServer.ServerName).exists(isOrcaEntry) then Some(content) @@ -124,13 +122,13 @@ private[gemini] object GeminiSettings: val rest = servers - AskUserMcpServer.ServerName val stripped = if rest.isEmpty then top - "mcpServers" - else top + ("mcpServers" -> RawJson(writeToString(rest))) - Option.when(stripped.nonEmpty)(writeToString(stripped)) + else top + ("mcpServers" -> RawJson(JsonCodec.writeString(rest))) + Option.when(stripped.nonEmpty)(JsonCodec.writeString(stripped)) private def mcpServers(top: Map[String, RawJson]): Map[String, RawJson] = top .get("mcpServers") - .map(raw => readFromString[Map[String, RawJson]](raw.value)) + .map(raw => JsonCodec.readString[Map[String, RawJson]](raw.value)) .getOrElse(Map.empty) /** Whether `raw` has the shape [[withOrca]] writes, pointing at an @@ -138,5 +136,5 @@ private[gemini] object GeminiSettings: * with a different timeout matches. */ private def isOrcaEntry(raw: RawJson): Boolean = - Try(readFromString[OrcaServerEntry](raw.value)).toOption + Try(JsonCodec.readString[OrcaServerEntry](raw.value)).toOption .exists(entry => McpHost.isHostUrl(entry.httpUrl)) diff --git a/gemini/src/main/scala/orca/backend/gemini/jsonl/InboundEvent.scala b/gemini/src/main/scala/orca/backend/gemini/jsonl/InboundEvent.scala index 1d19a02c1..5718a3b56 100644 --- a/gemini/src/main/scala/orca/backend/gemini/jsonl/InboundEvent.scala +++ b/gemini/src/main/scala/orca/backend/gemini/jsonl/InboundEvent.scala @@ -1,10 +1,7 @@ package orca.backend.gemini.jsonl import orca.events.Usage -import orca.json.RawJson - -import com.github.plokhotnyuk.jsoniter_scala.core.readFromString -import com.github.plokhotnyuk.jsoniter_scala.macros.ConfiguredJsonValueCodec +import orca.json.{JsonCodec, RawJson} /** Typed classification of a `message` event's `role` field. gemini spells the * assistant side `model`/`assistant` across versions, so any *present* value @@ -74,7 +71,7 @@ private[gemini] object InboundEvent: * skip or fail. */ def parse(line: String): InboundEvent = - val envelope = readFromString[TopEnvelope](line) + val envelope = JsonCodec.readString[TopEnvelope](line) envelope.`type` match case "init" => parseInit(line) case "message" => parseMessage(line) @@ -85,11 +82,11 @@ private[gemini] object InboundEvent: case other => Unknown(other) private def parseInit(line: String): InboundEvent = - val w = readFromString[InitWire](line) + val w = JsonCodec.readString[InitWire](line) Init(w.session_id, w.model) private def parseMessage(line: String): InboundEvent = - val w = readFromString[MessageWire](line) + val w = JsonCodec.readString[MessageWire](line) Message(roleOf(w.role), w.content.getOrElse("")) /** Classify the wire `role` — see [[Role]]. Operates on the `Option` so a @@ -101,7 +98,7 @@ private[gemini] object InboundEvent: case None => Role.Unknown private def parseToolUse(line: String): InboundEvent = - val w = readFromString[ToolUseWire](line) + val w = JsonCodec.readString[ToolUseWire](line) ToolUse( toolName = w.tool_name.getOrElse(""), toolId = w.tool_id.getOrElse(""), @@ -109,7 +106,7 @@ private[gemini] object InboundEvent: ) private def parseToolResult(line: String): InboundEvent = - val w = readFromString[ToolResultWire](line) + val w = JsonCodec.readString[ToolResultWire](line) ToolResult( toolId = w.tool_id.getOrElse(""), status = ToolStatus.of(w.status), @@ -117,11 +114,11 @@ private[gemini] object InboundEvent: ) private def parseError(line: String): InboundEvent = - val w = readFromString[ErrorWire](line) + val w = JsonCodec.readString[ErrorWire](line) Error(w.message.getOrElse("")) private def parseResult(line: String): InboundEvent = - val w = readFromString[ResultWire](line) + val w = JsonCodec.readString[ResultWire](line) val s = w.stats.getOrElse(StatsWire()) Result( Usage.inclusiveInput( @@ -143,8 +140,7 @@ private[gemini] object InboundEvent: // --- Wire shapes --- - private case class TopEnvelope(`type`: String) - derives ConfiguredJsonValueCodec + private case class TopEnvelope(`type`: String) derives JsonCodec /** `session_id` is required (no default): missing it throws * `JsonReaderException` rather than silently becoming `Init("")`. @@ -152,27 +148,26 @@ private[gemini] object InboundEvent: private case class InitWire( session_id: String, model: Option[String] = None - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec private case class MessageWire( role: Option[String] = None, content: Option[String] = None - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec private case class ToolUseWire( tool_name: Option[String] = None, tool_id: Option[String] = None, parameters: Option[RawJson] = None - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec private case class ToolResultWire( tool_id: Option[String] = None, status: Option[String] = None, output: Option[String] = None - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec - private case class ErrorWire(message: Option[String] = None) - derives ConfiguredJsonValueCodec + private case class ErrorWire(message: Option[String] = None) derives JsonCodec // `total_tokens` is decoded only as the cross-check `Usage.inclusiveInput` // runs against the axes above; no orca axis is derived from it. @@ -182,9 +177,9 @@ private[gemini] object InboundEvent: cached: Option[Long] = None, cached_input_tokens: Option[Long] = None, total_tokens: Option[Long] = None - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec private case class ResultWire( status: Option[String] = None, stats: Option[StatsWire] = None - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec diff --git a/gemini/src/test/scala/orca/backend/gemini/GeminiBackendTest.scala b/gemini/src/test/scala/orca/backend/gemini/GeminiBackendTest.scala index b4dac2064..d7b62adb2 100644 --- a/gemini/src/test/scala/orca/backend/gemini/GeminiBackendTest.scala +++ b/gemini/src/test/scala/orca/backend/gemini/GeminiBackendTest.scala @@ -1,6 +1,7 @@ package orca.backend.gemini -import orca.testkit.OpenTurn +import orca.effect.Network +import orca.testkit.{Effects, OpenTurn} import orca.backend.{SupervisedBackend, SystemPromptComposer} import orca.agents.{ TurnDispatch, @@ -30,7 +31,14 @@ class GeminiBackendTest extends munit.FunSuite: workDir: os.Path = TempDirs.dir() )( body: ox.Ox ?=> GeminiBackend => T - ): T = SupervisedBackend.using(new GeminiBackend(runner, workDir = workDir))( + ): T = SupervisedBackend.using( + new GeminiBackend( + Network.ambient, + Effects.fs, + runner, + workDir = workDir + ) + )( body ) @@ -331,7 +339,9 @@ class GeminiBackendTest extends munit.FunSuite: "test invariant: --list-sessions stdout must NOT contain the client id" ) val stub = new StubCliRunner(CliResult(0, listOutput, "")) - SupervisedBackend.using(new GeminiBackend(stub)): backend => + SupervisedBackend.using( + new GeminiBackend(Network.ambient, Effects.fs, stub, os.pwd) + ): backend => backend.sessions.rehydrate(clientForProbe, serverForProbe) assertEquals( backend.sessions.dispatchFor(clientForProbe).asTurnDispatch, @@ -350,7 +360,9 @@ class GeminiBackendTest extends munit.FunSuite: // (the process cwd, under `--worktree`) cannot see the flow's sessions. val workDir = TempDirs.dir() val stub = new StubCliRunner(CliResult(0, serverForProbe.value, "")) - SupervisedBackend.using(new GeminiBackend(stub, workDir)): backend => + SupervisedBackend.using( + new GeminiBackend(Network.ambient, Effects.fs, stub, workDir) + ): backend => backend.sessions.rehydrate(clientForProbe, serverForProbe) val _ = backend.sessions.dispatchFor(clientForProbe) assertEquals(stub.calls.map(_.cwd), List(workDir)) @@ -362,7 +374,9 @@ class GeminiBackendTest extends munit.FunSuite: // run (and must not pass the client id to --list-sessions). val stub = new StubCliRunner(CliResult(0, "client-uuid 2024-01-01T00:00:00", "")) - SupervisedBackend.using(new GeminiBackend(stub)): backend => + SupervisedBackend.using( + new GeminiBackend(Network.ambient, Effects.fs, stub, os.pwd) + ): backend => assertEquals( backend.sessions.dispatchFor(clientForProbe).asTurnDispatch, TurnDispatch.Fresh @@ -373,7 +387,9 @@ class GeminiBackendTest extends munit.FunSuite: ): val stub = new StubCliRunner(CliResult(0, "sess-other 2024-01-01T00:00:00", "")) - SupervisedBackend.using(new GeminiBackend(stub)): backend => + SupervisedBackend.using( + new GeminiBackend(Network.ambient, Effects.fs, stub, os.pwd) + ): backend => backend.sessions.rehydrate(clientForProbe, serverForProbe) assertEquals( backend.sessions.dispatchFor(clientForProbe).asTurnDispatch, @@ -384,7 +400,9 @@ class GeminiBackendTest extends munit.FunSuite: "a rehydrated id opens fresh when gemini --list-sessions exits non-zero" ): val stub = new StubCliRunner(CliResult(1, "sess-abc-123", "")) - SupervisedBackend.using(new GeminiBackend(stub)): backend => + SupervisedBackend.using( + new GeminiBackend(Network.ambient, Effects.fs, stub, os.pwd) + ): backend => backend.sessions.rehydrate(clientForProbe, serverForProbe) assertEquals( backend.sessions.dispatchFor(clientForProbe).asTurnDispatch, @@ -401,7 +419,9 @@ class GeminiBackendTest extends munit.FunSuite: env: Map[String, String], cwd: os.Path ): CliResult = throw new RuntimeException("binary not found") - SupervisedBackend.using(new GeminiBackend(stub)): backend => + SupervisedBackend.using( + new GeminiBackend(Network.ambient, Effects.fs, stub, os.pwd) + ): backend => backend.sessions.rehydrate(clientForProbe, serverForProbe) assertEquals( backend.sessions.dispatchFor(clientForProbe).asTurnDispatch, diff --git a/gemini/src/test/scala/orca/backend/gemini/GeminiIntegrationTest.scala b/gemini/src/test/scala/orca/backend/gemini/GeminiIntegrationTest.scala index 38845efd7..ca1debefc 100644 --- a/gemini/src/test/scala/orca/backend/gemini/GeminiIntegrationTest.scala +++ b/gemini/src/test/scala/orca/backend/gemini/GeminiIntegrationTest.scala @@ -9,8 +9,9 @@ import orca.agents.{ WireSessionId } import orca.backend.SupervisedBackend +import orca.effect.Network import orca.subprocess.OsProcCliRunner -import orca.testkit.TempDirs +import orca.testkit.{Effects, TempDirs} /** End-to-end tests against the real `gemini` CLI. Gated on the * `ORCA_INTEGRATION` environment variable so `sbt test` without the flag @@ -33,7 +34,12 @@ class GeminiIntegrationTest extends munit.FunSuite: private def withBackend(body: ox.Ox ?=> GeminiBackend => Unit): Unit = SupervisedBackend.using( - new GeminiBackend(OsProcCliRunner, workDir = TempDirs.dir()) + new GeminiBackend( + Network.ambient, + Effects.fs, + OsProcCliRunner(Effects.procs), + workDir = TempDirs.dir() + ) )(body) // A cheap, widely-available model; override via ORCA_GEMINI_MODEL. diff --git a/gemini/src/test/scala/orca/backend/gemini/GeminiSettingsTest.scala b/gemini/src/test/scala/orca/backend/gemini/GeminiSettingsTest.scala index a9c353ed9..362451df4 100644 --- a/gemini/src/test/scala/orca/backend/gemini/GeminiSettingsTest.scala +++ b/gemini/src/test/scala/orca/backend/gemini/GeminiSettingsTest.scala @@ -3,10 +3,12 @@ package orca.backend.gemini import com.github.plokhotnyuk.jsoniter_scala.core.readFromString import com.github.plokhotnyuk.jsoniter_scala.macros.JsonCodecMaker import orca.OrcaFlowException -import orca.testkit.TempDirs +import orca.testkit.{Effects, TempDirs} class GeminiSettingsTest extends munit.FunSuite: + private val fs = Effects.fs + private given mapCodec : com.github.plokhotnyuk.jsoniter_scala.core.JsonValueCodec[ Map[String, orca.json.RawJson] @@ -22,7 +24,7 @@ class GeminiSettingsTest extends munit.FunSuite: "register creates settings.json with the orca MCP server when none exists" ): val workDir = TempDirs.dir() - val _ = GeminiSettings.register(workDir, "http://127.0.0.1:9999/mcp") + val _ = GeminiSettings.register(fs, workDir, "http://127.0.0.1:9999/mcp") val file = settingsFile(workDir) assert(os.exists(file), "settings.json should be created") val keys = topLevel(os.read(file)) @@ -55,14 +57,14 @@ class GeminiSettingsTest extends munit.FunSuite: test("close removes a .gemini directory it created"): val workDir = TempDirs.dir() - val restore = GeminiSettings.register(workDir, "http://x/mcp") + val restore = GeminiSettings.register(fs, workDir, "http://x/mcp") assert(os.exists(settingsFile(workDir))) restore.close() assert(!os.exists(workDir / ".gemini")) test("close keeps a .gemini directory it created that gained other files"): val workDir = TempDirs.dir() - val restore = GeminiSettings.register(workDir, "http://x/mcp") + val restore = GeminiSettings.register(fs, workDir, "http://x/mcp") val other = workDir / ".gemini" / "other.json" os.write(other, "{}") restore.close() @@ -72,7 +74,7 @@ class GeminiSettingsTest extends munit.FunSuite: test("close keeps a pre-existing .gemini directory"): val workDir = TempDirs.dir() os.makeDir(workDir / ".gemini") - val restore = GeminiSettings.register(workDir, "http://x/mcp") + val restore = GeminiSettings.register(fs, workDir, "http://x/mcp") restore.close() assert(!os.exists(settingsFile(workDir))) assert(os.isDir(workDir / ".gemini")) @@ -84,7 +86,7 @@ class GeminiSettingsTest extends munit.FunSuite: """{"theme":"dark","mcpServers":{"github":{"httpUrl":"http://gh/mcp"}}}""" os.write(file, original, createFolders = true) - val restore = GeminiSettings.register(workDir, "http://orca/mcp") + val restore = GeminiSettings.register(fs, workDir, "http://orca/mcp") val keys = topLevel(os.read(file)) assertEquals(keys("theme").value, "\"dark\"") val servers = topLevel(keys("mcpServers").value) @@ -110,14 +112,14 @@ class GeminiSettingsTest extends munit.FunSuite: s"""{"theme":"dark","mcpServers":$StaleEntry}""", createFolders = true ) - GeminiSettings.register(workDir, "http://orca/mcp").close() + GeminiSettings.register(fs, workDir, "http://orca/mcp").close() assertEquals(topLevel(os.read(file)).keySet, Set("theme")) test("close removes a .gemini directory that held only a stale orca entry"): val workDir = TempDirs.dir() val file = settingsFile(workDir) os.write(file, s"""{"mcpServers":$StaleEntry}""", createFolders = true) - GeminiSettings.register(workDir, "http://orca/mcp").close() + GeminiSettings.register(fs, workDir, "http://orca/mcp").close() assert(!os.exists(workDir / ".gemini")) test("register keeps a user's orca entry of another shape"): @@ -125,7 +127,7 @@ class GeminiSettingsTest extends munit.FunSuite: val file = settingsFile(workDir) val original = """{"mcpServers":{"orca":{"command":"my-orca"}}}""" os.write(file, original, createFolders = true) - GeminiSettings.register(workDir, "http://orca/mcp").close() + GeminiSettings.register(fs, workDir, "http://orca/mcp").close() assertEquals(os.read(file), original) test("register keeps a user's orca entry pointing off the loopback host"): @@ -134,7 +136,7 @@ class GeminiSettingsTest extends munit.FunSuite: val original = """{"mcpServers":{"orca":{"httpUrl":"http://example.com/mcp","timeout":1}}}""" os.write(file, original, createFolders = true) - GeminiSettings.register(workDir, "http://orca/mcp").close() + GeminiSettings.register(fs, workDir, "http://orca/mcp").close() assertEquals(os.read(file), original) test("register refuses a symlinked .gemini directory"): @@ -142,7 +144,7 @@ class GeminiSettingsTest extends munit.FunSuite: val target = TempDirs.dir() os.symlink(workDir / ".gemini", target) val _ = intercept[OrcaFlowException]( - GeminiSettings.register(workDir, "http://orca/mcp") + GeminiSettings.register(fs, workDir, "http://orca/mcp") ) assertEquals(os.list(target), Seq.empty) @@ -153,6 +155,6 @@ class GeminiSettingsTest extends munit.FunSuite: os.makeDir(workDir / ".gemini") os.symlink(settingsFile(workDir), target) val _ = intercept[OrcaFlowException]( - GeminiSettings.register(workDir, "http://orca/mcp") + GeminiSettings.register(fs, workDir, "http://orca/mcp") ) assertEquals(os.read(target), "{}") diff --git a/gemini/src/test/scala/orca/backend/gemini/GeminiTurnTest.scala b/gemini/src/test/scala/orca/backend/gemini/GeminiTurnTest.scala index 2830e1d6e..7f45fa564 100644 --- a/gemini/src/test/scala/orca/backend/gemini/GeminiTurnTest.scala +++ b/gemini/src/test/scala/orca/backend/gemini/GeminiTurnTest.scala @@ -1,6 +1,7 @@ package orca.backend.gemini import orca.agents.{Model, WireSessionId} +import orca.effect.Network import orca.events.TurnDebit import orca.testkit.Usages.usage import orca.{OrcaFlowException, OrcaInteractiveCancelled} @@ -544,7 +545,7 @@ class GeminiTurnTest extends munit.FunSuite: live.cancel() live.awaitResult() match case Left(_: OrcaInteractiveCancelled) => () - case other => + case other => fail(s"expected Left(OrcaInteractiveCancelled), got: $other") assertEquals(process.sigIntCount, 1) @@ -588,7 +589,7 @@ class GeminiTurnTest extends munit.FunSuite: val process = new FakePipedCliProcess() val live = GeminiTurn( process, - askUser = AskUserChannel.Mcp(AskUserSession.allocate()) + askUser = AskUserChannel.Mcp(AskUserSession.allocate(Network.ambient)) ) process.enqueueStdout("""{"type":"init","session_id":"s"}""") @@ -625,7 +626,7 @@ class GeminiTurnTest extends munit.FunSuite: supervised: given BufferCapacity = BufferCapacity(8) val process = new FakePipedCliProcess() - val askUser = AskUserSession.allocate() + val askUser = AskUserSession.allocate(Network.ambient) val live = GeminiTurn(process, askUser = AskUserChannel.Mcp(askUser)) val bridge = askUser.bridge diff --git a/runner/src/main/resources/orca/runner/prompts/stack-discovery.md b/lifecycle/src/main/resources/orca/lifecycle/prompts/stack-discovery.md similarity index 100% rename from runner/src/main/resources/orca/runner/prompts/stack-discovery.md rename to lifecycle/src/main/resources/orca/lifecycle/prompts/stack-discovery.md diff --git a/runner/src/main/scala/orca/RunTarget.scala b/lifecycle/src/main/scala/orca/RunTarget.scala similarity index 100% rename from runner/src/main/scala/orca/RunTarget.scala rename to lifecycle/src/main/scala/orca/RunTarget.scala diff --git a/runner/src/main/scala/orca/runner/BranchHandoff.scala b/lifecycle/src/main/scala/orca/lifecycle/BranchHandoff.scala similarity index 94% rename from runner/src/main/scala/orca/runner/BranchHandoff.scala rename to lifecycle/src/main/scala/orca/lifecycle/BranchHandoff.scala index e8d06c09d..d08566cec 100644 --- a/runner/src/main/scala/orca/runner/BranchHandoff.scala +++ b/lifecycle/src/main/scala/orca/lifecycle/BranchHandoff.scala @@ -1,11 +1,11 @@ -package orca.runner +package orca.lifecycle import orca.progress.BranchMode /** Where a successful run leaves HEAD, derived from the run rather than asked * for by the flow script — see [[BranchHandoff.of]] for the rule. */ -private[runner] enum BranchHandoff: +private[lifecycle] enum BranchHandoff: /** Check the branch the run started on back out, keeping the feature branch. */ case ReturnToStart @@ -13,7 +13,7 @@ private[runner] enum BranchHandoff: /** Leave HEAD where the run ended — on the feature branch. */ case StayPut -private[runner] object BranchHandoff: +private[lifecycle] object BranchHandoff: /** The handoff for a run bound in `branchMode` that happened in `worktree` * and left `published` behind: * diff --git a/runner/src/main/scala/orca/runner/ClosingSummary.scala b/lifecycle/src/main/scala/orca/lifecycle/ClosingSummary.scala similarity index 95% rename from runner/src/main/scala/orca/runner/ClosingSummary.scala rename to lifecycle/src/main/scala/orca/lifecycle/ClosingSummary.scala index b6a67fab3..3042310b6 100644 --- a/runner/src/main/scala/orca/runner/ClosingSummary.scala +++ b/lifecycle/src/main/scala/orca/lifecycle/ClosingSummary.scala @@ -1,4 +1,4 @@ -package orca.runner +package orca.lifecycle import orca.gitref.{BranchName, CommitHash, Head} import orca.progress.PublishedWork @@ -11,7 +11,7 @@ import orca.progress.PublishedWork * Only tracked files are counted, so the count is exactly what the `git diff` * this summary offers can show. */ -private[runner] case class RunChanges( +private[lifecycle] case class RunChanges( base: CommitHash, filesChanged: Int, countedOn: BranchName @@ -24,7 +24,7 @@ private[runner] case class RunChanges( * a run that produced nothing has its throwaway branch deleted and ends back * where it started. */ -private[runner] object ClosingSummary: +private[lifecycle] object ClosingSummary: /** One line per fact, each emitted as its own `OrcaEvent.Step` so listeners * that render an event per line keep them aligned. * @@ -57,7 +57,7 @@ private[runner] object ClosingSummary: case None => s"done — you are on ${head.describe}" case Some(path) => s"done — the work is in $path on ${on.describe}" val summary = changes match - case None => List(where(head)) + case None => List(where(head)) case Some(RunChanges(_, 0, _)) => List(where(head), "no files changed") case Some(RunChanges(base, files, countedOn)) => diff --git a/runner/src/main/scala/orca/runner/DirtyTreePolicy.scala b/lifecycle/src/main/scala/orca/lifecycle/DirtyTreePolicy.scala similarity index 54% rename from runner/src/main/scala/orca/runner/DirtyTreePolicy.scala rename to lifecycle/src/main/scala/orca/lifecycle/DirtyTreePolicy.scala index a1c63c615..2b16add73 100644 --- a/runner/src/main/scala/orca/runner/DirtyTreePolicy.scala +++ b/lifecycle/src/main/scala/orca/lifecycle/DirtyTreePolicy.scala @@ -1,12 +1,9 @@ -package orca.runner - -import scala.annotation.tailrec -import scala.io.StdIn +package orca.lifecycle /** What setup does with a working tree holding uncommitted or untracked files * (ADR 0018 §2.5) — the outcome of [[DirtyTreePolicy.decide]]. */ -private[runner] enum DirtyTreeChoice: +private[orca] enum DirtyTreeChoice: /** Stash everything, so the run starts from committed content. */ case Stash @@ -19,7 +16,7 @@ private[runner] enum DirtyTreeChoice: /** The run facts [[DirtyTreePolicy.decide]] reads. Named rather than four bare * arguments in a row, so a transposed one cannot silently invert the decision. */ -private[runner] case class DirtyTreeFacts( +private[lifecycle] case class DirtyTreeFacts( // Presence, NOT resumability: a corrupt log is present but resumes nothing // (`bindBranch` warns and starts fresh from it). Both cases must still // start from a clean tree — the stash is the only thing that reverts a @@ -34,7 +31,7 @@ private[runner] case class DirtyTreeFacts( * carries it out so every cell of it is testable without a repository or a * terminal. */ -private[runner] object DirtyTreePolicy: +private[orca] object DirtyTreePolicy: /** Which of [[DirtyTreeChoice]] applies, given the run's facts. * @@ -60,53 +57,14 @@ private[runner] object DirtyTreePolicy: else if tty() then ask(facts.dirtyCount) else DirtyTreeChoice.Stash - /** The production `ask`: a numbered menu on stderr, one line from stdin. - * - * stderr, not stdout, so the menu never lands in a redirected transcript; - * `StdIn` rather than any richer input, since the runner deliberately owns - * no terminal UI of its own. `setup`'s gate requires BOTH streams to be a - * terminal, so neither half of the exchange can end up invisible. - * - * This writes the terminal outside `TerminalOutput`'s prompt transaction, - * which is otherwise the single owner of the cursor. What makes that safe is - * WHEN it runs: before any stage or setup activity raises the status row, so - * nothing is pinned at the bottom and the animator's `tick` is a no-op. - * Routing it through the transaction instead would mean handing setup the - * run's `Interaction` — including every embedder's non-terminal one. - * Residual: the read blocks uninterruptibly, so a fork failing elsewhere in - * the run's scope waits for the answer. - */ - def promptOnStderr(dirtyCount: Int): DirtyTreeChoice = - Console.err.println( - s"$dirtyCount uncommitted/untracked file(s) in the working tree:\n" + - " 1) stash them and start from a clean tree (default)\n" + - " 2) keep them in place for the flow to work on (if the flow fails " + - "before its first commit, kept changes to tracked files are lost)\n" + - " 3) abort" - ) - readAnswer() - - /** One `choice [1]:` line + read, repeated until [[parse]] accepts the answer - * — an unrecognized one must not silently pick the stash default against the - * user's intent. EOF and an empty line are accepted (as the default), so a - * closed stdin cannot loop. - */ - @tailrec - private def readAnswer(): DirtyTreeChoice = - Console.err.print("choice [1]: ") - Console.err.flush() - parse(Option(StdIn.readLine())) match - case Some(choice) => choice - case None => - Console.err.println("unrecognized answer — enter 1, 2, or 3") - readAnswer() - /** One menu answer: "1"/"2"/"3" name their entries; an empty line and EOF * (`readLine` yields `null` when stdin closes mid-prompt) mean the stash * default, matching what a headless run does. Any other answer is `None` — - * unrecognized, for [[readAnswer]] to re-ask. + * unrecognized, for `DirtyTreePrompt` to re-ask. */ - private[runner] def parse(answer: Option[String]): Option[DirtyTreeChoice] = + private[orca] def parse( + answer: Option[String] + ): Option[DirtyTreeChoice] = answer.map(_.trim) match case None | Some("") | Some("1") => Some(DirtyTreeChoice.Stash) case Some("2") => Some(DirtyTreeChoice.Keep) diff --git a/runner/src/main/scala/orca/runner/FlowLifecycle.scala b/lifecycle/src/main/scala/orca/lifecycle/FlowLifecycle.scala similarity index 94% rename from runner/src/main/scala/orca/runner/FlowLifecycle.scala rename to lifecycle/src/main/scala/orca/lifecycle/FlowLifecycle.scala index bf0ba95cb..17615acf1 100644 --- a/runner/src/main/scala/orca/runner/FlowLifecycle.scala +++ b/lifecycle/src/main/scala/orca/lifecycle/FlowLifecycle.scala @@ -1,11 +1,11 @@ -package orca.runner +package orca.lifecycle +import orca.effect.FileSystem import orca.{ BranchNamingStrategy, FlowContext, FlowControl, InStage, - OrcaArgs, OrcaDir, OrcaFlowException, ReportedFailure, @@ -32,18 +32,17 @@ import orca.progress.{ NotASlugRefused } import orca.settings.{AgentSettings, SettingsFile, SettingsScope} -import orca.subprocess.TtyProbe import orca.tools.{PrReadiness, RuntimeGit, UntrackedFiles} -import org.slf4j.LoggerFactory -import ox.either.orThrow - -import scala.util.control.NonFatal +import orca.safe.Logger +import orca.safe.prelude.{NonFatal, orThrow} /** Flow setup/teardown/recovery lifecycle (ADR 0018 §2.4/§2.5). Owns the * privileged, outside-any-user-stage git and progress-store mutations that * bracket the body. */ -object FlowLifecycle: +private[orca] object FlowLifecycle: + + private val flowLog = Logger("orca.flow") /** One run's phases, in mandated order: body → disjoint success/failure * teardown (ADR 0018 §2.4/§2.5). [[setup]] (branch + log binding) already @@ -63,7 +62,6 @@ object FlowLifecycle: flowSetup: FlowSetup, debug: Boolean )(body: (FlowContext, FlowControl) ?=> Unit): Unit = - val log = LoggerFactory.getLogger("orca.flow") // The whole flow body runs as a top-level stage: an otherwise unhandled // exception surfaces as a single Error event. `teardownFailure` runs only // here in the body phase, so a success-teardown error can never trigger @@ -83,8 +81,8 @@ object FlowLifecycle: catch case NonFatal(t) => f.addSuppressed(t) - log.debug("teardownFailure failed after body failure", t) - if debug then t.printStackTrace(System.err) + flowLog.debug("teardownFailure failed after body failure", t) + if debug then t.printStackTrace() ctx.emit( OrcaEvent.Step( "warning: workspace reset failed after the flow failure — " + @@ -108,8 +106,8 @@ object FlowLifecycle: case NonFatal(e) => val reported = ReportedFailure.reportOnce(e): cause => emit(OrcaEvent.Error(TextUtil.throwableMessage(cause))) - LoggerFactory.getLogger("orca.flow").debug("flow aborted", reported) - if debug then reported.printStackTrace(System.err) + flowLog.debug("flow aborted", reported) + if debug then reported.printStackTrace() throw reported /** Outcome of [[setup]] (ADR 0019 stack settings included). @@ -152,7 +150,7 @@ object FlowLifecycle: * takes this attempt's, a resumed one reads back what its header recorded, * so both describe the RUN rather than this attempt. */ - private[orca] case class BranchBinding( + private[lifecycle] case class BranchBinding( featureBranch: FeatureBranch, startingHead: Head, branchMode: BranchMode, @@ -177,7 +175,7 @@ object FlowLifecycle: * current feature branch. */ private[orca] def setup( - args: OrcaArgs, + args: SetupRequest, // The resolved coding-role agent (ADR 0020): branch-name resolution and // stack discovery — the run's privileged pre-body model calls — run here. agent: Agent[?], @@ -192,6 +190,13 @@ object FlowLifecycle: // Stamped into a freshly-written header (`freshRun`). flowSource: Option[FlowSource], emit: OrcaEvent => Unit, + // Where the run happens when it is an orca worktree (`WorktreeRun`): the + // closing summary names it. + worktree: Option[os.Path], + fs: FileSystem, + // Whether a command word resolves on PATH in a directory, for checking + // discovered stack commands (`PathProbe.resolves` in production). + onPath: (String, os.Path) => Boolean, // The dirty-tree prompt's two terminal dependencies, injected so tests // (and any headless caller) decide without one. Production probes real // stdin and prompts on stderr; `ask` takes the dirty-file count, which @@ -199,8 +204,8 @@ object FlowLifecycle: // Both streams of the exchange must be a terminal: the menu is printed // on stderr, so `orca ... 2> run.log` from an interactive shell has a tty // stdin but nowhere visible to ask. - tty: () => Boolean = () => TtyProbe.stdin() && TtyProbe.stderr(), - ask: Int => DirtyTreeChoice = DirtyTreePolicy.promptOnStderr + tty: () => Boolean, + ask: Int => DirtyTreeChoice ): FlowSetup = val preflight = SetupPreflight.run( args, @@ -209,6 +214,7 @@ object FlowLifecycle: resolution, store, emit, + fs, tty, ask ) @@ -228,7 +234,8 @@ object FlowLifecycle: // Discovery (ADR 0019) is sequenced after the stash, which would sweep a // just-written untracked file straight back out of the tree, and before // binding, so a failed discovery leaves no branch or header behind. - val stack = resolveStackSettings(agent, workDir, resolution, emit) + val stack = + resolveStackSettings(agent, workDir, resolution, emit, fs, onPath) val binding = session.bindBranch(preflight.startingHead, preflight.protectedBranches) emit(OrcaEvent.BranchBound(binding.featureBranch)) @@ -247,10 +254,7 @@ object FlowLifecycle: binding.branchMode, startingTree, binding.startingCommit, - // From where the run IS, not from the flag: the shell relaunches a resume - // inside the worktree its log was found in WITHOUT `--worktree` (the flag - // would re-derive a path), and that run has to name the tree too. - Option.when(WorktreeRun.isWorktreeRun(workDir))(workDir), + worktree, binding.prReadiness ) @@ -260,7 +264,7 @@ object FlowLifecycle: * every call site. */ private final class SetupSession( - args: OrcaArgs, + args: SetupRequest, agent: Agent[?], git: RuntimeGit, branchNaming: Option[BranchNamingStrategy], @@ -348,8 +352,7 @@ object FlowLifecycle: * logger keeps the DEBUG trace. */ private def warnCorruptLog(reason: String): Unit = - val log = LoggerFactory.getLogger("orca.flow") - log.warn( + flowLog.warn( s"progress log at ${store.path} is corrupt ($reason); starting fresh" ) emit( @@ -552,7 +555,9 @@ object FlowLifecycle: agent: Agent[?], workDir: os.Path, resolution: SettingsResolution, - emit: OrcaEvent => Unit + emit: OrcaEvent => Unit, + fs: FileSystem, + onPath: (String, os.Path) => Boolean )(using InStage): StackOutcome = resolution match case SettingsResolution.Resolved(settings) => @@ -562,12 +567,19 @@ object FlowLifecycle: case SettingsResolution.NeedsDiscovery(existingContent) => val (settings, entries) = withActivity("Discovering format, lint & test commands", emit): - StackDiscovery.discover(agent, workDir, emit, existingContent) + StackDiscovery.discover( + agent, + workDir, + emit, + existingContent, + fs, + onPath + ) val fileText = existingContent match - case None => SettingsFile.render(entries) + case None => SettingsFile.render(entries) case Some(content) => content + "\n" + SettingsFile.renderAppend(entries) - OrcaDir.settingsFile(workDir).replace(fileText) + OrcaDir(fs).settingsFile(workDir).replace(fileText) emit( OrcaEvent.Step( "written to .orca/settings.properties — review and edit as needed." @@ -598,7 +610,7 @@ object FlowLifecycle: * untouched). Content is captured pre-stash so a hand-written file the stash * sweeps out of a dirty tree is not lost. */ - private[runner] enum SettingsResolution: + private[orca] enum SettingsResolution: case Overridden(settings: StackSettings) case Resolved(settings: StackSettings) case NeedsDiscovery(existingContent: Option[String]) @@ -607,7 +619,7 @@ object FlowLifecycle: * resolution, plus the project- and user-global agent keys kept separate so * `runFlow` can track each role's source for the announcement `Step`. */ - private[runner] case class SettingsRead( + private[orca] case class SettingsRead( stack: SettingsResolution, projectAgents: AgentSettings, globalAgents: AgentSettings @@ -629,6 +641,7 @@ object FlowLifecycle: * resolves; an absent, blank, or stack-silent file needs discovery. */ private[orca] def readSettings( + fs: FileSystem, workDir: os.Path, globalSettingsPath: os.Path, stackOverride: Option[StackSettings] @@ -637,18 +650,18 @@ object FlowLifecycle: // A repo can commit `.orca/settings.properties` as a symlink pointing // outside the tree, which `os.read` follows. Refused here, before any tree // mutation, so the abort leaves the current branch untouched. - OrcaDir.assertNoOrcaSymlinks(workDir, projectPath) + OrcaDir(fs).assertNoOrcaSymlinks(workDir, projectPath) val projectContent: Option[String] = - if os.exists(projectPath) then Some(readOrAbort(projectPath)) + if fs.exists(projectPath) then Some(readOrAbort(fs, projectPath)) else None val projectParsed = projectContent.map(c => parseOrAbort(c, SettingsScope.Project, projectPath) ) val globalAgents: AgentSettings = - if os.exists(globalSettingsPath) then + if fs.exists(globalSettingsPath) then parseOrAbort( - readOrAbort(globalSettingsPath), + readOrAbort(fs, globalSettingsPath), SettingsScope.UserGlobal, globalSettingsPath ).agents @@ -658,17 +671,17 @@ object FlowLifecycle: val stack: SettingsResolution = stackOverride match case Some(settings) => SettingsResolution.Overridden(settings) - case None => + case None => projectParsed.flatMap(_.stack) match case Some(settings) => SettingsResolution.Resolved(settings) - case None => + case None => SettingsResolution.NeedsDiscovery( projectContent.filterNot(_.isBlank) ) SettingsRead(stack, projectAgents, globalAgents) - private def readOrAbort(path: os.Path): String = - try os.read(path) + private def readOrAbort(fs: FileSystem, path: os.Path): String = + try fs.read(path) catch case NonFatal(e) => throw new OrcaFlowException( @@ -681,7 +694,7 @@ object FlowLifecycle: path: os.Path ): orca.settings.ParsedSettings = SettingsFile.parse(content, scope) match - case Right(s) => s + case Right(s) => s case Left(err) => throw new OrcaFlowException( s"invalid settings at $path: ${err.message}" @@ -708,7 +721,7 @@ object FlowLifecycle: val fallback = resolveFallback(userPrompt, protectedBranches) val protectionChecked = FeatureBranch.resolve(resolvedName, protectedBranches) match - case Right(featureBranch) => featureBranch + case Right(featureBranch) => featureBranch case Left(ProtectedBranchRefused(name)) => emit( OrcaEvent.Step( @@ -778,7 +791,7 @@ object FlowLifecycle: ) case Head.OnBranch(current) => FeatureBranch.resolveReused(current, protectedBranches) match - case Right(featureBranch) => featureBranch + case Right(featureBranch) => featureBranch case Left(ProtectedBranchRefused(name)) => throw new OrcaFlowException( s"cannot skip branch creation: '$name' is a protected branch — " + @@ -798,7 +811,7 @@ object FlowLifecycle: val fallbackName = BranchNamingStrategy.flowFallbackName(userPrompt) FeatureBranch.resolve(fallbackName, protectedBranches) match case Right(featureBranch) => featureBranch - case Left(_) => + case Left(_) => throw new OrcaFlowException( s"internal error: deterministic fallback branch name " + s"'$fallbackName' is itself a protected branch" @@ -831,7 +844,7 @@ object FlowLifecycle: ) git.createBranch(candidate) match case Right(()) => candidate - case Left(_) => + case Left(_) => if fallback.value == candidate.value then doubleCollisionAbort(fallback.value) else @@ -858,11 +871,10 @@ object FlowLifecycle: * escaping teardown. */ private def bestEffortRead[T](what: String)(op: => Option[T]): Option[T] = - val log = LoggerFactory.getLogger("orca.flow") try op catch case NonFatal(e) => - log.debug(s"teardownSuccess: $what failed (cosmetic, swallowed)", e) + flowLog.debug(s"teardownSuccess: $what failed (cosmetic, swallowed)", e) None /** Successful teardown (ADR 0018 §2.5): remove the progress-log file in a @@ -881,7 +893,7 @@ object FlowLifecycle: * branch the user is left on — which is why [[RunChanges]] carries the * branch it was counted on. */ - private[runner] def teardownSuccess( + private[orca] def teardownSuccess( git: RuntimeGit, setup: FlowSetup, published: PublishedState, diff --git a/runner/src/main/scala/orca/runner/GitPreconditions.scala b/lifecycle/src/main/scala/orca/lifecycle/GitPreconditions.scala similarity index 92% rename from runner/src/main/scala/orca/runner/GitPreconditions.scala rename to lifecycle/src/main/scala/orca/lifecycle/GitPreconditions.scala index 336335e0d..5184847e4 100644 --- a/runner/src/main/scala/orca/runner/GitPreconditions.scala +++ b/lifecycle/src/main/scala/orca/lifecycle/GitPreconditions.scala @@ -1,4 +1,4 @@ -package orca.runner +package orca.lifecycle /** What every entry point needs of git before a run can start, worded once. * @@ -7,7 +7,7 @@ package orca.runner * hanging the wording off the later and much larger of the two would point the * dependency backwards from the order things run in. */ -private[runner] object GitPreconditions: +private[lifecycle] object GitPreconditions: /** Covers both "not a git repository" and "a repository with no commits" — * the same next step fixes either, and a caller can rarely tell them apart diff --git a/runner/src/main/scala/orca/runner/PublishedState.scala b/lifecycle/src/main/scala/orca/lifecycle/PublishedState.scala similarity index 93% rename from runner/src/main/scala/orca/runner/PublishedState.scala rename to lifecycle/src/main/scala/orca/lifecycle/PublishedState.scala index 0517993f9..ba128b12f 100644 --- a/runner/src/main/scala/orca/runner/PublishedState.scala +++ b/lifecycle/src/main/scala/orca/lifecycle/PublishedState.scala @@ -1,4 +1,4 @@ -package orca.runner +package orca.lifecycle import orca.json.JsonFile import orca.progress.{ProgressLog, PublishedWork} @@ -11,7 +11,7 @@ import orca.progress.{ProgressLog, PublishedWork} * published work was pushed from cannot be recovered, while keeping one * nothing points at costs a stale branch. */ -private[runner] enum PublishedState: +private[orca] enum PublishedState: case Published(where: PublishedWork) case NotPublished case Unknown @@ -31,7 +31,7 @@ private[runner] enum PublishedState: case NotPublished => true case Published(_) | Unknown => false -private[runner] object PublishedState: +private[orca] object PublishedState: def from(result: JsonFile.Read[ProgressLog]): PublishedState = result match case JsonFile.Read.Loaded(log) => diff --git a/runner/src/main/scala/orca/runner/RoleAgents.scala b/lifecycle/src/main/scala/orca/lifecycle/RoleAgents.scala similarity index 96% rename from runner/src/main/scala/orca/runner/RoleAgents.scala rename to lifecycle/src/main/scala/orca/lifecycle/RoleAgents.scala index 505a9108f..bba22c34d 100644 --- a/runner/src/main/scala/orca/runner/RoleAgents.scala +++ b/lifecycle/src/main/scala/orca/lifecycle/RoleAgents.scala @@ -1,16 +1,18 @@ -package orca.runner +package orca.lifecycle import orca.AgentSet import orca.agents.Agent import orca.review.ReviewerPrompts import orca.settings.{AgentSettings, AgentSpec} -import ox.{ResourceScope, tap} +import orca.concurrency.releaseAfterScope +import orca.safe.prelude.tap +import ox.ResourceScope /** The three role agents resolved for one run — every field an existentially * typed [[Agent]] since planning/coding/review can each land on a different * backend. */ -private[runner] case class ResolvedRoles( +private[orca] case class ResolvedRoles( planning: Agent[?], coding: Agent[?], review: Agent[?] @@ -20,7 +22,7 @@ private[runner] case class ResolvedRoles( * role-announcement `Step`'s `(source)` suffix; the winning [[AgentSpec]] (if * any) drives the `harness[:model]` part. */ -private[runner] enum RoleSource: +private[lifecycle] enum RoleSource: case Override, Project, Global, Default /** The three per-role programmatic overrides passed to `flow(...)` — selector- @@ -78,7 +80,7 @@ private[orca] object RoleAgents: project: AgentSettings, global: AgentSettings, overrides: RoleOverrides, - agents: WiredAgents + agents: AgentSet )(using ResourceScope): RoleResolution = val planning = resolveOne( @@ -117,7 +119,7 @@ private[orca] object RoleAgents: ) private def closeIfForeign(c: RoleChoice)(using ResourceScope): Unit = - if c.foreign then WiredAgents.closeAfterScope(List(c.agent)) + if c.foreign then releaseAfterScope(c.agent.close()) /** One role's resolved agent plus the provenance the announcement reads. * `harness`/`model` are precomputed at resolution time (the only place that @@ -157,7 +159,7 @@ private[orca] object RoleAgents: projectSpec: Option[AgentSpec], globalSpec: Option[AgentSpec], overrideSelect: Option[AgentSet => Agent[?]], - agents: WiredAgents + agents: AgentSet ): RoleChoice = val choice = overrideSelect match case Some(select) => diff --git a/runner/src/main/scala/orca/runner/SetupPreflight.scala b/lifecycle/src/main/scala/orca/lifecycle/SetupPreflight.scala similarity index 91% rename from runner/src/main/scala/orca/runner/SetupPreflight.scala rename to lifecycle/src/main/scala/orca/lifecycle/SetupPreflight.scala index 3de3c6fec..ada0c71bf 100644 --- a/runner/src/main/scala/orca/runner/SetupPreflight.scala +++ b/lifecycle/src/main/scala/orca/lifecycle/SetupPreflight.scala @@ -1,6 +1,7 @@ -package orca.runner +package orca.lifecycle -import orca.{OrcaArgs, OrcaDir, OrcaFlowException} +import orca.{OrcaDir, OrcaFlowException} +import orca.effect.FileSystem import orca.events.OrcaEvent import orca.gitref.{BranchName, Head} import orca.progress.{ @@ -10,16 +11,15 @@ import orca.progress.{ ProgressStore, ScannedProgressLog } -import orca.runner.FlowLifecycle.SettingsResolution +import orca.lifecycle.FlowLifecycle.SettingsResolution import orca.tools.RuntimeGit +import orca.safe.prelude.NonFatal import orca.util.TextUtil -import scala.util.control.NonFatal - /** What [[SetupPreflight.run]] established. `peeked` is this run's own log as * it was before the stash, kept only to be restored ([[PeekedLog]]). */ -private[runner] case class Preflight( +private[lifecycle] case class Preflight( startingHead: Head, // The protected set both binding arms enforce: the always-protected floor // (`main`/`master`) plus the repo's detected default branch. @@ -30,14 +30,14 @@ private[runner] case class Preflight( ) /** The cleanliness verdict once refusal is off the table. */ -private[runner] enum TreePlan: +private[lifecycle] enum TreePlan: case Stash, Keep /** [[FlowLifecycle.setup]]'s first phase: reads only, so every refusal leaves * the tree and branch as the user left them. Nothing here holds a capability * token. */ -private[runner] object SetupPreflight: +private[lifecycle] object SetupPreflight: /** Every read-only check and refusal of [[FlowLifecycle.setup]], ahead of any * mutation. The dirty-tree question comes last, so a user who answered it is @@ -45,12 +45,13 @@ private[runner] object SetupPreflight: * may be stale by the time a long-open prompt is answered. */ def run( - args: OrcaArgs, + args: SetupRequest, git: RuntimeGit, workDir: os.Path, stack: SettingsResolution, store: ProgressStore, emit: OrcaEvent => Unit, + fs: FileSystem, tty: () => Boolean, ask: Int => DirtyTreeChoice ): Preflight = @@ -64,7 +65,7 @@ private[runner] object SetupPreflight: .fold(reason => throw unreadableLog(store.path, reason), identity) startingHead match case Head.OnBranch(branch) => - abortIfBranchBusy(peeked, store.path, workDir, branch) + abortIfBranchBusy(fs, peeked, store.path, workDir, branch) // No branch for another run to have claimed. case Head.Detached(_) => () // No recorded default branch leaves just the floor. @@ -89,7 +90,7 @@ private[runner] object SetupPreflight: * turns the decision away from every keep path. */ private def decideTree( - args: OrcaArgs, + args: SetupRequest, peeked: PeekedLog, dirtyCount: Int, tty: () => Boolean, @@ -114,7 +115,7 @@ private[runner] object SetupPreflight: "or re-run with --keep-changes to leave them in place" ) - private[runner] def unreadableLog( + private[lifecycle] def unreadableLog( path: os.Path, reason: String ): OrcaFlowException = @@ -138,7 +139,7 @@ private[runner] object SetupPreflight: * the branch. */ private def abortIfRequestedBranchRefused( - args: OrcaArgs, + args: SetupRequest, peeked: PeekedLog, git: RuntimeGit, protectedBranches: Set[String] @@ -170,6 +171,7 @@ private[runner] object SetupPreflight: * which is fine here: this check only ever refuses. */ private def abortIfBranchBusy( + fs: FileSystem, peeked: PeekedLog, ownPath: os.Path, workDir: os.Path, @@ -177,8 +179,8 @@ private[runner] object SetupPreflight: ): Unit = peeked match case PeekedLog.Parseable(_) => () - case _ => - busyBranchLog(ownPath, workDir, startingBranch).foreach: log => + case _ => + busyBranchLog(fs, ownPath, workDir, startingBranch).foreach: log => throw new OrcaFlowException( branchBusyMessage(log, workDir, startingBranch) ) @@ -191,15 +193,19 @@ private[runner] object SetupPreflight: * that is otherwise fine. */ private def busyBranchLog( + fs: FileSystem, ownPath: os.Path, workDir: os.Path, startingBranch: BranchName ): Option[ScannedProgressLog] = try - ProgressScan + ProgressScan(fs) .progressLogs(workDir) .filter(l => l.path != ownPath && l.header.branch == startingBranch) - .maxByOption(l => os.mtime(l.path)) + // Read before `maxByOption`, whose function argument must be pure. + .map(l => (l, fs.modifiedAt(l.path))) + .maxByOption(_._2) + .map(_._1) catch case NonFatal(_) => None /** Identifies the run being refused — its recorded prompt and flow, plus the @@ -264,7 +270,7 @@ private[runner] object SetupPreflight: ) /** `name` as a [[FeatureBranch]]; throws when it is protected. */ - private[runner] def requestedBranch( + private[lifecycle] def requestedBranch( name: BranchName, protectedBranches: Set[String] ): FeatureBranch = @@ -272,13 +278,13 @@ private[runner] object SetupPreflight: // valid user names such as `feature/JIRA-123`. FeatureBranch.resolveReused(name, protectedBranches) match case Right(featureBranch) => featureBranch - case Left(refused) => + case Left(refused) => throw new OrcaFlowException( s"--branch '${refused.name}' is a protected branch in this repo — " + "pick another --branch name" ) - private[runner] def requestedBranchExists( + private[lifecycle] def requestedBranchExists( name: BranchName ): OrcaFlowException = new OrcaFlowException( diff --git a/lifecycle/src/main/scala/orca/lifecycle/SetupRequest.scala b/lifecycle/src/main/scala/orca/lifecycle/SetupRequest.scala new file mode 100644 index 000000000..555ce527f --- /dev/null +++ b/lifecycle/src/main/scala/orca/lifecycle/SetupRequest.scala @@ -0,0 +1,13 @@ +package orca.lifecycle + +import orca.RunTarget +import orca.gitref.BranchName +import orca.tools.PrReadiness + +/** The parts of a run's `OrcaArgs` that [[FlowLifecycle.setup]] reads. */ +private[orca] case class SetupRequest( + userPrompt: String, + target: RunTarget, + branch: Option[BranchName], + prReadiness: PrReadiness +) diff --git a/runner/src/main/scala/orca/runner/StackDiscovery.scala b/lifecycle/src/main/scala/orca/lifecycle/StackDiscovery.scala similarity index 89% rename from runner/src/main/scala/orca/runner/StackDiscovery.scala rename to lifecycle/src/main/scala/orca/lifecycle/StackDiscovery.scala index ac8763cb7..a79d33f3f 100644 --- a/runner/src/main/scala/orca/runner/StackDiscovery.scala +++ b/lifecycle/src/main/scala/orca/lifecycle/StackDiscovery.scala @@ -1,18 +1,19 @@ -package orca.runner +package orca.lifecycle import orca.{InStage, StackSettings} +import orca.effect.FileSystem import orca.agents.{Agent, Announce, PromptEvent} import orca.events.OrcaEvent import orca.json.{JsonData, given} import orca.settings.{SettingsEntry, StackCommand, StackKey, StackValue} -import orca.subprocess.PathProbe +import orca.safe.prelude.{subPath, given} import orca.util.{PromptResource, TextUtil} /** One command the discovery agent proposes for a gate, with the repo-relative * file that justifies it and an optional free-text note (key/line/why) — the * evidence that makes the written settings file reviewable (ADR 0019). */ -private[runner] case class DiscoveredCommand( +private[orca] case class DiscoveredCommand( command: String, evidencePath: String, evidenceNote: Option[String] = None @@ -24,7 +25,7 @@ private[runner] case class DiscoveredCommand( * `"unsetReason": null` for whichever side doesn't apply. The Scala-side * defaults keep the jsoniter parse lenient about a genuinely omitted field. */ -private[runner] case class DiscoveredGate( +private[orca] case class DiscoveredGate( commands: List[DiscoveredCommand] = Nil, unsetReason: Option[String] = None ) derives JsonData @@ -33,7 +34,7 @@ private[runner] case class DiscoveredGate( * [[orca.StackSettings]] (ADR 0019). Sent over the wire inside * [[StackDiscoveryReply]]. */ -private[runner] case class StackDiscoveryResult( +private[orca] case class StackDiscoveryResult( format: DiscoveredGate, lint: DiscoveredGate, test: DiscoveredGate @@ -47,10 +48,10 @@ private[runner] case class StackDiscoveryResult( * matches every other cheap-tier payload in production * (`SelectedReviewers.names`, `ReviewResult.findings`). */ -private[runner] case class StackDiscoveryReply(result: StackDiscoveryResult) +private[orca] case class StackDiscoveryReply(result: StackDiscoveryResult) derives JsonData -private[runner] object StackDiscoveryReply: +private[orca] object StackDiscoveryReply: /** Silent — discovery narrates each written line as its own `Step`, so an * auto-announced payload would compete. */ @@ -61,13 +62,13 @@ private[runner] object StackDiscoveryReply: * mechanical checks demote unresolvable ones to commented-out lines; the * assembled entries render into `.orca/settings.properties`. */ -private[runner] object StackDiscovery: +private[lifecycle] object StackDiscovery: /** The principle-based discovery prompt (no real stack→command examples, one * fictional-stack example only — ADR 0019). */ - private[runner] val Prompt: String = - PromptResource.load("/orca/runner/prompts/stack-discovery.md") + private[lifecycle] val Prompt: String = + PromptResource.load("/orca/lifecycle/prompts/stack-discovery.md") /** Run discovery end to end (ADR 0019 § Auto-discovery). The cheap-tier agent * run is read-only — it inspects the repo with its own tools; nothing is @@ -86,7 +87,9 @@ private[runner] object StackDiscovery: agent: Agent[?], workDir: os.Path, emit: OrcaEvent => Unit, - existingContent: Option[String] + existingContent: Option[String], + fs: FileSystem, + onPath: (String, os.Path) => Boolean )(using InStage): (StackSettings, List[SettingsEntry]) = emit(OrcaEvent.Step(startMessage(existingContent))) val result = agent.cheap.withReadOnly @@ -96,8 +99,8 @@ private[runner] object StackDiscovery: .result val (entries, settings) = toEntries( result, - unresolvedReason(_, workDir), - evidenceExists(_, workDir) + unresolvedReason(onPath, _, workDir), + evidenceExists(fs, _, workDir) ) narrateEntries(entries, emit) warnDisabledGates(settings, emit) @@ -108,10 +111,10 @@ private[runner] object StackDiscovery: * two cases `FlowLifecycle.readSettings` funnels into `NeedsDiscovery`, told * apart by whether it captured any `existingContent`. */ - private[runner] def startMessage(existingContent: Option[String]): String = + private[lifecycle] def startMessage(existingContent: Option[String]): String = val purpose = "discovering how to format, lint & test this project" existingContent match - case None => s"no .orca/settings.properties — $purpose" + case None => s"no .orca/settings.properties — $purpose" case Some(_) => s".orca/settings.properties configures no stack keys — $purpose" @@ -170,7 +173,8 @@ private[runner] object StackDiscovery: * stage-time `bash -c` inherits). cwd is `workDir`, so a repo-relative path * like `./script.sh` resolves as it will at stage time. */ - private[runner] def unresolvedReason( + private[lifecycle] def unresolvedReason( + onPath: (String, os.Path) => Boolean, command: String, workDir: os.Path ): Option[String] = @@ -179,9 +183,9 @@ private[runner] object StackDiscovery: // visible demoted line in the settings file, hand-fixable. val words = command.trim.split("\\s+").toList.filter(_.nonEmpty) words.dropWhile(w => AssignmentToken.findPrefixOf(w).isDefined) match - case Nil => Some("empty command") + case Nil => Some("empty command") case word :: _ => - if PathProbe.resolves(word, workDir) then None + if onPath(word, workDir) then None else Some(s"$word: not found on PATH") /** Second mechanical check (ADR 0019): the cited evidence file must exist @@ -189,9 +193,12 @@ private[runner] object StackDiscovery: * (`os.SubPath` refuses both), so an agent-proposed citation can only ever * probe inside `workDir`. */ - private[runner] def evidenceExists(path: String, workDir: os.Path): Boolean = - try os.exists(workDir / os.SubPath(path)) - catch case _: IllegalArgumentException => false + private[lifecycle] def evidenceExists( + fs: FileSystem, + path: String, + workDir: os.Path + ): Boolean = + subPath(path).exists(p => fs.exists(workDir / p)) /** Assemble the agent's `result` into settings-file entries plus the * [[StackSettings]] the run uses. The two mechanical checks are injected as @@ -235,7 +242,7 @@ private[runner] object StackDiscovery: case StackValue.Run(command) => demotionReason(command, cmd.evidencePath) match case Some(reason) => demoted(reason) - case None => + case None => SettingsEntry.Command( key, command, diff --git a/runner/src/main/scala/orca/runner/StartingTree.scala b/lifecycle/src/main/scala/orca/lifecycle/StartingTree.scala similarity index 95% rename from runner/src/main/scala/orca/runner/StartingTree.scala rename to lifecycle/src/main/scala/orca/lifecycle/StartingTree.scala index 49b29a0a8..ded95448e 100644 --- a/runner/src/main/scala/orca/runner/StartingTree.scala +++ b/lifecycle/src/main/scala/orca/lifecycle/StartingTree.scala @@ -1,4 +1,4 @@ -package orca.runner +package orca.lifecycle import orca.WorkspaceWrite import orca.events.OrcaEvent @@ -7,7 +7,7 @@ import orca.tools.{RuntimeGit, UncommittedSnapshot, UntrackedFiles} /** What the working tree held when the body started, which decides what failure * teardown may delete and what it puts back. */ -private[runner] enum StartingTree: +private[orca] enum StartingTree: /** Clean, or stashed clean by setup: every untracked file is the run's. */ case Clean @@ -60,7 +60,7 @@ private[runner] enum StartingTree: ) case Kept(None) | Clean => () -private[runner] object StartingTree: +private[orca] object StartingTree: /** The starting tree for setup's `untracked` verdict. Call after setup's last * commit, so a snapshot's base is where HEAD stays until the first stage @@ -74,10 +74,10 @@ private[runner] object StartingTree: )(using WorkspaceWrite): StartingTree = untracked match case UntrackedFiles.Remove => Clean - case UntrackedFiles.Keep => + case UntrackedFiles.Keep => git.snapshotUncommitted() match case Right(snapshot) => Kept(snapshot) - case Left(failed) => + case Left(failed) => emit( OrcaEvent.Step( "warning: could not snapshot the kept changes " + diff --git a/runner/src/main/scala/orca/runner/WorktreeRun.scala b/lifecycle/src/main/scala/orca/lifecycle/WorktreeRun.scala similarity index 78% rename from runner/src/main/scala/orca/runner/WorktreeRun.scala rename to lifecycle/src/main/scala/orca/lifecycle/WorktreeRun.scala index b44815116..07c48f6b6 100644 --- a/runner/src/main/scala/orca/runner/WorktreeRun.scala +++ b/lifecycle/src/main/scala/orca/lifecycle/WorktreeRun.scala @@ -1,7 +1,9 @@ -package orca.runner +package orca.lifecycle import orca.{OrcaDir, RunKey} +import orca.effect.{FileSystem, Processes} import orca.gitref.BranchName +import orca.safe.prelude.{up, given} import orca.tools.{ MainCheckoutFailure, StartBranchFailure, @@ -18,14 +20,13 @@ import orca.tools.{ * that is. (The shell's resume relaunch does not use the flag at all — it runs * in the directory the log was found in.) */ -private[orca] object WorktreeRun: +private[orca] final class WorktreeRun( + procs: Processes, + fs: FileSystem, + lock: WorktreeLock +): - /** What is at the derived path. A registered worktree whose directory is gone - * — what `git clean -xdff` leaves — counts as [[Absent]]: `Worktrees.add` - * reclaims that entry. - */ - private enum PathState: - case Reusable, Occupied, Absent + private val worktrees = Worktrees(procs, fs) /** The directory a `--worktree` run should use, created if it does not exist * yet. `invokingDir` is where orca was started, which may itself be a @@ -50,23 +51,24 @@ private[orca] object WorktreeRun: invokingDir: os.Path, key: RunKey ): Either[String, os.Path] = - Worktrees + worktrees .mainCheckoutOrReason(invokingDir) .left .map(noMainCheckout) .flatMap: mainCheckout => val path = OrcaDir.worktreesPath(mainCheckout) / key.value - FlowLock.worktreeLocked(mainCheckout, key): + lock.worktreeLocked(mainCheckout, key): pathState(invokingDir, path) match - case PathState.Reusable => reuse(mainCheckout, path) - case PathState.Occupied => + case WorktreeRun.PathState.Reusable => reuse(mainCheckout, path) + case WorktreeRun.PathState.Occupied => Left( s"$path already exists but is not a worktree of this " + "repository — orca will not take over a directory it did " + "not create" ) - case PathState.Absent => create(invokingDir, mainCheckout, path) + case WorktreeRun.PathState.Absent => + create(invokingDir, mainCheckout, path) /** What to tell the user when no main checkout could be named. Each case gets * its own sentence: telling someone whose repository is fine to `git init` @@ -98,10 +100,10 @@ private[orca] object WorktreeRun: mainCheckout: os.Path, path: os.Path ): Either[String, os.Path] = - val _ = OrcaDir.ensureWorktrees(mainCheckout) + val _ = OrcaDir(fs).ensureWorktrees(mainCheckout) // Anything but a definite branch goes through the repair, unreadable // included. - if Worktrees.onABranch(path) then Right(path) else bindBranch(path) + if worktrees.onABranch(path) then Right(path) else bindBranch(path) /** Whether `workDir` is a worktree orca made for its own repository — the * question the closing summary asks, since a run ends up in one either by @@ -110,14 +112,18 @@ private[orca] object WorktreeRun: * both routes answer the same. */ def isWorktreeRun(workDir: os.Path): Boolean = - Worktrees + worktrees .mainCheckout(workDir) - .exists(main => workDir / os.up == OrcaDir.worktreesPath(main)) + .exists(main => workDir / up == OrcaDir.worktreesPath(main)) - private def pathState(invokingDir: os.Path, path: os.Path): PathState = - if !os.exists(path) then PathState.Absent - else if Worktrees.list(invokingDir).contains(path) then PathState.Reusable - else PathState.Occupied + private def pathState( + invokingDir: os.Path, + path: os.Path + ): WorktreeRun.PathState = + if !fs.exists(path) then WorktreeRun.PathState.Absent + else if worktrees.list(invokingDir).contains(path) then + WorktreeRun.PathState.Reusable + else WorktreeRun.PathState.Occupied private def create( invokingDir: os.Path, @@ -127,9 +133,9 @@ private[orca] object WorktreeRun: // The ignore marker goes in before the worktree, never after: until it is // there, `git add -A` in the main checkout stages the new worktree as an // embedded git repository. - val _ = OrcaDir.ensureWorktrees(mainCheckout) - Worktrees.add(invokingDir, path) match - case Right(()) => bindBranch(path) + val _ = OrcaDir(fs).ensureWorktrees(mainCheckout) + worktrees.add(invokingDir, path) match + case Right(()) => bindBranch(path) case Left(WorktreeAddFailure.NoCommitsYet) => Left(GitPreconditions.needsRepoWithCommit) case Left(WorktreeAddFailure.GitFailed(message)) => @@ -152,8 +158,8 @@ private[orca] object WorktreeRun: path: os.Path, branch: BranchName ): Either[String, os.Path] = - Worktrees.startBranch(path, branch) match - case Right(()) => Right(path) + worktrees.startBranch(path, branch) match + case Right(()) => Right(path) case Left(StartBranchFailure.WouldLoseCommits(name)) => Left( s"branch '${name.value}' already has commits this run would not " + @@ -165,3 +171,19 @@ private[orca] object WorktreeRun: s"the worktree at $path exists but could not be put on branch " + s"'${branch.value}': $message" ) + +private[orca] object WorktreeRun: + + /** What is at the derived path. A registered worktree whose directory is gone + * — what `git clean -xdff` leaves — counts as [[Absent]]: `Worktrees.add` + * reclaims that entry. + */ + private[lifecycle] enum PathState: + case Reusable, Occupied, Absent + +/** Serialises `--worktree` resolution of one prompt across processes. */ +private[orca] trait WorktreeLock: + /** Runs `op` holding the lock for `key` in `mainCheckout`; throws + * [[orca.OrcaFlowException]] when another process holds it. + */ + def worktreeLocked[T](mainCheckout: os.Path, key: RunKey)(op: => T): T diff --git a/runner/src/test/scala/orca/runner/BranchHandoffTest.scala b/lifecycle/src/test/scala/orca/lifecycle/BranchHandoffTest.scala similarity index 98% rename from runner/src/test/scala/orca/runner/BranchHandoffTest.scala rename to lifecycle/src/test/scala/orca/lifecycle/BranchHandoffTest.scala index ed06c46be..acd70c7a7 100644 --- a/runner/src/test/scala/orca/runner/BranchHandoffTest.scala +++ b/lifecycle/src/test/scala/orca/lifecycle/BranchHandoffTest.scala @@ -1,4 +1,4 @@ -package orca.runner +package orca.lifecycle import orca.progress.{BranchMode, PublishedWork} diff --git a/runner/src/test/scala/orca/runner/ClosingSummaryTest.scala b/lifecycle/src/test/scala/orca/lifecycle/ClosingSummaryTest.scala similarity index 99% rename from runner/src/test/scala/orca/runner/ClosingSummaryTest.scala rename to lifecycle/src/test/scala/orca/lifecycle/ClosingSummaryTest.scala index 28f71b688..854cedff6 100644 --- a/runner/src/test/scala/orca/runner/ClosingSummaryTest.scala +++ b/lifecycle/src/test/scala/orca/lifecycle/ClosingSummaryTest.scala @@ -1,4 +1,4 @@ -package orca.runner +package orca.lifecycle import orca.gitref.{CommitHash, Head} import orca.progress.PublishedWork diff --git a/runner/src/test/scala/orca/runner/DirtyTreePolicyTest.scala b/lifecycle/src/test/scala/orca/lifecycle/DirtyTreePolicyTest.scala similarity index 69% rename from runner/src/test/scala/orca/runner/DirtyTreePolicyTest.scala rename to lifecycle/src/test/scala/orca/lifecycle/DirtyTreePolicyTest.scala index 8fe119504..fa2d9ecec 100644 --- a/runner/src/test/scala/orca/runner/DirtyTreePolicyTest.scala +++ b/lifecycle/src/test/scala/orca/lifecycle/DirtyTreePolicyTest.scala @@ -1,4 +1,4 @@ -package orca.runner +package orca.lifecycle import java.util.concurrent.atomic.AtomicInteger @@ -101,43 +101,3 @@ class DirtyTreePolicyTest extends munit.FunSuite: test("an unrecognized answer is invalid, not a silent stash"): assertEquals(DirtyTreePolicy.parse(Some("keep")), None) assertEquals(DirtyTreePolicy.parse(Some("22")), None) - - test("the production prompt asks on stderr, reads stdin, keeps stdout clean"): - val out = new java.io.ByteArrayOutputStream() - val err = new java.io.ByteArrayOutputStream() - val choice = Console.withIn(new java.io.StringReader("2\n")): - Console.withOut(out): - Console.withErr(err): - DirtyTreePolicy.promptOnStderr(dirtyCount = 2) - assertEquals(choice, DirtyTreeChoice.Keep) - val menu = err.toString("UTF-8") - assert(menu.contains("2 uncommitted/untracked file(s)"), menu) - assert( - menu.contains("1)") && menu.contains("2)") && menu.contains("3)"), - s"all three options must be offered: $menu" - ) - assert( - menu.contains("kept changes to tracked files are lost"), - s"the keep option must warn about the failure-teardown data loss: $menu" - ) - assertEquals(out.toString("UTF-8"), "", "stdout must stay clean") - - test("the production prompt stashes when stdin is at EOF"): - val choice = Console.withIn(new java.io.StringReader("")): - Console.withErr(new java.io.ByteArrayOutputStream()): - DirtyTreePolicy.promptOnStderr(dirtyCount = 1) - assertEquals(choice, DirtyTreeChoice.Stash) - - test( - "the production prompt re-asks an unrecognized answer, then honors the next valid one" - ): - val err = new java.io.ByteArrayOutputStream() - val choice = Console.withIn(new java.io.StringReader("keep\n2\n")): - Console.withErr(err): - DirtyTreePolicy.promptOnStderr(dirtyCount = 1) - assertEquals(choice, DirtyTreeChoice.Keep) - val transcript = err.toString("UTF-8") - assert( - transcript.contains("unrecognized answer"), - s"the re-ask must say why it is asking again: $transcript" - ) diff --git a/runner/src/test/scala/orca/runner/RoleAgentsTest.scala b/lifecycle/src/test/scala/orca/lifecycle/RoleAgentsTest.scala similarity index 95% rename from runner/src/test/scala/orca/runner/RoleAgentsTest.scala rename to lifecycle/src/test/scala/orca/lifecycle/RoleAgentsTest.scala index aab4d25d1..2dd4a18a0 100644 --- a/runner/src/test/scala/orca/runner/RoleAgentsTest.scala +++ b/lifecycle/src/test/scala/orca/lifecycle/RoleAgentsTest.scala @@ -1,5 +1,6 @@ -package orca.runner +package orca.lifecycle +import orca.AgentSet import orca.agents.{ Agent, AgentConfig, @@ -16,9 +17,9 @@ import orca.settings.{AgentSettings, AgentSpec} import orca.testkit.{ScriptedBackend, TestAgent} /** Pins [[RoleAgents.resolveAll]]'s mapping from settings to the run's - * [[WiredAgents]]: unset stays claude, a bare spec picks the matching wired + * [[AgentSet]]: unset stays claude, a bare spec picks the matching wired * backend, and a model pin produces a `withModel` sibling that still shares - * the wired agent's backend — the same sharing [[LeadAgentIdentityTest]] pins + * the wired agent's backend — the same sharing `LeadAgentIdentityTest` pins * for the `_.claude.opus` selector shape, here exercised through * settings-driven resolution instead of a flow selector. * @@ -295,7 +296,7 @@ class RoleAgentsTest extends munit.FunSuite: project: AgentSettings, global: AgentSettings, overrides: RoleOverrides, - agents: WiredAgents + agents: AgentSet ): RoleResolution = ox.resourceScope( RoleAgents.resolveAll(project, global, overrides, agents) @@ -306,7 +307,7 @@ class RoleAgentsTest extends munit.FunSuite: */ private def resolvedRoles( settings: AgentSettings, - agents: WiredAgents + agents: AgentSet ): ResolvedRoles = resolveInScope( project = settings, @@ -332,5 +333,13 @@ class RoleAgentsTest extends munit.FunSuite: opencode: OpencodeAgent = stub(BackendTag.Opencode), pi: PiAgent = stub(BackendTag.Pi), gemini: GeminiAgent = stub(BackendTag.Gemini) - ): WiredAgents = - new WiredAgents(claude, codex, opencode, pi, gemini) + ): AgentSet = + WiredSet(claude, codex, opencode, pi, gemini) + + private final class WiredSet( + val claude: ClaudeAgent, + val codex: CodexAgent, + val opencode: OpencodeAgent, + val pi: PiAgent, + val gemini: GeminiAgent + ) extends AgentSet diff --git a/lifecycle/src/test/scala/orca/lifecycle/StackDiscoverySchemaParityTest.scala b/lifecycle/src/test/scala/orca/lifecycle/StackDiscoverySchemaParityTest.scala new file mode 100644 index 000000000..db29c09ac --- /dev/null +++ b/lifecycle/src/test/scala/orca/lifecycle/StackDiscoverySchemaParityTest.scala @@ -0,0 +1,10 @@ +package orca.lifecycle + +import orca.json.given + +@scala.annotation.experimental +class StackDiscoverySchemaParityTest extends orca.json.SchemaParitySuite: + parity[DiscoveredCommand] + parity[DiscoveredGate] + parity[StackDiscoveryResult] + parity[StackDiscoveryReply] diff --git a/runner/src/test/scala/orca/runner/StackDiscoveryTest.scala b/lifecycle/src/test/scala/orca/lifecycle/StackDiscoveryTest.scala similarity index 92% rename from runner/src/test/scala/orca/runner/StackDiscoveryTest.scala rename to lifecycle/src/test/scala/orca/lifecycle/StackDiscoveryTest.scala index 05cb0ce8c..79b55637b 100644 --- a/runner/src/test/scala/orca/runner/StackDiscoveryTest.scala +++ b/lifecycle/src/test/scala/orca/lifecycle/StackDiscoveryTest.scala @@ -1,7 +1,8 @@ -package orca.runner +package orca.lifecycle import com.github.plokhotnyuk.jsoniter_scala.core.readFromString import orca.StackSettings +import orca.subprocess.PathProbe import orca.json.JsonData import orca.settings.{ SettingsEntry, @@ -11,10 +12,12 @@ import orca.settings.{ StackKey, StackValue } -import orca.testkit.TempDirs +import orca.testkit.{Effects, TempDirs} class StackDiscoveryTest extends munit.FunSuite: + private val onPath = PathProbe.resolves(Effects.procs, _, _) + private def command(raw: String): StackCommand = StackValue.parse(raw) match case StackValue.Run(command) => command @@ -362,33 +365,38 @@ class StackDiscoveryTest extends munit.FunSuite: test("unresolvedReason: a plain word on PATH resolves"): assertEquals( - StackDiscovery.unresolvedReason("echo hello", TempDirs.dir()), + StackDiscovery.unresolvedReason(onPath, "echo hello", TempDirs.dir()), None ) test("unresolvedReason: an unknown word demotes, naming it"): assertEquals( StackDiscovery - .unresolvedReason("definitely-not-a-cmd-xyz --flag", TempDirs.dir()), + .unresolvedReason( + onPath, + "definitely-not-a-cmd-xyz --flag", + TempDirs.dir() + ), Some("definitely-not-a-cmd-xyz: not found on PATH") ) test("unresolvedReason: a leading VAR= assignment token is stripped"): assertEquals( - StackDiscovery.unresolvedReason("FOO=bar echo hi", TempDirs.dir()), + StackDiscovery + .unresolvedReason(onPath, "FOO=bar echo hi", TempDirs.dir()), None ) test("unresolvedReason: two leading VAR= assignment tokens are stripped"): assertEquals( StackDiscovery - .unresolvedReason("FOO=bar B_2=x echo hi", TempDirs.dir()), + .unresolvedReason(onPath, "FOO=bar B_2=x echo hi", TempDirs.dir()), None ) test("unresolvedReason: an all-assignments (empty) command demotes"): assertEquals( - StackDiscovery.unresolvedReason("FOO=bar", TempDirs.dir()), + StackDiscovery.unresolvedReason(onPath, "FOO=bar", TempDirs.dir()), Some("empty command") ) @@ -399,7 +407,8 @@ class StackDiscoveryTest extends munit.FunSuite: // the script text: `echo; touch marker` must not create `marker`, whatever // the resolution verdict on the (unresolvable) word `echo;` is. val dir = TempDirs.dir() - val result = StackDiscovery.unresolvedReason("echo; touch marker", dir) + val result = + StackDiscovery.unresolvedReason(onPath, "echo; touch marker", dir) assert( !os.exists(dir / "marker"), "the probe must not execute any part of the command" @@ -411,29 +420,39 @@ class StackDiscoveryTest extends munit.FunSuite: os.write(dir / "script.sh", "#!/bin/sh\n") os.perms.set(dir / "script.sh", "rwxr-xr-x") assertEquals( - StackDiscovery.unresolvedReason("./script.sh --check", dir), + StackDiscovery.unresolvedReason(onPath, "./script.sh --check", dir), None ) test("unresolvedReason: an absent ./script.sh demotes"): assertEquals( - StackDiscovery.unresolvedReason("./missing.sh", TempDirs.dir()), + StackDiscovery.unresolvedReason(onPath, "./missing.sh", TempDirs.dir()), Some("./missing.sh: not found on PATH") ) test("evidenceExists: a present repo-relative file is accepted"): val dir = TempDirs.dir() os.write(dir / "Cargo.toml", "[package]\n") - assert(StackDiscovery.evidenceExists("Cargo.toml", dir)) + assert(StackDiscovery.evidenceExists(Effects.fs, "Cargo.toml", dir)) test("evidenceExists: a missing repo-relative file is rejected"): - assert(!StackDiscovery.evidenceExists("Cargo.toml", TempDirs.dir())) + assert( + !StackDiscovery.evidenceExists( + Effects.fs, + "Cargo.toml", + TempDirs.dir() + ) + ) test("evidenceExists: ../ traversal and absolute paths are rejected"): val dir = TempDirs.dir() os.write(dir / "real.txt", "x") - assert(!StackDiscovery.evidenceExists("../escape.txt", dir)) - assert(!StackDiscovery.evidenceExists("/etc/passwd", dir)) + assert( + !StackDiscovery.evidenceExists(Effects.fs, "../escape.txt", dir) + ) + assert( + !StackDiscovery.evidenceExists(Effects.fs, "/etc/passwd", dir) + ) test("startMessage: absent/blank file names the missing file"): assertEquals( diff --git a/lifecycle/src/test/scala/orca/lifecycle/WorktreeRunTest.scala b/lifecycle/src/test/scala/orca/lifecycle/WorktreeRunTest.scala new file mode 100644 index 000000000..7e60a7ace --- /dev/null +++ b/lifecycle/src/test/scala/orca/lifecycle/WorktreeRunTest.scala @@ -0,0 +1,174 @@ +package orca.lifecycle + +import orca.{OrcaDir, RunKey} +import orca.progress.ProgressStore +import orca.testkit.{Effects, GitRepo, TempDirs} +import ox.discard + +class WorktreeRunTest extends munit.FunSuite: + + private val worktreeRun = + WorktreeRun(Effects.procs, Effects.fs, NoLock) + + private object NoLock extends WorktreeLock: + def worktreeLocked[T](mainCheckout: os.Path, key: RunKey)(op: => T): T = op + + test("the worktree lives under .orca/worktrees, keyed by the prompt"): + val repo = GitRepo.seeded() + val path = resolved(repo, "task A") + assertEquals(path / os.up, OrcaDir.worktreesPath(repo)) + assertNotEquals(path, resolved(repo, "task B")) + // Same key as the progress log inside it: that coupling is what makes a + // resumed run land where its log is. + assert( + ProgressStore + .default(Effects.fs, path, RunKey.of("task A")) + .path + .last + .contains(path.last), + path.last + ) + + test("re-resolving the same prompt reuses the worktree already there"): + val repo = GitRepo.seeded() + val first = resolved(repo, "task A") + assertEquals(resolved(repo, "task A"), first) + // The main checkout plus exactly one worktree: no second one was created. + assertEquals(Effects.worktrees.list(repo), List(repo, first)) + + test("a new worktree is put on a branch of its own, not left detached"): + val repo = GitRepo.seeded() + val path = resolved(repo, "task A") + assert(Effects.worktrees.onABranch(path)) + + test("resolving from inside the worktree lands on that same worktree"): + val repo = GitRepo.seeded() + val path = resolved(repo, "task A") + assertEquals(worktreeRun.resolve(path, RunKey.of("task A")), Right(path)) + + test("the ignore marker is written before the worktree is created"): + val repo = GitRepo.empty() + // Creation cannot succeed on an unborn HEAD, so the marker is on disk only + // if it was written first. + assert(worktreeRun.resolve(repo, RunKey.of("task A")).isLeft) + assert(os.exists(OrcaDir.worktreesPath(repo) / ".gitignore")) + + test("reuse puts the ignore marker back when it went missing"): + val repo = GitRepo.seeded() + val path = resolved(repo, "task A") + // What `git clean -xdf` in the main checkout removes: the marker only, the + // worktree itself skipped as a repository. + os.remove(OrcaDir.worktreesPath(repo) / ".gitignore").discard + assertEquals(worktreeRun.resolve(repo, RunKey.of("task A")), Right(path)) + assert(os.exists(OrcaDir.worktreesPath(repo) / ".gitignore")) + + test("a worktree whose directory was deleted is recreated"): + val repo = GitRepo.seeded() + val path = resolved(repo, "task A") + // What `git clean -xdff` leaves: the directory gone, git's administrative + // entry still listing it. + os.remove.all(path) + assertEquals(worktreeRun.resolve(repo, RunKey.of("task A")), Right(path)) + assert(os.exists(path / ".git")) + + test("recreating one worktree leaves the repository's others registered"): + val repo = GitRepo.seeded() + val path = resolved(repo, "task A") + val unrelated = repo / "unrelated" + assertEquals(Effects.worktrees.add(repo, unrelated), Right(())) + // Both directories unreachable — a stale entry orca does not own must + // survive the reclaim of the one it does. + os.remove.all(path) + os.remove.all(unrelated) + assertEquals(worktreeRun.resolve(repo, RunKey.of("task A")), Right(path)) + assert(Effects.worktrees.list(repo).contains(unrelated)) + + test("a plain directory at the path is refused, not taken over"): + val repo = GitRepo.seeded() + val path = resolved(repo, "task A") + git(repo, "worktree", "remove", "--force", path.toString) + os.makeDir.all(path) + val refusal = refusalOf(repo) + assert(refusal.contains(path.toString), refusal) + + test("outside a git repository, orca's own wording is used"): + assertEquals( + worktreeRun.resolve(TempDirs.dir(), RunKey.of("task A")), + Left(GitPreconditions.needsRepoWithCommit) + ) + + test("a repository without commits gets orca's wording, not git's"): + assertEquals( + worktreeRun.resolve(GitRepo.empty(), RunKey.of("task A")), + Left(GitPreconditions.needsRepoWithCommit) + ) + + test("a main worktree that is not a checkout is named as the reason"): + val checkout = GitRepo.seededSeparateGitDir() + val worktree = checkout / os.up / "wt" + assertEquals(Effects.worktrees.add(checkout, worktree), Right(())) + // Resolving from here finds no main checkout, and the user is told it is + // the repository's layout — not that they have no repository. + val refusal = refusalOf(worktree) + assert(refusal.contains("--separate-git-dir"), refusal) + + test("a git that does not answer the query is named as the reason"): + // rev-parse prints one path per line, so a repository path holding a + // newline makes the answer unreadable — as does a git too old for + // `--path-format`, which is what the user is pointed at. + val repo = TempDirs.dir() / "two\nlines" + os.makeDir.all(repo) + git(repo, "init", "-b", "main") + val refusal = refusalOf(repo) + assert(refusal.contains("git 2.31 or newer"), refusal) + + test("reuse puts a detached worktree back on its branch"): + val repo = GitRepo.seeded() + val path = resolved(repo, "task A") + // How a half-created worktree looks: `add` succeeded, the branch step did + // not. + git(path, "checkout", "--detach", "-q") + assert(!Effects.worktrees.onABranch(path)) + assertEquals(worktreeRun.resolve(repo, RunKey.of("task A")), Right(path)) + assert(Effects.worktrees.onABranch(path)) + + test("reuse refuses when the run's branch has gained commits"): + val repo = GitRepo.seeded() + val path = resolved(repo, "task A") + val branch = s"orca-worktree-${path.last}" + os.write(path / "work.txt", "work") + git(path, "add", "-A") + git(path, "commit", "-q", "-m", "work in the worktree") + // Detached BEHIND the branch, so putting the run back on it would strand + // that commit — which is what orca refuses to do. + git(path, "checkout", "--detach", "-q", "HEAD~1") + val refusal = refusalOf(repo) + assert(refusal.contains(branch), refusal) + // The remedy names the branch, which is what holds the commits. Removing + // the worktree does not help: the next run recreates it detached and hits + // this same refusal. + assert(refusal.contains(s"git branch -D $branch"), refusal) + + test("isWorktreeRun recognises orca's own worktree and nothing else"): + val repo = GitRepo.seeded() + val ours = resolved(repo, "task A") + val theirs = repo / "review-their-branch" + assertEquals(Effects.worktrees.add(repo, theirs), Right(())) + assert(worktreeRun.isWorktreeRun(ours), "orca's own worktree") + assert(!worktreeRun.isWorktreeRun(repo), "the main checkout is not one") + assert(!worktreeRun.isWorktreeRun(theirs), "a worktree orca did not make") + assert(!worktreeRun.isWorktreeRun(TempDirs.dir()), "not a repository") + + private def resolved(repo: os.Path, userPrompt: String): os.Path = + worktreeRun + .resolve(repo, RunKey.of(userPrompt)) + .getOrElse(fail("expected a resolved worktree")) + + private def refusalOf(cwd: os.Path): String = + worktreeRun + .resolve(cwd, RunKey.of("task A")) + .left + .getOrElse(fail("expected a refusal")) + + private def git(cwd: os.Path, args: String*): Unit = + os.proc("git" +: args).call(cwd = cwd).discard diff --git a/opencode/src/main/scala/orca/backend/opencode/JavaNetOpencodeHttp.scala b/opencode/src/main/scala/orca/backend/opencode/JavaNetOpencodeHttp.scala deleted file mode 100644 index 62f84218a..000000000 --- a/opencode/src/main/scala/orca/backend/opencode/JavaNetOpencodeHttp.scala +++ /dev/null @@ -1,129 +0,0 @@ -package orca.backend.opencode - -import orca.OrcaFlowException -import orca.backend.StreamSource - -import java.io.{BufferedReader, InputStreamReader} -import java.net.URI -import java.net.http.{HttpClient, HttpRequest, HttpResponse} -import java.nio.charset.StandardCharsets -import java.time.Duration -import java.util.Base64 -import ox.{sleep, tapException} -import scala.concurrent.duration.{DurationInt, FiniteDuration} -import scala.util.control.NonFatal - -/** [[OpencodeHttp]] over the JDK `java.net.http` client (ADR 0014). One - * instance per server; the SSE `events` stream is a lazy line iterator over - * the response body that `interrupt()` closes. - */ -private[opencode] object JavaNetOpencodeHttp: - /** Build a client for an already-listening server and block until `GET /doc` - * answers (the bind line can precede readiness — e.g. a first-run DB - * migration). Throws if it never becomes healthy. - */ - def start(baseUrl: String, password: String): OpencodeHttp = - val http = new JavaNetOpencodeHttp(baseUrl, password) - http - .awaitHealthy(attempts = 50, delay = 200.millis) - .tapException(_ => http.close()) - http - - /** Bounds a request/response call, so a hung server fails it rather than - * blocking the caller (e.g. a turn cancel's abort POST). - */ - private val RequestTimeout: Duration = Duration.ofSeconds(30) - - /** A health ping is retried, so a hung one should give way to the next. */ - private val PingTimeout: Duration = Duration.ofSeconds(1) - -private[opencode] class JavaNetOpencodeHttp(baseUrl: String, password: String) - extends OpencodeHttp: - - // JDK java.net.http, not sttp (whose client isn't a dependency — only - // sttp-apispec/tapir are). Pinned to HTTP/1.1: the server hangs on the h2c - // upgrade for a POST-with-body (a GET upgrades fine), wedging the first - // `POST /session`. - private val client: HttpClient = - HttpClient.newBuilder().version(HttpClient.Version.HTTP_1_1).build() - - // Preemptive Basic auth set by hand (java.net.http's `Authenticator` is - // challenge/response only). User is the server default "opencode"; the - // password reached the server via our env on spawn. - private val authHeader: String = - "Basic " + Base64.getEncoder.encodeToString( - s"opencode:$password".getBytes(StandardCharsets.UTF_8) - ) - - def postJson(path: String, body: String): String = - val req = request(path) - .timeout(JavaNetOpencodeHttp.RequestTimeout) - .header("Content-Type", "application/json") - .POST(HttpRequest.BodyPublishers.ofString(body)) - .build() - val resp = client.send(req, HttpResponse.BodyHandlers.ofString()) - if resp.statusCode() / 100 == 2 then resp.body() - else - throw OrcaFlowException( - s"opencode POST $path failed: ${resp.statusCode()} ${resp.body()}" - ) - - override def getStatus(path: String): Int = - try - val req = - request(path).timeout(JavaNetOpencodeHttp.RequestTimeout).GET().build() - client.send(req, HttpResponse.BodyHandlers.discarding()).statusCode() - catch case NonFatal(_) => 0 - - def events(): StreamSource = - // No timeout: on newer JDKs it also bounds reading the body, and this - // stream stays open for the whole turn. - val req = request("/event").GET().build() - // `ofInputStream` returns once headers arrive; we read lines off the raw - // body ourselves. Closing the InputStream reliably unblocks a thread parked - // in `readLine()` on the open-ended SSE stream — `ofLines().close()` does - // not, which deadlocks the reader at turn end. - val body = - client.send(req, HttpResponse.BodyHandlers.ofInputStream()).body() - val reader = - new BufferedReader(new InputStreamReader(body, StandardCharsets.UTF_8)) - new StreamSource: - def lines: Iterator[String] = - Iterator.continually(reader.readLine()).takeWhile(_ != null) - def errorLines: Iterator[String] = Iterator.empty - def interrupt(): Unit = body.close() - def tryExitCode: Option[Int] = Some(0) - - /** Forceful shutdown — does not wait for in-flight requests (an SSE stream - * may still be open at teardown), so `shutdownNow` over the blocking - * `close`. - */ - override def close(): Unit = client.shutdownNow() - - /** Poll `GET /doc` until it answers 200 or attempts run out, sleeping only - * between attempts. - */ - private def awaitHealthy(attempts: Int, delay: FiniteDuration): Unit = - val healthy = Iterator - .range(0, attempts) - .exists: attempt => - if attempt > 0 then sleep(delay) - pingOk() - if !healthy then - throw OrcaFlowException( - s"opencode server at $baseUrl never became healthy" - ) - - private def pingOk(): Boolean = - try - val req = - request("/doc").timeout(JavaNetOpencodeHttp.PingTimeout).GET().build() - client - .send(req, HttpResponse.BodyHandlers.discarding()) - .statusCode() == 200 - catch case NonFatal(_) => false - - private def request(path: String): HttpRequest.Builder = - HttpRequest - .newBuilder(URI.create(baseUrl + path)) - .header("Authorization", authHeader) diff --git a/opencode/src/main/scala/orca/backend/opencode/NetworkOpencodeHttp.scala b/opencode/src/main/scala/orca/backend/opencode/NetworkOpencodeHttp.scala new file mode 100644 index 000000000..ba49e3eee --- /dev/null +++ b/opencode/src/main/scala/orca/backend/opencode/NetworkOpencodeHttp.scala @@ -0,0 +1,81 @@ +package orca.backend.opencode + +import orca.OrcaFlowException +import orca.backend.StreamSource +import orca.effect.{HttpClient, Network} +import orca.concurrency.sleep +import orca.safe.prelude.{DurationInt, NonFatal} +import ox.ResourceScope + +import scala.concurrent.duration.FiniteDuration + +/** [[OpencodeHttp]] over [[Network.httpClient]] (ADR 0014). */ +private[opencode] object NetworkOpencodeHttp: + + /** Bounds a request/response call, so a hung server fails it rather than + * blocking the caller (e.g. a turn cancel's abort POST). + */ + private val RequestTimeout: FiniteDuration = 30.seconds + + /** A health ping is retried, so a hung one should give way to the next. */ + private val PingTimeout: FiniteDuration = 1.second + + /** A client for an already-listening server, once `GET /doc` answers (the + * bind line can precede readiness, e.g. a first-run DB migration). Throws if + * it never does. The client lives until the enclosing scope ends. + */ + def start(net: Network, baseUrl: String, password: String)(using + ResourceScope + ): OpencodeHttp = + // User is the server default; the password reached it via the spawn env. + connect(net.httpClient(baseUrl, "opencode", password), baseUrl) + + /** [[start]] over an already-built `client`. */ + private[opencode] def connect( + client: HttpClient, + baseUrl: String + ): OpencodeHttp = + awaitHealthy(client, baseUrl, attempts = 50, delay = 200.millis) + new NetworkOpencodeHttp(client) + + private def awaitHealthy( + client: HttpClient, + baseUrl: String, + attempts: Int, + delay: FiniteDuration + ): Unit = + val healthy = (0 until attempts).exists: attempt => + if attempt > 0 then sleep(delay) + pingOk(client) + if !healthy then + throw OrcaFlowException( + s"opencode server at $baseUrl never became healthy" + ) + + private def pingOk(client: HttpClient): Boolean = + try client.get("/doc", PingTimeout).status == 200 + catch case NonFatal(_) => false + +private[opencode] final class NetworkOpencodeHttp(client: HttpClient) + extends OpencodeHttp: + import NetworkOpencodeHttp.RequestTimeout + + def postJson(path: String, body: String): String = + val reply = client.postJson(path, body, RequestTimeout) + if reply.isSuccess then reply.body + else + throw OrcaFlowException( + s"opencode POST $path failed: ${reply.status} ${reply.body}" + ) + + override def getStatus(path: String): Int = + try client.get(path, RequestTimeout).status + catch case NonFatal(_) => 0 + + def events(): StreamSource = + val stream = client.openLines("/event") + new StreamSource: + def lines: Iterator[String] = stream.lines + def errorLines: Iterator[String] = Nil.iterator + def interrupt(): Unit = stream.close() + def tryExitCode: Option[Int] = Some(0) diff --git a/opencode/src/main/scala/orca/backend/opencode/OpencodeApi.scala b/opencode/src/main/scala/orca/backend/opencode/OpencodeApi.scala index 5ecf3ad70..7ccafa270 100644 --- a/opencode/src/main/scala/orca/backend/opencode/OpencodeApi.scala +++ b/opencode/src/main/scala/orca/backend/opencode/OpencodeApi.scala @@ -1,6 +1,6 @@ package orca.backend.opencode -import com.github.plokhotnyuk.jsoniter_scala.macros.ConfiguredJsonValueCodec +import orca.json.JsonCodec import orca.json.RawJson /** Wire DTOs for the OpenCode HTTP server (ADR 0014). @@ -18,18 +18,15 @@ private[opencode] object OpencodeApi: // --- Request bodies (written) --- /** `model` on a message / session-create body. */ - case class ModelRef(providerID: String, modelID: String) - derives ConfiguredJsonValueCodec + case class ModelRef(providerID: String, modelID: String) derives JsonCodec /** A `text` message part — the only part kind orca sends. */ - case class MessagePart(`type`: String, text: String) - derives ConfiguredJsonValueCodec + case class MessagePart(`type`: String, text: String) derives JsonCodec /** `format` for structured output; `schema` is the caller's JSON Schema held * verbatim. */ - case class OutputFormat(`type`: String, schema: RawJson) - derives ConfiguredJsonValueCodec + case class OutputFormat(`type`: String, schema: RawJson) derives JsonCodec /** Body of `POST /session/{id}/prompt_async` (and `…/message`). No field * defaults — a write shape is exact (see the file header); `tools` in @@ -44,22 +41,20 @@ private[opencode] object OpencodeApi: tools: Option[Map[String, Boolean]], format: Option[OutputFormat], variant: Option[String] - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec /** Body of `POST /session`. */ - case class SessionCreateBody(title: Option[String] = None) - derives ConfiguredJsonValueCodec + case class SessionCreateBody(title: Option[String] = None) derives JsonCodec /** Body of `POST /question/{id}/reply` — one answer (selected labels) per * question, in order. */ - case class QuestionReplyBody(answers: List[List[String]]) - derives ConfiguredJsonValueCodec + case class QuestionReplyBody(answers: List[List[String]]) derives JsonCodec /** Body of `POST /permission/{id}/reply`; `reply` is one of * [[PermissionReply]]. */ - case class PermissionReplyBody(reply: String) derives ConfiguredJsonValueCodec + case class PermissionReplyBody(reply: String) derives JsonCodec /** The verdicts `POST /permission/{id}/reply` accepts. */ object PermissionReply: @@ -70,19 +65,18 @@ private[opencode] object OpencodeApi: // --- Response bodies (read) --- /** `POST /session` response. */ - case class SessionCreated(id: String) derives ConfiguredJsonValueCodec + case class SessionCreated(id: String) derives JsonCodec // --- Assistant message metadata (read from `message.updated`) --- - case class Cache(read: Long = 0L, write: Long = 0L) - derives ConfiguredJsonValueCodec + case class Cache(read: Long = 0L, write: Long = 0L) derives JsonCodec case class Tokens( input: Long = 0L, output: Long = 0L, reasoning: Long = 0L, cache: Cache = Cache() - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec /** The assistant `info` carried by `message.updated`. `structured` is present * only in structured mode; `error` is present only on a failed turn. @@ -96,28 +90,28 @@ private[opencode] object OpencodeApi: modelID: Option[String] = None, finish: Option[String] = None, error: Option[RawJson] = None - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec // --- ask_user / permission request payloads (read from SSE `properties`) --- case class QuestionOption(label: String, description: String = "") - derives ConfiguredJsonValueCodec + derives JsonCodec case class QuestionInfo( question: String, header: Option[String] = None, options: List[QuestionOption] = Nil - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec case class QuestionRequest( id: String, sessionID: String, questions: List[QuestionInfo] = Nil - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec case class PermissionRequest( id: String, sessionID: String, permission: String, patterns: List[String] = Nil - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec diff --git a/opencode/src/main/scala/orca/backend/opencode/OpencodeBackend.scala b/opencode/src/main/scala/orca/backend/opencode/OpencodeBackend.scala index dcae8df26..c31f2f661 100644 --- a/opencode/src/main/scala/orca/backend/opencode/OpencodeBackend.scala +++ b/opencode/src/main/scala/orca/backend/opencode/OpencodeBackend.scala @@ -1,9 +1,7 @@ package orca.backend.opencode -import com.github.plokhotnyuk.jsoniter_scala.core.{ - readFromString, - writeToString -} +import orca.json.JsonCodec +import orca.effect.Network import orca.backend.{ AskUserChannel, LiveTurn, @@ -31,7 +29,7 @@ import orca.subprocess.CliRunner import orca.backend.opencode.OpencodeApi.{SessionCreateBody, SessionCreated} import ox.Ox -import scala.util.control.NonFatal +import orca.safe.prelude.NonFatal /** Seam between the backend and the shared `opencode serve` owner — lets tests * substitute a fake without a real process. `http` may spawn on first call. @@ -60,10 +58,17 @@ private[orca] object OpencodeBackend: cli: CliRunner, workDir: os.Path, events: OrcaListener, + net: Network, launcher: OpencodeLauncher = OpencodeLauncher.default )(using Ox): OpencodeBackend = new OpencodeBackend( - OpencodeServer(cli, workDir, events, launcher), + OpencodeServer( + cli, + workDir, + events, + launcher, + (url, password) => NetworkOpencodeHttp.start(net, url, password) + ), workDir ) @@ -79,7 +84,7 @@ private[orca] object OpencodeBackend: */ private[orca] class OpencodeBackend( server: OpencodeServerHandle, - override val workDir: os.Path = os.pwd + override val workDir: os.Path ) extends AgentBackend[BackendTag.Opencode.type]: /** Probe `http` for the given session id via `GET /session/` → status @@ -137,9 +142,10 @@ private[orca] class OpencodeBackend( ): String = dispatch match case Dispatch.Resume(serverId, _) => WireSessionId.value(serverId) - case Dispatch.Fresh(_) => - val resp = http.postJson("/session", writeToString(SessionCreateBody())) - readFromString[SessionCreated](resp).id + case Dispatch.Fresh(_) => + val resp = + http.postJson("/session", JsonCodec.writeString(SessionCreateBody())) + JsonCodec.readString[SessionCreated](resp).id /** Resolve the server session, THEN open the SSE stream: [[serverSessionFor]] * can throw (fresh `POST /session`, or a bad resume id), and opening the @@ -198,7 +204,7 @@ private[orca] class OpencodeBackend( try val _ = http.postJson( s"/session/$serverSession/prompt_async", - writeToString(body) + JsonCodec.writeString(body) ) catch // The reader is already live on the SSE stream; cancel it so it doesn't diff --git a/opencode/src/main/scala/orca/backend/opencode/OpencodeDecoder.scala b/opencode/src/main/scala/orca/backend/opencode/OpencodeDecoder.scala index 14ee9b9a0..ad9fdb18d 100644 --- a/opencode/src/main/scala/orca/backend/opencode/OpencodeDecoder.scala +++ b/opencode/src/main/scala/orca/backend/opencode/OpencodeDecoder.scala @@ -1,6 +1,6 @@ package orca.backend.opencode -import com.github.plokhotnyuk.jsoniter_scala.core.writeToString +import orca.json.JsonCodec import orca.backend.{ AgentResult, ApprovalDecision, @@ -28,7 +28,7 @@ import orca.backend.opencode.OpencodeApi.{ import ox.Ox -import scala.util.control.NonFatal +import orca.safe.prelude.NonFatal /** Decodes one OpenCode turn off its `GET /event` SSE stream (ADR 0014): SSE * frame → [[OpencodeEvent]] → `TurnEvent`, deriving the [[AgentResult]] from @@ -124,7 +124,7 @@ private[opencode] final class OpencodeDecoder( // wrongly colliding two distinct id-less parts (BB5). partId match case Some(id) if state.startedTools.contains(id) => Step.continue(state) - case Some(id) => + case Some(id) => Step.continue( state.copy(startedTools = state.startedTools + id), call @@ -176,7 +176,7 @@ private[opencode] final class OpencodeDecoder( private def finishTurn(state: State): Out = state.info.flatMap(_.error) match case Some(err) => failTurn(state, OpencodeEvent.errorMessage(err)) - case None => + case None => if state.info.isEmpty && state.text.isEmpty then failTurn(state, "session went idle without an assistant message") else settleResult(state) @@ -245,7 +245,7 @@ private[opencode] final class OpencodeDecoder( private def replyToQuestion(req: QuestionRequest)(answer: String): Unit = val _ = http.postJson( s"/question/${req.id}/reply", - writeToString(QuestionReplyBody(List(List(answer)))) + JsonCodec.writeString(QuestionReplyBody(List(List(answer)))) ) private def replyToPermission( @@ -256,7 +256,7 @@ private[opencode] final class OpencodeDecoder( case ApprovalDecision.Deny => PermissionReply.Reject val _ = http.postJson( s"/permission/${req.id}/reply", - writeToString(PermissionReplyBody(verdict)) + JsonCodec.writeString(PermissionReplyBody(verdict)) ) private[opencode] object OpencodeDecoder: diff --git a/opencode/src/main/scala/orca/backend/opencode/OpencodeEvent.scala b/opencode/src/main/scala/orca/backend/opencode/OpencodeEvent.scala index a61c47f2a..ab8fe4525 100644 --- a/opencode/src/main/scala/orca/backend/opencode/OpencodeEvent.scala +++ b/opencode/src/main/scala/orca/backend/opencode/OpencodeEvent.scala @@ -1,7 +1,6 @@ package orca.backend.opencode -import com.github.plokhotnyuk.jsoniter_scala.core.readFromString -import com.github.plokhotnyuk.jsoniter_scala.macros.ConfiguredJsonValueCodec +import orca.json.JsonCodec import orca.backend.opencode.OpencodeApi.{ AssistantInfo, PermissionRequest, @@ -95,21 +94,21 @@ private[opencode] object OpencodeEvent: * [[OpencodeEvent.Ignored]]. */ def parse(json: String): OpencodeEvent = - readFromString[Envelope](json).`type` match + JsonCodec.readString[Envelope](json).`type` match case "message.part.delta" => parsePartDelta(json) case "message.part.updated" => parsePartUpdated(json) case "message.updated" => parseMessageUpdated(json) - case "session.idle" => - Idle(readFromString[SessionFrame](json).properties.sessionID) - case "session.error" => parseError(json) + case "session.idle" => + Idle(JsonCodec.readString[SessionFrame](json).properties.sessionID) + case "session.error" => parseError(json) case "question.asked" => - QuestionAsked(readFromString[QuestionFrame](json).properties) + QuestionAsked(JsonCodec.readString[QuestionFrame](json).properties) case "permission.asked" => - PermissionAsked(readFromString[PermissionFrame](json).properties) + PermissionAsked(JsonCodec.readString[PermissionFrame](json).properties) case _ => Ignored private def parsePartDelta(json: String): OpencodeEvent = - val p = readFromString[PartDeltaFrame](json).properties + val p = JsonCodec.readString[PartDeltaFrame](json).properties p.field match case "text" => TextDelta(p.sessionID, p.partID, p.delta) case "reasoning" => ReasoningDelta(p.sessionID, p.delta) @@ -122,7 +121,7 @@ private[opencode] object OpencodeEvent: * parts (text/step/…) carry no event. */ private def parsePartUpdated(json: String): OpencodeEvent = - val props = readFromString[PartUpdatedFrame](json).properties + val props = JsonCodec.readString[PartUpdatedFrame](json).properties val part = props.part if part.`type` == "reasoning" then ReasoningPart(props.sessionID.orElse(part.sessionID), part.id) @@ -166,14 +165,14 @@ private[opencode] object OpencodeEvent: * the result. */ private def parseMessageUpdated(json: String): OpencodeEvent = - val props = readFromString[MessageUpdatedFrame](json).properties + val props = JsonCodec.readString[MessageUpdatedFrame](json).properties val info = props.info if !info.role.contains("assistant") then Ignored else MessageUpdated(props.sessionID.orElse(info.sessionID).getOrElse(""), info) private def parseError(json: String): OpencodeEvent = - val p = readFromString[SessionFrame](json).properties + val p = JsonCodec.readString[SessionFrame](json).properties Errored(p.sessionID, p.error.map(errorMessage).getOrElse("session error")) /** The human-readable text of an error payload. OpenCode wraps errors as @@ -182,7 +181,7 @@ private[opencode] object OpencodeEvent: */ private[opencode] def errorMessage(error: RawJson): String = scala.util - .Try(readFromString[ErrorBody](error.value)) + .Try(JsonCodec.readString[ErrorBody](error.value)) .toOption .flatMap(b => b.message.orElse(b.data.flatMap(_.message))) .filter(_.nonEmpty) @@ -190,13 +189,13 @@ private[opencode] object OpencodeEvent: // --- Wire shapes (only the fields the decoder inspects) --- - private case class Envelope(`type`: String) derives ConfiguredJsonValueCodec + private case class Envelope(`type`: String) derives JsonCodec private case class ToolState( status: Option[String] = None, input: Option[RawJson] = None, output: Option[String] = None - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec private case class Part( `type`: String, @@ -204,46 +203,44 @@ private[opencode] object OpencodeEvent: tool: Option[String] = None, sessionID: Option[String] = None, state: Option[ToolState] = None - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec private case class PartDeltaProps( sessionID: String, partID: Option[String] = None, field: String = "", delta: String = "" - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec private case class PartDeltaFrame(properties: PartDeltaProps) - derives ConfiguredJsonValueCodec + derives JsonCodec private case class PartUpdatedProps( part: Part, sessionID: Option[String] = None - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec private case class PartUpdatedFrame(properties: PartUpdatedProps) - derives ConfiguredJsonValueCodec + derives JsonCodec private case class MessageUpdatedProps( info: AssistantInfo, sessionID: Option[String] = None - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec private case class MessageUpdatedFrame(properties: MessageUpdatedProps) - derives ConfiguredJsonValueCodec + derives JsonCodec private case class SessionProps( sessionID: Option[String] = None, error: Option[RawJson] = None - ) derives ConfiguredJsonValueCodec - private case class SessionFrame(properties: SessionProps) - derives ConfiguredJsonValueCodec + ) derives JsonCodec + private case class SessionFrame(properties: SessionProps) derives JsonCodec - private case class ErrorData(message: Option[String] = None) - derives ConfiguredJsonValueCodec + private case class ErrorData(message: Option[String] = None) derives JsonCodec private case class ErrorBody( message: Option[String] = None, data: Option[ErrorData] = None - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec private case class QuestionFrame(properties: QuestionRequest) - derives ConfiguredJsonValueCodec + derives JsonCodec private case class PermissionFrame(properties: PermissionRequest) - derives ConfiguredJsonValueCodec + derives JsonCodec diff --git a/opencode/src/main/scala/orca/backend/opencode/OpencodeHttp.scala b/opencode/src/main/scala/orca/backend/opencode/OpencodeHttp.scala index aeaa638a9..cc4595f8b 100644 --- a/opencode/src/main/scala/orca/backend/opencode/OpencodeHttp.scala +++ b/opencode/src/main/scala/orca/backend/opencode/OpencodeHttp.scala @@ -7,8 +7,8 @@ import orca.backend.StreamSource * SSE stream as a [[StreamSource]]. * * A small seam so [[OpencodeTurn]] and [[OpencodeBackend]] are testable - * without a live server — implemented over `java.net.http` in - * [[OpencodeServer]], stubbed in tests. + * without a live server — implemented by [[NetworkOpencodeHttp]], stubbed in + * tests. */ private[opencode] trait OpencodeHttp: /** POST a JSON `body` to `path` (relative to the server base, e.g. @@ -28,9 +28,3 @@ private[opencode] trait OpencodeHttp: * reason — callers treat any non-200 as "session not found". */ def getStatus(path: String): Int = 0 - - /** Release transport resources (the HTTP client). Default no-op for stubs; - * the real client shuts down its connection pool and selector thread. Called - * once at scope teardown. - */ - def close(): Unit = () diff --git a/opencode/src/main/scala/orca/backend/opencode/OpencodeServer.scala b/opencode/src/main/scala/orca/backend/opencode/OpencodeServer.scala index d2be32623..340aedb48 100644 --- a/opencode/src/main/scala/orca/backend/opencode/OpencodeServer.scala +++ b/opencode/src/main/scala/orca/backend/opencode/OpencodeServer.scala @@ -4,24 +4,20 @@ import orca.OrcaFlowException import orca.events.OrcaListener import orca.subprocess.{CliRunner, PipedCliProcess} import orca.backend.EnvCookieSweep -import ox.{ - Fork, - Ox, +import orca.safe.Logger +import orca.concurrency.{ + bufferedChannel, fork, forkDiscard, forkUnsupervised, forever, releaseAfterScope, - supervised, - useInScope + rendezvousChannel, + supervised } -import ox.channels.{Channel, Sink, Source} -import ox.either.{catching, orThrow} - -import org.slf4j.LoggerFactory - -import java.util.UUID -import scala.util.control.NonFatal +import orca.safe.prelude.{NonFatal, catching, orThrow, randomUuid} +import ox.{Fork, Ox, ResourceScope} +import ox.channels.{Sink, Source} /** An attempt's shared `opencode serve` process (ADR 0014). The process, its * output drains and its HTTP client live in one daemon fork of the scope that @@ -47,7 +43,7 @@ private[opencode] final class OpencodeServer private ( * `ChannelClosedException` once the owning scope has ended. */ def http(): OpencodeHttp = - val replyTo = Channel.buffered[Either[Throwable, OpencodeHttp]](1) + val replyTo = bufferedChannel[Either[Throwable, OpencodeHttp]](1) requests.send(replyTo) replyTo.receive().orThrow @@ -56,7 +52,7 @@ private[opencode] object OpencodeServer: /** Where the server answers one [[OpencodeServer.http]] call. */ private type ReplyTo = Sink[Either[Throwable, OpencodeHttp]] - private val log = LoggerFactory.getLogger(classOf[OpencodeServer]) + private val log = Logger("orca.backend.opencode.OpencodeServer") private val ListeningLine = """listening on (https?://\S+)""".r @@ -67,16 +63,17 @@ private[opencode] object OpencodeServer: * [[OpencodeServer.http]] call, so a backend wired but never used spawns * nothing, and is destroyed when the scope ends. Work the process detached * and left running is then reported to `events`, which may no longer reach a - * closed terminal. + * closed terminal. `httpFor(baseUrl, password)` builds the client in the + * server's own scope, which releases it. */ def apply( cli: CliRunner, workDir: os.Path, events: OrcaListener, launcher: OpencodeLauncher = OpencodeLauncher.default, - httpFor: (String, String) => OpencodeHttp = JavaNetOpencodeHttp.start + httpFor: (String, String) => ResourceScope ?=> OpencodeHttp )(using Ox): OpencodeServer = - val requests = Channel.rendezvous[ReplyTo] + val requests = rendezvousChannel[ReplyTo] val owner = new ServerOwner(cli, workDir, events, launcher, httpFor) forkDiscard: // `error`, not `done`: it also fails sends already waiting. @@ -96,7 +93,7 @@ private[opencode] object OpencodeServer: workDir: os.Path, events: OrcaListener, launcher: OpencodeLauncher, - httpFor: (String, String) => OpencodeHttp + httpFor: (String, String) => ResourceScope ?=> OpencodeHttp ): /** Nothing is spawned until the first request. A start that fails answers @@ -111,13 +108,11 @@ private[opencode] object OpencodeServer: firstReplyTo: ReplyTo, requests: Source[ReplyTo] )(using Ox): Unit = - val password = UUID.randomUUID.toString + val password = randomUuid() spawn(password).catching[Throwable] match - case Left(e) => firstReplyTo.send(Left(e)) + case Left(e) => firstReplyTo.send(Left(e)) case Right(process) => - releaseAfterScope( - EnvCookieSweep.afterScope(process.envCookie, events) - ) + releaseAfterScope(EnvCookieSweep.afterScope(process, events)) // Destroying the process here, before the scope joins the drains, // ends their pipe reads. Tree-destroy, as a launch wrapper (ollama) // forks the real serve. @@ -151,13 +146,13 @@ private[opencode] object OpencodeServer: val baseUrl = forkUnsupervised(out.flatMap(parseBaseUrl).nextOption()) .join() .getOrElse(failedToStart(process, errTail)) - log.debug("opencode server started, listening on {}", baseUrl) + log.debug(s"opencode server started, listening on $baseUrl") // Resumes the same iterator past the bind line, so the server's log // output can't back-fill the pipe and stall it. forkDiscard: try out.foreach(_ => ()) catch case NonFatal(e) => log.debug("opencode stdout drain ended", e) - useInScope(httpFor(baseUrl, password))(_.close()) + httpFor(baseUrl, password) /** Drains stderr so a chatty launcher can't fill the pipe, returning the * last lines for a start-failure message. @@ -165,7 +160,7 @@ private[opencode] object OpencodeServer: private def stderrTail(process: PipedCliProcess): Vector[String] = try process.stderrLines.foldLeft(Vector.empty[String]): (tail, line) => - log.debug("opencode serve stderr: {}", line) + log.debug(s"opencode serve stderr: $line") (tail :+ line).takeRight(MaxErrTailLines) catch case NonFatal(e) => diff --git a/opencode/src/test/scala/orca/backend/opencode/NetworkOpencodeHttpTest.scala b/opencode/src/test/scala/orca/backend/opencode/NetworkOpencodeHttpTest.scala new file mode 100644 index 000000000..cff7d229e --- /dev/null +++ b/opencode/src/test/scala/orca/backend/opencode/NetworkOpencodeHttpTest.scala @@ -0,0 +1,69 @@ +package orca.backend.opencode + +import orca.OrcaFlowException +import orca.effect.{HttpClient, HttpLines, HttpReply} + +import java.io.IOException +import scala.concurrent.duration.FiniteDuration + +class NetworkOpencodeHttpTest extends munit.FunSuite: + + /** Answers each GET with the next of `gets`, and every POST with `post`. */ + private class FakeClient( + gets: List[() => HttpReply], + post: HttpReply = HttpReply(200, "") + ) extends HttpClient: + private var remaining = gets + var getPaths: List[String] = Nil + var closed = false + def get(path: String, timeout: FiniteDuration): HttpReply = + getPaths = getPaths :+ path + val next = remaining.head + remaining = remaining.tail + next() + def postJson( + path: String, + json: String, + timeout: FiniteDuration + ): HttpReply = + post + def openLines(path: String): HttpLines = new HttpLines: + def lines: Iterator[String] = Iterator("data: x") + def close(): Unit = closed = true + + private val healthy: () => HttpReply = () => HttpReply(200, "") + + test("connect polls /doc until the server answers 200"): + val client = new FakeClient( + List( + () => throw IOException("refused"), + () => HttpReply(503, ""), + healthy + ) + ) + val _ = NetworkOpencodeHttp.connect(client, "http://x") + assertEquals(client.getPaths, List("/doc", "/doc", "/doc")) + + test("postJson returns the body of a 2xx reply"): + val client = new FakeClient(List(healthy), post = HttpReply(201, "made")) + val http = NetworkOpencodeHttp.connect(client, "http://x") + assertEquals(http.postJson("/session", "{}"), "made") + + test("postJson throws on a non-2xx reply, naming the status"): + val client = new FakeClient(List(healthy), post = HttpReply(500, "boom")) + val http = NetworkOpencodeHttp.connect(client, "http://x") + val e = intercept[OrcaFlowException](http.postJson("/session", "{}")) + assert(e.getMessage.contains("500 boom"), e.getMessage) + + test("getStatus is 0 when the request fails"): + val client = new FakeClient( + List(healthy, () => throw IOException("reset")) + ) + val http = NetworkOpencodeHttp.connect(client, "http://x") + assertEquals(http.getStatus("/session/s"), 0) + + test("interrupting the event stream closes it"): + val client = new FakeClient(List(healthy)) + val events = NetworkOpencodeHttp.connect(client, "http://x").events() + events.interrupt() + assert(client.closed) diff --git a/opencode/src/test/scala/orca/backend/opencode/OpencodeBackendTest.scala b/opencode/src/test/scala/orca/backend/opencode/OpencodeBackendTest.scala index 1c7e53445..0571afec5 100644 --- a/opencode/src/test/scala/orca/backend/opencode/OpencodeBackendTest.scala +++ b/opencode/src/test/scala/orca/backend/opencode/OpencodeBackendTest.scala @@ -43,6 +43,9 @@ class OpencodeBackendTest extends munit.FunSuite: extends OpencodeServerHandle: def http(): OpencodeHttp = httpThunk + private def backendOn(server: OpencodeServerHandle): OpencodeBackend = + new OpencodeBackend(server, os.pwd) + private def data(json: String): String = s"data: $json" private def turn( @@ -76,7 +79,7 @@ class OpencodeBackendTest extends munit.FunSuite: ) ) ) - val backend = new OpencodeBackend(new FakeHandle(http)) + val backend = backendOn(new FakeHandle(http)) val client = fresh val result = backend.runAutonomous("hi", client, AgentConfig()) @@ -106,7 +109,7 @@ class OpencodeBackendTest extends munit.FunSuite: ): supervised: val http = new FakeHttp(turn("ses_server1", "stop", Nil)) - val backend = new OpencodeBackend(new FakeHandle(http)) + val backend = backendOn(new FakeHandle(http)) val client = fresh val _ = backend.runAutonomous("one", client, AgentConfig()) @@ -120,14 +123,14 @@ class OpencodeBackendTest extends munit.FunSuite: // `format: json_schema` (the body field asserted in OpencodeArgsTest) makes // the server deliver the payload through an injected StructuredOutput tool. supervised: - val backend = new OpencodeBackend(new FakeHandle(new FakeHttp(Nil))) + val backend = backendOn(new FakeHandle(new FakeHttp(Nil))) assertEquals(backend.structuredOutputMode, StructuredOutputMode.Tool) // Provider-matched, so incidental work doesn't pull in a second provider's // auth. test("an openai-led agent's cheap tier is openai luna"): supervised: - val backend = new OpencodeBackend(new FakeHandle(new FakeHttp(Nil))) + val backend = backendOn(new FakeHandle(new FakeHttp(Nil))) assertEquals( backend.cheapModel(Some(Model("openai/gpt-6-sol"))), Some(Model("openai/gpt-6-luna")) @@ -135,7 +138,7 @@ class OpencodeBackendTest extends munit.FunSuite: test("any other agent's cheap tier is anthropic haiku"): supervised: - val backend = new OpencodeBackend(new FakeHandle(new FakeHttp(Nil))) + val backend = backendOn(new FakeHandle(new FakeHttp(Nil))) assertEquals( backend.cheapModel(Some(Model("ollama/llama3.1"))), Some(Model("anthropic/claude-haiku-5-5")) @@ -144,7 +147,7 @@ class OpencodeBackendTest extends munit.FunSuite: test("registerSession lets a later call resume that server session directly"): supervised: val http = new FakeHttp(turn("ses_X", "stop", Nil)) - val backend = new OpencodeBackend(new FakeHandle(http)) + val backend = backendOn(new FakeHandle(http)) val client = fresh backend.sessions.register( client, @@ -171,7 +174,7 @@ class OpencodeBackendTest extends munit.FunSuite: ) ) ) - val backend = new OpencodeBackend(new FakeHandle(http)) + val backend = backendOn(new FakeHandle(http)) val live = OpenTurn.interactive(backend)( "q", fresh, @@ -195,7 +198,7 @@ class OpencodeBackendTest extends munit.FunSuite: // Nothing has mapped this client id, so `dispatchFor` must // short-circuit WITHOUT forcing `http` — the fake // handle fails the test if it is ever forced. - val backend = new OpencodeBackend( + val backend = backendOn( new FakeHandle(fail("must not spawn")) ) val client = fresh @@ -217,7 +220,7 @@ class OpencodeBackendTest extends munit.FunSuite: Nil, path => if path == "/session/ses_server1" then 200 else 404 ) - val backend = new OpencodeBackend(new FakeHandle(http)) + val backend = backendOn(new FakeHandle(http)) val client = fresh backend.sessions.rehydrate( client, @@ -239,7 +242,7 @@ class OpencodeBackendTest extends munit.FunSuite: turn(existingId, "stop", Nil), path => if path == s"/session/$existingId" then 200 else 404 ) - val backend = new OpencodeBackend(new FakeHandle(http)) + val backend = backendOn(new FakeHandle(http)) val _ = backend.runAutonomous("hi", fresh, AgentConfig()) // A different, unmapped client id resolves to no server id → false. @@ -254,13 +257,13 @@ class OpencodeBackendTest extends munit.FunSuite: Nil, path => if path == "/session/ses_abc" then 200 else 404 ) - val backend = new OpencodeBackend(new FakeHandle(http)) + val backend = backendOn(new FakeHandle(http)) assert(backend.probeSession("ses_abc", http)) test("probeSession returns false when getStatus is 404"): supervised: val http = new FakeHttp(Nil, _ => 404) - val backend = new OpencodeBackend(new FakeHandle(http)) + val backend = backendOn(new FakeHandle(http)) assert(!backend.probeSession("ses_missing", http)) test( @@ -268,7 +271,7 @@ class OpencodeBackendTest extends munit.FunSuite: ): supervised: val http = new FakeHttp(Nil, _ => 404) - val backend = new OpencodeBackend(new FakeHandle(http)) + val backend = backendOn(new FakeHandle(http)) val client = fresh backend.sessions.rehydrate( client, @@ -286,7 +289,7 @@ class OpencodeBackendTest extends munit.FunSuite: val http = new FakeHttp(Nil): override def getStatus(path: String): Int = throw new java.io.IOException("connection refused") - val backend = new OpencodeBackend(new FakeHandle(http)) + val backend = backendOn(new FakeHandle(http)) assert(!backend.probeSession("ses_abc", http)) test( @@ -302,7 +305,7 @@ class OpencodeBackendTest extends munit.FunSuite: if path == "/session" then throw new OrcaFlowException("boom: session create failed") else super.postJson(path, body) - val backend = new OpencodeBackend(new FakeHandle(http)) + val backend = backendOn(new FakeHandle(http)) val _ = intercept[OrcaFlowException]: backend.runAutonomous("hi", fresh, AgentConfig()) assert( diff --git a/opencode/src/test/scala/orca/backend/opencode/OpencodeIntegrationTest.scala b/opencode/src/test/scala/orca/backend/opencode/OpencodeIntegrationTest.scala index c47356a55..643d6939a 100644 --- a/opencode/src/test/scala/orca/backend/opencode/OpencodeIntegrationTest.scala +++ b/opencode/src/test/scala/orca/backend/opencode/OpencodeIntegrationTest.scala @@ -9,9 +9,10 @@ import orca.agents.{ WireSessionId, ToolSet } +import orca.effect.Network import orca.events.OrcaListener import orca.subprocess.OsProcCliRunner -import orca.testkit.TempDirs +import orca.testkit.{Effects, TempDirs} /** End-to-end tests against a real `opencode serve`. Gated on the * `ORCA_INTEGRATION` environment variable, so `sbt test` without it behaves as @@ -44,7 +45,12 @@ class OpencodeIntegrationTest extends munit.FunSuite: body: ox.Ox ?=> OpencodeBackend => Unit ): Unit = SupervisedBackend.using( - OpencodeBackend(OsProcCliRunner, workDir, OrcaListener.noop) + OpencodeBackend( + OsProcCliRunner(Effects.procs), + workDir, + OrcaListener.noop, + Network.ambient + ) )(body) private def fresh = SessionId.fresh[BackendTag.Opencode.type] diff --git a/opencode/src/test/scala/orca/backend/opencode/OpencodeServerTest.scala b/opencode/src/test/scala/orca/backend/opencode/OpencodeServerTest.scala index f87d2f5f9..ecae95228 100644 --- a/opencode/src/test/scala/orca/backend/opencode/OpencodeServerTest.scala +++ b/opencode/src/test/scala/orca/backend/opencode/OpencodeServerTest.scala @@ -12,9 +12,10 @@ import orca.subprocess.{ } import orca.backend.SweepFixtures import orca.testkit.TempDirs -import ox.{fork, supervised, timeout} +import ox.{fork, releaseAfterScope, supervised, timeout} import ox.channels.ChannelClosedException +import java.util.concurrent.atomic.AtomicBoolean import scala.concurrent.duration.* class OpencodeServerTest extends munit.FunSuite with SweepFixtures: @@ -152,23 +153,25 @@ class OpencodeServerTest extends munit.FunSuite with SweepFixtures: assertEquals(server.http(), stubHttp) assertEquals(runner.spawnCalls.size, 2) - test("scope end destroys the process and closes the client"): + test("scope end destroys the process and releases the client"): // Stdout/stderr stay open after the bind line, so the drains are blocked // until the process is destroyed. val proc = new FakePipedCliProcess() proc.enqueueStdout("opencode server listening on http://127.0.0.1:4096") - val client = new TrackingHttp + val released = new AtomicBoolean(false) supervised: val server = OpencodeServer( new SpawnStubCliRunner(List(proc)), TempDirs.dir(), OrcaListener.noop, - httpFor = (_, _) => client + httpFor = (_, _) => + releaseAfterScope(released.set(true)) + stubHttp ) val _ = server.http() assert(proc.isAlive) assert(!proc.isAlive, "scope end must destroy the serve process") - assert(client.closed, "scope end must close the http client") + assert(released.get, "scope end must release the http client") test("nothing is spawned when the server is never used"): val runner = new SpawnStubCliRunner(Nil) @@ -236,12 +239,6 @@ class OpencodeServerTest extends munit.FunSuite with SweepFixtures: val _ = awaitPid(pidFile) finally killPid(pidFile) - private class TrackingHttp extends OpencodeHttp: - @volatile var closed: Boolean = false - def postJson(path: String, body: String): String = "ok" - def events(): StreamSource = throw new UnsupportedOperationException - override def close(): Unit = closed = true - /** Runs `script` in place of `opencode serve`. */ private class ScriptRunner(script: String) extends CliRunner: def run( diff --git a/pi/src/main/scala/orca/backend/pi/PiAskUserExtension.scala b/pi/src/main/scala/orca/backend/pi/PiAskUserExtension.scala index 4b035cd46..89e937607 100644 --- a/pi/src/main/scala/orca/backend/pi/PiAskUserExtension.scala +++ b/pi/src/main/scala/orca/backend/pi/PiAskUserExtension.scala @@ -1,6 +1,8 @@ package orca.backend.pi -import orca.backend.TurnResources +import orca.effect.FileSystem +import orca.safe.prelude.given +import orca.util.PromptResource import ox.ResourceScope /** Temporary Pi extension exposing Orca's `ask_user` turn event through Pi's @@ -19,16 +21,10 @@ private[pi] object PiAskUserExtension: "do not ask if you can make a reasonable assumption." /** Write the extension to a temp file removed when the turn ends. */ - def write()(using ResourceScope): os.Path = - val file = TurnResources.tempDir("orca-pi-ask-user-") / "ask-user.ts" - os.write(file, loadSource().replace("__TOOL_NAME__", ToolName)) + def write(fs: FileSystem)(using ResourceScope): os.Path = + val file = fs.tempDir("orca-pi-ask-user-") / "ask-user.ts" + fs.writeNew(file, source.replace("__TOOL_NAME__", ToolName)) file - private def loadSource(): String = - val stream = getClass.getResourceAsStream("/orca/backend/pi/ask-user.ts") - require( - stream != null, - "ask-user.ts resource missing from the pi module jar" - ) - try String(stream.readAllBytes(), java.nio.charset.StandardCharsets.UTF_8) - finally stream.close() + private def source: String = + PromptResource.load("/orca/backend/pi/ask-user.ts") diff --git a/pi/src/main/scala/orca/backend/pi/PiBackend.scala b/pi/src/main/scala/orca/backend/pi/PiBackend.scala index bd35cec57..478d2e37b 100644 --- a/pi/src/main/scala/orca/backend/pi/PiBackend.scala +++ b/pi/src/main/scala/orca/backend/pi/PiBackend.scala @@ -1,6 +1,7 @@ package orca.backend.pi import orca.OrcaDir +import orca.effect.FileSystem import orca.agents.{ Model, AutoApprove, @@ -20,10 +21,10 @@ import orca.backend.{ IdScheme, SessionSupport, SubprocessSpawn, - SystemPromptComposer, - TurnResources + SystemPromptComposer } import orca.subprocess.CliRunner +import orca.safe.prelude.given import ox.{Ox, ResourceScope} @@ -41,21 +42,18 @@ import java.time.Instant * resumes on later ones. */ private[orca] class PiBackend private[pi] ( + fs: FileSystem, cli: CliRunner, - /** Working directory every spawn runs in. The `os.pwd` default serves test - * construction; the runtime passes the flow's real `workDir`. - */ - override val workDir: os.Path = os.pwd, - /** Wall clock for the session-cache retention cutoff (see - * [[PiSessionStore.Retention]]). - */ - clock: () => Instant = () => Instant.now() + /** [[PiBackend.transcriptProbe]] in production; given [[workDir]]. */ + transcriptExists: (os.Path, String) -> Boolean, + /** Working directory every spawn runs in, and where the probe looks. */ + override val workDir: os.Path ) extends AgentBackend[BackendTag.Pi.type]: // One session dir per Orca session id gives caller-stable continuity. The // claimed id is committed only after a successful turn, so both a retried // open-failure and a resumed run resolve fresh-vs-resume from the dir itself: - // `--continue` whenever `hasTranscript` finds a transcript pi already seeded + // `--continue` whenever `transcriptExists` finds a transcript pi already seeded // there, and a fresh seed when it says no. /** Durable: each session's transcript lives under @@ -64,14 +62,10 @@ private[orca] class PiBackend private[pi] ( * is a best-effort on-disk probe (ADR 0018 §2.6). */ val sessions: SessionSupport[BackendTag.Pi.type] = - SessionSupport.durable(IdScheme.ClientClaimed, hasTranscript) - - // A read path: probing must not create `.orca` (under the `os.pwd` default - // that would be the wrong tree) — only the spawn path creates. - private def hasTranscript(id: String): Boolean = - PiSessionStore - .dirFor(workDir, id) - .exists(dir => PiSessionStore.resumable(dir, workDir, clock())) + SessionSupport.durable( + IdScheme.ClientClaimed, + id => transcriptExists(workDir, id) + ) val tag: BackendTag.Pi.type = BackendTag.Pi @@ -93,7 +87,7 @@ private[orca] class PiBackend private[pi] ( val extraHint = Option.when(mode.isInteractive)(PiAskUserExtension.Hint) val systemPromptFile = writeSystemPrompt(config, extraHint) val askUserExtension = - Option.when(mode.isInteractive)(PiAskUserExtension.write()) + Option.when(mode.isInteractive)(PiAskUserExtension.write(fs)) SubprocessSpawn.open("pi RPC", events) { val args = PiArgs.rpc( @@ -101,7 +95,7 @@ private[orca] class PiBackend private[pi] ( // inside `/`, so the base must exist by spawn time. // Ensured per spawn, so a cache deleted mid-run is recreated. sessionDir = - OrcaDir.ensurePiSessions(workDir) / SessionId.value(session), + OrcaDir(fs).ensurePiSessions(workDir) / SessionId.value(session), dispatch = dispatch.asTurnDispatch, config = config, systemPromptFile = Some(systemPromptFile), @@ -126,28 +120,43 @@ private[orca] class PiBackend private[pi] ( extraHint: Option[String] )(using ResourceScope): os.Path = val file = - TurnResources.tempDir("orca-pi-system-prompt-") / "system-prompt.md" - os.write(file, SystemPromptComposer.combine(config, extraHint)) + fs.tempDir("orca-pi-system-prompt-") / "system-prompt.md" + fs.writeNew(file, SystemPromptComposer.combine(config, extraHint)) file private[orca] object PiBackend: - /** The runtime's door: builds a backend and prunes its session cache before - * handing it out, so no probe can call a dir the next prune would take - * resumable. The bare constructor stays effect-free because constructing - * with the `os.pwd` default must not prune the repo's own cache — that's - * what [[forInspection]] exists for. + /** The runtime's door: prunes the session cache under `workDir`, then builds + * a backend, so no probe can call a dir the next prune would take resumable. */ private[orca] def create( + fs: FileSystem, cli: CliRunner, + transcriptExists: (os.Path, String) -> Boolean, workDir: os.Path, - clock: () => Instant = () => Instant.now() - ): PiBackend = - val backend = new PiBackend(cli, workDir, clock) - PiSessionStore.prune(workDir, clock()) - backend + now: Instant + ): PiBackend^{fs} = + PiSessionStore(fs).prune(workDir, now) + new PiBackend(fs, cli, transcriptExists, workDir) + + /** Whether `id`'s session dir under `workDir` holds a transcript pi resumes. + * A read path: it never creates `.orca`, only spawning does. + */ + def transcriptProbe( + fs: FileSystem, + clock: () -> Instant + ): (os.Path, String) ->{fs} Boolean = + val store = PiSessionStore(fs) + (workDir, id) => + PiSessionStore + .dirFor(workDir, id) + .exists(store.resumable(_, workDir, clock())) /** A backend built only to be inspected (enforcement tables, arg wiring), - * never spawned or probed: no workDir, no prune, no disk effects. + * never spawned or probed: no prune, no disk effects. */ - private[orca] def forInspection(cli: CliRunner): PiBackend = - new PiBackend(cli) + private[orca] def forInspection( + fs: FileSystem, + cli: CliRunner, + workDir: os.Path + ): PiBackend^{fs} = + new PiBackend(fs, cli, (_, _) => false, workDir) diff --git a/pi/src/main/scala/orca/backend/pi/PiSessionStore.scala b/pi/src/main/scala/orca/backend/pi/PiSessionStore.scala index d86cceebd..eca778090 100644 --- a/pi/src/main/scala/orca/backend/pi/PiSessionStore.scala +++ b/pi/src/main/scala/orca/backend/pi/PiSessionStore.scala @@ -1,41 +1,24 @@ package orca.backend.pi import orca.OrcaDir +import orca.effect.FileSystem import orca.agents.SessionId +import orca.json.JsonCodec +import orca.safe.Logger +import orca.safe.prelude.{NonFatal, resolvePath, utf8String, given} -import com.github.plokhotnyuk.jsoniter_scala.core.readFromString -import com.github.plokhotnyuk.jsoniter_scala.macros.ConfiguredJsonValueCodec -import org.slf4j.LoggerFactory - -import java.nio.charset.StandardCharsets import java.time.Instant -import scala.util.control.NonFatal /** Where pi keeps a session's transcripts on disk, and whether one can still be * `--continue`d from — the single owner of that layout, of the resumability * predicate ([[resumable]]), and of the retention that prunes it - * ([[Retention]], [[prune]]). [[PiBackend]] points pi at these dirs and probes - * them through [[resumable]]; the shell's resume command builds its - * `--session-dir` argv from the same predicate, so the two can't drift. + * ([[PiSessionStore.Retention]], [[prune]]). [[PiBackend]] points pi at these + * dirs and probes them through [[resumable]]; the shell's resume command + * builds its `--session-dir` argv from the same predicate, so the two can't + * drift. The companion names the paths, without touching the filesystem. */ -private[orca] object PiSessionStore: - - private val log = LoggerFactory.getLogger(getClass) - - /** How long an untouched session dir survives in `.orca/cache/pi-sessions`, - * matching claude's own transcript retention. A pruned session is not a lost - * turn: [[resumable]] then reports absence and the runtime re-seeds. - */ - private[orca] val Retention: java.time.Duration = - java.time.Duration.ofDays(30) - - /** The dir pi writes `id`'s transcripts into, or `None` when `id` is not a - * usable directory name. Ids orca mints always are; the check is for ids - * read back from an event log or progress log, which are files on disk that - * could have been hand-edited into a path that escapes the store. - */ - def dirFor(workDir: os.Path, id: String): Option[os.Path] = - Option.when(SessionId.isSafe(id))(OrcaDir.piSessionsPath(workDir) / id) +private[orca] final class PiSessionStore(fs: FileSystem): + import PiSessionStore.* /** Will `pi --session-dir --continue` pick a transcript up from `dir`, * and will it survive the next [[prune]]? Composes the transcript check @@ -55,23 +38,11 @@ private[orca] object PiSessionStore: // second file, and `--continue` picks the most recent — the one the retry // wrote. try - os.isDir(dir) && - os.list - .stream(dir) + fs.isDir(dir) && + fs.list(dir) .exists(f => f.last.endsWith(".jsonl") && headerMatches(f, workDir)) catch case NonFatal(_) => false - // Verified against pi's shipped session loader - // (@earendil-works/pi-coding-agent/dist/core/session-manager.js, - // `findMostRecentSession`): it reads the file's first 512 bytes, JSON-parses - // the first line, and requires `type == "session"`, a string `id`, and a - // non-empty `cwd` that lexically path-resolves equal to pi's process cwd. - private case class TranscriptHeader( - `type`: String, - id: Option[String] = None, - cwd: Option[String] = None - ) derives ConfiguredJsonValueCodec - /** Would pi's `--continue` accept `file` as a transcript for a process run in * `workDir`? Any read or parse failure counts as no-match — pi silently * starts an empty session for such a file, which orca must report as "not @@ -79,22 +50,13 @@ private[orca] object PiSessionStore: */ private def headerMatches(file: os.Path, workDir: os.Path): Boolean = try - val firstLine = new String( - os.read.bytes(file, offset = 0, count = 512), - StandardCharsets.UTF_8 - ).takeWhile(_ != '\n') - val header = readFromString[TranscriptHeader](firstLine) + val firstLine = + utf8String(fs.readPrefix(file, maxBytes = 512)).takeWhile(_ != '\n') + val header = JsonCodec.readString[TranscriptHeader](firstLine) header.`type` == "session" && header.id.isDefined && header.cwd.exists(resolvesTo(_, workDir)) catch case NonFatal(_) => false - // Lexical resolution like node's `path.resolve` (pi resolves the header cwd - // against its process cwd, which orca sets to workDir): os.Path normalises - // `.`/`..` without touching the filesystem. - private def resolvesTo(cwd: String, workDir: os.Path): Boolean = - try cwd.nonEmpty && os.Path(os.FilePath(cwd), workDir) == workDir - catch case NonFatal(_) => false - /** Delete session dirs untouched since `now` minus [[Retention]]. Pi prunes * its own default session store but not the dir orca points it at, so * without this the cache grows by one dir per session forever. Run by @@ -110,14 +72,14 @@ private[orca] object PiSessionStore: */ def prune(workDir: os.Path, now: Instant): Unit = val root = OrcaDir.piSessionsPath(workDir) - if os.exists(root) && symlinkFree(workDir, root) then + if fs.exists(root) && symlinkFree(workDir, root) then val stale = now.minus(Retention) try - os.list(root) + fs.list(root) .foreach: dir => try if isPrunableSessionDir(dir) && !touchedSince(dir, stale) then - os.remove.all(dir) + fs.removeAll(dir) catch case NonFatal(e) => log.debug(s"skipping unprunable session dir $dir", e) @@ -127,17 +89,17 @@ private[orca] object PiSessionStore: private def symlinkFree(workDir: os.Path, root: os.Path): Boolean = try - OrcaDir.assertNoOrcaSymlinks(workDir, root) + OrcaDir(fs).assertNoOrcaSymlinks(workDir, root) true catch case NonFatal(e) => log.warn(s"not pruning pi session cache: ${e.getMessage}") false - // A symlinked or oddly-named entry is left alone: `os.remove.all` on it could - // reach outside the store. + // A symlinked or oddly-named entry is left alone: removing it recursively + // could reach outside the store. private def isPrunableSessionDir(dir: os.Path): Boolean = - SessionId.isSafe(dir.last) && os.isDir(dir, followLinks = false) + SessionId.isSafe(dir.last) && fs.isDir(dir) && !fs.isLink(dir) /** Was `dir` — or any file in it — modified at or after `cutoff`? Its own * mtime is not enough: appending to an existing transcript doesn't bump the @@ -146,6 +108,43 @@ private[orca] object PiSessionStore: * costs one stat and a stale scan stops at the first fresh file. */ private def touchedSince(dir: os.Path, cutoff: Instant): Boolean = - def touched(p: os.Path) = - !Instant.ofEpochMilli(os.mtime(p)).isBefore(cutoff) - touched(dir) || os.list.stream(dir).exists(touched) + def touched(p: os.Path) = !fs.modifiedAt(p).isBefore(cutoff) + touched(dir) || fs.list(dir).exists(touched) + +private[orca] object PiSessionStore: + + private val log = Logger(classOf[PiSessionStore].getName) + + /** How long an untouched session dir survives in `.orca/cache/pi-sessions`, + * matching claude's own transcript retention. A pruned session is not a lost + * turn: [[PiSessionStore.resumable]] then reports absence and the runtime + * re-seeds. + */ + private[orca] val Retention: java.time.Duration = + java.time.Duration.ofDays(30) + + /** The dir pi writes `id`'s transcripts into, or `None` when `id` is not a + * usable directory name. Ids orca mints always are; the check is for ids + * read back from an event log or progress log, which are files on disk that + * could have been hand-edited into a path that escapes the store. + */ + def dirFor(workDir: os.Path, id: String): Option[os.Path] = + Option.when(SessionId.isSafe(id))(OrcaDir.piSessionsPath(workDir) / id) + + // Verified against pi's shipped session loader + // (@earendil-works/pi-coding-agent/dist/core/session-manager.js, + // `findMostRecentSession`): it reads the file's first 512 bytes, JSON-parses + // the first line, and requires `type == "session"`, a string `id`, and a + // non-empty `cwd` that lexically path-resolves equal to pi's process cwd. + private case class TranscriptHeader( + `type`: String, + id: Option[String] = None, + cwd: Option[String] = None + ) derives JsonCodec + + // Lexical resolution like node's `path.resolve` (pi resolves the header cwd + // against its process cwd, which orca sets to workDir): os.Path normalises + // `.`/`..` without touching the filesystem. + private def resolvesTo(cwd: String, workDir: os.Path): Boolean = + try cwd.nonEmpty && resolvePath(cwd, workDir) == workDir + catch case NonFatal(_) => false diff --git a/pi/src/main/scala/orca/backend/pi/rpc/InboundEvent.scala b/pi/src/main/scala/orca/backend/pi/rpc/InboundEvent.scala index a79ac4d7b..ab90b7da8 100644 --- a/pi/src/main/scala/orca/backend/pi/rpc/InboundEvent.scala +++ b/pi/src/main/scala/orca/backend/pi/rpc/InboundEvent.scala @@ -1,18 +1,9 @@ package orca.backend.pi.rpc +import com.github.plokhotnyuk.jsoniter_scala.core.JsonValueCodec import orca.events.Usage -import orca.json.RawJson - -import com.github.plokhotnyuk.jsoniter_scala.core.{ - JsonValueCodec, - readFromString -} -import com.github.plokhotnyuk.jsoniter_scala.macros.{ - ConfiguredJsonValueCodec, - JsonCodecMaker -} - -import scala.util.control.NonFatal +import orca.json.{JsonCodec, RawJson} +import orca.safe.prelude.NonFatal /** One JSONL message read from `pi --mode rpc` stdout. The decoder models only * fields needed by Orca and collapses protocol additions to [[Unknown]]. @@ -48,7 +39,7 @@ private[pi] case class AgentMessage( private[pi] object InboundEvent: def parse(line: String): InboundEvent = - val envelope = readFromString[TopEnvelope](line) + val envelope = JsonCodec.readString[TopEnvelope](line) envelope.`type` match case "response" => parseResponse(line) case "message_update" => parseMessageUpdate(line) @@ -60,29 +51,29 @@ private[pi] object InboundEvent: case other => Unknown(other) private def parseResponse(line: String): InboundEvent = - val wire = readFromString[ResponseWire](line) + val wire = JsonCodec.readString[ResponseWire](line) Response(wire.id, wire.command, wire.success.getOrElse(false), wire.error) private def parseMessageUpdate(line: String): InboundEvent = - val wire = readFromString[MessageUpdateWire](line) + val wire = JsonCodec.readString[MessageUpdateWire](line) wire.assistantMessageEvent match case Some(raw) => MessageUpdate(parseDelta(raw)) case None => Unknown("message_update") private def parseDelta(raw: RawJson): MessageDelta = - val wire = readFromString[DeltaWire](raw.value) + val wire = JsonCodec.readString[DeltaWire](raw.value) wire.`type` match - case "text_delta" => MessageDelta.Text(wire.delta.getOrElse("")) + case "text_delta" => MessageDelta.Text(wire.delta.getOrElse("")) case "thinking_delta" => MessageDelta.Thinking(wire.delta.getOrElse("")) case other => MessageDelta.Other(other) private def parseMessageEnd(line: String): InboundEvent = - val wire = readFromString[MessageEndWire](line) + val wire = JsonCodec.readString[MessageEndWire](line) MessageEnd(parseAgentMessage(wire.message)) private def parseAgentMessage(raw: RawJson): AgentMessage = - val wire = readFromString[AgentMessageWire](raw.value) + val wire = JsonCodec.readString[AgentMessageWire](raw.value) AgentMessage( role = wire.role.getOrElse(""), text = wire.content.map(renderContent).getOrElse(""), @@ -92,14 +83,14 @@ private[pi] object InboundEvent: ) private def parseToolExecutionStart(line: String): InboundEvent = - val wire = readFromString[ToolExecutionStartWire](line) + val wire = JsonCodec.readString[ToolExecutionStartWire](line) ToolExecutionStart( wire.toolName.getOrElse(""), wire.args.map(_.value).getOrElse("{}") ) private def parseToolExecutionEnd(line: String): InboundEvent = - val wire = readFromString[ToolExecutionEndWire](line) + val wire = JsonCodec.readString[ToolExecutionEndWire](line) ToolExecutionEnd( toolName = wire.toolName.getOrElse(""), ok = !wire.isError.getOrElse(false), @@ -107,7 +98,7 @@ private[pi] object InboundEvent: ) private def parseExtensionUiRequest(line: String): InboundEvent = - val wire = readFromString[ExtensionUiRequestWire](line) + val wire = JsonCodec.readString[ExtensionUiRequestWire](line) // A missing/unknown method falls through to the decoder's `other` branch, // which replies with a cancel so Pi never blocks waiting on us. val method = wire.method.getOrElse("") @@ -126,10 +117,11 @@ private[pi] object InboundEvent: val trimmed = raw.value.trim if trimmed.isEmpty || trimmed == "null" then "" else if trimmed.startsWith("\"") then - try readFromString[String](trimmed) + try JsonCodec.readString[String](trimmed) catch case NonFatal(_) => trimmed else if trimmed.startsWith("[") then - try renderTextBlocks(readFromString[List[ContentBlockWire]](trimmed)) + try + renderTextBlocks(JsonCodec.readString[List[ContentBlockWire]](trimmed)) catch case NonFatal(_) => trimmed else trimmed @@ -137,7 +129,8 @@ private[pi] object InboundEvent: val trimmed = raw.value.trim if trimmed.isEmpty || trimmed == "null" then "" else - try renderTextBlocks(readFromString[ToolResultWire](trimmed).content) + try + renderTextBlocks(JsonCodec.readString[ToolResultWire](trimmed).content) catch case NonFatal(_) => trimmed /** Concatenate the `"type":"text"` blocks; other kinds (tool, image, …) are @@ -152,30 +145,28 @@ private[pi] object InboundEvent: val trimmed = raw.value.trim if trimmed.isEmpty || trimmed == "null" then None else - try Some(readFromString[String](trimmed)) + try Some(JsonCodec.readString[String](trimmed)) catch case NonFatal(_) => Some(trimmed) - private case class TopEnvelope(`type`: String) - derives ConfiguredJsonValueCodec + private case class TopEnvelope(`type`: String) derives JsonCodec private case class ResponseWire( id: Option[String] = None, command: Option[String] = None, success: Option[Boolean] = None, error: Option[String] = None - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec private case class MessageUpdateWire( assistantMessageEvent: Option[RawJson] = None - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec private case class DeltaWire( `type`: String, delta: Option[String] = None - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec - private case class MessageEndWire(message: RawJson) - derives ConfiguredJsonValueCodec + private case class MessageEndWire(message: RawJson) derives JsonCodec private case class AgentMessageWire( role: Option[String] = None, @@ -183,7 +174,7 @@ private[pi] object InboundEvent: usage: Option[UsageWire] = None, model: Option[String] = None, errorMessage: Option[String] = None - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec private case class UsageWire( input: Option[Long] = None, @@ -191,7 +182,7 @@ private[pi] object InboundEvent: cacheRead: Option[Long] = None, cacheWrite: Option[Long] = None, cost: Option[CostWire] = None - ) derives ConfiguredJsonValueCodec: + ) derives JsonCodec: // pi subtracts cached tokens from `input` on providers that include them, // so `input` is the fresh prompt on every provider it fronts. It also // reports `cacheWrite1h`, the 1h-TTL subset of `cacheWrite` — unparsed @@ -209,31 +200,31 @@ private[pi] object InboundEvent: ) private case class CostWire(total: Option[BigDecimal] = None) - derives ConfiguredJsonValueCodec + derives JsonCodec private case class ToolExecutionStartWire( toolName: Option[String] = None, args: Option[RawJson] = None - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec private case class ToolExecutionEndWire( toolName: Option[String] = None, result: Option[RawJson] = None, isError: Option[Boolean] = None - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec private case class ToolResultWire( content: List[ContentBlockWire] = Nil - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec private case class ContentBlockWire( `type`: String, text: Option[String] = None - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec - private given JsonValueCodec[String] = JsonCodecMaker.make[String] + private given JsonValueCodec[String] = JsonCodec.make[String] private given JsonValueCodec[List[ContentBlockWire]] = - JsonCodecMaker.make[List[ContentBlockWire]] + JsonCodec.make[List[ContentBlockWire]] private case class ExtensionUiRequestWire( id: String, @@ -242,4 +233,4 @@ private[pi] object InboundEvent: message: Option[RawJson] = None, placeholder: Option[String] = None, prefill: Option[String] = None - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec diff --git a/pi/src/main/scala/orca/backend/pi/rpc/OutboundMessage.scala b/pi/src/main/scala/orca/backend/pi/rpc/OutboundMessage.scala index 458363921..4fccc6fcc 100644 --- a/pi/src/main/scala/orca/backend/pi/rpc/OutboundMessage.scala +++ b/pi/src/main/scala/orca/backend/pi/rpc/OutboundMessage.scala @@ -1,7 +1,6 @@ package orca.backend.pi.rpc -import com.github.plokhotnyuk.jsoniter_scala.core.writeToString -import com.github.plokhotnyuk.jsoniter_scala.macros.ConfiguredJsonValueCodec +import orca.json.JsonCodec /** JSONL commands written to Pi RPC stdin. */ private[pi] object OutboundMessage: @@ -9,27 +8,29 @@ private[pi] object OutboundMessage: val PromptId: String = "orca-prompt" def prompt(text: String): String = - writeToString(PromptCommand(PromptId, "prompt", text)) + JsonCodec.writeString(PromptCommand(PromptId, "prompt", text)) def extensionUiValue(id: String, value: String): String = - writeToString(ExtensionUiValueResponse("extension_ui_response", id, value)) + JsonCodec.writeString( + ExtensionUiValueResponse("extension_ui_response", id, value) + ) def extensionUiCancelled(id: String): String = - writeToString( + JsonCodec.writeString( ExtensionUiCancelledResponse("extension_ui_response", id, true) ) private case class PromptCommand(id: String, `type`: String, message: String) - derives ConfiguredJsonValueCodec + derives JsonCodec private case class ExtensionUiValueResponse( `type`: String, id: String, value: String - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec private case class ExtensionUiCancelledResponse( `type`: String, id: String, cancelled: Boolean - ) derives ConfiguredJsonValueCodec + ) derives JsonCodec diff --git a/pi/src/test/scala/orca/backend/pi/PiAskUserExtensionTest.scala b/pi/src/test/scala/orca/backend/pi/PiAskUserExtensionTest.scala index 8ba436bd8..3f620ba67 100644 --- a/pi/src/test/scala/orca/backend/pi/PiAskUserExtensionTest.scala +++ b/pi/src/test/scala/orca/backend/pi/PiAskUserExtensionTest.scala @@ -1,11 +1,14 @@ package orca.backend.pi +import orca.testkit.Effects + import ox.supervised class PiAskUserExtensionTest extends munit.FunSuite: test("write() writes an extension file with the tool name substituted"): - val content = supervised(os.read(PiAskUserExtension.write())) + val content = + supervised(os.read(PiAskUserExtension.write(Effects.fs))) assert(content.contains("registerTool"), content) assert( content.contains(s"""name: "${PiAskUserExtension.ToolName}""""), diff --git a/pi/src/test/scala/orca/backend/pi/PiBackendTest.scala b/pi/src/test/scala/orca/backend/pi/PiBackendTest.scala index 57f6d97f9..2e54ff2df 100644 --- a/pi/src/test/scala/orca/backend/pi/PiBackendTest.scala +++ b/pi/src/test/scala/orca/backend/pi/PiBackendTest.scala @@ -1,6 +1,6 @@ package orca.backend.pi -import orca.testkit.OpenTurn +import orca.testkit.{Effects, OpenTurn} import orca.OrcaDir import orca.backend.SystemPromptComposer import orca.testkit.Usages.usage @@ -24,13 +24,21 @@ class PiBackendTest extends munit.FunSuite: private def sid: SessionId[BackendTag.Pi.type] = SessionId[BackendTag.Pi.type]("00000000-0000-0000-0000-000000000000") - /** A backend on a throwaway `workDir` — Pi's session dirs live under - * `/.orca/cache/pi-sessions/`, so the `os.pwd` default would write - * them into the repo. Tests that assert on the working dir itself construct - * the backend directly with their own. + private val fs = Effects.fs + + /** Pi's session dirs live under `/.orca/cache/pi-sessions/`, so + * `workDir` defaults to a throwaway dir rather than the repo. */ - private def backendWith(runner: SpawnStubCliRunner): PiBackend = - new PiBackend(runner, workDir = TempDirs.dir()) + private def backendWith( + runner: SpawnStubCliRunner, + workDir: os.Path = TempDirs.dir() + ): PiBackend = + new PiBackend( + fs, + runner, + PiBackend.transcriptProbe(fs, () => Instant.now()), + workDir + ) private def successfulProcess( message: String = "hello", @@ -53,7 +61,7 @@ class PiBackendTest extends munit.FunSuite: val process = successfulProcess("answer", 10L, 5L) val runner = new SpawnStubCliRunner(List(process)) val workDir = TempDirs.dir() - val backend = new PiBackend(runner, workDir = workDir) + val backend = backendWith(runner, workDir) val result = backend.runAutonomous("do it", sid, AgentConfig()) @@ -257,14 +265,14 @@ class PiBackendTest extends munit.FunSuite: // The read path (a wire id replayed from the progress log, then probed) // must stay effect-free — only spawning pi creates the session dirs. val workDir = TempDirs.dir() - val backend = new PiBackend(new SpawnStubCliRunner(Nil), workDir = workDir) + val backend = backendWith(new SpawnStubCliRunner(Nil), workDir) backend.sessions.rehydrate(sid, sid.onWire) val _ = backend.sessions.dispatchFor(sid) assert(!os.exists(workDir / ".orca")) test("a rehydrated session whose dir holds a transcript resumes"): val workDir = TempDirs.dir() - val backend = new PiBackend(new SpawnStubCliRunner(Nil), workDir = workDir) + val backend = backendWith(new SpawnStubCliRunner(Nil), workDir) os.write( OrcaDir.piSessionsPath(workDir) / SessionId.value(sid) / "session.jsonl", transcript(workDir), @@ -323,7 +331,13 @@ class PiBackendTest extends munit.FunSuite: * — so it prunes before it is handed back. */ private def backendAt(workDir: os.Path): PiBackend = - PiBackend.create(new SpawnStubCliRunner(Nil), workDir, () => now) + PiBackend.create( + fs, + new SpawnStubCliRunner(Nil), + PiBackend.transcriptProbe(fs, () => now), + workDir, + now + ) test("a session dir untouched since before the cutoff is pruned"): val workDir = TempDirs.dir() @@ -405,7 +419,7 @@ class PiBackendTest extends munit.FunSuite: val victim = target / "victim" os.makeDir.all(victim) os.mtime.set(victim, cutoff.minusMillis(1).toEpochMilli): Unit - val _ = OrcaDir.ensureCache(workDir) + val _ = Effects.orcaDir.ensureCache(workDir) os.symlink(OrcaDir.piSessionsPath(workDir), target) val _ = backendAt(workDir) diff --git a/pi/src/test/scala/orca/backend/pi/PiIntegrationTest.scala b/pi/src/test/scala/orca/backend/pi/PiIntegrationTest.scala index 92ff13114..8e6d5e4e8 100644 --- a/pi/src/test/scala/orca/backend/pi/PiIntegrationTest.scala +++ b/pi/src/test/scala/orca/backend/pi/PiIntegrationTest.scala @@ -9,7 +9,9 @@ import orca.agents.{ onWire } import orca.subprocess.OsProcCliRunner -import orca.testkit.TempDirs +import orca.testkit.{Effects, TempDirs} + +import java.time.Instant /** End-to-end smoke test against the real `pi` CLI. Gated on `ORCA_INTEGRATION` * so normal unit test runs do not require Pi to be installed or authenticated. @@ -24,11 +26,22 @@ class PiIntegrationTest extends munit.FunSuite: import scala.concurrent.duration.DurationInt 2.minutes + /** Built the way `PiAgents.default` builds it. */ + private def realBackend(workDir: os.Path): PiBackend = + val fs = Effects.fs + PiBackend.create( + fs, + OsProcCliRunner(Effects.procs), + PiBackend.transcriptProbe(fs, () => Instant.now()), + workDir, + Instant.now() + ) + private def fresh: SessionId[BackendTag.Pi.type] = SessionId.fresh[BackendTag.Pi.type] test("RPC autonomous prompt returns requested literal output"): - val backend = PiBackend.create(OsProcCliRunner, workDir = TempDirs.dir()) + val backend = realBackend(TempDirs.dir()) val result = backend.runAutonomous( prompt = "Reply with the single word: READY", session = fresh, @@ -44,8 +57,7 @@ class PiIntegrationTest extends munit.FunSuite: val session = fresh val config = AgentConfig().copy(tools = ToolSet.ReadOnly) - val _ = PiBackend - .create(OsProcCliRunner, workDir = workDir) + val _ = realBackend(workDir) .runAutonomous( prompt = "Remember the word BANANA. Reply with the single word: OK", session = session, @@ -54,7 +66,7 @@ class PiIntegrationTest extends munit.FunSuite: // A second instance stands in for the next orca run: the wire id comes back // from the run's event log, and only Pi's on-disk session dir carries context. - val next = PiBackend.create(OsProcCliRunner, workDir = workDir) + val next = realBackend(workDir) next.sessions.rehydrate(session, session.onWire) assertEquals( next.sessions.dispatchFor(session).asTurnDispatch, diff --git a/pi/src/test/scala/orca/backend/pi/PiSessionStoreTest.scala b/pi/src/test/scala/orca/backend/pi/PiSessionStoreTest.scala index 855902994..673b3604f 100644 --- a/pi/src/test/scala/orca/backend/pi/PiSessionStoreTest.scala +++ b/pi/src/test/scala/orca/backend/pi/PiSessionStoreTest.scala @@ -1,11 +1,13 @@ package orca.backend.pi -import orca.testkit.TempDirs +import orca.testkit.{Effects, TempDirs} import java.time.Instant class PiSessionStoreTest extends munit.FunSuite: + private val store = PiSessionStore(Effects.fs) + test( "dirFor places a session under .orca/cache/pi-sessions, creating nothing" ): @@ -36,7 +38,7 @@ class PiSessionStoreTest extends munit.FunSuite: val workDir = TempDirs.dir() val dir = sessionDir(workDir) os.write(dir / "session.jsonl", header(workDir)) - assert(PiSessionStore.resumable(dir, workDir, Instant.now())) + assert(store.resumable(dir, workDir, Instant.now())) test("not resumable when the header cwd names a different checkout"): // pi's `--continue` filters transcripts by header cwd and would silently @@ -44,21 +46,21 @@ class PiSessionStoreTest extends munit.FunSuite: val workDir = TempDirs.dir() val dir = sessionDir(workDir) os.write(dir / "session.jsonl", header(TempDirs.dir())) - assert(!PiSessionStore.resumable(dir, workDir, Instant.now())) + assert(!store.resumable(dir, workDir, Instant.now())) test("not resumable when the transcript has no parseable session header"): val workDir = TempDirs.dir() val dir = sessionDir(workDir) os.write(dir / "session.jsonl", "{}\n") - assert(!PiSessionStore.resumable(dir, workDir, Instant.now())) + assert(!store.resumable(dir, workDir, Instant.now())) test("not resumable when the dir holds only non-transcript files"): val workDir = TempDirs.dir() val dir = sessionDir(workDir) os.write(dir / "notes.txt", "x") - assert(!PiSessionStore.resumable(dir, workDir, Instant.now())) + assert(!store.resumable(dir, workDir, Instant.now())) test("not resumable when the dir isn't there"): val workDir = TempDirs.dir() val dir = PiSessionStore.dirFor(workDir, "a-session").get - assert(!PiSessionStore.resumable(dir, workDir, Instant.now())) + assert(!store.resumable(dir, workDir, Instant.now())) diff --git a/pi/src/test/scala/orca/backend/pi/PiTurnTest.scala b/pi/src/test/scala/orca/backend/pi/PiTurnTest.scala index 35ca215d0..c02efb8da 100644 --- a/pi/src/test/scala/orca/backend/pi/PiTurnTest.scala +++ b/pi/src/test/scala/orca/backend/pi/PiTurnTest.scala @@ -258,7 +258,7 @@ class PiTurnTest extends munit.FunSuite: live.cancel() live.awaitResult() match case Left(_: OrcaInteractiveCancelled) => () - case other => + case other => fail(s"expected cancellation after test cleanup, got $other") // Ctrl-C at an interactive prompt abandons a turn that has already been diff --git a/project/Dependencies.scala b/project/Dependencies.scala index d570a596c..5d3d0f515 100644 --- a/project/Dependencies.scala +++ b/project/Dependencies.scala @@ -2,7 +2,7 @@ import sbt.* object Dependencies { object V { - val scala = "3.9.0" + val scala = "3.10.0" val osLib = "0.11.8" val jsoniter = "2.38.17" // Tapir and sttp-apispec follow what V.chimp's chimp-server declares, so @@ -18,6 +18,7 @@ object Dependencies { val jsonSchemaValidator = "1.5.4" val chimp = "0.5.2" val logback = "1.5.18" + val slf4j = "2.0.17" } val osLib = "com.lihaoyi" %% "os-lib" % V.osLib @@ -52,8 +53,8 @@ object Dependencies { val munitScalacheck = "org.scalameta" %% "munit-scalacheck" % V.munitScalacheck % Test - // The Scala 3 compiler as a library, so the CC-enforcement negative-compile - // suite (orca.CcNegativeCompileTest) can invoke the compiler's entry point + // The Scala 3 compiler as a library, so the negative-compile suites + // (orca.testkit.Dotc) can invoke the compiler's entry point // (`dotty.tools.dotc.Main` — the class the `scalac`/`scala` launchers // themselves run) in-process on fixture sources. Pinned to V.scala so the // checked-compilation phases match the build's own compiler. Test-only. @@ -70,6 +71,10 @@ object Dependencies { val tapirNettySync = "com.softwaremill.sttp.tapir" %% "tapir-netty-server-sync" % V.tapir + // The logging API alone, for modules that log but leave the binding to the + // application. + val slf4j = "org.slf4j" % "slf4j-api" % V.slf4j + // SLF4J binding so chimp/tapir/netty stop printing the "No SLF4J providers // found" warning; a logback.xml on the classpath pins the noisy loggers to // ERROR so the orca event log isn't drowned in framework chatter. diff --git a/runner/src/main/scala/orca/FlowSourceProperty.scala b/runner/src/main/scala/orca/FlowSourceProperty.scala index 936eb5374..47356b6ad 100644 --- a/runner/src/main/scala/orca/FlowSourceProperty.scala +++ b/runner/src/main/scala/orca/FlowSourceProperty.scala @@ -30,7 +30,7 @@ private[orca] object FlowSourceProperty: .flatMap: raw => decode(raw) match case Right(source) => Some(source) - case Left(error) => + case Left(error) => System.err.println( s"[orca] ignoring the $Name property ($error); the orca shell " + "won't offer to resume this run" diff --git a/runner/src/main/scala/orca/OrcaArgs.scala b/runner/src/main/scala/orca/OrcaArgs.scala index fc8ea9e4c..16196abfa 100644 --- a/runner/src/main/scala/orca/OrcaArgs.scala +++ b/runner/src/main/scala/orca/OrcaArgs.scala @@ -30,7 +30,7 @@ case class OrcaArgs( case RunTarget.NewBranch(Uncommitted.Stash) => Nil case RunTarget.NewBranch(Uncommitted.Keep) => Seq("--keep-changes") case RunTarget.CurrentBranch(Uncommitted.Stash) => Seq("--skip-branch") - case RunTarget.CurrentBranch(Uncommitted.Keep) => + case RunTarget.CurrentBranch(Uncommitted.Keep) => Seq("--skip-branch", "--keep-changes") case RunTarget.Worktree => Seq("--worktree") val branchArgv = branch.toList.flatMap(name => Seq("--branch", name.value)) @@ -39,6 +39,9 @@ case class OrcaArgs( case PrReadiness.Ready => Nil promptArgv ++ verboseArgv ++ targetArgv ++ branchArgv ++ draftArgv + private[orca] def setupRequest: lifecycle.SetupRequest = + lifecycle.SetupRequest(userPrompt, target, branch, prReadiness) + object OrcaArgs: /** Parse the given argv or return a human-readable error — including for a diff --git a/claude/src/main/scala/orca/backend/claude/ClaudeAgents.scala b/runner/src/main/scala/orca/backend/claude/ClaudeAgents.scala similarity index 81% rename from claude/src/main/scala/orca/backend/claude/ClaudeAgents.scala rename to runner/src/main/scala/orca/backend/claude/ClaudeAgents.scala index 0bde9b203..e515babda 100644 --- a/claude/src/main/scala/orca/backend/claude/ClaudeAgents.scala +++ b/runner/src/main/scala/orca/backend/claude/ClaudeAgents.scala @@ -2,7 +2,9 @@ package orca.backend.claude import orca.agents.{Agent, AgentConfig, ClaudeAgent} import orca.backend.AgentWiring +import orca.effect.{FileSystem, Network, Processes} import orca.subprocess.OsProcCliRunner +import orca.tools.OsGitTool /** The default claude agent and the Claude-specific builders. Bare `claude` * runs Opus with the 1M-token context window (the coder); the tier accessors @@ -17,8 +19,17 @@ object ClaudeAgents: * [[ClaudeBackend.workDir]]). */ def default(wiring: AgentWiring): ClaudeAgent = + val fs = FileSystem.ambient + val procs = Processes.ambient Agent( - backend = new ClaudeBackend(OsProcCliRunner, workDir = wiring.workDir), + backend = new ClaudeBackend( + Network.ambient, + fs, + OsProcCliRunner(procs), + new OsGitTool(_, procs, fs), + ClaudeBackend.transcriptProbe(fs, os.home / ".claude" / "projects"), + workDir = wiring.workDir + ), config = AgentConfig(model = Some(ClaudeModels.Opus1M)), prompts = wiring.prompts, events = wiring.events, diff --git a/codex/src/main/scala/orca/backend/codex/CodexAgents.scala b/runner/src/main/scala/orca/backend/codex/CodexAgents.scala similarity index 73% rename from codex/src/main/scala/orca/backend/codex/CodexAgents.scala rename to runner/src/main/scala/orca/backend/codex/CodexAgents.scala index 208b78ab8..f8804ee5e 100644 --- a/codex/src/main/scala/orca/backend/codex/CodexAgents.scala +++ b/runner/src/main/scala/orca/backend/codex/CodexAgents.scala @@ -2,6 +2,7 @@ package orca.backend.codex import orca.agents.{Agent, AgentConfig, CodexAgent} import orca.backend.AgentWiring +import orca.effect.{FileSystem, Network, Processes} import orca.subprocess.OsProcCliRunner /** The default codex agent and the codex model tiers. Bare `codex` pins the @@ -14,8 +15,15 @@ object CodexAgents: * `.mini` opts down for cheap one-shots. */ def default(wiring: AgentWiring): CodexAgent = + val fs = FileSystem.ambient Agent( - backend = new CodexBackend(OsProcCliRunner, workDir = wiring.workDir), + backend = new CodexBackend( + Network.ambient, + fs, + OsProcCliRunner(Processes.ambient), + CodexBackend.rolloutProbe(fs, os.home / ".codex" / "sessions"), + workDir = wiring.workDir + ), config = AgentConfig(model = Some(CodexModels.Sol)), prompts = wiring.prompts, events = wiring.events, diff --git a/gemini/src/main/scala/orca/backend/gemini/GeminiAgents.scala b/runner/src/main/scala/orca/backend/gemini/GeminiAgents.scala similarity index 76% rename from gemini/src/main/scala/orca/backend/gemini/GeminiAgents.scala rename to runner/src/main/scala/orca/backend/gemini/GeminiAgents.scala index 64a736b76..722f9e7fe 100644 --- a/gemini/src/main/scala/orca/backend/gemini/GeminiAgents.scala +++ b/runner/src/main/scala/orca/backend/gemini/GeminiAgents.scala @@ -2,6 +2,7 @@ package orca.backend.gemini import orca.agents.{Agent, AgentConfig, GeminiAgent} import orca.backend.AgentWiring +import orca.effect.{FileSystem, Network, Processes} import orca.subprocess.OsProcCliRunner /** The default gemini agent and the Gemini model tiers. */ @@ -12,7 +13,12 @@ object GeminiAgents: */ def default(wiring: AgentWiring): GeminiAgent = Agent( - backend = new GeminiBackend(OsProcCliRunner, workDir = wiring.workDir), + backend = new GeminiBackend( + Network.ambient, + FileSystem.ambient, + OsProcCliRunner(Processes.ambient), + workDir = wiring.workDir + ), config = AgentConfig(model = Some(GeminiModels.Pro)), prompts = wiring.prompts, events = wiring.events, diff --git a/opencode/src/main/scala/orca/backend/opencode/OpencodeAgents.scala b/runner/src/main/scala/orca/backend/opencode/OpencodeAgents.scala similarity index 95% rename from opencode/src/main/scala/orca/backend/opencode/OpencodeAgents.scala rename to runner/src/main/scala/orca/backend/opencode/OpencodeAgents.scala index 953896b8a..9a84ae3ed 100644 --- a/opencode/src/main/scala/orca/backend/opencode/OpencodeAgents.scala +++ b/runner/src/main/scala/orca/backend/opencode/OpencodeAgents.scala @@ -2,6 +2,7 @@ package orca.backend.opencode import orca.agents.{Agent, AgentConfig, Model, OpencodeAgent} import orca.backend.AgentWiring +import orca.effect.{Network, Processes} import orca.subprocess.OsProcCliRunner import ox.Ox @@ -25,9 +26,10 @@ object OpencodeAgents: )(using Ox): OpencodeAgent = Agent( backend = OpencodeBackend( - OsProcCliRunner, + OsProcCliRunner(Processes.ambient), wiring.workDir, wiring.events, + Network.ambient, launcher ), config = AgentConfig(), diff --git a/pi/src/main/scala/orca/backend/pi/PiAgents.scala b/runner/src/main/scala/orca/backend/pi/PiAgents.scala similarity index 67% rename from pi/src/main/scala/orca/backend/pi/PiAgents.scala rename to runner/src/main/scala/orca/backend/pi/PiAgents.scala index 2ad6521e5..c0439730f 100644 --- a/pi/src/main/scala/orca/backend/pi/PiAgents.scala +++ b/runner/src/main/scala/orca/backend/pi/PiAgents.scala @@ -2,8 +2,11 @@ package orca.backend.pi import orca.agents.{Agent, AgentConfig, PiAgent} import orca.backend.AgentWiring +import orca.effect.{FileSystem, Processes} import orca.subprocess.OsProcCliRunner +import java.time.Instant + /** The default pi agent. Pi supports many providers and fuzzy model patterns * through its own CLI, so it has no named tiers: pin one with * `pi.withModel(Model("..."))`. @@ -12,8 +15,15 @@ object PiAgents: /** The default pi agent for a run: standard config. */ def default(wiring: AgentWiring): PiAgent = + val fs = FileSystem.ambient Agent( - backend = PiBackend.create(OsProcCliRunner, wiring.workDir), + backend = PiBackend.create( + fs, + OsProcCliRunner(Processes.ambient), + PiBackend.transcriptProbe(fs, () => Instant.now()), + wiring.workDir, + Instant.now() + ), config = AgentConfig(), prompts = wiring.prompts, events = wiring.events, diff --git a/runner/src/main/scala/orca/exports.scala b/runner/src/main/scala/orca/exports.scala index ade1f04c9..36ec45e1a 100644 --- a/runner/src/main/scala/orca/exports.scala +++ b/runner/src/main/scala/orca/exports.scala @@ -1,5 +1,10 @@ package orca +// Re-exporting `@assumeSafe` definitions (`JsonData`) needs an experimental +// scope; without one, 3.10.0 reports "class assumeSafe is marked @experimental" +// with no position. +import language.experimental.captureChecking + // Re-exports the user-facing surface from each sub-package so flow scripts can // pull everything in with a single `import orca.{*, given}`. Mirrors the public // API the README documents; deliberately omits customisation-only knobs (e.g. diff --git a/runner/src/main/scala/orca/flowEntry.scala b/runner/src/main/scala/orca/flowEntry.scala index 140880881..631d0d5f5 100644 --- a/runner/src/main/scala/orca/flowEntry.scala +++ b/runner/src/main/scala/orca/flowEntry.scala @@ -1,6 +1,7 @@ package orca import orca.backend.{AgentWiring, Interaction} +import orca.effect.{FileSystem, Processes} import orca.events.{ CostResolvingDispatcher, CostTracker, @@ -22,28 +23,26 @@ import orca.agents.{ Prompts } import orca.progress.ProgressStore -import orca.review.ReviewerCatalog +import orca.review.{OsGateShell, ReviewerCatalog} import orca.runner.{ DefaultFlowContext, DefaultFlowControl, - FlowLifecycle, + DirtyTreePrompt, FlowLock, FlowWiring, LoggingListener, OrcaBanner, OrcaLog, - RoleAgents, - RoleOverrides, RunGitHubTool, RunLogStarter, RunRequest, SetupOptions, - WiredAgents, - WorktreeRun + WiredAgents } +import orca.lifecycle.{FlowLifecycle, RoleAgents, RoleOverrides, WorktreeRun} import orca.runlog.{AttemptOutcome, RunEventLog} -import orca.runner.terminal.TerminalInteraction -import orca.subprocess.OsProcCliRunner +import orca.runner.terminal.{TerminalInteraction, TtyProbe} +import orca.subprocess.{OsProcCliRunner, PathProbe} import org.slf4j.LoggerFactory import orca.tools.FsTool import orca.tools.RuntimeGit @@ -157,6 +156,8 @@ def flow( // progress header via `runFlow`/`FlowLifecycle.setup`). val flowSource = FlowSourceProperty.read() val runKey = RunKey.of(args.userPrompt) + val worktreeRun = + WorktreeRun(Processes.ambient, FileSystem.ambient, FlowLock) // Where the run happens. This settles before the directory's first consumer, // the attempt's trace below; everything downstream is handed `workDir` @@ -168,7 +169,7 @@ def flow( // same prompt, a symlinked or unwritable `.orca`, a git that won't start. // One `Left` shape for every outcome keeps the reporting below the only // way out. - try WorktreeRun.resolve(workDir, runKey) + try worktreeRun.resolve(workDir, runKey) catch case NonFatal(e) => Left(TextUtil.throwableMessage(e)) // The attempt's trace file, capturing every stage, prompt and @@ -228,6 +229,7 @@ def flow( pricing = pricing, startRunLog = new RunLogStarter: def start(using Ox): RunEventLog = RunEventLog.start( + FileSystem.ambient, dir, runKey, attemptId, @@ -241,7 +243,7 @@ def flow( branchNaming = branchNaming, stackSettings = stackSettings, roles = RoleOverrides(planningAgent, codingAgent, reviewAgent), - configHome = ConfigHome.default, + configHome = ConfigHome.resolve(sys.env.get, os.home), flowSource = flowSource ) ) @@ -302,7 +304,7 @@ private[orca] def runFlow(request: RunRequest)( try runLogged(request, runLog)(body) catch // An interrupt means the scope is shutting down and has interrupted the - // log's actors too; an ask to them would never be answered. + // log's actor and writer too; an ask would never be answered. case e: InterruptedException => throw e // Fatal errors too, so the queued events are flushed and the attempt // is recorded as failed, not crashed. @@ -350,11 +352,22 @@ private def runLogged(request: RunRequest, runLog: RunEventLog)( ) val agents = WiredAgents.build(wiring, agentWiring) val runtimeGit = - wiring.git.getOrElse(new OsGitTool(workDir, dispatcher)) + wiring.git.getOrElse( + new OsGitTool( + workDir, + Processes.ambient, + FileSystem.ambient, + dispatcher + ) + ) val ghTool = wiring.gh.getOrElse( - new OsGitHubTool(OsProcCliRunner, workDir, events = dispatcher) + new OsGitHubTool( + OsProcCliRunner(Processes.ambient), + workDir, + events = dispatcher + ) ) - val fsTool = wiring.fs.getOrElse(new OsFsTool(workDir)) + val fsTool = wiring.fs.getOrElse(new OsFsTool(workDir, FileSystem.ambient)) runInContext( args = request.args, workDir = workDir, @@ -392,7 +405,7 @@ private def runInContext( )(body: (FlowContext, FlowControl) ?=> Unit): Unit = val debug = OrcaDebug.enabled || args.verbose val runKey = RunKey.of(args.userPrompt) - val store = ProgressStore.default(workDir, runKey) + val store = ProgressStore.default(FileSystem.ambient, workDir, runKey) // This method takes no `Ox`, as `resourceScope` can't start where one is // visible. resourceScope: @@ -407,6 +420,7 @@ private def runInContext( val (resolvedRoles, settingsRead) = surfaced: val read = FlowLifecycle.readSettings( + FileSystem.ambient, workDir, options.configHome.settings, options.stackSettings @@ -428,9 +442,11 @@ private def runInContext( val projectReviewersPath = OrcaDir.reviewersPath(workDir) // `ReviewerCatalog.discover` reads through `os.isDir`, which follows // links, so the tier directory is guarded here. - OrcaDir.assertNoOrcaSymlinks(workDir, projectReviewersPath) + OrcaDir(FileSystem.ambient) + .assertNoOrcaSymlinks(workDir, projectReviewersPath) val catalog = ReviewerCatalog.discover( + FileSystem.ambient, projectReviewersPath, options.configHome.reviewers ) @@ -440,7 +456,7 @@ private def runInContext( // its outcome is a constructor input; it drives the CODING role. val flowSetup = surfaced( FlowLifecycle.setup( - args, + args.setupRequest, resolvedRoles.coding, runtimeGit, workDir, @@ -449,7 +465,19 @@ private def runInContext( store, runLog, flowSource = options.flowSource, - emit = dispatcher.onEvent + emit = dispatcher.onEvent, + // From where the run IS, not from the flag: the shell relaunches a + // resume inside the worktree its log was found in WITHOUT `--worktree` + // (the flag would re-derive a path), and that run has to name the tree + // too. + worktree = Option.when( + WorktreeRun(Processes.ambient, FileSystem.ambient, FlowLock) + .isWorktreeRun(workDir) + )(workDir), + fs = FileSystem.ambient, + onPath = PathProbe.resolves(Processes.ambient, _, _), + tty = () => TtyProbe.stdin() && TtyProbe.stderr(), + ask = DirtyTreePrompt.promptOnStderr ) ) val ctx = new DefaultFlowContext( @@ -464,7 +492,8 @@ private def runInContext( gh = RunGitHubTool(ghTool, flowSetup.prReadiness), fs = fsTool, stackSettings = flowSetup.stackSettings, - reviewerCatalog = reviewerCatalog + reviewerCatalog = reviewerCatalog, + gateShell = OsGateShell(Processes.ambient, FileSystem.ambient) ) val control = new DefaultFlowControl( progressStore = store, diff --git a/runner/src/main/scala/orca/runner/DefaultFlowContext.scala b/runner/src/main/scala/orca/runner/DefaultFlowContext.scala index 5d166c62d..014c8d68f 100644 --- a/runner/src/main/scala/orca/runner/DefaultFlowContext.scala +++ b/runner/src/main/scala/orca/runner/DefaultFlowContext.scala @@ -1,7 +1,7 @@ package orca.runner import orca.{FlowContext, StackSettings} -import orca.review.ReviewerCatalog +import orca.review.{GateShell, ReviewerCatalog} import orca.tools.{FsTool, GitHubTool, RuntimeGit} import orca.agents.Agent import orca.events.{OrcaEvent, OrcaListener} @@ -32,7 +32,8 @@ private[orca] class DefaultFlowContext( * tiers): resolved by `runFlow` before the context exists, so it arrives * frozen like `stackSettings`. */ - val reviewerCatalog: ReviewerCatalog + val reviewerCatalog: ReviewerCatalog, + private[orca] val gateShell: GateShell ) extends FlowContext: export wired.{claude, codex, opencode, pi, gemini} diff --git a/runner/src/main/scala/orca/runner/DirtyTreePrompt.scala b/runner/src/main/scala/orca/runner/DirtyTreePrompt.scala new file mode 100644 index 000000000..59c2b20c9 --- /dev/null +++ b/runner/src/main/scala/orca/runner/DirtyTreePrompt.scala @@ -0,0 +1,49 @@ +package orca.runner + +import orca.lifecycle.{DirtyTreeChoice, DirtyTreePolicy} +import scala.annotation.tailrec +import scala.io.StdIn + +/** The terminal half of [[DirtyTreePolicy]]. */ +private[orca] object DirtyTreePrompt: + + /** The production `ask`: a numbered menu on stderr, one line from stdin. + * + * stderr, not stdout, so the menu never lands in a redirected transcript; + * `StdIn` rather than any richer input, since the runner deliberately owns + * no terminal UI of its own. `setup`'s gate requires BOTH streams to be a + * terminal, so neither half of the exchange can end up invisible. + * + * This writes the terminal outside `TerminalOutput`'s prompt transaction, + * which is otherwise the single owner of the cursor. What makes that safe is + * WHEN it runs: before any stage or setup activity raises the status row, so + * nothing is pinned at the bottom and the animator's `tick` is a no-op. + * Routing it through the transaction instead would mean handing setup the + * run's `Interaction` — including every embedder's non-terminal one. + * Residual: the read blocks uninterruptibly, so a fork failing elsewhere in + * the run's scope waits for the answer. + */ + def promptOnStderr(dirtyCount: Int): DirtyTreeChoice = + Console.err.println( + s"$dirtyCount uncommitted/untracked file(s) in the working tree:\n" + + " 1) stash them and start from a clean tree (default)\n" + + " 2) keep them in place for the flow to work on (if the flow fails " + + "before its first commit, kept changes to tracked files are lost)\n" + + " 3) abort" + ) + readAnswer() + + /** One `choice [1]:` line + read, repeated until [[DirtyTreePolicy.parse]] + * accepts the answer — an unrecognized one must not silently pick the stash + * default against the user's intent. EOF and an empty line are accepted (as + * the default), so a closed stdin cannot loop. + */ + @tailrec + private def readAnswer(): DirtyTreeChoice = + Console.err.print("choice [1]: ") + Console.err.flush() + DirtyTreePolicy.parse(Option(StdIn.readLine())) match + case Some(choice) => choice + case None => + Console.err.println("unrecognized answer — enter 1, 2, or 3") + readAnswer() diff --git a/runner/src/main/scala/orca/runner/FlowLock.scala b/runner/src/main/scala/orca/runner/FlowLock.scala index 69a32310b..8a4e89ba9 100644 --- a/runner/src/main/scala/orca/runner/FlowLock.scala +++ b/runner/src/main/scala/orca/runner/FlowLock.scala @@ -1,6 +1,8 @@ package orca.runner +import orca.lifecycle.WorktreeLock import orca.{OrcaDir, OrcaFlowException, RunKey} +import orca.effect.FileSystem import ox.discard import java.nio.ByteBuffer @@ -30,7 +32,7 @@ import java.util.concurrent.atomic.AtomicBoolean * commit: [[orca.OrcaDir.ensureCache]] writes the cache's `.gitignore` before * the lock file exists. */ -private[orca] object FlowLock: +private[orca] object FlowLock extends WorktreeLock: private val processFlowLock = new AtomicBoolean(false) @@ -46,14 +48,14 @@ private[orca] object FlowLock: * process holds it. */ def worktreeLocked[T](mainCheckout: os.Path, key: RunKey)(op: => T): T = - OrcaDir.ensureCache(mainCheckout).discard + OrcaDir(FileSystem.ambient).ensureCache(mainCheckout).discard locked(OrcaDir.worktreeLockPath(mainCheckout, key), "for this prompt")(op) /** Runs `op` holding the lock of the run in `workDir`; throws when another * process holds it. */ def workdirLocked[T](workDir: os.Path)(op: => T): T = - OrcaDir.ensureCache(workDir).discard + OrcaDir(FileSystem.ambient).ensureCache(workDir).discard locked(OrcaDir.flowLockPath(workDir), "in this working tree")(op) /** Runs `op` holding the OS lock on `lockPath`; throws when another process diff --git a/runner/src/main/scala/orca/runner/LoggingListener.scala b/runner/src/main/scala/orca/runner/LoggingListener.scala index e43c30ff5..0f75ea803 100644 --- a/runner/src/main/scala/orca/runner/LoggingListener.scala +++ b/runner/src/main/scala/orca/runner/LoggingListener.scala @@ -24,11 +24,11 @@ private[orca] class LoggingListener extends OrcaListener: log.info("stage end: {} ({})", path.display, outcome) case OrcaEvent.ActivityStarted(label) => log.info("activity start: {}", label) - case _: OrcaEvent.ActivityEnded => log.info("activity end") - case OrcaEvent.Step(message) => log.info("step: {}", message) + case _: OrcaEvent.ActivityEnded => log.info("activity end") + case OrcaEvent.Step(message) => log.info("step: {}", message) case OrcaEvent.Bookkeeping(message) => log.info("bookkeeping: {}", message) - case OrcaEvent.Caveat(message) => log.info("caveat: {}", message) + case OrcaEvent.Caveat(message) => log.info("caveat: {}", message) case OrcaEvent.BranchBound(branch) => log.info("branch bound: {}", branch.value) case OrcaEvent.UserPrompt(text) => log.debug("prompt sent:\n{}", text) diff --git a/runner/src/main/scala/orca/runner/OrcaLog.scala b/runner/src/main/scala/orca/runner/OrcaLog.scala index 8b6aef545..7ba88affa 100644 --- a/runner/src/main/scala/orca/runner/OrcaLog.scala +++ b/runner/src/main/scala/orca/runner/OrcaLog.scala @@ -11,6 +11,7 @@ import ch.qos.logback.core.rolling.{ } import ch.qos.logback.core.util.{Duration, FileSize} import orca.{AttemptId, OrcaDir, RunKey} +import orca.effect.FileSystem import org.slf4j.{Logger, LoggerFactory} import java.nio.charset.StandardCharsets.UTF_8 @@ -60,7 +61,7 @@ private[orca] object OrcaLog: * trace. */ def start(workDir: os.Path, key: RunKey, id: AttemptId): OrcaLog = - val _ = OrcaDir.ensureRunDir(workDir, key) + val _ = OrcaDir(FileSystem.ambient).ensureRunDir(workDir, key) val file = OrcaDir.traceLogPath(workDir, key, id) loggerContext() match case Some(ctx) => diff --git a/runner/src/main/scala/orca/runner/RunRequest.scala b/runner/src/main/scala/orca/runner/RunRequest.scala index 5ce08119c..96a399233 100644 --- a/runner/src/main/scala/orca/runner/RunRequest.scala +++ b/runner/src/main/scala/orca/runner/RunRequest.scala @@ -1,5 +1,6 @@ package orca.runner +import orca.lifecycle.RoleOverrides import orca.{BranchNamingStrategy, ConfigHome, OrcaArgs, StackSettings} import orca.backend.Interaction import orca.events.{OrcaListener, PricingTable} diff --git a/runner/src/main/scala/orca/runner/WiredAgents.scala b/runner/src/main/scala/orca/runner/WiredAgents.scala index ffce7c153..673847467 100644 --- a/runner/src/main/scala/orca/runner/WiredAgents.scala +++ b/runner/src/main/scala/orca/runner/WiredAgents.scala @@ -42,12 +42,6 @@ private[orca] final class WiredAgents( */ def all: List[Agent[?]] = byTag.values.toList - /** True when `a` runs on one of the five wired backends — a wired agent - * itself or a builder-derived sibling (`_.claude.opus`, `.withReadOnly`, …). - * Used only for the foreign-lead warning. - */ - def isWiredBackend(a: Agent[?]): Boolean = all.exists(_.sharesBackendWith(a)) - private[orca] object WiredAgents: /** Wire the attempt's agents, filling every `None` override with the diff --git a/runner/src/main/scala/orca/runner/terminal/TerminalEventRenderer.scala b/runner/src/main/scala/orca/runner/terminal/TerminalEventRenderer.scala index e301ff7fe..148bcdcd0 100644 --- a/runner/src/main/scala/orca/runner/terminal/TerminalEventRenderer.scala +++ b/runner/src/main/scala/orca/runner/terminal/TerminalEventRenderer.scala @@ -94,7 +94,7 @@ private[terminal] class TerminalEventRenderer( // collapsed and truncated, in the `●` style — ADR 0008 requires it stay // visible since the streamed JSON was suppressed. announcement match - case Announcement.Silent => () + case Announcement.Silent => () case Announcement.Say(text) => val who = attribution(agent) output.log( diff --git a/runner/src/main/scala/orca/runner/terminal/ToolInputSummary.scala b/runner/src/main/scala/orca/runner/terminal/ToolInputSummary.scala index 92475b93b..0d33d6578 100644 --- a/runner/src/main/scala/orca/runner/terminal/ToolInputSummary.scala +++ b/runner/src/main/scala/orca/runner/terminal/ToolInputSummary.scala @@ -110,7 +110,7 @@ private[terminal] object ToolInputSummary: maxLength ) case HeadlineKind.Command => CommandHeadline.render(value, maxLength) - case HeadlineKind.Search => + case HeadlineKind.Search => Text.oneLine(scopedPattern(value, json, workDir), maxLength) case HeadlineKind.Plain => Text.oneLine(value, maxLength) @@ -197,7 +197,7 @@ private[terminal] object ToolInputSummary: json.charAt(i) match case '"' => findStringEnd(json, i + 1) match - case None => names.reverse + case None => names.reverse case Some(end) => val isKey = depth == 1 && json.startsWith(":", skipSpaces(json, end + 1)) diff --git a/tools/src/main/scala/orca/subprocess/TtyProbe.scala b/runner/src/main/scala/orca/runner/terminal/TtyProbe.scala similarity index 98% rename from tools/src/main/scala/orca/subprocess/TtyProbe.scala rename to runner/src/main/scala/orca/runner/terminal/TtyProbe.scala index 7dfbb0751..456fe4304 100644 --- a/tools/src/main/scala/orca/subprocess/TtyProbe.scala +++ b/runner/src/main/scala/orca/runner/terminal/TtyProbe.scala @@ -1,4 +1,4 @@ -package orca.subprocess +package orca.runner.terminal import scala.util.control.NonFatal diff --git a/claude/src/test/scala/orca/backend/claude/ClaudeAgentsTest.scala b/runner/src/test/scala/orca/backend/claude/ClaudeAgentsTest.scala similarity index 100% rename from claude/src/test/scala/orca/backend/claude/ClaudeAgentsTest.scala rename to runner/src/test/scala/orca/backend/claude/ClaudeAgentsTest.scala diff --git a/gemini/src/test/scala/orca/backend/gemini/GeminiAgentsTest.scala b/runner/src/test/scala/orca/backend/gemini/GeminiAgentsTest.scala similarity index 90% rename from gemini/src/test/scala/orca/backend/gemini/GeminiAgentsTest.scala rename to runner/src/test/scala/orca/backend/gemini/GeminiAgentsTest.scala index b0bcd9639..711fc0c7c 100644 --- a/gemini/src/test/scala/orca/backend/gemini/GeminiAgentsTest.scala +++ b/runner/src/test/scala/orca/backend/gemini/GeminiAgentsTest.scala @@ -2,8 +2,9 @@ package orca.backend.gemini import orca.backend.SupervisedBackend import orca.agents.{AgentConfig, GeminiAgent} +import orca.effect.Network import orca.subprocess.{FakePipedCliProcess, SpawnStubCliRunner} -import orca.testkit.TestAgent +import orca.testkit.{Effects, TestAgent} import orca.backend.gemini.GeminiAgents.flash class GeminiAgentsTest extends munit.FunSuite: @@ -27,7 +28,9 @@ class GeminiAgentsTest extends munit.FunSuite: runner: SpawnStubCliRunner, config: AgentConfig )(body: GeminiAgent => Unit): Unit = - SupervisedBackend.using(new GeminiBackend(runner)): backend => + SupervisedBackend.using( + new GeminiBackend(Network.ambient, Effects.fs, runner, os.pwd) + ): backend => body(TestAgent(backend, config = config)) test("the base tool's pinned model reaches the CLI --model flag"): diff --git a/opencode/src/test/scala/orca/backend/opencode/OpencodeAgentsTest.scala b/runner/src/test/scala/orca/backend/opencode/OpencodeAgentsTest.scala similarity index 100% rename from opencode/src/test/scala/orca/backend/opencode/OpencodeAgentsTest.scala rename to runner/src/test/scala/orca/backend/opencode/OpencodeAgentsTest.scala diff --git a/runner/src/test/scala/orca/runner/CannedDiscoveryAgent.scala b/runner/src/test/scala/orca/runner/CannedDiscoveryAgent.scala index ad90f2919..58cf90f6a 100644 --- a/runner/src/test/scala/orca/runner/CannedDiscoveryAgent.scala +++ b/runner/src/test/scala/orca/runner/CannedDiscoveryAgent.scala @@ -1,5 +1,6 @@ package orca.runner +import orca.lifecycle.{StackDiscoveryReply, StackDiscoveryResult} import orca.agents.{Agent, BackendTag, ClaudeAgent} import orca.testkit.{ScriptedBackend, TestAgent} diff --git a/runner/src/test/scala/orca/runner/DirtyTreePromptTest.scala b/runner/src/test/scala/orca/runner/DirtyTreePromptTest.scala new file mode 100644 index 000000000..f32520d42 --- /dev/null +++ b/runner/src/test/scala/orca/runner/DirtyTreePromptTest.scala @@ -0,0 +1,44 @@ +package orca.runner + +import orca.lifecycle.DirtyTreeChoice +class DirtyTreePromptTest extends munit.FunSuite: + + test("the production prompt asks on stderr, reads stdin, keeps stdout clean"): + val out = new java.io.ByteArrayOutputStream() + val err = new java.io.ByteArrayOutputStream() + val choice = Console.withIn(new java.io.StringReader("2\n")): + Console.withOut(out): + Console.withErr(err): + DirtyTreePrompt.promptOnStderr(dirtyCount = 2) + assertEquals(choice, DirtyTreeChoice.Keep) + val menu = err.toString("UTF-8") + assert(menu.contains("2 uncommitted/untracked file(s)"), menu) + assert( + menu.contains("1)") && menu.contains("2)") && menu.contains("3)"), + s"all three options must be offered: $menu" + ) + assert( + menu.contains("kept changes to tracked files are lost"), + s"the keep option must warn about the failure-teardown data loss: $menu" + ) + assertEquals(out.toString("UTF-8"), "", "stdout must stay clean") + + test("the production prompt stashes when stdin is at EOF"): + val choice = Console.withIn(new java.io.StringReader("")): + Console.withErr(new java.io.ByteArrayOutputStream()): + DirtyTreePrompt.promptOnStderr(dirtyCount = 1) + assertEquals(choice, DirtyTreeChoice.Stash) + + test( + "the production prompt re-asks an unrecognized answer, then honors the next valid one" + ): + val err = new java.io.ByteArrayOutputStream() + val choice = Console.withIn(new java.io.StringReader("keep\n2\n")): + Console.withErr(err): + DirtyTreePrompt.promptOnStderr(dirtyCount = 1) + assertEquals(choice, DirtyTreeChoice.Keep) + val transcript = err.toString("UTF-8") + assert( + transcript.contains("unrecognized answer"), + s"the re-ask must say why it is asking again: $transcript" + ) diff --git a/runner/src/test/scala/orca/runner/EnforcementTableTest.scala b/runner/src/test/scala/orca/runner/EnforcementTableTest.scala index 503c2e06b..3b400d156 100644 --- a/runner/src/test/scala/orca/runner/EnforcementTableTest.scala +++ b/runner/src/test/scala/orca/runner/EnforcementTableTest.scala @@ -2,8 +2,10 @@ package orca.runner import orca.agents.{AutoApprove, BackendTag, Enforcement, ToolSet, TurnDispatch} import orca.backend.AgentBackend +import orca.effect.Network import orca.events.OrcaListener import orca.subprocess.StubCliRunner +import orca.tools.OsGitTool import orca.backend.claude.ClaudeBackend import orca.backend.codex.CodexBackend import orca.backend.gemini.GeminiBackend @@ -11,7 +13,7 @@ import orca.backend.opencode.OpencodeBackend import orca.backend.pi.PiBackend import ox.supervised -import orca.testkit.{RepoRoot, TempDirs} +import orca.testkit.{Effects, RepoRoot, TempDirs} /** The [[AutoApprove]] values the matrix distinguishes. `Only` is split by * emptiness because "pre-approve nothing" is a shape some backends encode as @@ -182,11 +184,29 @@ class EnforcementTableTest extends munit.FunSuite: supervised: val cli = new StubCliRunner() val backends = List[AgentBackend[?]]( - new ClaudeBackend(cli), - new CodexBackend(cli), - new GeminiBackend(cli), - OpencodeBackend(cli, TempDirs.dir(), OrcaListener.noop), - PiBackend.forInspection(cli) + new ClaudeBackend( + Network.ambient, + Effects.fs, + cli, + new OsGitTool(_, Effects.procs, Effects.fs), + (_, _) => false, + os.pwd + ), + new CodexBackend( + Network.ambient, + Effects.fs, + cli, + _ => false, + os.pwd + ), + new GeminiBackend(Network.ambient, Effects.fs, cli, os.pwd), + OpencodeBackend( + cli, + TempDirs.dir(), + OrcaListener.noop, + Network.ambient + ), + PiBackend.forInspection(Effects.fs, cli, os.pwd) ) use( (for diff --git a/runner/src/test/scala/orca/runner/FlowHarness.scala b/runner/src/test/scala/orca/runner/FlowHarness.scala index 0a7463fe3..df0ed106e 100644 --- a/runner/src/test/scala/orca/runner/FlowHarness.scala +++ b/runner/src/test/scala/orca/runner/FlowHarness.scala @@ -1,5 +1,6 @@ package orca.runner +import orca.lifecycle.RoleOverrides import orca.{ AgentSet, BranchNamingStrategy, diff --git a/runner/src/test/scala/orca/runner/FlowLifecycleTest.scala b/runner/src/test/scala/orca/runner/FlowLifecycleTest.scala index 472f2bd6f..4b7576ebd 100644 --- a/runner/src/test/scala/orca/runner/FlowLifecycleTest.scala +++ b/runner/src/test/scala/orca/runner/FlowLifecycleTest.scala @@ -1,5 +1,16 @@ package orca.runner +import orca.lifecycle.{ + DirtyTreeChoice, + DiscoveredCommand, + DiscoveredGate, + FlowLifecycle, + PublishedState, + StackDiscoveryResult, + StartingTree, + WorktreeRun +} +import orca.subprocess.PathProbe import orca.ReportedFailure import orca.{ BranchNamingStrategy, @@ -38,8 +49,7 @@ import orca.tools.{ OsGitTool, PrReadiness, RuntimeGit, - UntrackedFiles, - Worktrees + UntrackedFiles } import orca.pr.PrSummary import ox.supervised @@ -50,12 +60,13 @@ import java.util.concurrent.atomic.{AtomicInteger, AtomicReference} import java.util.concurrent.ConcurrentLinkedQueue import scala.jdk.CollectionConverters.* import orca.testkit.{ + Effects, GitRepo, PushlessGit, ScriptedBackend, StubGitHubTool, - TestAgent, TempDirs, + TestAgent, branchName, currentBranch, prHandle @@ -70,6 +81,9 @@ import orca.testkit.{ */ class FlowLifecycleTest extends munit.FunSuite: + private val worktreeRun = + WorktreeRun(Effects.procs, Effects.fs, FlowLock) + /** A well-formed hash naming no commit in any test repo — what a header * carries when the test does not care about the whole-run diff base. */ @@ -104,7 +118,11 @@ class FlowLifecycleTest extends munit.FunSuite: .text() .trim assertEquals(branch, "main") - val store = ProgressStore.default(workDir, RunKey.of("lifecycle-success")) + val store = ProgressStore.default( + Effects.fs, + workDir, + RunKey.of("lifecycle-success") + ) assert(!os.exists(store.path), s"progress log ${store.path} should be gone") // The one call site that wires the closing block to a real listener. val rendered = out.toString(java.nio.charset.StandardCharsets.UTF_8) @@ -150,7 +168,9 @@ class FlowLifecycleTest extends munit.FunSuite: ranIn.set(Some(ctx.workDir)) logSeen.set( os.exists( - ProgressStore.default(ctx.workDir, RunKey.of(prompt)).path + ProgressStore + .default(Effects.fs, ctx.workDir, RunKey.of(prompt)) + .path ) ) os.write.over(ctx.workDir / "made-by-the-run.txt", "worktree work") @@ -201,14 +221,14 @@ class FlowLifecycleTest extends munit.FunSuite: ranIn.set(None) run() assertEquals(ranIn.get(), Some(worktree), "a re-run returns to it") - assertEquals(Worktrees.list(workDir), List(workDir, worktree)) + assertEquals(Effects.worktrees.list(workDir), List(workDir, worktree)) test( "--worktree: a failed run in the worktree is resumable, not refused" ): val invokedIn = GitRepo.seeded() val prompt = "worktree-resume" - val worktree = WorktreeRun + val worktree = worktreeRun .resolve(invokedIn, RunKey.of(prompt)) .getOrElse(fail("the worktree must resolve")) val stageOneRuns = new AtomicInteger(0) @@ -237,9 +257,10 @@ class FlowLifecycleTest extends munit.FunSuite: // and a newly created untracked one from a second stage. Then apply failure // teardown. val workDir = GitRepo.seeded() - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) val prompt = "lifecycle-failure" - val store = ProgressStore.default(workDir, RunKey.of(prompt)) + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of(prompt)) given WorkspaceWrite = WorkspaceWrite.unsafe @@ -312,12 +333,13 @@ class FlowLifecycleTest extends munit.FunSuite: // feature branch without deleting the log). val workDir = GitRepo.seeded() val prompt = "lifecycle-resume" - val store = ProgressStore.default(workDir, RunKey.of(prompt)) + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of(prompt)) val invocations = new AtomicInteger(0) given WorkspaceWrite = WorkspaceWrite.unsafe - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) val _ = git.createBranch(branchName("feat/lifecycle-resume")) store.writeHeader( ProgressHeader( @@ -387,8 +409,9 @@ class FlowLifecycleTest extends munit.FunSuite: // leaves us on the feature branch with stage 1's commit + log entry intact. val workDir = GitRepo.seeded() val prompt = "crash-feature" - val store = ProgressStore.default(workDir, RunKey.of(prompt)) - val git = new OsGitTool(workDir) + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of(prompt)) + val git = Effects.gitTool(workDir) val startBranch = git.currentBranch() // The body failure escapes `runFlow` wrapped in `ReportedFailure` (it @@ -425,7 +448,7 @@ class FlowLifecycleTest extends munit.FunSuite: // successful second run returns to the start branch. val workDir = GitRepo.seeded() val prompt = "resume-feature" - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) val startBranch = git.currentBranch() val stageOneRuns = new AtomicInteger(0) @@ -598,8 +621,9 @@ class FlowLifecycleTest extends munit.FunSuite: // must abort rather than replay against the wrong branch. val workDir = GitRepo.seeded() val prompt = "merged-hazard" - val store = ProgressStore.default(workDir, RunKey.of(prompt)) - val git = new OsGitTool(workDir) + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of(prompt)) + val git = Effects.gitTool(workDir) given WorkspaceWrite = WorkspaceWrite.unsafe // Commit the log on `main` (HEAD) while it names a different feature branch. @@ -643,27 +667,21 @@ class FlowLifecycleTest extends munit.FunSuite: // as an `OrcaEvent.Step` so a listener (e.g. Slack) sees it. val workDir = GitRepo.seeded() val prompt = "corrupt-log-fresh" - val store = ProgressStore.default(workDir, RunKey.of(prompt)) - val git = new OsGitTool(workDir) + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of(prompt)) + val git = Effects.gitTool(workDir) val startBranch = git.currentBranch() os.makeDir.all(store.path / os.up) os.write.over(store.path, "not json {{{", createFolders = true) val emitted = new AtomicReference[List[OrcaEvent]](Nil) - val setup = FlowLifecycle.setup( - args = OrcaArgs(prompt), - agent = StubAgent.claude, - git = git, - workDir = workDir, - branchNaming = None, - resolution = FlowLifecycle - .readSettings(workDir, noGlobalSettings, Some(StackSettings.empty)) - .stack, - flowSource = None, - store = store, - runLog = new InMemoryRunEventLog, - emit = e => { val _ = emitted.updateAndGet(e :: _) } + val setup = runSetup( + workDir, + Some(StackSettings.empty), + prompt, + emit = e => { val _ = emitted.updateAndGet(e :: _) }, + git = Some(git) ) val steps = emitted.get().collect { case s: OrcaEvent.Step => s } @@ -691,24 +709,17 @@ class FlowLifecycleTest extends munit.FunSuite: // directory at the log's path is unreadable for any user, root included. val workDir = GitRepo.seeded() val prompt = "unreadable-log" - val git = new OsGitTool(workDir) - val store = ProgressStore.default(workDir, RunKey.of(prompt)) + val git = Effects.gitTool(workDir) + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of(prompt)) os.makeDir.all(store.path) os.write(workDir / "wip.txt", "user's edit") val thrown = intercept[orca.OrcaFlowException]: - val _ = FlowLifecycle.setup( - args = OrcaArgs(prompt), - agent = StubAgent.claude, - git = git, - workDir = workDir, - branchNaming = None, - resolution = FlowLifecycle - .readSettings(workDir, noGlobalSettings, Some(StackSettings.empty)) - .stack, - flowSource = None, - store = store, - runLog = new InMemoryRunEventLog, - emit = _ => () + val _ = runSetup( + workDir, + Some(StackSettings.empty), + prompt, + git = Some(git) ) assert(thrown.getMessage.contains("cannot be read"), thrown.getMessage) assert(thrown.getMessage.contains("permissions"), thrown.getMessage) @@ -737,7 +748,8 @@ class FlowLifecycleTest extends munit.FunSuite: flow: Option[FlowSource] = Some(FlowSource.Catalog("implement.sc")) ): ProgressStore = given WorkspaceWrite = WorkspaceWrite.unsafe - val store = ProgressStore.default(workDir, RunKey.of(prompt)) + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of(prompt)) store.writeHeader( ProgressHeader( startingBranch = Some(branchName("main")), @@ -752,7 +764,7 @@ class FlowLifecycleTest extends munit.FunSuite: store private def setupFresh(workDir: os.Path): FlowLifecycle.FlowSetup = - setupForSettings(workDir, Some(StackSettings.empty), "a brand new task") + runSetup(workDir, Some(StackSettings.empty), "a brand new task") test("setup: a repository with no commits is refused with a named next step"): val workDir = GitRepo.empty() @@ -764,7 +776,7 @@ class FlowLifecycleTest extends munit.FunSuite: "setup: a FRESH run refuses to start on a branch another run's log claims" ): val workDir = GitRepo.seeded() - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) val startBranch = git.currentBranch() val _ = writeForeignLog(workDir, branch = startBranch) // Dirty on purpose: the refusal must land before the cleanliness policy, @@ -790,27 +802,20 @@ class FlowLifecycleTest extends munit.FunSuite: // The check sits ahead of every mode decision; skip-branch would otherwise // bind straight onto the branch the interrupted run is using. val workDir = GitRepo.seeded() - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) given WorkspaceWrite = WorkspaceWrite.unsafe assert(git.createBranch(branchName("my-work")).isRight) val _ = writeForeignLog(workDir, branch = "my-work") val thrown = intercept[orca.OrcaFlowException]: - val _ = FlowLifecycle.setup( - args = OrcaArgs( - "a brand new task", - target = RunTarget.CurrentBranch(Uncommitted.Stash) + val prompt = "a brand new task" + val _ = runSetup( + workDir, + Some(StackSettings.empty), + prompt, + args = Some( + OrcaArgs(prompt, target = RunTarget.CurrentBranch(Uncommitted.Stash)) ), - agent = StubAgent.claude, - git = git, - workDir = workDir, - branchNaming = None, - resolution = FlowLifecycle - .readSettings(workDir, noGlobalSettings, Some(StackSettings.empty)) - .stack, - flowSource = None, - store = ProgressStore.default(workDir, RunKey.of("a brand new task")), - runLog = new InMemoryRunEventLog, - emit = _ => () + git = Some(git) ) assert(thrown.getMessage.contains("my-work"), thrown.getMessage) @@ -821,7 +826,7 @@ class FlowLifecycleTest extends munit.FunSuite: // not reach the terminal, a multi-line task must not break the message // apart, and a long one must not bury the guidance after it. val workDir = GitRepo.seeded() - val startBranch = new OsGitTool(workDir).currentBranch() + val startBranch = Effects.gitTool(workDir).currentBranch() val esc = 27.toChar val _ = writeForeignLog( workDir, @@ -847,7 +852,7 @@ class FlowLifecycleTest extends munit.FunSuite: // the one they'd actually resume — newest by mtime, as the shell's resume // offer picks. val workDir = GitRepo.seeded() - val startBranch = new OsGitTool(workDir).currentBranch() + val startBranch = Effects.gitTool(workDir).currentBranch() val older = writeForeignLog( workDir, startBranch, @@ -886,13 +891,13 @@ class FlowLifecycleTest extends munit.FunSuite: test("setup: a log naming a DIFFERENT branch leaves a fresh run alone"): val workDir = GitRepo.seeded() - val startBranch = new OsGitTool(workDir).currentBranch() + val startBranch = Effects.gitTool(workDir).currentBranch() val _ = writeForeignLog(workDir, branch = "feat/elsewhere") assertNotEquals(setupFresh(workDir).featureBranch.value, startBranch) test("setup: a CORRUPT foreign log never refuses — it claims no branch"): val workDir = GitRepo.seeded() - val startBranch = new OsGitTool(workDir).currentBranch() + val startBranch = Effects.gitTool(workDir).currentBranch() val foreign = writeForeignLog(workDir, branch = startBranch) os.write.over(foreign.path, "not json {{{") assertNotEquals(setupFresh(workDir).featureBranch.value, startBranch) @@ -904,13 +909,13 @@ class FlowLifecycleTest extends munit.FunSuite: // case, not a conflict — and a second log naming that branch doesn't turn // it into one. val workDir = GitRepo.seeded() - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) val prompt = "resume-me" given WorkspaceWrite = WorkspaceWrite.unsafe val startBranch = git.currentBranch() assert(git.createBranch(branchName("feat/resume-me")).isRight) ProgressStore - .default(workDir, RunKey.of(prompt)) + .default(Effects.fs, workDir, RunKey.of(prompt)) .writeHeader( ProgressHeader( startingBranch = Some(branchName(startBranch)), @@ -924,7 +929,7 @@ class FlowLifecycleTest extends munit.FunSuite: ) val _ = writeForeignLog(workDir, branch = "feat/resume-me") - val setup = setupForSettings(workDir, Some(StackSettings.empty), prompt) + val setup = runSetup(workDir, Some(StackSettings.empty), prompt) assertEquals(setup.featureBranch.value, "feat/resume-me") assertEquals(setup.startingHead, Head.OnBranch(branchName(startBranch))) @@ -934,7 +939,7 @@ class FlowLifecycleTest extends munit.FunSuite: // Read at the binding point, so the run's own header commit — which moves // HEAD — is inside the range a whole-run review sees, not before it. val workDir = GitRepo.seeded() - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) val boundAt = git.headCommit() val setup = setupFresh(workDir) assertEquals(setup.startingCommit, boundAt) @@ -947,26 +952,21 @@ class FlowLifecycleTest extends munit.FunSuite: test("setup: skip-branch mode records the commit it bound at too"): // No branch is created here, so nothing else would capture the commit. val workDir = GitRepo.seeded() - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) given WorkspaceWrite = WorkspaceWrite.unsafe assert(git.createBranch(branchName("my-work")).isRight) val boundAt = git.headCommit() val prompt = "skip-branch starting commit" - val store = ProgressStore.default(workDir, RunKey.of(prompt)) - val setup = FlowLifecycle.setup( - args = - OrcaArgs(prompt, target = RunTarget.CurrentBranch(Uncommitted.Stash)), - agent = StubAgent.claude, - git = git, - workDir = workDir, - branchNaming = None, - resolution = FlowLifecycle - .readSettings(workDir, noGlobalSettings, Some(StackSettings.empty)) - .stack, - flowSource = None, - store = store, - runLog = new InMemoryRunEventLog, - emit = _ => () + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of(prompt)) + val setup = runSetup( + workDir, + Some(StackSettings.empty), + prompt, + args = Some( + OrcaArgs(prompt, target = RunTarget.CurrentBranch(Uncommitted.Stash)) + ), + git = Some(git) ) assertEquals(setup.startingCommit, boundAt) assertEquals(Some(store.load().get.header.startingCommit), boundAt) @@ -975,7 +975,7 @@ class FlowLifecycleTest extends munit.FunSuite: // This attempt's HEAD has moved on by everything the interrupted one // committed, and a whole-run review must still see those changes. val workDir = GitRepo.seeded() - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) val boundAt = git.headCommit() val _ = setupFresh(workDir) given WorkspaceWrite = WorkspaceWrite.unsafe @@ -988,7 +988,7 @@ class FlowLifecycleTest extends munit.FunSuite: // names nothing HEAD descends from, so diffing against it would widen the // review to unrelated history. Losing the base is the designed outcome. val workDir = GitRepo.seeded() - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) val prompt = "resume-unreachable-commit" given WorkspaceWrite = WorkspaceWrite.unsafe val startBranch = git.currentBranch() @@ -996,7 +996,7 @@ class FlowLifecycleTest extends munit.FunSuite: git.createBranch(branchName("feat/resume-unreachable-commit")).isRight ) ProgressStore - .default(workDir, RunKey.of(prompt)) + .default(Effects.fs, workDir, RunKey.of(prompt)) .writeHeader( ProgressHeader( startingBranch = Some(branchName(startBranch)), @@ -1008,7 +1008,7 @@ class FlowLifecycleTest extends munit.FunSuite: prReadiness = PrReadiness.Ready ) ) - val setup = setupForSettings(workDir, Some(StackSettings.empty), prompt) + val setup = runSetup(workDir, Some(StackSettings.empty), prompt) assertEquals(setup.startingCommit, None) // --- the branch the run bound to, announced as an event --- @@ -1017,7 +1017,7 @@ class FlowLifecycleTest extends munit.FunSuite: workDir: os.Path ): (FlowLifecycle.FlowSetup, List[String]) = val emitted = new AtomicReference[List[OrcaEvent]](Nil) - val setup = setupForSettings( + val setup = runSetup( workDir, Some(StackSettings.empty), "a brand new task", @@ -1037,12 +1037,12 @@ class FlowLifecycleTest extends munit.FunSuite: test("setup: skip-branch mode emits the current branch as the bound one"): val workDir = GitRepo.seeded() - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) given WorkspaceWrite = WorkspaceWrite.unsafe assert(git.createBranch(branchName("my-work")).isRight) val prompt = "skip-branch bound event" val emitted = new AtomicReference[List[OrcaEvent]](Nil) - val _ = setupForSettings( + val _ = runSetup( workDir, Some(StackSettings.empty), prompt, @@ -1064,7 +1064,7 @@ class FlowLifecycleTest extends munit.FunSuite: git: RuntimeGit ): FlowLifecycle.FlowSetup = val prompt = "a task with a chosen branch" - setupForSettings( + runSetup( workDir, Some(StackSettings.empty), prompt, @@ -1081,14 +1081,14 @@ class FlowLifecycleTest extends munit.FunSuite: test("setup: --branch wins over the branch naming strategy"): val workDir = GitRepo.seeded() - val setup = setupWithBranch(workDir, "feat/x", new OsGitTool(workDir)) + val setup = setupWithBranch(workDir, "feat/x", Effects.gitTool(workDir)) assertEquals(setup.featureBranch.value, "feat/x") test( "setup: an existing --branch is refused with the --skip-branch hint, before anything is stashed" ): val workDir = GitRepo.seeded() - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) given WorkspaceWrite = WorkspaceWrite.unsafe assert(git.createBranch(branchName("feat/x")).isRight) assert(git.checkout(branchName("main")).isRight) @@ -1105,7 +1105,7 @@ class FlowLifecycleTest extends munit.FunSuite: test("setup: --branch differing from the resumed run's branch is refused"): val workDir = GitRepo.seeded() - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) val _ = setupWithBranch(workDir, "feat/x", git) val thrown = intercept[orca.OrcaFlowException]: setupWithBranch(workDir, "feat/y", git) @@ -1116,7 +1116,7 @@ class FlowLifecycleTest extends munit.FunSuite: test("setup: --branch equal to the resumed run's branch resumes"): val workDir = GitRepo.seeded() - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) val _ = setupWithBranch(workDir, "feat/x", git) assertEquals( setupWithBranch(workDir, "feat/x", git).featureBranch.value, @@ -1131,7 +1131,7 @@ class FlowLifecycleTest extends munit.FunSuite: prReadiness: PrReadiness ): FlowLifecycle.FlowSetup = val prompt = "a task with a chosen readiness" - setupForSettings( + runSetup( workDir, Some(StackSettings.empty), prompt, @@ -1157,7 +1157,7 @@ class FlowLifecycleTest extends munit.FunSuite: test("setup: --branch naming the detected default branch is refused"): val workDir = GitRepo.seeded() - val git = new OsGitTool(workDir): + val git = new OsGitTool(workDir, Effects.procs, Effects.fs): override def defaultBranch(): Option[String] = Some("develop") val thrown = intercept[orca.OrcaFlowException]: setupWithBranch(workDir, "develop", git) @@ -1172,7 +1172,7 @@ class FlowLifecycleTest extends munit.FunSuite: // a flow body actually reads when it asks for the whole-run diff base. val workDir = GitRepo.seeded() val prompt = "starting-commit-threading" - val boundAt = new OsGitTool(workDir).headCommit() + val boundAt = Effects.gitTool(workDir).headCommit() var seen: Option[CommitHash] = None runFlowForTest(workDir, prompt): seen = summon[orca.FlowControl].startingCommit @@ -1195,7 +1195,7 @@ class FlowLifecycleTest extends munit.FunSuite: prompt: String ): List[String] = val workDir = GitRepo.seeded() - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) gitignore.foreach: body => given WorkspaceWrite = WorkspaceWrite.unsafe os.write(workDir / ".gitignore", body) @@ -1206,21 +1206,13 @@ class FlowLifecycleTest extends munit.FunSuite: "format = echo fmt\n", createFolders = true ) - val store = ProgressStore.default(workDir, RunKey.of(prompt)) val emitted = new AtomicReference[List[OrcaEvent]](Nil) - val _ = FlowLifecycle.setup( - args = OrcaArgs(prompt), - agent = StubAgent.claude, - git = git, - workDir = workDir, - branchNaming = None, - resolution = FlowLifecycle - .readSettings(workDir, noGlobalSettings, settingsOverride) - .stack, - flowSource = None, - store = store, - runLog = new InMemoryRunEventLog, - emit = e => { val _ = emitted.updateAndGet(e :: _) } + val _ = runSetup( + workDir, + settingsOverride, + prompt, + emit = e => { val _ = emitted.updateAndGet(e :: _) }, + git = Some(git) ) emitted.get().collect { case s: OrcaEvent.Step => s.message } @@ -1267,13 +1259,11 @@ class FlowLifecycleTest extends munit.FunSuite: // --- stack-settings resolution during setup (ADR 0019) -------------------- - /** Drives `setup` directly against `workDir` with a throwaway store — the - * fixture for the stack-settings resolution tests, and (via the defaulted - * `emit`/`tty`/`ask`) the dirty-tree prompt tests. Headless by default, so a - * fresh dirty run stashes without asking. `args` overrides the plain - * `OrcaArgs(prompt)`; the store stays keyed on `prompt`. + /** Drives `setup` against `workDir` with the production probes and a store + * keyed on `prompt`. Headless by default, so a fresh dirty run stashes + * without asking. `args` overrides the plain `OrcaArgs(prompt)`. */ - private def setupForSettings( + private def runSetup( workDir: os.Path, settingsOverride: Option[StackSettings] = None, prompt: String = "settings-resolution", @@ -1281,28 +1271,38 @@ class FlowLifecycleTest extends munit.FunSuite: tty: () => Boolean = () => false, ask: Int => DirtyTreeChoice = _ => DirtyTreeChoice.Stash, args: Option[OrcaArgs] = None, - git: Option[RuntimeGit] = None + git: Option[RuntimeGit] = None, + agent: Agent[?] = StubAgent.claude, + flowSource: Option[FlowSource] = None ): FlowLifecycle.FlowSetup = FlowLifecycle.setup( - args = args.getOrElse(OrcaArgs(prompt)), - agent = StubAgent.claude, - git = git.getOrElse(new OsGitTool(workDir)), + args = args.getOrElse(OrcaArgs(prompt)).setupRequest, + agent = agent, + git = git.getOrElse(Effects.gitTool(workDir)), workDir = workDir, branchNaming = None, resolution = FlowLifecycle - .readSettings(workDir, noGlobalSettings, settingsOverride) + .readSettings( + Effects.fs, + workDir, + noGlobalSettings, + settingsOverride + ) .stack, - flowSource = None, - store = ProgressStore.default(workDir, RunKey.of(prompt)), + flowSource = flowSource, + store = ProgressStore.default(Effects.fs, workDir, RunKey.of(prompt)), runLog = new InMemoryRunEventLog, emit = emit, + worktree = None, + fs = Effects.fs, + onPath = PathProbe.resolves(Effects.procs, _, _), tty = tty, ask = ask ) test("setup: a committed settings file resolves into FlowSetup"): val workDir = GitRepo.seeded() - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) given WorkspaceWrite = WorkspaceWrite.unsafe os.write( OrcaDir.settingsPath(workDir), @@ -1310,7 +1310,7 @@ class FlowLifecycleTest extends munit.FunSuite: createFolders = true ) assert(git.commit("add stack settings").isRight) - val setup = setupForSettings(workDir) + val setup = runSetup(workDir) assertEquals( setup.stackSettings, StackSettings( @@ -1324,7 +1324,7 @@ class FlowLifecycleTest extends munit.FunSuite: "setup: a fresh run's header records userPrompt and the given flow source" ): val workDir = GitRepo.seeded() - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) given WorkspaceWrite = WorkspaceWrite.unsafe os.write( OrcaDir.settingsPath(workDir), @@ -1333,19 +1333,13 @@ class FlowLifecycleTest extends munit.FunSuite: ) assert(git.commit("add stack settings").isRight) val prompt = "resume-header-fields" - val store = ProgressStore.default(workDir, RunKey.of(prompt)) - val _ = FlowLifecycle.setup( - args = OrcaArgs(prompt), - agent = StubAgent.claude, - git = git, - workDir = workDir, - branchNaming = None, - resolution = - FlowLifecycle.readSettings(workDir, noGlobalSettings, None).stack, - store = store, - runLog = new InMemoryRunEventLog, - flowSource = Some(FlowSource.Catalog("implement.sc")), - emit = _ => () + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of(prompt)) + val _ = runSetup( + workDir, + prompt = prompt, + git = Some(git), + flowSource = Some(FlowSource.Catalog("implement.sc")) ) val loaded = store.load() assertEquals(loaded.map(_.header.userPrompt), Some(prompt)) @@ -1359,8 +1353,9 @@ class FlowLifecycleTest extends munit.FunSuite: ): val workDir = GitRepo.seeded() val prompt = "no-flow-name" - val store = ProgressStore.default(workDir, RunKey.of(prompt)) - val _ = setupForSettings( + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of(prompt)) + val _ = runSetup( workDir, settingsOverride = Some(StackSettings.empty), prompt = prompt @@ -1380,7 +1375,7 @@ class FlowLifecycleTest extends munit.FunSuite: "lint = npm run lint\n", createFolders = true ) - val setup = setupForSettings(workDir) + val setup = runSetup(workDir) assert( !os.exists(OrcaDir.settingsPath(workDir)), "stash must have swept the untracked settings file" @@ -1394,7 +1389,7 @@ class FlowLifecycleTest extends munit.FunSuite: "setup: a malformed settings file aborts BEFORE ensureClean — no stash, branch unchanged" ): val workDir = GitRepo.seeded() - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) os.write( OrcaDir.settingsPath(workDir), "not-a-key = cargo fmt\n", @@ -1402,7 +1397,7 @@ class FlowLifecycleTest extends munit.FunSuite: ) val startBranch = git.currentBranch() val thrown = intercept[orca.OrcaFlowException]: - setupForSettings(workDir) + runSetup(workDir) assert( thrown.getMessage.contains("invalid settings") && thrown.getMessage.contains(OrcaDir.settingsPath(workDir).toString) && @@ -1423,19 +1418,21 @@ class FlowLifecycleTest extends munit.FunSuite: val content = "format =\n" os.write(OrcaDir.settingsPath(workDir), content, createFolders = true) assertEquals( - FlowLifecycle.readSettings(workDir, noGlobalSettings, None).stack, + FlowLifecycle + .readSettings(Effects.fs, workDir, noGlobalSettings, None) + .stack, FlowLifecycle.SettingsResolution.NeedsDiscovery(Some(content)) ) test("setup: an explicit override wins over a present file, file untouched"): val workDir = GitRepo.seeded() - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) given WorkspaceWrite = WorkspaceWrite.unsafe val fileContent = "format = cargo fmt\n" os.write(OrcaDir.settingsPath(workDir), fileContent, createFolders = true) assert(git.commit("add stack settings").isRight) val override_ = StackSettings(format = List("scalafmt")) - val setup = setupForSettings(workDir, settingsOverride = Some(override_)) + val setup = runSetup(workDir, settingsOverride = Some(override_)) assertEquals(setup.stackSettings, override_) assertEquals(os.read(OrcaDir.settingsPath(workDir)), fileContent) @@ -1451,19 +1448,12 @@ class FlowLifecycleTest extends munit.FunSuite: target: RunTarget = RunTarget.NewBranch(Uncommitted.Stash) ): (FlowLifecycle.FlowSetup, List[String]) = val emitted = new AtomicReference[List[OrcaEvent]](Nil) - val setup = FlowLifecycle.setup( - args = OrcaArgs(prompt, target = target), - agent = agent, - git = new OsGitTool(workDir), - workDir = workDir, - branchNaming = None, - resolution = FlowLifecycle - .readSettings(workDir, noGlobalSettings, None) - .stack, - flowSource = None, - store = ProgressStore.default(workDir, RunKey.of(prompt)), - runLog = new InMemoryRunEventLog, - emit = e => { val _ = emitted.updateAndGet(e :: _) } + val setup = runSetup( + workDir, + prompt = prompt, + emit = e => { val _ = emitted.updateAndGet(e :: _) }, + args = Some(OrcaArgs(prompt, target = target)), + agent = agent ) ( setup, @@ -1515,7 +1505,7 @@ class FlowLifecycleTest extends munit.FunSuite: ) setup.startingTree match case StartingTree.Kept(Some(snapshot)) => - assertEquals(Some(snapshot.base), new OsGitTool(workDir).headCommit()) + assertEquals(Some(snapshot.base), Effects.gitTool(workDir).headCommit()) case other => fail(s"expected a kept snapshot, got $other") test( @@ -1625,8 +1615,9 @@ class FlowLifecycleTest extends munit.FunSuite: ): val workDir = GitRepo.seeded() val prompt = "discover-resume" - val store = ProgressStore.default(workDir, RunKey.of(prompt)) - val git = new OsGitTool(workDir) + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of(prompt)) + val git = Effects.gitTool(workDir) given WorkspaceWrite = WorkspaceWrite.unsafe // The delete-to-rediscover fixture: feature branch, committed header — and // NO settings file when the run resumes. @@ -1677,7 +1668,7 @@ class FlowLifecycleTest extends munit.FunSuite: "discovery: legacy-ignored repo — file written, no dedicated commit, stays untracked, ignored-warning fires" ): val workDir = GitRepo.seeded() - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) locally: given WorkspaceWrite = WorkspaceWrite.unsafe os.write(workDir / ".gitignore", ".orca/\n") @@ -1878,7 +1869,8 @@ class FlowLifecycleTest extends munit.FunSuite: // still complete: nothing should escape teardown and turn success into exit 1. val workDir = GitRepo.seeded() val prompt = "bestEffort-teardown" - val store = ProgressStore.default(workDir, RunKey.of(prompt)) + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of(prompt)) runFlowForTest(workDir, prompt): val _ = stage("stage-one"): os.write(workDir / "one.txt", "content") @@ -1939,7 +1931,7 @@ class FlowLifecycleTest extends munit.FunSuite: // the branch should be gone. val workDir = GitRepo.seeded() val prompt = "throwaway-flow" - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) supervised: val interaction = TerminalInteraction.start( out = new PrintStream(new ByteArrayOutputStream()), @@ -1973,7 +1965,7 @@ class FlowLifecycleTest extends munit.FunSuite: ): val workDir = GitRepo.seeded() val prompt = "code-flow" - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) var featureBranchName = "" supervised: val interaction = TerminalInteraction.start( @@ -2051,14 +2043,14 @@ class FlowLifecycleTest extends munit.FunSuite: /** A run that writes a file and then calls [[openPrIfGitHub]] on GitHub. */ private def handoffRun(workDir: os.Path = GitRepo.seeded()): HandoffRun = val prompt = "pr-handoff" - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) var featureBranch = "" runFlowForTest( workDir, prompt, claude = summarisingClaude, gh = Some(new StubGh), - git = Some(new PushlessGit(new OsGitTool(workDir))) + git = Some(new PushlessGit(Effects.gitTool(workDir))) ): featureBranch = summon[FlowContext].git.currentBranch() val _ = stage("write code"): @@ -2113,7 +2105,8 @@ class FlowLifecycleTest extends munit.FunSuite: orca.pr.recordOpenedPr(handoffPr) "done" val events = - RunEventReader.read(OrcaDir.eventLogPath(workDir, RunKey.of(prompt))) + new RunEventReader(Effects.fs) + .read(OrcaDir.eventLogPath(workDir, RunKey.of(prompt))) // Only the dispatcher delivers stage events to the log. assert( events.exists: @@ -2141,7 +2134,8 @@ class FlowLifecycleTest extends munit.FunSuite: val _ = stage[String]("crash"): throw new RuntimeException("boom") val events = - RunEventReader.read(OrcaDir.eventLogPath(workDir, RunKey.of(prompt))) + new RunEventReader(Effects.fs) + .read(OrcaDir.eventLogPath(workDir, RunKey.of(prompt))) assert( !events.exists(_.isInstanceOf[RunEvent.RunSucceeded]), s"a failed run must not record success: $events" @@ -2155,7 +2149,7 @@ class FlowLifecycleTest extends munit.FunSuite: // --worktree looks exactly like this — so `flowSetup.worktree` is the only // thing keeping teardown from checking the worktree's own branch out. val repo = GitRepo.seeded() - val worktree = WorktreeRun.resolve(repo, RunKey.of("pr-handoff")) match + val worktree = worktreeRun.resolve(repo, RunKey.of("pr-handoff")) match case Right(dir) => dir case Left(msg) => fail(s"could not create the worktree: $msg") val r = handoffRun(worktree) @@ -2166,7 +2160,7 @@ class FlowLifecycleTest extends munit.FunSuite: // A flow that crashes must NOT delete the branch — it needs to stay for resume. val workDir = GitRepo.seeded() val prompt = "failure-keeps-branch" - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) var featureBranchName = "" val _ = intercept[ReportedFailure]: runFlowForTest(workDir, prompt): @@ -2229,7 +2223,7 @@ class FlowLifecycleTest extends munit.FunSuite: // a protecting Step. val workDir = GitRepo.seeded() val prompt = "fresh-protected" - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) val listener = new RecordingListener var featureBranchName = "" supervised: @@ -2279,7 +2273,7 @@ class FlowLifecycleTest extends munit.FunSuite: // name: it must fall back and leave the pre-existing branch untouched. val workDir = GitRepo.seeded() val prompt = "adoption-hazard" - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) val listener = new RecordingListener given WorkspaceWrite = WorkspaceWrite.unsafe val _ = git.createBranch(branchName("taken-name")) @@ -2351,7 +2345,7 @@ class FlowLifecycleTest extends munit.FunSuite: // resolving to "main". val workDir = GitRepo.seeded() val prompt = "fallback-collision-hazard" - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) val expectedFallback = BranchNamingStrategy.flowFallbackName(prompt) given WorkspaceWrite = WorkspaceWrite.unsafe val _ = git.createBranch(branchName(expectedFallback)) @@ -2402,7 +2396,7 @@ class FlowLifecycleTest extends munit.FunSuite: ): val workDir = GitRepo.seeded() val prompt = "skip-branch-reuse" - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) given WorkspaceWrite = WorkspaceWrite.unsafe val _ = git.createBranch(branchName("my-work")) val branchesBefore = branchNames(workDir) @@ -2451,7 +2445,7 @@ class FlowLifecycleTest extends munit.FunSuite: // output. Teardown's untracked removal must therefore be off here. val workDir = GitRepo.seeded() val prompt = "skip-branch-dirty-teardown" - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) given WorkspaceWrite = WorkspaceWrite.unsafe val _ = git.createBranch(branchName("my-work")) os.write(workDir / "handoff.md", "the plan") @@ -2501,7 +2495,7 @@ class FlowLifecycleTest extends munit.FunSuite: test("skip-branch mode refuses to bind to a protected branch"): val workDir = GitRepo.seeded() // starts on "main" val prompt = "skip-branch-protected" - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) val listener = new RecordingListener val thrown = intercept[ReportedFailure]: supervised: @@ -2588,7 +2582,7 @@ class FlowLifecycleTest extends munit.FunSuite: val _ = stage("stage-two")("two-done") assertEquals(stageOneRuns.get(), 1, "stage one must replay, not re-run") // Nothing but orca's log landed on the branch, so it is deleted. - assertEquals(new OsGitTool(workDir).head(), Head.Detached(start)) + assertEquals(Effects.gitTool(workDir).head(), Head.Detached(start)) assert(!branchNames(workDir).contains(featureBranch), branchNames(workDir)) /** `git stash list`, one entry per line, for asserting no/some stash was @@ -2609,7 +2603,7 @@ class FlowLifecycleTest extends munit.FunSuite: ): val workDir = GitRepo.seeded() // commits "seed.txt" val prompt = "skip-branch-dirty-modified" - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) given WorkspaceWrite = WorkspaceWrite.unsafe val _ = git.createBranch(branchName("my-work")) os.write.over(workDir / "seed.txt", "modified in place") @@ -2676,7 +2670,7 @@ class FlowLifecycleTest extends munit.FunSuite: ): val workDir = GitRepo.seeded() val prompt = "skip-branch-dirty-untracked" - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) given WorkspaceWrite = WorkspaceWrite.unsafe val _ = git.createBranch(branchName("my-work")) os.write(workDir / "docs" / "plan.md", "plan notes", createFolders = true) @@ -2740,7 +2734,7 @@ class FlowLifecycleTest extends munit.FunSuite: ): val workDir = GitRepo.seeded() val prompt = "skip-branch-dirty-mixed" - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) given WorkspaceWrite = WorkspaceWrite.unsafe val _ = git.createBranch(branchName("my-work")) os.write.over(workDir / "seed.txt", "modified in place") @@ -2781,8 +2775,9 @@ class FlowLifecycleTest extends munit.FunSuite: // tolerance covers an ABSENT log only. val workDir = GitRepo.seeded() val prompt = "skip-branch-corrupt-log" - val store = ProgressStore.default(workDir, RunKey.of(prompt)) - val git = new OsGitTool(workDir) + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of(prompt)) + val git = Effects.gitTool(workDir) given WorkspaceWrite = WorkspaceWrite.unsafe val _ = git.createBranch(branchName("my-work")) os.write(store.path, "not json {{{", createFolders = true) @@ -2820,8 +2815,9 @@ class FlowLifecycleTest extends munit.FunSuite: ): val workDir = GitRepo.seeded() val prompt = "skip-branch-resume-dirty" - val store = ProgressStore.default(workDir, RunKey.of(prompt)) - val git = new OsGitTool(workDir) + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of(prompt)) + val git = Effects.gitTool(workDir) given WorkspaceWrite = WorkspaceWrite.unsafe val _ = git.createBranch(branchName("my-work")) // A committed header, as if a prior run got this far before being killed. @@ -2880,7 +2876,7 @@ class FlowLifecycleTest extends munit.FunSuite: ): val workDir = GitRepo.seeded() // commits "seed.txt" val prompt = "keep-changes-fresh" - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) os.write.over(workDir / "seed.txt", "modified in place") os.write(workDir / "docs" / "plan.md", "plan notes", createFolders = true) val listener = new RecordingListener @@ -2946,8 +2942,9 @@ class FlowLifecycleTest extends munit.FunSuite: ): val workDir = GitRepo.seeded() val prompt = "keep-changes-resume" - val store = ProgressStore.default(workDir, RunKey.of(prompt)) - val git = new OsGitTool(workDir) + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of(prompt)) + val git = Effects.gitTool(workDir) given WorkspaceWrite = WorkspaceWrite.unsafe val _ = git.createBranch(branchName("feat/keep-changes-resume")) // A committed header, as if a prior run got this far before being killed. @@ -2999,8 +2996,8 @@ class FlowLifecycleTest extends munit.FunSuite: s"resume must say the flag was ignored: $steps" ) - /** [[setupForSettings]] with the dirty-tree prompt injected: `answer` is what - * the user "types", `tty` whether there is a terminal to type at. Counts the + /** [[runSetup]] with the dirty-tree prompt injected: `answer` is what the + * user "types", `tty` whether there is a terminal to type at. Counts the * asks, and records the emitted events, for the assertions that care. */ private def setupAsking( @@ -3011,7 +3008,7 @@ class FlowLifecycleTest extends munit.FunSuite: emitted: AtomicReference[List[OrcaEvent]] = new AtomicReference(Nil), asks: AtomicInteger = new AtomicInteger(0) ): FlowLifecycle.FlowSetup = - setupForSettings( + runSetup( workDir, settingsOverride = Some(StackSettings.empty), prompt = prompt, @@ -3152,7 +3149,7 @@ class FlowLifecycleTest extends munit.FunSuite: os.write.over(workDir / "seed.txt", "edited elsewhere") throw new RuntimeException("boom body") assertEquals( - new OsGitTool(workDir).head(), + Effects.gitTool(workDir).head(), Head.OnBranch(branchName("elsewhere")) ) assertEquals(os.read(workDir / "seed.txt"), "edited elsewhere") @@ -3170,7 +3167,7 @@ class FlowLifecycleTest extends munit.FunSuite: // must not sweep the file in. val workDir = GitRepo.seeded() val prompt = "keep-changes-no-stage" - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) os.write(workDir / "handoff.md", "the plan") supervised: val interaction = TerminalInteraction.start( @@ -3209,7 +3206,7 @@ class FlowLifecycleTest extends munit.FunSuite: // `ThrowawayBranch` never treats as throwaway. val workDir = GitRepo.seeded() val prompt = "skip-branch-throwaway" - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) given WorkspaceWrite = WorkspaceWrite.unsafe val _ = git.createBranch(branchName("my-empty-work")) supervised: @@ -3241,9 +3238,13 @@ class FlowLifecycleTest extends munit.FunSuite: // (skip-branch mode) must block the delete regardless of what // `startingCommit` claims. val workDir = GitRepo.seeded() // "main" - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) val startedAt = git.headCommit() - val store = ProgressStore.default(workDir, RunKey.of("gated-delete")) + val store = ProgressStore.default( + Effects.fs, + workDir, + RunKey.of("gated-delete") + ) given WorkspaceWrite = WorkspaceWrite.unsafe val _ = git.createBranch(branchName("reused-branch")) // diff-blank vs "main" @@ -3282,9 +3283,10 @@ class FlowLifecycleTest extends munit.FunSuite: withCode: Boolean ): (OsGitTool, os.Path, FlowLifecycle.FlowSetup) = val workDir = GitRepo.seeded() // "main" - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) val startedAt = git.headCommit() - val store = ProgressStore.default(workDir, RunKey.of("handoff")) + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of("handoff")) given WorkspaceWrite = WorkspaceWrite.unsafe val _ = git.createBranch(branchName("feat/work")) if withCode then @@ -3371,7 +3373,8 @@ class FlowLifecycleTest extends munit.FunSuite: // was pushed from cannot be recovered. val workDir = GitRepo.seeded() val prompt = "corrupt-log-throwaway" - val store = ProgressStore.default(workDir, RunKey.of(prompt)) + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of(prompt)) var featureBranch = "" runFlowForTest(workDir, prompt): featureBranch = summon[FlowContext].git.currentBranch() @@ -3406,8 +3409,12 @@ class FlowLifecycleTest extends munit.FunSuite: */ private def teardownPushFixture(): TeardownPushRepo = val workDir = GitRepo.seeded() // "main" - val git = new OsGitTool(workDir) - val store = ProgressStore.default(workDir, RunKey.of("teardown-push")) + val git = Effects.gitTool(workDir) + val store = ProgressStore.default( + Effects.fs, + workDir, + RunKey.of("teardown-push") + ) given WorkspaceWrite = WorkspaceWrite.unsafe val _ = git.createBranch(branchName(TeardownPushBranch)) os.write.over(store.path, "# progress\n", createFolders = true) @@ -3541,7 +3548,7 @@ class FlowLifecycleTest extends munit.FunSuite: // The default path: the branch the count was taken on is the one the user // is left on, so `git diff ` there already shows the work. val workDir = GitRepo.seeded() - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) given WorkspaceWrite = WorkspaceWrite.unsafe val base = git.headCommit().get val _ = git.createBranch(branchName("closing-stay")) @@ -3549,7 +3556,8 @@ class FlowLifecycleTest extends munit.FunSuite: os.write(workDir / "b.txt", "two") assert(git.commit("the run's work").isRight) val setup = closingSetup( - ProgressStore.default(workDir, RunKey.of("closing-stay")), + ProgressStore + .default(Effects.fs, workDir, RunKey.of("closing-stay")), "closing-stay", BranchMode.Created, Some(base) @@ -3572,7 +3580,7 @@ class FlowLifecycleTest extends munit.FunSuite: // The published line is the only difference from the unpublished runs // below. val workDir = GitRepo.seeded() - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) given WorkspaceWrite = WorkspaceWrite.unsafe val base = git.headCommit().get val _ = git.createBranch(branchName("closing-work")) @@ -3580,7 +3588,8 @@ class FlowLifecycleTest extends munit.FunSuite: os.write(workDir / "b.txt", "two") assert(git.commit("the run's work").isRight) val setup = closingSetup( - ProgressStore.default(workDir, RunKey.of("closing-work")), + ProgressStore + .default(Effects.fs, workDir, RunKey.of("closing-work")), "closing-work", BranchMode.Created, Some(base) @@ -3599,13 +3608,14 @@ class FlowLifecycleTest extends munit.FunSuite: "closing summary: a throwaway branch is deleted first, so the block names the branch the user is left on" ): val workDir = GitRepo.seeded() - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) given WorkspaceWrite = WorkspaceWrite.unsafe val base = git.headCommit().get val _ = git.createBranch(branchName("closing-throwaway")) // no user code on it val setup = closingSetup( - ProgressStore.default(workDir, RunKey.of("closing-throwaway")), + ProgressStore + .default(Effects.fs, workDir, RunKey.of("closing-throwaway")), "closing-throwaway", BranchMode.Created, Some(base) @@ -3617,13 +3627,14 @@ class FlowLifecycleTest extends munit.FunSuite: test("closing summary: no recorded starting commit omits the file count"): val workDir = GitRepo.seeded() - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) given WorkspaceWrite = WorkspaceWrite.unsafe val _ = git.createBranch(branchName("closing-no-base")) os.write(workDir / "a.txt", "one") assert(git.commit("the run's work").isRight) val setup = closingSetup( - ProgressStore.default(workDir, RunKey.of("closing-no-base")), + ProgressStore + .default(Effects.fs, workDir, RunKey.of("closing-no-base")), "closing-no-base", BranchMode.Reused, startingCommit = None @@ -3638,8 +3649,9 @@ class FlowLifecycleTest extends munit.FunSuite: ): val workDir = GitRepo.seeded() val prompt = "resume-announcement" - val store = ProgressStore.default(workDir, RunKey.of(prompt)) - val git = new OsGitTool(workDir) + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of(prompt)) + val git = Effects.gitTool(workDir) given WorkspaceWrite = WorkspaceWrite.unsafe val base = git.headCommit().get val _ = git.createBranch(branchName("resumed-work")) @@ -3702,10 +3714,11 @@ class FlowLifecycleTest extends munit.FunSuite: // cross-check). val workDir = GitRepo.seeded() val prompt = "skip-branch-resume" - val store = ProgressStore.default(workDir, RunKey.of(prompt)) + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of(prompt)) val invocations = new AtomicInteger(0) given WorkspaceWrite = WorkspaceWrite.unsafe - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) val _ = git.createBranch(branchName("Feature/JIRA-123")) store.writeHeader( ProgressHeader( @@ -3767,10 +3780,11 @@ class FlowLifecycleTest extends munit.FunSuite: // discarding already-recorded progress. val workDir = GitRepo.seeded() val prompt = "resume-dirty-json-break" - val store = ProgressStore.default(workDir, RunKey.of(prompt)) + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of(prompt)) val invocations = new AtomicInteger(0) given WorkspaceWrite = WorkspaceWrite.unsafe - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) val _ = git.createBranch(branchName("feat/resume-dirty-json-break")) store.writeHeader( ProgressHeader( @@ -3830,10 +3844,11 @@ class FlowLifecycleTest extends munit.FunSuite: // every completed stage over the interrupted stage's leftovers. val workDir = GitRepo.seeded() val prompt = "keep-changes-dirty-json-break" - val store = ProgressStore.default(workDir, RunKey.of(prompt)) + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of(prompt)) val invocations = new AtomicInteger(0) given WorkspaceWrite = WorkspaceWrite.unsafe - val git = new OsGitTool(workDir) + val git = Effects.gitTool(workDir) val _ = git.createBranch(branchName("feat/keep-changes-dirty-json-break")) store.writeHeader( ProgressHeader( @@ -3890,8 +3905,9 @@ class FlowLifecycleTest extends munit.FunSuite: // marked as surfaced. val workDir = GitRepo.seeded() val prompt = "surfaced-tampered" - val store = ProgressStore.default(workDir, RunKey.of(prompt)) - val git = new OsGitTool(workDir) + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of(prompt)) + val git = Effects.gitTool(workDir) given WorkspaceWrite = WorkspaceWrite.unsafe val _ = git.createBranch(branchName("feat/surfaced-tampered")) store.writeHeader( @@ -3936,8 +3952,9 @@ class FlowLifecycleTest extends munit.FunSuite: // `FlowLifecycle.setup`, not just at the `RecoveryCheckTest` unit level. val workDir = GitRepo.seeded() val prompt = "resume-protected-branch" - val store = ProgressStore.default(workDir, RunKey.of(prompt)) - val git = new OsGitTool(workDir) + val store = + ProgressStore.default(Effects.fs, workDir, RunKey.of(prompt)) + val git = Effects.gitTool(workDir) given WorkspaceWrite = WorkspaceWrite.unsafe val _ = git.createBranch(branchName("feat/resume-protected-branch")) store.writeHeader( @@ -4182,7 +4199,8 @@ class FlowLifecycleTest extends munit.FunSuite: * the reset error is attached as suppressed rather than masking the * original. */ - private class ResetThrowingGit(workDir: os.Path) extends OsGitTool(workDir): + private class ResetThrowingGit(workDir: os.Path) + extends OsGitTool(workDir, Effects.procs, Effects.fs): override def discardUncommitted(untracked: UntrackedFiles)(using WorkspaceWrite ): Unit = diff --git a/runner/src/test/scala/orca/runner/LockHolder.scala b/runner/src/test/scala/orca/runner/LockHolder.scala index e22f446d9..fb99c19ef 100644 --- a/runner/src/test/scala/orca/runner/LockHolder.scala +++ b/runner/src/test/scala/orca/runner/LockHolder.scala @@ -26,7 +26,7 @@ object LockHolder: /** Starts a holder of `lock` and waits until it is [[Ready]]. */ def spawn(lock: Lock): os.SubProcess = val args = lock match - case Lock.Workdir(workDir) => List(workDir.toString) + case Lock.Workdir(workDir) => List(workDir.toString) case Lock.Worktree(mainCheckout, prompt) => List(mainCheckout.toString, prompt) os.proc( @@ -59,7 +59,7 @@ object LockHolder: Option(holder.stdout.readLine()) match case Some(AcquiredLine) => Outcome.Acquired case Some(refusal) => Outcome.Refused(refusal) - case None => + case None => throw new IllegalStateException( s"the lock holder exited with code ${holder.wrapped.waitFor()}" ) @@ -69,7 +69,7 @@ object LockHolder: spawn(lock).tap: holder => go(holder) outcome(holder) match - case Outcome.Acquired => () + case Outcome.Acquired => () case Outcome.Refused(message) => throw new IllegalStateException( s"the lock holder was refused: $message" @@ -84,7 +84,7 @@ object LockHolder: def main(args: Array[String]): Unit = val lock = args.toList match - case List(workDir) => Lock.Workdir(os.Path(workDir)) + case List(workDir) => Lock.Workdir(os.Path(workDir)) case List(mainCheckout, prompt) => Lock.Worktree(os.Path(mainCheckout), prompt) case other => sys.error(s"unexpected arguments: $other") diff --git a/runner/src/test/scala/orca/runner/ReviewerDiscoveryFlowTest.scala b/runner/src/test/scala/orca/runner/ReviewerDiscoveryFlowTest.scala index fc4e7e8e4..44914ef3e 100644 --- a/runner/src/test/scala/orca/runner/ReviewerDiscoveryFlowTest.scala +++ b/runner/src/test/scala/orca/runner/ReviewerDiscoveryFlowTest.scala @@ -2,8 +2,7 @@ package orca.runner import orca.ReportedFailure import orca.{ConfigHome, FlowContext, OrcaDir, StackSettings} -import orca.testkit.{GitRepo, TempDirs, currentBranch} -import orca.tools.OsGitTool +import orca.testkit.{Effects, GitRepo, TempDirs, currentBranch} import java.util.concurrent.atomic.AtomicReference @@ -51,12 +50,12 @@ class ReviewerDiscoveryFlowTest extends munit.FunSuite: "---\nname: orca\n---\n\n## Scope\n", createFolders = true ) - val startBranch = new OsGitTool(workDir).currentBranch() + val startBranch = Effects.gitTool(workDir).currentBranch() val e = intercept[ReportedFailure]: driveFlow(workDir)(fail("the body must not run")) assert(e.cause.getMessage.contains("description:"), e.cause.getMessage) // The abort precedes `ensureClean` and any branch creation. - assertEquals(new OsGitTool(workDir).currentBranch(), startBranch) + assertEquals(Effects.gitTool(workDir).currentBranch(), startBranch) test("a symlinked .orca/reviewers directory aborts before the branch"): // `os.isDir` follows links, so the per-file check cannot see this; the @@ -66,11 +65,11 @@ class ReviewerDiscoveryFlowTest extends munit.FunSuite: writeReviewer(outside, "from outside the tree") os.makeDir.all(OrcaDir.rootPath(workDir)) os.symlink(OrcaDir.reviewersPath(workDir), outside) - val startBranch = new OsGitTool(workDir).currentBranch() + val startBranch = Effects.gitTool(workDir).currentBranch() val e = intercept[ReportedFailure]: driveFlow(workDir)(fail("the body must not run")) assert(e.cause.getMessage.contains("symlink"), e.cause.getMessage) - assertEquals(new OsGitTool(workDir).currentBranch(), startBranch) + assertEquals(Effects.gitTool(workDir).currentBranch(), startBranch) private def writeReviewer(dir: os.Path, description: String): Unit = os.write( diff --git a/runner/src/test/scala/orca/runner/RoleSettingsFlowTest.scala b/runner/src/test/scala/orca/runner/RoleSettingsFlowTest.scala index 210184907..ea6d83911 100644 --- a/runner/src/test/scala/orca/runner/RoleSettingsFlowTest.scala +++ b/runner/src/test/scala/orca/runner/RoleSettingsFlowTest.scala @@ -1,12 +1,12 @@ package orca.runner +import orca.lifecycle.{DiscoveredCommand, DiscoveredGate, StackDiscoveryResult} import orca.ReportedFailure import orca.{ConfigHome, FlowContext, OrcaDir, OrcaFlowException, StackSettings} import orca.agents.{Agent, BackendTag, CodexAgent, GeminiAgent, Model} import orca.events.OrcaListener import orca.settings.SettingsFile -import orca.testkit.{GitRepo, TempDirs, currentBranch} -import orca.tools.OsGitTool +import orca.testkit.{Effects, GitRepo, TempDirs, currentBranch} import java.util.concurrent.atomic.AtomicReference @@ -320,7 +320,7 @@ class RoleSettingsFlowTest extends munit.FunSuite: val linkPath = OrcaDir.settingsPath(workDir) os.makeDir.all(linkPath / os.up) os.symlink(linkPath, outside) - val startBranch = new OsGitTool(workDir).currentBranch() + val startBranch = Effects.gitTool(workDir).currentBranch() // A canned discovery agent that would succeed and write, so it's the abort, // not a discovery failure, that keeps the target file from being created. val canned = CannedDiscoveryAgent( @@ -339,7 +339,7 @@ class RoleSettingsFlowTest extends munit.FunSuite: "no discovery write must go through the symlink" ) assertEquals( - new OsGitTool(workDir).currentBranch(), + Effects.gitTool(workDir).currentBranch(), startBranch, "the symlink abort must precede any branch mutation" ) @@ -356,7 +356,7 @@ class RoleSettingsFlowTest extends munit.FunSuite: val outside = TempDirs.dir() / "outside-orca" os.makeDir.all(outside) os.symlink(OrcaDir.rootPath(workDir), outside) - val startBranch = new OsGitTool(workDir).currentBranch() + val startBranch = Effects.gitTool(workDir).currentBranch() // A canned discovery agent that would succeed and write, so it's the abort, // not a discovery failure, that keeps the target from being written. val canned = CannedDiscoveryAgent( @@ -375,7 +375,7 @@ class RoleSettingsFlowTest extends munit.FunSuite: "no write must go through the symlinked `.orca` directory" ) assertEquals( - new OsGitTool(workDir).currentBranch(), + Effects.gitTool(workDir).currentBranch(), startBranch, "the symlinked-`.orca` abort must precede any branch mutation" ) @@ -394,7 +394,7 @@ class RoleSettingsFlowTest extends munit.FunSuite: os.makeDir.all(outside) os.makeDir.all(OrcaDir.rootPath(workDir)) os.symlink(OrcaDir.rootPath(workDir) / "cache", outside) - val startBranch = new OsGitTool(workDir).currentBranch() + val startBranch = Effects.gitTool(workDir).currentBranch() // A canned discovery agent that would succeed and write, so it's the abort, // not a discovery failure, that keeps the target from being written. val canned = CannedDiscoveryAgent( @@ -413,7 +413,7 @@ class RoleSettingsFlowTest extends munit.FunSuite: "no write must go through the symlinked `.orca/cache` directory" ) assertEquals( - new OsGitTool(workDir).currentBranch(), + Effects.gitTool(workDir).currentBranch(), startBranch, "the symlinked-`.orca/cache` abort must precede any branch mutation" ) @@ -480,7 +480,7 @@ class RoleSettingsFlowTest extends munit.FunSuite: configHome: ConfigHome = absentGlobal(), stackSettings: Option[StackSettings] = None ): Unit = - val startBranch = new OsGitTool(workDir).currentBranch() + val startBranch = Effects.gitTool(workDir).currentBranch() val _ = intercept[ReportedFailure]: driveFlow( workDir, @@ -489,7 +489,7 @@ class RoleSettingsFlowTest extends munit.FunSuite: wiring = wiringWith(claude = StubAgent.claude) )(()) assertEquals( - new OsGitTool(workDir).currentBranch(), + Effects.gitTool(workDir).currentBranch(), startBranch, "the malformed-file abort must precede any branch mutation" ) diff --git a/runner/src/test/scala/orca/runner/WorktreeRunTest.scala b/runner/src/test/scala/orca/runner/WorktreeRunTest.scala index 0aab6ccf0..eeb514c06 100644 --- a/runner/src/test/scala/orca/runner/WorktreeRunTest.scala +++ b/runner/src/test/scala/orca/runner/WorktreeRunTest.scala @@ -1,128 +1,20 @@ package orca.runner +import orca.lifecycle.WorktreeRun import orca.{OrcaDir, OrcaFlowException, RunKey} -import orca.progress.ProgressStore -import orca.testkit.{GitRepo, TempDirs} -import orca.tools.Worktrees -import ox.discard +import orca.testkit.{Effects, GitRepo} class WorktreeRunTest extends munit.FunSuite: - test("the worktree lives under .orca/worktrees, keyed by the prompt"): - val repo = GitRepo.seeded() - val path = resolved(repo, "task A") - assertEquals(path / os.up, OrcaDir.worktreesPath(repo)) - assertNotEquals(path, resolved(repo, "task B")) - // Same key as the progress log inside it: that coupling is what makes a - // resumed run land where its log is. - assert( - ProgressStore - .default(path, RunKey.of("task A")) - .path - .last - .contains(path.last), - path.last - ) - - test("re-resolving the same prompt reuses the worktree already there"): - val repo = GitRepo.seeded() - val first = resolved(repo, "task A") - assertEquals(resolved(repo, "task A"), first) - // The main checkout plus exactly one worktree: no second one was created. - assertEquals(Worktrees.list(repo), List(repo, first)) - - test("a new worktree is put on a branch of its own, not left detached"): - val repo = GitRepo.seeded() - val path = resolved(repo, "task A") - assert(Worktrees.onABranch(path)) - - test("resolving from inside the worktree lands on that same worktree"): - val repo = GitRepo.seeded() - val path = resolved(repo, "task A") - assertEquals(WorktreeRun.resolve(path, RunKey.of("task A")), Right(path)) - - test("the ignore marker is written before the worktree is created"): - val repo = GitRepo.empty() - // Creation cannot succeed on an unborn HEAD, so the marker is on disk only - // if it was written first. - assert(WorktreeRun.resolve(repo, RunKey.of("task A")).isLeft) - assert(os.exists(OrcaDir.worktreesPath(repo) / ".gitignore")) - - test("reuse puts the ignore marker back when it went missing"): - val repo = GitRepo.seeded() - val path = resolved(repo, "task A") - // What `git clean -xdf` in the main checkout removes: the marker only, the - // worktree itself skipped as a repository. - os.remove(OrcaDir.worktreesPath(repo) / ".gitignore").discard - assertEquals(WorktreeRun.resolve(repo, RunKey.of("task A")), Right(path)) - assert(os.exists(OrcaDir.worktreesPath(repo) / ".gitignore")) - - test("a worktree whose directory was deleted is recreated"): - val repo = GitRepo.seeded() - val path = resolved(repo, "task A") - // What `git clean -xdff` leaves: the directory gone, git's administrative - // entry still listing it. - os.remove.all(path) - assertEquals(WorktreeRun.resolve(repo, RunKey.of("task A")), Right(path)) - assert(os.exists(path / ".git")) - - test("recreating one worktree leaves the repository's others registered"): - val repo = GitRepo.seeded() - val path = resolved(repo, "task A") - val unrelated = repo / "unrelated" - assertEquals(Worktrees.add(repo, unrelated), Right(())) - // Both directories unreachable — a stale entry orca does not own must - // survive the reclaim of the one it does. - os.remove.all(path) - os.remove.all(unrelated) - assertEquals(WorktreeRun.resolve(repo, RunKey.of("task A")), Right(path)) - assert(Worktrees.list(repo).contains(unrelated)) - - test("a plain directory at the path is refused, not taken over"): - val repo = GitRepo.seeded() - val path = resolved(repo, "task A") - git(repo, "worktree", "remove", "--force", path.toString) - os.makeDir.all(path) - val refusal = refusalOf(repo) - assert(refusal.contains(path.toString), refusal) - - test("outside a git repository, orca's own wording is used"): - assertEquals( - WorktreeRun.resolve(TempDirs.dir(), RunKey.of("task A")), - Left(GitPreconditions.needsRepoWithCommit) - ) - - test("a repository without commits gets orca's wording, not git's"): - assertEquals( - WorktreeRun.resolve(GitRepo.empty(), RunKey.of("task A")), - Left(GitPreconditions.needsRepoWithCommit) - ) - - test("a main worktree that is not a checkout is named as the reason"): - val checkout = GitRepo.seededSeparateGitDir() - val worktree = checkout / os.up / "wt" - assertEquals(Worktrees.add(checkout, worktree), Right(())) - // Resolving from here finds no main checkout, and the user is told it is - // the repository's layout — not that they have no repository. - val refusal = refusalOf(worktree) - assert(refusal.contains("--separate-git-dir"), refusal) - - test("a git that does not answer the query is named as the reason"): - // rev-parse prints one path per line, so a repository path holding a - // newline makes the answer unreadable — as does a git too old for - // `--path-format`, which is what the user is pointed at. - val repo = TempDirs.dir() / "two\nlines" - os.makeDir.all(repo) - git(repo, "init", "-b", "main") - val refusal = refusalOf(repo) - assert(refusal.contains("git 2.31 or newer"), refusal) + private val worktreeRun = + WorktreeRun(Effects.procs, Effects.fs, FlowLock) test("resolving is refused while a live process holds the worktree lock"): val repo = GitRepo.seeded() val holder = LockHolder.acquire(LockHolder.Lock.Worktree(repo, "task A")) try val thrown = intercept[OrcaFlowException]: - WorktreeRun.resolve(repo, RunKey.of("task A")) + worktreeRun.resolve(repo, RunKey.of("task A")) assertEquals( thrown.getMessage, s"a flow is already running for this prompt (pid " + @@ -130,54 +22,3 @@ class WorktreeRunTest extends munit.FunSuite: ) assert(!os.exists(OrcaDir.worktreesPath(repo)), "nothing was created") finally LockHolder.kill(holder) - - test("reuse puts a detached worktree back on its branch"): - val repo = GitRepo.seeded() - val path = resolved(repo, "task A") - // How a half-created worktree looks: `add` succeeded, the branch step did - // not. - git(path, "checkout", "--detach", "-q") - assert(!Worktrees.onABranch(path)) - assertEquals(WorktreeRun.resolve(repo, RunKey.of("task A")), Right(path)) - assert(Worktrees.onABranch(path)) - - test("reuse refuses when the run's branch has gained commits"): - val repo = GitRepo.seeded() - val path = resolved(repo, "task A") - val branch = s"orca-worktree-${path.last}" - os.write(path / "work.txt", "work") - git(path, "add", "-A") - git(path, "commit", "-q", "-m", "work in the worktree") - // Detached BEHIND the branch, so putting the run back on it would strand - // that commit — which is what orca refuses to do. - git(path, "checkout", "--detach", "-q", "HEAD~1") - val refusal = refusalOf(repo) - assert(refusal.contains(branch), refusal) - // The remedy names the branch, which is what holds the commits. Removing - // the worktree does not help: the next run recreates it detached and hits - // this same refusal. - assert(refusal.contains(s"git branch -D $branch"), refusal) - - test("isWorktreeRun recognises orca's own worktree and nothing else"): - val repo = GitRepo.seeded() - val ours = resolved(repo, "task A") - val theirs = repo / "review-their-branch" - assertEquals(Worktrees.add(repo, theirs), Right(())) - assert(WorktreeRun.isWorktreeRun(ours), "orca's own worktree") - assert(!WorktreeRun.isWorktreeRun(repo), "the main checkout is not one") - assert(!WorktreeRun.isWorktreeRun(theirs), "a worktree orca did not make") - assert(!WorktreeRun.isWorktreeRun(TempDirs.dir()), "not a repository") - - private def resolved(repo: os.Path, userPrompt: String): os.Path = - WorktreeRun - .resolve(repo, RunKey.of(userPrompt)) - .getOrElse(fail("expected a resolved worktree")) - - private def refusalOf(cwd: os.Path): String = - WorktreeRun - .resolve(cwd, RunKey.of("task A")) - .left - .getOrElse(fail("expected a refusal")) - - private def git(cwd: os.Path, args: String*): Unit = - os.proc("git" +: args).call(cwd = cwd).discard diff --git a/runner/src/test/scala/orca/runner/scalacli/ScalaCliSmokeTest.scala b/runner/src/test/scala/orca/runner/scalacli/ScalaCliSmokeTest.scala index 93b5ed91e..231ad8ad5 100644 --- a/runner/src/test/scala/orca/runner/scalacli/ScalaCliSmokeTest.scala +++ b/runner/src/test/scala/orca/runner/scalacli/ScalaCliSmokeTest.scala @@ -23,11 +23,12 @@ class ScalaCliSmokeTest extends munit.FunSuite: import scala.concurrent.duration.DurationInt 20.minutes - // Keep in sync with V.scala (project/Dependencies.scala) — not reachable - // from test code (build-definition sources aren't on the runtime/test - // classpath), so pinned here as a literal. A stale value here fails loudly: - // the published library's TASTy won't parse under an older scala-cli pin. - private val scalaVersion = "3.9.0" + private val scalaVersion: String = sys.props.getOrElse( + "orca.build.scalaVersion", + sys.error( + "-Dorca.build.scalaVersion unset — see scalaVersionProperty in build.sbt" + ) + ) private val version: String = sys.props.getOrElse( "orca.build.version", diff --git a/tools/src/test/scala/orca/subprocess/TtyProbeTest.scala b/runner/src/test/scala/orca/runner/terminal/TtyProbeTest.scala similarity index 93% rename from tools/src/test/scala/orca/subprocess/TtyProbeTest.scala rename to runner/src/test/scala/orca/runner/terminal/TtyProbeTest.scala index 4da656755..bb5b1e9a5 100644 --- a/tools/src/test/scala/orca/subprocess/TtyProbeTest.scala +++ b/runner/src/test/scala/orca/runner/terminal/TtyProbeTest.scala @@ -1,4 +1,4 @@ -package orca.subprocess +package orca.runner.terminal class TtyProbeTest extends munit.FunSuite: diff --git a/shell/src/main/scala/orca/shell/Main.scala b/shell/src/main/scala/orca/shell/Main.scala index 328c4cd0c..0342d70f7 100644 --- a/shell/src/main/scala/orca/shell/Main.scala +++ b/shell/src/main/scala/orca/shell/Main.scala @@ -1,6 +1,7 @@ package orca.shell import orca.shell.cli.{Cli, CliHelp} +import orca.effect.Processes import orca.shell.menu.{MenuContext, SettingsMenu, ShellMenu} import orca.shell.ui.{ShellOutput, ShellUi} import orca.shell.wizard.{FirstRun, FirstRunStatus, Wizard} @@ -20,7 +21,7 @@ object Main: def main(args: Array[String]): Unit = given ShellEnv = ShellEnv.current() args.headOption match - case None => runInteractiveShell() + case None => runInteractiveShell() case Some("--help") | Some("-h") | Some("help") => println(CliHelp.topLevel) sys.exit(0) @@ -57,7 +58,7 @@ object Main: val ui = ShellUi.make(terminal) val wizard = Wizard( ui, - PathProbe.resolves(_, env.workDir), + PathProbe.resolves(Processes.ambient, _, env.workDir), env.configHome.settings ) runWizardIfFirstRun(wizard, env.configHome.settings) @@ -78,7 +79,7 @@ object Main: case Right(FirstRunStatus.FirstRun) => wizard.run(reconfigure = false).discard case Right(FirstRunStatus.AlreadyConfigured) => () - case Left(error) => + case Left(error) => ShellOutput.error( s"the global settings file is malformed — ${error.message}" ) diff --git a/shell/src/main/scala/orca/shell/OrcaBuild.scala b/shell/src/main/scala/orca/shell/OrcaBuild.scala index 216b3c761..c1240944c 100644 --- a/shell/src/main/scala/orca/shell/OrcaBuild.scala +++ b/shell/src/main/scala/orca/shell/OrcaBuild.scala @@ -19,7 +19,7 @@ private[shell] enum OrcaBuild: /** The `//> using` lines that make a flow script resolve this build. */ def usingDirectives: String = this match - case Release(v) => s"""//> using dep "${OrcaBuild.Module}:$v"""" + case Release(v) => s"""//> using dep "${OrcaBuild.Module}:$v"""" case Snapshot(v) => s"""//> using dep "${OrcaBuild.Module}:$v" |//> using repository ivy2Local""".stripMargin @@ -28,7 +28,7 @@ private[shell] enum OrcaBuild: * own orca pin. */ def forceArgs: Seq[String] = this match - case Release(v) => Seq("--dep", s"${OrcaBuild.Module}:$v") + case Release(v) => Seq("--dep", s"${OrcaBuild.Module}:$v") case Snapshot(v) => Seq("--dep", s"${OrcaBuild.Module}:$v", "--repository", "ivy2Local") @@ -49,3 +49,7 @@ private[shell] object OrcaBuild: /** This shell's build, from the version resource `build.sbt` generates. */ lazy val current: OrcaBuild = of(PromptResource.load("/orca/shell/version").trim) + + /** The Scala version flow scripts pin: the one orca is built with. */ + lazy val scalaVersion: String = + PromptResource.load("/orca/shell/scala-version").trim diff --git a/shell/src/main/scala/orca/shell/Tier.scala b/shell/src/main/scala/orca/shell/Tier.scala index 2f54558cb..9c1261f45 100644 --- a/shell/src/main/scala/orca/shell/Tier.scala +++ b/shell/src/main/scala/orca/shell/Tier.scala @@ -1,6 +1,7 @@ package orca.shell import orca.OrcaDir +import orca.effect.FileSystem import ox.tap /** Where the shell writes a flow or settings file: the project's committed @@ -17,8 +18,9 @@ private[shell] object Tier: */ def ensureFlowsDir(using env: ShellEnv): os.Path = tier match - case Tier.Project => OrcaDir.ensureFlows(env.workDir) - case Tier.Global => env.configHome.flows.tap(os.makeDir.all(_)) + case Tier.Project => + OrcaDir(FileSystem.ambient).ensureFlows(env.workDir) + case Tier.Global => env.configHome.flows.tap(os.makeDir.all(_)) /** The tier's settings file: `.orca/settings.properties` under `workDir`, * or the global `settings.properties`. diff --git a/shell/src/main/scala/orca/shell/WorktreeScan.scala b/shell/src/main/scala/orca/shell/WorktreeScan.scala index 34d12dbd9..5e6fdc00d 100644 --- a/shell/src/main/scala/orca/shell/WorktreeScan.scala +++ b/shell/src/main/scala/orca/shell/WorktreeScan.scala @@ -1,5 +1,6 @@ package orca.shell +import orca.effect.{FileSystem, Processes} import orca.OrcaDir import orca.tools.Worktrees @@ -57,7 +58,7 @@ private[shell] object WorktreeScan: val root = OrcaDir.worktreesPath(workDir) ScanDirs( workDir, - Worktrees + Worktrees(Processes.ambient, FileSystem.ambient) .list(workDir) .filter(_ / os.up == root) // Decorate-sort-undecorate: `sortBy` re-evaluates its key on every diff --git a/shell/src/main/scala/orca/shell/actions/AuthorAction.scala b/shell/src/main/scala/orca/shell/actions/AuthorAction.scala index e4eea0f19..2f531383b 100644 --- a/shell/src/main/scala/orca/shell/actions/AuthorAction.scala +++ b/shell/src/main/scala/orca/shell/actions/AuthorAction.scala @@ -2,6 +2,7 @@ package orca.shell.actions import org.jline.terminal.Terminal import orca.{OrcaArgs, OrcaDir, RunTarget, Uncommitted} +import orca.effect.FileSystem import orca.shell.{OrcaBuild, ShellEnv} import orca.shell.create.{ AuthoringSandbox, @@ -155,7 +156,7 @@ private[shell] object AuthorAction: private def extractApiMaterial(sandbox: os.Path): os.Path = FlowAuthoring.extractApiMaterial( - OrcaDir.ensureCache(sandbox), + OrcaDir(FileSystem.ambient).ensureCache(sandbox), OrcaBuild.current.version ) diff --git a/shell/src/main/scala/orca/shell/actions/ConfigSummary.scala b/shell/src/main/scala/orca/shell/actions/ConfigSummary.scala index a2ddec222..c6b95c56b 100644 --- a/shell/src/main/scala/orca/shell/actions/ConfigSummary.scala +++ b/shell/src/main/scala/orca/shell/actions/ConfigSummary.scala @@ -1,7 +1,8 @@ package orca.shell.actions +import orca.effect.{FileSystem, Processes} import orca.gitref.Head -import orca.runner.RoleAgents +import orca.lifecycle.RoleAgents import orca.settings.{AgentSpec, StackKey} import orca.tools.OsGitTool @@ -30,8 +31,8 @@ private[shell] object ConfigSummary: ConfigAction.show(globalSettingsPath), ConfigAction.showProject(workDir) ) match - case (Left(error), _) => s"agents: $error" - case (_, Left(error)) => s"agents: $error" + case (Left(error), _) => s"agents: $error" + case (_, Left(error)) => s"agents: $error" case (Right(global), Right(project)) => val roles = List( "planning" -> RoleAgents.projectOverGlobal( @@ -56,7 +57,9 @@ private[shell] object ConfigSummary: */ def branchLine(workDir: os.Path): Option[String] = scala.util - .Try(new OsGitTool(workDir).head()) + .Try( + new OsGitTool(workDir, Processes.ambient, FileSystem.ambient).head() + ) .toOption .map: case Head.OnBranch(name) => s"branch: ${name.value}" diff --git a/shell/src/main/scala/orca/shell/actions/FlowResolution.scala b/shell/src/main/scala/orca/shell/actions/FlowResolution.scala index b0732e82e..fd07dc1cd 100644 --- a/shell/src/main/scala/orca/shell/actions/FlowResolution.scala +++ b/shell/src/main/scala/orca/shell/actions/FlowResolution.scala @@ -1,6 +1,7 @@ package orca.shell.actions import orca.OrcaDir +import orca.effect.FileSystem import orca.shell.{OrcaBuild, ShellEnv} import orca.discovery.Origin import orca.progress.FlowSource @@ -31,7 +32,8 @@ private[shell] object FlowResolution: def list(using env: ShellEnv): Either[String, List[DiscoveredFlow]] = val projectFlows = OrcaDir.flowsPath(env.workDir) try - OrcaDir.assertNoOrcaSymlinks(env.workDir, projectFlows) + OrcaDir(FileSystem.ambient) + .assertNoOrcaSymlinks(env.workDir, projectFlows) Right( FlowCatalog.list( projectFlows, @@ -59,7 +61,7 @@ private[shell] object FlowResolution: ) pathHit match case Some(path) => Right(fromPath(path)) - case None => + case None => if ref.contains("/") then Left(s"no such flow file: $ref") else byName(ref) @@ -81,7 +83,7 @@ private[shell] object FlowResolution: ): Either[String, DiscoveredFlow] = source match case FlowSource.Catalog(name) => byName(name) - case FlowSource.File(path) => + case FlowSource.File(path) => recordedFile(path).filter(os.isFile) match case Some(file) => Right(fromPath(file)) case None => Left(s"no flow file at $path") @@ -108,7 +110,7 @@ private[shell] object FlowResolution: flows: List[DiscoveredFlow] ): String = nearMatches(ref, flows) match - case Nil => s"no flow named '$ref' found in the catalog" + case Nil => s"no flow named '$ref' found in the catalog" case matches => s"no flow named '$ref'; did you mean: ${matches.mkString(", ")}?" diff --git a/shell/src/main/scala/orca/shell/actions/SessionAction.scala b/shell/src/main/scala/orca/shell/actions/SessionAction.scala index dc678144c..cfb2d429d 100644 --- a/shell/src/main/scala/orca/shell/actions/SessionAction.scala +++ b/shell/src/main/scala/orca/shell/actions/SessionAction.scala @@ -7,6 +7,7 @@ import orca.shell.ui.ShellOutput import orca.subprocess.QuietProc import orca.backend.gemini.GeminiSessionList import orca.backend.pi.PiSessionStore +import orca.effect.FileSystem import java.time.Instant import scala.util.Try @@ -76,7 +77,7 @@ private[shell] object SessionAction: s"the recorded pi session id `${displayable(wireId)}` is not a directory name" ) .flatMap: dir => - if PiSessionStore.resumable(dir, workDir, Instant.now()) then Right(dir) + if piSessions.resumable(dir, workDir, Instant.now()) then Right(dir) else Left( s"no resumable pi transcript at $dir — pruned by the session-cache " + @@ -84,6 +85,8 @@ private[shell] object SessionAction: "checkout path, or never written" ) + private val piSessions = PiSessionStore(FileSystem.ambient) + // The id just failed the safe-charset check, so it's arbitrary bytes from an // editable file — never interpolated into terminal output raw. private def displayable(raw: String): String = @@ -110,7 +113,7 @@ private[shell] object SessionAction: selection: SessionSelection ): Either[String, Int] = validatedWorkDir(selection.attempt.workDir) match - case Left(reason) => Left(s"can't resume — $reason") + case Left(reason) => Left(s"can't resume — $reason") case Right(workDir) => def geminiIndex(uuid: String): Option[Int] = try @@ -123,7 +126,7 @@ private[shell] object SessionAction: piSessionDir = piSessionDir(_, workDir) ) match case Left(reason) => Left(s"can't resume — $reason") - case Right(argv) => + case Right(argv) => try val exitCode = ChildTerminal.withChild(terminal): os.proc(argv) diff --git a/shell/src/main/scala/orca/shell/actions/SettingsEditAction.scala b/shell/src/main/scala/orca/shell/actions/SettingsEditAction.scala index 200ac80fc..bedbb54fd 100644 --- a/shell/src/main/scala/orca/shell/actions/SettingsEditAction.scala +++ b/shell/src/main/scala/orca/shell/actions/SettingsEditAction.scala @@ -2,6 +2,7 @@ package orca.shell.actions import orca.OrcaDir import orca.agents.BackendTag +import orca.effect.FileSystem import orca.settings.{AgentSettings, AgentSpec, SettingsFile} import orca.shell.{ShellEnv, Tier} import ox.discard @@ -47,9 +48,9 @@ private[shell] object SettingsEditAction: case Tier.Global => if !os.exists(path) then ConfigAction.set(path, AgentSettings.empty) case Tier.Project => - OrcaDir.assertNoOrcaSymlinks(env.workDir, path) + OrcaDir(FileSystem.ambient).assertNoOrcaSymlinks(env.workDir, path) if !os.exists(path) then - OrcaDir.ensureRoot(env.workDir).discard + OrcaDir(FileSystem.ambient).ensureRoot(env.workDir).discard os.write(path, ProjectTemplate) /** Re-parses `tier`'s settings file after the editor exits, reusing diff --git a/shell/src/main/scala/orca/shell/actions/StackAction.scala b/shell/src/main/scala/orca/shell/actions/StackAction.scala index fca14f6ee..18f526830 100644 --- a/shell/src/main/scala/orca/shell/actions/StackAction.scala +++ b/shell/src/main/scala/orca/shell/actions/StackAction.scala @@ -1,6 +1,7 @@ package orca.shell.actions import orca.OrcaDir +import orca.effect.FileSystem import orca.StackSettings import orca.settings.{SettingsFile, SettingsScope, StackKey} import orca.shell.ui.ShellOutput @@ -29,7 +30,7 @@ private[shell] object StackAction: def status(workDir: os.Path): Either[String, StackStatus] = val path = OrcaDir.settingsPath(workDir) try - OrcaDir.assertNoOrcaSymlinks(workDir, path) + OrcaDir(FileSystem.ambient).assertNoOrcaSymlinks(workDir, path) if !os.exists(path) then Right(StackStatus.NoSettings) else val content = os.read(path) @@ -51,7 +52,7 @@ private[shell] object StackAction: * run re-triggers discovery. */ def clear(workDir: os.Path, content: String): Unit = - OrcaDir + OrcaDir(FileSystem.ambient) .settingsFile(workDir) .replace(SettingsFile.stripStackLines(content)) diff --git a/shell/src/main/scala/orca/shell/cli/Cli.scala b/shell/src/main/scala/orca/shell/cli/Cli.scala index 05f4a2422..fdf53661e 100644 --- a/shell/src/main/scala/orca/shell/cli/Cli.scala +++ b/shell/src/main/scala/orca/shell/cli/Cli.scala @@ -16,7 +16,7 @@ import orca.shell.{ShellEnv, Tier, WorktreeScan} import orca.shell.run.LaunchResult import orca.shell.sessions.ObservedStatus import orca.shell.ui.ShellUi -import orca.subprocess.TtyProbe +import orca.runner.terminal.TtyProbe /** Exit codes shared by every subcommand (ADR 0021 §10/§4). */ private[shell] object ExitCodes: @@ -129,7 +129,7 @@ private[shell] object Cli: Console.err.println(usage) ExitCodes.UsageError case Right(exitCode: Int) => exitCode - case Right(_) => + case Right(_) => ExitCodes.ActionFailed // unreachable: every @main below returns Int /** `orca --help`'s text: `name`'s own mainargs-rendered signature and diff --git a/shell/src/main/scala/orca/shell/cli/ConfigCli.scala b/shell/src/main/scala/orca/shell/cli/ConfigCli.scala index 369aad5af..818db79db 100644 --- a/shell/src/main/scala/orca/shell/cli/ConfigCli.scala +++ b/shell/src/main/scala/orca/shell/cli/ConfigCli.scala @@ -40,7 +40,7 @@ private[cli] object ConfigCli: ) ) case Some(tier) => runEdit(tier, tty) - case None => + case None => runConfig(env.configHome.settings, planning, coding, review, force) /** `config`'s behavior over the settings file at `path`. */ diff --git a/shell/src/main/scala/orca/shell/cli/RunCli.scala b/shell/src/main/scala/orca/shell/cli/RunCli.scala index 02201077f..44f89399e 100644 --- a/shell/src/main/scala/orca/shell/cli/RunCli.scala +++ b/shell/src/main/scala/orca/shell/cli/RunCli.scala @@ -73,7 +73,7 @@ private[cli] object RunCli: prompt match case Some(text) if text.trim.nonEmpty => Right(text.trim) case Some(_) => Left("prompt can't be empty") - case None => + case None => if tty then Left( "no prompt given, and stdin is a terminal — " + diff --git a/shell/src/main/scala/orca/shell/create/FlowAuthoring.scala b/shell/src/main/scala/orca/shell/create/FlowAuthoring.scala index ccc9b2f5c..3412076cf 100644 --- a/shell/src/main/scala/orca/shell/create/FlowAuthoring.scala +++ b/shell/src/main/scala/orca/shell/create/FlowAuthoring.scala @@ -355,16 +355,13 @@ private[shell] object FlowAuthoring: dir /** The exact `//> using` header lines a new flow file starts with: `build`'s - * [[OrcaBuild.usingDirectives]] between the scala and jvm pins (the - * "3.9.0"/`21` literals are kept in lockstep with `V.scala` in - * `project/Dependencies.scala` by hand — `updateDocs` only rewrites - * `.md`/`.sc` files, so this text is invisible to it). Shared by + * [[OrcaBuild.usingDirectives]] between the scala and jvm pins. Shared by * [[initialPrompt]] (states it as an instruction to the authoring agent) and * [[skeletonFlow]] (writes it verbatim), so a hand-written skeleton and an * agent-authored file pin the same way. */ private def versionPinLines(build: OrcaBuild): String = - s"""//> using scala 3.9.0 + s"""//> using scala ${OrcaBuild.scalaVersion} |${build.usingDirectives} |//> using jvm 21""".stripMargin diff --git a/shell/src/main/scala/orca/shell/menu/AuthoringMenu.scala b/shell/src/main/scala/orca/shell/menu/AuthoringMenu.scala index 5b79f66e4..88138c6e3 100644 --- a/shell/src/main/scala/orca/shell/menu/AuthoringMenu.scala +++ b/shell/src/main/scala/orca/shell/menu/AuthoringMenu.scala @@ -137,7 +137,7 @@ private[menu] object AuthoringMenu: ShellEnv ): Option[FlowDestination] = ui.input("Filename for the new flow", default = Some("new-flow.sc")) match - case UiOutcome.Cancelled => None + case UiOutcome.Cancelled => None case UiOutcome.Selected(name) => val prepared = for diff --git a/shell/src/main/scala/orca/shell/menu/Prompts.scala b/shell/src/main/scala/orca/shell/menu/Prompts.scala index 55ccd4135..93e40f5b5 100644 --- a/shell/src/main/scala/orca/shell/menu/Prompts.scala +++ b/shell/src/main/scala/orca/shell/menu/Prompts.scala @@ -16,7 +16,7 @@ private[menu] object Prompts: emptyError: String ): Option[String] = ui.inputMultiline(label) match - case UiOutcome.Cancelled => None + case UiOutcome.Cancelled => None case UiOutcome.Selected(text) if text.trim.isEmpty => ShellOutput.error(emptyError) nonBlankMultiline(ui, label, emptyError) diff --git a/shell/src/main/scala/orca/shell/menu/ResumeDetector.scala b/shell/src/main/scala/orca/shell/menu/ResumeDetector.scala index 580fe1c69..b4bfb9ec5 100644 --- a/shell/src/main/scala/orca/shell/menu/ResumeDetector.scala +++ b/shell/src/main/scala/orca/shell/menu/ResumeDetector.scala @@ -1,6 +1,7 @@ package orca.shell.menu import orca.gitref.BranchName +import orca.effect.FileSystem import orca.json.JsonFile import orca.progress.{FlowSource, ProgressHeader, ProgressLog, ProgressScan} import orca.shell.actions.FlowResolution @@ -52,7 +53,7 @@ private[menu] object ResumeDetector: def detect(dirs: List[os.Path]): Option[InterruptedRun] = try newestProgressLog(dirs).flatMap: found => - JsonFile.read[ProgressLog](found.path) match + JsonFile(FileSystem.ambient).read[ProgressLog](found.path) match case JsonFile.Read.Loaded(log) => fromHeader(log.header, found) case _ => None catch case scala.util.control.NonFatal(_) => None @@ -98,7 +99,7 @@ private[menu] object ResumeDetector: */ private def logsIn(dir: os.Path): List[FoundLog] = try - ProgressScan + ProgressScan(orca.effect.FileSystem.ambient) .progressLogPaths(dir) .flatMap: path => scala.util diff --git a/shell/src/main/scala/orca/shell/menu/RunMenu.scala b/shell/src/main/scala/orca/shell/menu/RunMenu.scala index 05cd84ffc..1ab32dcac 100644 --- a/shell/src/main/scala/orca/shell/menu/RunMenu.scala +++ b/shell/src/main/scala/orca/shell/menu/RunMenu.scala @@ -99,7 +99,7 @@ private[menu] object RunMenu: ui: ShellUi ): UiOutcome[Option[BranchName]] = ui.input("Branch name (Enter to derive from the prompt)") match - case UiOutcome.Cancelled => UiOutcome.Cancelled + case UiOutcome.Cancelled => UiOutcome.Cancelled case UiOutcome.Selected(raw) if raw.trim.isEmpty => UiOutcome.Selected(None) case UiOutcome.Selected(raw) => diff --git a/shell/src/main/scala/orca/shell/menu/ShellMenu.scala b/shell/src/main/scala/orca/shell/menu/ShellMenu.scala index 52ffbd999..279ac6ab6 100644 --- a/shell/src/main/scala/orca/shell/menu/ShellMenu.scala +++ b/shell/src/main/scala/orca/shell/menu/ShellMenu.scala @@ -54,7 +54,7 @@ private[shell] object ShellMenu: MainMenu.choices(continueSessionCount, resumeOffer, scanDirs.own) ) match case UiOutcome.Cancelled | UiOutcome.Selected(MenuItem.Exit) => () - case UiOutcome.Selected(item) => + case UiOutcome.Selected(item) => handle(item, context, resumeOffer, scanDirs) loop(context) @@ -82,9 +82,9 @@ private[shell] object ShellMenu: SettingsMenu.editSettings(ui, terminal, spawnEditor) case MenuItem.RediscoverStack => SettingsMenu.rediscoverStack(ui) case MenuItem.ViewFlow => viewFlow(ui, context.tty) - case MenuItem.EditFlow => + case MenuItem.EditFlow => AuthoringMenu.editFlow(ui, terminal, spawnEditor) - case MenuItem.RunFlow => RunMenu.runFlow(ui, terminal, launch) + case MenuItem.RunFlow => RunMenu.runFlow(ui, terminal, launch) case MenuItem.CreateFlow => AuthoringMenu.createNewFlow(ui, terminal, spawnEditor) case MenuItem.ForkFlow => @@ -133,7 +133,7 @@ private[shell] object ShellMenu: "Continue which session?", SessionPicker.sessionRows(index, expanded) ) match - case UiOutcome.Cancelled => () + case UiOutcome.Cancelled => () case UiOutcome.Selected(SessionPicker.PickerRow.ShowMore) => continueSession(ui, terminal, index, expanded = true) case UiOutcome.Selected(SessionPicker.PickerRow.Resume(selection)) => diff --git a/shell/src/main/scala/orca/shell/run/FlowLauncher.scala b/shell/src/main/scala/orca/shell/run/FlowLauncher.scala index bc05394f8..9906ea840 100644 --- a/shell/src/main/scala/orca/shell/run/FlowLauncher.scala +++ b/shell/src/main/scala/orca/shell/run/FlowLauncher.scala @@ -287,7 +287,7 @@ private[shell] object FlowLauncher: case NextAction.Succeed => LaunchResult.Ok case NextAction.ReportFailure(exit) => LaunchResult.Failed(exit) case NextAction.CancelledBySignal => LaunchResult.Cancelled - case NextAction.OfferFallback => + case NextAction.OfferFallback => fallback match case FallbackPolicy.Ask(ui) => ui.confirm(fallbackQuestion(build), default = true) match diff --git a/shell/src/main/scala/orca/shell/sessions/ContinuableAttempts.scala b/shell/src/main/scala/orca/shell/sessions/ContinuableAttempts.scala index 1c26161c3..f2e487cfb 100644 --- a/shell/src/main/scala/orca/shell/sessions/ContinuableAttempts.scala +++ b/shell/src/main/scala/orca/shell/sessions/ContinuableAttempts.scala @@ -1,6 +1,7 @@ package orca.shell.sessions import orca.{AttemptId, OrcaDir} +import orca.effect.FileSystem import orca.runlog.{RunEvent, RunEventReader} import java.time.Instant @@ -98,7 +99,7 @@ private[shell] object ContinuableAttempts: */ private def datedEventLogs(workDir: os.Path): List[DatedLog] = val dir = OrcaDir.cacheRunsPath(workDir) - OrcaDir.assertNoOrcaSymlinks(workDir, dir) + OrcaDir(FileSystem.ambient).assertNoOrcaSymlinks(workDir, dir) if !os.exists(dir) then Nil else eventLogs(workDir, dir).flatMap: log => @@ -126,7 +127,7 @@ private[shell] object ContinuableAttempts: * counts. */ private def attemptsWithSessions(log: os.Path): List[AttemptSessions] = - RunEventReader + new RunEventReader(orca.effect.FileSystem.ambient) .readOnly(log, CountedTypes) .groupBy(_.attempt) .toList @@ -155,7 +156,7 @@ private[shell] object ContinuableAttempts: processAlive: AttemptRecord => Boolean ): AttemptListing = val dir = OrcaDir.cacheRunsPath(workDir) - OrcaDir.assertNoOrcaSymlinks(workDir, dir) + OrcaDir(FileSystem.ambient).assertNoOrcaSymlinks(workDir, dir) if !os.exists(dir) then AttemptListing(Nil, Nil) else val perLog = eventLogs(workDir, dir).map(readEventLog(_, processAlive)) @@ -169,7 +170,7 @@ private[shell] object ContinuableAttempts: .toList .map(_ / OrcaDir.EventLogName) .filter: log => - OrcaDir.assertNoOrcaSymlinks(workDir, log) + OrcaDir(FileSystem.ambient).assertNoOrcaSymlinks(workDir, log) os.isFile(log) private def readEventLog( @@ -184,7 +185,7 @@ private[shell] object ContinuableAttempts: * attempts would otherwise drop out of the listing unnoticed. */ private def readEvents(file: os.Path): Either[String, List[RunEvent]] = - RunEventReader + new RunEventReader(orca.effect.FileSystem.ambient) .readOrError(file) .left .map(e => s"skipping $file: ${firstLine(e)}") diff --git a/shell/src/main/scala/orca/shell/sessions/ResumeCommand.scala b/shell/src/main/scala/orca/shell/sessions/ResumeCommand.scala index 2e4394806..04617b32d 100644 --- a/shell/src/main/scala/orca/shell/sessions/ResumeCommand.scala +++ b/shell/src/main/scala/orca/shell/sessions/ResumeCommand.scala @@ -51,7 +51,7 @@ private[shell] object ResumeCommand: Right(Seq(binary, "--resume", wireId)) case BackendTag.Codex => Right(Seq(binary, "resume", wireId)) case BackendTag.Opencode => Right(Seq(binary, "--session", wireId)) - case BackendTag.Gemini => + case BackendTag.Gemini => geminiIndex(wireId) match case Some(index) => Right(Seq(binary, "--resume", index.toString)) diff --git a/shell/src/main/scala/orca/shell/sessions/SessionIndex.scala b/shell/src/main/scala/orca/shell/sessions/SessionIndex.scala index fa644e533..0834df1cf 100644 --- a/shell/src/main/scala/orca/shell/sessions/SessionIndex.scala +++ b/shell/src/main/scala/orca/shell/sessions/SessionIndex.scala @@ -59,7 +59,7 @@ private[shell] case class SessionIndex private ( */ def resolve(selector: Option[String]): Either[String, SessionSelection] = selector match - case None => newest + case None => newest case Some(s) => SessionRef.parse(s) match case Some(ref) => byRef(ref, s) @@ -96,7 +96,7 @@ private[shell] case class SessionIndex private ( ) case (_, Nil) => resolveByName(selector, byName) case (Nil, _) => resolveByBranch(selector, byBranch) - case _ => + case _ => Left( s"'$selector' names both a session and a branch; ${SessionIndex.pickFromList}" ) diff --git a/shell/src/main/scala/orca/shell/ui/ConsoleUiShell.scala b/shell/src/main/scala/orca/shell/ui/ConsoleUiShell.scala index 56d273834..8d78fec3e 100644 --- a/shell/src/main/scala/orca/shell/ui/ConsoleUiShell.scala +++ b/shell/src/main/scala/orca/shell/ui/ConsoleUiShell.scala @@ -100,7 +100,7 @@ private[ui] final class ConsoleUiShell(terminal: Terminal) extends ShellUi: } list.addPrompt().discard runOrCancelled(consolePrompt, builder.build()) match - case UiOutcome.Cancelled => UiOutcome.Cancelled + case UiOutcome.Cancelled => UiOutcome.Cancelled case UiOutcome.Selected(results) => val selectedId = results.get("select").asInstanceOf[ListResult].getSelectedId @@ -126,7 +126,7 @@ private[ui] final class ConsoleUiShell(terminal: Terminal) extends ShellUi: .addPrompt() .discard runOrCancelled(consolePrompt, builder.build()) match - case UiOutcome.Cancelled => UiOutcome.Cancelled + case UiOutcome.Cancelled => UiOutcome.Cancelled case UiOutcome.Selected(results) => val confirmed = results.get("confirm").asInstanceOf[ConfirmResult].getConfirmed @@ -144,7 +144,7 @@ private[ui] final class ConsoleUiShell(terminal: Terminal) extends ShellUi: */ def input(prompt: String, default: Option[String] = None): UiOutcome[String] = default match - case None => plainLineInput(prompt) + case None => plainLineInput(prompt) case Some(text) => val consolePrompt = newConsolePrompt() val builder = consolePrompt.getPromptBuilder @@ -153,7 +153,7 @@ private[ui] final class ConsoleUiShell(terminal: Terminal) extends ShellUi: inputBuilder.defaultValue(text).discard inputBuilder.addPrompt().discard runOrCancelled(consolePrompt, builder.build()) match - case UiOutcome.Cancelled => UiOutcome.Cancelled + case UiOutcome.Cancelled => UiOutcome.Cancelled case UiOutcome.Selected(results) => UiOutcome.Selected( results.get("input").asInstanceOf[InputResult].getResult diff --git a/shell/src/main/scala/orca/shell/ui/NumberedUi.scala b/shell/src/main/scala/orca/shell/ui/NumberedUi.scala index 981527a01..0f40ff26f 100644 --- a/shell/src/main/scala/orca/shell/ui/NumberedUi.scala +++ b/shell/src/main/scala/orca/shell/ui/NumberedUi.scala @@ -51,7 +51,7 @@ private[ui] final class NumberedUi(in: BufferedReader, out: PrintStream) case "" => UiOutcome.Selected(default) case "y" | "yes" => UiOutcome.Selected(true) case "n" | "no" => UiOutcome.Selected(false) - case _ => + case _ => out.println("Please answer y or n.") loop() diff --git a/shell/src/main/scala/orca/shell/wizard/ModelCatalog.scala b/shell/src/main/scala/orca/shell/wizard/ModelCatalog.scala index 41993080c..fb4390f9b 100644 --- a/shell/src/main/scala/orca/shell/wizard/ModelCatalog.scala +++ b/shell/src/main/scala/orca/shell/wizard/ModelCatalog.scala @@ -111,7 +111,7 @@ private[shell] object ModelCatalog: ).flatMap: case ModelPick.Curated(id) => UiOutcome.Selected(Some(id)) case ModelPick.Default => UiOutcome.Selected(None) - case ModelPick.Manual => + case ModelPick.Manual => val hint = clearAffordance(current) freeText(ui, s"$label$hint", current) diff --git a/shell/src/main/scala/orca/shell/wizard/Wizard.scala b/shell/src/main/scala/orca/shell/wizard/Wizard.scala index d5cd43dd6..dca8903e3 100644 --- a/shell/src/main/scala/orca/shell/wizard/Wizard.scala +++ b/shell/src/main/scala/orca/shell/wizard/Wizard.scala @@ -8,7 +8,7 @@ import ox.discard /** The welcome wizard (ADR 0021 §4): detects installed harnesses, asks the user * to pick one per role plus its model, and writes the user-global settings - * file. `probe` is `PathProbe.resolves(_, env.workDir)` in production, + * file. `probe` is `PathProbe.resolves(procs, _, env.workDir)` in production, * injected so tests never touch a real PATH; `globalSettingsPath` is likewise * injected so tests never touch the developer's `~/.config`. */ @@ -69,7 +69,7 @@ private[shell] class Wizard( yield AgentSettings(Some(planning), Some(coding), Some(review)) chosen match - case UiOutcome.Cancelled => false + case UiOutcome.Cancelled => false case UiOutcome.Selected(agents) => ConfigAction.set(globalSettingsPath, agents) true @@ -137,7 +137,7 @@ private[shell] class Wizard( // parse splits a typed `@` off the model AgentSpec.parse(AgentSpec(tag, model).render) match case Right(typed) if typed.effort.isDefined => UiOutcome.Selected(typed) - case Right(typed) => + case Right(typed) => UiOutcome.Selected( current.fold(typed)(_.repinned(tag, typed.model)) ) diff --git a/shell/src/test/scala/orca/shell/WorktreeScanTest.scala b/shell/src/test/scala/orca/shell/WorktreeScanTest.scala index 1be1a7499..27581af36 100644 --- a/shell/src/test/scala/orca/shell/WorktreeScanTest.scala +++ b/shell/src/test/scala/orca/shell/WorktreeScanTest.scala @@ -1,8 +1,7 @@ package orca.shell import orca.{OrcaDir, RunKey} -import orca.testkit.{GitRepo, TempDirs} -import orca.tools.Worktrees +import orca.testkit.{Effects, GitRepo, TempDirs} import ox.discard class WorktreeScanTest extends munit.FunSuite: @@ -16,14 +15,14 @@ class WorktreeScanTest extends munit.FunSuite: test("an orca worktree is scanned alongside the checkout"): val repo = GitRepo.seeded() - val worktree = OrcaDir.ensureWorktrees(repo) / "0123456789ab" - assertEquals(Worktrees.add(repo, worktree), Right(())) + val worktree = Effects.orcaDir.ensureWorktrees(repo) / "0123456789ab" + assertEquals(Effects.worktrees.add(repo, worktree), Right(())) assertEquals(WorktreeScan.dirs(repo).all, List(repo, worktree)) test("a worktree scans only itself, not the checkout that made it"): val repo = GitRepo.seeded() - val worktree = OrcaDir.ensureWorktrees(repo) / "0123456789ab" - assertEquals(Worktrees.add(repo, worktree), Right(())) + val worktree = Effects.orcaDir.ensureWorktrees(repo) / "0123456789ab" + assertEquals(Effects.worktrees.add(repo, worktree), Right(())) // git lists the checkout and the worktree from either one; the scan is // scoped to the `.orca/worktrees/` of the directory it is asked about, and // a worktree's own is empty. @@ -31,10 +30,10 @@ class WorktreeScanTest extends munit.FunSuite: test("a worktree does not scan its siblings"): val repo = GitRepo.seeded() - val ours = OrcaDir.ensureWorktrees(repo) / "0123456789ab" - val sibling = OrcaDir.ensureWorktrees(repo) / "ba9876543210" - assertEquals(Worktrees.add(repo, ours), Right(())) - assertEquals(Worktrees.add(repo, sibling), Right(())) + val ours = Effects.orcaDir.ensureWorktrees(repo) / "0123456789ab" + val sibling = Effects.orcaDir.ensureWorktrees(repo) / "ba9876543210" + assertEquals(Effects.worktrees.add(repo, ours), Right(())) + assertEquals(Effects.worktrees.add(repo, sibling), Right(())) assertEquals(WorktreeScan.dirs(ours).all, List(ours)) // Both are visible from the checkout they hang off. assertEquals(WorktreeScan.dirs(repo).all.toSet, Set(repo, ours, sibling)) @@ -42,7 +41,7 @@ class WorktreeScanTest extends munit.FunSuite: test("a worktree orca did not create is left out"): val repo = GitRepo.seeded() val theirs = repo / "review-their-branch" - assertEquals(Worktrees.add(repo, theirs), Right(())) + assertEquals(Effects.worktrees.add(repo, theirs), Right(())) // Its branch's committed progress log would otherwise become a resume // offer, handing that branch's prompt to an agent. assertEquals(WorktreeScan.dirs(repo).all, List(repo)) @@ -51,8 +50,8 @@ class WorktreeScanTest extends munit.FunSuite: val repo = GitRepo.seeded() val worktrees = List("aaaaaaaaaaaa", "bbbbbbbbbbbb", "cccccccccccc").map: name => - val path = OrcaDir.ensureWorktrees(repo) / name - assertEquals(Worktrees.add(repo, path), Right(())) + val path = Effects.orcaDir.ensureWorktrees(repo) / name + assertEquals(Effects.worktrees.add(repo, path), Right(())) os.write(eventLog(path), "", createFolders = true) path // Newest attempt last-recorded wins; the shell's own checkout stays first. @@ -68,8 +67,8 @@ class WorktreeScanTest extends munit.FunSuite: val repo = GitRepo.seeded() val List(oneRun, twoRuns) = List("aaaaaaaaaaaa", "bbbbbbbbbbbb").map: name => - val path = OrcaDir.ensureWorktrees(repo) / name - assertEquals(Worktrees.add(repo, path), Right(())) + val path = Effects.orcaDir.ensureWorktrees(repo) / name + assertEquals(Effects.worktrees.add(repo, path), Right(())) path val older = OrcaDir.eventLogPath(twoRuns, RunKey.of("older")) List(eventLog(oneRun), eventLog(twoRuns), older).foreach: @@ -82,8 +81,8 @@ class WorktreeScanTest extends munit.FunSuite: test("only the most recently used worktrees are scanned, up to the cap"): val repo = GitRepo.seeded() val worktrees = (0 to WorktreeScan.MaxScannedWorktrees).toList.map: i => - val path = OrcaDir.ensureWorktrees(repo) / f"wt$i%012d" - assertEquals(Worktrees.add(repo, path), Right(())) + val path = Effects.orcaDir.ensureWorktrees(repo) / f"wt$i%012d" + assertEquals(Effects.worktrees.add(repo, path), Right(())) os.write(eventLog(path), "", createFolders = true) // Ascending mtimes, so index 0 is the least recently used. os.mtime.set(eventLog(path), 1000L + i * 1000L).discard diff --git a/shell/src/test/scala/orca/shell/actions/AuthorActionTest.scala b/shell/src/test/scala/orca/shell/actions/AuthorActionTest.scala index c8fa541c7..3b511ca80 100644 --- a/shell/src/test/scala/orca/shell/actions/AuthorActionTest.scala +++ b/shell/src/test/scala/orca/shell/actions/AuthorActionTest.scala @@ -177,8 +177,7 @@ class AuthorActionTest extends munit.FunSuite: val copied = sandbox / ".orca" / "cache" / s"orca-api-${OrcaBuild.current.version}" / "fork-source" / "implement.sc" - promptSourcePathExisted = os.isFile(copied) - ) + promptSourcePathExisted = os.isFile(copied)) val result = AuthorAction.fork( source, diff --git a/shell/src/test/scala/orca/shell/create/FlowAuthoringTest.scala b/shell/src/test/scala/orca/shell/create/FlowAuthoringTest.scala index e28d1b09b..f2e66b259 100644 --- a/shell/src/test/scala/orca/shell/create/FlowAuthoringTest.scala +++ b/shell/src/test/scala/orca/shell/create/FlowAuthoringTest.scala @@ -13,6 +13,13 @@ class FlowAuthoringTest extends munit.FunSuite: private val release = OrcaBuild.Release("0.0.18") + /** The build's Scala version, independent of the resource the shell reads. */ + private val scalaPin: String = + "//> using scala " + sys.props.getOrElse( + "orca.build.scalaVersion", + sys.error("-Dorca.build.scalaVersion unset — see build.sbt") + ) + private def resourceText(name: String): String = val stream = getClass.getResourceAsStream(resourcePrefix + name) assert(stream != null, s"missing resource $name") @@ -67,7 +74,7 @@ class FlowAuthoringTest extends munit.FunSuite: assert(text.contains("\n | mode | effect |"), text) test("initialPrompt states the verbatim version-pinned header"): - assert(prompt.contains("//> using scala 3.9.0")) + assert(prompt.contains(scalaPin)) assert(prompt.contains("""//> using dep "org.virtuslab::orca:0.0.18"""")) assert(prompt.contains("//> using jvm 21")) @@ -649,7 +656,7 @@ class FlowAuthoringTest extends munit.FunSuite: ) result match case Left(message) => assert(message.contains("already exists")) - case Right(path) => + case Right(path) => fail(s"expected a collision refusal, got Right($path)") // --- create/fork filename guard: no path separators (security review) --- @@ -685,7 +692,7 @@ class FlowAuthoringTest extends munit.FunSuite: test("skeletonFlow states the same version pins as initialPrompt"): val skeleton = FlowAuthoring.skeletonFlow(release) - assert(skeleton.contains("//> using scala 3.9.0")) + assert(skeleton.contains(scalaPin)) assert(skeleton.contains("""//> using dep "org.virtuslab::orca:0.0.18"""")) assert(skeleton.contains("//> using jvm 21")) diff --git a/shell/src/test/scala/orca/shell/menu/ResumeDetectorTest.scala b/shell/src/test/scala/orca/shell/menu/ResumeDetectorTest.scala index d925c3af2..fa24d4b7f 100644 --- a/shell/src/test/scala/orca/shell/menu/ResumeDetectorTest.scala +++ b/shell/src/test/scala/orca/shell/menu/ResumeDetectorTest.scala @@ -5,7 +5,7 @@ import orca.tools.PrReadiness import orca.{OrcaDir, RunKey, WorkspaceWrite} import orca.gitref.CommitHash import orca.progress.{BranchMode, FlowSource, ProgressHeader, ProgressStore} -import orca.testkit.{TempDirs, branchName} +import orca.testkit.{Effects, TempDirs, branchName} class ResumeDetectorTest extends munit.FunSuite: @@ -37,7 +37,7 @@ class ResumeDetectorTest extends munit.FunSuite: test("detect finds a fresh log's recorded flow, prompt, and branch"): val workDir = TempDirs.dir() ProgressStore - .default(workDir, RunKey.of("fix the flaky test")) + .default(Effects.fs, workDir, RunKey.of("fix the flaky test")) .writeHeader(header()) assertEquals( ResumeDetector.detect(List(workDir)), @@ -57,7 +57,7 @@ class ResumeDetectorTest extends munit.FunSuite: ): val workDir = TempDirs.dir() ProgressStore - .default(workDir, RunKey.of("fix the flaky test")) + .default(Effects.fs, workDir, RunKey.of("fix the flaky test")) .writeHeader(header(flow = None)) assertEquals(ResumeDetector.detect(List(workDir)), None) @@ -65,7 +65,7 @@ class ResumeDetectorTest extends munit.FunSuite: val workDir = TempDirs.dir() val source = FlowSource.File("/home/u/scratch/implement.sc") ProgressStore - .default(workDir, RunKey.of("fix the flaky test")) + .default(Effects.fs, workDir, RunKey.of("fix the flaky test")) .writeHeader(header(flow = Some(source))) assertEquals(ResumeDetector.detect(List(workDir)).map(_.flow), Some(source)) @@ -73,27 +73,39 @@ class ResumeDetectorTest extends munit.FunSuite: // FlowResolutionTest covers which paths those are. val workDir = TempDirs.dir() ProgressStore - .default(workDir, RunKey.of("fix the flaky test")) + .default(Effects.fs, workDir, RunKey.of("fix the flaky test")) .writeHeader(header(flow = Some(FlowSource.File("scratch/x.sc")))) assertEquals(ResumeDetector.detect(List(workDir)), None) test("detect is None for a corrupt (unparseable) log, silently"): val workDir = TempDirs.dir() - val store = ProgressStore.default(workDir, RunKey.of("fix the flaky test")) + val store = ProgressStore.default( + Effects.fs, + workDir, + RunKey.of("fix the flaky test") + ) store.writeHeader(header()) os.write.over(store.path, "not json {{{") assertEquals(ResumeDetector.detect(List(workDir)), None) test("detect picks the newest of multiple unfinished logs by mtime"): val workDir = TempDirs.dir() - val older = ProgressStore.default(workDir, RunKey.of("older prompt")) + val older = ProgressStore.default( + Effects.fs, + workDir, + RunKey.of("older prompt") + ) older.writeHeader( header( userPrompt = "older prompt", flow = Some(FlowSource.Catalog("a.sc")) ) ) - val newer = ProgressStore.default(workDir, RunKey.of("newer prompt")) + val newer = ProgressStore.default( + Effects.fs, + workDir, + RunKey.of("newer prompt") + ) newer.writeHeader( header( userPrompt = "newer prompt", @@ -119,7 +131,7 @@ class ResumeDetectorTest extends munit.FunSuite: val shellDir = TempDirs.dir() val worktree = TempDirs.dir() ProgressStore - .default(worktree, RunKey.of("fix the flaky test")) + .default(Effects.fs, worktree, RunKey.of("fix the flaky test")) .writeHeader(header()) assertEquals( ResumeDetector.detect(List(shellDir, worktree)), @@ -137,7 +149,11 @@ class ResumeDetectorTest extends munit.FunSuite: test("detect: the newest wins across directories, not within each"): val shellDir = TempDirs.dir() val worktree = TempDirs.dir() - val older = ProgressStore.default(worktree, RunKey.of("older prompt")) + val older = ProgressStore.default( + Effects.fs, + worktree, + RunKey.of("older prompt") + ) older.writeHeader( header( userPrompt = "older prompt", @@ -145,7 +161,7 @@ class ResumeDetectorTest extends munit.FunSuite: ) ) ProgressStore - .default(shellDir, RunKey.of("newer prompt")) + .default(Effects.fs, shellDir, RunKey.of("newer prompt")) .writeHeader( header( userPrompt = "newer prompt", @@ -170,7 +186,7 @@ class ResumeDetectorTest extends munit.FunSuite: test("detect: an unreadable directory costs only its own logs"): val shellDir = TempDirs.dir() ProgressStore - .default(shellDir, RunKey.of("fix the flaky test")) + .default(Effects.fs, shellDir, RunKey.of("fix the flaky test")) .writeHeader(header()) // `.orca` present but not listable — the scan must still offer the log it // can read rather than dropping the whole thing. diff --git a/shell/src/test/scala/orca/shell/sessions/EventLogRoundTripTest.scala b/shell/src/test/scala/orca/shell/sessions/EventLogRoundTripTest.scala index 21a35f921..f05413d66 100644 --- a/shell/src/test/scala/orca/shell/sessions/EventLogRoundTripTest.scala +++ b/shell/src/test/scala/orca/shell/sessions/EventLogRoundTripTest.scala @@ -4,7 +4,7 @@ import orca.{AttemptId, RunKey, StagePath} import orca.agents.{BackendTag, SessionKey} import orca.events.OrcaEvent import orca.runlog.{AttemptOutcome, RunEventLog} -import orca.testkit.{StageEvents, TempDirs} +import orca.testkit.{Effects, StageEvents, TempDirs} import ox.channels.BufferCapacity import ox.supervised @@ -24,6 +24,7 @@ class EventLogRoundTripTest extends munit.FunSuite: supervised: given BufferCapacity = BufferCapacity(8) val log = RunEventLog.start( + Effects.fs, workDir, RunKey.of("a prompt"), AttemptId(Instant.now(), pid = 1), diff --git a/shell/src/test/scala/orca/shell/wizard/WizardTest.scala b/shell/src/test/scala/orca/shell/wizard/WizardTest.scala index 0ccd68cfe..f9362ac56 100644 --- a/shell/src/test/scala/orca/shell/wizard/WizardTest.scala +++ b/shell/src/test/scala/orca/shell/wizard/WizardTest.scala @@ -511,13 +511,14 @@ class WizardTest extends munit.FunSuite: |""".stripMargin ) val ui = ScriptedUi( - selectScript = List( - UiOutcome.Selected(BackendTag.ClaudeCode), - UiOutcome.Selected(ModelCatalog.ModelPick.Default), - UiOutcome.Selected(BackendTag.ClaudeCode), - UiOutcome.Selected(ModelCatalog.ModelPick.Curated("sonnet")), - UiOutcome.Selected(BackendTag.Gemini) - ), + selectScript = + List( + UiOutcome.Selected(BackendTag.ClaudeCode), + UiOutcome.Selected(ModelCatalog.ModelPick.Default), + UiOutcome.Selected(BackendTag.ClaudeCode), + UiOutcome.Selected(ModelCatalog.ModelPick.Curated("sonnet")), + UiOutcome.Selected(BackendTag.Gemini) + ), inputScript = List(UiOutcome.Selected("")) ) assert(Wizard(ui, probe, path).run(reconfigure = true)) diff --git a/tools/src/main/scala/orca/CheckedPar.scala b/tools/src/main/scala/orca/CheckedPar.scala index 8a6dfd655..57a01e1f8 100644 --- a/tools/src/main/scala/orca/CheckedPar.scala +++ b/tools/src/main/scala/orca/CheckedPar.scala @@ -3,28 +3,26 @@ package orca import language.experimental.captureChecking import language.experimental.separationChecking -import ox.flow.Flow - /** The capture-checked concurrency funnel: orca's internal parallel fan-outs go * through here rather than calling Ox's combinators directly, so separation * checking can enforce the [[InStage]]/[[WorkspaceWrite]] capability split at * the fork boundary (ADR 0018 §6). * * Separation checking rejects an exclusive capability the moment it is widened - * away. For a fan-out that is the thunk list's impure `() => T` element type: a - * closure capturing an exclusive `WorkspaceWrite`/`FlowControl` fails with a + * away. For a fan-out that is the thunk list's impure `() => T` element type: + * a closure capturing an exclusive `WorkspaceWrite`/`FlowControl` fails with a * separation failure at the call site, while a shared `InStage` capture (a * `caps.SharedCapability`, separation-exempt) compiles — reviewers reach a * gated LLM `run` from inside their fork. The check is armed by the impure * element type at the call site, not by this wrapper's signature; the funnel's - * job is to keep fan-out call sites in that checked shape, and it is the pinned - * pattern the negative-compile suite compiles fixtures against. + * job is to keep fan-out call sites in that checked shape, and it is the + * pinned pattern the negative-compile suite compiles fixtures against. * * Raw Ox stays the unchecked escape hatch until Ox itself is capture-checked. - * These wrappers are a bridge for deletion: once `ox.fork`/`Flow` carry capture - * annotations, the rejection happens at the Ox call directly. Enforcement is - * per compilation unit — a call site is only checked if its own file carries - * the two language imports above. + * These wrappers are a bridge for deletion: once `ox.fork`/`Flow` carry + * capture annotations, the rejection happens at the Ox call directly. + * Enforcement is per compilation unit — a call site is only checked if its own + * file carries the two language imports above. * * The wrappers delegate 1:1 to Ox with no behavioral additions. */ @@ -34,17 +32,13 @@ private[orca] object CheckedPar: * collecting thread, and returns every result as a list. * * The capture-set parameter `C^` is not interchangeable with plain impure - * thunks (`Seq[() => T]`): that signature does not compile, since applying the - * elements leaks the reach capability `thunks*` into the method's capture - * scope. `C^` names the elements' captures so the body type-checks, while the - * call site keeps the enforcement role (see the object doc). `onResult` is a - * plain impure closure, unconstrained by `C`. + * thunks (`Seq[() => T]`): that signature does not compile, since applying + * the elements leaks the reach capability `thunks*` into the method's + * capture scope. `C^` names the elements' captures so the body type-checks, + * while the call site keeps the enforcement role (see the object doc). + * `onResult` is a plain impure closure, unconstrained by `C`. */ def mapParUnordered[T, C^]( parallelism: Int )(thunks: Seq[() ->{C} T])(onResult: T => Unit): List[T] = - Flow - .fromIterable(thunks) - .mapParUnordered(parallelism)(_.apply()) - .tap(onResult) - .runToList() + concurrency.mapParUnordered(parallelism)(thunks)(_.apply())(onResult) diff --git a/tools/src/main/scala/orca/ConfigHome.scala b/tools/src/main/scala/orca/ConfigHome.scala index 50fbba45a..1e370f09c 100644 --- a/tools/src/main/scala/orca/ConfigHome.scala +++ b/tools/src/main/scala/orca/ConfigHome.scala @@ -1,5 +1,7 @@ package orca +import orca.safe.prelude.given + /** The user-global config home: `$XDG_CONFIG_HOME/orca/`, defaulting to * `~/.config/orca/` (XDG Base Directory spec). `root` is that directory; each * global tier lives directly under it, so one resolved home fixes all three. @@ -17,5 +19,3 @@ private[orca] case class ConfigHome(root: os.Path): private[orca] object ConfigHome: def resolve(env: String => Option[String], home: os.Path): ConfigHome = ConfigHome(XdgDirs.configHome(env, home) / "orca") - - def default: ConfigHome = resolve(sys.env.get, os.home) diff --git a/tools/src/main/scala/orca/InStage.scala b/tools/src/main/scala/orca/InStage.scala index 9a1087215..6a2c0330a 100644 --- a/tools/src/main/scala/orca/InStage.scala +++ b/tools/src/main/scala/orca/InStage.scala @@ -12,7 +12,7 @@ import scala.annotation.implicitNotFound * reviewer fan-out's shared `InStage` capture is load-bearing) and is exempt * from separation rules. The EXCLUSIVE half — [[WorkspaceWrite]] — gates * mutations that must NOT cross a fork boundary. `caps.SharedCapability` is - * non-experimental on 3.9.0, so — unlike [[WorkspaceWrite]] — this file needs + * non-experimental on 3.10.0, so — unlike [[WorkspaceWrite]] — this file needs * no `captureChecking` language import. * * Carries no state; it is a real class only so capture checking has a diff --git a/tools/src/main/scala/orca/OrcaDir.scala b/tools/src/main/scala/orca/OrcaDir.scala index b9c849f02..6d09a0005 100644 --- a/tools/src/main/scala/orca/OrcaDir.scala +++ b/tools/src/main/scala/orca/OrcaDir.scala @@ -1,20 +1,156 @@ package orca +import orca.effect.FileSystem +import orca.safe.prelude.{*, given} import orca.util.AtomicFile -import ox.tap -import java.nio.charset.StandardCharsets - -/** Layout of the `.orca/` directory (ADR 0019): committed project metadata - * lives at the root, ephemeral state lives under `cache/`, which self-ignores - * via its own `.gitignore` and carries a `CACHEDIR.TAG` so backup tools skip - * it. All `.orca` creation routes through this object, so the exclusion - * markers are always in place before anything is written into the cache. Whole - * files are written through an [[OrcaFile]], which only this object creates. +/** Creates and guards the `.orca/` directory (ADR 0019): committed project + * metadata lives at the root, ephemeral state lives under `cache/`, which + * self-ignores via its own `.gitignore` and carries a `CACHEDIR.TAG` so backup + * tools skip it. All `.orca` creation routes through this class, so the + * exclusion markers are always in place before anything is written into the + * cache. Whole files are written through an [[OrcaDir.OrcaFile]], which only + * this class creates. The companion names the paths, without touching the + * filesystem. * * `worktrees/` is the one root entry that is not committed: it self-ignores * like the cache but is not disposable like it — see [[ensureWorktrees]]. */ +private[orca] final class OrcaDir(fs: FileSystem): + import OrcaDir.* + + /** Idempotently ensure `.orca/` exists and return it. */ + def ensureRoot(workDir: os.Path): os.Path = + ensureDir(workDir, rootPath(workDir)) + + /** Idempotently ensure `.orca/cache/` exists, writing its self-ignoring + * `.gitignore` and `CACHEDIR.TAG` before returning so nothing lands in the + * dir before the exclusion is in place. Markers are written only when + * absent, so repeated calls do not churn mtimes. + */ + def ensureCache(workDir: os.Path): os.Path = + ensureDir(workDir, cachePath(workDir)).tap: cache => + writeIfAbsent(cache / ".gitignore", gitignoreContents) + writeIfAbsent(cache / "CACHEDIR.TAG", cachedirTagContents) + + /** Idempotently ensure [[runDirPath]] exists, with the cache markers in + * place, and return it. + */ + def ensureRunDir(workDir: os.Path, key: RunKey): os.Path = + ensureCacheDir(workDir, runDirPath(workDir, key)) + + /** [[ensureRunDir]], then [[eventLogPath]] for appending; refuses a symlinked + * event log as well as a symlinked directory. + */ + def ensureEventLog(workDir: os.Path, key: RunKey): os.Path = + val _ = ensureRunDir(workDir, key) + eventLogPath(workDir, key).tap(abortIfOrcaComponentSymlink(workDir, _)) + + /** Idempotently ensure `.orca/cache/pi-sessions/` exists and return it, for + * the write side (spawning pi at a session dir under it). + */ + def ensurePiSessions(workDir: os.Path): os.Path = + ensureCacheDir(workDir, piSessionsPath(workDir)) + + /** Idempotently ensure `.orca/worktrees/` exists, writing its self-ignoring + * `.gitignore` before returning: without the marker, `git add -A` in this + * checkout stages a worktree under it as an embedded git repository, so the + * marker must be in place before the first worktree is created. + * + * Unlike [[ensureCache]] there is deliberately no `CACHEDIR.TAG` — the tag + * tells backup tools to skip the directory, and a worktree holds unmerged + * work plus the only copy of the progress log that resumes a failed run. + */ + def ensureWorktrees(workDir: os.Path): os.Path = + ensureDir(workDir, worktreesPath(workDir)).tap: worktrees => + writeIfAbsent(worktrees / ".gitignore", gitignoreContents) + + /** Idempotently ensure `.orca/flows/` exists and return it. */ + def ensureFlows(workDir: os.Path): os.Path = + ensureDir(workDir, flowsPath(workDir)) + + /** The project settings file, cleared for writing. */ + def settingsFile(workDir: os.Path): OrcaFile = + committedFile(workDir, settingsPath(workDir)) + + /** The progress log of the run keyed `key`, cleared for writing. */ + def progressFile(workDir: os.Path, key: RunKey): OrcaFile = + committedFile(workDir, progressPath(workDir, key)) + + /** `file`, committed with the rest of `.orca`'s root. */ + private def committedFile(workDir: os.Path, file: os.Path): OrcaFile = + OrcaFile( + ensureDir(workDir, file / up) / file.last, + ensureCache(workDir), + fs + ) + + /** Idempotently create `dir` under `.orca`, refusing to write through a + * symlinked component. + */ + private def ensureDir(workDir: os.Path, dir: os.Path): os.Path = + abortIfOrcaComponentSymlink(workDir, dir) + dir.tap(fs.makeDirs) + + /** [[ensureDir]] for a directory under `.orca/cache`, with the cache's + * exclusion markers in place first. + */ + private def ensureCacheDir(workDir: os.Path, dir: os.Path): os.Path = + val _ = ensureCache(workDir) + ensureDir(workDir, dir) + + /** Read-only counterpart to [[abortIfOrcaComponentSymlink]], for callers that + * only need to verify before reading (e.g. flow discovery) and must not + * create `.orca` as a side effect. A no-op when there is no `.orca` entry — + * nothing to guard, and discovery already tolerates a missing tier + * directory. + */ + def assertNoOrcaSymlinks(workDir: os.Path, dir: os.Path): Unit = + if fs.entryExists(rootPath(workDir)) then + abortIfOrcaComponentSymlink(workDir, dir) + + /** Refuse if `.orca` — or any orca-created directory from it down to `dir` + * (inclusive) — is a symlink, before any directory creation or write through + * it. A committed symlink at any component (git mode 120000) would redirect + * orca's writes to the link's target, outside the working tree. Each segment + * is checked on its own because `isLink` (lstat, no-follow) inspects only + * the final component, so a symlinked ancestor would be invisible to a + * leaf-only check; the no-follow also catches a dangling final link. + * + * Accepted residual (TOCTOU): a purely LOCAL race could swap a plain + * component for a symlink between the check and the write. Out of scope + * under the committed-repo-symlink threat model, so left open deliberately. + */ + private def abortIfOrcaComponentSymlink( + workDir: os.Path, + dir: os.Path + ): Unit = + val r = rootPath(workDir) + // Every path segment from `.orca` down to `dir`, inclusive, walking up from + // `dir` so the check order is root-first. + @scala.annotation.tailrec + def componentsUpTo(p: os.Path, acc: List[os.Path]): List[os.Path] = + if p == r then r :: acc + else componentsUpTo(p / up, p :: acc) + componentsUpTo(dir, Nil).foreach: component => + if fs.isLink(component) then + throw new OrcaFlowException( + s"$component is a symlink — refusing to create or write through it " + + "(a committed symlink at or below .orca could redirect orca's " + + "writes outside the working tree)" + ) + + // On first-ever cache creation two processes can both see the file absent + // before the flow lock exists to serialize them; the loser's `writeNew` + // (CREATE_NEW) throws. Both writers carry identical contents, so losing the + // race is harmless. + private def writeIfAbsent(path: os.Path, contents: String): Unit = + if !fs.exists(path) then + try fs.writeNew(path, contents) + catch + case _: java.nio.file.FileAlreadyExistsException => + () // lost the create race; content is identical + private[orca] object OrcaDir: private val gitignoreContents = "# Automatically created by orca.\n*\n" @@ -63,17 +199,13 @@ private[orca] object OrcaDir: /** Repo-relative form of the settings path, for git probes that take a path * relative to the repository root. */ - val settingsSubPath: os.SubPath = os.sub / Name / "settings.properties" + val settingsSubPath: os.SubPath = sub / Name / "settings.properties" /** `/.orca/settings.properties` (ADR 0019), for the read side. * [[settingsFile]] is the write-side counterpart. */ def settingsPath(workDir: os.Path): os.Path = workDir / settingsSubPath - /** The project settings file, cleared for writing. */ - def settingsFile(workDir: os.Path): OrcaFile = - committedFile(workDir, settingsPath(workDir)) - /** `/.orca/runs`, passively — the committed directory of the per-run * progress logs, for the read side (`ProgressScan`'s listing), which must * not create `.orca` as a side effect. @@ -87,18 +219,10 @@ private[orca] object OrcaDir: def progressPath(workDir: os.Path, key: RunKey): os.Path = runsPath(workDir) / s"${key.value}$ProgressLogSuffix" - /** The progress log of the run keyed `key`, cleared for writing. */ - def progressFile(workDir: os.Path, key: RunKey): OrcaFile = - committedFile(workDir, progressPath(workDir, key)) - /** Whether `file` is named like a progress log under [[runsPath]]. */ def isProgressLog(file: os.Path): Boolean = file.last.endsWith(ProgressLogSuffix) - /** Idempotently ensure `.orca/` exists and return it. */ - def ensureRoot(workDir: os.Path): os.Path = - ensureDir(workDir, rootPath(workDir)) - /** `/.orca/cache`, passively — for callers that only need the path * (a file to delete, an argument to hand a subprocess) and must not create * anything. [[ensureCache]] is the write-side counterpart. @@ -126,12 +250,6 @@ private[orca] object OrcaDir: def runDirPath(workDir: os.Path, key: RunKey): os.Path = cacheRunsPath(workDir) / key.value - /** Idempotently ensure [[runDirPath]] exists, with the cache markers in - * place, and return it. - */ - def ensureRunDir(workDir: os.Path, key: RunKey): os.Path = - ensureCacheDir(workDir, runDirPath(workDir, key)) - /** File name of a run's event log inside [[runDirPath]]. */ val EventLogName: String = "events.jsonl" @@ -141,23 +259,6 @@ private[orca] object OrcaDir: def eventLogPath(workDir: os.Path, key: RunKey): os.Path = runDirPath(workDir, key) / EventLogName - /** [[ensureRunDir]], then [[eventLogPath]] for appending; refuses a symlinked - * event log as well as a symlinked directory. - */ - def ensureEventLog(workDir: os.Path, key: RunKey): os.Path = - val _ = ensureRunDir(workDir, key) - eventLogPath(workDir, key).tap(abortIfOrcaComponentSymlink(workDir, _)) - - /** Idempotently ensure `.orca/cache/` exists, writing its self-ignoring - * `.gitignore` and `CACHEDIR.TAG` before returning so nothing lands in the - * dir before the exclusion is in place. Markers are written only when - * absent, so repeated calls do not churn mtimes. - */ - def ensureCache(workDir: os.Path): os.Path = - ensureDir(workDir, cachePath(workDir)).tap: cache => - writeIfAbsent(cache / ".gitignore", gitignoreContents) - writeIfAbsent(cache / "CACHEDIR.TAG", cachedirTagContents) - /** `/.orca/cache/runs//.trace.log` — the trace log of * attempt `id` of the run keyed `key`; its rolled-over part ends in * [[RolledTraceLogSuffix]]. @@ -195,30 +296,11 @@ private[orca] object OrcaDir: def piSessionsPath(workDir: os.Path): os.Path = cachePath(workDir) / "pi-sessions" - /** Idempotently ensure `.orca/cache/pi-sessions/` exists and return it, for - * the write side (spawning pi at a session dir under it). - */ - def ensurePiSessions(workDir: os.Path): os.Path = - ensureCacheDir(workDir, piSessionsPath(workDir)) - /** `/.orca/worktrees`, passively — a run's worktree is derived from * this path before anything decides whether to create it. */ def worktreesPath(workDir: os.Path): os.Path = rootPath(workDir) / "worktrees" - /** Idempotently ensure `.orca/worktrees/` exists, writing its self-ignoring - * `.gitignore` before returning: without the marker, `git add -A` in this - * checkout stages a worktree under it as an embedded git repository, so the - * marker must be in place before the first worktree is created. - * - * Unlike [[ensureCache]] there is deliberately no `CACHEDIR.TAG` — the tag - * tells backup tools to skip the directory, and a worktree holds unmerged - * work plus the only copy of the progress log that resumes a failed run. - */ - def ensureWorktrees(workDir: os.Path): os.Path = - ensureDir(workDir, worktreesPath(workDir)).tap: worktrees => - writeIfAbsent(worktrees / ".gitignore", gitignoreContents) - /** `/.orca/flows` — project-tier flow scripts (ADR 0021 §5), * committed like the rest of `.orca`'s root. */ @@ -229,95 +311,24 @@ private[orca] object OrcaDir: */ def reviewersPath(workDir: os.Path): os.Path = rootPath(workDir) / "reviewers" - /** Idempotently ensure `.orca/flows/` exists and return it. */ - def ensureFlows(workDir: os.Path): os.Path = - ensureDir(workDir, flowsPath(workDir)) - /** A file under `.orca/` cleared for writing: its directory exists and no * directory from `.orca` down to it is a symlink. Only [[OrcaDir]] creates * one, so a writer holding it cannot have skipped that check. */ final class OrcaFile private[OrcaDir] ( val path: os.Path, - stagingDir: os.Path - ): + stagingDir: os.Path, + fs: FileSystem + ) extends caps.SharedCapability: + // So OrcaDir's factories return `OrcaFile`, not `OrcaFile^{fs}`. + /** Replace the file's content atomically ([[orca.util.AtomicFile]]). The * temp file of a committed file is staged in the self-ignored cache, so a * kill mid-write never leaves a stray file for a stage's `git add -A`. */ - def replace(bytes: Array[Byte]): Unit = - AtomicFile.replace(path, stagingDir, bytes) + def replace(bytes: IArray[Byte]): Unit = + AtomicFile.replace(fs, path, stagingDir, bytes) /** [[replace]] with `text` in UTF-8. */ def replace(text: String): Unit = - replace(text.getBytes(StandardCharsets.UTF_8)) - - /** `file`, committed with the rest of `.orca`'s root. */ - private def committedFile(workDir: os.Path, file: os.Path): OrcaFile = - OrcaFile(ensureDir(workDir, file / os.up) / file.last, ensureCache(workDir)) - - /** Idempotently create `dir` under `.orca`, refusing to write through a - * symlinked component. - */ - private def ensureDir(workDir: os.Path, dir: os.Path): os.Path = - abortIfOrcaComponentSymlink(workDir, dir) - dir.tap(os.makeDir.all(_)) - - /** [[ensureDir]] for a directory under `.orca/cache`, with the cache's - * exclusion markers in place first. - */ - private def ensureCacheDir(workDir: os.Path, dir: os.Path): os.Path = - val _ = ensureCache(workDir) - ensureDir(workDir, dir) - - /** Read-only counterpart to [[abortIfOrcaComponentSymlink]], for callers that - * only need to verify before reading (e.g. flow discovery) and must not - * create `.orca` as a side effect. A no-op when there is no `.orca` entry — - * nothing to guard, and discovery already tolerates a missing tier - * directory. - */ - private[orca] def assertNoOrcaSymlinks(workDir: os.Path, dir: os.Path): Unit = - if os.exists(rootPath(workDir), followLinks = false) then - abortIfOrcaComponentSymlink(workDir, dir) - - /** Refuse if `.orca` — or any orca-created directory from it down to `dir` - * (inclusive) — is a symlink, before any `os.makeDir.all`/write through it. - * A committed symlink at any component (git mode 120000) would redirect - * orca's writes to the link's target, outside the working tree. Each segment - * is checked on its own because `os.isLink` (lstat, no-follow) inspects only - * the final component, so a symlinked ancestor would be invisible to a - * leaf-only check; the no-follow also catches a dangling final link. - * - * Accepted residual (TOCTOU): a purely LOCAL race could swap a plain - * component for a symlink between the check and the write. Out of scope - * under the committed-repo-symlink threat model, so left open deliberately. - */ - private def abortIfOrcaComponentSymlink( - workDir: os.Path, - dir: os.Path - ): Unit = - val r = rootPath(workDir) - // Every path segment from `.orca` down to `dir`, inclusive, walking up from - // `dir` so the check order is root-first. - @scala.annotation.tailrec - def componentsUpTo(p: os.Path, acc: List[os.Path]): List[os.Path] = - if p == r then r :: acc - else componentsUpTo(p / os.up, p :: acc) - componentsUpTo(dir, Nil).foreach: component => - if os.isLink(component) then - throw new OrcaFlowException( - s"$component is a symlink — refusing to create or write through it " + - "(a committed symlink at or below .orca could redirect orca's " + - "writes outside the working tree)" - ) - - // On first-ever cache creation two processes can both see the file absent - // before the flow lock exists to serialize them; the loser's `os.write` - // (CREATE_NEW) throws. Both writers carry identical contents, so losing the - // race is harmless. - private def writeIfAbsent(path: os.Path, contents: String): Unit = - if !os.exists(path) then - try os.write(path, contents) - catch - case _: java.nio.file.FileAlreadyExistsException => - () // lost the create race; content is identical + replace(utf8Bytes(text)) diff --git a/tools/src/main/scala/orca/Par.scala b/tools/src/main/scala/orca/Par.scala index 3db18b513..477e03b63 100644 --- a/tools/src/main/scala/orca/Par.scala +++ b/tools/src/main/scala/orca/Par.scala @@ -1,7 +1,5 @@ package orca -import ox.flow.Flow - /** The sanctioned fan-out for flow scripts: run `f` over `items` concurrently * and collect the results, without importing Ox directly. * @@ -21,4 +19,4 @@ object Par: * interrupting the in-flight siblings. */ def mapUnordered[A, R](parallelism: Int)(items: Seq[A])(f: A => R): List[R] = - Flow.fromIterable(items).mapParUnordered(parallelism)(f).runToList() + concurrency.mapParUnordered(parallelism)(items)(f)(_ => ()) diff --git a/tools/src/main/scala/orca/RunKey.scala b/tools/src/main/scala/orca/RunKey.scala index 7b169856e..8ef7d454a 100644 --- a/tools/src/main/scala/orca/RunKey.scala +++ b/tools/src/main/scala/orca/RunKey.scala @@ -1,5 +1,7 @@ package orca +import orca.safe.prelude.{UTF_8, sha256} + /** The key of a run — one prompt's flow execution across every process that * resumes it: the first 12 hex chars of SHA-256(prompt). Names the run's * progress log, session records and worktree (`OrcaDir`), so re-running the @@ -10,9 +12,10 @@ opaque type RunKey = String object RunKey: /** The key of the run for `userPrompt`. */ def of(userPrompt: String): RunKey = - val md = java.security.MessageDigest.getInstance("SHA-256") - val digest = md.digest(userPrompt.getBytes("UTF-8")) - digest.iterator.take(6).map(b => f"${b & 0xff}%02x").mkString + sha256(userPrompt.getBytes(UTF_8)).iterator + .take(6) + .map(b => f"${b & 0xff}%02x") + .mkString extension (k: RunKey) /** The string form, for file and directory names. */ diff --git a/tools/src/main/scala/orca/StagePath.scala b/tools/src/main/scala/orca/StagePath.scala index 63551eab0..9fdf8ec0e 100644 --- a/tools/src/main/scala/orca/StagePath.scala +++ b/tools/src/main/scala/orca/StagePath.scala @@ -5,12 +5,7 @@ import com.github.plokhotnyuk.jsoniter_scala.core.{ JsonValueCodec, JsonWriter } -import com.github.plokhotnyuk.jsoniter_scala.macros.{ - ConfiguredJsonValueCodec, - JsonCodecMaker -} -import orca.json.JsonData -import sttp.tapir.Schema +import orca.json.{JsonCodec, JsonData} import scala.annotation.tailrec @@ -46,7 +41,7 @@ enum StagePath: acc: List[StagePath.Segment] ): List[StagePath.Segment] = path match - case FlowBody => acc + case FlowBody => acc case Stage(parent, name, occurrence) => loop(parent, StagePath.Segment(name, occurrence) :: acc) loop(this, Nil) @@ -55,7 +50,7 @@ object StagePath: private case class Segment(name: String, occurrence: Int) private val segmentsCodec: JsonValueCodec[List[Segment]] = - JsonCodecMaker.make + JsonCodec.make // The list codec reads `null` as its default, which would be the flow body. private def decode(in: JsonReader): StagePath = @@ -75,8 +70,8 @@ object StagePath: /** A stage's path, refusing the flow body's `[]`. */ given stageJsonData: JsonData[Stage] = JsonData( - Schema.schemaForIterable[Segment, List](using Schema.derived).as[Stage], - new ConfiguredJsonValueCodec[Stage]: + JsonData.listSchema(using JsonData.derivedSchema[Segment]).as[Stage], + new JsonCodec[Stage]: def decodeValue(in: JsonReader, default: Stage): Stage = decode(in) match case s: Stage => s diff --git a/tools/src/main/scala/orca/WorkspaceWrite.scala b/tools/src/main/scala/orca/WorkspaceWrite.scala index 90e1f5cf6..ce1081b9d 100644 --- a/tools/src/main/scala/orca/WorkspaceWrite.scala +++ b/tools/src/main/scala/orca/WorkspaceWrite.scala @@ -2,6 +2,8 @@ package orca import language.experimental.captureChecking +import orca.concurrency.{ThreadIdentity, currentThread} + import scala.annotation.implicitNotFound /** In-stage workspace-mutation token: the EXCLUSIVE half of the capability @@ -14,7 +16,7 @@ import scala.annotation.implicitNotFound * Extends `caps.ExclusiveCapability` (where [[InStage]] is a * `SharedCapability`), so separation checking forbids two concurrent closures * from both capturing it — the type-level encoding of "these mutations are - * flow-thread-only". `ExclusiveCapability` is `@experimental` on 3.9.0, so + * flow-thread-only". `ExclusiveCapability` is `@experimental` on 3.10.0, so * this file carries `import language.experimental.captureChecking`; that * taints only this compilation unit, not consumers (see ADR 0018 §6). * @@ -25,14 +27,14 @@ import scala.annotation.implicitNotFound @implicitNotFound( "git/file/GitHub writes and progress-log writes must be made inside a `stage(...)` body — and, unlike LLM calls, must NOT be captured into a `fork`. Move this write into a stage (not a fork within one). If this is a helper meant to run inside a stage, declare it `(using WorkspaceWrite)` so its caller's token flows through." ) -final class WorkspaceWrite private (owner: Thread) +final class WorkspaceWrite private (owner: ThreadIdentity) extends caps.ExclusiveCapability: /** Throws unless called on the thread that minted this token. Called first by * every gated write, with `what` naming the write. */ private[orca] def check(what: String): Unit = - if Thread.currentThread() ne owner then + if currentThread() != owner then throw new OrcaFlowException( s"$what called off the flow thread (inside a `fork`, `supervised` " + "body, `Par.mapUnordered` or `timeout`) — return the data from " + @@ -45,4 +47,4 @@ object WorkspaceWrite: * code; library code must never call this directly. */ private[orca] def unsafe: WorkspaceWrite = - new WorkspaceWrite(Thread.currentThread()) + new WorkspaceWrite(currentThread()) diff --git a/tools/src/main/scala/orca/XdgDirs.scala b/tools/src/main/scala/orca/XdgDirs.scala index ef13e5ab3..8ff3d546b 100644 --- a/tools/src/main/scala/orca/XdgDirs.scala +++ b/tools/src/main/scala/orca/XdgDirs.scala @@ -1,5 +1,7 @@ package orca +import orca.safe.prelude.{absolutePath, given} + /** The per-user base directories of the XDG Base Directory spec. A relative, * empty, or root-climbing variable is ignored and falls back like an unset * one, as the spec mandates. @@ -18,8 +20,4 @@ private[orca] object XdgDirs: variable: String, fallback: os.Path ): os.Path = - env(variable) - // `os.Path` accepts only absolute paths, so a relative, empty, or - // root-climbing value throws and falls back — no separate pre-filter. - .flatMap(v => scala.util.Try(os.Path(v)).toOption) - .getOrElse(fallback) + env(variable).flatMap(absolutePath).getOrElse(fallback) diff --git a/tools/src/main/scala/orca/agents/Agent.scala b/tools/src/main/scala/orca/agents/Agent.scala index d24716c23..558caab04 100644 --- a/tools/src/main/scala/orca/agents/Agent.scala +++ b/tools/src/main/scala/orca/agents/Agent.scala @@ -5,12 +5,10 @@ import orca.backend.{AgentBackend, AgentResult, Interaction} import orca.events.{OrcaEvent, OrcaListener} import orca.json.JsonData import orca.util.TextUtil -import org.slf4j.LoggerFactory -import ox.tap +import orca.safe.Logger +import orca.safe.prelude.{NonFatal, tap} -import scala.util.control.NonFatal - -private val log = LoggerFactory.getLogger("orca.agents") +private val log = Logger("orca.agents") /** An LLM adapter usable from flow scripts — the handle you call from a * `flow(...)` block (`claude`, `codex`, etc.). Three rungs, by how long the @@ -44,7 +42,7 @@ final class Agent[B <: BackendTag] private ( prompts: Prompts, events: OrcaListener, interaction: Interaction, - naming: AgentName, + private val naming: AgentName, /** Role tag for this agent in the event stream — a second axis on * `OrcaEvent.UnpricedTurn` alongside [[name]]. The review loop sets * `Some("reviewer")` via [[withRole]] so `CostTracker` can subtotal @@ -276,7 +274,7 @@ final class Agent[B <: BackendTag] private ( ): Unit = cause match case Some(e) => log.warn(s"$purpose one-shot failed: $reason", e) - case None => log.warn("{} one-shot failed: {}", purpose, reason) + case None => log.warn(s"$purpose one-shot failed: $reason") events.onEvent( OrcaEvent.Step( s"$purpose agent failed ($reason) — using the default $purpose instead" @@ -342,6 +340,14 @@ final class Agent[B <: BackendTag] private ( private[orca] def sharesBackendWith(other: Agent[?]): Boolean = backend eq other.backend + /** `other` as an `Agent[B]` — its settings on this agent's backend — or + * `None` when it does not run on that backend ([[sharesBackendWith]]). + */ + private[orca] def sibling(other: Agent[?]): Option[Agent[B]] = + Option.when(sharesBackendWith(other))( + copy(config = other.config, naming = other.naming, role = other.role) + ) + /** What the NEXT call on `session` does with the backend's conversation: * continue one it already holds, or open a fresh one that needs re-seeding — * see [[orca.backend.SessionSupport.dispatchFor]]. diff --git a/tools/src/main/scala/orca/agents/AgentCall.scala b/tools/src/main/scala/orca/agents/AgentCall.scala index 9b7e3d305..0d9df3a17 100644 --- a/tools/src/main/scala/orca/agents/AgentCall.scala +++ b/tools/src/main/scala/orca/agents/AgentCall.scala @@ -4,7 +4,8 @@ import orca.AgentTurnFailed import orca.backend.{Interaction, AgentBackend} import orca.events.{OrcaEvent, OrcaListener} import orca.json.JsonData -import ox.resilience.{ResultPolicy, RetryConfig, retry} +import orca.concurrency +import orca.concurrency.retry /** Structured-output gateway — obtained via `agent.resultAs[O]`. Splits the * autonomous-vs-interactive choice into two sibling objects so the call site @@ -180,11 +181,9 @@ final class AgentCall[B <: BackendTag, O] private[orca] ( ) throw e - val retryConfig = RetryConfig( + val retryConfig = concurrency.retryConfig( config.retrySchedule, - ResultPolicy.retryWhen[Throwable, O](e => - !e.isInstanceOf[AgentTurnFailed] - ) + concurrency.retryWhen[Throwable, O](e => !e.isInstanceOf[AgentTurnFailed]) ) try retry(retryConfig)(tryOnce()) diff --git a/tools/src/main/scala/orca/agents/AgentConfig.scala b/tools/src/main/scala/orca/agents/AgentConfig.scala index 65efc0acb..c5ae459ba 100644 --- a/tools/src/main/scala/orca/agents/AgentConfig.scala +++ b/tools/src/main/scala/orca/agents/AgentConfig.scala @@ -1,8 +1,8 @@ package orca.agents +import orca.concurrency import ox.scheduling.Schedule - -import scala.concurrent.duration.DurationInt +import orca.safe.prelude.DurationInt case class AgentConfig( model: Option[Model] = None, @@ -47,14 +47,14 @@ case class AgentConfig( */ def autoApproveAlso(tool: String): AgentConfig = autoApprove match - case AutoApprove.All => this + case AutoApprove.All => this case AutoApprove.Only(tools) => copy(autoApprove = AutoApprove.Only(tools + tool)) object AgentConfig: val defaultRetrySchedule: Schedule = - Schedule.exponentialBackoff(1.second).maxRetries(3) + concurrency.exponentialBackoff(1.second).maxRetries(3) enum AutoApprove: case All diff --git a/tools/src/main/scala/orca/agents/AgentInput.scala b/tools/src/main/scala/orca/agents/AgentInput.scala index a302966a6..aea72181a 100644 --- a/tools/src/main/scala/orca/agents/AgentInput.scala +++ b/tools/src/main/scala/orca/agents/AgentInput.scala @@ -1,7 +1,6 @@ package orca.agents -import com.github.plokhotnyuk.jsoniter_scala.core.writeToString -import orca.json.JsonData +import orca.json.{JsonCodec, JsonData} /** Serializes an arbitrary value into the string embedded in the prompt sent to * the LLM, so callers don't have to pre-stringify their arguments. @@ -17,4 +16,4 @@ object AgentInput: def serialize(a: String): String = a given [A](using jd: JsonData[A]): AgentInput[A] with - def serialize(a: A): String = writeToString(a)(using jd.codec) + def serialize(a: A): String = JsonCodec.writeString(a)(using jd.codec) diff --git a/tools/src/main/scala/orca/agents/Announce.scala b/tools/src/main/scala/orca/agents/Announce.scala index f09eee9e3..ad44cf24a 100644 --- a/tools/src/main/scala/orca/agents/Announce.scala +++ b/tools/src/main/scala/orca/agents/Announce.scala @@ -35,7 +35,7 @@ object Announce: ): Announcement = instance match case _: NoSpecific[?] => Announcement.Unannounced - case specific => + case specific => specific .message(value) .filter(_.nonEmpty) @@ -51,11 +51,11 @@ object Announce: * then "" else "…")` without the empty-branch landing on screen as a blank * line. */ - def from[O](f: O => String): Announce[O] = (value: O) => + def from[O](f: O -> String): Announce[O] = (value: O) => f(value) match case "" => None case msg => Some(msg) /** Construct from a function returning `Option[String]` directly. */ - def fromOption[O](f: O => Option[String]): Announce[O] = + def fromOption[O](f: O -> Option[String]): Announce[O] = (value: O) => f(value) diff --git a/tools/src/main/scala/orca/agents/BackendTag.scala b/tools/src/main/scala/orca/agents/BackendTag.scala index 1a3e32d8a..644557286 100644 --- a/tools/src/main/scala/orca/agents/BackendTag.scala +++ b/tools/src/main/scala/orca/agents/BackendTag.scala @@ -1,9 +1,8 @@ package orca.agents -import com.github.plokhotnyuk.jsoniter_scala.macros.{ - CodecMakerConfig, - ConfiguredJsonValueCodec -} +import com.github.plokhotnyuk.jsoniter_scala.macros.ConfiguredJsonValueCodec +import orca.json.JsonCodec +import orca.safe.prelude.randomUuid /** Compile-time type tag for a concrete LLM backend. Carried as the `B` * parameter on [[SessionId]], [[orca.backend.AgentResult]], @@ -28,8 +27,8 @@ enum BackendTag: object BackendTag: given codec: ConfiguredJsonValueCodec[BackendTag] = - ConfiguredJsonValueCodec.derived[BackendTag](using - CodecMakerConfig.withDiscriminatorFieldName(None) + JsonCodec.derived[BackendTag](using + JsonCodec.config.withDiscriminatorFieldName(None) ) /** Orca's stable client handle for one agent conversation. Library-internal: @@ -77,7 +76,7 @@ private[orca] object SessionId: * backend stores a client→server mapping keyed on this value. */ def fresh[B <: BackendTag]: SessionId[B] = - java.util.UUID.randomUUID.toString + randomUuid() /** The id a backend actually resumes a conversation against on the wire — * distinct from [[SessionId]], orca's stable client handle. For claude (and diff --git a/tools/src/main/scala/orca/agents/Chat.scala b/tools/src/main/scala/orca/agents/Chat.scala index a1b8f2222..3abde499c 100644 --- a/tools/src/main/scala/orca/agents/Chat.scala +++ b/tools/src/main/scala/orca/agents/Chat.scala @@ -50,12 +50,11 @@ final class Chat[B <: BackendTag] private[agents] ( * holds the conversation; otherwise throws [[AgentOnOtherBackend]]. */ def withAgent(f: Agent[?] => Agent[?]): Chat[B] = - val variant = f(agent) - if !agent.sharesBackendWith(variant) then throw new AgentOnOtherBackend // `Agent[?]`, not `Agent[B]`: a lambda can't be typed against the `B` of a - // wildcard `Chat[?]` held in a `def` (such as a role agent's chat). The - // shared backend instance above guarantees the same `B`. - new Chat(variant.asInstanceOf[Agent[B]], id, origin) + // wildcard `Chat[?]` held in a `def` (such as a role agent's chat). + val variant = + agent.sibling(f(agent)).getOrElse(throw new AgentOnOtherBackend) + new Chat(variant, id, origin) /** Fix the output type for structured turns continuing this conversation — * both `autonomous` and `interactive` modes, mirroring `agent.resultAs[O]`. @@ -68,7 +67,7 @@ final class Chat[B <: BackendTag] private[agents] ( * opened or lost. */ private[agents] def requireHeld(): Unit = origin match - case ChatOrigin.Minted => () + case ChatOrigin.Minted => () case ChatOrigin.Adopted => agent.dispatchFor(id) match case Dispatch.Resume(_, _) => () diff --git a/tools/src/main/scala/orca/agents/EnforcementNotice.scala b/tools/src/main/scala/orca/agents/EnforcementNotice.scala index 99716f95a..b4e9d8cce 100644 --- a/tools/src/main/scala/orca/agents/EnforcementNotice.scala +++ b/tools/src/main/scala/orca/agents/EnforcementNotice.scala @@ -1,9 +1,8 @@ package orca.agents -import orca.backend.AgentBackend import orca.events.{OrcaEvent, OrcaListener} - -import org.slf4j.LoggerFactory +import orca.safe.Logger +import orca.concurrency.Ref /** Says, when a turn asks a backend for a restriction it cannot mechanically * apply, that the restriction is not mechanical — the runtime's only consumer @@ -15,64 +14,69 @@ import org.slf4j.LoggerFactory * says in plain words what the agent may still do, and the WARN behind it * carries the cell's rationale and level names for whoever is diagnosing. * - * One instance per backend, held by [[AgentBackend]] and shared with any - * sibling backend derived from it, so a twenty-reviewer fan-out says each - * thing once. + * One instance per backend, held by [[orca.backend.AgentBackend]] and shared + * with any sibling backend derived from it, so a twenty-reviewer fan-out says + * each thing once. * * Said at the first turn that asks, not at run start: a flow picks a turn's * `ToolSet` when it makes the call, so the run header cannot know the tiers. */ private[orca] final class EnforcementNotice: + // Shared across a fan-out's reviewer forks, so neither method-local nor + // `caps.Mutable` (fails separation checking there); no I/O in the update, so + // no actor. /** Sentences already said. Keyed on the sentence rather than the cell, so a * different tier, dispatch or approval list repeats the notice exactly when * it changes the wording. */ - private val said = - java.util.concurrent.ConcurrentHashMap.newKeySet[String]() + private val said: Ref[Set[String]] = Ref(Set.empty) /** Report `config`'s unmet restriction for the turn about to run as - * `dispatch`, unless the same sentence has already been said. + * `dispatch` on backend `tag`, unless the same sentence has already been + * said. `cell` is backend `tag`'s cell for `config` and `dispatch`; + * `freshCell` the one for `config` on a fresh dispatch. * * Reporting, not refusing: a weaker gate is a documented property of the * backend, and a restricted turn's prompt still carries * [[orca.backend.SystemPromptComposer.ReadOnlyTurn]]. */ - def announceShortfall[B <: BackendTag]( - backend: AgentBackend[B], + def announceShortfall( + tag: BackendTag, config: AgentConfig, dispatch: TurnDispatch, + cell: EnforcementCell, + freshCell: EnforcementCell, events: OrcaListener ): Unit = - val cell = - backend.enforcementCell(config.tools, config.autoApprove, dispatch) EnforcementNotice - .summary(backend, config, cell, dispatch) + .summary(tag, config, dispatch, cell, freshCell) .foreach: summary => - if said.add(summary) then + if said.modify(s => (s + summary, !s.contains(summary))) then events.onEvent(OrcaEvent.Caveat(summary)) - EnforcementNotice.log.warn("{} — {}", summary, cell.rationale) + EnforcementNotice.log.warn(s"$summary — ${cell.rationale}") private[orca] object EnforcementNotice: - private val log = LoggerFactory.getLogger(classOf[EnforcementNotice]) + private val log = Logger(classOf[EnforcementNotice].getName) /** The sentence to say, or `None` when the turn asked for nothing this * backend had to encode, or got what it asked for. */ private def summary( - backend: AgentBackend[?], + tag: BackendTag, config: AgentConfig, + dispatch: TurnDispatch, cell: EnforcementCell, - dispatch: TurnDispatch + freshCell: EnforcementCell ): Option[String] = for request <- unmetRequest( config, - turnWording(backend, config, cell, dispatch) + turnWording(config, dispatch, cell, freshCell) ) consequence <- consequenceOf(cell.level) - yield s"${backend.tag} cannot $request — $consequence" + yield s"$tag cannot $request — $consequence" /** What the caller asked this backend to withhold. The two axes a caller can * restrict on: the tier withholds the write tools (or, on `NoTools`, every @@ -89,7 +93,7 @@ private[orca] object EnforcementNotice: Some(s"stop $turn from calling tools") case ToolSet.Full => config.autoApprove match - case AutoApprove.All => None + case AutoApprove.All => None case AutoApprove.Only(_) => Some(s"hold $turn to the tools it was asked to auto-approve") @@ -98,7 +102,7 @@ private[orca] object EnforcementNotice: */ private def consequenceOf(level: Enforcement): Option[String] = level match - case Enforcement.Hard => None + case Enforcement.Hard => None case Enforcement.PromptOnly => Some("only the turn's own prompt asks it not to") case Enforcement.SandboxApprox => @@ -114,16 +118,12 @@ private[orca] object EnforcementNotice: * change. */ private def turnWording( - backend: AgentBackend[?], config: AgentConfig, + dispatch: TurnDispatch, cell: EnforcementCell, - dispatch: TurnDispatch + freshCell: EnforcementCell ): String = - val weakenedByResuming = cell.level.weakerThan( - backend - .enforcementCell(config.tools, config.autoApprove, TurnDispatch.Fresh) - .level - ) + val weakenedByResuming = cell.level.weakerThan(freshCell.level) dispatch match case TurnDispatch.Resumed if weakenedByResuming => s"a resumed ${config.tools} turn" diff --git a/tools/src/main/scala/orca/agents/JsonSchemaGen.scala b/tools/src/main/scala/orca/agents/JsonSchemaGen.scala index 77cfcb26b..2cb73a43a 100644 --- a/tools/src/main/scala/orca/agents/JsonSchemaGen.scala +++ b/tools/src/main/scala/orca/agents/JsonSchemaGen.scala @@ -1,28 +1,13 @@ package orca.agents -import _root_.io.circe.{Json, JsonObject} -import _root_.io.circe.syntax.EncoderOps import orca.OrcaFlowException -import sttp.apispec.circe.encoderSchema +import orca.json.JsonSchemas import sttp.tapir.Schema -import sttp.tapir.docs.apispec.schema.TapirSchemaToJsonSchema -/** JSON-Schema string produced for a given Scala type's `tapir.Schema`, post- - * processed so it satisfies OpenAI's "strict" structured-output dialect (used - * by both `codex exec --output-schema` and `claude --json-schema`): - * - * - every object node carries `additionalProperties: false`; - * - every object's `required` array lists every key in `properties`. - * - * Tapir's default output is more permissive than strict mode, so codex rejects - * it with `invalid_json_schema`. Optional fields and fields with Scala-side - * defaults are marked nullable via `markOptionsAsNullable = true`, so - * requiring them is safe — the agent emits `null` or an empty list rather than - * omitting. - * - * A `Map[String, _]` field has no fixed key set, so it can't be expressed in - * the strict dialect; such a field makes this throw [[orca.OrcaFlowException]] - * rather than emit a schema the backend would reject later and more opaquely. +/** JSON-Schema string for a type's `tapir.Schema`, in OpenAI's strict + * structured-output dialect ([[JsonSchemas.strict]]); a type with no strict + * form throws [[OrcaFlowException]], rather than emit a schema the backend + * would reject later and more opaquely. * * The post-processing applies to every backend, which is why it lives at the * backend-agnostic `resultAs[O]` seam. The one schema string per `O` is both @@ -33,76 +18,5 @@ import sttp.tapir.docs.apispec.schema.TapirSchemaToJsonSchema * dialect any backend requires is safe as the single common form. */ private[orca] object JsonSchemaGen: - def apply[O](using schema: Schema[O]): String = - val jsonSchema = - TapirSchemaToJsonSchema(schema, markOptionsAsNullable = true) - // Tapir stamps `$schema: .../draft/2020-12/schema` on its output; the - // claude CLI (observed on 2.1.207) has no 2020-12 meta-schema registered - // and rejects any schema declaring that dialect ("no schema with key or ref - // ...") before the turn starts. No consumer needs the declaration, so strip - // it. - toOpenAiStrict(jsonSchema.asJson).mapObject(_.remove("$schema")).noSpaces - - /** Walk every object subtree and inject the two OpenAI-strict-mode - * constraints. Exposed for tests; production code uses [[apply]] which - * applies it automatically. - */ - private[agents] def toOpenAiStrict(json: Json): Json = - json.fold( - jsonNull = json, - jsonBoolean = _ => json, - jsonNumber = _ => json, - jsonString = _ => json, - jsonArray = arr => Json.fromValues(arr.map(toOpenAiStrict)), - jsonObject = obj => Json.fromJsonObject(transformObject(obj)) - ) - - private def transformObject(obj: JsonObject): JsonObject = - val recursed = JsonObject.fromIterable( - obj.toIterable.map((k, v) => k -> toOpenAiStrict(v)) - ) - rejectMapShapedAdditionalProperties(recursed) - if isObjectSchemaNode(recursed) then addStrictConstraints(recursed) - else recursed - - /** Tapir renders a `Map[String, T]` field as an object node whose - * `additionalProperties` is `T`'s own sub-schema (no `properties` key — the - * keys are unbounded). Strict mode requires `additionalProperties: false` on - * every object node, which would silently discard the value-type constraint - * if overwritten — so fail fast here, with a message naming the fix, rather - * than letting the schema bounce back as an opaque `invalid_json_schema` - * after a stage has already run. - */ - private def rejectMapShapedAdditionalProperties(obj: JsonObject): Unit = - obj("additionalProperties").filterNot(_.isBoolean).foreach { _ => - throw OrcaFlowException( - "resultAs[O]: output schema has a Map[String, _]-shaped field " + - "(an object with a value-type `additionalProperties` schema), " + - "which OpenAI's strict structured-output mode doesn't support. " + - "Model it as a List of key/value case classes instead." - ) - } - - /** A node is an "object schema" — eligible for the strict-mode constraints — - * when it declares `"type": "object"` AND carries a non-empty `"properties"` - * object. The properties check rules out empty/marker objects like - * `{"type":"object"}`. - */ - private def isObjectSchemaNode(obj: JsonObject): Boolean = - obj("type").flatMap(_.asString).contains("object") && - obj("properties").flatMap(_.asObject).exists(_.nonEmpty) - - private def addStrictConstraints(obj: JsonObject): JsonObject = - val props = - obj("properties").flatMap(_.asObject).getOrElse(JsonObject.empty) - val allKeys = props.keys.toList - // The Map[String, T] shape was already rejected above, so - // `additionalProperties` here is absent or already boolean — don't clobber - // an explicit boolean, only fill in the missing default. - val withAdditional = - if obj.contains("additionalProperties") then obj - else obj.add("additionalProperties", Json.False) - withAdditional.add( - "required", - Json.fromValues(allKeys.map(Json.fromString)) - ) + def apply[O](using Schema[O]): String = + JsonSchemas.strict[O].fold(m => throw OrcaFlowException(m), identity) diff --git a/tools/src/main/scala/orca/agents/Model.scala b/tools/src/main/scala/orca/agents/Model.scala index 1761b8543..38aef0a26 100644 --- a/tools/src/main/scala/orca/agents/Model.scala +++ b/tools/src/main/scala/orca/agents/Model.scala @@ -5,6 +5,7 @@ import com.github.plokhotnyuk.jsoniter_scala.core.{ JsonValueCodec, JsonWriter } +import orca.json.JsonData import sttp.tapir.Schema /** The concrete model id a backend served (or will serve) a call with — @@ -30,4 +31,4 @@ object Model: "Model cannot decode JSON null; wrap in Option at the wire layer" ) - given Schema[Model] = Schema.string[Model] + given Schema[Model] = JsonData.stringSchema[Model] diff --git a/tools/src/main/scala/orca/agents/ResponseParser.scala b/tools/src/main/scala/orca/agents/ResponseParser.scala index 2a7a236a6..af675acf2 100644 --- a/tools/src/main/scala/orca/agents/ResponseParser.scala +++ b/tools/src/main/scala/orca/agents/ResponseParser.scala @@ -2,10 +2,9 @@ package orca.agents import com.github.plokhotnyuk.jsoniter_scala.core.{ JsonReaderException, - JsonValueCodec, - readFromString + JsonValueCodec } -import io.circe.parser.{parse as parseCirceJson} +import orca.json.{JsonCodec, RawJson} import scala.annotation.tailrec import scala.util.matching.Regex @@ -45,12 +44,12 @@ private[agents] object ResponseParser: val parseResults = candidates.map(tryParse[O]) parseResults.collectFirst { case Right(value) => value } match case Some(value) => value - case None => + case None => candidates.flatMap(unwrapInputEnvelope[O]).collectFirst { case Right(value) => value } match case Some(value) => value - case None => + case None => val lastError = parseResults.collect { case Left(e) => e }.last throw new MalformedAgentOutputException( rawOutput = raw, @@ -61,7 +60,7 @@ private[agents] object ResponseParser: private def tryParse[O: JsonValueCodec]( candidate: String ): Either[JsonReaderException, O] = - try Right(readFromString[O](candidate)) + try Right(JsonCodec.readString[O](candidate)) catch case e: JsonReaderException => Left(e) /** Some backends (observed: opencode routing a reviewer call through @@ -77,14 +76,20 @@ private[agents] object ResponseParser: private def unwrapInputEnvelope[O: JsonValueCodec]( candidate: String ): Option[Either[JsonReaderException, O]] = - for - json <- parseCirceJson(candidate).toOption - obj <- json.asObject - if obj.keys.toList == List("input") - value <- obj("input") - yield value.asString match - case Some(inner) => tryParse[O](inner) - case None => tryParse[O](value.noSpaces) + tryParse[InputEnvelope](candidate).toOption.map: envelope => + // jsoniter's raw value keeps the whitespace that follows the colon. + val value = envelope.input.value.trim + if value.startsWith("\"") then + tryParse[String](value).flatMap(tryParse[O]) + else tryParse[O](value) + + /** A top-level object with the single key `input`. */ + private case class InputEnvelope(input: RawJson) + + private given JsonValueCodec[InputEnvelope] = + JsonCodec.make(using JsonCodec.config.withSkipUnexpectedFields(false)) + + private given JsonValueCodec[String] = JsonCodec.make private def stripFences(raw: String): String = raw.trim match diff --git a/tools/src/main/scala/orca/agents/SessionKey.scala b/tools/src/main/scala/orca/agents/SessionKey.scala index 73f1924da..f982c565f 100644 --- a/tools/src/main/scala/orca/agents/SessionKey.scala +++ b/tools/src/main/scala/orca/agents/SessionKey.scala @@ -1,8 +1,8 @@ package orca.agents import com.github.plokhotnyuk.jsoniter_scala.core.JsonValueCodec -import com.github.plokhotnyuk.jsoniter_scala.macros.JsonCodecMaker import orca.StagePath +import orca.json.JsonCodec /** What identifies a durable session: the `name` it was minted under (its role, * e.g. `implementer`) and the stage that minted it. Two stages can therefore @@ -35,4 +35,4 @@ object SessionKey: * form, for the run's event log. */ private[orca] given codec: JsonValueCodec[SessionKey] = - JsonCodecMaker.make[SessionKey] + JsonCodec.make[SessionKey] diff --git a/tools/src/main/scala/orca/backend/AgentBackend.scala b/tools/src/main/scala/orca/backend/AgentBackend.scala index b6d1a34e3..91c0d1542 100644 --- a/tools/src/main/scala/orca/backend/AgentBackend.scala +++ b/tools/src/main/scala/orca/backend/AgentBackend.scala @@ -1,7 +1,5 @@ package orca.backend -import java.util.concurrent.atomic.AtomicBoolean - import orca.events.OrcaListener import orca.agents.{ AutoApprove, @@ -15,7 +13,9 @@ import orca.agents.{ ToolSet, TurnDispatch } -import ox.{Ox, supervised} +import orca.concurrency.Ref +import orca.concurrency.supervised +import ox.Ox /** SPI implemented per backend (Claude, Codex, …), called from the * autonomous-text and structured-output paths ([[orca.agents.Agent]], @@ -209,10 +209,14 @@ private[orca] trait AgentBackend[B <: BackendTag]: dispatch: Dispatch[B], events: OrcaListener ): Unit = + val turnDispatch = dispatch.asTurnDispatch enforcementNotice.announceShortfall( - this, + tag, config, - dispatch.asTurnDispatch, + turnDispatch, + cell = enforcementCell(config.tools, config.autoApprove, turnDispatch), + freshCell = + enforcementCell(config.tools, config.autoApprove, TurnDispatch.Fresh), events ) @@ -239,8 +243,9 @@ private[orca] trait AgentBackend[B <: BackendTag]: // The use-after-close latch lives on the backend, not the Agent instance: // every builder constructs a new agent sharing this same backend — a // per-agent flag would reset to "open" on every derived handle, letting a - // leaked handle bypass the guard. - private val closedFlag: AtomicBoolean = new AtomicBoolean(false) + // leaked handle bypass the guard. Shared across reviewer forks, so not + // `caps.Mutable` (fails separation checking there). + private val closedFlag: Ref[Boolean] = Ref(false) /** Which enforcement notices this backend has already given. */ private val enforcementNotice: EnforcementNotice = new EnforcementNotice @@ -254,7 +259,7 @@ private[orca] trait AgentBackend[B <: BackendTag]: /** Whether [[markClosed]] has run — i.e. the flow that created this backend * (and every agent handle sharing it) has ended. */ - private[orca] final def isClosed: Boolean = closedFlag.get() + private[orca] final def isClosed: Boolean = closedFlag.get /** Refuse a turn against a backend whose flow has ended, so a leaked agent * handle can't emit to a closed flow's dispatcher. [[runAutonomous]] / diff --git a/tools/src/main/scala/orca/backend/ChannelEvent.scala b/tools/src/main/scala/orca/backend/ChannelEvent.scala index 0d0262ac3..bb68ef05c 100644 --- a/tools/src/main/scala/orca/backend/ChannelEvent.scala +++ b/tools/src/main/scala/orca/backend/ChannelEvent.scala @@ -21,7 +21,7 @@ enum ChannelEvent: * user typed; the backend feeds the answer back as a tool result. Only * emitted by turns whose `LiveTurn.canAskUser` is true. */ - case UserQuestion private[orca] (question: String, respond: String => Unit) + case UserQuestion private[orca] (question: String, respond: String -> Unit) /** Channel's answer to a [[ChannelEvent.ApproveTool]] prompt. */ enum ApprovalDecision: diff --git a/tools/src/main/scala/orca/backend/DecodedTurn.scala b/tools/src/main/scala/orca/backend/DecodedTurn.scala index 073c64ae2..ffdaf975b 100644 --- a/tools/src/main/scala/orca/backend/DecodedTurn.scala +++ b/tools/src/main/scala/orca/backend/DecodedTurn.scala @@ -5,20 +5,11 @@ import orca.backend.mcp.AskUserBridge import orca.util.{OrcaDebug, TerminalControl} import orca.{AgentTurnFailed, OrcaInteractiveCancelled} -import ox.{ - Fork, - Ox, - UnsupervisedFork, - discard, - fork, - forkDiscard, - forkUnsupervised, - pipe -} -import ox.channels.{Channel, ChannelClosed} - -import java.util.concurrent.atomic.AtomicBoolean -import scala.util.control.NonFatal +import orca.concurrency +import orca.concurrency.{fork, forkDiscard, forkUnsupervised} +import orca.safe.prelude.{NonFatal, discard, pipe} +import ox.{Fork, Ox, UnsupervisedFork} +import ox.channels.Channel /** A [[LiveTurn]] over a line [[StreamSource]], decoded by a [[LineDecoder]]. * @@ -47,39 +38,36 @@ private[backend] object DecodedTurn: spec: DecodedTurnSpec, decoder: LineDecoder[B, S] )(using Ox): LiveTurn[B] = - val channel = Channel.buffered[TurnEvent](ChannelCapacity) + val channel = concurrency.bufferedChannel[TurnEvent](ChannelCapacity) def neutral(event: NeutralEvent): Unit = channel.sendOrClosed(event).discard spec.openingPrompt.foreach(p => neutral(TurnEvent.UserMessage(p))) - val cancelled = new AtomicBoolean(false) val stderr = fork(drainStderr(source, decoder, neutral)) spec.askUser match case AskUserChannel.Mcp(session) => forkDiscard(drainQuestions(session.bridge, neutral)) case AskUserChannel.Unavailable | AskUserChannel.Native => () val reader = forkUnsupervised( - Reader(source, decoder, channel, cancelled, stderr).run() + Reader(source, decoder, channel, stderr).run() ) - Live(spec, source, channel, cancelled, reader) + Live(spec, source, channel, reader) private type Outcome[B <: BackendTag] = Either[OrcaInteractiveCancelled | AgentTurnFailed, AgentResult[B]] + /** Only `cancel` closes `channel` before the reader's own `finally`, so the + * reader takes a closed channel to mean the turn was cancelled. + */ private final class Live[B <: BackendTag]( spec: DecodedTurnSpec, source: StreamSource, channel: Channel[TurnEvent], - cancelled: AtomicBoolean, reader: UnsupervisedFork[Outcome[B]] ) extends LiveTurn[B]: def outputSchema: Option[String] = spec.outputSchema def canAskUser: Boolean = spec.askUser.isAvailable - val events: Iterator[TurnEvent] = - Iterator - .continually(channel.receiveOrClosed()) - .takeWhile(!_.isInstanceOf[ChannelClosed]) - .collect { case e: TurnEvent => e } + val events: Iterator[TurnEvent] = concurrency.receiveUntilClosed(channel) /** Every failure of a turn that ran surfaces as [[AgentTurnFailed]]: the * wire session may already exist, so a retry against the same id would @@ -93,18 +81,19 @@ private[backend] object DecodedTurn: case Left(c: OrcaInteractiveCancelled) => Left(c) case Left(f: AgentTurnFailed) => throw f - /** Stops the source, frees a reader blocked on a full channel (events - * already buffered are still delivered), and waits for the reader, so its - * unsettled-end handling completes before the turn scope ends. SIGINT and - * the forcible kill are back to back: this same path runs in the routine - * `finally` of every turn, where the process has already exited. Never - * called from the reader, which would then wait for itself. + /** Closes the channel, which marks the turn cancelled and frees a reader + * blocked on a full channel (events already buffered are still delivered), + * before stopping the source, so the reader steps no line read after the + * cancel; then waits for the reader, so its unsettled-end handling + * completes before the turn scope ends. SIGINT and the forcible kill are + * back to back: this same path runs in the routine `finally` of every + * turn, where the process has already exited. Never called from the + * reader, which would then wait for itself. */ def cancel(): Unit = - cancelled.set(true) + channel.doneOrClosed().discard source.interrupt() source.destroyForcibly() - channel.doneOrClosed().discard reader.join().discard /** Where the reader is in the message grammar. */ @@ -123,11 +112,13 @@ private[backend] object DecodedTurn: source: StreamSource, decoder: LineDecoder[B, S], channel: Channel[TurnEvent], - cancelled: AtomicBoolean, stderr: Fork[StderrLog] )(using Ox): private val name = decoder.backendName + // Before this reader's `finally`, only `Live.cancel` closes the channel. + private def cancelled: Boolean = channel.isClosedForSend + /** Returns the outcome and never throws, so `join` always yields one. */ def run(): Outcome[B] = var progress = Progress[B, S](decoder.init, Phase.BetweenMessages()) @@ -136,10 +127,10 @@ private[backend] object DecodedTurn: try for line <- source.lines do OrcaDebug.traceStream(name, "stdout", line) - if !cancelled.get() then + if !cancelled then progress = progress.phase match case Phase.Done(_) => progress - case Phase.BetweenMessages() | Phase.InMessage() => + case _: (Phase.BetweenMessages[?] | Phase.InMessage[?]) => step(progress, line) None catch @@ -148,8 +139,8 @@ private[backend] object DecodedTurn: Some(e) val stderrLog = stderr.join() progress.phase match - case Phase.Done(_) => () - case Phase.BetweenMessages() | Phase.InMessage() => + case Phase.Done(_) => () + case _: (Phase.BetweenMessages[?] | Phase.InMessage[?]) => decoder.onUnsettledEnd() outcome(progress, readError, stderrLog) catch @@ -236,18 +227,18 @@ private[backend] object DecodedTurn: else ctx.mkString(s"$message\n ", "\n ", "") ) progress.phase match - case Phase.Done(Settled.Succeeded(result)) => Right(result) + case Phase.Done(Settled.Succeeded(result)) => Right(result) case Phase.Done(Settled.Failed(message, debit)) => Left(new AgentTurnFailed(withContext(message), debit)) - case Phase.BetweenMessages() | Phase.InMessage() => + case _: (Phase.BetweenMessages[?] | Phase.InMessage[?]) => val debit = decoder.failedTurnDebit(progress.state) // A cancel's kill can make the in-flight read throw rather than EOF, // so `cancelled` is checked first: a Ctrl-C is never a failure. - if cancelled.get() then Left(new OrcaInteractiveCancelled(debit)) + if cancelled then Left(new OrcaInteractiveCancelled(debit)) else readError match case Some(e) => Left(new AgentTurnFailed(describe(e), debit, e)) - case None => + case None => Left( new AgentTurnFailed( withContext(unsettledExit(source.tryExitCode)), @@ -263,7 +254,7 @@ private[backend] object DecodedTurn: case Some(0) => s"$name exited cleanly but never sent ${decoder.terminalMessageNoun}" case Some(code) => s"$name exited with code $code" - case None => + case None => s"$name's output ended without ${decoder.terminalMessageNoun}, " + "while the process was still running" diff --git a/tools/src/main/scala/orca/backend/EnvCookieSweep.scala b/tools/src/main/scala/orca/backend/EnvCookieSweep.scala index 9199bd53d..5380db824 100644 --- a/tools/src/main/scala/orca/backend/EnvCookieSweep.scala +++ b/tools/src/main/scala/orca/backend/EnvCookieSweep.scala @@ -1,47 +1,35 @@ package orca.backend import orca.events.{OrcaEvent, OrcaListener} -import orca.subprocess.EnvCookie - -import org.slf4j.LoggerFactory -import ox.{discard, sleep} - -import java.nio.charset.StandardCharsets.UTF_8 -import java.nio.file.{Files, Path} -import scala.concurrent.duration.* -import scala.jdk.StreamConverters.* -import scala.util.control.NonFatal - -/** Finds processes still carrying an agent process's [[EnvCookie]] once the - * scope that owned the process has torn it down — the backstop for work an - * agent detached from orca's process tree (`nohup … &`, `setsid`, a double - * fork), which parent-link teardown cannot reach at all. +import orca.safe.Logger +import orca.concurrency.sleep +import orca.safe.prelude.* +import orca.subprocess.PipedCliProcess + +import scala.concurrent.duration.FiniteDuration +import scala.util.Properties.envOrNone + +/** Finds processes still carrying an agent process's + * [[orca.subprocess.EnvCookie]] once the scope that owned the process has torn + * it down — the backstop for work an agent detached from orca's process tree + * (`nohup … &`, `setsid`, a double fork), which parent-link teardown cannot + * reach at all. * * '''Report-only by default.''' A survivor is announced (an `OrcaEvent.Step` * plus a WARN log) and left running; `ORCA_SWEEP_KILL=1` makes the same sweep * kill it. * - * '''Linux only.''' The scan reads `/proc//environ`. No equivalent has - * been verified on macOS, so [[sweep]] there finds nothing and the whole - * feature is silently inert — deliberately, since a user on a platform it - * cannot support has nothing to act on. A sweep that appears to do nothing on - * a Mac is working as intended, not broken. + * '''Linux only''' ([[PipedCliProcess.survivors]]): elsewhere the sweep finds + * nothing and the whole feature is silently inert — deliberately, since a user + * on a platform it cannot support has nothing to act on. */ private[backend] object EnvCookieSweep: - private val log = LoggerFactory.getLogger("orca.backend") + private val log = Logger("orca.backend") /** `ORCA_SWEEP_KILL=1` — kill survivors instead of only naming them. */ private val killsSurvivors: Boolean = - sys.env.get("ORCA_SWEEP_KILL").contains("1") - - private val ProcFs: Path = Path.of("/proc") - - /** `environ`'s entry separator. */ - private val Nul: Char = 0 - - /** Whether this machine exposes readable per-process environments. */ - val supported: Boolean = Files.isDirectory(ProcFs) + envOrNone("ORCA_SWEEP_KILL").contains("1") /** Work the agent was already shutting down — an MCP server noticing stdin * EOF, say — can still be in the process table when teardown returns, and @@ -51,50 +39,17 @@ private[backend] object EnvCookieSweep: */ private val SettleDelay: FiniteDuration = 250.millis - /** Sweep for `cookie` and announce what still carries it. Runs as the owning - * scope's finalizer, so it is total: a failure is logged and swallowed - * rather than left to mask the scope's own outcome. `None` — a process - * without a cookie — is a no-op, as is a platform [[sweep]] cannot scan. + /** Announce what still carries `process`'s cookie. Runs as the owning scope's + * finalizer, so it is total: a failure is logged and swallowed rather than + * left to mask the scope's own outcome. */ - def afterScope(cookie: Option[EnvCookie], events: OrcaListener): Unit = - cookie.foreach: c => - try - if sweep(c).nonEmpty then - sleep(SettleDelay) - val settled = sweep(c) - if settled.nonEmpty then announce(settled, events) - catch case NonFatal(e) => log.debug("environment-cookie sweep failed", e) - - /** Live processes whose environment carries `cookie`; empty where the scan is - * unsupported. Orca's own JVM is skipped, so a sweep can never name — or, - * with the kill switch on, kill — the process running it. - */ - def sweep(cookie: EnvCookie): List[ProcessHandle] = - if !supported then Nil - else - val entry = environEntry(cookie) - val self = ProcessHandle.current().pid - ProcessHandle - .allProcesses() - .toScala(List) - .filter(h => h.pid != self && carriesEntry(h.pid, entry)) - - /** Compares whole NUL-delimited entries, so a variable whose name merely ends - * with ours cannot match. A process that exited mid-scan and one owned by - * another user both read as `false` — the file is gone or unreadable, which - * is all this needs to decide. - */ - /** The `NAME=value` pair as it appears in `/proc//environ`. */ - private def environEntry(cookie: EnvCookie): String = - s"${EnvCookie.VarName}=${cookie.value}" - - private def carriesEntry(pid: Long, entry: String): Boolean = + def afterScope(process: PipedCliProcess, events: OrcaListener): Unit = try - new String( - Files.readAllBytes(ProcFs.resolve(pid.toString).resolve("environ")), - UTF_8 - ).split(Nul).contains(entry) - catch case NonFatal(_) => false + if process.survivors().nonEmpty then + sleep(SettleDelay) + val settled = process.survivors() + if settled.nonEmpty then announce(settled, events) + catch case NonFatal(e) => log.debug("environment-cookie sweep failed", e) private def announce( survivors: List[ProcessHandle], diff --git a/tools/src/main/scala/orca/backend/ObservedTurn.scala b/tools/src/main/scala/orca/backend/ObservedTurn.scala index fec285334..835a466dc 100644 --- a/tools/src/main/scala/orca/backend/ObservedTurn.scala +++ b/tools/src/main/scala/orca/backend/ObservedTurn.scala @@ -2,7 +2,7 @@ package orca.backend import orca.agents.BackendTag import orca.events.{OrcaEvent, OrcaListener} -import ox.either.orThrow +import orca.safe.prelude.orThrow /** A [[LiveTurn]] whose display events go to an [[orca.events.OrcaListener]], * on both the autonomous and the interactive path, leaving the consumer only @@ -39,22 +39,24 @@ final class ObservedTurn[B <: BackendTag] private[orca] ( * Throws [[orca.OrcaInteractiveCancelled]] if the turn was cancelled. */ def drain(answer: ChannelEvent => Unit): AgentResult[B] = - val buffer = new MessageBuffer( - closingProse(live), - text => listener.onEvent(OrcaEvent.AssistantMessage(text)) - ) + var buffer = MessageBuffer.empty(closingProse(live)) + def emit(texts: List[String]): Unit = + texts.foreach(text => listener.onEvent(OrcaEvent.AssistantMessage(text))) + def step(next: (MessageBuffer, List[String])): Unit = + buffer = next._1 + emit(next._2) try live.events.foreach: event => - if event.opensMessage then buffer.onActivity() + if event.opensMessage then step(buffer.onActivity) event match case TurnEvent.AssistantToolCall(name, raw) => listener.onEvent(OrcaEvent.ToolUse(name, raw)) case TurnEvent.AssistantTextDelta(delta) => - buffer.append(delta) + buffer = buffer.append(delta) case TurnEvent.AssistantThinkingDelta(_) => () - case TurnEvent.AssistantMessageEnd => buffer.messageEnd() + case TurnEvent.AssistantMessageEnd => step(buffer.messageEnd) case TurnEvent.ToolResult(_, _, _) => () - case TurnEvent.ToolDenied(toolName) => + case TurnEvent.ToolDenied(toolName) => listener.onEvent(OrcaEvent.ToolDenied(toolName, None)) case TurnEvent.Error(message) => listener.onEvent(OrcaEvent.Error(message)) @@ -62,10 +64,10 @@ final class ObservedTurn[B <: BackendTag] private[orca] ( listener.onEvent(OrcaEvent.UserPrompt(text)) case TurnEvent.Approval(request) => answer(request) case TurnEvent.Question(request) => answer(request) - buffer.finishNormally() + emit(buffer.finishNormally) catch case t: Throwable => - buffer.finishAbnormally() + emit(buffer.finishAbnormally) throw t live.awaitResult().orThrow @@ -89,8 +91,10 @@ private[orca] object ObservedTurn: if live.outputSchema.isDefined then ClosingProse.Withhold else ClosingProse.Render - /** Renders each completed assistant message's prose as one `emit`, holding - * back the CLOSING message when it carries the structured payload. + /** The prose of the assistant message in progress, and a completed one held + * back: each transition returns the messages to render as one + * `AssistantMessage` each, holding back the CLOSING message when it carries + * the structured payload. * * A message can only be recognised as closing after the fact, so a completed * message is parked in `withheld` and released by [[onActivity]] — the first @@ -98,51 +102,41 @@ private[orca] object ObservedTurn: * ahead of the tool calls it announces. What survives to end-of-stream is * the closing message, and only that is dropped. A [[ChannelEvent]] opens no * message, so it is never stuck behind a withheld one. - * - * State is confined to the drain's single-threaded event loop — the mutable - * `StringBuilder`/`var` here is a deliberate, reviewed deviation from the - * codebase's Ox-concurrency default, not an actor oversight: this runs on - * one thread only, so no channel/actor would buy anything. */ - private final class MessageBuffer( + private final case class MessageBuffer( closing: ClosingProse, - emit: String => Unit + current: Vector[String], + withheld: Option[String] ): - private val current = new StringBuilder - private var withheld: Option[String] = None - /** Assistant activity opening a message (`TurnEvent.opensMessage`): the - * parked message isn't the closing one after all, so release it. Call + * parked message isn't the closing one after all, so release it. Apply * before rendering the triggering event, so a message's narration lands * ahead of the tool call it announces. */ - def onActivity(): Unit = - withheld.foreach(emit) - withheld = None + def onActivity: (MessageBuffer, List[String]) = + (copy(withheld = None), withheld.toList) - def append(delta: String): Unit = - val _ = current.append(delta) + def append(delta: String): MessageBuffer = copy(current = current :+ delta) - def messageEnd(): Unit = - if current.nonEmpty then - val text = current.toString - current.clear() - closing match - case ClosingProse.Withhold => withheld = Some(text) - case ClosingProse.Render => emit(text) + def messageEnd: (MessageBuffer, List[String]) = + currentText match + case None => (this, Nil) + case Some(text) => + val cleared = copy(current = Vector.empty) + closing match + case ClosingProse.Withhold => + (cleared.copy(withheld = Some(text)), Nil) + case ClosingProse.Render => (cleared, List(text)) /** Normal end of stream: the parked message IS the payload — drop it (the - * caller emits StructuredResult); flush any unfinished current buffer. + * caller emits StructuredResult); flush any unfinished current message. * * Since DecodedTurn closes every completed message, a normal turn ends its * last message with an `AssistantMessageEnd` that already ran - * `messageEnd()`, leaving `current` empty. `flushCurrent()` is a safety - * net for a message the stream left open (abnormal termination - * mid-message). + * [[messageEnd]], leaving `current` empty. The flush is a safety net for a + * message the stream left open (abnormal termination mid-message). */ - def finishNormally(): Unit = - withheld = None - flushCurrent() + def finishNormally: List[String] = currentText.toList /** The drain threw — reading the stream or answering an event: the message * boundary that would have told the payload from narration never arrived, @@ -154,12 +148,11 @@ private[orca] object ObservedTurn: * like any other closing message — an unfinished one still flushes from * `current`. */ - def finishAbnormally(): Unit = - withheld.foreach(emit) - withheld = None - flushCurrent() + def finishAbnormally: List[String] = withheld.toList ++ currentText + + private def currentText: Option[String] = + Some(current.mkString).filter(_.nonEmpty) - private def flushCurrent(): Unit = - if current.nonEmpty then - emit(current.toString) - current.clear() + private object MessageBuffer: + def empty(closing: ClosingProse): MessageBuffer = + MessageBuffer(closing, Vector.empty, None) diff --git a/tools/src/main/scala/orca/backend/SessionSupport.scala b/tools/src/main/scala/orca/backend/SessionSupport.scala index 547f8726b..64866b49b 100644 --- a/tools/src/main/scala/orca/backend/SessionSupport.scala +++ b/tools/src/main/scala/orca/backend/SessionSupport.scala @@ -3,9 +3,9 @@ package orca.backend import orca.AgentTurnFailed import orca.events.TurnDebit import orca.agents.{BackendTag, SessionId, TurnDispatch, WireSessionId, onWire} -import org.slf4j.LoggerFactory - -import scala.util.control.NonFatal +import orca.safe.Logger +import orca.concurrency.Ref +import orca.safe.prelude.NonFatal /** What the next turn against a client id does with the backend's conversation * — the one answer both the prompt side (re-seed or not) and the wire side @@ -74,16 +74,16 @@ private[orca] enum IdScheme: */ private[orca] final class SessionSupport[B <: BackendTag] private ( scheme: IdScheme, - probe: Option[String => Boolean] + probe: Option[String -> Boolean] ): + // Shared across reviewer forks, so not `caps.Mutable` (fails separation + // checking there); not an actor, which would serialise `probe` runs. /** client id → what is known about its wire id. Under * [[IdScheme.ClientClaimed]] the stored wire id is the client id itself. */ - private val entries = - new java.util.concurrent.ConcurrentHashMap[String, SessionSupport.Entry[ - B - ]]() + private val entries: Ref[Map[String, SessionSupport.Entry[B]]] = + Ref(Map.empty) /** What the next turn against `client` does — see [[Dispatch]]. * @@ -100,23 +100,27 @@ private[orca] final class SessionSupport[B <: BackendTag] private ( */ def dispatchFor(client: SessionId[B]): Dispatch[B] = val key = SessionId.value(client) - Option(entries.get(key)) match + entries.get.get(key) match case Some(SessionSupport.Entry.Confirmed(wire, origin)) => Dispatch.Resume(wire, origin) case Some(rehydrated @ SessionSupport.Entry.Rehydrated(wire)) => if probe.forall(holds(_, WireSessionId.value(wire))) then - val _ = - entries.replace(key, rehydrated, confirmedEarlierAttempt(wire)) + val confirmed = confirmedEarlierAttempt(wire) + entries.update: m => + if m.get(key).contains(rehydrated) then m.updated(key, confirmed) + else m Dispatch.Resume(wire, ResumeOrigin.EarlierAttempt) else // Dropped so this attempt's next commit (`putIfAbsent`) records the // conversation the re-seeded turn opens, not the lost one. - val _ = entries.remove(key, rehydrated) + entries.update: m => + if m.get(key).contains(rehydrated) then m - key else m fresh(client) case None => heldClaim(client) match case Some(claim) => - val _ = entries.putIfAbsent(key, confirmedEarlierAttempt(claim)) + val confirmed = confirmedEarlierAttempt(claim) + entries.update(SessionSupport.putIfAbsent(_, key, confirmed)) Dispatch.Resume(claim, ResumeOrigin.EarlierAttempt) case None => fresh(client) @@ -139,10 +143,9 @@ private[orca] final class SessionSupport[B <: BackendTag] private ( */ def rehydrate(client: SessionId[B], wire: WireSessionId[B]): Unit = if isRecordable(wire) then - val _ = entries.putIfAbsent( - SessionId.value(client), - SessionSupport.Entry.Rehydrated(wire) - ) + val key = SessionId.value(client) + val entry = SessionSupport.Entry.Rehydrated(wire) + entries.update(SessionSupport.putIfAbsent(_, key, entry)) /** Record the client→wire mapping after an interactive turn, so a follow-up * call on the same client id resumes the right thread. @@ -160,8 +163,8 @@ private[orca] final class SessionSupport[B <: BackendTag] private ( val safe = SessionId.isSafe(WireSessionId.value(wire)) if !safe then SessionSupport.log.error( - "refusing to record invalid wire id ('{}') for resume; the next call re-seeds", - WireSessionId.value(wire) + s"refusing to record invalid wire id ('${WireSessionId.value(wire)}') " + + "for resume; the next call re-seeds" ) safe @@ -190,10 +193,9 @@ private[orca] final class SessionSupport[B <: BackendTag] private ( val wire = scheme match case IdScheme.ClientClaimed => client.onWire case IdScheme.ServerMinted => server - val _ = entries.putIfAbsent( - SessionId.value(client), - SessionSupport.Entry.Confirmed(wire, ResumeOrigin.ThisAttempt) - ) + val key = SessionId.value(client) + val entry = SessionSupport.Entry.Confirmed(wire, ResumeOrigin.ThisAttempt) + entries.update(SessionSupport.putIfAbsent(_, key, entry)) /** Under [[IdScheme.ClientClaimed]] the client id IS the wire id, so a * conversation the backend already holds under it resumes with nothing @@ -213,7 +215,7 @@ private[orca] final class SessionSupport[B <: BackendTag] private ( private def heldClaim(client: SessionId[B]): Option[WireSessionId[B]] = val id = SessionId.value(client) scheme match - case IdScheme.ServerMinted => None + case IdScheme.ServerMinted => None case IdScheme.ClientClaimed => Option.when(SessionId.isSafe(id) && probe.exists(holds(_, id)))( client.onWire @@ -232,8 +234,7 @@ private[orca] final class SessionSupport[B <: BackendTag] private ( * claim is known only once a [[dispatchFor]] has asked. */ def persistableWireId(client: SessionId[B]): Option[WireSessionId[B]] = - if probe.isDefined then - Option(entries.get(SessionId.value(client))).map(_.wire) + if probe.isDefined then entries.get.get(SessionId.value(client)).map(_.wire) else None /** The one identity this conversation is known by across events — see @@ -252,7 +253,10 @@ private[orca] final class SessionSupport[B <: BackendTag] private ( ) private[orca] object SessionSupport: - private val log = LoggerFactory.getLogger(classOf[SessionSupport[?]]) + private val log = Logger(classOf[SessionSupport[?]].getName) + + private def putIfAbsent[K, V](m: Map[K, V], k: K, v: V): Map[K, V] = + if m.contains(k) then m else m.updated(k, v) /** What [[SessionSupport]] knows about one client's wire id. */ private enum Entry[B <: BackendTag]: @@ -270,7 +274,7 @@ private[orca] object SessionSupport: */ def durable[B <: BackendTag]( scheme: IdScheme, - probe: String => Boolean + probe: String -> Boolean ): SessionSupport[B] = new SessionSupport(scheme, Some(probe)) diff --git a/tools/src/main/scala/orca/backend/StreamSource.scala b/tools/src/main/scala/orca/backend/StreamSource.scala index 0ee20cbfb..913725b1e 100644 --- a/tools/src/main/scala/orca/backend/StreamSource.scala +++ b/tools/src/main/scala/orca/backend/StreamSource.scala @@ -1,8 +1,7 @@ package orca.backend import orca.subprocess.PipedCliProcess - -import ox.discard +import orca.safe.prelude.discard /** The line-oriented source a [[orca.backend.DecodedTurn]] drives: a primary * line stream, an optional secondary diagnostic stream, a way to stop it, and diff --git a/tools/src/main/scala/orca/backend/SubprocessSpawn.scala b/tools/src/main/scala/orca/backend/SubprocessSpawn.scala index c66e08ce6..9edb3ad5c 100644 --- a/tools/src/main/scala/orca/backend/SubprocessSpawn.scala +++ b/tools/src/main/scala/orca/backend/SubprocessSpawn.scala @@ -2,9 +2,9 @@ package orca.backend import orca.OrcaFlowException import orca.events.OrcaListener +import orca.concurrency.releaseAfterScope import orca.subprocess.PipedCliProcess - -import ox.{ResourceScope, releaseAfterScope} +import ox.ResourceScope /** Spawns one turn's agent process and wraps it in a [[LiveTurn]], for the * subprocess backends (claude/codex/gemini/pi). @@ -26,7 +26,7 @@ private[backend] object SubprocessSpawn: spawn: => PipedCliProcess )(build: PipedCliProcess => C)(using ResourceScope): C = val process = spawn - releaseAfterScope(EnvCookieSweep.afterScope(process.envCookie, events)) + releaseAfterScope(EnvCookieSweep.afterScope(process, events)) try build(process) catch case e: Exception => diff --git a/tools/src/main/scala/orca/backend/TurnResources.scala b/tools/src/main/scala/orca/backend/TurnResources.scala index 01513a977..30d51fc96 100644 --- a/tools/src/main/scala/orca/backend/TurnResources.scala +++ b/tools/src/main/scala/orca/backend/TurnResources.scala @@ -1,13 +1,14 @@ package orca.backend -import org.slf4j.LoggerFactory +import orca.safe.Logger +import orca.concurrency.useInScope +import orca.safe.prelude.NonFatal import ox.ResourceScope -import scala.util.control.NonFatal - /** Registers what one turn allocates — temp files, MCP bindings, config edits — * with the turn's scope, released in reverse order once the scope has joined - * its forks. + * its forks. Temp files come from `FileSystem.tempFile`/`tempDir`, which + * register with the scope themselves. * * A failing release is logged and swallowed: by then the turn has its outcome, * and a teardown error must not turn a finished turn into a failed (and @@ -15,28 +16,12 @@ import scala.util.control.NonFatal */ private[backend] object TurnResources: - private val log = LoggerFactory.getLogger("orca.backend") + private val log = Logger("orca.backend") def use[T](acquire: => T)(release: T => Unit)(using ResourceScope): T = - ox.useInScope(acquire): resource => + useInScope(acquire): resource => try release(resource) catch case NonFatal(e) => log.warn("turn resource release failed", e) - def useCloseable[T <: AutoCloseable](acquire: => T)(using ResourceScope): T = + def useCloseable[T <: AutoCloseable^](acquire: => T)(using ResourceScope): T = use(acquire)(_.close()) - - // Temp files and dirs skip `deleteOnExit`: the scope removes them, and each - // registration would stay in the JVM's exit-hook list for the rest of a flow - // that runs hundreds of turns. - - /** A temp file holding `contents`, removed when the turn ends. */ - def tempFile(contents: String, prefix: String, suffix: String)(using - ResourceScope - ): os.Path = - use( - os.temp(contents, prefix = prefix, suffix = suffix, deleteOnExit = false) - )(path => os.remove(path): Unit) - - /** A temp directory, removed with its contents when the turn ends. */ - def tempDir(prefix: String)(using ResourceScope): os.Path = - use(os.temp.dir(prefix = prefix, deleteOnExit = false))(os.remove.all(_)) diff --git a/tools/src/main/scala/orca/backend/mcp/AskUserBridge.scala b/tools/src/main/scala/orca/backend/mcp/AskUserBridge.scala index bbf7fb201..4a59aa06c 100644 --- a/tools/src/main/scala/orca/backend/mcp/AskUserBridge.scala +++ b/tools/src/main/scala/orca/backend/mcp/AskUserBridge.scala @@ -1,9 +1,8 @@ package orca.backend.mcp +import orca.concurrency +import orca.safe.prelude.discard import ox.channels.{BufferCapacity, Channel} -import ox.discard - -import java.util.concurrent.atomic.AtomicBoolean /** Synchronous rendezvous between the MCP `ask_user` tool handler (which needs * a string answer to return to the agent) and the host process (the @@ -23,7 +22,7 @@ import java.util.concurrent.atomic.AtomicBoolean private[backend] class AskUserBridge(using BufferCapacity): private val pending: Channel[(String, Channel[String])] = - Channel.bufferedDefault + concurrency.bufferedDefaultChannel /** Called by the MCP handler. Blocks the calling thread until the host * answers. Each call gets its own one-shot reply channel so concurrent @@ -36,7 +35,7 @@ private[backend] class AskUserBridge(using BufferCapacity): * `send` raise a recoverable `ChannelClosedException` instead. */ def ask(question: String): String = - val reply: Channel[String] = Channel.rendezvous + val reply: Channel[String] = concurrency.rendezvousChannel try pending.send((question, reply)) reply.receive() @@ -46,18 +45,17 @@ private[backend] class AskUserBridge(using BufferCapacity): * the closure that delivers the answer to the originating [[ask]]. Blocks if * no question is queued. * - * The returned `respond` closure is idempotent (only the first call - * delivers) to protect against a renderer that double-fires. It uses - * `sendOrClosed` so that if the originating `ask` has already exited (its - * reply channel `done`'d), the caller sees a no-op rather than an exception - * escaping into the renderer's dispatch loop. + * A second call of the returned `respond` after the first returns is a + * no-op, protecting against a renderer that double-fires: `respond` `done`s + * the reply channel after sending. `sendOrClosed` also makes a send after + * the originating `ask` has exited a no-op rather than an exception escaping + * into the renderer's dispatch loop. */ def nextQuestion(): PendingQuestion = val (question, reply) = pending.receive() - val delivered = new AtomicBoolean(false) - val respond: String => Unit = answer => - if delivered.compareAndSet(false, true) then - reply.sendOrClosed(answer).discard + val respond: String -> Unit = answer => + reply.sendOrClosed(answer).discard + reply.doneOrClosed().discard PendingQuestion(question, respond) /** Single pending invocation of `ask_user`: the question text the agent @@ -66,5 +64,5 @@ private[backend] class AskUserBridge(using BufferCapacity): */ private[mcp] final case class PendingQuestion( question: String, - respond: String => Unit + respond: String -> Unit ) diff --git a/tools/src/main/scala/orca/backend/mcp/AskUserMcpServer.scala b/tools/src/main/scala/orca/backend/mcp/AskUserMcpServer.scala index 0f9516b02..8b581942a 100644 --- a/tools/src/main/scala/orca/backend/mcp/AskUserMcpServer.scala +++ b/tools/src/main/scala/orca/backend/mcp/AskUserMcpServer.scala @@ -1,16 +1,17 @@ package orca.backend.mcp -import chimp.server.* -import io.circe.Codec +import orca.effect.Network +import orca.json.JsonData +import orca.mcp import ox.Ox -import sttp.tapir.Schema +import orca.safe.prelude.DurationInt -import scala.concurrent.duration.{DurationInt, FiniteDuration} +import scala.concurrent.duration.FiniteDuration /** Input shape of the `ask_user` MCP tool: the agent fills in `question` and * gets the typed answer back as the tool result. */ -private[mcp] case class AskUserInput(question: String) derives Codec, Schema +private[mcp] case class AskUserInput(question: String) derives JsonData /** The `ask_user` MCP tool: each invocation enqueues the question on an * [[AskUserBridge]] and blocks until the host supplies an answer. @@ -43,16 +44,12 @@ private[backend] object AskUserMcpServer: /** Mount the `ask_user` tool on a fresh [[McpHost]]. The handler blocks until * the host user types an answer, which is why [[ToolTimeout]] is an hour. */ - def start(bridge: AskUserBridge)(using Ox): McpHost = - val askUserTool = - tool(AskUserMcpServer.ToolSlug) - .description( - "Ask the host user a clarifying question and receive their " + - "typed answer." - ) - .input[AskUserInput] - .handle(in => ToolResult.text(bridge.ask(in.question))) - McpHost.start(List(askUserTool), ToolTimeout) + def start(net: Network, bridge: AskUserBridge)(using Ox): McpHost = + val askUserTool = mcp.tool[AskUserInput]( + AskUserMcpServer.ToolSlug, + "Ask the host user a clarifying question and receive their typed answer." + )(in => mcp.text(bridge.ask(in.question))) + McpHost.start(net, List(askUserTool), ToolTimeout) /** Short system-prompt hint telling the agent it has an `ask_user` tool for * clarifying questions. Worded conservatively — agents over-use tools diff --git a/tools/src/main/scala/orca/backend/mcp/AskUserSession.scala b/tools/src/main/scala/orca/backend/mcp/AskUserSession.scala index 0a4335588..068ee02e8 100644 --- a/tools/src/main/scala/orca/backend/mcp/AskUserSession.scala +++ b/tools/src/main/scala/orca/backend/mcp/AskUserSession.scala @@ -1,5 +1,6 @@ package orca.backend.mcp +import orca.effect.Network import ox.Ox import ox.channels.BufferCapacity @@ -15,6 +16,6 @@ private[backend] case class AskUserSession( private[backend] object AskUserSession: - def allocate()(using Ox, BufferCapacity): AskUserSession = + def allocate(net: Network)(using Ox, BufferCapacity): AskUserSession = val bridge = new AskUserBridge - AskUserSession(bridge, AskUserMcpServer.start(bridge)) + AskUserSession(bridge, AskUserMcpServer.start(net, bridge)) diff --git a/tools/src/main/scala/orca/backend/mcp/GitHubMcpServer.scala b/tools/src/main/scala/orca/backend/mcp/GitHubMcpServer.scala index 20d47a9bc..46bc2971f 100644 --- a/tools/src/main/scala/orca/backend/mcp/GitHubMcpServer.scala +++ b/tools/src/main/scala/orca/backend/mcp/GitHubMcpServer.scala @@ -1,19 +1,19 @@ package orca.backend.mcp -import chimp.server.* -import io.circe.Codec +import orca.effect.Network +import orca.json.JsonData +import orca.mcp import orca.tools.{Comment, GitHubTool, Issue, IssueHandle} import ox.Ox -import sttp.tapir.Schema +import orca.safe.prelude.DurationInt -import scala.concurrent.duration.{DurationInt, FiniteDuration} +import scala.concurrent.duration.FiniteDuration private[mcp] case class GitHubIssueInput( owner: String, repo: String, number: Int -) derives Codec, - Schema +) derives JsonData /** The GitHub read a planner needs: an issue or PR's body and conversation. * Served host-side through [[GitHubTool]], so the agent never gets `gh` and @@ -36,16 +36,14 @@ private[backend] object GitHubMcpServer: */ private[orca] val ToolTimeout: FiniteDuration = 2.minutes - def start(github: GitHubTool)(using Ox): McpHost = - val issueTool = tool(IssueSlug) - .description( - "Read a GitHub issue or pull request: its title, state, author, body " + - "and conversation comments. Issues and pull requests share one " + - "number sequence, so either kind of number works." - ) - .input[GitHubIssueInput] - .handle(in => ToolResult.fromEither(read(github, in))) - McpHost.start(List(issueTool), ToolTimeout) + def start(net: Network, github: GitHubTool)(using Ox): McpHost = + val issueTool = mcp.tool[GitHubIssueInput]( + IssueSlug, + "Read a GitHub issue or pull request: its title, state, author, body " + + "and conversation comments. Issues and pull requests share one " + + "number sequence, so either kind of number works." + )(in => mcp.fromEither(read(github, in))) + McpHost.start(net, List(issueTool), ToolTimeout) /** `IssueHandle.parse` does the validation: `owner` and `repo` are spliced * into a `gh api` request path, and its patterns are the ones that already diff --git a/tools/src/main/scala/orca/backend/mcp/McpHost.scala b/tools/src/main/scala/orca/backend/mcp/McpHost.scala index 011cc1052..e45319354 100644 --- a/tools/src/main/scala/orca/backend/mcp/McpHost.scala +++ b/tools/src/main/scala/orca/backend/mcp/McpHost.scala @@ -1,20 +1,11 @@ package orca.backend.mcp -import chimp.protocol.ToolContent -import chimp.server.{ - McpServer, - NoStructuredOutput, - ServerContext, - ServerTool, - ToolResult -} -import orca.backend.TurnResources +import orca.effect.Network +import orca.mcp import ox.Ox -import sttp.shared.Identity -import sttp.tapir.server.netty.sync.NettySyncServer +import orca.safe.prelude.NonFatal -import scala.concurrent.duration.{DurationInt, FiniteDuration} -import scala.util.control.NonFatal +import scala.concurrent.duration.FiniteDuration /** One MCP server orca stands up for a single agent turn: a Netty binding on * `127.0.0.1` at an ephemeral port, serving whatever tools it was started @@ -28,7 +19,7 @@ private[backend] class McpHost private[mcp] (val port: Int): /** The URL an MCP client (claude's `.mcp.json`, codex's * `mcp_servers..url`) should target. */ - val url: String = s"${McpHost.UrlPrefix}$port/mcp" + val url: String = s"${McpHost.UrlPrefix}$port/${mcp.endpointPath}" private[backend] object McpHost: @@ -54,12 +45,9 @@ private[backend] object McpHost: else text.take(MaxOutputChars) + CutMarker private def bounded( - result: ToolResult[NoStructuredOutput] - ): ToolResult[NoStructuredOutput] = - result.copy(content = result.content.map: - case t: ToolContent.Text => t.copy(text = bounded(t.text)) - case other => other - ) + result: mcp.Result + ): mcp.Result = + mcp.mapText(result)(bounded) /** The two guarantees every result served here carries: its text stays within * [[MaxOutputChars]] whether the result is an error or not, and a handler @@ -67,53 +55,35 @@ private[backend] object McpHost: * cannot read. */ private def guardedResult( - result: => ToolResult[NoStructuredOutput] - ): ToolResult[NoStructuredOutput] = + result: => mcp.Result + ): mcp.Result = try bounded(result) catch case NonFatal(e) => - bounded(ToolResult.error(Option(e.getMessage).getOrElse(e.toString))) + bounded(mcp.error(Option(e.getMessage).getOrElse(e.toString))) /** Put a tool's logic behind [[guardedResult]]. [[start]] applies this to * every tool it binds, so the guard is not something a tool opts into. * - * The [[NoStructuredOutput]] bound keeps a second, unbounded channel out of - * reach: `withStructured` moves the type parameter, and `ServerTool` is - * invariant in it, so a tool built that way no longer fits [[start]]'s list. + * The `NoStructuredOutput` bound in [[mcp.Tool]] keeps a second, unbounded + * channel out of reach: `withStructured` moves the type parameter, and + * `ServerTool` is invariant in it, so a tool built that way no longer fits + * [[start]]'s list. */ - private[mcp] def guarded[I]( - t: ServerTool[I, NoStructuredOutput, Identity, ServerContext[Identity]] - ): ServerTool[I, NoStructuredOutput, Identity, ServerContext[Identity]] = + private[mcp] def guarded[I](t: mcp.Tool[I]): mcp.Tool[I] = t.copy(logic = (in, ctx, headers) => guardedResult(t.logic(in, ctx, headers)) ) /** Bind `tools` on a fresh port in the enclosing scope, each [[guarded]]. * - * `toolTimeout` becomes Netty's request timeout, raising it from the - * framework default so a slow tool doesn't have its connection closed - * mid-call; `idleTimeout` adds a minute of slop because Netty requires it to - * exceed the request timeout. - * - * The binding stops when the scope ends: tapir's `start()` leaves teardown - * to the caller, and a leaked binding strands its event-loop threads for the - * life of the JVM. + * `toolTimeout` becomes the request timeout, raising it from the framework + * default so a slow tool doesn't have its connection closed mid-call. */ private[mcp] def start( - tools: List[ - ServerTool[?, NoStructuredOutput, Identity, ServerContext[Identity]] - ], + net: Network, + tools: List[mcp.Tool[?]], toolTimeout: FiniteDuration )(using Ox): McpHost = - val binding = TurnResources.use( - NettySyncServer() - .port(0) - .modifyConfig( - _.requestTimeout(toolTimeout).idleTimeout(toolTimeout + 1.minute) - ) - .addEndpoint( - McpServer(tools = tools.map(t => guarded(t))).endpoint(List("mcp")) - ) - .start() - )(_.stop()) - new McpHost(binding.port) + val endpoint = mcp.endpoint(tools.map(t => guarded(t))) + new McpHost(net.serveLocal(List(endpoint), toolTimeout)) diff --git a/tools/src/main/scala/orca/backend/mcp/RepoMcpServer.scala b/tools/src/main/scala/orca/backend/mcp/RepoMcpServer.scala index 3dac8c71c..6eca4769e 100644 --- a/tools/src/main/scala/orca/backend/mcp/RepoMcpServer.scala +++ b/tools/src/main/scala/orca/backend/mcp/RepoMcpServer.scala @@ -1,28 +1,25 @@ package orca.backend.mcp -import chimp.server.* -import io.circe.Codec +import orca.effect.Network +import orca.json.JsonData +import orca.mcp import orca.tools.{GitReadFailed, GitTool, ShowDetail} import ox.Ox -import sttp.tapir.Schema +import orca.safe.prelude.DurationInt -import scala.concurrent.duration.{DurationInt, FiniteDuration} +import scala.concurrent.duration.FiniteDuration -/** `paths` and `stat` are `Option`, not defaulted fields: the derived circe - * codec does not honour a Scala default, so a missing key on a plain - * `List`/`Boolean` is a decode failure. The tool advertises both as optional, - * so the agent omits them. +/** `paths` and `stat` are `Option`, not defaulted fields, so the tool + * advertises both as optional and a call that omits them decodes. */ private[mcp] case class GitShowInput( rev: String, paths: Option[List[String]], stat: Option[Boolean] -) derives Codec, - Schema +) derives JsonData private[mcp] case class GitFileAtInput(rev: String, path: String) - derives Codec, - Schema + derives JsonData /** MCP tools giving read-only turns the git reads they lose with the shell. Two * tools, both structured — the agent names a revision and paths, never a @@ -56,33 +53,29 @@ private[backend] object RepoMcpServer: private[orca] val ToolSlugs: Seq[String] = Seq(ShowSlug, FileAtSlug) /** Mount both tools on a fresh [[McpHost]], reading through `git`. */ - def start(git: GitTool)(using Ox): McpHost = - val showTool = tool(ShowSlug) - .description( - "Show a commit: its message and diff, or with stat=true just which " + - "files it changed. Optionally narrow the diff to specific " + - "repository-relative paths." - ) - .input[GitShowInput] - .handle: in => - val detail = - if in.stat.getOrElse(false) then ShowDetail.StatOnly - else ShowDetail.Full - render(git.show(in.rev, in.paths.getOrElse(Nil), detail)) - val fileAtTool = tool(FileAtSlug) - .description( - "Read one file's full contents as of a commit, e.g. to see what it " + - "looked like before the change under review." - ) - .input[GitFileAtInput] - .handle(in => render(git.fileAt(in.rev, in.path))) - McpHost.start(List(showTool, fileAtTool), ToolTimeout) + def start(net: Network, git: GitTool)(using Ox): McpHost = + val showTool = mcp.tool[GitShowInput]( + ShowSlug, + "Show a commit: its message and diff, or with stat=true just which " + + "files it changed. Optionally narrow the diff to specific " + + "repository-relative paths." + ): in => + val detail = + if in.stat.getOrElse(false) then ShowDetail.StatOnly + else ShowDetail.Full + render(git.show(in.rev, in.paths.getOrElse(Nil), detail)) + val fileAtTool = mcp.tool[GitFileAtInput]( + FileAtSlug, + "Read one file's full contents as of a commit, e.g. to see what it " + + "looked like before the change under review." + )(in => render(git.fileAt(in.rev, in.path))) + McpHost.start(net, List(showTool, fileAtTool), ToolTimeout) /** Map a read outcome onto MCP's success/error channels. */ private def render( result: Either[GitReadFailed, String] - ): ToolResult[NoStructuredOutput] = - ToolResult.fromEither(result.left.map(_.getMessage)) + ): mcp.Result = + mcp.fromEither(result.left.map(_.getMessage)) /** System-prompt hint naming the tools. Read-only turns have no shell, so * without this the agent has no reason to look for them. diff --git a/tools/src/main/scala/orca/events/Usage.scala b/tools/src/main/scala/orca/events/Usage.scala index f4bb6c01d..3ab671354 100644 --- a/tools/src/main/scala/orca/events/Usage.scala +++ b/tools/src/main/scala/orca/events/Usage.scala @@ -1,6 +1,6 @@ package orca.events -import org.slf4j.LoggerFactory +import orca.safe.Logger /** Token + cost accounting for one or more LLM calls. * @@ -64,7 +64,7 @@ case class Usage( ) object Usage: - private val log = LoggerFactory.getLogger("orca.events") + private val log = Logger("orca.events") val empty: Usage = Usage( freshInputTokens = 0L, diff --git a/tools/src/main/scala/orca/gitref/BranchName.scala b/tools/src/main/scala/orca/gitref/BranchName.scala index 40fef0c2e..83d63ad44 100644 --- a/tools/src/main/scala/orca/gitref/BranchName.scala +++ b/tools/src/main/scala/orca/gitref/BranchName.scala @@ -52,7 +52,7 @@ object BranchName: raw.split("/", -1).toList // Ordered: the first broken rule names the refusal. - private val refFormatRules: List[(String => Boolean, String)] = List( + private val refFormatRules: List[(String -> Boolean, String)] = List( (_.isEmpty, "is empty"), (_ == "HEAD", "is the reserved name HEAD"), (_ == "@", "is the reserved name @"), diff --git a/tools/src/main/scala/orca/json/JsonFile.scala b/tools/src/main/scala/orca/json/JsonFile.scala index 17a27beb2..4525aa110 100644 --- a/tools/src/main/scala/orca/json/JsonFile.scala +++ b/tools/src/main/scala/orca/json/JsonFile.scala @@ -1,44 +1,20 @@ package orca.json -import com.github.plokhotnyuk.jsoniter_scala.core.{ - JsonValueCodec, - readFromArray, - writeToArray -} - +import com.github.plokhotnyuk.jsoniter_scala.core.JsonValueCodec import orca.OrcaDir +import orca.effect.FileSystem +import orca.safe.prelude.NonFatal -import scala.util.control.NonFatal - -/** Whole-file JSON documents that a later process reads back, such as the - * progress log. Each is rewritten whole on every change, so [[write]] replaces - * it atomically — a plain `os.write.over` torn by a kill would leave the - * reader unparseable content where a resume was expected. +/** Reads whole-file JSON documents that an earlier process wrote, such as the + * progress log ([[JsonFile.write]]). */ -object JsonFile: - - /** Outcome of [[read]]. The two failure arms call for opposite actions: - * content orca wrote and cannot parse is safe to replace, while a file it - * could not read at all may still hold data, and overwriting it would - * destroy that. - */ - enum Read[+A]: - /** No file at the path. */ - case Absent - - /** The read itself failed — permissions, or a directory at the path. */ - case Unreadable(reason: String) - - /** The file was read but does not parse: truncated, or externally edited. - */ - case Corrupt(reason: String) - - case Loaded(value: A) +final class JsonFile(fs: FileSystem): + import JsonFile.* def read[A](path: os.Path)(using JsonValueCodec[A]): Read[A] = readBytes(path) match - case Left(reason) => Read.Unreadable(reason) - case Right(None) => Read.Absent + case Left(reason) => Read.Unreadable(reason) + case Right(None) => Read.Absent case Right(Some(bytes)) => decode[A](bytes).fold(Read.Corrupt(_), Read.Loaded(_)) @@ -49,19 +25,44 @@ object JsonFile: path: os.Path ): Either[String, Option[IArray[Byte]]] = // Absence is the read's own `NoSuchFileException` rather than an - // `os.exists` pre-check, so there is no window between the two for the - // file to vanish in — the progress log is removed by teardown while the - // runtime still reads it. - try Right(Some(IArray.unsafeFromArray(os.read.bytes(path)))) + // `exists` pre-check, so there is no window between the two for the file + // to vanish in — the progress log is removed by teardown while the runtime + // still reads it. + try Right(Some(fs.readBytes(path))) catch case _: java.nio.file.NoSuchFileException => Right(None) case NonFatal(e) => Left(describe(e)) +/** Whole-file JSON documents that a later process reads back. Each is rewritten + * whole on every change, so [[write]] replaces it atomically — a plain + * overwrite torn by a kill would leave the reader unparseable content where a + * resume was expected. + */ +object JsonFile: + + /** Outcome of [[JsonFile!.read]]. The two failure arms call for opposite + * actions: content orca wrote and cannot parse is safe to replace, while a + * file it could not read at all may still hold data, and overwriting it + * would destroy that. + */ + enum Read[+A]: + /** No file at the path. */ + case Absent + + /** The read itself failed — permissions, or a directory at the path. */ + case Unreadable(reason: String) + + /** The file was read but does not parse: truncated, or externally edited. + */ + case Corrupt(reason: String) + + case Loaded(value: A) + /** `bytes` as an `A`, or why they do not parse. */ private[orca] def decode[A](bytes: IArray[Byte])(using JsonValueCodec[A] ): Either[String, A] = - try Right(readFromArray[A](IArray.genericWrapArray(bytes).toArray)) + try Right(JsonCodec.readBytes[A](bytes)) catch case NonFatal(e) => Left(describe(e)) /** Replace `file`'s contents with `value`'s JSON, atomically (see @@ -70,7 +71,7 @@ object JsonFile: private[orca] def write[A](file: OrcaDir.OrcaFile, value: A)(using JsonValueCodec[A] ): Unit = - file.replace(writeToArray(value)) + file.replace(JsonCodec.writeBytes(value)) /** A one-line account of `e` for a warning: its class and the first line of * its message. jsoniter appends a multi-line hex dump of the buffer to its diff --git a/tools/src/main/scala/orca/json/RawJson.scala b/tools/src/main/scala/orca/json/RawJson.scala index 877a221f0..6c31eafb2 100644 --- a/tools/src/main/scala/orca/json/RawJson.scala +++ b/tools/src/main/scala/orca/json/RawJson.scala @@ -5,6 +5,7 @@ import com.github.plokhotnyuk.jsoniter_scala.core.{ JsonValueCodec, JsonWriter } +import orca.safe.prelude.UTF_8 /** Opaque wrapper for a JSON subtree held in its serialized form. Used for * message fields whose shape varies too much to model (tool inputs, MCP @@ -29,13 +30,10 @@ object RawJson: given JsonValueCodec[RawJson] with def decodeValue(in: JsonReader, default: RawJson): RawJson = // The next JSON value verbatim, re-interpreted as UTF-8. - new String( - in.readRawValAsBytes(), - java.nio.charset.StandardCharsets.UTF_8 - ) + new String(in.readRawValAsBytes(), UTF_8) def encodeValue(x: RawJson, out: JsonWriter): Unit = - out.writeRawVal(x.value.getBytes(java.nio.charset.StandardCharsets.UTF_8)) + out.writeRawVal(x.value.getBytes(UTF_8)) def nullValue: RawJson = "null" @@ -44,4 +42,4 @@ object RawJson: * `--json-schema`; `string` is the closest primitive for an opaque * serialized subtree. */ - given sttp.tapir.Schema[RawJson] = sttp.tapir.Schema.string + given sttp.tapir.Schema[RawJson] = JsonData.stringSchema diff --git a/tools/src/main/scala/orca/subprocess/OsProcCliRunner.scala b/tools/src/main/scala/orca/subprocess/OsProcCliRunner.scala index 6e22ddd1a..9aca885bd 100644 --- a/tools/src/main/scala/orca/subprocess/OsProcCliRunner.scala +++ b/tools/src/main/scala/orca/subprocess/OsProcCliRunner.scala @@ -1,23 +1,12 @@ package orca.subprocess -import org.slf4j.LoggerFactory -import ox.{abandonOnInterruptReads, discard, raceResult, sleep} +import orca.effect.Processes -import java.io.{BufferedReader, InputStream, InputStreamReader} -import java.nio.charset.StandardCharsets.UTF_8 -import java.util.concurrent.atomic.AtomicReference -import scala.concurrent.duration.* -import scala.jdk.CollectionConverters.given - -/** Runs external commands via os-lib. `check = false` is intentional — callers - * inspect `exitCode` and `stderr` rather than handle exceptions, since - * non-zero exits from tools like `claude -p` carry actionable information. +/** Runs external commands through `procs`. Non-zero exits are returned, not + * thrown: callers inspect `exitCode` and `stderr`, since non-zero exits from + * tools like `claude -p` carry actionable information. */ -object OsProcCliRunner extends CliRunner: - // Traces command + flags + exit code to the per-run log. Prompts travel over - // stdin, not argv, so this doesn't leak prompt bodies (logged separately via - // the `UserPrompt` event). - private val log = LoggerFactory.getLogger("orca.proc") +final class OsProcCliRunner(procs: Processes) extends CliRunner: def run( args: Seq[String], @@ -25,162 +14,11 @@ object OsProcCliRunner extends CliRunner: env: Map[String, String], cwd: os.Path ): CliResult = - // Delegates to QuietProc, which guarantees captured stderr so subprocess - // output never bypasses the renderer's StatusBar (see [[QuietProc]]). - log.debug("exec: {} (cwd={})", args.mkString(" "), cwd) - val result = QuietProc.call(args, cwd = cwd, env = env, stdin = stdin) - log.debug("exit {}: {}", result.exitCode, args.headOption.getOrElse("?")) - CliResult(result.exitCode, result.out.text(), result.err.text()) + procs.call(args, cwd = cwd, env = env, stdin = stdin) def spawnPiped( args: Seq[String], env: Map[String, String], cwd: os.Path ): PipedCliProcess = - log.debug("spawn: {} (cwd={})", args.mkString(" "), cwd) - // os-lib applies `env` on top of the inherited environment (replacing it - // only with `propagateEnv = false`), so this adds one variable and changes - // nothing else the CLI sees. - val cookie = EnvCookie.mint() - val sub = os - .proc(args) - .spawn( - cwd = cwd, - env = env + (EnvCookie.VarName -> cookie.value), - stdin = os.Pipe, - stdout = os.Pipe, - stderr = os.Pipe - ) - new OsPipedSubProcess(sub, cookie) - -/** Takes the cookie unwrapped: a process orca actually spawned always carries - * one, so it cannot be constructed without it. - */ -private final class OsPipedSubProcess( - sub: os.SubProcess, - cookie: EnvCookie -) extends PipedCliProcess: - - def envCookie: Option[EnvCookie] = Some(cookie) - - // Memoised so repeated calls return the same iterator, avoiding a second - // `BufferedReader` leak against the pipe. - private lazy val stdoutIterator: Iterator[String] = - linesOf(sub.wrapped.getInputStream) - private lazy val stderrIterator: Iterator[String] = - linesOf(sub.wrapped.getErrorStream) - - /** Lines of `pipe`, ending at EOF or once the process has exited and no line - * arrives within [[OsPipedSubProcess.OrphanedPipeGrace]]. - * - * Everything the process wrote is in the pipe buffer by the time it exits, - * so a read still blocked after the grace waits only on a descendant that - * inherited the pipe and outlived the process. That descendant keeps - * running, and so does the pipe's detached reader thread: blocked in the - * read until the descendant exits or writes, and parked for good if it - * writes. An unterminated last line is dropped in that case. - */ - private def linesOf(pipe: InputStream): Iterator[String] = - val reader = BufferedReader( - InputStreamReader(abandonOnInterruptReads(pipe), UTF_8) - ) - // `raceResult`, so a failed read surfaces at once. Not `reader.lines()`: its - // iterator reads again on each `hasNext` after the end. - Iterator - .continually(raceResult(reader.readLine(), endOfOutput())) - .takeWhile(_ != null) - - /** `null`, as `readLine` at EOF, once the process has exited and the grace - * has passed. - */ - private def endOfOutput(): String = - sub.wrapped.waitFor().discard - sleep(OsPipedSubProcess.OrphanedPipeGrace) - null - - /** Descendants that were alive when the root was signalled. Signalling the - * root can make it exit, at which point its children are reparented to init - * and drop out of the root's `descendants()` walk — so a later tree kill - * would silently miss exactly the processes it exists to reap. Killing a - * remembered `ProcessHandle` cannot hit an unrelated process that recycled - * the pid: the JDK signals only if the pid still has the start time the - * handle was taken with. - * - * The snapshot only ever fills when orca signals a root that is still alive - * — a turn that settles, or a cancel. A root that exited on its own (an - * agent crash, a clean exit with no terminal message) was never signalled, - * so nothing was recorded and its descendants survive. - */ - private val signalledDescendants: AtomicReference[List[ProcessHandle]] = - new AtomicReference(Nil) - - // Signals the root PID only, deliberately: a spawned child inherits the JVM's - // process group (`ProcessBuilder` does not setsid), so `kill -INT -` - // would signal orca itself. Descendants are covered by `destroyForciblyTree`, - // which walks parent→child links instead of the group. - def sendSigInt(): Unit = - // Walked outside the CAS: `updateAndGet` may retry its function, and an OS - // process walk is neither cheap nor idempotent enough to re-run. - val toRemember = liveDescendants() - signalledDescendants - .updateAndGet(prev => (prev ++ toRemember).distinct) - .discard - // Narrows (does not close) the window where the pid has been recycled by an - // unrelated process — unlike the ProcessHandle path, a shelled-out `kill` - // has no start-time check to protect it. SIGINT rather than - // `ProcessHandle.destroy`'s SIGTERM because that is the signal agent CLIs - // treat as "user interrupted me". - if sub.isAlive() then - QuietProc.call(Seq("kill", "-INT", sub.wrapped.pid.toString)).discard - - def isAlive: Boolean = sub.isAlive() - - def destroyForcibly(): Unit = - val _ = sub.wrapped.destroyForcibly() - - override def destroyForciblyTree(): Unit = - // Remembered handles are re-expanded rather than killed flat: by now the - // root is usually gone, so they are the only reachable branch of the tree, - // and each may have spawned children since the snapshot was taken. - // Reachability is best-effort throughout — a process that detached itself - // (`nohup … &`, `setsid`) appears in neither the walk nor the snapshot. A - // process group established here would not extend to it either: every agent - // CLI runs its tool calls in a session of their own, so orca's group never - // contains them (measured: a claude bash tool call's session id is its own - // pid). Reaching a detached process needs a marker it carries — e.g. a - // per-turn cookie injected into the environment and swept at teardown — or - // an OS-level container such as a cgroup scope. - val remembered = - signalledDescendants.get().flatMap(h => h +: descendantsOf(h)) - (liveDescendants() ++ remembered).foreach(_.destroyForcibly().discard) - sub.wrapped.toHandle.destroyForcibly().discard - - private def liveDescendants(): List[ProcessHandle] = - descendantsOf(sub.wrapped.toHandle) - - private def descendantsOf(handle: ProcessHandle): List[ProcessHandle] = - handle.descendants().iterator().asScala.toList - - def waitForExit(): Int = - val _ = sub.join() - sub.exitCode() - - def writeLine(line: String): Unit = - sub.stdin.writeLine(line) - sub.stdin.flush() - - def closeStdin(): Unit = sub.stdin.close() - - def stdoutLines: Iterator[String] = stdoutIterator - - def stderrLines: Iterator[String] = stderrIterator - - def tryExitCode: Option[Int] = - if sub.isAlive() then None else Some(sub.exitCode()) - -private[subprocess] object OsPipedSubProcess: - - /** How long a pipe may stay silent after the process exited before its stream - * ends. - */ - val OrphanedPipeGrace: FiniteDuration = 1.second + procs.spawnPiped(args, cwd = cwd, env = env) diff --git a/tools/src/main/scala/orca/subprocess/PathProbe.scala b/tools/src/main/scala/orca/subprocess/PathProbe.scala index 37c81bb16..9a9ff5364 100644 --- a/tools/src/main/scala/orca/subprocess/PathProbe.scala +++ b/tools/src/main/scala/orca/subprocess/PathProbe.scala @@ -1,7 +1,9 @@ package orca.subprocess +import orca.effect.Processes + /** Whether a word resolves to a runnable command on PATH (ADR 0021 §4). Shared - * by [[orca.runner.StackDiscovery]]'s mechanical PATH check and the shell + * by [[orca.lifecycle.StackDiscovery]]'s mechanical PATH check and the shell * wizard's harness-binary probe. */ private[orca] object PathProbe: @@ -12,7 +14,7 @@ private[orca] object PathProbe: * Exit 0 ⇒ resolvable (builtins included, same environment stage-time `bash * -c` inherits). */ - def resolves(word: String, cwd: os.Path): Boolean = - os.proc("bash", "-c", """command -v -- "$1"""", "bash", word) - .call(cwd = cwd, check = false, stdout = os.Pipe, stderr = os.Pipe) + def resolves(procs: Processes, word: String, cwd: os.Path): Boolean = + procs + .call(Seq("bash", "-c", """command -v -- "$1"""", "bash", word), cwd) .exitCode == 0 diff --git a/tools/src/main/scala/orca/tools/FsTool.scala b/tools/src/main/scala/orca/tools/FsTool.scala index bbc3ea15a..a9f7a7f48 100644 --- a/tools/src/main/scala/orca/tools/FsTool.scala +++ b/tools/src/main/scala/orca/tools/FsTool.scala @@ -1,8 +1,8 @@ package orca.tools import orca.{OrcaDir, OrcaFlowException, WorkspaceWrite} - -import java.nio.file.FileSystems +import orca.effect.FileSystem +import orca.safe.prelude.{*, given} /** Filesystem adapter usable from flow scripts — the handle behind the `fs` * accessor. Reads, writes, and globs files against the flow's working @@ -31,44 +31,38 @@ trait FsTool: def write(path: String, content: String)(using WorkspaceWrite): Unit def list(glob: String): List[String] -/** `FsTool` implementation backed by os-lib. Path resolution and glob semantics - * are specified on the trait; the `list` traversal is narrowed to the deepest +/** `FsTool` over the local filesystem. Path resolution and glob semantics are + * specified on the trait; the `list` traversal is narrowed to the deepest * wildcard-free prefix of the glob. */ -private[orca] class OsFsTool(base: os.Path = os.pwd) extends FsTool: +private[orca] class OsFsTool(base: os.Path, fs: FileSystem) extends FsTool: def read(path: String): Option[String] = val p = resolve(path) - if os.isFile(p) then Some(os.read(p)) else None + if fs.isFile(p) then Some(fs.read(p)) else None def write(path: String, content: String)(using ws: WorkspaceWrite): Unit = ws.check("fs.write") val target = resolve(path) validateWriteTarget(path, target) - os.write.over(target, content, createFolders = true) + fs.writeOver(target, content) def list(glob: String): List[String] = validateGlob(glob) - val matcher = - FileSystems.getDefault.getPathMatcher(s"glob:$glob") + val matches = globMatcher(glob) val root = globRoot(glob) - if !os.exists(root) then Nil + if !fs.exists(root) then Nil else - os.walk - .stream(root) - .filter(os.isFile) - .filter(p => matcher.matches(p.relativeTo(base).toNIO)) - .map(p => p.relativeTo(base).toString) - .toList + fs.files(root, p => matches(p.relativeTo(base))) + .map(_.relativeTo(base).toString) - private def resolve(path: String): os.Path = - os.Path(path, base) + private def resolve(path: String): os.Path = resolvePath(path, base) /** Refuses `target` unless, symlinks resolved, it lies in the working * directory and outside the directories orca manages. */ private def validateWriteTarget(path: String, target: os.Path): Unit = - val realBase = os.Path(base.toNIO.toRealPath()) + val realBase = fs.realPath(base) val real = realLocation(path, target) if !real.startsWith(realBase) then throw OrcaFlowException( @@ -90,13 +84,13 @@ private[orca] class OsFsTool(base: os.Path = os.pwd) extends FsTool: * ancestor (itself included), plus the segments still to be created. */ private def realLocation(path: String, target: os.Path): os.Path = - if os.exists(target) then os.Path(target.toNIO.toRealPath()) - else if os.isLink(target) then + if fs.exists(target) then fs.realPath(target) + else if fs.isLink(target) then throw OrcaFlowException( s"fs.write: '$path' goes through the dangling symlink $target — " + "remove the link or write elsewhere" ) - else realLocation(path, target / os.up) / target.last + else realLocation(path, target / up) / target.last /** Reject glob shapes `globRoot`'s segment fold can't handle cleanly: a * leading `/` (would only ever match nothing, since found paths are relative diff --git a/tools/src/main/scala/orca/tools/GhJson.scala b/tools/src/main/scala/orca/tools/GhJson.scala index 313d89c55..9157ebd2f 100644 --- a/tools/src/main/scala/orca/tools/GhJson.scala +++ b/tools/src/main/scala/orca/tools/GhJson.scala @@ -1,10 +1,7 @@ package orca.tools import com.github.plokhotnyuk.jsoniter_scala.core.JsonValueCodec -import com.github.plokhotnyuk.jsoniter_scala.macros.{ - ConfiguredJsonValueCodec, - JsonCodecMaker -} +import orca.json.JsonCodec // Wire-format DTOs for the `gh`/`git` JSON that `OsGitHubTool` parses. Kept // separate from the public GitHubTool contract: they mirror the GitHub CLI/API @@ -18,16 +15,16 @@ private[tools] case class GhCheck( conclusion: Option[String] = None, state: Option[String] = None, name: Option[String] = None -) derives ConfiguredJsonValueCodec +) derives JsonCodec private[tools] case class GhCheckRollup( statusCheckRollup: List[GhCheck] -) derives ConfiguredJsonValueCodec +) derives JsonCodec private[tools] case class GhCommentJson( body: String, user: GhUserJson -) derives ConfiguredJsonValueCodec +) derives JsonCodec /** Comment JSON with its GitHub-issued numeric id, used by [[OsGitHubTool]] for * the PATCH path in `upsertComment`. Callers see the id-free [[Comment]] type. @@ -36,7 +33,7 @@ private[tools] case class GhIdentifiedCommentJson( id: Long, body: String, user: GhUserJson -) derives ConfiguredJsonValueCodec +) derives JsonCodec /** Minimal PR fields from `gh pr list --json number,url`, used by * [[OsGitHubTool.findOpenPr]] to map a head branch to an open PR. The handle @@ -45,7 +42,7 @@ private[tools] case class GhIdentifiedCommentJson( private[tools] case class GhPrListJson( number: Int, url: String -) derives ConfiguredJsonValueCodec +) derives JsonCodec /** What `gh repo view --json url` returns for the repository gh resolves from * the checkout's remotes: the browser URL, which carries host, owner and repo @@ -53,10 +50,9 @@ private[tools] case class GhPrListJson( */ private[tools] case class GhRepoViewJson( url: String -) derives ConfiguredJsonValueCodec +) derives JsonCodec -private[tools] case class GhUserJson(login: String) - derives ConfiguredJsonValueCodec +private[tools] case class GhUserJson(login: String) derives JsonCodec private[tools] case class GhIssueJson( title: String, @@ -65,12 +61,12 @@ private[tools] case class GhIssueJson( body: Option[String] = None, user: GhUserJson, state: String -) derives ConfiguredJsonValueCodec +) derives JsonCodec private[tools] given ghCommentListCodec: JsonValueCodec[List[GhCommentJson]] = - JsonCodecMaker.make + JsonCodec.make private[tools] given ghIdentifiedCommentListCodec : JsonValueCodec[List[GhIdentifiedCommentJson]] = - JsonCodecMaker.make + JsonCodec.make private[tools] given ghPrListCodec: JsonValueCodec[List[GhPrListJson]] = - JsonCodecMaker.make + JsonCodec.make diff --git a/tools/src/main/scala/orca/tools/GitHubAvailability.scala b/tools/src/main/scala/orca/tools/GitHubAvailability.scala index 8bed8a5ed..5ff5b2091 100644 --- a/tools/src/main/scala/orca/tools/GitHubAvailability.scala +++ b/tools/src/main/scala/orca/tools/GitHubAvailability.scala @@ -48,7 +48,7 @@ enum GitHubUnavailable: * its own next action to. */ def explanation: String = this match - case NoRemote => "this checkout has no git remote" + case NoRemote => "this checkout has no git remote" case NotGitHub(host) => s"origin is on $host, which gh holds no credential for — if it is a " + s"GitHub Enterprise host, `gh auth login --hostname $host` lets orca " + @@ -56,5 +56,5 @@ enum GitHubUnavailable: case NoHost(remote) => s"origin is a local remote ($remote), not a GitHub repository" case Unreachable(host, reason) => s"cannot reach GitHub ($host): $reason" - case GitUnusable(reason) => + case GitUnusable(reason) => s"could not run git to find the remote: $reason" diff --git a/tools/src/main/scala/orca/tools/GitHubTool.scala b/tools/src/main/scala/orca/tools/GitHubTool.scala index e3e7db693..040f35e70 100644 --- a/tools/src/main/scala/orca/tools/GitHubTool.scala +++ b/tools/src/main/scala/orca/tools/GitHubTool.scala @@ -1,16 +1,17 @@ package orca.tools -import com.github.plokhotnyuk.jsoniter_scala.core.readFromString import orca.{OrcaFlowException, WorkspaceWrite} import orca.events.{OrcaEvent, OrcaListener} import orca.subprocess.{CliResult, CliRunner} import orca.util.TextUtil -import ox.sleep -import ox.resilience.{ResultPolicy, RetryConfig, retry} +import orca.json.JsonCodec +import orca.concurrency +import orca.concurrency.{nanoTime, retry, sleep} +import ox.resilience.RetryConfig import ox.scheduling.Schedule +import orca.safe.prelude.{DurationInt, NonFatal} -import scala.concurrent.duration.{DurationInt, FiniteDuration} -import scala.util.control.NonFatal +import scala.concurrent.duration.FiniteDuration case class IssueHandle(owner: String, repo: String, number: Int): /** Canonical GitHub short-form `/#`. */ @@ -206,7 +207,7 @@ trait GitHubTool: else availability() match case GitHubAvailability.Available(host, _, _) => Right(host) - case GitHubAvailability.Unavailable(why) => + case GitHubAvailability.Unavailable(why) => Left(why.explanation) handle <- PrHandle.from(host, issue.owner, issue.repo, issue.number) yield handle @@ -310,7 +311,7 @@ trait GitHubTool: */ private[orca] class OsGitHubTool( cli: CliRunner, - workDir: os.Path = os.pwd, + workDir: os.Path, pollInterval: FiniteDuration = 30.seconds, events: OrcaListener = OrcaListener.noop, readRetry: Schedule = OsGitHubTool.defaultReadRetry @@ -323,9 +324,9 @@ private[orca] class OsGitHubTool( * i.e. an `OrcaFlowException`) under a bounded backoff. */ private val readRetryConfig: RetryConfig[Throwable, String] = - RetryConfig( + concurrency.retryConfig( readRetry, - ResultPolicy.retryWhen[Throwable, String]( + concurrency.retryWhen[Throwable, String]( _.isInstanceOf[OrcaFlowException] ) ) @@ -336,14 +337,13 @@ private[orca] class OsGitHubTool( * start is not retried, as in [[readRetryConfig]]. */ private val probeRetryConfig: RetryConfig[Throwable, CliResult] = - RetryConfig( + concurrency.retryConfig( readRetry, - ResultPolicy( + concurrency.resultPolicy( isSuccess = r => r.exitCode == 0 || (classifyFailure(r) match case GhFailure.Hard => true - case GhFailure.Transient => false - ), + case GhFailure.Transient => false), isWorthRetrying = _ => false ) ) @@ -354,9 +354,9 @@ private[orca] class OsGitHubTool( originUrl() match case OriginProbe.GitUnusable(reason) => Unavailable(GitUnusable(reason)) case OriginProbe.NoOrigin => Unavailable(NoRemote) - case OriginProbe.Origin(url) => + case OriginProbe.Origin(url) => OsGitTool.remoteHost(url) match - case None => Unavailable(NoHost(url)) + case None => Unavailable(NoHost(url)) case Some(host) => credential(host) match case CredentialProbe.Present => repoGhResolves(host) @@ -423,7 +423,7 @@ private[orca] class OsGitHubTool( GitHubUnavailable.Unreachable(gitHost, reason) ) tryRepoView() match - case Left(reason) => unreachable(reason) + case Left(reason) => unreachable(reason) case Right(view) if view.exitCode != 0 => unreachable( ghReason( @@ -443,7 +443,7 @@ private[orca] class OsGitHubTool( */ private def repoFromView(stdout: String): Either[String, GitHubAvailability] = try - val url = readFromString[GhRepoViewJson](stdout).url + val url = JsonCodec.readString[GhRepoViewJson](stdout).url url.stripSuffix("/").stripSuffix(".git") match case RepoUrlPattern(host, owner, repo) => Right( @@ -557,7 +557,9 @@ private[orca] class OsGitHubTool( "--json", "number,url" ) - readFromString[List[GhPrListJson]](output).headOption + JsonCodec + .readString[List[GhPrListJson]](output) + .headOption .flatMap(entry => PrHandle.fromUrl(entry.url)) /** gh's fully qualified `HOST/OWNER/REPO` `--repo` form. */ @@ -567,7 +569,7 @@ private[orca] class OsGitHubTool( def readIssue(issue: IssueHandle): Issue = val target = GhTarget(issue) val output = ghReadApi(target.host, target.issuePath) - val parsed = readFromString[GhIssueJson](output) + val parsed = JsonCodec.readString[GhIssueJson](output) Issue( title = parsed.title, body = parsed.body.getOrElse(""), @@ -586,8 +588,10 @@ private[orca] class OsGitHubTool( private def readCommentsAt(target: GhTarget): List[Comment] = val output = ghReadApi(target.host, "--paginate", target.commentsPath) - readFromString[List[GhCommentJson]](output).map: c => - Comment(author = c.user.login, body = c.body) + JsonCodec + .readString[List[GhCommentJson]](output) + .map: c => + Comment(author = c.user.login, body = c.body) def updatePr(pr: PrHandle, title: String, body: String)(using ws: WorkspaceWrite @@ -672,7 +676,7 @@ private[orca] class OsGitHubTool( target: GhTarget ): List[GhIdentifiedCommentJson] = val output = ghReadApi(target.host, "--paginate", target.commentsPath) - readFromString[List[GhIdentifiedCommentJson]](output) + JsonCodec.readString[List[GhIdentifiedCommentJson]](output) /** PATCH an existing issue/PR comment body via the REST API. */ private def patchComment(target: GhTarget, id: Long, body: String): Unit = @@ -695,7 +699,7 @@ private[orca] class OsGitHubTool( "--json", "statusCheckRollup" ) - val rollup = readFromString[GhCheckRollup](output) + val rollup = JsonCodec.readString[GhCheckRollup](output) val outcome = aggregateOutcome(rollup.statusCheckRollup) val log = rollup.statusCheckRollup .map: c => @@ -703,7 +707,7 @@ private[orca] class OsGitHubTool( // Called out explicitly so the log distinguishes an unrecognised // shape from a real CI failure. case CheckState.Unknown(raw) => s"unknown ($raw)" - case _ => + case _ => c.conclusion.orElse(c.state).orElse(c.status).getOrElse("?") s"${c.name.getOrElse("?")}: $tag" .mkString("\n") @@ -714,7 +718,7 @@ private[orca] class OsGitHubTool( timeout: FiniteDuration, noChecksGrace: FiniteDuration = 90.seconds ): Either[BuildWaitFailed, BuildStatus] = - val start = System.nanoTime() + val start = nanoTime() val deadline = start + timeout.toNanos val noChecksDeadline = start + noChecksGrace.toNanos @@ -723,7 +727,7 @@ private[orca] class OsGitHubTool( // `buildStatus` already retries transient gh/GitHub blips internally; a // failure here means the read failed past that budget. val status = buildStatus(pr) - val now = System.nanoTime() + val now = nanoTime() // Sticky watermark: once one non-empty rollup is seen, the "no CI // configured" hypothesis is disproven, so a later transient empty rollup // can't fire NoChecksConfigured. Driven by structured `checkCount`, not @@ -897,7 +901,7 @@ private[orca] object OsGitHubTool: * schedule. */ val defaultReadRetry: Schedule = - Schedule.exponentialBackoff(1.second).maxRetries(4) + concurrency.exponentialBackoff(1.second).maxRetries(4) // --- Recoverable `gh pr create` stderr/stdout predicates --- // diff --git a/tools/src/main/scala/orca/tools/OsGitTool.scala b/tools/src/main/scala/orca/tools/OsGitTool.scala index 33d21c058..b8bf68608 100644 --- a/tools/src/main/scala/orca/tools/OsGitTool.scala +++ b/tools/src/main/scala/orca/tools/OsGitTool.scala @@ -3,16 +3,14 @@ package orca.tools import orca.{OrcaFlowException, WorkspaceWrite} import orca.events.{OrcaEvent, OrcaListener} import orca.gitref.{BranchName, CommitHash, Head} -import orca.subprocess.{CappedResult, QuietProc} -import ox.either -import ox.either.ok +import orca.effect.{FileSystem, Processes} +import orca.subprocess.{CappedResult, CliResult} -import java.nio.charset.StandardCharsets -import scala.util.control.NonFatal +import orca.safe.prelude.{*, given} -/** `RuntimeGit` implementation that shells out to the `git` CLI via os-lib. - * Contract semantics are specified on the trait; this class handles the - * subprocess plumbing. +/** `RuntimeGit` implementation that shells out to the `git` CLI. Contract + * semantics are specified on the trait; this class handles the subprocess + * plumbing. * * `events` publishes [[OrcaEvent.Step]]s for operations shown in the event log * (branch switches, commits, pushes), and [[OrcaEvent.Bookkeeping]] for the @@ -20,7 +18,9 @@ import scala.util.control.NonFatal * `OrcaListener.noop`. */ private[orca] class OsGitTool( - workDir: os.Path = os.pwd, + workDir: os.Path, + procs: Processes, + fs: FileSystem, events: OrcaListener = OrcaListener.noop ) extends RuntimeGit: @@ -118,12 +118,12 @@ private[orca] class OsGitTool( */ private def gitWithDiagnostics(args: String*): String = val result = gitProc("git" +: args) - if result.exitCode == 0 then result.out.text() + if result.exitCode == 0 then result.stdout else throw OrcaFlowException( OsGitTool.gitFailureMessage( args.mkString(" "), - result.err.text(), + result.stderr, gitDiagnostics() ) ) @@ -136,9 +136,9 @@ private[orca] class OsGitTool( private def gitDiagnostics(): OsGitTool.GitDiagnostics = def tryRun(args: String*): String = val r = gitProc("git" +: args) - if r.exitCode == 0 then r.out.text() + if r.exitCode == 0 then r.stdout else - s"" + s"" OsGitTool.GitDiagnostics( status = tryRun("status", "--porcelain"), fsck = tryRun("fsck", "--no-progress") @@ -156,7 +156,7 @@ private[orca] class OsGitTool( step("Pushed to origin") Right(()) else - val stderr = result.err.text() + val stderr = result.stderr // Order isn't load-bearing: `push()` targets a single ref per call, so // its stderr carries at most one rejection reason (divergence vs. policy // decline are mutually exclusive for the same ref). @@ -170,7 +170,7 @@ private[orca] class OsGitTool( def head(): Head = val result = gitProc(Seq("git", "symbolic-ref", "--quiet", "HEAD")) result.exitCode match - case 0 => Head.OnBranch(OsGitTool.branchOf(result.out.text().trim)) + case 0 => Head.OnBranch(OsGitTool.branchOf(result.stdout.trim)) case 1 => Head.Detached( headCommit().getOrElse( @@ -210,13 +210,13 @@ private[orca] class OsGitTool( gitProc(Seq("git", "symbolic-ref", "--quiet", OsGitTool.OriginHeadRef)) result.exitCode match case 0 => - Some(result.out.text().trim.stripPrefix(OsGitTool.OriginRefsPrefix)) + Some(result.stdout.trim.stripPrefix(OsGitTool.OriginRefsPrefix)) case 1 => None // Not `fail`: this refusal stops a run at preflight, so it names the fix. case exit => throw OrcaFlowException( s"cannot read the default branch from origin/HEAD (exit $exit: " + - s"${result.err.text().trim}) — fix the repository, or reset " + + s"${result.stderr.trim}) — fix the repository, or reset " + "origin/HEAD with `git remote set-head origin -a`, and re-run" ) @@ -256,9 +256,9 @@ private[orca] class OsGitTool( ): Either[SnapshotFailed, Option[UncommittedSnapshot]] = ws.check("git.snapshotUncommitted") val result = gitProc(Seq("git", "stash", "create")) - val out = result.out.text().trim + val out = result.stdout.trim if result.exitCode != 0 then - Left(SnapshotFailed(s"git stash create: ${result.err.text().trim}")) + Left(SnapshotFailed(s"git stash create: ${result.stderr.trim}")) // `stash create` prints nothing when no tracked file differs from HEAD. else if out.isEmpty then Right(None) else @@ -277,7 +277,7 @@ private[orca] class OsGitTool( val result = gitProc(Seq("git", "stash", "apply", "--index", snapshot.commit.value)) if result.exitCode == 0 then Right(()) - else Left(SnapshotFailed(s"git stash apply: ${result.err.text().trim}")) + else Left(SnapshotFailed(s"git stash apply: ${result.stderr.trim}")) def uncommittedDiff(): String = trackedDiff("HEAD") @@ -422,7 +422,7 @@ private[orca] class OsGitTool( acc: List[OsGitTool.UntrackedPiece] ): List[OsGitTool.UntrackedPiece] = remaining match - case Nil => acc.reverse + case Nil => acc.reverse case path :: rest => val piece = if size >= budget then @@ -439,7 +439,7 @@ private[orca] class OsGitTool( // would let rendering continue past a cut and put the cut marker // mid-document. private def utf8Size(piece: String): Int = - piece.getBytes(StandardCharsets.UTF_8).length + piece.getBytes(UTF_8).length /** Untracked, non-`.orca/` paths anywhere in the repository, relative to * `workDir`. Untracked directories are recursed into, so an entry is @@ -520,15 +520,18 @@ private[orca] class OsGitTool( * path inside (`error: Could not access 'x/null'`), which takes * [[untrackedFileDiff]]'s failure branch. A nested repository is recognised * by its `.git` entry — a directory in a clone, a file in a linked worktree, - * hence `os.exists` rather than `os.isDir`. + * hence `exists` rather than `isDir`. * * Symlinks to a file or to nothing are diffable, as mode 120000. */ private def undiffableReason(relPath: String): Option[String] = - val path = workDir / os.RelPath(relPath) - if os.isLink(path) && os.isDir(path) then Some("symlink to a directory") - else if os.exists(path / ".git") then Some("nested git repository") - else None + orca.safe.prelude.relPath(relPath).map(workDir / _) match + case None => Some("not a relative path") + case Some(path) if fs.isLink(path) && fs.isDir(path) => + Some("symlink to a directory") + case Some(path) if fs.exists(path / ".git") => + Some("nested git repository") + case Some(_) => None // `--end-of-options` so a `base` spelled like a flag is refused as a // revision rather than obeyed (`--output=` would write a file). @@ -671,8 +674,8 @@ private[orca] class OsGitTool( * git (or ssh it spawns) can block the flow on an interactive credential or * passphrase prompt. */ - private def gitProc(args: Seq[String]): os.CommandResult = - QuietProc.call(args, cwd = workDir, env = OsGitTool.nonInteractiveEnv) + private def gitProc(args: Seq[String]): CliResult = + procs.call(args, cwd = workDir, env = OsGitTool.nonInteractiveEnv) /** Trimmed stdout of a read-only git command that exited 0; `None` on a * non-zero exit, empty output, or a subprocess error — the shared shape of @@ -692,14 +695,14 @@ private[orca] class OsGitTool( private def probeStdout(args: Seq[String]): Option[String] = try val result = gitProc("git" +: args) - if result.exitCode == 0 then Some(result.out.text()) else None + if result.exitCode == 0 then Some(result.stdout) else None catch case NonFatal(_) => None /** [[gitProc]] for a command whose output size is set by what it reads rather * than by its arguments, keeping at most [[OsGitTool.MaxReadBytes]] of it. */ private def gitProcCapped(args: Seq[String]): CappedResult = - QuietProc.callCapped( + procs.callCapped( args, maxBytes = OsGitTool.MaxReadBytes, cwd = workDir, @@ -710,20 +713,20 @@ private[orca] class OsGitTool( * unrecoverable git failure. Callers handle the EXPECTED non-zero exits * (rejected push, "already exists") as `Left`s before reaching here. */ - private def fail(label: String, result: os.CommandResult): Nothing = - failWith(label, result.exitCode, result.err.text().trim) + private def fail(label: String, result: CliResult): Nothing = + failWith(label, result.exitCode, result.stderr.trim) private def failWith(label: String, exitCode: Int, stderr: String): Nothing = throw OrcaFlowException(s"$label failed (exit $exitCode): $stderr") private def git(args: String*): String = - // Route through QuietProc so git's stderr ("Switched to a new branch", + // Route through `procs` so git's stderr ("Switched to a new branch", // etc.) is captured rather than leaked to the parent terminal, where it // would tear the renderer's status row. Branch-state changes surface in the // event log via the `step` calls in the public methods above. val result = gitProc("git" +: args) if result.exitCode != 0 then fail(s"git ${args.mkString(" ")}", result) - result.out.text() + result.stdout /** [[git]] for a command whose output is bounded only by what it reads. * [[marked]] stays outside rather than folded in, since @@ -837,7 +840,7 @@ private[orca] object OsGitTool: * a user's custom `GIT_SSH_COMMAND` is preserved. */ private[tools] val nonInteractiveEnv: Map[String, String] = - val baseSsh = sys.env.getOrElse("GIT_SSH_COMMAND", "ssh") + val baseSsh = scala.util.Properties.envOrElse("GIT_SSH_COMMAND", "ssh") Map( "GIT_TERMINAL_PROMPT" -> "0", "GIT_SSH_COMMAND" -> s"$baseSsh -o BatchMode=yes" diff --git a/tools/src/main/scala/orca/tools/Worktrees.scala b/tools/src/main/scala/orca/tools/Worktrees.scala index 5422c2dee..c28c7d8b4 100644 --- a/tools/src/main/scala/orca/tools/Worktrees.scala +++ b/tools/src/main/scala/orca/tools/Worktrees.scala @@ -1,9 +1,9 @@ package orca.tools +import orca.effect.{FileSystem, Processes} import orca.gitref.BranchName -import orca.subprocess.QuietProc - -import scala.util.control.NonFatal +import orca.subprocess.CliResult +import orca.safe.prelude.{*, given} /** Why [[Worktrees.add]] refused. `NoCommitsYet` is a case of its own because * the caller reports it in orca's own words; for anything else git's message @@ -54,9 +54,7 @@ private[orca] enum StartBranchFailure: * the refusal into words. The writes, [[add]] and [[startBranch]], return * their failures. */ -private[orca] object Worktrees: - - private val WorktreeLine = "worktree (.*)".r +private[orca] final class Worktrees(procs: Processes, fs: FileSystem): /** The main checkout of the repository containing `cwd`, identical whether * `cwd` is that checkout or any linked worktree of it. `None` when `cwd` is @@ -97,13 +95,13 @@ private[orca] object Worktrees: case None => Left(MainCheckoutFailure.NotARepository) case Some(List(gitDir, commonDir, topLevel)) => if gitDir == commonDir then - absolute(topLevel).toRight(MainCheckoutFailure.Unsupported) + absolutePath(topLevel).toRight(MainCheckoutFailure.Unsupported) // Creating worktrees under a git directory would write a checkout // inside the repository's own metadata store, so a candidate that is // not itself a checkout is refused rather than written to. else worktrees.headOption - .filter(p => os.exists(p / ".git")) + .filter(p => fs.exists(p / ".git")) .toRight(MainCheckoutFailure.MainWorktreeNotACheckout) case Some(_) => Left(MainCheckoutFailure.Unsupported) @@ -116,8 +114,8 @@ private[orca] object Worktrees: probe(cwd, "worktree", "list", "--porcelain").toList .flatMap(_.linesIterator) .flatMap: - case WorktreeLine(path) => absolute(path) - case _ => None + case Worktrees.WorktreeLine(path) => absolutePath(path) + case _ => None /** Create a worktree at `path`, detached at `cwd`'s current HEAD. * @@ -151,7 +149,7 @@ private[orca] object Worktrees: git(cwd, "worktree", "add", "--force", "--detach", path.toString) if result.exitCode == 0 then Right(()) else - val stderr = result.err.text().trim + val stderr = result.stderr.trim Left( WorktreeAddFailure.GitFailed( if stderr.nonEmpty then stderr @@ -179,7 +177,7 @@ private[orca] object Worktrees: else val result = git(worktree, "checkout", "-B", name.value) if result.exitCode == 0 then Right(()) - else Left(StartBranchFailure.GitFailed(result.err.text().trim)) + else Left(StartBranchFailure.GitFailed(result.stderr.trim)) /** Whether HEAD is DEFINITELY on a branch — `false` for a detached checkout * and for a worktree that cannot be read, so a caller repairing a detached @@ -199,16 +197,8 @@ private[orca] object Worktrees: "HEAD" ).exitCode != 0 - /** Git prints absolute paths for both reads above; anything else means the - * command did not do what was asked, which is "not known" rather than a - * failure to raise at a caller that has nowhere to put one — `os.Path` - * throws on a relative or empty string. - */ - private def absolute(path: String): Option[os.Path] = - Option.when(path.startsWith("/"))(os.Path(path)) - - private def git(cwd: os.Path, args: String*): os.CommandResult = - QuietProc.call("git" +: args, cwd = cwd, env = OsGitTool.nonInteractiveEnv) + private def git(cwd: os.Path, args: String*): CliResult = + procs.call("git" +: args, cwd = cwd, env = OsGitTool.nonInteractiveEnv) /** Stdout of a read-only git command that exited 0; `None` on anything else, * including a subprocess that could not be started. @@ -216,5 +206,8 @@ private[orca] object Worktrees: private def probe(cwd: os.Path, args: String*): Option[String] = try val result = git(cwd, args*) - if result.exitCode == 0 then Some(result.out.text()) else None + if result.exitCode == 0 then Some(result.stdout) else None catch case NonFatal(_) => None + +private[orca] object Worktrees: + private val WorktreeLine = "worktree (.*)".r diff --git a/tools/src/main/scala/orca/util/AtomicFile.scala b/tools/src/main/scala/orca/util/AtomicFile.scala index 4989a7c2c..840cf2467 100644 --- a/tools/src/main/scala/orca/util/AtomicFile.scala +++ b/tools/src/main/scala/orca/util/AtomicFile.scala @@ -1,12 +1,7 @@ package orca.util -import java.nio.ByteBuffer -import java.nio.channels.FileChannel -import java.nio.file.StandardOpenOption -import java.util.UUID - -import scala.util.Using -import scala.util.control.NonFatal +import orca.effect.FileSystem +import orca.safe.prelude.{*, given} /** Whole-file replacement that a kill cannot tear: the content goes to a temp * file, is flushed to disk, then renamed over the target. A reader sees the @@ -19,11 +14,16 @@ private[orca] object AtomicFile: * symlink at `path` rather than writing through it. `path` is untouched and * the temp file gone when this throws. */ - def replace(path: os.Path, stagingDir: os.Path, bytes: Array[Byte]): Unit = - val tmp = stagingDir / s".${path.last}.${UUID.randomUUID()}.tmp" + def replace( + fs: FileSystem, + path: os.Path, + stagingDir: os.Path, + bytes: IArray[Byte] + ): Unit = + val tmp = stagingDir / s".${path.last}.${randomUuid()}.tmp" try - writeDurably(tmp, bytes) - try os.move(tmp, path, replaceExisting = true, atomicMove = true) + fs.writeNewDurably(tmp, bytes) + try fs.moveAtomically(tmp, path) catch // Some filesystems (network mounts, some container overlay/bind mounts) // reject ATOMIC_MOVE even for a same-directory rename. Torn writes are @@ -31,24 +31,8 @@ private[orca] object AtomicFile: // concurrent readers is unavailable), so a plain move is a safe // fallback. case _: java.nio.file.AtomicMoveNotSupportedException => - os.move(tmp, path, replaceExisting = true) + fs.moveReplacing(tmp, path) catch case NonFatal(e) => - if os.exists(tmp) then os.remove(tmp): Unit + if fs.exists(tmp) then fs.remove(tmp) throw e - - /** Create `file` and flush `bytes` to disk before returning. `CREATE_NEW` - * rather than `os.temp`, so the file gets the umask's permissions, not - * owner-only ones the rename would carry onto the target. - */ - private def writeDurably(file: os.Path, bytes: Array[Byte]): Unit = - Using.resource( - FileChannel.open( - file.toNIO, - StandardOpenOption.CREATE_NEW, - StandardOpenOption.WRITE - ) - ): channel => - val buffer = ByteBuffer.wrap(bytes) - while buffer.hasRemaining do channel.write(buffer): Unit - channel.force(true) diff --git a/tools/src/main/scala/orca/util/OrcaDebug.scala b/tools/src/main/scala/orca/util/OrcaDebug.scala index 599b7fd3b..647eaddf6 100644 --- a/tools/src/main/scala/orca/util/OrcaDebug.scala +++ b/tools/src/main/scala/orca/util/OrcaDebug.scala @@ -1,5 +1,9 @@ package orca.util +import orca.safe.Logger + +import scala.util.Properties.envOrNone + /** One place that knows which `ORCA_*` debug environment variables exist. The * flags are read once at object initialization, so the booleans are safe to * pin in cold paths. @@ -13,17 +17,17 @@ private[orca] object OrcaDebug: /** `ORCA_DEBUG=1` — print full JVM stack traces on errors that the stage * machinery would otherwise format. Read by [[orca.flow]]. */ - val enabled: Boolean = sys.env.get("ORCA_DEBUG").contains("1") + val enabled: Boolean = envOrNone("ORCA_DEBUG").contains("1") /** `ORCA_DEBUG_STREAM=1` — dump every inbound NDJSON / JSONL line from the * agent subprocess to the parent's stderr before parsing. Read by * [[orca.backend.DecodedTurn]] and the decoders. */ - val streamTrace: Boolean = sys.env.get("ORCA_DEBUG_STREAM").contains("1") + val streamTrace: Boolean = envOrNone("ORCA_DEBUG_STREAM").contains("1") /** Prints one raw line of a backend's `stream` to stderr under * [[streamTrace]]. */ def traceStream(backendName: String, stream: String, line: String): Unit = if streamTrace then - System.err.println(s"[orca-debug $backendName-$stream] $line") + Logger.stderr(s"[orca-debug $backendName-$stream] $line") diff --git a/tools/src/main/scala/orca/util/PromptResource.scala b/tools/src/main/scala/orca/util/PromptResource.scala index 9f48d97a6..fbda272e8 100644 --- a/tools/src/main/scala/orca/util/PromptResource.scala +++ b/tools/src/main/scala/orca/util/PromptResource.scala @@ -1,5 +1,7 @@ package orca.util +import orca.safe.prelude.resourceText + /** Loads prompt templates from classpath resources, one `.md` file per template * under `src/main/resources//prompts/.md`. * @@ -15,13 +17,11 @@ private[orca] object PromptResource: * the classpath root (use a leading `/`). */ def load(path: String): String = - val stream = Option(getClass.getResourceAsStream(path)).getOrElse( + resourceText(path).getOrElse( throw new RuntimeException( s"prompt resource not found on classpath: $path" ) ) - try scala.io.Source.fromInputStream(stream, "UTF-8").mkString - finally stream.close() /** Substitute `{{name}}` placeholders in `template` with the supplied `(name * -> value)` pairs. Unreferenced substitutions are ignored; a placeholder diff --git a/tools/src/main/scala/orca/util/TerminalControl.scala b/tools/src/main/scala/orca/util/TerminalControl.scala index 7cb84e914..752b24a07 100644 --- a/tools/src/main/scala/orca/util/TerminalControl.scala +++ b/tools/src/main/scala/orca/util/TerminalControl.scala @@ -29,7 +29,7 @@ private[orca] object TerminalControl: case Osc => i = skipStringControl(text, i + 1) case Dcs | Sos | Pm | Apc => i = skipStringControl(text, i + 1) case c if isUnsafeControl(c) => i += 1 - case c => + case c => val _ = out.append(c) i += 1 out.toString diff --git a/tools/src/test/scala/orca/OrcaDirTest.scala b/tools/src/test/scala/orca/OrcaDirTest.scala index bc423dbe9..c6fa5e602 100644 --- a/tools/src/test/scala/orca/OrcaDirTest.scala +++ b/tools/src/test/scala/orca/OrcaDirTest.scala @@ -1,7 +1,7 @@ package orca import ox.discard -import orca.testkit.TempDirs +import orca.testkit.{Effects, TempDirs} import java.time.Instant @@ -9,7 +9,7 @@ class OrcaDirTest extends munit.FunSuite: test("ensureCache creates the cache dir with exact marker file contents"): val wd = TempDirs.dir() - val cache = OrcaDir.ensureCache(wd) + val cache = Effects.orcaDir.ensureCache(wd) assertEquals(cache, wd / ".orca" / "cache") assert(os.isDir(cache)) assertEquals( @@ -24,17 +24,17 @@ class OrcaDirTest extends munit.FunSuite: test("second ensureCache call leaves existing marker files untouched"): val wd = TempDirs.dir() - val cache = OrcaDir.ensureCache(wd) + val cache = Effects.orcaDir.ensureCache(wd) // Surviving canaries prove the files are written only when absent. os.write.over(cache / ".gitignore", "canary-gitignore") os.write.over(cache / "CACHEDIR.TAG", "canary-tag") - assertEquals(OrcaDir.ensureCache(wd), cache) + assertEquals(Effects.orcaDir.ensureCache(wd), cache) assertEquals(os.read(cache / ".gitignore"), "canary-gitignore") assertEquals(os.read(cache / "CACHEDIR.TAG"), "canary-tag") test("ensureRoot creates .orca only, without the cache dir"): val wd = TempDirs.dir() - val root = OrcaDir.ensureRoot(wd) + val root = Effects.orcaDir.ensureRoot(wd) assertEquals(root, wd / ".orca") assert(os.isDir(root)) assert(!os.exists(root / "cache")) @@ -46,7 +46,7 @@ class OrcaDirTest extends munit.FunSuite: test("ensureWorktrees writes the .gitignore but no CACHEDIR.TAG"): val wd = TempDirs.dir() - val worktrees = OrcaDir.ensureWorktrees(wd) + val worktrees = Effects.orcaDir.ensureWorktrees(wd) assertEquals(worktrees, wd / ".orca" / "worktrees") assertEquals( os.read(worktrees / ".gitignore"), @@ -60,7 +60,7 @@ class OrcaDirTest extends munit.FunSuite: os.makeDir.all(outside) os.makeDir.all(OrcaDir.rootPath(wd)) os.symlink(OrcaDir.worktreesPath(wd), outside) - intercept[OrcaFlowException](OrcaDir.ensureWorktrees(wd)).discard + intercept[OrcaFlowException](Effects.orcaDir.ensureWorktrees(wd)).discard assert(os.list(outside).isEmpty, "no write must go through the symlink") test("flowsPath points at .orca/flows without creating it"): @@ -70,10 +70,10 @@ class OrcaDirTest extends munit.FunSuite: test("ensureFlows creates .orca/flows and is idempotent"): val wd = TempDirs.dir() - val flows = OrcaDir.ensureFlows(wd) + val flows = Effects.orcaDir.ensureFlows(wd) assertEquals(flows, wd / ".orca" / "flows") assert(os.isDir(flows)) - assertEquals(OrcaDir.ensureFlows(wd), flows) + assertEquals(Effects.orcaDir.ensureFlows(wd), flows) test("ensureFlows aborts on a symlinked .orca/flows"): val wd = TempDirs.dir() @@ -81,12 +81,12 @@ class OrcaDirTest extends munit.FunSuite: os.makeDir.all(outside) os.makeDir.all(OrcaDir.rootPath(wd)) os.symlink(OrcaDir.rootPath(wd) / "flows", outside) - intercept[OrcaFlowException](OrcaDir.ensureFlows(wd)).discard + intercept[OrcaFlowException](Effects.orcaDir.ensureFlows(wd)).discard assert(os.list(outside).isEmpty, "no write must go through the symlink") test("assertNoOrcaSymlinks is a no-op when .orca doesn't exist"): val wd = TempDirs.dir() - OrcaDir.assertNoOrcaSymlinks(wd, OrcaDir.flowsPath(wd)) + Effects.orcaDir.assertNoOrcaSymlinks(wd, OrcaDir.flowsPath(wd)) assert( !os.exists(OrcaDir.rootPath(wd)), "must not create .orca as a side effect" @@ -94,8 +94,8 @@ class OrcaDirTest extends munit.FunSuite: test("assertNoOrcaSymlinks passes silently for a real .orca/flows dir"): val wd = TempDirs.dir() - OrcaDir.ensureFlows(wd).discard - OrcaDir.assertNoOrcaSymlinks(wd, OrcaDir.flowsPath(wd)) + Effects.orcaDir.ensureFlows(wd).discard + Effects.orcaDir.assertNoOrcaSymlinks(wd, OrcaDir.flowsPath(wd)) test("assertNoOrcaSymlinks aborts on a symlinked .orca/flows"): val wd = TempDirs.dir() @@ -104,13 +104,13 @@ class OrcaDirTest extends munit.FunSuite: os.makeDir.all(OrcaDir.rootPath(wd)) os.symlink(OrcaDir.rootPath(wd) / "flows", outside) val ex = intercept[OrcaFlowException]( - OrcaDir.assertNoOrcaSymlinks(wd, OrcaDir.flowsPath(wd)) + Effects.orcaDir.assertNoOrcaSymlinks(wd, OrcaDir.flowsPath(wd)) ) assert(ex.getMessage.contains("symlink"), ex.getMessage) test("a committed file's replace leaves only the file in its directory"): val wd = TempDirs.dir() - val file = OrcaDir.progressFile(wd, RunKey.of("p")) + val file = Effects.orcaDir.progressFile(wd, RunKey.of("p")) file.replace("{}") assertEquals(os.list(OrcaDir.runsPath(wd)).toList, List(file.path)) @@ -121,7 +121,7 @@ class OrcaDirTest extends munit.FunSuite: os.makeDir.all(OrcaDir.rootPath(wd)) os.symlink(OrcaDir.runsPath(wd), outside) intercept[OrcaFlowException]( - OrcaDir.progressFile(wd, RunKey.of("p")) + Effects.orcaDir.progressFile(wd, RunKey.of("p")) ).discard test("runsPath points at .orca/runs without creating anything"): @@ -142,7 +142,7 @@ class OrcaDirTest extends munit.FunSuite: os.write(outside, "kept") os.makeDir.all(OrcaDir.rootPath(wd)) os.symlink(OrcaDir.settingsPath(wd), outside) - OrcaDir.settingsFile(wd).replace("format = x\n") + Effects.orcaDir.settingsFile(wd).replace("format = x\n") assert(!os.isLink(OrcaDir.settingsPath(wd))) assertEquals(os.read(OrcaDir.settingsPath(wd)), "format = x\n") assertEquals(os.read(outside), "kept") @@ -182,7 +182,7 @@ class OrcaDirTest extends munit.FunSuite: test("ensureRunDir creates .orca/cache/runs/, with the cache markers"): val wd = TempDirs.dir() val key = RunKey.of("p") - val runDir = OrcaDir.ensureRunDir(wd, key) + val runDir = Effects.orcaDir.ensureRunDir(wd, key) assertEquals(runDir, OrcaDir.runDirPath(wd, key)) assert(os.isDir(runDir)) assert(os.exists(wd / ".orca" / "cache" / ".gitignore")) @@ -195,13 +195,13 @@ class OrcaDirTest extends munit.FunSuite: os.makeDir.all(outside) os.makeDir.all(OrcaDir.cacheRunsPath(wd)) os.symlink(OrcaDir.runDirPath(wd, key), outside) - intercept[OrcaFlowException](OrcaDir.ensureRunDir(wd, key)).discard + intercept[OrcaFlowException](Effects.orcaDir.ensureRunDir(wd, key)).discard test( "ensurePiSessions creates .orca/cache/pi-sessions, including the cache markers" ): val wd = TempDirs.dir() - val sessions = OrcaDir.ensurePiSessions(wd) + val sessions = Effects.orcaDir.ensurePiSessions(wd) assertEquals(sessions, wd / ".orca" / "cache" / "pi-sessions") assert(os.isDir(sessions)) assert(os.exists(wd / ".orca" / "cache" / ".gitignore")) @@ -210,9 +210,9 @@ class OrcaDirTest extends munit.FunSuite: val wd = TempDirs.dir() val outside = TempDirs.dir() / "outside-sessions" os.makeDir.all(outside) - OrcaDir.ensureCache(wd).discard + Effects.orcaDir.ensureCache(wd).discard os.symlink(OrcaDir.piSessionsPath(wd), outside) - intercept[OrcaFlowException](OrcaDir.ensurePiSessions(wd)).discard + intercept[OrcaFlowException](Effects.orcaDir.ensurePiSessions(wd)).discard test( "piSessionsPath points at .orca/cache/pi-sessions without creating anything" diff --git a/tools/src/test/scala/orca/StagePathTest.scala b/tools/src/test/scala/orca/StagePathTest.scala index 79dc64391..40881db78 100644 --- a/tools/src/test/scala/orca/StagePathTest.scala +++ b/tools/src/test/scala/orca/StagePathTest.scala @@ -5,7 +5,9 @@ import com.github.plokhotnyuk.jsoniter_scala.core.{ readFromString, writeToString } +import io.circe.parser.parse import munit.FunSuite +import orca.json.{JsonData, SchemaParitySuite} /** Stage path identity and its persisted form (ADR 0018 §2.1). */ class StagePathTest extends FunSuite: @@ -35,3 +37,25 @@ class StagePathTest extends FunSuite: StagePath.FlowBody.child("Implement", 0).child("Task", 1).display, "Implement#0/Task#1" ) + + test("a stage's schema is an array of name-occurrence segments"): + val schema = summon[JsonData[StagePath.Stage]].schema + assertEquals( + SchemaParitySuite.render(schema), + parse("""{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$defs": { + "Segment": { + "title": "Segment", + "type": "object", + "required": ["name", "occurrence"], + "properties": { + "name": {"type": "string"}, + "occurrence": {"type": "integer", "format": "int32"} + } + } + }, + "type": "array", + "items": {"$ref": "#/$defs/Segment"} + }""").toTry.get + ) diff --git a/tools/src/test/scala/orca/agents/JsonSchemaGenTest.scala b/tools/src/test/scala/orca/agents/JsonSchemaGenTest.scala index bc036b264..4aab629e3 100644 --- a/tools/src/test/scala/orca/agents/JsonSchemaGenTest.scala +++ b/tools/src/test/scala/orca/agents/JsonSchemaGenTest.scala @@ -1,84 +1,18 @@ package orca.agents import io.circe.parser.parse -import io.circe.{Json, JsonObject} +import orca.json.SchemaDerivation +import sttp.tapir.Schema class JsonSchemaGenTest extends munit.FunSuite: - private def strict(input: String): Json = - JsonSchemaGen.toOpenAiStrict(parse(input).toOption.get) - - test("object node gains additionalProperties:false and full required list"): - // OpenAI strict mode demands every object declare additionalProperties - // false AND list every property in required (nullability is how - // optionality is expressed). Tapir's output doesn't supply either. - val out = strict( - """{"type":"object","properties":{"a":{"type":"string"},"b":{"type":"integer"}}}""" - ) - val obj = out.asObject.get - assertEquals(obj("additionalProperties"), Some(Json.False)) - val required = - obj("required").get.asArray.get.map(_.asString.get).toSet - assertEquals(required, Set("a", "b")) - - test("object without properties is left alone"): - // A marker `{"type":"object"}` (no properties) carries no constraints - // to enforce — don't graft additionalProperties onto an empty shell. - val out = strict("""{"type":"object"}""") - assertEquals(out.asObject.get("additionalProperties"), None) - assertEquals(out.asObject.get("required"), None) - - test("nested object schemas inside $defs are transformed too"): - // `$defs` is where Tapir parks the case class definitions referenced by - // `$ref` — they're the meaningful object schemas and must be strict too. - val out = strict( - """{ - | "type":"object", - | "properties":{"t":{"$ref":"#/$defs/Task"}}, - | "$defs":{ - | "Task":{"type":"object","properties":{"title":{"type":"string"}}} - | } - |}""".stripMargin - ) - val task = - out.hcursor - .downField("$defs") - .downField("Task") - .as[JsonObject] - .toOption - .get - assertEquals(task("additionalProperties"), Some(Json.False)) - assertEquals( - task("required").get.asArray.get.map(_.asString.get).toList, - List("title") - ) - - test( - "a Map-shaped additionalProperties field (Tapir's real Map shape) fails fast with an actionable message" - ): - // The realistic shape Tapir emits for a Map[String,T] field: an object - // node with additionalProperties but no `properties` key at all. - // `rejectMapShapedAdditionalProperties` runs before the `properties` - // check, so this fails fast here rather than letting the non-strict - // schema reach codex/claude and bounce back as an opaque - // `invalid_json_schema`. - val ex = intercept[orca.OrcaFlowException]: - strict( - """{"type":"object","additionalProperties":{"type":"integer"}}""" - ) - assert( - ex.getMessage.contains("List of key/value case classes"), - s"expected actionable Map-field message, got: ${ex.getMessage}" - ) - test("apply emits no $schema dialect declaration"): // TapirSchemaToJsonSchema stamps `$schema: .../draft/2020-12/schema` on its // output, but the claude CLI's `--json-schema` validator has no 2020-12 // meta-schema registered and rejects any schema declaring that dialect. No // consumer needs the declaration, so `apply` strips it. - import sttp.tapir.Schema - import sttp.tapir.generic.auto.given case class Sample(name: String) + given Schema[Sample] = SchemaDerivation.derived val parsed = parse(JsonSchemaGen[Sample]).toOption.get assertEquals(parsed.asObject.get("$schema"), None) @@ -86,9 +20,8 @@ class JsonSchemaGenTest extends munit.FunSuite: // End-to-end check that the strict transform doesn't collapse Tapir's // nullable encoding — the agent must be able to emit `null` for an // Option-typed field that's now `required`. - import sttp.tapir.Schema - import sttp.tapir.generic.auto.given case class Sample(name: String, note: Option[String]) + given Schema[Sample] = SchemaDerivation.derived val parsed = parse(JsonSchemaGen[Sample]).toOption.get val noteType = parsed.hcursor .downField("properties") diff --git a/tools/src/test/scala/orca/agents/ResponseParserTest.scala b/tools/src/test/scala/orca/agents/ResponseParserTest.scala index d326dbe80..94e39c8a5 100644 --- a/tools/src/test/scala/orca/agents/ResponseParserTest.scala +++ b/tools/src/test/scala/orca/agents/ResponseParserTest.scala @@ -61,6 +61,11 @@ class ResponseParserTest extends munit.FunSuite: """{"input":"{\"name\":\"widget\",\"count\":3}"}""" assertEquals(ResponseParser.parse[ParsedSample](wrapped), expected) + test("unwraps a string-encoded envelope value preceded by whitespace"): + val wrapped = + """{"input": "{\"name\":\"widget\",\"count\":3}"}""" + assertEquals(ResponseParser.parse[ParsedSample](wrapped), expected) + test("unwraps a lone input-key envelope, nested-object value"): val wrapped = """{"input":{"name":"widget","count":3}}""" assertEquals(ResponseParser.parse[ParsedSample](wrapped), expected) diff --git a/tools/src/test/scala/orca/backend/DecodedTurnTeardownTest.scala b/tools/src/test/scala/orca/backend/DecodedTurnTeardownTest.scala index a40c29b2f..5cebaf6f1 100644 --- a/tools/src/test/scala/orca/backend/DecodedTurnTeardownTest.scala +++ b/tools/src/test/scala/orca/backend/DecodedTurnTeardownTest.scala @@ -1,5 +1,7 @@ package orca.backend +import orca.testkit.Effects + import orca.AgentTurnFailed import orca.agents.{BackendTag, WireSessionId} import orca.events.{TurnDebit, Usage} @@ -60,7 +62,7 @@ class DecodedTurnTeardownTest extends munit.FunSuite: check: (LiveTurn[BackendTag.ClaudeCode.type], Long) => Unit ): Unit = supervised: - val process = OsProcCliRunner.spawnPiped( + val process = OsProcCliRunner(Effects.procs).spawnPiped( Seq("bash", "-c", script), env = Map.empty, cwd = os.pwd diff --git a/tools/src/test/scala/orca/backend/EnvCookieSweepTest.scala b/tools/src/test/scala/orca/backend/EnvCookieSweepTest.scala index 6e638d076..306fa003b 100644 --- a/tools/src/test/scala/orca/backend/EnvCookieSweepTest.scala +++ b/tools/src/test/scala/orca/backend/EnvCookieSweepTest.scala @@ -1,14 +1,11 @@ package orca.backend -import orca.subprocess.{EnvCookie, PipedCliProcess} +import orca.subprocess.PipedCliProcess class EnvCookieSweepTest extends munit.FunSuite with SweepFixtures: - private def cookieOf(process: PipedCliProcess): EnvCookie = - process.envCookie.getOrElse(fail("spawnPiped must inject a cookie")) - - private def survivorPids(cookie: EnvCookie): List[Long] = - EnvCookieSweep.sweep(cookie).map(_.pid) + private def survivorPids(process: PipedCliProcess): List[Long] = + process.survivors().map(_.pid) private def isAlive(pid: Long): Boolean = ProcessHandle.of(pid).map[Boolean](_.isAlive).orElse(false) @@ -25,11 +22,10 @@ class EnvCookieSweepTest extends munit.FunSuite with SweepFixtures: // Wait so the launching shell is reaped: only the detached worker can // then be carrying the cookie. val _ = process.waitForExit() - val cookie = cookieOf(process) - assertEquals(survivorPids(cookie), List(detachedPid)) + assertEquals(survivorPids(process), List(detachedPid)) val listener = new StepRecorder - EnvCookieSweep.afterScope(Some(cookie), listener) + EnvCookieSweep.afterScope(process, listener) assertEquals(listener.steps.size, 1) assert( listener.steps.head.contains(detachedPid.toString), @@ -43,7 +39,7 @@ class EnvCookieSweepTest extends munit.FunSuite with SweepFixtures: val decoy = spawn("echo $$; sleep 60") try val sweptPid = swept.stdoutLines.next().trim.toLong - assertEquals(survivorPids(cookieOf(swept)), List(sweptPid)) + assertEquals(survivorPids(swept), List(sweptPid)) finally swept.destroyForciblyTree() decoy.destroyForciblyTree() diff --git a/tools/src/test/scala/orca/backend/MessageGrammarTest.scala b/tools/src/test/scala/orca/backend/MessageGrammarTest.scala index 15ae32f0f..a86a6b99c 100644 --- a/tools/src/test/scala/orca/backend/MessageGrammarTest.scala +++ b/tools/src/test/scala/orca/backend/MessageGrammarTest.scala @@ -44,7 +44,7 @@ class MessageGrammarTest extends munit.FunSuite: ) case "messageend" => Step.continue((), TurnEvent.AssistantMessageEnd) case "error" => Step.continue((), TurnEvent.Error("boom")) - case "succeed" => + case "succeed" => Step.Settle( (), Nil, diff --git a/tools/src/test/scala/orca/backend/SweepFixtures.scala b/tools/src/test/scala/orca/backend/SweepFixtures.scala index bee89a32c..8d2256b47 100644 --- a/tools/src/test/scala/orca/backend/SweepFixtures.scala +++ b/tools/src/test/scala/orca/backend/SweepFixtures.scala @@ -1,7 +1,9 @@ package orca.backend +import orca.testkit.Effects + import orca.events.{OrcaEvent, OrcaListener} -import orca.subprocess.{OsProcCliRunner, PipedCliProcess} +import orca.subprocess.{CliRunner, CookieScan, OsProcCliRunner, PipedCliProcess} import java.util.concurrent.atomic.AtomicReference @@ -15,11 +17,13 @@ trait SweepFixtures: protected def onLinux( name: String )(body: => Any)(using munit.Location): Unit = - if EnvCookieSweep.supported then test(name)(body) + if CookieScan.supported then test(name)(body) else test(name.ignore)(body) + protected val runner: CliRunner = OsProcCliRunner(Effects.procs) + protected def spawn(script: String): PipedCliProcess = - OsProcCliRunner.spawnPiped( + runner.spawnPiped( Seq("bash", "-c", script), env = Map.empty, cwd = os.pwd diff --git a/tools/src/test/scala/orca/backend/mcp/AskUserMcpServerTest.scala b/tools/src/test/scala/orca/backend/mcp/AskUserMcpServerTest.scala index aa1803a24..8731e367a 100644 --- a/tools/src/test/scala/orca/backend/mcp/AskUserMcpServerTest.scala +++ b/tools/src/test/scala/orca/backend/mcp/AskUserMcpServerTest.scala @@ -1,5 +1,6 @@ package orca.backend.mcp +import orca.effect.Network import ox.{forkDiscard, supervised} import ox.channels.BufferCapacity @@ -17,7 +18,7 @@ class AskUserMcpServerTest extends munit.FunSuite: supervised: given BufferCapacity = BufferCapacity(8) val bridge = new AskUserBridge - val server = AskUserMcpServer.start(bridge) + val server = AskUserMcpServer.start(Network.ambient, bridge) val resp = post( server.url, @@ -38,7 +39,7 @@ class AskUserMcpServerTest extends munit.FunSuite: .start: () => supervised: given BufferCapacity = BufferCapacity(8) - val session = AskUserSession.allocate() + val session = AskUserSession.allocate(Network.ambient) forkDiscard: post( session.server.url, diff --git a/tools/src/test/scala/orca/backend/mcp/McpHostTest.scala b/tools/src/test/scala/orca/backend/mcp/McpHostTest.scala index 7025aee6a..9f0e3f0c5 100644 --- a/tools/src/test/scala/orca/backend/mcp/McpHostTest.scala +++ b/tools/src/test/scala/orca/backend/mcp/McpHostTest.scala @@ -3,6 +3,10 @@ package orca.backend.mcp import chimp.protocol.ToolContent import chimp.server.{NoStructuredOutput, ServerContext, ToolResult, tool} import io.circe.Codec +import io.circe.parser.parse +import orca.effect.Network +import orca.json.{JsonData, SchemaDerivation} +import orca.mcp import ox.supervised import sttp.shared.Identity import sttp.tapir.Schema @@ -11,7 +15,12 @@ import java.net.URI import java.net.http.{HttpClient, HttpRequest, HttpResponse} import scala.concurrent.duration.DurationInt -private case class Probe(text: String) derives Codec, Schema +private case class Probe(text: String) derives Codec + +private object Probe: + given Schema[Probe] = SchemaDerivation.derived + +private case class Count(n: Int) derives JsonData class McpHostTest extends munit.FunSuite: @@ -67,7 +76,7 @@ class McpHostTest extends munit.FunSuite: tool("probe") .input[Probe] .handle(_ => throw new RuntimeException(message)) - val host = McpHost.start(List(throwing), 10.seconds) + val host = McpHost.start(Network.ambient, List(throwing), 10.seconds) val response = call( host.url, """{"jsonrpc":"2.0","id":1,"method":"tools/call",""" + @@ -82,6 +91,25 @@ class McpHostTest extends munit.FunSuite: response.body().takeRight(200) ) + test("input that fails to decode gets a one-line JSON-RPC error"): + // chimp answers a decode failure with an `Invalid arguments` error, not a + // tool result; without the facade's decoder its message would carry + // jsoniter's multi-line buffer dump. + supervised: + val counting = mcp.tool[Count]("count", "")(c => mcp.text(c.n.toString)) + val host = McpHost.start(Network.ambient, List(counting), 10.seconds) + val response = call( + host.url, + """{"jsonrpc":"2.0","id":1,"method":"tools/call",""" + + """"params":{"name":"count","arguments":{"n":"7"}}}""" + ) + assertEquals(response.statusCode(), 200, response.body()) + val error = parse(response.body()).toOption.get.hcursor.downField("error") + assertEquals(error.get[Int]("code"), Right(-32602), response.body()) + val message = error.get[String]("message").toOption.get + assert(message.contains("illegal number"), message) + assert(!message.contains('\n'), message) + /** POST one JSON-RPC message to a running host. The client is closed rather * than left to a finalizer: its idle keep-alive connection otherwise holds * the binding open, and `stop()` waits ten seconds for it. diff --git a/tools/src/test/scala/orca/backend/mcp/McpInputSchemaParityTest.scala b/tools/src/test/scala/orca/backend/mcp/McpInputSchemaParityTest.scala new file mode 100644 index 000000000..ddbef17dd --- /dev/null +++ b/tools/src/test/scala/orca/backend/mcp/McpInputSchemaParityTest.scala @@ -0,0 +1,16 @@ +package orca.backend.mcp + +import orca.json.given + +import scala.annotation.experimental + +// tapir's Magnolia derivation copies the annotations of these safe-mode +// classes, `@assumeSafe` among them, which only experimental code may use. +// Not `-experimental` on tools' tests: it marks every test definition +// experimental, which the backend modules' tests reuse. +@experimental +class McpInputSchemaParityTest extends orca.json.SchemaParitySuite: + parity[AskUserInput] + parity[GitHubIssueInput] + parity[GitShowInput] + parity[GitFileAtInput] diff --git a/tools/src/test/scala/orca/backend/mcp/RepoMcpServerTest.scala b/tools/src/test/scala/orca/backend/mcp/RepoMcpServerTest.scala index 4ba53646c..71063d56a 100644 --- a/tools/src/test/scala/orca/backend/mcp/RepoMcpServerTest.scala +++ b/tools/src/test/scala/orca/backend/mcp/RepoMcpServerTest.scala @@ -1,16 +1,21 @@ package orca.backend.mcp -import io.circe.parser.decode +import com.github.plokhotnyuk.jsoniter_scala.core.readFromString +import orca.json.JsonData + +import scala.util.Try class RepoMcpServerTest extends munit.FunSuite: test("git_show decodes a call that omits the optional arguments"): // The tool's description and the system-prompt hint both say `paths` and - // `stat` are optional, so the agent omits them. Scala default values do not - // reach the derived circe codec, which rejects a missing non-Option field — - // measured live as `DecodingFailure at .paths: Missing required field`, - // leaving the agent with one working git tool instead of two. + // `stat` are optional, so the agent omits them; failing to decode such a + // call leaves the agent with one working git tool instead of two. // Asserted on the outcome rather than the constructor shape, so restoring // defaulted fields fails this test rather than failing to compile it. - val decoded = decode[GitShowInput]("""{"rev":"HEAD"}""") - assert(decoded.isRight, decoded) + val decoded = Try( + readFromString[GitShowInput]("""{"rev":"HEAD"}""")(using + summon[JsonData[GitShowInput]].codec + ) + ) + assert(decoded.isSuccess, decoded) diff --git a/tools/src/test/scala/orca/json/JsonFileTest.scala b/tools/src/test/scala/orca/json/JsonFileTest.scala index 6847594bc..af5e67e3c 100644 --- a/tools/src/test/scala/orca/json/JsonFileTest.scala +++ b/tools/src/test/scala/orca/json/JsonFileTest.scala @@ -3,8 +3,8 @@ package orca.json import com.github.plokhotnyuk.jsoniter_scala.core.JsonValueCodec import com.github.plokhotnyuk.jsoniter_scala.macros.JsonCodecMaker import munit.FunSuite -import orca.{OrcaDir, RunKey} -import orca.testkit.TempDirs +import orca.RunKey +import orca.testkit.{Effects, TempDirs} private case class Doc(v: Int) private given JsonValueCodec[Doc] = JsonCodecMaker.make @@ -15,30 +15,33 @@ class JsonFileTest extends FunSuite: private def path(): os.Path = TempDirs.dir() / "dir" / "doc.json" test("a missing file is Absent"): - assertEquals(JsonFile.read[Doc](path()), JsonFile.Read.Absent) + assertEquals(Effects.jsonFile.read[Doc](path()), JsonFile.Read.Absent) test("write then read is Loaded"): - val file = OrcaDir.progressFile(TempDirs.dir(), RunKey.of("p")) + val file = Effects.orcaDir.progressFile(TempDirs.dir(), RunKey.of("p")) JsonFile.write(file, Doc(1)) - assertEquals(JsonFile.read[Doc](file.path), JsonFile.Read.Loaded(Doc(1))) + assertEquals( + Effects.jsonFile.read[Doc](file.path), + JsonFile.Read.Loaded(Doc(1)) + ) test("a file that does not parse is Corrupt, with a reason"): val p = path() os.write(p, "not json {{{", createFolders = true) - JsonFile.read[Doc](p) match + Effects.jsonFile.read[Doc](p) match case JsonFile.Read.Corrupt(reason) => assert(reason.nonEmpty) case other => fail(s"expected Corrupt, got $other") test("well-formed JSON the codec rejects is Corrupt"): val p = path() os.write(p, """{"w":1}""", createFolders = true) - JsonFile.read[Doc](p) match + Effects.jsonFile.read[Doc](p) match case JsonFile.Read.Corrupt(_) => () case other => fail(s"expected Corrupt, got $other") test("a directory at the path is Unreadable, not Corrupt"): val p = path() os.makeDir.all(p) - JsonFile.read[Doc](p) match + Effects.jsonFile.read[Doc](p) match case JsonFile.Read.Unreadable(reason) => assert(reason.nonEmpty) case other => fail(s"expected Unreadable, got $other") diff --git a/tools/src/test/scala/orca/subprocess/FakePipedCliProcess.scala b/tools/src/test/scala/orca/subprocess/FakePipedCliProcess.scala index ebd58e5e7..81c18500c 100644 --- a/tools/src/test/scala/orca/subprocess/FakePipedCliProcess.scala +++ b/tools/src/test/scala/orca/subprocess/FakePipedCliProcess.scala @@ -52,7 +52,7 @@ class FakePipedCliProcess( /** No OS process, so nothing to sweep for — and a cookie here would make * every fake-driven turn pay for a real `/proc` scan. */ - def envCookie: Option[EnvCookie] = None + def survivors(): List[ProcessHandle] = Nil // The OS-backed process throws from the flush that follows every write, so // the fake rejects a late write too. diff --git a/tools/src/test/scala/orca/subprocess/OsProcCliRunnerTest.scala b/tools/src/test/scala/orca/subprocess/OsProcCliRunnerTest.scala index 5901e03f3..abf0189f4 100644 --- a/tools/src/test/scala/orca/subprocess/OsProcCliRunnerTest.scala +++ b/tools/src/test/scala/orca/subprocess/OsProcCliRunnerTest.scala @@ -1,7 +1,7 @@ package orca.subprocess import orca.testkit.ProcessProbe.{alive, awaitDead, awaitTrue} -import orca.testkit.TempDirs +import orca.testkit.{Effects, TempDirs} import ox.discard class OsProcCliRunnerTest extends munit.FunSuite: @@ -17,7 +17,7 @@ class OsProcCliRunnerTest extends munit.FunSuite: private def withBackgroundedChild( body: (PipedCliProcess, Long) => Unit ): Unit = - val proc = OsProcCliRunner.spawnPiped( + val proc = OsProcCliRunner(Effects.procs).spawnPiped( Seq("bash", "-c", "sleep 30 & echo $!; wait"), env = Map.empty, cwd = os.pwd @@ -84,7 +84,7 @@ class OsProcCliRunnerTest extends munit.FunSuite: val script = """bash -c 'while [ ! -e "$TRIGGER" ]; do sleep 0.02; done""" + """; sleep 30 & echo $! > "$PIDFILE"; wait' & echo $!; wait""" - val proc = OsProcCliRunner.spawnPiped( + val proc = OsProcCliRunner(Effects.procs).spawnPiped( Seq("bash", "-c", script), env = Map("TRIGGER" -> trigger.toString, "PIDFILE" -> pidFile.toString), cwd = os.pwd @@ -120,7 +120,7 @@ class OsProcCliRunnerTest extends munit.FunSuite: * empty stream, not a failure. */ test("destroyForciblyTree on an already-exited process is a no-op"): - val proc = OsProcCliRunner.spawnPiped( + val proc = OsProcCliRunner(Effects.procs).spawnPiped( Seq("bash", "-c", "exit 0"), env = Map.empty, cwd = os.pwd @@ -142,21 +142,20 @@ class OsProcCliRunnerTest extends munit.FunSuite: test("spawnPiped adds its cookie to the inherited environment"): val home = sys.env.getOrElse("HOME", "") assume(home.nonEmpty, "needs an inherited variable to check against") - val proc = OsProcCliRunner.spawnPiped( - Seq("bash", "-c", s"""echo "$$${EnvCookie.VarName}|$$HOME|$$EXTRA""""), + val proc = OsProcCliRunner(Effects.procs).spawnPiped( + Seq( + "bash", + "-c", + s"""echo "$${${EnvCookie.VarName}:+set}|$$HOME|$$EXTRA"""" + ), env = Map("EXTRA" -> "from-caller"), cwd = os.pwd ) - try - val cookie = proc.envCookie.getOrElse(fail("no cookie was injected")) - assertEquals( - proc.stdoutLines.next(), - s"${cookie.value}|$home|from-caller" - ) + try assertEquals(proc.stdoutLines.next(), s"set|$home|from-caller") finally proc.destroyForciblyTree() test("spawnPiped hands the child's stderr to stderrLines"): - val proc = OsProcCliRunner.spawnPiped( + val proc = OsProcCliRunner(Effects.procs).spawnPiped( Seq("bash", "-c", "echo oops >&2"), cwd = os.pwd ) @@ -168,7 +167,7 @@ class OsProcCliRunnerTest extends munit.FunSuite: */ test("output a process sends after a long silence is still read"): val silence = (OsPipedSubProcess.OrphanedPipeGrace * 2).toMillis / 1000.0 - val proc = OsProcCliRunner.spawnPiped( + val proc = OsProcCliRunner(Effects.procs).spawnPiped( Seq("bash", "-c", s"sleep $silence; echo late"), cwd = os.pwd ) diff --git a/tools/src/test/scala/orca/subprocess/PathProbeTest.scala b/tools/src/test/scala/orca/subprocess/PathProbeTest.scala index d07381687..8c591c901 100644 --- a/tools/src/test/scala/orca/subprocess/PathProbeTest.scala +++ b/tools/src/test/scala/orca/subprocess/PathProbeTest.scala @@ -1,14 +1,15 @@ package orca.subprocess -import orca.testkit.TempDirs +import orca.testkit.{Effects, TempDirs} class PathProbeTest extends munit.FunSuite: + private val procs = Effects.procs test("resolves: a word on PATH resolves"): - assert(PathProbe.resolves("bash", os.pwd)) + assert(PathProbe.resolves(procs, "bash", os.pwd)) test("resolves: an unknown word does not resolve"): - assert(!PathProbe.resolves("definitely-not-a-binary-xyz", os.pwd)) + assert(!PathProbe.resolves(procs, "definitely-not-a-binary-xyz", os.pwd)) test( "resolves: shell metacharacters in the word are never interpreted — no side effects" @@ -17,6 +18,6 @@ class PathProbeTest extends munit.FunSuite: // into the script text: this must not create `marker`, whatever the // resolution verdict on the (unresolvable) word is. val dir = TempDirs.dir() - val result = PathProbe.resolves("echo; touch marker", dir) + val result = PathProbe.resolves(procs, "echo; touch marker", dir) assert(!result) assert(!os.exists(dir / "marker")) diff --git a/tools/src/test/scala/orca/testkit/Effects.scala b/tools/src/test/scala/orca/testkit/Effects.scala new file mode 100644 index 000000000..bce151d1f --- /dev/null +++ b/tools/src/test/scala/orca/testkit/Effects.scala @@ -0,0 +1,22 @@ +package orca.testkit + +import orca.OrcaDir +import orca.effect.{FileSystem, Processes} +import orca.events.OrcaListener +import orca.json.JsonFile +import orca.tools.{OsGitTool, Worktrees} + +/** Ambient capabilities, and the effectful helpers built from them, for tests. + */ +object Effects: + val fs: FileSystem = FileSystem.ambient + val procs: Processes = Processes.ambient + val orcaDir: OrcaDir = OrcaDir(fs) + val jsonFile: JsonFile = JsonFile(fs) + val worktrees: Worktrees = Worktrees(procs, fs) + + def gitTool( + workDir: os.Path, + events: OrcaListener = OrcaListener.noop + ): OsGitTool = + OsGitTool(workDir, procs, fs, events) diff --git a/tools/src/test/scala/orca/tools/OsFsToolTest.scala b/tools/src/test/scala/orca/tools/OsFsToolTest.scala index 2b0d10b3d..c13a9f91a 100644 --- a/tools/src/test/scala/orca/tools/OsFsToolTest.scala +++ b/tools/src/test/scala/orca/tools/OsFsToolTest.scala @@ -1,7 +1,7 @@ package orca.tools import orca.{OrcaFlowException, WorkspaceWrite} -import orca.testkit.TempDirs +import orca.testkit.{Effects, TempDirs} import ox.discard class OsFsToolTest extends munit.FunSuite: @@ -12,7 +12,7 @@ class OsFsToolTest extends munit.FunSuite: private def withFs(body: (OsFsTool, os.Path) => Unit): Unit = val tmp = TempDirs.dir() - body(new OsFsTool(base = tmp), tmp) + body(new OsFsTool(tmp, Effects.fs), tmp) test("write creates missing parent directories and read round-trips content"): withFs: (fs, tmp) => diff --git a/tools/src/test/scala/orca/tools/OsGitHubIntegrationTest.scala b/tools/src/test/scala/orca/tools/OsGitHubIntegrationTest.scala index 53d02f025..e3c70d642 100644 --- a/tools/src/test/scala/orca/tools/OsGitHubIntegrationTest.scala +++ b/tools/src/test/scala/orca/tools/OsGitHubIntegrationTest.scala @@ -1,7 +1,7 @@ package orca.tools import orca.subprocess.OsProcCliRunner -import orca.testkit.prHandle +import orca.testkit.{Effects, prHandle} /** End-to-end tests against the real `gh` CLI. Gated on the `ORCA_INTEGRATION` * environment variable and require `gh auth login` on the host. The tests read @@ -14,7 +14,8 @@ class OsGitHubIntegrationTest extends munit.FunSuite: else Nil test("gh CLI is installed and invokable via the CliRunner"): - val result = OsProcCliRunner.run(Seq("gh", "--version"), cwd = os.pwd) + val result = OsProcCliRunner(Effects.procs) + .run(Seq("gh", "--version"), cwd = os.pwd) assertEquals(result.exitCode, 0) assert( result.stdout.toLowerCase.contains("gh version"), @@ -22,7 +23,8 @@ class OsGitHubIntegrationTest extends munit.FunSuite: ) test("gh api user succeeds under the authenticated session"): - val result = OsProcCliRunner.run(Seq("gh", "api", "user"), cwd = os.pwd) + val result = OsProcCliRunner(Effects.procs) + .run(Seq("gh", "api", "user"), cwd = os.pwd) assertEquals( result.exitCode, 0, @@ -33,7 +35,7 @@ class OsGitHubIntegrationTest extends munit.FunSuite: test("readPrComments succeeds against a public issue-or-PR endpoint"): // GitHub's /issues/{n}/comments endpoint is shared between issues and PRs; // Hello-World #1 is stable public data. - val gh = new OsGitHubTool(OsProcCliRunner) + val gh = new OsGitHubTool(OsProcCliRunner(Effects.procs), os.pwd) val handle = prHandle("https://github.com/octocat/Hello-World/pull/1") val comments = gh.readPrComments(handle) assert(comments.forall(_.author.nonEmpty)) diff --git a/tools/src/test/scala/orca/tools/OsGitHubToolTest.scala b/tools/src/test/scala/orca/tools/OsGitHubToolTest.scala index 5705877b7..050cb46d3 100644 --- a/tools/src/test/scala/orca/tools/OsGitHubToolTest.scala +++ b/tools/src/test/scala/orca/tools/OsGitHubToolTest.scala @@ -26,7 +26,7 @@ class OsGitHubToolTest extends munit.FunSuite: private def stubGh(response: CliResult): (StubCliRunner, OsGitHubTool) = val cli = new StubCliRunner(response) - (cli, new OsGitHubTool(cli, pollInterval = 10.millis)) + (cli, new OsGitHubTool(cli, os.pwd, pollInterval = 10.millis)) private class CapturingListener extends OrcaListener: private val seen = new ConcurrentLinkedQueue[OrcaEvent]() @@ -97,11 +97,11 @@ class OsGitHubToolTest extends munit.FunSuite: /** A tool whose probe never retries, so a failing gh answers at once. */ private def probeGh(cli: CliRunner): OsGitHubTool = - new OsGitHubTool(cli, readRetry = Schedule.immediate.maxRetries(0)) + new OsGitHubTool(cli, os.pwd, readRetry = Schedule.immediate.maxRetries(0)) /** A tool whose probe retries twice, with no delay. */ private def retryingProbeGh(cli: CliRunner): OsGitHubTool = - new OsGitHubTool(cli, readRetry = Schedule.immediate.maxRetries(2)) + new OsGitHubTool(cli, os.pwd, readRetry = Schedule.immediate.maxRetries(2)) test("availability retries a repo view that fails once on the network"): val cli = new SequencedCliRunner( @@ -390,7 +390,7 @@ class OsGitHubToolTest extends munit.FunSuite: val cli = createPrRunner( CliResult(0, "https://github.com/acme/widgets/pull/42\n", "") ) - val gh = new OsGitHubTool(cli) + val gh = new OsGitHubTool(cli, os.pwd) val _ = gh.createPr("feat: hi", "hello").orThrow // gh must be given the branch symbolic-ref resolved, so pin the order. assertEquals( @@ -408,7 +408,7 @@ class OsGitHubToolTest extends munit.FunSuite: val cli = createPrRunner( CliResult(0, "https://github.com/acme/widgets/pull/42\n", "") ) - val _ = new OsGitHubTool(cli) + val _ = new OsGitHubTool(cli, os.pwd) .createPr("feat: hi", "hello", readiness) .orThrow cli.calls.last.args @@ -425,7 +425,7 @@ class OsGitHubToolTest extends munit.FunSuite: "" ) ) - val pr = new OsGitHubTool(cli) + val pr = new OsGitHubTool(cli, os.pwd) .createPr("feat: hi", "hello", Some(PrReadiness.Draft)) .orThrow assertEquals(pr, samplePr) @@ -443,14 +443,14 @@ class OsGitHubToolTest extends munit.FunSuite: val cli = createPrRunner( CliResult(0, "https://github.com/acme/widgets/pull/42\n", "") ) - val gh = new OsGitHubTool(cli) + val gh = new OsGitHubTool(cli, os.pwd) assertEquals(gh.createPr("feat: hi", "hello").orThrow, samplePr) test("createPr keeps the host of a GitHub Enterprise PR URL"): val cli = createPrRunner( CliResult(0, "https://ghe.example.com/acme/widgets/pull/42\n", "") ) - val gh = new OsGitHubTool(cli) + val gh = new OsGitHubTool(cli, os.pwd) assertEquals( gh.createPr("feat: hi", "hello").orThrow, prHandle("https://ghe.example.com/acme/widgets/pull/42") @@ -461,7 +461,7 @@ class OsGitHubToolTest extends munit.FunSuite: val cli = createPrRunner( CliResult(0, "https://github.com/acme/widgets/pull/42\n", "") ) - val gh = new OsGitHubTool(cli, events = listener) + val gh = new OsGitHubTool(cli, os.pwd, events = listener) val _ = gh.createPr("feat: hi", "hello").orThrow assertEquals( listener.events, @@ -469,7 +469,8 @@ class OsGitHubToolTest extends munit.FunSuite: ) test("createPr throws when gh output does not contain a PR URL"): - val gh = new OsGitHubTool(createPrRunner(CliResult(0, "no url here", ""))) + val gh = + new OsGitHubTool(createPrRunner(CliResult(0, "no url here", "")), os.pwd) val _ = intercept[OrcaFlowException](gh.createPr("t", "b")) test( @@ -481,7 +482,7 @@ class OsGitHubToolTest extends munit.FunSuite: CliResult(1, "", "a pull request for branch 'feat' already exists"), CliResult(0, "[]", "") // gh pr list — no open PR ) - val gh = new OsGitHubTool(cli, readRetry = Schedule.immediate) + val gh = new OsGitHubTool(cli, os.pwd, readRetry = Schedule.immediate) assert(gh.createPr("t", "b").left.exists(_.isInstanceOf[PrAlreadyExists])) test( @@ -494,7 +495,8 @@ class OsGitHubToolTest extends munit.FunSuite: "", "pull request create failed: No commits between main and feat" ) - ) + ), + os.pwd ) assert(gh.createPr("t", "b").left.exists(_.isInstanceOf[NoCommitsToPr])) @@ -511,14 +513,15 @@ class OsGitHubToolTest extends munit.FunSuite: "GraphQL: Head sha can't be blank, No commits between main and feat, " + "Head ref must be a branch (createPullRequest)" ) - ) + ), + os.pwd ) assert(gh.createPr("t", "b").left.exists(_.isInstanceOf[BranchNotPushed])) test("createPr throws when not on a branch (detached HEAD)"): // `symbolic-ref --quiet` exits 1, printing nothing, when HEAD is detached. val cli = new SequencedCliRunner(List(CliResult(1, "", ""))) - val gh = new OsGitHubTool(cli) + val gh = new OsGitHubTool(cli, os.pwd) val e = intercept[OrcaFlowException](gh.createPr("t", "b")) assert(e.getMessage.contains("not on a branch"), e.getMessage) @@ -751,7 +754,7 @@ class OsGitHubToolTest extends munit.FunSuite: val successJson = """{"statusCheckRollup":[{"status":"COMPLETED","conclusion":"SUCCESS","name":"t"}]}""" val cli = new StubCliRunner(CliResult(0, pendingJson, "")) - val gh = new OsGitHubTool(cli, pollInterval = 10.millis) + val gh = new OsGitHubTool(cli, os.pwd, pollInterval = 10.millis) // Flip the stub to success after a couple of polls. val watcher = new Thread(() => @@ -772,7 +775,11 @@ class OsGitHubToolTest extends munit.FunSuite: CliResult(0, issueJson, "") ) ) - val gh = new OsGitHubTool(cli, readRetry = Schedule.immediate.maxRetries(2)) + val gh = new OsGitHubTool( + cli, + os.pwd, + readRetry = Schedule.immediate.maxRetries(2) + ) assertEquals(gh.readIssue(IssueHandle("a", "b", 1)).title, "t") assertEquals(cli.callCount, 2) // one failed attempt, then success @@ -791,6 +798,7 @@ class OsGitHubToolTest extends munit.FunSuite: ) val gh = new OsGitHubTool( cli, + os.pwd, pollInterval = 1.milli, readRetry = Schedule.immediate.maxRetries(3) ) @@ -806,6 +814,7 @@ class OsGitHubToolTest extends munit.FunSuite: ) val gh = new OsGitHubTool( cli, + os.pwd, pollInterval = 1.milli, readRetry = Schedule.immediate.maxRetries(2) ) @@ -825,7 +834,7 @@ class OsGitHubToolTest extends munit.FunSuite: val pendingJson = """{"statusCheckRollup":[{"status":"IN_PROGRESS","name":"t"}]}""" val cli = new StubCliRunner(CliResult(0, pendingJson, "")) - val gh = new OsGitHubTool(cli, pollInterval = 5.millis) + val gh = new OsGitHubTool(cli, os.pwd, pollInterval = 5.millis) val watcher = new Thread(() => // Wait past the grace window, then flip to an empty rollup. The // sticky `sawAnyCheck` should prevent NoChecksConfigured. @@ -862,7 +871,7 @@ class OsGitHubToolTest extends munit.FunSuite: CliResult(0, pendingJson, "") ) ) - val gh = new OsGitHubTool(cli, pollInterval = 1.millis) + val gh = new OsGitHubTool(cli, os.pwd, pollInterval = 1.millis) val result = gh.waitForBuild( samplePr, timeout = 200.millis, @@ -878,7 +887,7 @@ class OsGitHubToolTest extends munit.FunSuite: // fast-path fired — it's only reachable via the grace branch. val cli = new StubCliRunner(CliResult(0, """{"statusCheckRollup":[]}""", "")) - val gh = new OsGitHubTool(cli, pollInterval = 10.millis) + val gh = new OsGitHubTool(cli, os.pwd, pollInterval = 10.millis) val result = gh.waitForBuild( samplePr, timeout = 5.seconds, @@ -913,6 +922,7 @@ class OsGitHubToolTest extends munit.FunSuite: ) val gh = new OsGitHubTool( cli, + os.pwd, events = listener, readRetry = Schedule.immediate ) @@ -940,7 +950,7 @@ class OsGitHubToolTest extends munit.FunSuite: CliResult(1, "", "a pull request for branch 'feat' already exists"), CliResult(0, prListJson, "") ) - val gh = new OsGitHubTool(cli, readRetry = Schedule.immediate) + val gh = new OsGitHubTool(cli, os.pwd, readRetry = Schedule.immediate) assertEquals( gh.createPr("feat: hi", "hello").orThrow, prHandle("https://ghe.example.com/acme/widgets/pull/42") @@ -953,7 +963,7 @@ class OsGitHubToolTest extends munit.FunSuite: val cli = createPrRunner( CliResult(0, "https://github.com/acme/widgets/pull/42\n", "") ) - val gh = new OsGitHubTool(cli) + val gh = new OsGitHubTool(cli, os.pwd) val _ = gh.createPr("feat: hi", "hello").orThrow val symbolicRefCall = cli.calls.headOption.getOrElse(fail("expected a git symbolic-ref call")) @@ -978,7 +988,7 @@ class OsGitHubToolTest extends munit.FunSuite: CliResult(0, "", "") // writeComment create ) ) - val gh = new OsGitHubTool(cli, readRetry = Schedule.immediate) + val gh = new OsGitHubTool(cli, os.pwd, readRetry = Schedule.immediate) gh.upsertComment(samplePr, "", "Rejected.") // The create call must embed the marker in the body val createCall = cli.calls.last @@ -997,7 +1007,7 @@ class OsGitHubToolTest extends munit.FunSuite: CliResult(0, "", "") // PATCH update ) ) - val gh = new OsGitHubTool(cli, readRetry = Schedule.immediate) + val gh = new OsGitHubTool(cli, os.pwd, readRetry = Schedule.immediate) gh.upsertComment(samplePr, "", "New text.") // Exactly fetch + PATCH = 2 calls; no comment-create was made assertEquals(cli.callCount, 2, "expected exactly fetch + PATCH, not more") @@ -1025,7 +1035,7 @@ class OsGitHubToolTest extends munit.FunSuite: CliResult(0, "", "") // writeComment create ) ) - val gh = new OsGitHubTool(cli, readRetry = Schedule.immediate) + val gh = new OsGitHubTool(cli, os.pwd, readRetry = Schedule.immediate) val issue = IssueHandle("acme", "widgets", 7) gh.upsertComment(issue, "", "Not a bug.") val createCall = cli.calls.last @@ -1044,7 +1054,7 @@ class OsGitHubToolTest extends munit.FunSuite: CliResult(0, "", "") // PATCH update ) ) - val gh = new OsGitHubTool(cli, readRetry = Schedule.immediate) + val gh = new OsGitHubTool(cli, os.pwd, readRetry = Schedule.immediate) val issue = IssueHandle("acme", "widgets", 7) gh.upsertComment(issue, "", "Updated.") // Exactly fetch + PATCH = 2 calls; no comment-create was made diff --git a/tools/src/test/scala/orca/tools/OsGitToolTest.scala b/tools/src/test/scala/orca/tools/OsGitToolTest.scala index a94b167da..f5365b820 100644 --- a/tools/src/test/scala/orca/tools/OsGitToolTest.scala +++ b/tools/src/test/scala/orca/tools/OsGitToolTest.scala @@ -3,7 +3,7 @@ package orca.tools import orca.{OrcaFlowException, WorkspaceWrite} import orca.events.{OrcaEvent, OrcaListener} import orca.gitref.{CommitHash, Head} -import orca.testkit.{GitRepo, branchName} +import orca.testkit.{Effects, GitRepo, branchName} import ox.either.orThrow import java.util.concurrent.atomic.AtomicReference @@ -17,11 +17,11 @@ class OsGitToolTest extends munit.FunSuite: private def withRepo(body: (OsGitTool, os.Path) => Unit): Unit = val dir = GitRepo.empty() - body(new OsGitTool(dir), dir) + body(Effects.gitTool(dir), dir) private def withSeededRepo(body: (OsGitTool, os.Path) => Unit): Unit = val dir = GitRepo.seeded() - body(new OsGitTool(dir), dir) + body(Effects.gitTool(dir), dir) /** True when a read never reached git: the revision failed validation. */ private def rejected(result: Either[GitReadFailed, String]): Boolean = @@ -35,7 +35,7 @@ class OsGitToolTest extends munit.FunSuite: val seen = new AtomicReference[List[OrcaEvent]](Nil) val listener: OrcaListener = (e: OrcaEvent) => val _ = seen.updateAndGet(e :: _) - body(new OsGitTool(dir, listener), dir, seen) + body(Effects.gitTool(dir, listener), dir, seen) test("createBranch switches to the new branch"): withSeededRepo: (git, _) => @@ -88,7 +88,7 @@ class OsGitToolTest extends munit.FunSuite: test("isIgnored is false (not a failure) outside a git repository"): val dir = TempDirs.dir() - assert(!new OsGitTool(dir).isIgnored(os.sub / "whatever.txt")) + assert(!Effects.gitTool(dir).isIgnored(os.sub / "whatever.txt")) test("commitOnly commits exactly the given path, leaving other files out"): withSeededRepo: (git, dir) => @@ -193,7 +193,7 @@ class OsGitToolTest extends munit.FunSuite: test("defaultBranch throws when git cannot answer"): val error = intercept[OrcaFlowException]: - new OsGitTool(TempDirs.dir()).defaultBranch() + Effects.gitTool(TempDirs.dir()).defaultBranch() assert(error.getMessage.contains("git remote set-head"), error.getMessage) test("diffVsBase returns the cumulative branch diff vs base"): @@ -386,7 +386,7 @@ class OsGitToolTest extends munit.FunSuite: val _ = pushToLocalRemote(git, dir) // Paths are cwd-relative, so a tool rooted at `nested` must see its own // `f.txt` — not miss it by looking for `f.txt` at the repo root. - val nestedGit = new OsGitTool(dir / "nested") + val nestedGit = Effects.gitTool(dir / "nested") assert(nestedGit.upstreamHas(dir / "nested" / "f.txt")) test("nonInteractiveEnv disables git and ssh interactive prompts"): @@ -635,10 +635,10 @@ class OsGitToolTest extends munit.FunSuite: os.makeDir.all(dir / "sub") os.write(dir / "top.txt", "one\n") os.write(dir / "sub" / "inner.txt", "one\n") - val root = new OsGitTool(dir) + val root = Effects.gitTool(dir) root.commit("seed").orThrow os.write.over(dir / "top.txt", "two\n") - val inSub = new OsGitTool(dir / "sub") + val inSub = Effects.gitTool(dir / "sub") assert( inSub.uncommittedDiff().contains("top.txt"), inSub.uncommittedDiff() @@ -652,10 +652,10 @@ class OsGitToolTest extends munit.FunSuite: withRepo: (_, dir) => os.makeDir.all(dir / "sub") os.write(dir / "sub" / "seed.txt", "seed") - val root = new OsGitTool(dir) + val root = Effects.gitTool(dir) root.commit("seed").orThrow os.write(dir / "sub" / "new.txt", "hello") - val inSub = new OsGitTool(dir / "sub") + val inSub = Effects.gitTool(dir / "sub") assertEquals(inSub.pendingChanges().newFiles, List("new.txt")) val diff = inSub.reviewChanges().diff assert(diff.contains("+hello"), diff) @@ -846,7 +846,7 @@ class OsGitToolTest extends munit.FunSuite: git.commit("seed").orThrow os.write.over(dir / "sub" / "inner.txt", "two\n") val section = - new OsGitTool(dir / "sub").reviewChanges().sections.get("inner.txt") + Effects.gitTool(dir / "sub").reviewChanges().sections.get("inner.txt") assert(section.exists(_.contains("+two")), section) test("reviewChanges gives a file replaced by a symlink one section"): @@ -906,11 +906,11 @@ class OsGitToolTest extends munit.FunSuite: os.makeDir.all(dir / "sub") os.write(dir / "top.txt", "one") os.write(dir / "sub" / "inner.txt", "one") - new OsGitTool(dir).commit("seed").orThrow + Effects.gitTool(dir).commit("seed").orThrow os.write.over(dir / "top.txt", "two") val _ = os.proc("git", "config", "diff.relative", "true").call(cwd = dir) assertEquals( - new OsGitTool(dir / "sub").changedFiles(), + Effects.gitTool(dir / "sub").changedFiles(), List("../top.txt") ) diff --git a/tools/src/test/scala/orca/tools/WorktreesTest.scala b/tools/src/test/scala/orca/tools/WorktreesTest.scala index a84ed6e57..58dddea9a 100644 --- a/tools/src/test/scala/orca/tools/WorktreesTest.scala +++ b/tools/src/test/scala/orca/tools/WorktreesTest.scala @@ -1,7 +1,6 @@ package orca.tools -import orca.OrcaDir -import orca.testkit.{GitRepo, TempDirs, branchName} +import orca.testkit.{Effects, GitRepo, TempDirs, branchName} import ox.discard class WorktreesTest extends munit.FunSuite: @@ -9,8 +8,8 @@ class WorktreesTest extends munit.FunSuite: test("add creates a worktree that list then reports"): val repo = GitRepo.seeded() val path = repo / "wt" - assertEquals(Worktrees.add(repo, path), Right(())) - assertEquals(Worktrees.list(repo), List(repo, path)) + assertEquals(Effects.worktrees.add(repo, path), Right(())) + assertEquals(Effects.worktrees.list(repo), List(repo, path)) test("add starts the worktree detached at the invoking checkout's HEAD"): val repo = GitRepo.seeded() @@ -19,7 +18,7 @@ class WorktreesTest extends munit.FunSuite: git(repo, "add", "-A").discard git(repo, "commit", "-q", "-m", "feature").discard val path = repo / "wt" - assertEquals(Worktrees.add(repo, path), Right(())) + assertEquals(Effects.worktrees.add(repo, path), Right(())) // The commit only 'feature' has: a worktree started from the default // branch, or from anything but cwd's HEAD, would not have it. assertEquals(head(path), head(repo)) @@ -27,37 +26,37 @@ class WorktreesTest extends munit.FunSuite: test("the .orca/worktrees marker keeps a worktree out of git add -A"): val repo = GitRepo.seeded() - val path = OrcaDir.ensureWorktrees(repo) / "run" - assertEquals(Worktrees.add(repo, path), Right(())) + val path = Effects.orcaDir.ensureWorktrees(repo) / "run" + assertEquals(Effects.worktrees.add(repo, path), Right(())) git(repo, "add", "-A").discard assertEquals(gitOut(repo, "status", "--porcelain"), "") test("mainCheckout answers the same path from the repo and its worktree"): val repo = GitRepo.seeded() val path = repo / "wt" - assertEquals(Worktrees.add(repo, path), Right(())) - assertEquals(Worktrees.mainCheckout(repo), Some(repo)) - assertEquals(Worktrees.mainCheckout(path), Some(repo)) + assertEquals(Effects.worktrees.add(repo, path), Right(())) + assertEquals(Effects.worktrees.mainCheckout(repo), Some(repo)) + assertEquals(Effects.worktrees.mainCheckout(path), Some(repo)) test("mainCheckout answers the checkout when the git dir lives outside it"): val checkout = GitRepo.seededSeparateGitDir() - assertEquals(Worktrees.mainCheckout(checkout), Some(checkout)) + assertEquals(Effects.worktrees.mainCheckout(checkout), Some(checkout)) test("mainCheckout is empty outside a git repository"): - assertEquals(Worktrees.mainCheckout(TempDirs.dir()), None) + assertEquals(Effects.worktrees.mainCheckout(TempDirs.dir()), None) test("add reports an unborn HEAD as its own failure"): val repo = GitRepo.empty() assertEquals( - Worktrees.add(repo, repo / "wt"), + Effects.worktrees.add(repo, repo / "wt"), Left(WorktreeAddFailure.NoCommitsYet) ) test("add carries git's message for every other failure"): val repo = GitRepo.seeded() val path = repo / "wt" - assertEquals(Worktrees.add(repo, path), Right(())) - Worktrees.add(repo, path) match + assertEquals(Effects.worktrees.add(repo, path), Right(())) + Effects.worktrees.add(repo, path) match case Left(WorktreeAddFailure.GitFailed(message)) => assert(message.nonEmpty, "git's explanation must survive") case other => fail(s"expected a git failure, got $other") @@ -73,9 +72,9 @@ class WorktreesTest extends munit.FunSuite: test("startBranch refuses to move a branch holding commits HEAD lacks"): val repo = GitRepo.seeded() val path = repo / "wt" - assertEquals(Worktrees.add(repo, path), Right(())) + assertEquals(Effects.worktrees.add(repo, path), Right(())) assertEquals( - Worktrees.startBranch(path, branchName("wt-branch")), + Effects.worktrees.startBranch(path, branchName("wt-branch")), Right(()) ) os.write(path / "work.txt", "work") @@ -83,9 +82,9 @@ class WorktreesTest extends munit.FunSuite: git(path, "commit", "-q", "-m", "work in the worktree").discard // Every removal route leaves the branch behind, so a re-run finds it. git(repo, "worktree", "remove", "--force", path.toString).discard - assertEquals(Worktrees.add(repo, path), Right(())) + assertEquals(Effects.worktrees.add(repo, path), Right(())) assertEquals( - Worktrees.startBranch(path, branchName("wt-branch")), + Effects.worktrees.startBranch(path, branchName("wt-branch")), Left(StartBranchFailure.WouldLoseCommits(branchName("wt-branch"))) ) // The commit is still reachable from the branch orca refused to move. @@ -97,10 +96,13 @@ class WorktreesTest extends munit.FunSuite: test("startBranch takes a freshly added worktree off its detached HEAD"): val repo = GitRepo.seeded() val path = repo / "wt" - assertEquals(Worktrees.add(repo, path), Right(())) - assert(!Worktrees.onABranch(path), "a fresh worktree starts detached") + assertEquals(Effects.worktrees.add(repo, path), Right(())) + assert( + !Effects.worktrees.onABranch(path), + "a fresh worktree starts detached" + ) assertEquals( - Worktrees.startBranch(path, branchName("wt-branch")), + Effects.worktrees.startBranch(path, branchName("wt-branch")), Right(()) ) assertEquals(gitOut(path, "rev-parse", "--abbrev-ref", "HEAD"), "wt-branch") @@ -108,9 +110,9 @@ class WorktreesTest extends munit.FunSuite: test("a worktree that cannot be read reads as not on a branch"): val repo = GitRepo.seeded() val path = repo / "wt" - assertEquals(Worktrees.add(repo, path), Right(())) + assertEquals(Effects.worktrees.add(repo, path), Right(())) assertEquals( - Worktrees.startBranch(path, branchName("wt-branch")), + Effects.worktrees.startBranch(path, branchName("wt-branch")), Right(()) ) // What `git clean -xdff` leaves: git cannot be started in a directory that @@ -119,26 +121,29 @@ class WorktreesTest extends munit.FunSuite: // prints a stack trace to stderr — the probe catches it, and the test log // shows it even while passing. os.remove.all(path) - assert(!Worktrees.onABranch(path), "unknown must not read as on a branch") + assert( + !Effects.worktrees.onABranch(path), + "unknown must not read as on a branch" + ) test("mainCheckout never answers the git dir, even from a linked worktree"): val checkout = GitRepo.seededSeparateGitDir() val worktree = checkout / os.up / "wt" - assertEquals(Worktrees.add(checkout, worktree), Right(())) + assertEquals(Effects.worktrees.add(checkout, worktree), Right(())) // git names the git dir as the main worktree here, so there is no checkout // to answer with: refusing beats writing a worktree inside `.git`. - assertEquals(Worktrees.mainCheckout(worktree), None) + assertEquals(Effects.worktrees.mainCheckout(worktree), None) // And the reason is the layout, not "you have no repository" — the caller // words those two very differently. assertEquals( - Worktrees.mainCheckoutOrReason(worktree), + Effects.worktrees.mainCheckoutOrReason(worktree), Left(MainCheckoutFailure.MainWorktreeNotACheckout) ) test("mainCheckoutOrReason separates a non-repository from a bad layout"): assertEquals( - Worktrees.mainCheckoutOrReason(TempDirs.dir()), + Effects.worktrees.mainCheckoutOrReason(TempDirs.dir()), Left(MainCheckoutFailure.NotARepository) ) val repo = GitRepo.seeded() - assertEquals(Worktrees.mainCheckoutOrReason(repo), Right(repo)) + assertEquals(Effects.worktrees.mainCheckoutOrReason(repo), Right(repo)) diff --git a/tools/src/test/scala/orca/util/AtomicFileTest.scala b/tools/src/test/scala/orca/util/AtomicFileTest.scala index 03bb5f1e5..6c93509cf 100644 --- a/tools/src/test/scala/orca/util/AtomicFileTest.scala +++ b/tools/src/test/scala/orca/util/AtomicFileTest.scala @@ -1,7 +1,7 @@ package orca.util import munit.FunSuite -import orca.testkit.TempDirs +import orca.testkit.{Effects, TempDirs} class AtomicFileTest extends FunSuite: @@ -11,7 +11,7 @@ class AtomicFileTest extends FunSuite: os.write(p / "inner", "kept", createFolders = true) val stagingDir = TempDirs.dir() val _ = intercept[java.nio.file.FileSystemException]( - AtomicFile.replace(p, stagingDir, Array[Byte](1)) + AtomicFile.replace(Effects.fs, p, stagingDir, IArray(1.toByte)) ) assertEquals(os.read(p / "inner"), "kept") assertEquals(os.list(stagingDir).toList, Nil)