Cache config file layer discovery (#319)

.github/dependabot.yml

@@ -43,6 +43,7 @@ updates:
     directories:
       - "/"
       - "/test_support"
+      - "/tests/ui/cli_configuration_pass"
     open-pull-requests-limit: 5
     labels:
       - "dependencies"

docs/developers-guide.md

@@ -2386,8 +2386,9 @@ a two-pass approach when no explicit config path is provided:
 Because `MergeComposer` uses last-wins semantics, pushing the project layers
 after user layers gives them higher precedence.
 
-Early JSON resolution reuses this logic through
-`collect_diag_file_layers_with_sources`, before full configuration merging.
+Early JSON resolution calls `discover_file_layers`, which returns a
+`DiscoveryOutcome` containing the cached layers, resolved JSON preference, and
+deferred bounded diagnostics before full configuration merging.
 
 ### Layer precedence
 
@@ -2405,10 +2406,10 @@ Private helper functions for config discovery and JSON-output resolution.
 
 Configuration merge helpers:
 
-- `config_discovery(directory, env_source) -> ConfigDiscovery` builds the
-  single-pass OrthoConfig discovery scanner with an optional project-root
-  anchor and the environment adapter selected at the composition root.
-- `project_scope_file(directory: Option<&Path>) -> Option<PathBuf>`
+- `config_discovery(directory: Option<&PathBuf>) -> ConfigDiscovery` builds
+  the single-pass OrthoConfig discovery scanner with an optional project-root
+  anchor.
+- `project_scope_file_str(directory: Option<&Path>) -> Option<String>`
   resolves the expected project `.netsuke.toml` path for project-layer
   detection.
 - `project_scope_layers(directory)` loads the project-scope config directly,
@@ -2419,19 +2420,19 @@ Configuration merge helpers:
   `PathBuf`.
 - `explicit_config_path_with_env(cli, env) -> Option<PathBuf>` resolves explicit
   config selection from `--config` and `NETSUKE_CONFIG`.
-- `push_file_layers_with_sources(cli, composer, errors, sources) -> ()` pushes
-  explicit or discovered file layers onto a `MergeComposer`. Explicit load
-  errors are pushed into `errors`, and automatic discovery is not attempted
-  after an explicit selector fails.
-- `collect_diag_file_layers_with_sources(cli, sources)` reuses the same
-  file-layer precedence for early JSON resolution.
-- `collect_file_layers_with_env_source(directory, env_source)` builds the
-  fallback discovery layer chain, applies the project-layer second pass, and
-  returns `OrthoResult<Vec<MergeLayer<'static>>>`.
+- `discover_file_layers(cli, env) -> DiscoveryOutcome` performs the single
+  explicit-or-automatic discovery and load pass, retaining its layers, errors,
+  resolved JSON preference, and deferred bounded diagnostics for the
+  composition boundary.
+- `push_discovered_file_layers(composer, errors, layers) -> ()` consumes the
+  cached layers and discovery errors while composing the full configuration.
+- `collect_file_layers(directory)` builds the fallback discovery layer chain,
+  applies the project-layer second pass, and returns
+  `OrthoResult<Vec<MergeLayer<'static>>>`.
 - `is_empty_value(value: &serde_json::Value) -> bool` detects an empty CLI
   override object.
-- `json_from_layer(value: &serde_json::Value) -> Option<bool>` extracts `json`
-  from a configuration value.
+- `json_from_value(value: &serde_json::Value) -> Option<bool>` extracts `json`
+  from a configuration value while discovery retains the resulting preference.
 - `json_from_matches(cli, matches, discovered) -> bool` applies an explicit
   root `--json` override to the discovered value.
 - `cli_overrides_from_matches(matches: &ArgMatches) -> OrthoValue` extracts
@@ -2441,35 +2442,66 @@ Configuration merge helpers:
 
 ### Environment lookup seams
 
-`cli::discovery::EnvProvider` is the port for raw environment access during
-early CLI configuration resolution; `src/cli/mod.rs` re-exports it as
-`ConfigEnvProvider` (and `StdEnvProvider` as `ConfigStdEnvProvider`), so
-external callers see only the `Config*` names below. The production
-`StdEnvProvider` adapter delegates to the process environment; tests can inject
-map-backed providers without mutating process-global state.
+CLI configuration uses the `mockable::Env` seam for environment access.
+Production wrappers bind `mockable::DefaultEnv`; tests inject
+`mockable::MockEnv` and do not mutate process-global environment variables.
+The public entry points have these signatures:
 
 ```rust
-pub trait ConfigEnvProvider {
-    fn get(&self, key: &str) -> Option<std::ffi::OsString>;
-    fn entries(&self) -> Vec<(std::ffi::OsString, std::ffi::OsString)>;
-}
+pub fn resolve_merged_json(
+    cli: &Cli,
+    matches: &ArgMatches,
+) -> OrthoResult<bool>;
+pub fn resolve_merged_json_with_env(
+    cli: &Cli,
+    matches: &ArgMatches,
+    env: &impl mockable::Env,
+) -> OrthoResult<bool>;
+pub fn resolve_json_and_layers_with_env(
+    cli: &Cli,
+    matches: &ArgMatches,
+    env: &impl mockable::Env,
+) -> OrthoResult<(bool, DiscoveredLayers)>;
+pub fn resolve_json_and_layers_outcome_with_env(
+    cli: &Cli,
+    matches: &ArgMatches,
+    env: &impl mockable::Env,
+) -> (OrthoResult<bool>, DiscoveryOutcome);
+pub fn merge_with_config(cli: &Cli, matches: &ArgMatches) -> OrthoResult<Cli>;
+pub fn merge_with_config_and_env(
+    cli: &Cli,
+    matches: &ArgMatches,
+    env: &impl mockable::Env,
+) -> OrthoResult<Cli>;
+pub fn merge_with_layers(
+    cli: &Cli,
+    matches: &ArgMatches,
+    env: &impl mockable::Env,
+    layers: DiscoveredLayers,
+) -> OrthoResult<Cli>;
+pub fn merge_with_process_environment_layers(
+    cli: &Cli,
+    matches: &ArgMatches,
+    layers: DiscoveredLayers,
+) -> OrthoResult<Cli>;
 ```
 
-`get` owns selector lookup, while `entries` supplies the complete snapshot for
-the layered `NETSUKE_*` merge. A selector-only provider may retain the empty
-default for `entries`. Full-merge adapters must return a stable owned snapshot
-so discovery and value merging observe one environment. Keep this port scoped
-to CLI configuration; runner, manifest, locale, and stdlib environment seams
-remain separate because their input and lifetime contracts differ.
-
-`DiscoverySources` is a crate-private composition input owned by
-`src/cli/discovery.rs`. Only full merge and early JSON resolution may construct
-it. Ambient entry points pair `ConfigStdEnvProvider` with OrthoConfig
-`ProcessEnv`; injected entry points project the same `ConfigEnvProvider` into a
-closed `MapEnv` containing only `NETSUKE_CONFIG`, `HOME`, `USERPROFILE`,
-`XDG_CONFIG_HOME`, `XDG_CONFIG_DIRS`, `APPDATA`, and `LOCALAPPDATA`. Do not
-reuse this fixed-key projection as a general environment-copy helper;
-`EnvironmentLayer` alone enumerates the full `NETSUKE_*` value environment.
+`resolve_json_and_layers_with_env` returns the resolved JSON preference and the
+`DiscoveredLayers` from that same discovery pass. The caller must pass those
+layers to `merge_with_layers` for the subsequent full merge; this handoff
+prevents configuration files from being discovered and loaded again.
+`resolve_json_and_layers_outcome_with_env` returns a `DiscoveryOutcome` that
+retains the layers and bounded diagnostics when diagnostic resolution fails.
+The startup boundary calls `emit_diagnostics()` after enabling its tracing
+filter, then calls `into_layers()` and passes the layers to
+`merge_with_process_environment_layers`.
+
+`merge_with_process_environment_layers` accepts pre-discovered layers and
+reads raw process environment entries only at the composition boundary. This
+preserves non-Unicode entries for the environment-layer policy while keeping
+discovery and diagnostic lookups behind the injected seam. Standalone callers
+can use `merge_with_config_and_env` to discover and merge with an injected
+environment; callers with cached layers should pass them to `merge_with_layers`.
 
 `explicit_config_path_with_env` is the crate-internal seam for explicit
 config-file selection. It evaluates the precedence chain in this order:
@@ -2481,50 +2513,46 @@ config-file selection. It evaluates the precedence chain in this order:
 environment value into `PathBuf`. Both full merging and early JSON resolution
 use the same injected selector and file-layer implementation.
 
-The ambient public APIs `merge_with_config` and `resolve_merged_json` each
-accept two arguments. Their injected counterparts accept a `ConfigEnvProvider`:
-
-```rust
-pub fn merge_with_config(cli: &Cli, matches: &ArgMatches) -> OrthoResult<Cli>;
-pub fn merge_with_config_and_env(
-    cli: &Cli,
-    matches: &ArgMatches,
-    env: &impl ConfigEnvProvider,
-) -> OrthoResult<Cli>;
-pub fn resolve_merged_json(cli: &Cli, matches: &ArgMatches) -> OrthoResult<bool>;
-pub fn resolve_merged_json_with_env(
-    cli: &Cli,
-    matches: &ArgMatches,
-    env: &impl ConfigEnvProvider,
-) -> OrthoResult<bool>;
-```
-
-The `cli` module re-exports this trait publicly as `ConfigEnvProvider` (and
-`StdEnvProvider` as `ConfigStdEnvProvider`) to keep the CLI seam distinct from
-the unrelated `LocaleEnvProvider` in `locale_resolution`; crate-internal code
-uses the bare `EnvProvider` name.
-
-Tests for injected configuration discovery should provide a map-backed
-`ConfigEnvProvider`. End-to-end tests of the ambient `ProcessEnv` adapter must
-run in an isolated child configured with `env_clear()` followed by
-`Command::env`. `EnvLock` is reserved for tests that change the process working
-directory alongside `CwdGuard`; it does not justify environment mutation.
+Tests for Netsuke's environment seam should inject `mockable::MockEnv`
+directly. End-to-end tests may configure a child process with `env_clear()` and
+`Command::env`, but in-process tests must not mutate the process environment.
 
 Unit tests that only need to verify explicit config path precedence should test
 `explicit_config_path_with_env` with an injected provider instead of mutating
 the process environment.
 
 Config selector resolution remains a pure query: `resolve_config_selector`
 records the winning selector, its optional path, and every environment lookup
-evaluated, and emits no tracing itself. Structured diagnostics are emitted only
-at the file-layer boundary, where `collect_file_layers_with_env` calls
-`trace_config_path_resolution` after resolution completes.
-
-Tracing never logs full paths or formatted parser errors. Path values are
-bounded to a `path_hash` correlation identifier plus `path_file_name`, and load
-failures are classified with the `ConfigLoadFailureKind` enum instead of the
-formatted error text. `path_hash` is a bounded identifier for correlating
-events, not a cryptographic guarantee.
+evaluated. `DiscoveryOutcome` also retains bounded branch metadata from that
+pass. `emit_diagnostics()` replays the environment lookups, resolved selector,
+file-layer branch, and applicable project-scope outcome without performing
+another environment lookup, filesystem scan, path normalization, or
+configuration-file load.
+
+#### Cached discovery telemetry
+
+The full-merge boundary increments
+`netsuke_cli_config_discovery_cache_total` exactly once per merge. Its sole
+`outcome` label belongs to a closed two-value set:
+
+- `reused` — `merge_with_layers` or
+  `merge_with_process_environment_layers` consumed layers discovered by an
+  earlier phase.
+- `bypass` — `merge_with_config` or `merge_with_config_and_env` performed
+  standalone discovery immediately before merging.
+
+This counter measures the layer handoff only. Configuration-load success,
+failure, and duration belong to the startup observability boundary, so this
+metric does not duplicate them. It carries no selectors, paths, error text, or
+configuration values. Recorder-backed tests pin both outcomes through local
+`metrics_util::DebuggingRecorder` instances without installing process-global
+state.
+
+Tracing never logs full paths, file names, or formatted parser errors. Path
+values are bounded to a `path_hash` correlation identifier plus a presence
+indicator. Load failures are classified with the `ConfigLoadFailureKind` enum
+instead of the formatted error text. `path_hash` is a bounded identifier for
+correlating events, not a cryptographic guarantee.
 
 #### `json` contract
 
@@ -2761,16 +2789,16 @@ split diagnostics, path comparison, and tests out of the main discovery flow:
 - `discovery_event_assertions.rs` — shared test-only helpers:
   `capture_events` runs a closure under a TRACE capturing subscriber,
   `find_event` locates one emitted event by substring, and `EventAssertion`
-  bundles an event with its path to assert bounded `path_hash`/`path_file_name`
+  bundles an event with its path to assert bounded `path_hash` and presence
   fields, the absence of the raw path or formatted error text, and to normalize
   the hash before an `insta` snapshot.
 - `discovery_tracing_tests.rs` — tests selector precedence
   (`--config` versus `NETSUKE_CONFIG`), the removed legacy
   `NETSUKE_CONFIG_PATH` alias, and event-schema snapshots for both selection
   and explicit load failures.
-- `discovery_layer_tests.rs` — tests the test-only
-  `collect_diag_file_layers_with_env` wrapper (explicit path versus automatic
-  discovery) and the project-scope second pass in `collect_file_layers`.
+- `discovery_layer_tests.rs` — tests `discover_file_layers` directly for
+  explicit-path and automatic-discovery branches, plus the project-scope second
+  pass in `collect_file_layers`.
 
 Both test modules import `capture_events`, `find_event`, and `EventAssertion`
 from `discovery_event_assertions` rather than duplicating them. The `insta`

docs/netsuke-design.md

@@ -2776,8 +2776,12 @@ flowchart LR
 Netsuke configuration discovery is implemented in `src/cli/discovery.rs`.
 Explicit file selection is handled by `resolve_config_selector(...)`, which
 applies the precedence `--config` > `NETSUKE_CONFIG`. Layer loading and
-automatic discovery are handled by `push_file_layers_with_sources(...)`, which
-also applies the `-C/--directory` flag as the project-discovery root.
+automatic discovery are handled by `discover_file_layers(...)`, which honours
+the `-C/--directory` project-discovery root and returns a `DiscoveryOutcome`
+containing the loaded layers and deferred bounded diagnostics. Startup emits
+those diagnostics after configuring its tracing filter, consumes
+`into_layers()`, and passes the resulting layers to the full merge so discovery
+and loading happen only once.
 
 **Figure: Explicit Config Selector Resolution** — This diagram shows how
 Netsuke chooses the configuration file before automatic discovery. Netsuke
@@ -2883,28 +2887,26 @@ manual flag repetition.
   selectors before automatic discovery so missing or invalid explicit files
   remain hard errors.
 - The `merge_with_config()` function in `src/cli/merge.rs` orchestrates the
-  full layer composition: it calls `push_file_layers(...)` to load explicit,
-  project, and user file layers, merges them with defaults, adds environment
-  variables via Figment, and finally applies CLI overrides extracted from
-  `ArgMatches`.
-- The `config_discovery()` function uses OrthoConfig's builder API with the
-  application name, environment selector, and an environment source selected at
-  the CLI composition root. Ambient runs use `ProcessEnv`; injected runs use a
-  closed `MapEnv` projected from Netsuke's environment port, preventing tests
-  from falling through to host directories.
-- A missing optional candidate means no configuration layer and therefore
-  built-in defaults. A candidate that exists but cannot load is retained as an
-  error when no candidate succeeds, so malformed configuration and a missing
-  `extends` parent are never mistaken for absence.
+  full layer composition: it performs one `discover_file_layers(...)` pass,
+  emits its deferred diagnostics, consumes the resulting layers, merges them
+  with defaults, adds environment variables via Figment, and finally applies
+  CLI overrides extracted from `ArgMatches`.
+- The `config_discovery()` function uses OrthoConfig's builder API without
+  further customization beyond the application name and environment variable
+  override, relying on OrthoConfig's platform-specific defaults for standard
+  directory resolution.
 - Netsuke-owned environment reads for explicit config selection and early JSON
-  resolution go through the `EnvProvider` port in `src/cli/discovery.rs`.
-  Production code uses `StdEnvProvider`; tests can inject a map-backed provider
-  instead of mutating the process environment. The v0.9.0 adapter projects only
-  the documented discovery keys into OrthoConfig, while `EnvironmentLayer`
-  retains the complete `NETSUKE_*` value merge boundary.
-- Configuration files use TOML. OrthoConfig's optional YAML provider remains
-  disabled; Netsukefile YAML continues to be parsed by the separate
-  `serde-saphyr` manifest boundary.
+  resolution take an injected `&impl mockable::Env` in the public
+  `*_with_env` entry points. Production wrappers supply
+  `mockable::DefaultEnv`; tests supply `mockable::MockEnv` without mutating the
+  process environment. Startup obtains a `DiscoveryOutcome` from
+  `resolve_json_and_layers_outcome_with_env`, emits its deferred diagnostics,
+  then passes `into_layers()` to `merge_with_process_environment_layers`, so
+  file discovery and loading happen once. OrthoConfig discovery remains an
+  external boundary and may still read platform environment variables directly.
+- Configuration files use TOML format by default. JSON5 (`.json`, `.json5`) and
+  YAML (`.yaml`, `.yml`) formats are supported when the corresponding Cargo
+  features are enabled.
 - Explicit config selection is handled outside OrthoConfig's built-in discovery
   override surface so Netsuke keeps its custom two-pass project-over-user merge
   behaviour for automatic discovery. If an explicit selector is set, the