diff --git a/CHANGELOG.md b/CHANGELOG.md index 233dd1b..fa2589b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -23,6 +23,20 @@ Node pack, and the Bedrock clients, their region and credential checks are pinned by unit tests. **Upgrade note:** none - the Claude API backend is the default and behaves as before; `llm.backend` now also accepts `bedrock`. +### Changed + +1. **The `resources` folder is the one next to the application file; the working directory is never searched.** + `mercury-serve app.py` reads `resources/application.yml` (`.yaml`, `.properties`) beside `app.py`, and a direct + `python app.py` beside the script Python ran, wherever the command runs from - the engines' classpath convention: the + configuration travels with the application and does not depend on the launch directory. Before, the working directory + was searched first and the application's folder only as a fallback, so a direct `python llm_helper.py` from the + repository root started with no configuration at all (the default name and port, no health dependencies), and a stray + `resources/` in the launch directory could shadow the application's own. `--config ` still names another file; + the companion `app-log-context.yaml` follows the same folder; `AppConfig.source` is now the absolute path of the file + found and `AppConfig.resources_dir` names the folder; the startup log reports `Configuration: `. + **Upgrade note:** an application that kept `resources/` in the working directory with its application file elsewhere + passes `--config`, or moves the folder next to the file. + ## Version 4.12.21, 10/7/2026 The lock-step round with the engines: the pack moves from 4.12.15 to 4.12.21, the number the Java engine and the diff --git a/README.md b/README.md index 686351d..9ab4ee2 100644 --- a/README.md +++ b/README.md @@ -135,9 +135,11 @@ The same conventions as the engines, so a polyglot installation stays uniform: | `log.format` | `text`, `json` (pretty-printed) or `compact` (single-line JSONL) | `text` | | `log.level` | log level (`LOG_LEVEL` env var wins) | `INFO` | -Configuration lives in the `resources` folder, mirroring the engines: -`resources/application.yml` (or `.yaml` / `.properties`) in the working directory or next -to the application file, or an explicit `--config` path — see +Configuration lives in the `resources` folder next to the application file, mirroring the +engines' classpath resources: `resources/application.yml` (or `.yaml` / `.properties`) beside +the file `mercury-serve` is given, or beside the script `python app.py` ran — never the working +directory, so the configuration does not depend on where the app is launched from — or an +explicit `--config` path; see [`examples/demo-app/resources/application.yml`](examples/demo-app/resources/application.yml) for a worked sample. Values support `${ENV_VAR:default}` substitution. Runtime parameter overrides use the same `-D` syntax as the Java engine and the Rust port — checked first on every read diff --git a/docs/guides/config-logging-actuators.md b/docs/guides/config-logging-actuators.md index dc54a8c..97dd646 100644 --- a/docs/guides/config-logging-actuators.md +++ b/docs/guides/config-logging-actuators.md @@ -19,7 +19,9 @@ keywords: [configuration, resources, log format, actuator, health, kubernetes, l ## Configuration — the engines' conventions Configuration lives in the `resources` folder (`resources/application.yml`, `.yaml` -or `.properties`), in the working directory or next to the application file. Values +or `.properties`) next to the application file — the file `mercury-serve` is given, or the +script `python app.py` ran — never in the working directory, so the configuration does not +depend on where the app is launched from (`--config` names another file). Values support `${ENV_VAR:default}` substitution; `-Dkey=value` command-line arguments are runtime overrides checked first on every read — the same syntax as the Java engine's JVM system properties and the Rust port's `-D` arguments: diff --git a/docs/guides/configuration-reference.md b/docs/guides/configuration-reference.md index 7a06bab..775f7be 100644 --- a/docs/guides/configuration-reference.md +++ b/docs/guides/configuration-reference.md @@ -13,8 +13,9 @@ keywords: [configuration, reference, keys, substitution, overrides] 1. `-Dkey=value` command-line overrides (and programmatic `AppConfig.set`) — checked first on every read, the engines' `f:setConfig` analog. -2. The configuration file: `resources/application.yml` | `.yaml` | `.properties`, in - the working directory or next to the application file, or `--config `. +2. The configuration file: `resources/application.yml` | `.yaml` | `.properties` next to + the application file (the file `mercury-serve` is given, or the script `python app.py` + ran), never in the working directory; or `--config `. 3. `${ENV_VAR:default}` substitution inside values: environment first, then a base configuration key of that name, then the default. diff --git a/examples/demo-app/README.md b/examples/demo-app/README.md index 0a01ae0..4763534 100644 --- a/examples/demo-app/README.md +++ b/examples/demo-app/README.md @@ -19,8 +19,12 @@ JavaScript handler has no sync flavour), with the same behaviour. ```bash pip install mercury-composable mercury-serve examples/demo-app/demo_app.py +python examples/demo-app/demo_app.py # the same without mercury-serve ``` +Either way the app reads `resources/application.yml` next to `demo_app.py`, whichever directory the +command runs from. + The app listens on port 8086 (`rest.server.port` in `resources/application.yml`; `-Drest.server.port=8090` overrides it). The [LLM helper](../llm-helper/README.md) defaults to 8086 as well, so give one of the two another port when you run both on one machine. Map a route from the engine's `event-over-http.yaml`: diff --git a/examples/demo-app/resources/application.yml b/examples/demo-app/resources/application.yml index 6101d0e..e13feec 100644 --- a/examples/demo-app/resources/application.yml +++ b/examples/demo-app/resources/application.yml @@ -1,6 +1,7 @@ # Sample configuration - the engines' "resources" convention. -# mercury-serve loads resources/application.yml from the working directory, or from -# the resources folder next to the application file (this one). Any key can be +# mercury-serve (and a direct `python demo_app.py`) loads resources/application.yml from +# the resources folder next to the application file (this one), never from the working +# directory. Any key can be # overridden at run time with the engines' -D syntax, e.g.: # mercury-serve examples/demo-app/demo_app.py -Drest.server.port=8090 -Dlog.format=compact diff --git a/examples/llm-helper/README.md b/examples/llm-helper/README.md index 655284b..6b52aca 100644 --- a/examples/llm-helper/README.md +++ b/examples/llm-helper/README.md @@ -21,8 +21,12 @@ speaks the same contract. Both run one shared vector file, so they cannot drift pip install 'mercury-composable[llm]' # the SDK is this app's one dependency export ANTHROPIC_API_KEY=... # from the environment, never from a config file mercury-serve examples/llm-helper/llm_helper.py +python examples/llm-helper/llm_helper.py # the same without mercury-serve ``` +Either way the helper reads `resources/application.yml` next to `llm_helper.py`, whichever directory the +command runs from; `-Dkey=value` overrides any key. + On AWS Bedrock instead (see Backends), the `bedrock` extra adds boto3, which signs each call with the AWS credential chain, and a region is required: diff --git a/examples/llm-helper/resources/application.yml b/examples/llm-helper/resources/application.yml index 16fc8c7..fe1e322 100644 --- a/examples/llm-helper/resources/application.yml +++ b/examples/llm-helper/resources/application.yml @@ -1,6 +1,7 @@ # The LLM helper's configuration - the engines' "resources" convention. -# mercury-serve loads resources/application.yml from the working directory, or from the -# resources folder next to the application file (this one). Any key can be overridden at +# mercury-serve (and a direct `python llm_helper.py`) loads resources/application.yml from the +# resources folder next to the application file (this one), never from the working directory. +# Any key can be overridden at # run time with the engines' -D syntax, e.g.: # mercury-serve examples/llm-helper/llm_helper.py -Dllm.model=claude-haiku-4-5 diff --git a/src/mercury_composable/cli.py b/src/mercury_composable/cli.py index 06b796b..b8e328d 100644 --- a/src/mercury_composable/cli.py +++ b/src/mercury_composable/cli.py @@ -3,6 +3,9 @@ mercury-serve app.py --port 8086 mercury-serve app.py --config application.yml + +The configuration is the ``resources`` folder next to ``app.py`` - never the working +directory - unless ``--config`` names a file. """ from __future__ import annotations @@ -22,7 +25,8 @@ def main() -> int: help="Event API port (default: rest.server.port or 8085)") parser.add_argument("--host", default="127.0.0.1", help="Bind address") parser.add_argument("--config", default=None, - help="Configuration file (default: resources/application.yml|properties)") + help="Configuration file (default: resources/application.yml|yaml|properties " + "next to the application file)") # -Dkey=value runtime overrides are consumed by AppConfig from sys.argv args, _unknown = parser.parse_known_args() # argparse Namespace attributes are untyped - pin the types at the boundary @@ -31,22 +35,14 @@ def main() -> int: host_arg: str = args.host config_arg: str | None = args.config - from .config import DEFAULT_CANDIDATES, load_config + from .config import load_config app_path = os.path.abspath(app_arg) - config_path: str | None = config_arg - if config_path is None and not any(os.path.isfile(c) for c in DEFAULT_CANDIDATES): - # fall back to a resources folder next to the application file - app_dir = os.path.dirname(app_path) - for candidate in DEFAULT_CANDIDATES: - probe = os.path.join(app_dir, candidate) - if os.path.isfile(probe): - config_path = probe - break - load_config(config_path) # before logging/server setup so their keys apply - if not os.path.isfile(app_path): print(f"Application file not found - {app_path}", file=sys.stderr) return 1 + # the resources folder is the one next to the application file, wherever the command runs + # from; --config names another file. Loaded before logging/server setup so their keys apply. + load_config(config_arg, app_dir=os.path.dirname(app_path)) spec = importlib.util.spec_from_file_location("mercury_user_app", app_path) if spec is None or spec.loader is None: print(f"Unable to load application - {app_path}", file=sys.stderr) diff --git a/src/mercury_composable/config.py b/src/mercury_composable/config.py index f254e97..e5489bd 100644 --- a/src/mercury_composable/config.py +++ b/src/mercury_composable/config.py @@ -4,9 +4,13 @@ Same conventions as the Java/Rust AppConfigReader so a polyglot installation uses one configuration style everywhere: -- Configuration lives in the ``resources`` folder, mirroring the engines: - ``resources/application.yml`` (or ``.yaml`` / ``.properties``), or an - explicit path via ``AppConfig(path=...)`` / ``mercury-serve --config``. +- Configuration lives in the ``resources`` folder NEXT TO THE APPLICATION + FILE, mirroring the engines' classpath resources: ``resources/application.yml`` + (or ``.yaml`` / ``.properties``) beside the file ``mercury-serve`` is given, + or beside the script Python ran (``python app.py``); or an explicit path via + ``AppConfig(path=...)`` / ``mercury-serve --config``. The working directory + is never searched, so the configuration an application starts with does not + depend on where it is launched from. - Dotted keys over nested YAML (``rest.server.port``). - ``-Dkey=value`` command-line arguments are runtime parameter overrides, checked first on every read — the same syntax as the Java engine's JVM @@ -46,6 +50,16 @@ "resources/application.properties", ] +RESOURCES_FOLDER = "resources" + + +def main_script_dir() -> str | None: + """The folder of the script Python ran (``python app.py``) - where a direct run + finds its ``resources`` folder; None in an interactive session, which has no script.""" + main = sys.modules.get("__main__") + file = getattr(main, "__file__", None) + return os.path.dirname(os.path.abspath(file)) if file else None + def _flatten(prefix: str, node: Any, out: dict[str, Any]) -> None: if isinstance(node, dict): @@ -81,19 +95,32 @@ def parse_d_args(argv: list[str]) -> dict[str, Any]: class AppConfig: """Flat, dot-addressed application configuration.""" - def __init__(self, path: str | None = None, argv: list[str] | None = None): + def __init__(self, path: str | None = None, argv: list[str] | None = None, + app_dir: str | None = None): + """``path`` names the configuration file explicitly; otherwise the file is one of + DEFAULT_CANDIDATES under ``app_dir`` - the application file's folder, defaulting to the + folder of the script Python ran. The working directory is never searched.""" self._store: dict[str, Any] = {} self._overrides: dict[str, Any] = parse_d_args(sys.argv[1:] if argv is None else argv) self._source = "none" - candidates = [path] if path else DEFAULT_CANDIDATES - for candidate in candidates: - if candidate and os.path.isfile(candidate): - self._load(candidate) - self._source = candidate - break - else: - if path: + self._resources_dir: str | None = None + if path: + if not os.path.isfile(path): raise FileNotFoundError(f"Configuration file not found - {path}") + self._load(path) + self._source = path + self._resources_dir = os.path.dirname(os.path.abspath(path)) + return + folder = app_dir if app_dir is not None else main_script_dir() + if folder is None: + return + self._resources_dir = os.path.join(folder, RESOURCES_FOLDER) + for candidate in DEFAULT_CANDIDATES: + probe = os.path.join(folder, candidate) + if os.path.isfile(probe): + self._load(probe) + self._source = probe + break def _load(self, path: str) -> None: with open(path, "r", encoding="utf-8") as f: @@ -108,8 +135,17 @@ def _load(self, path: str) -> None: @property def source(self) -> str: + """The configuration file loaded (an explicit path as given, else the absolute path of + the file found next to the application file), or ``none``.""" return self._source + @property + def resources_dir(self) -> str | None: + """The ``resources`` folder companion files (``app-log-context.yaml``) share with the + configuration file - beside an explicit file, else next to the application file; + None when there is neither.""" + return self._resources_dir + def set(self, key: str, value: Any) -> None: """Runtime override, checked first on every read (f:setConfig analog).""" if not key or not str(key).strip(): @@ -169,7 +205,8 @@ def _resolve_ref(self, ref: str) -> Any: def app_config() -> AppConfig: - """The shared AppConfig singleton (created on first use).""" + """The shared AppConfig singleton (created on first use - a direct ``python app.py`` run + resolves its ``resources`` folder next to the script).""" global _instance with _lock: instance = _instance @@ -179,10 +216,11 @@ def app_config() -> AppConfig: return instance -def load_config(path: str | None = None) -> AppConfig: - """Replace the shared AppConfig (used by the CLI before startup).""" +def load_config(path: str | None = None, app_dir: str | None = None) -> AppConfig: + """Replace the shared AppConfig (used by the CLI before startup, with the application + file's folder as ``app_dir``).""" global _instance with _lock: - instance = AppConfig(path) + instance = AppConfig(path, app_dir=app_dir) _instance = instance return instance diff --git a/src/mercury_composable/log_context.py b/src/mercury_composable/log_context.py index 786b938..7845139 100644 --- a/src/mercury_composable/log_context.py +++ b/src/mercury_composable/log_context.py @@ -131,12 +131,12 @@ def _load() -> tuple[LogContextConfig, str | None]: config = app_config() if (config.get_property(FEATURE_FLAG, "true") or "true").lower() == "false": return LogContextConfig(None), None - # an application override replaces the default entirely - same resources - # convention as application.yml - source = config.source - folder = os.path.dirname(source) if source != "none" else "resources" - candidate = os.path.join(folder or "resources", CONFIG_FILE) - if os.path.isfile(candidate): + # an application override replaces the default entirely - it sits in the same + # resources folder as application.yml (next to the application file, or beside + # an explicit --config file); the working directory is never searched + folder = config.resources_dir + candidate = os.path.join(folder, CONFIG_FILE) if folder else None + if candidate and os.path.isfile(candidate): with open(candidate, "r", encoding="utf-8") as f: section = _context_section(yaml.safe_load(f.read()) or {}) if section is None: diff --git a/src/mercury_composable/server.py b/src/mercury_composable/server.py index c53a42c..c3144b1 100644 --- a/src/mercury_composable/server.py +++ b/src/mercury_composable/server.py @@ -289,6 +289,7 @@ def run(self, port: int | None = None, host: str = "127.0.0.1") -> None: for route, service in sorted(self.registry.routes().items()): visibility = "PRIVATE" if service.private else "PUBLIC" log.info("Loaded %s %s, instances=%d", visibility, route, service.instances) + log.info("Configuration: %s", config.source) log.info("%s - Event API service started on port %d", app_name, actual_port) app = server.create_app() if exporter is not None: diff --git a/tests/test_config.py b/tests/test_config.py index 6ded039..e6abfb8 100644 --- a/tests/test_config.py +++ b/tests/test_config.py @@ -1,6 +1,8 @@ """AppConfig tests: resources/ convention, -D overrides, ${ENV:default} substitution.""" import os +import subprocess +import sys from pathlib import Path import pytest @@ -14,16 +16,94 @@ def write(path: str, text: str) -> None: f.write(text) -def test_resources_location_convention(tmp_path: Path, monkeypatch: pytest.MonkeyPatch): - write(str(tmp_path / "resources" / "application.yml"), +def test_resources_next_to_the_application_file(tmp_path: Path): + app_dir = tmp_path / "app" + write(str(app_dir / "resources" / "application.yml"), "application:\n name: 'demo-app'\nrest:\n server:\n port: 8086\n") - monkeypatch.chdir(tmp_path) - config = AppConfig(argv=[]) - assert config.source == "resources/application.yml" + config = AppConfig(argv=[], app_dir=str(app_dir)) + assert config.source == str(app_dir / "resources" / "application.yml") + assert config.resources_dir == str(app_dir / "resources") assert config.get("application.name") == "demo-app" assert config.get("rest.server.port") == 8086 +def test_the_working_directory_is_never_searched(tmp_path: Path, monkeypatch: pytest.MonkeyPatch): + # a decoy in the working directory, nothing next to the application file + write(str(tmp_path / "cwd" / "resources" / "application.yml"), "application:\n name: 'decoy'\n") + monkeypatch.chdir(tmp_path / "cwd") + app_dir = tmp_path / "app" + app_dir.mkdir() + config = AppConfig(argv=[], app_dir=str(app_dir)) + assert config.source == "none" + assert config.get("application.name") is None + assert config.resources_dir == str(app_dir / "resources") # where a companion file would sit + + +def test_a_direct_run_anchors_on_the_script_python_ran(tmp_path: Path, + monkeypatch: pytest.MonkeyPatch): + from mercury_composable import config as config_module + write(str(tmp_path / "resources" / "application.yml"), "application:\n name: 'from-script'\n") + monkeypatch.setattr(config_module, "main_script_dir", lambda: str(tmp_path)) + config = AppConfig(argv=[]) + assert config.source == str(tmp_path / "resources" / "application.yml") + assert config.get("application.name") == "from-script" + + +def test_an_interactive_session_has_no_script_and_loads_nothing(monkeypatch: pytest.MonkeyPatch): + from mercury_composable import config as config_module + monkeypatch.setattr(config_module, "main_script_dir", lambda: None) + config = AppConfig(argv=[]) + assert config.source == "none" + assert config.resources_dir is None + + +def test_an_explicit_file_keeps_its_companions_beside_it(tmp_path: Path): + path = str(tmp_path / "conf" / "application.yml") + write(path, "application:\n name: 'explicit'\n") + config = AppConfig(path=path, argv=[]) + assert config.source == path + assert config.resources_dir == str(tmp_path / "conf") + + +APP_SCRIPT = """\ +from mercury_composable import app_config +print(app_config().get("application.name"), app_config().source) +raise SystemExit(0) # before any function registers - the configuration is the point +""" + + +def _app_in_a_decoy_cwd(tmp_path: Path) -> tuple[Path, Path, str]: + """An application file with its own resources folder, and a working directory holding a decoy.""" + app = tmp_path / "app" / "app.py" + write(str(app), APP_SCRIPT) + write(str(tmp_path / "app" / "resources" / "application.yml"), "application:\n name: 'from-app'\n") + write(str(tmp_path / "cwd" / "resources" / "application.yml"), "application:\n name: 'decoy'\n") + return app, tmp_path / "cwd", str(tmp_path / "app" / "resources" / "application.yml") + + +def _run(args: list[str], cwd: Path) -> str: + result = subprocess.run([sys.executable, *args], cwd=str(cwd), capture_output=True, + text=True, timeout=120, check=False) + assert result.returncode == 0, result.stderr + return result.stdout.strip() + + +def test_python_app_py_resolves_next_to_the_script(tmp_path: Path): + # `python app.py` from a decoy working directory finds the resources folder next to app.py + app, cwd, expected = _app_in_a_decoy_cwd(tmp_path) + assert _run([str(app)], cwd) == f"from-app {expected}" + + +def test_mercury_serve_resolves_next_to_the_application_file(tmp_path: Path): + # the CLI, run from a decoy working directory, loads the resources folder next to the app + app, cwd, expected = _app_in_a_decoy_cwd(tmp_path) + assert _run(["-m", "mercury_composable.cli", str(app)], cwd) == f"from-app {expected}" + # --config names another file + other = str(tmp_path / "other.yml") + write(other, "application:\n name: 'other'\n") + assert _run(["-m", "mercury_composable.cli", str(app), "--config", other], cwd) == f"other {other}" + + def test_properties_format(tmp_path: Path): path = str(tmp_path / "application.properties") write(path, "# comment\nrest.server.port=8087\napplication.name=props-app\n")