fix(eval): use configs/ and load .env from cwd for local evals

[AKSHEXXXX]

Summary

Fixes #1295.

Local sam eval expected a .configs/ layout while sam init creates configs/. This change aligns eval paths with the init layout.

.env is now loaded from the project working directory (via find_dotenv(usecwd=True)) before parsing the test suite so broker and model _VAR references resolve correctly.

Changes

• Use configs/ for eval_backend.yaml, shared config seeding, and summary builder paths
• Add load_dotenv_from_cwd() and call it at the start of evaluation and from helpers/subscriber
• Remove obsolete .configs/ gitignore entry
• Add unit tests for helper env resolution

Testing

• pytest tests/unit/cli/commands/test_eval_cmd.py
• pytest tests/unit/evaluation/test_helpers.py

[Copilot]

Pull request overview

Aligns local sam eval behavior with the project layout created by sam init by switching eval config paths from .configs/ to configs/, and ensures .env variables are loaded from the project working directory early enough for _VAR environment resolution during local eval runs.

Changes:

• Updated evaluation codepaths to reference configs/eval_backend.yaml (instead of .configs/) across loader, runner, and summary builder.
• Added load_dotenv_from_cwd() and integrated it into evaluation entrypoints and subscriber initialization.
• Added unit tests for dotenv loading / _VAR resolution and updated repo docs / gitignore accordingly.

Reviewed changes

Copilot reviewed 8 out of 9 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
evaluation/shared/helpers.py	Introduces load_dotenv_from_cwd() and uses it to ensure env vars are loaded before resolving _VAR keys.
evaluation/run.py	Loads dotenv-from-CWD early in evaluation flow; switches eval backend config seeding to configs/.
evaluation/shared/test_suite_loader.py	Updates auto-injected eval backend path to configs/eval_backend.yaml.
evaluation/summary_builder.py	Updates artifact config loading path to configs/eval_backend.yaml.
evaluation/subscriber.py	Uses the new helper to load .env from the working directory at import time.
evaluation/shared/__init__.py	Re-exports load_dotenv_from_cwd.
tests/unit/evaluation/test_helpers.py	Adds tests covering dotenv-from-CWD behavior and _VAR resolution.
README.md	Adds local development setup section.
.gitignore	Removes obsolete .configs/ ignore entry.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

find_dotenv(usecwd=True) will walk up parent directories from the CWD until it finds a .env, so this may load env vars from an unexpected parent directory (not strictly “from the current working directory” as the docstring says). If the intent is to only honor a project-local .env in the working directory, consider loading Path.cwd() / '.env' directly (and no-op if it doesn't exist) to avoid surprising/unsafe env injection.

[Copilot]

This test writes a .env and relies on load_dotenv_from_cwd() (via resolve_env_vars) which mutates os.environ, but the env var it introduces (BROKER_HOST_FROM_ENV) is not cleaned up afterward. To avoid leaking state into other tests, save/restore the previous value (or explicitly os.environ.pop/monkeypatch.delenv in a teardown/finally block).

Suggested change

	resolved = resolve_env_vars(
	{
	"SOLACE_BROKER_URL_VAR": "BROKER_HOST_FROM_ENV",
	"vpn_name": "default",
	}
	)
	assert resolved["SOLACE_BROKER_URL"] == "mqtt.example.com"
	assert resolved["vpn_name"] == "default"
	broker_host_key = "BROKER_HOST_FROM_ENV"
	had_broker_host = broker_host_key in os.environ
	previous_broker_host = os.environ.get(broker_host_key)
	try:
	resolved = resolve_env_vars(
	{
	"SOLACE_BROKER_URL_VAR": "BROKER_HOST_FROM_ENV",
	"vpn_name": "default",
	}
	)
	assert resolved["SOLACE_BROKER_URL"] == "mqtt.example.com"
	assert resolved["vpn_name"] == "default"
	finally:
	if had_broker_host:
	os.environ[broker_host_key] = previous_broker_host
	else:
	os.environ.pop(broker_host_key, None)

[Copilot]

load_dotenv_from_cwd() sets environment variables in the process; this test asserts the variable is set but doesn’t restore the prior value afterward. Even with a unique key, restoring the previous value (or unsetting it) makes the test more isolated and prevents order-dependent failures if future tests start using the same name.

Suggested change

	monkeypatch.delenv(key, raising=False)
	(tmp_path / ".env").write_text(f"{key}=from_dotenv\n")
	monkeypatch.chdir(tmp_path)
	load_dotenv_from_cwd()
	assert os.environ.get(key) == "from_dotenv"
	original_value = os.environ.get(key)
	monkeypatch.delenv(key, raising=False)
	(tmp_path / ".env").write_text(f"{key}=from_dotenv\n")
	monkeypatch.chdir(tmp_path)
	try:
	load_dotenv_from_cwd()
	assert os.environ.get(key) == "from_dotenv"
	finally:
	if original_value is None:
	os.environ.pop(key, None)
	else:
	os.environ[key] = original_value