security: Sprint 0 emergency fixes

Dockerfile

@@ -8,6 +8,10 @@ RUN pip install --no-cache-dir .
 COPY entrypoint.sh /entrypoint.sh
 RUN chmod +x /entrypoint.sh
 
+RUN useradd -r -s /bin/false agent
+RUN mkdir -p /workspace && chown -R agent:agent /app /workspace
+USER agent
+
 WORKDIR /workspace
 ENTRYPOINT ["/entrypoint.sh"]
 CMD ["serve", "--config", "/workspace/agents.yml"]

SECURITY.md

@@ -0,0 +1,123 @@
+# Security Policy
+
+## Supported Versions
+
+| Version | Supported |
+|---------|-----------|
+| `main` (beta) | Yes — active development, receives security fixes |
+| < 1.0 releases | No — no stable releases yet |
+
+This project is in **beta**. Security fixes are applied to `main` and shipped in the next container image build (`ghcr.io/extra-org/extra`). There are no long-term support branches.
+
+---
+
+## Reporting a Vulnerability
+
+If you discover a security vulnerability, please report it **privately** — do not open a public GitHub issue.
+
+**Preferred:** [GitHub Security Advisories](https://github.com/extra-org/extra/security/advisories/new)
+
+### What to include
+
+- Description of the vulnerability
+- Steps to reproduce
+- Affected version or commit
+- Potential impact assessment
+- Any suggested fix (if applicable)
+
+### Response timeline
+
+| Step | Target |
+|------|--------|
+| Acknowledgment | 72 hours |
+| Triage and severity assessment | 7 days |
+| Fix or mitigation | 14 days for critical, 30 days for others |
+| Public disclosure | After fix is released, coordinated with reporter |
+
+We follow **coordinated disclosure**. Please do not publicly disclose the vulnerability until a fix is available and you have been notified.
+
+---
+
+## Important: Beta Software — Known Security Gaps
+
+This project is **not production-ready**. The following critical security gaps are known and documented. **Do not expose this software to an untrusted network without addressing them.**
+
+| Gap | Risk | Status |
+|-----|------|--------|
+| **No authentication** on any API endpoint (`/invoke`, `/stream`, `/conversations`) | Critical | Open — Sprint 1 planned |
+| **Access control not enforced** — the `protected` node feature is structurally wired but the Security/Context Gate that populates real identity is not implemented | Critical | Open — Sprint 1 planned |
+| **No rate limiting** — no per-IP or per-endpoint request throttling | High | Open — Sprint 1 planned |
+| **No built-in TLS** — must be handled by a reverse proxy | High | Open — Sprint 3 planned |
+| **No security headers** — missing `X-Content-Type-Options`, `X-Frame-Options`, `Strict-Transport-Security` | Medium | Open — Sprint 1 planned |
+| **No SSE disconnect detection** — engine continues processing after client disconnects | Medium | Open — Sprint 2 planned |
+| **No server timeouts** — uvicorn keep-alive and request timeouts not configured | Medium | Open — Sprint 2 planned |
+
+For the full remediation plan, see [docs/SECURITY_SPRINTS.md](docs/SECURITY_SPRINTS.md).
+
+---
+
+## Security Features Already Implemented
+
+Despite the gaps above, the following security practices are in place:
+
+| Practice | Description |
+|----------|-------------|
+| YAML secret scanning | Rejects literal secrets and credential shapes; only variable references are allowed |
+| Non-root Docker user | Container runs as `agent` user, not root |
+| CORS default deny | Empty allowed origins list by default; must be explicitly configured |
+| Request size limits | Pydantic `max_length` on all string fields in request schemas |
+| Execution cost guardrails | Per-run token and cost limits prevent runaway LLM spend |
+| MCP auth headers never logged | Authentication headers passed to MCP servers are excluded from logs |
+| SQL injection prevention | All database access goes through SQLAlchemy ORM |
+| Safe YAML loading | YAML is loaded without code execution or arbitrary object construction |
+| Sensitive argument masking | HITL approval prompts mask sensitive tool arguments |
+| Request ID sanitization | Request IDs are regex-validated and capped at 64 characters |
+
+---
+
+## Deployment Security Recommendations
+
+Before exposing Extra to any network, you **must**:
+
+1. **Implement authentication** — write a custom auth plugin that validates API keys or JWT tokens. See the plugin architecture in [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md).
+
+2. **Run behind a reverse proxy** — use nginx, Caddy, or a cloud load balancer to terminate TLS and add security headers.
+
+3. **Restrict CORS origins** — set `CORS_ORIGINS` in your `.env` to the exact domains that should access the API.
+
+4. **Use Docker network isolation** — bind to `127.0.0.1:port:port` and expose through the container platform's networking.
+
+5. **Monitor logs** — watch for authentication failures, approval rejections, and protected-node access attempts.
+
+6. **Pin container versions** — use `ghcr.io/extra-org/extra:v1.2.3` (specific tag), not `:latest`.
+
+---
+
+## Security Roadmap
+
+| Sprint | Focus | Status |
+|--------|-------|--------|
+| Sprint 0 | Emergency fixes (error responses, non-root Docker, request limits) | Completed |
+| Sprint 1 | Auth & access control (authentication middleware, rate limiting, security headers) | Planned |
+| Sprint 2 | Input hardening (timeouts, SSE cleanup, path validation, log redaction) | Planned |
+| Sprint 3 | Production readiness (TLS docs, health checks, audit logging) | Planned |
+
+Full details: [docs/SECURITY_SPRINTS.md](docs/SECURITY_SPRINTS.md)
+
+---
+
+## Security Update Process
+
+- Security fixes are merged to `main` and released via [release-please](https://github.com/googleapis/release-please) automation.
+- Container images are published to `ghcr.io/extra-org/extra` on each release.
+- Users should subscribe to [GitHub Releases](https://github.com/extra-org/extra/releases) for notifications.
+- For critical vulnerabilities, we will publish a security advisory on GitHub with affected versions and upgrade instructions.
+
+---
+
+## References
+
+- [Security Sprints](docs/SECURITY_SPRINTS.md) — full remediation plan
+- [Architecture](docs/ARCHITECTURE.md) — system design and security boundaries
+- [Runtime Lifecycle](docs/RUNTIME_LIFECYCLE.md) — request flow and security gate
+- [Plugin Context/Auth](docs/SIDECAR_CONTEXT_AUTH.md) — access control design

docker-compose.yml

@@ -0,0 +1,33 @@
+# Local development compose — exposes services on 127.0.0.1 only.
+# Not intended for production deployment.
+
+services:
+  engine:
+    build: .
+    ports:
+      - "127.0.0.1:8090:8090"
+    volumes:
+      - ./examples:/workspace:ro
+    env_file:
+      - .env
+    command: ["serve", "--host", "0.0.0.0", "--config", "/workspace/enterprise-knowledge-assistant/agents.yaml"]
+    healthcheck:
+      test: ["CMD", "python", "-c", "import httpx; httpx.get('http://127.0.0.1:8090/health')"]
+      interval: 30s
+      timeout: 5s
+      retries: 3
+
+  manager:
+    build: .
+    ports:
+      - "127.0.0.1:8100:8100"
+    volumes:
+      - ./examples:/workspace:ro
+    env_file:
+      - .env
+    command: ["agent-manager", "--host", "0.0.0.0", "--config", "/workspace/enterprise-knowledge-assistant/agents.yaml"]
+    healthcheck:
+      test: ["CMD", "python", "-c", "import httpx; httpx.get('http://127.0.0.1:8100/health')"]
+      interval: 30s
+      timeout: 5s
+      retries: 3

docs/SECURITY_SPRINTS.md

@@ -0,0 +1,83 @@
+# Security Sprints
+
+Actionable remediation plan from the security audit (July 2026).
+Each sprint is ordered — do them in sequence.
+
+---
+
+## Sprint 0 — Emergency Fixes (1-2 days)
+
+Quick wins, low effort, high impact.
+
+| # | Task | Files | Effort |
+|---|------|-------|--------|
+| 1 | **Generic error responses** — replace `str(exc)` with sanitized messages in all API error returns. Log full exceptions server-side only. | `agent_engine/api/app.py:243,297,332` `agent_manager/api/routes.py:64,115` | 2h |
+| 2 | **Non-root Docker user** — add `RUN useradd -r -s /bin/false agent` and `USER agent` before entrypoint. | `Dockerfile` | 30m |
+| 3 | **Request size limits** — add `max_length=4096` (or similar) to all `str` fields in request schemas. | `agent_manager/api/schemas.py` `agent_engine/api/app.py` | 1h |
+| 4 | **Default host 127.0.0.1** — change default `--host` from `0.0.0.0` to `127.0.0.1` in both servers. | `agentctl/main.py:190` `agent_manager/config.py:29` | 30m |
+| 5 | **Docker Compose** — add `docker-compose.yml` with `engine` (8090) and `manager` (8100) services, SQLite volume, health checks, `.env` passthrough. | `docker-compose.yml` | 30m |
+
+**Definition of done:** `make check` passes. No `str(exc)` in HTTP responses. Docker runs as non-root. `docker compose up` starts both services.
+
+---
+
+## Sprint 1 — Auth & Access Control (1-2 weeks)
+
+The critical missing piece.
+
+| # | Task | Files | Effort |
+|---|------|-------|--------|
+| 1 | **Auth middleware** — implement a FastAPI dependency that validates API key or JWT bearer token. Apply to all routes in both APIs. | `agent_engine/api/app.py` `agent_manager/api/app.py` `agent_manager/api/deps.py` | 3-5d |
+| 2 | **Wire auth context to AccessFilter** — populate `AuthContext` from the verified identity so `protected` nodes are actually enforced. | `engine/langgraph/engine.py:669-674` `engine/langgraph/filters.py:42-67` `docs/SIDECAR_CONTEXT_AUTH.md` | 2-3d |
+| 3 | **Rate limiting** — add `slowapi` or equivalent. Per-IP and per-endpoint limits, especially on `/invoke` and `/stream`. | `agent_engine/api/app.py` `agent_manager/api/app.py` | 1d |
+| 4 | **Security headers** — add middleware for `X-Content-Type-Options`, `X-Frame-Options`, `Strict-Transport-Security`, `Referrer-Policy`. | Both API app files | 2h |
+| 5 | **Tighten CORS** — restrict `allow_methods` and `allow_headers` to specific values instead of `*`. | `agent_manager/api/web.py:24-25` | 30m |
+
+**Definition of done:** All endpoints require valid credentials. `protected: true` nodes reject unauthenticated callers. `make check` passes.
+
+---
+
+## Sprint 2 — Input Hardening (1 week)
+
+Defense-in-depth.
+
+| # | Task | Files | Effort |
+|---|------|-------|--------|
+| 1 | **Server timeouts** — configure uvicorn `timeout_keep_alive` and request timeouts. | `agentctl/main.py:208` `agent_manager/cli.py:38` | 2h |
+| 2 | **SSE disconnect detection** — abort engine processing when client disconnects mid-stream. | `agent_engine/api/app.py:264-303` `agent_manager/api/routes.py:91-119` | 1d |
+| 3 | **Path validation** — verify `import_from_path()` resolves within the expected `plugins/` directory. | `loaders/_import.py:22-29` | 2h |
+| 4 | **Dependency lockfile** — generate `uv.lock` or `pip-compile` output for reproducible builds. | `pyproject.toml` | 1h |
+| 5 | **Log redaction** — add a redaction pass in `StructuredFormatter` for values matching secret patterns at DEBUG level. | `logging_config.py` `observability/providers/logging/provider.py` | 4h |
+
+**Definition of done:** Timeouts configured. SSE cleans up on disconnect. Lockfile committed. `make check` passes.
+
+---
+
+## Sprint 3 — Production Readiness (ongoing)
+
+Operational security.
+
+| # | Task | Files | Effort |
+|---|------|-------|--------|
+| 1 | **TLS docs** — document that TLS must be handled by reverse proxy. Add `--ssl-keyfile`/`--ssl-certfile` options to `agentctl serve`. | `agentctl/main.py` `docs/` | 1d |
+| 2 | **Health check** — add `GET /healthz` that returns 200 when engine is ready. Add `HEALTHCHECK` to Dockerfile. | `agent_engine/api/app.py` `Dockerfile` | 2h |
+| 3 | **Audit logging** — log all authentication failures, approval decisions, and protected-node access attempts. | `agent_engine/api/app.py` `engine/langgraph/filters.py` | 1d |
+| 4 | **Pentest checklist** — create a doc with test cases for an external security review. | `docs/SECURITY_PENTEST.md` | 4h |
+
+**Definition of done:** Deployment docs include TLS guidance. Docker image has health check. Audit log covers sensitive operations.
+
+---
+
+## Appendix: What's Already Done Well
+
+| Practice | Location |
+|----------|----------|
+| YAML secret scanning (keys + credential shapes) | `parsers/yaml/parser.py:483-498` |
+| Sensitive argument masking in HITL approvals | `approvals/sanitization.py` |
+| CORS default deny (empty origins list) | `agent_manager/config.py:33` |
+| Request ID sanitization (regex + 64-char cap) | `agent_engine/api/app.py:44-56` |
+| MCP auth headers never logged | `runtime/hooks/mcp.py:81-85` |
+| Execution limits per run (cost guardrails) | `runtime/execution.py` |
+| Access filter fail-closed on exceptions | `engine/langgraph/filters.py:51-67` |
+| `.env` files gitignored | `.gitignore` |
+| SQL injection prevention via ORM | `infrastructure/persistence/` |

src/agent_engine/api/app.py

@@ -14,7 +14,7 @@
 
 from fastapi import FastAPI, Header, HTTPException, Response
 from fastapi.responses import StreamingResponse
-from pydantic import BaseModel
+from pydantic import BaseModel, Field
 
 from agent_engine.approvals.decision import ApprovalDecision, parse_decision
 from agent_engine.approvals.errors import (
@@ -106,7 +106,7 @@ def _map_approval_error(exc: ApprovalError) -> HTTPException:
 
 
 class InvokeRequest(BaseModel):
-    message: str
+    message: str = Field(max_length=65536)
 
 
 class ToolRecord(BaseModel):
@@ -240,7 +240,7 @@ async def invoke(
             )
         except Exception as exc:
             log(logger, logging.ERROR, "request end", status="error", error=str(exc))
-            raise HTTPException(status_code=500, detail=str(exc)) from exc
+            raise HTTPException(status_code=500, detail="internal server error") from exc
         log(
             logger,
             logging.INFO,
@@ -294,7 +294,7 @@ async def event_stream():
                 )
             except Exception as exc:
                 log(logger, logging.ERROR, "request end", status="error", error=str(exc))
-                yield f"data: {json.dumps({'type': 'error', 'detail': str(exc)})}\n\n"
+                yield f"data: {json.dumps({'type': 'error', 'detail': 'internal server error'})}\n\n"
 
         return StreamingResponse(
             event_stream(),
@@ -329,7 +329,7 @@ async def _decide(
         except ApprovalError as exc:
             raise _map_approval_error(exc) from exc
         except Exception as exc:
-            raise HTTPException(status_code=500, detail=str(exc)) from exc
+            raise HTTPException(status_code=500, detail="internal server error") from exc
         return InvokeResponse(
             system_name=result.system_name,
             answer=result.answer,

src/agent_manager/api/routes.py

@@ -4,6 +4,7 @@
 
 import dataclasses
 import json
+import logging
 from collections.abc import AsyncIterator
 from typing import Annotated
 
@@ -28,6 +29,7 @@
 )
 
 router = APIRouter()
+logger = logging.getLogger(__name__)
 
 Service = Annotated[ConversationService, Depends(get_service)]
 
@@ -61,7 +63,8 @@ async def send_message(
     except ConversationTokenBudgetExceeded:
         raise HTTPException(status_code=429, detail="conversation token budget exceeded") from None
     except Exception as exc:  # engine failure
-        raise HTTPException(status_code=500, detail=str(exc)) from exc
+        logger.exception("send_message failed")
+        raise HTTPException(status_code=500, detail="internal server error") from exc
     return SendMessageResponse(
         answer=result.answer,
         visited=list(result.visited),
@@ -112,7 +115,8 @@ async def event_source() -> AsyncIterator[str]:
                 payload = _to_stream_event(event).model_dump(exclude_none=True)
                 yield f"event: {event.type}\ndata: {json.dumps(payload)}\n\n"
         except Exception as exc:
-            yield f"event: error\ndata: {json.dumps({'type': 'error', 'error': str(exc)})}\n\n"
+            logger.exception("stream_message failed")
+            yield f"event: error\ndata: {json.dumps({'type': 'error', 'error': 'internal server error'})}\n\n"
         finally:
             yield "event: done\ndata: [DONE]\n\n"
 

src/agent_manager/api/schemas.py

@@ -8,14 +8,14 @@
 
 from datetime import datetime
 
-from pydantic import BaseModel
+from pydantic import BaseModel, Field
 
 from agent_manager.domain import Role
 
 
 class CreateConversationRequest(BaseModel):
-    session_id: str | None = None
-    user_id: str | None = None
+    session_id: str | None = Field(default=None, max_length=128)
+    user_id: str | None = Field(default=None, max_length=128)
 
 
 class CreateConversationResponse(BaseModel):
@@ -30,8 +30,8 @@ class MessageOut(BaseModel):
 
 
 class SendMessageRequest(BaseModel):
-    message: str
-    user_id: str | None = None
+    message: str = Field(max_length=65536)
+    user_id: str | None = Field(default=None, max_length=128)
 
 
 class ToolRecord(BaseModel):