Add portal dry-run submit and mTLS API auth

docs/deploy/key-management.md

@@ -7,7 +7,9 @@ This guide covers the secrets used by `result_server/app.py`.
 Production deployments must provide:
 
 - `FLASK_SECRET_KEY`: at least 32 characters, generated randomly.
-- `RESULT_SERVER_KEYS`: one or more runner-scoped ingest keys.
+- `RESULT_SERVER_KEYS`: one or more runner-scoped ingest keys, or
+  `RESULT_SERVER_TRUSTED_PROXY_AUTH=mtls` when nginx verifies client
+  certificates before proxying ingest/query API requests.
 
 Use runner-scoped server keys instead of the legacy server-side
 `RESULT_SERVER_KEY` fallback:
@@ -17,20 +19,49 @@ RESULT_SERVER_KEYS=runner-a:<RUNNER_A_KEY>,runner-b:<RUNNER_B_KEY>
 ```
 
 `RESULT_SERVER_KEYS` is the server-side registry of accepted posting/query
-keys. It is intentionally broader than the current single-key CI setup so that
-the portal can later accept results from multiple trusted CI sources, such as
-the main BenchKit CI, site-managed runners, collaborator forks, or
-estimator-only pipelines.
-
-Each client job still receives a single `RESULT_SERVER_KEY` secret for its own
-uploads. This client-side key must match one entry in `RESULT_SERVER_KEYS`, and
-it is typically injected through GitLab CI/CD variables or another CI secret
-mechanism rather than stored on the runner host.
+keys for deployments that still use shared API keys. Client jobs in mTLS mode
+do not use `RESULT_SERVER_KEY` and do not send an `X-API-Key` header.
 
 Each key must be at least 32 characters and must not use known insecure
 examples such as `dev-api-key`, `changeme`, or `secret`. The production app
 refuses to start when these checks fail.
 
+## Client Certificate Mode
+
+Deployments can avoid shared ingest keys by terminating TLS at a trusted reverse
+proxy and requiring a client certificate for result API endpoints. Configure the
+portal with:
+
+```text
+RESULT_SERVER_TRUSTED_PROXY_AUTH=mtls
+```
+
+In this mode `RESULT_SERVER_KEYS` and the legacy `RESULT_SERVER_KEY` may be
+empty, provided nginx verifies the client certificate and forwards these headers
+only to the local Flask backend:
+
+```nginx
+proxy_set_header X-Result-Server-Client-Verify $ssl_client_verify;
+proxy_set_header X-Result-Server-Client-DN $ssl_client_s_dn;
+proxy_set_header X-Result-Server-Client-Fingerprint $ssl_client_fingerprint;
+```
+
+The nginx location must reject requests unless `$ssl_client_verify` is
+`SUCCESS`. Keep the backend bound to loopback or a Unix socket so clients cannot
+bypass nginx and provide these headers themselves.
+
+CI jobs can use host-managed certificates instead of GitLab CI/CD secret
+variables by mounting them read-only into a self-managed runner container and
+setting:
+
+```text
+RESULT_SERVER_CLIENT_CERT=/run/benchkit/result-server/client.crt
+RESULT_SERVER_CLIENT_KEY=/run/benchkit/result-server/client.key
+```
+
+The upload/query helper scripts use these variables automatically and do not
+send an `X-API-Key` header.
+
 ## Generation
 
 Generate random values with a local secret generator, for example:

docs/guides/developer-reference.md

@@ -250,7 +250,7 @@ For production portal deployments:
 - `app.py` binds to `127.0.0.1:8800` by default; set `RESULT_SERVER_HOST` and `RESULT_SERVER_PORT` explicitly when the deployment requires a different bind address.
 - Set runner-scoped ingest keys with `RESULT_SERVER_KEYS=runner-a:<RUNNER_A_KEY>,runner-b:<RUNNER_B_KEY>`.
 - `RESULT_SERVER_KEYS` is the server-side registry of accepted posting/query keys. It is intentionally broader than the current single-key CI setup so the portal can later accept results from multiple trusted CI sources such as main BenchKit CI, site-managed runners, collaborator forks, or estimator-only pipelines.
-- Each client job still receives a single `RESULT_SERVER_KEY` secret for its own upload/query operations, usually through GitLab CI/CD variables or another CI secret store. That client-side key must match one entry in server-side `RESULT_SERVER_KEYS`.
+- Client jobs on mTLS-protected deployments use `RESULT_SERVER_CLIENT_CERT` and `RESULT_SERVER_CLIENT_KEY` instead of `RESULT_SERVER_KEY`; they do not send an `X-API-Key` header.
 - `FLASK_SECRET_KEY` and each ingest key must be at least 32 characters and must not use known insecure examples such as `dev-api-key`, `changeme`, or `secret`; production startup refuses these values.
 - The legacy server-side `RESULT_SERVER_KEY` variable is still accepted as runner `default` for compatibility, but production portal deployments should rotate the accepted-key registry to `RESULT_SERVER_KEYS`.
 - See `docs/deploy/key-management.md` for generation and rotation guidance.

result_server/app.py

@@ -54,8 +54,14 @@ def _configure_redis(app, prefix):
 
     redis_url = os.environ.get("REDIS_URL", "redis://localhost:6379/0")
     app.config["REDIS_CONN"] = redis.from_url(redis_url, decode_responses=True)
-    app.config["REDIS_PREFIX"] = "dev:" if prefix == "/dev" else "main:"
-    app.config["SESSION_COOKIE_NAME"] = "session_dev" if prefix == "/dev" else "session_main"
+    app.config["REDIS_PREFIX"] = os.environ.get(
+        "RESULT_SERVER_REDIS_PREFIX",
+        "dev:" if prefix == "/dev" else "main:",
+    )
+    app.config["SESSION_COOKIE_NAME"] = os.environ.get(
+        "RESULT_SERVER_SESSION_COOKIE_NAME",
+        "session_dev" if prefix == "/dev" else "session_main",
+    )
     app.config["AUTH_REQUIRES_REDIS"] = True
 
 
@@ -109,6 +115,14 @@ def _configure_admin_policy(app):
     )
 
 
+def _configure_api_auth(app):
+    """Configure API authentication modes accepted behind the reverse proxy."""
+    app.config["TRUSTED_PROXY_AUTH"] = os.environ.get(
+        "RESULT_SERVER_TRUSTED_PROXY_AUTH",
+        "",
+    ).strip()
+
+
 def _configure_execution_profiles(app, base_dir):
     """Configure the site-local execution profile database path."""
     app.config["EXECUTION_PROFILE_DB_PATH"] = os.environ.get(
@@ -153,6 +167,7 @@ def create_app(prefix="", base_dir=None):
     _configure_result_directories(app, base_dir)
     _configure_upload_limits(app)
     _configure_admin_policy(app)
+    _configure_api_auth(app)
     _configure_execution_profiles(app, base_dir)
     init_csrf(app, exempt_blueprints=(api_bp,))
 

result_server/routes/admin.py

@@ -26,6 +26,7 @@
     load_execution_profiles,
     normalize_profile,
 )
+from utils.gitlab_pipeline import build_pipeline_plan, configured_gitlab_repo
 from utils.rate_limit import rate_limited
 from utils.user_store import get_user_store
 
@@ -122,6 +123,10 @@ def _parse_execution_profile_form():
     return raw_profile, errors
 
 
+def _parse_bool_form(name):
+    return request.form.get(name) == "on"
+
+
 def _user_affiliations(store, email):
     """Return the affiliations for a user, handling missing records uniformly."""
     if hasattr(store, "get_user"):
@@ -173,6 +178,7 @@ def execution_profiles():
     return render_template(
         "admin_execution_profiles.html",
         profile_result=profile_result,
+        dry_run_result=None,
     )
 
 
@@ -230,6 +236,88 @@ def upsert_execution_profile():
     return redirect(url_for("admin.execution_profiles"))
 
 
+@admin_bp.route("/execution-profiles/dry-run-submit", methods=["POST"])
+@admin_required
+@rate_limited(max_per_minute=20, key_fn=_admin_rate_key, scope="admin_write")
+def dry_run_execution_profile_submit():
+    """Resolve an execution profile and render a GitLab Pipeline API dry run."""
+    db_path = current_app.config.get("EXECUTION_PROFILE_DB_PATH")
+    store = ExecutionProfileStore(db_path)
+    target_ref = request.form.get("target_ref", "").strip() or "develop"
+    profile_id = request.form.get("profile_id", "").strip()
+    code = request.form.get("code", "").strip()
+    system = request.form.get("system", "").strip()
+    exp = request.form.get("exp", "").strip()
+    app = request.form.get("app", "").strip()
+    benchpark = _parse_bool_form("benchpark")
+    park_only = _parse_bool_form("park_only")
+    park_send = _parse_bool_form("park_send")
+
+    resolve_result = store.resolve_profile(
+        profile_id=profile_id,
+        code=code,
+        system=system,
+        exp=exp,
+    )
+    profile = resolve_result.profile
+    plan = build_pipeline_plan(
+        gitlab_repo=configured_gitlab_repo(),
+        target_ref=target_ref,
+        code=code,
+        system=system,
+        app=app,
+        benchpark=benchpark,
+        park_only=park_only,
+        park_send=park_send,
+        scheduler_extra_args=resolve_result.scheduler_extra_args,
+    )
+    errors = resolve_result.errors + plan.errors
+    status = "dry_run_ready" if not errors else "dry_run_blocked"
+    request_id = store.create_execution_request(
+        request_type="gitlab_pipeline",
+        status=status,
+        dry_run=True,
+        profile_id=profile["id"] if profile else profile_id,
+        target_ref=target_ref,
+        code=code,
+        system=system,
+        exp=exp,
+        payload={"api_url": plan.api_url, "payload": plan.payload},
+        errors=errors,
+        actor=session.get("user_email", ""),
+    )
+
+    audit_event(
+        "admin_execution_profile_submit_dry_run",
+        actor=session.get("user_email"),
+        target=profile["id"] if profile else profile_id,
+        result="success" if not errors else "failure",
+        details={
+            "request_id": request_id,
+            "target_ref": target_ref,
+            "code": code,
+            "system": system,
+            "exp": exp,
+            "errors": errors,
+        },
+    )
+
+    profile_result = load_execution_profiles(db_path)
+    return render_template(
+        "admin_execution_profiles.html",
+        profile_result=profile_result,
+        dry_run_result={
+            "request_id": request_id,
+            "status": status,
+            "profile": profile,
+            "api_url": plan.api_url,
+            "payload_json": json.dumps(plan.payload, indent=2, sort_keys=True),
+            "errors": errors,
+            "warnings": plan.warnings,
+        },
+    )
+
+
 @admin_bp.route("/users/add", methods=["POST"])
 @admin_required
 @rate_limited(max_per_minute=20, key_fn=_admin_rate_key, scope="admin_write")

result_server/routes/api.py

@@ -12,7 +12,7 @@
 import tempfile
 from datetime import datetime
 
-from utils.auth import verify_ingest_key
+from utils.auth import verify_ingest_key, verify_trusted_proxy_auth
 from utils.audit_logging import audit_event
 from utils.rate_limit import rate_limited
 
@@ -28,19 +28,29 @@
 def require_api_key():
     """Validate the request API key and return the authenticated runner id."""
     runner_id = verify_ingest_key(request.headers.get("X-API-Key", ""))
+    auth_method = "api_key"
+    if not runner_id:
+        runner_id = verify_trusted_proxy_auth(request.headers)
+        auth_method = "trusted_proxy"
     if not runner_id:
         audit_event(
             "api_auth_failed",
             result="failure",
             level=logging.WARNING,
-            details={"reason": "invalid_api_key"},
+            details={"reason": "invalid_api_key_or_proxy_auth"},
         )
         abort(401, description="Invalid API Key")
-    audit_event("api_auth_success", actor=runner_id, result="success")
+    audit_event(
+        "api_auth_success",
+        actor=runner_id,
+        result="success",
+        details={"auth_method": auth_method},
+    )
     current_app.logger.info(
-        "api key accepted",
+        "api auth accepted",
         extra={
             "runner_id": runner_id,
+            "auth_method": auth_method,
             "endpoint": request.path,
             "ip": request.remote_addr,
         },
@@ -50,7 +60,11 @@ def require_api_key():
 
 def _api_rate_key(req):
     """Return the runner-scoped API rate-limit key for a request."""
-    runner_id = verify_ingest_key(req.headers.get("X-API-Key", "")) or "unknown"
+    runner_id = (
+        verify_ingest_key(req.headers.get("X-API-Key", ""))
+        or verify_trusted_proxy_auth(req.headers)
+        or "unknown"
+    )
     return f"runner:{runner_id}"
 
 

result_server/templates/admin_execution_profiles.html

@@ -93,6 +93,17 @@
     }
     .btn-primary { background-color: #0f766e; color: #fff; border-color: #0f766e; }
     .btn-primary:hover { background-color: #0b5f59; color: #fff; }
+    .btn-secondary { background-color: #1f2937; color: #fff; border-color: #1f2937; }
+    .btn-secondary:hover { background-color: #111827; color: #fff; }
+    .profile-dry-run-output {
+        margin-top: 14px;
+        padding: 12px;
+        overflow-x: auto;
+        border: 1px solid #d8e2e8;
+        border-radius: 8px;
+        background: #0f172a;
+        color: #e2e8f0;
+    }
 </style>
 
 {% with messages = get_flashed_messages() %}
@@ -237,6 +248,88 @@ <h2 class="section-title">Create / Update Profile</h2>
     </form>
 </section>
 
+<section class="page-card">
+    <h2 class="section-title">GitLab Pipeline Dry Run</h2>
+    <p class="section-intro">
+        Resolve an approved profile for a target scope and preview the GitLab
+        Pipeline API request. This does not submit a pipeline.
+    </p>
+    <form method="POST" action="{{ url_for('admin.dry_run_execution_profile_submit') }}" class="profile-form">
+        {% if csrf_token is defined %}<input type="hidden" name="csrf_token" value="{{ csrf_token() }}">{% endif %}
+        <label>
+            Target Ref
+            <input type="text" name="target_ref" value="develop" required>
+        </label>
+        <label>
+            Profile ID
+            <input type="text" name="profile_id" placeholder="optional explicit profile">
+        </label>
+        <label>
+            Code
+            <input type="text" name="code" placeholder="qws">
+        </label>
+        <label>
+            System
+            <input type="text" name="system" placeholder="RIKYU">
+        </label>
+        <label>
+            Exp
+            <input type="text" name="exp" placeholder="case0">
+        </label>
+        <label>
+            BenchPark App
+            <input type="text" name="app" placeholder="osu-micro-benchmarks">
+        </label>
+        <div class="profile-form-actions">
+            <label class="profile-check-label">
+                <input type="checkbox" name="benchpark">
+                benchpark
+            </label>
+            <label class="profile-check-label">
+                <input type="checkbox" name="park_only">
+                park_only
+            </label>
+            <label class="profile-check-label">
+                <input type="checkbox" name="park_send">
+                park_send
+            </label>
+            <button type="submit" class="btn btn-secondary">Preview Payload</button>
+        </div>
+    </form>
+
+    {% if dry_run_result %}
+    <div class="inline-notice">
+        <strong>Dry-run request #{{ dry_run_result.request_id }}:</strong>
+        {{ dry_run_result.status }}
+        {% if dry_run_result.profile %}
+        using profile <span class="profile-mono">{{ dry_run_result.profile.id }}</span>
+        {% endif %}
+    </div>
+    {% if dry_run_result.errors %}
+    <div class="inline-notice">
+        <strong>Dry-run blockers:</strong>
+        <ul>
+            {% for error in dry_run_result.errors %}
+            <li>{{ error }}</li>
+            {% endfor %}
+        </ul>
+    </div>
+    {% endif %}
+    {% if dry_run_result.warnings %}
+    <div class="inline-notice">
+        <strong>Dry-run warnings:</strong>
+        <ul>
+            {% for warning in dry_run_result.warnings %}
+            <li>{{ warning }}</li>
+            {% endfor %}
+        </ul>
+    </div>
+    {% endif %}
+    <p><strong>API URL:</strong> <span class="profile-mono">{{ dry_run_result.api_url or 'not configured' }}</span></p>
+    <pre class="profile-dry-run-output">{{ dry_run_result.payload_json }}</pre>
+    {% endif %}
+</section>
+
 <section class="page-card table-card">
     <h2 class="section-title">Registered Profiles</h2>
     <div class="table-wrap">