Skip to content

Add daily Sentinel v2.4 operational-check script, runbook, and tests #142

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
OneFineStarstuff wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Add daily Sentinel v2.4 operational-check script, runbook, and tests #142

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
3fe97bd
fix: harden daily sentinel evidence checks
OneFineStarstuff Jul 6, 2026
20a9723
Merge branch 'main' into codex/perform-devsecops-operational-checks
OneFineStarstuff Jul 6, 2026
fb1237c
Merge branch 'main' into codex/perform-devsecops-operational-checks
OneFineStarstuff Jul 6, 2026
9207200
Merge branch 'main' into codex/perform-devsecops-operational-checks
OneFineStarstuff Jul 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
95 changes: 95 additions & 0 deletions docs/reports/DAILY_SENTINEL_V24_DEVSECOPS_RUNBOOK.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
# Daily Sentinel AI v2.4 DevSecOps Operational Check Runbook

This runbook defines the daily evidence pack used to validate the Omni-Sentinel Cognitive Execution Environment for G-SIFI and Fortune 500 financial-services deployments.

## Required checks

1. **Sentinel telemetry/dashboard health** — verify the Sentinel v2.4 dashboard or API endpoint returns a 2xx health result. The default endpoint is `https://sentinel.internal.g-sifi.local`.
2. **Global Systemic Risk Index (G-SRI)** — verify a signed, fresh G-SRI evidence bundle reports `current_value < threshold` under the approved risk-appetite policy version.
3. **PQC WORM audit batches** — verify `pqc_worm_logger.py` committed the latest audit batch to the designated S3 Object Lock bucket, including object version, Object Lock mode, retention-until timestamp, Merkle root, PQC signature algorithm, and commit-lag SLO.
4. **TEE/TPM attestation** — verify fresh attestation evidence reports `PCR_MATCH=true`, trusted TEE status, node identity, PCR policy hash, TPM quote ID, and verifier identity.

## Evidence JSON fields

### G-SRI evidence

```json
{
"timestamp_utc": "2026-05-29T12:00:00Z",
"current_value": 0.41,
"threshold": 0.70,
"policy_version": "gsri-risk-appetite-2026.05",
"signed": true
}
```

### WORM evidence

```json
{
"timestamp_utc": "2026-05-29T12:00:00Z",
"logger_name": "pqc_worm_logger.py",
"batch_status": "committed",
"bucket": "sentinel-object-lock-audit",
"object_key": "audit/2026/05/29/batch-0001.jsonl",
"object_version_id": "3HL4kqtJlcpXrof3vjVBH40Nrjfkd",
"object_lock_mode": "COMPLIANCE",
"retention_until_utc": "2033-05-29T12:00:00Z",
"merkle_root": "sha256:abc123",
"pqc_signature_alg": "ML-DSA-65",
"pqc_signature_verified": true,
"commit_lag_seconds": 42
}
```

### TEE/TPM attestation evidence

```json
{
"timestamp_utc": "2026-05-29T12:00:00Z",
"PCR_MATCH": true,
"tee_status": "trusted",
"node_id": "sentinel-prod-01",
"pcr_policy_hash": "sha256:def456",
"tpm_quote_id": "quote-20260529-0001",
"verifier_id": "attestor-v2.4.3",
"attestation_signature_verified": true
}
```

## CLI usage

Run a live check with dashboard/API reachability and the designated Object Lock bucket:

```bash
python -m scripts.daily_sentinel_operational_check \
--gsri-evidence /secure/evidence/gsri.json \
--worm-evidence /secure/evidence/worm.json \
--expected-worm-bucket sentinel-object-lock-audit \
--attestation-evidence /secure/evidence/attestation.json
```

Run an offline evidence-pack check when the dashboard network is intentionally unavailable:

```bash
python -m scripts.daily_sentinel_operational_check \
--skip-dashboard \
--gsri-evidence /secure/evidence/gsri.json \
--worm-evidence /secure/evidence/worm.json \
--attestation-evidence /secure/evidence/attestation.json
```

Use `--json` for machine-readable CI output. The JSON payload includes `generated_at_utc`, `overall_status`, and a `results` array so CI can route each failed check to the correct control owner. Use `--require-compliance-object-lock` when governance policy forbids S3 Object Lock GOVERNANCE mode for the audit bucket.

## Fail-closed rules

Treat the daily control state as **RED** when any of the following conditions occurs:

- Sentinel dashboard/API health is not 2xx and the outage is not covered by an approved maintenance window.

Check notice on line 88 in docs/reports/DAILY_SENTINEL_V24_DEVSECOPS_RUNBOOK.md

View check run for this annotation

Codacy Production / Codacy Static Code Analysis

docs/reports/DAILY_SENTINEL_V24_DEVSECOPS_RUNBOOK.md#L88 

Expected: 80; Actual: 107
- G-SRI evidence is missing, stale, unsigned, or at/above the policy threshold.
- `pqc_worm_logger.py` has not committed the latest WORM batch, the bucket does not match the designated Object Lock bucket, Object Lock metadata is missing, retention is expired, the PQC signature is not verified, or commit lag exceeds the SLO.
- TEE/TPM attestation is stale, `PCR_MATCH` is not true, TEE status is not trusted/verified, or the attestation signature is not verified.
Comment on lines +86 to +91

@coderabbitai coderabbitai Bot Jul 6, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Locate the runbook and any references to maintenance windows or the checker contract.
git ls-files 'docs/reports/DAILY_SENTINEL_V24_DEVSECOPS_RUNBOOK.md' \
  | xargs -r -I{} sh -c 'echo "### {}"; wc -l "{}"; echo'
echo "### Search for maintenance window references"
rg -n --hidden --no-ignore-vcs -S "maintenance window|maintenance-window|approved maintenance|checker contract|fail-closed|Sentinel dashboard/API health|RED when any" docs . || true

Repository: OneFineStarstuff/OneFineStarstuff.github.io

Length of output: 50399

🏁 Script executed:

#!/bin/bash
set -euo pipefail

file='docs/reports/DAILY_SENTINEL_V24_DEVSECOPS_RUNBOOK.md'

echo "### Runbook excerpt"
sed -n '70,105p' "$file" | cat -n

echo
echo "### Maintenance-window references near Sentinel docs"
rg -n -S "maintenance window|maintenance-window|approved maintenance|Sentinel dashboard/API health|checker contract|daily control state|RED when any" \
  docs/reports rag-agentic-dashboard --glob '!**/*.html' --glob '!**/node_modules/**' || true

echo
echo "### Contract-like references in the Sentinel generator"
rg -n -S "dashboard/API health|maintenance|checker|contract|fail-closed" rag-agentic-dashboard/gen-sentinel-ai-v24-governance.py || true

Repository: OneFineStarstuff/OneFineStarstuff.github.io

Length of output: 4355

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "### Locate the checker implementation"
rg -n -S "daily_sentinel_operational_check|skip-dashboard|require-compliance-object-lock|maintenance window|maintenance-window|overall_status|results array" \
  . --glob '!**/*.html' --glob '!**/node_modules/**' --glob '!**/dist/**' --glob '!**/build/**' || true

echo
echo "### Likely Python entrypoints"
fd -HI 'daily_sentinel_operational_check*|*operational_check*|*sentinel*check*' . || true

Repository: OneFineStarstuff/OneFineStarstuff.github.io

Length of output: 8171

🏁 Script executed:

#!/bin/bash
set -euo pipefail

file='scripts/daily_sentinel_operational_check.py'

echo "### CLI and dashboard-related functions"
ast-grep outline "$file" --view expanded

echo
echo "### Relevant slices"
sed -n '1,220p' "$file" | cat -n
echo
sed -n '220,470p' "$file" | cat -n

Repository: OneFineStarstuff/OneFineStarstuff.github.io

Length of output: 22050

Drop the maintenance-window exception unless the checker enforces it. The CLI only exposes --skip-dashboard; there’s no maintenance-window input or enforcement path, so this line suggests a bypass operators can’t actually use.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/reports/DAILY_SENTINEL_V24_DEVSECOPS_RUNBOOK.md` around lines 86 - 91,
Remove the maintenance-window exception from the RED-state criteria in the
runbook unless there is an actual enforcement path in the checker; the current
CLI only exposes `--skip-dashboard`, so update the guidance in the affected
section to match what the checker can truly evaluate and keep the remaining
Sentinel health, G-SRI, WORM, and TEE/TPM conditions unchanged.


## Regulatory mapping

The daily evidence pack supports EU AI Act technical documentation and systemic-risk GPAI monitoring, NIST AI RMF 1.0 and AI 600-1 operational risk management, ISO/IEC 42001 AIMS monitoring evidence, SR 11-7/SR 26-2 model-risk governance, DORA and NIS2 operational-resilience evidence, GDPR accountability, and Basel operational-risk oversight.
1 change: 1 addition & 0 deletions pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,3 +31,4 @@ governance = ["jsonschema"]

[project.scripts]
validate-gsifi-governance-assets = "scripts.validate_gsifi_governance_assets:main"
daily-sentinel-operational-check = "scripts.daily_sentinel_operational_check:main"
Loading
Loading