Skip to content

Add risk windows, exposure series and trust-center projections (long tail, part 2) #44

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
therandomsecurityguy merged 1 commit into from
Oct 28, 2026
Merged

Add risk windows, exposure series and trust-center projections (long tail, part 2) #44

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 26 additions & 11 deletions ARCHITECTURE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -198,17 +198,32 @@ graph, and the top row is the cheapest fix for the most (or worst) paths.

## Trajectory forecasting

The rule runner writes three more daily series beside the issue counts: the
backlog's total risk score (a few criticals weigh more than many lows), mean
time to resolution over issues resolved in the last 30 days, and the
admin-principal count from the policy evaluator's findings. `GET
/trends/:metric` fits an explainable least-squares line over the window and
returns 7/30-day projections with a residual band and a confidence level
derived from fit quality and point count — degraded a level, with the reason in
`note`, when no runner pass recorded in two days or the window has three or
more missing days. The Trends page renders the projection beside the series.
Publishing projections to the trust center is deferred until the publishable
discipline covers them.
The rule runner writes daily series beside the issue counts: the backlog's
total risk score (a few criticals weigh more than many lows), mean time to
resolution over issues resolved in the last 30 days, the admin-principal count
from the policy evaluator's findings, **SLO burn** (open findings past 80% of
their severity window — critical 7 days, high 30, medium 90, low 180 — derived
from creation dates the run already reads, no extra query) and
**exposure-by-type** (a bounded label-only read of internet-exposed vertices,
capped at 2000 with an explicit truncation flag). `GET /trends/:metric` fits
an explainable least-squares line over the window and returns 7/30-day
projections with a residual band and a confidence level derived from fit
quality and point count — degraded a level, with the reason in `note`, when no
runner pass recorded in two days or the window has three or more missing days.
The Trends page renders the projection beside the series, and each open issue
carries its own window state (`GET /issues` computes burn, `burning` and
`breached` on read from the severity window).

**Projections publish with the posture**: while the reconciliation record is
`OK`, `GET /compliance/reconciliation/publishable` attaches compact
projections (metric id, per-day slope, confidence, basis and horizon, the
30-day projection) for open findings, backlog risk score and SLO burn — the
same forecaster, so the page and the API never disagree. STALE or MISSING
publishes nothing, and unreadable trends omit the projections without failing
the response. The sync layer owns the public wording: it maps the fixed metric
ids to reviewed labels and drops anything unknown, and the sanitization
tripwires still scan the whole fragment, so the public page shows reviewed
strings beside plain numbers only.

## Rule quality and calibration

Expand Down
12 changes: 12 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,18 @@

All notable changes to Khalifa.

## [Unreleased] - Risk windows, exposure series and trust-center projections (long tail, part 2)

### Added

- **Per-issue risk windows**: a new engine module (`slo.ts`) sets an SLA window per severity (critical 7 days, high 30, medium 90, low 180) and derives, deterministically from `createdAt`, the age, burn (`age / window`), state (`ok` / `burning` at 80% / `breached` past the window) and days-to-breach. Open issues expose it on read (`GET /issues`, `GET /issues/:id` — no stored state), the Issues list marks burning and breached findings, and the issue page's Status card states the window.
- **Two new trend series**: `sloBurning` (open findings past 80% of their window, split by severity plus the breached count — derived from the open issues the snapshot already reads, no extra query) and `exposedByType` (internet-exposed vertices grouped by resource type — one bounded, label-only read capped at 2000 with an explicit truncation flag; the cost objection answered by bound, not by skipping the series). Both flow through the trends route with sanitized samples and appear on the Trends page.
- **Trust-center projections**: while the reconciliation record is `OK`, `GET /compliance/reconciliation/publishable` attaches compact series projections (metric id, per-day slope, confidence, basis/horizon days, 30-day value) for open findings, backlog risk score and SLO burn, sharing one forecaster with the trends route (extracted to `services/trend-forecast.ts`). The sync layer maps the fixed metric ids to **reviewed labels** (nothing else can reach the page), drops unknown metrics and out-of-range values, and the sanitization tripwires scan the whole fragment; the rendered page gains a "Where the posture is heading" block. STALE/MISSING publishes nothing and unreadable trends omit projections without failing the response.

### Notes

- This completes the predictive completion plan (`~/Documents/Testing/plans/2026-10-27-predictive-completion.md`); the one deliberate remainder is ASN attribution, deferred with its reason recorded (external dataset + licensing).

## [Unreleased] - UEBA breadth and EPSS/KEV enrichment (long tail, part 1)

### Added
Expand Down
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ One member role. Evaluation stays in-account.
| **Detection** | GuardDuty detector coverage per region (including `MISSING`), Security Hub findings joined to internet-exposed resources (RULE-017, RULE-018), CloudTrail anomaly signals: root activity, secret reads from IPs outside each principal's own 30-day baseline, AccessDenied deviations from each principal's own daily baseline (RULE-020 to RULE-022), canary credential tripwires (RULE-023), secret scanning of Lambda environment variables plus IMDSv1-with-role detection (RULE-024), and Amazon Inspector CVEs on internet-exposed workloads (RULE-003). |
| **Exposure** | Public EKS control planes (RULE-019) plus public S3/RDS and world-open security groups, derived from collected endpoint and network config, with optional observed traffic from VPC flow logs confirming where traffic actually goes. |
| **Compliance** | CIS AWS Foundations v3.0 and the Khalifa baseline, per-control evidence, mapped SOC 2 / ISO 27001 / PCI DSS / NIST 800-53 requirements, and CIS reconciliation against Security Hub before anything is publishable. |
| **Trust center** | Nightly static trust-center page built from the publishable posture (statuses only, in-repo site). |
| **Trust center** | Nightly static trust-center page built from the publishable posture: framework statuses plus straight-line series projections while the reconciliation record is OK (reviewed labels, numbers only, in-repo site). |
| **Runtime** | EKS Helm + `khalifa-eks` / `khalifa-data` / `khalifa-registry`. CDK Lambda remains for the in-repo pipeline. |
| **Shift-left** | `npm run risk:plan -- <plan.json>` predicts which rules a Terraform plan would feed (world-open security groups, account-root trusts, public buckets/RDS/EKS endpoints, secret policies, function URLs, IMDSv1) and fails the gate, naming the rule and the predicted path before anything is deployed. |

Expand Down
70 changes: 70 additions & 0 deletions api-service/src/__tests__/trust-center.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import {
getPublishableRequirements,
publishableFrom,
setReconciliationStoreForTests,
setTrendStoreForTests,
} from '../routes/trust-center';
import { requireTrustCenterSync } from '../middleware/rbac';

Expand Down Expand Up @@ -173,6 +174,75 @@ describe('GET /compliance/reconciliation/publishable', () => {
expect(res.body.status).toBe('OK');
expect(res.body.requirements).toHaveLength(1);
});

function thirtyPoints(metric: string, start: number, step: number) {
return Array.from({ length: 30 }, (_, index) => ({
metric,
date: new Date(Date.UTC(2026, 8, 1 + index)).toISOString().slice(0, 10),
value: start + step * index,
accountIds: [] as string[],
recordedAt: new Date().toISOString(),
}));
}

it('attaches projections to an OK response, via the trends forecaster', async () => {
const generatedAt = new Date(Date.now() - 60_000).toISOString();
setReconciliationStoreForTests({
getLatest: jest.fn().mockResolvedValue(record({ generatedAt })),
save: jest.fn(),
});
setTrendStoreForTests({
getSeries: jest
.fn()
.mockImplementation(async (metric: string) =>
metric === 'openRiskScore' ? thirtyPoints(metric, 900, -2) : []
),
} as never);
const res = makeRes();
await getPublishableRequirements({} as Request, res);
setTrendStoreForTests(null);

expect(res.body.status).toBe('OK');
expect(res.body.projections).toEqual([
expect.objectContaining({
metric: 'openRiskScore',
perDay: -2,
confidence: expect.any(String),
basisDays: 30,
horizonDays: 30,
}),
]);
expect(res.body.projections[0].projectedValue).toBeLessThan(900);
});

it('publishes no projections while the record is STALE, and survives unreadable trends', async () => {
setReconciliationStoreForTests({
getLatest: jest
.fn()
.mockResolvedValue(
record({ generatedAt: new Date(Date.now() - MAX_RECORD_AGE_MS - 60_000).toISOString() })
),
save: jest.fn(),
});
setTrendStoreForTests({
getSeries: jest.fn().mockRejectedValue(new Error('AccessDenied')),
} as never);
const stale = makeRes();
jest.spyOn(console, 'error').mockImplementation(() => {});
await getPublishableRequirements({} as Request, stale);
expect(stale.body.status).toBe('STALE');
expect(stale.body.projections).toBeUndefined();

setReconciliationStoreForTests({
getLatest: jest.fn().mockResolvedValue(record({ generatedAt: new Date().toISOString() })),
save: jest.fn(),
});
const unreadable = makeRes();
await getPublishableRequirements({} as Request, unreadable);
setTrendStoreForTests(null);
expect(unreadable.body.status).toBe('OK');
expect(unreadable.body.projections).toBeUndefined();
});
});

describe('requireTrustCenterSync', () => {
Expand Down
9 changes: 8 additions & 1 deletion api-service/src/routes/issues.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import type { Request, Response } from 'express';
import {
DynamoDBIssueRepository,
issueSlo,
PostureTrendStore,
type IssueListQuery,
} from '@khalifa/risk-engine';
Expand Down Expand Up @@ -63,7 +64,12 @@ export async function listIssues(req: Request, res: Response): Promise<void> {

const response = await issueStore.listIssues(query);

res.json(response);
res.json({
...response,
items: response.items.map((issue) =>
issue.status === 'open' ? { ...issue, slo: issueSlo(issue) } : issue
),
});
} catch (error) {
if (
error instanceof Error &&
Expand Down Expand Up @@ -158,6 +164,7 @@ export async function getIssue(req: Request<{ id: string }>, res: Response): Pro

const response: IssueDetailResponse = {
...issue,
...(issue.status === 'open' ? { slo: issueSlo(issue) } : {}),
attackPath: {
nodes: attackPathNodes,
edges: attackPathEdges,
Expand Down
81 changes: 38 additions & 43 deletions api-service/src/routes/trends.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
import type { Request, Response } from 'express';
import { forecastSeries, PostureTrendStore } from '@khalifa/risk-engine';
import { PostureTrendStore } from '@khalifa/risk-engine';
import { parseBoundedInteger } from '../middleware/gremlin-validator';
import type { SeverityCounts, TrendForecast, TrendPoint, TrendResponse } from '../types';
import { buildForecast } from '../services/trend-forecast';
import type { ExposureCounts, SeverityCounts, TrendPoint, TrendResponse } from '../types';

const trendStore = new PostureTrendStore();

Expand All @@ -15,49 +16,10 @@ const VALID_METRICS = [
'openRiskScore',
'mttrHours',
'adminPrincipals',
'sloBurning',
'exposedByType',
];

const DAY_MS = 86_400_000;
const STALE_RUN_MS = 2 * DAY_MS;
const MAX_SERIES_GAP_DAYS = 3;

// The engine's projection plus the series' own honesty check: no runner pass
// in two days, or three or more missing days inside the window, lowers the
// confidence a level and says why. This is the trend-store equivalent of the
// compliance engine's freshCoverage gate.
function buildForecast(metric: string, points: TrendPoint[]): TrendForecast | undefined {
const forecast = forecastSeries(
points.map((point) => ({ date: point.date, value: point.value }))
);
if (!forecast) return undefined;

let confidence = forecast.confidence;
const notes: string[] = [];
const last = points[points.length - 1];
const recordedMs = last.recordedAt ? Date.parse(last.recordedAt) : NaN;
if (Number.isFinite(recordedMs) && Date.now() - recordedMs > STALE_RUN_MS) {
confidence = downgrade(confidence);
notes.push('No runner pass in the last two days');
}
const spanDays = (Date.parse(last.date) - Date.parse(points[0].date)) / DAY_MS + 1;
if (spanDays - points.length >= MAX_SERIES_GAP_DAYS) {
confidence = downgrade(confidence);
notes.push('gaps in the series');
}

return {
...forecast,
confidence,
...(notes.length > 0 ? { note: notes.join('; ') } : {}),
};
}

function downgrade(confidence: TrendForecast['confidence']): TrendForecast['confidence'] {
if (confidence === 'high') return 'medium';
if (confidence === 'medium') return 'low';
return 'low';
}

export async function getTrend(req: Request<{ metric: string }>, res: Response): Promise<void> {
try {
const { metric } = req.params;
Expand All @@ -82,13 +44,24 @@ export async function getTrend(req: Request<{ metric: string }>, res: Response):
const points = await trendStore.getSeries(metric as any, days);
const trendPoints: TrendPoint[] = points.map((p) => {
const bySeverity = severityCounts(p.sample?.bySeverity);
const breached = nonNegativeInteger(p.sample?.breached);
const slo =
metric === 'sloBurning' && bySeverity && breached !== undefined
? { ...bySeverity, breached }
: undefined;
const exposure =
metric === 'exposedByType'
? exposureCounts(p.sample?.byType, p.sample?.truncated)
: undefined;
return {
metric: p.metric,
date: p.date,
value: p.value,
accountIds: p.accountIds,
recordedAt: p.recordedAt,
...(bySeverity ? { bySeverity } : {}),
...(slo ? { slo } : {}),
...(exposure ? { exposure } : {}),
};
});

Expand All @@ -113,6 +86,28 @@ export async function getTrend(req: Request<{ metric: string }>, res: Response):
}
}

// `sample` is free-form in storage; only reviewed shapes reach the client. The
// exposure map is bounded twice: at most 25 types, label-shaped keys only.
const MAX_EXPOSURE_TYPES = 25;
const EXPOSURE_LABEL = /^[A-Za-z][A-Za-z0-9]{0,40}$/;

function nonNegativeInteger(raw: unknown): number | undefined {
return Number.isSafeInteger(raw) && (raw as number) >= 0 ? (raw as number) : undefined;
}

function exposureCounts(byTypeRaw: unknown, truncatedRaw: unknown): ExposureCounts | undefined {
if (!byTypeRaw || typeof byTypeRaw !== 'object') return undefined;
const entries = Object.entries(byTypeRaw as Record<string, unknown>);
if (entries.length === 0 || entries.length > MAX_EXPOSURE_TYPES) return undefined;
const byType: Record<string, number> = {};
for (const [label, count] of entries) {
const parsed = nonNegativeInteger(count);
if (!EXPOSURE_LABEL.test(label) || parsed === undefined) return undefined;
byType[label] = parsed;
}
return { byType, truncated: truncatedRaw === true };
}

// `sample` is free-form in storage; only these four counts reach the client.
function severityCounts(raw: unknown): SeverityCounts | undefined {
if (!raw || typeof raw !== 'object') return undefined;
Expand Down
Loading
Loading