From f198f8d384483391449c05662e7f2ff1e0bb8826 Mon Sep 17 00:00:00 2001 From: Eric Conklin Date: Sat, 6 Jun 2026 15:20:03 -0500 Subject: [PATCH] Add reviewer demo package --- docs/demo/README.md | 74 +++++++++++++++ docs/demo/demo-narrative-one-pager.md | 63 +++++++++++++ docs/demo/iamscope-vs-pacu-pmapper.md | 30 +++++++ docs/demo/live-demo-runbook.md | 87 ++++++++++++++++++ docs/demo/recorded-demo-script.md | 125 ++++++++++++++++++++++++++ 5 files changed, 379 insertions(+) create mode 100644 docs/demo/README.md create mode 100644 docs/demo/demo-narrative-one-pager.md create mode 100644 docs/demo/iamscope-vs-pacu-pmapper.md create mode 100644 docs/demo/live-demo-runbook.md create mode 100644 docs/demo/recorded-demo-script.md diff --git a/docs/demo/README.md b/docs/demo/README.md new file mode 100644 index 0000000..06886e0 --- /dev/null +++ b/docs/demo/README.md @@ -0,0 +1,74 @@ +# IAMScope Reviewer Demo Package + +## What This Demo Is + +This demo package gives reviewers a short, bounded walkthrough of IAMScope’s differentiated value: evidence-grade IAM findings with explicit verdicts, required checks, blockers, `collection_context`, capability boundaries, replay, human labels, and an owner-confirmation trail. + +The strongest current milestone is the frozen tag: + +- `real-pilot-dev-001-final-calibrated-reviewed` + +The main case study is: + +- [`docs/case-studies/real-pilot-dev-001-human-review-summary.md`](../case-studies/real-pilot-dev-001-human-review-summary.md) + +The capability boundary reference is: + +- [`docs/reference/capability-honesty-matrix.md`](../reference/capability-honesty-matrix.md) + +## What This Demo Proves In The Narrow Sense + +The final calibrated real-pilot summary records: + +- 18 findings. +- 18 validated. +- 15 `cross_account_trust` findings. +- 3 `admin_reachability` findings. +- 18 labeled findings. +- 14 `valid_path` labels. +- 3 `expected_benign` labels. +- 1 `needs_more_evidence` label. +- 5 `owner_confirmed` labels. +- complete `collection_context`. +- sanitized output hygiene clean. + +This proves only a bounded workflow point: IAMScope can turn a collected IAM graph into reviewable, evidence-grade findings with explicit capability boundaries, human-review labels, and owner-confirmation metadata for the documented pilot. It does not prove broad correctness or safety. + +## What This Demo Does Not Prove + +No findings does not mean safe. Validated does not mean exploited. The demo evidence is bounded. + +This package does not claim: + +- production readiness. +- exploitability proof. +- full IAM safety. +- full AWS authorization semantics. +- complete IAM privilege-escalation coverage. +- broad IAMScope correctness. +- a composite score. +- a pass/fail benchmark label. + +## What To Open First + +1. [`demo-narrative-one-pager.md`](demo-narrative-one-pager.md) — shortest public-facing story. +2. [`iamscope-vs-pacu-pmapper.md`](iamscope-vs-pacu-pmapper.md) — positioning against Pacu and PMapper without disparaging either tool. +3. [`recorded-demo-script.md`](recorded-demo-script.md) — 7-10 minute recording script. +4. [`live-demo-runbook.md`](live-demo-runbook.md) — safe no-AWS and authorized-AWS demo modes. +5. [`../case-studies/real-pilot-dev-001-human-review-summary.md`](../case-studies/real-pilot-dev-001-human-review-summary.md) — final calibrated real-pilot evidence. + +## How To Use The Final Real-Pilot Case Study + +Use the case study as the evidence anchor, not as a broad score. The useful review path is: + +- confirm the final calibrated replay counts; +- inspect how `cross_account_trust` and `admin_reachability` findings are separated; +- explain how `collection_context` and non-claims keep the evidence bounded; +- show how human labels and owner-confirmation add reviewer accountability without claiming exploitability. + +## Related Demo Files + +- [`recorded-demo-script.md`](recorded-demo-script.md) +- [`live-demo-runbook.md`](live-demo-runbook.md) +- [`iamscope-vs-pacu-pmapper.md`](iamscope-vs-pacu-pmapper.md) +- [`demo-narrative-one-pager.md`](demo-narrative-one-pager.md) diff --git a/docs/demo/demo-narrative-one-pager.md b/docs/demo/demo-narrative-one-pager.md new file mode 100644 index 0000000..e0c30aa --- /dev/null +++ b/docs/demo/demo-narrative-one-pager.md @@ -0,0 +1,63 @@ +# IAMScope Demo Narrative + +IAMScope is not trying to be Pacu. + +IAMScope is not trying to be a full replacement for PMapper. + +IAMScope is trying to answer a narrower reviewer question: what IAM paths can we support with evidence, what remains uncertain, and what should a human review first? + +## The Short Version + +Many IAM tools help you exploit, query, or graph an AWS environment. Those are useful jobs. IAMScope is focused on a different job: evidence-grade review. + +In the current final calibrated real-pilot milestone, IAMScope produced: + +- 18 findings. +- 18 validated. +- 15 `cross_account_trust`. +- 3 `admin_reachability`. +- 14 `valid_path`. +- 3 `expected_benign`. +- 1 `needs_more_evidence`. +- 5 `owner_confirmed`. +- complete `collection_context`. + +The point is not that these numbers are a score. They are not. The point is that each finding can be discussed as a review artifact with checks, evidence boundaries, labels, and non-claims. + +## Why This Is Different + +IAMScope keeps the reviewer workflow visible: + +- collect or replay the IAM graph; +- run bounded reasoners; +- emit findings with verdicts and required checks; +- show blockers and uncertainty; +- preserve `collection_context`; +- map findings to human labels; +- add owner-confirmation when available; +- keep raw artifacts local unless sanitized. + +This is why the demo uses phrases like evidence-grade, bounded, reviewable, and owner-confirmed instead of “exploited” or “safe.” + +## The Lines We Do Not Cross + +No findings does not mean safe. + +Validated does not mean exploited. + +The demo evidence is bounded. + +This demo does not claim: + +- production readiness. +- exploitability proof. +- full IAM safety. +- full AWS authorization semantics. +- complete IAM privilege-escalation coverage. +- broad IAMScope correctness. +- a composite score. +- a pass/fail benchmark label. + +## Reviewer Takeaway + +IAMScope is for the moment after a graph looks scary but before a reviewer can act. It helps separate “the evidence supports this,” “this looks expected-benign,” “this needs more evidence,” and “this is outside current modeled capability.” diff --git a/docs/demo/iamscope-vs-pacu-pmapper.md b/docs/demo/iamscope-vs-pacu-pmapper.md new file mode 100644 index 0000000..0606a03 --- /dev/null +++ b/docs/demo/iamscope-vs-pacu-pmapper.md @@ -0,0 +1,30 @@ +# IAMScope vs Pacu vs PMapper + +Pacu and PMapper are useful tools with different goals. This comparison is positioning, not a ranking. + +| Tool | Primary purpose | What it is good at | What IAMScope is not trying to replace | IAMScope difference | +| --- | --- | --- | --- | --- | +| Pacu | Offensive AWS exploitation / attack module framework. | Running AWS attack modules in authorized security testing, demonstrating exploitation workflows, and exploring attacker tradecraft. | IAMScope is not trying to replace Pacu as an exploitation framework or attack-module runner. | IAMScope does not attempt exploitation. It produces evidence-grade findings with verdicts, required checks, blockers, `collection_context`, capability boundaries, replay, human labels, and owner-confirmation trail. | +| PMapper | IAM graph/query/local authorization simulation / privilege-escalation path mapping. | Building IAM relationship graphs, querying policies, local authorization reasoning, and mapping privilege-escalation paths. | IAMScope is not trying to be a full replacement for PMapper’s graph/query and simulation workflows. | IAMScope focuses on reviewer-facing findings: what is validated, blocked, inconclusive, expected-benign, unsupported, or needs more evidence. It emphasizes capability honesty and bounded non-claims. | +| IAMScope | Evidence-grade IAM finding workflow. | Collecting or replaying IAM graph artifacts, running reasoners, emitting verdicts and required checks, preserving blockers and `collection_context`, supporting human labels, owner-confirmation, and sanitized review summaries. | IAMScope is not trying to replace offensive testing tools or general IAM graph/query tools. | IAMScope’s value is the review workflow: it says what the evidence supports, what remains uncertain, and what a human should review first. | + +## Positioning Summary + +- Pacu helps authorized testers exercise offensive AWS techniques. +- PMapper helps users inspect IAM graph and authorization relationships. +- IAMScope helps reviewers handle evidence, verdicts, blockers, capability boundaries, `collection_context`, human labels, and owner-confirmation without claiming exploitation. + +## Non-Claims + +IAMScope’s demo evidence does not claim: + +- production readiness. +- exploitability proof. +- full IAM safety. +- full AWS authorization semantics. +- complete IAM privilege-escalation coverage. +- broad IAMScope correctness. +- a composite score. +- a pass/fail benchmark label. + +No findings does not mean safe. Validated does not mean exploited. diff --git a/docs/demo/live-demo-runbook.md b/docs/demo/live-demo-runbook.md new file mode 100644 index 0000000..0df83a2 --- /dev/null +++ b/docs/demo/live-demo-runbook.md @@ -0,0 +1,87 @@ +# Live Demo Runbook + +This runbook supports two demo modes: a no-AWS walkthrough and an explicitly authorized AWS walkthrough. The default and safest path is the no-AWS demo. + +## Safety Boundary + +- Do not run live AWS by default. +- Do not run Terraform by default. +- Do not show raw account IDs or raw IAM/STS ARNs on screen unless explicitly authorized. +- Do not commit raw `scenario.json`, `findings.json`, labels, logs, or generated review artifacts. +- Do not present any result as production readiness, exploitability proof, full IAM safety, a composite score, or a pass/fail benchmark label. + +## Mode A — No-AWS Demo + +Use this mode for recorded demos, public walkthroughs, and first-pass reviewer conversations. + +Steps: + +1. Open [`README.md`](README.md). +2. Open [`demo-narrative-one-pager.md`](demo-narrative-one-pager.md). +3. Open [`iamscope-vs-pacu-pmapper.md`](iamscope-vs-pacu-pmapper.md). +4. Open [`../reference/capability-honesty-matrix.md`](../reference/capability-honesty-matrix.md). +5. Open [`../case-studies/real-pilot-dev-001-human-review-summary.md`](../case-studies/real-pilot-dev-001-human-review-summary.md). +6. If a sanitized review table is present locally, show only sanitized columns and avoid raw account IDs or raw IAM/STS ARNs. + +Facts to state: + +- 18 findings. +- 18 validated. +- 15 `cross_account_trust`. +- 3 `admin_reachability`. +- 14 `valid_path`. +- 3 `expected_benign`. +- 1 `needs_more_evidence`. +- 5 `owner_confirmed`. +- complete `collection_context`. +- sanitized output hygiene clean. + +Explain that this is evidence-grade review material, not live exploitation and not a safety certificate. + +## Mode B — Authorized AWS Demo + +Use this mode only in a sandbox, non-production account, or explicitly authorized environment controlled by the reviewer. + +Preconditions: + +- written or clearly recorded authorization; +- scoped AWS profile; +- expected account checked before collection; +- no production admin access requested; +- output path outside the repository, preferably under `/tmp`; +- redaction plan agreed before screen sharing or publication. + +Suggested flow: + +1. Confirm the profile and expected account out of band with the environment owner. +2. Run collection or replay only if authorized. +3. Produce sanitized reviewer output under `/tmp` or another non-repo path. +4. Run hygiene grep against generated demo output before showing or sharing it. +5. Show findings, required checks, blockers, `collection_context`, labels, and owner-confirmation status. +6. Keep raw account IDs, raw IAM/STS ARNs, raw findings, and logs off-screen unless explicitly authorized. +7. Do not commit generated artifacts. + +Stop conditions: + +- profile/account does not match the expected environment; +- collection emits unexpected raw artifacts into the repo tree; +- raw account IDs or raw IAM/STS ARNs would be exposed without authorization; +- reviewer asks to stop; +- any command would mutate resources without explicit approval; +- any result is being framed as exploitability proof, production readiness, or a pass/fail benchmark label. + +## Hygiene Checks For Demo Outputs + +Before sharing sanitized output, run equivalent local checks against the output directory: + +- no raw 12-digit account IDs unless explicitly authorized; +- no raw IAM/STS ARNs unless explicitly authorized; +- no Terraform state, plan, lock file, or output JSON; +- no raw AWS logs; +- no generated artifacts committed to git. + +## Demo Close + +End with: + +> “IAMScope is not an exploitation framework. It is an evidence-grade IAM finding workflow. No findings does not mean safe, and validated does not mean exploited.” diff --git a/docs/demo/recorded-demo-script.md b/docs/demo/recorded-demo-script.md new file mode 100644 index 0000000..2ca9331 --- /dev/null +++ b/docs/demo/recorded-demo-script.md @@ -0,0 +1,125 @@ +# Recorded Demo Script + +Target length: 7-10 minutes. + +## 0:00-0:45 — Problem + +IAM trust is messy. A reviewer does not only need a scary graph edge or an exploit module; they need to know what is supported by evidence, what is blocked, what is uncertain, and what should be reviewed first. + +Say: + +> “IAMScope is built for evidence-grade IAM review. It does not try to prove the account is safe, and it does not claim exploitability.” + +## 0:45-1:45 — Positioning: Pacu vs PMapper vs IAMScope + +Show [`iamscope-vs-pacu-pmapper.md`](iamscope-vs-pacu-pmapper.md). + +Talk track: + +- Pacu is useful as an offensive AWS exploitation and attack-module framework. +- PMapper is useful for IAM graph/query work, local authorization simulation, and privilege-escalation path mapping. +- IAMScope is narrower: it turns collection or replay into findings with verdicts, required checks, blockers, `collection_context`, capability honesty, human labels, and owner-confirmation trail. + +Do not disparage Pacu or PMapper. They solve different jobs. + +## 1:45-3:00 — Capability-Honesty Matrix + +Open [`../reference/capability-honesty-matrix.md`](../reference/capability-honesty-matrix.md). + +Call out: + +- modeled areas; +- unsupported or static-only areas; +- places where IAMScope refuses to turn missing evidence into a stronger claim; +- why no composite score and no pass/fail benchmark label are used. + +Say: + +> “The matrix is part of the product. It tells reviewers what not to believe.” + +## 3:00-4:30 — Real-Pilot Case Study And Final Calibrated Replay + +Open [`../case-studies/real-pilot-dev-001-human-review-summary.md`](../case-studies/real-pilot-dev-001-human-review-summary.md). + +Show the final calibrated replay: + +- 18 findings. +- 18 validated. +- 15 `cross_account_trust`. +- 3 `admin_reachability`. +- 18 labeled. +- 14 `valid_path`. +- 3 `expected_benign`. +- 1 `needs_more_evidence`. +- 5 `owner_confirmed`. +- complete `collection_context`. + +Say: + +> “This is bounded real-pilot evidence. It is not production readiness, exploitability proof, or full IAM safety.” + +## 4:30-6:00 — Cross-Account Trust Finding + +Walk through one `cross_account_trust` row from the sanitized local review material if it is present locally. If the raw or sanitized table is not available, use the case-study summary instead. + +Explain: + +- the finding is a trust-structure review row; +- wildcard-principal trust findings were repeatedly classified as `valid_path`; +- expected-benign rows are still useful because they represent real structures owners may need to confirm; +- owner-confirmed rows strengthen reviewability but do not create a full owner-confirmed truth set. + +Avoid showing raw account IDs or raw IAM/STS ARNs unless the demo owner explicitly authorizes it. + +## 6:00-7:30 — Admin Reachability Finding + +Walk through one `admin_reachability` row from the sanitized local review material if present. + +Explain the calibrated evidence chain: + +- source role has `sts:AssumeRole` to `ProdDBAdminRole`; +- the target trust is conditioned account-root trust narrowed by `aws:PrincipalArn`; +- the admin witness is AWS-managed `AdministratorAccess`; +- `clean_witness_check` is pass; +- `source_has_assume_role`, `reaches_at_least_one_admin`, and `walk_terminated_within_depth_limit` are pass. + +Say: + +> “Validated does not mean exploited. It means IAMScope’s modeled checks for this finding passed under the current bounded evidence.” + +## 7:30-8:30 — Collection Context And Non-Claims + +Show that `collection_context` is complete: + +- `graph_collection_complete`: true. +- `has_collection_failures`: false. +- `has_policy_parse_failures`: false. + +Then read the non-claims: + +- no production readiness. +- no exploitability proof. +- no full IAM safety. +- no full AWS authorization semantics. +- no complete IAM privilege-escalation coverage. +- no composite score. +- no pass/fail benchmark label. + +## 8:30-9:30 — Owner-Confirmation Layer + +Explain why owner-confirmation matters: + +- labels are human review, not an automatic truth oracle; +- five priority trust findings were owner-confirmed; +- owner confirmation is bounded to those findings only; +- this creates a review trail without claiming broad IAMScope correctness. + +## 9:30-10:00 — Close + +Close with: + +> “IAMScope does not prove the account is safe. It gives a reviewer evidence they can act on.” + +Then repeat: + +> “No findings does not mean safe. Validated does not mean exploited. The demo evidence is bounded.”