feat: exploitability reasoning (--exploit) + LLM-ready output (-o llm) by slimm609 · Pull Request #348 · slimm609/checksec

slimm609 · 2026-07-04T11:53:25Z

Summary

Two stacked, independently-designed research features that push checksec beyond reporting raw mitigation posture. Both were built spec-first (design docs), executed task-by-task under strict red/green TDD, and passed a final whole-branch review.

1. Exploitability reasoning (`--exploit` / `--chain`)

A "so-what" layer over the existing checks: instead of only reporting the mitigation posture, it reasons about which memory-corruption techniques that posture fails to obstruct, citing the evidence (imports, relocations, segment permissions) behind each verdict.

New self-contained pkg/exploit/ package: an Evidence collector → a rule engine that abstains on missing evidence → 7 technique rules (stack-bof-overwrite, ret2plt, ret2libc, got-overwrite, shellcode-injection, format-string, ret2dlresolve) → renderers.
Framing is mitigation-obstruction, never "exploitable." Six honest tiers (VIABLE / LIKELY / REQUIRES-LEAK / REQUIRES-INPUT-CONTROL / ENABLER / BLOCKED); every VIABLE verdict must cite a present primitive (enforced by an invariant test).
Renders inline in the table and embeds under each report's exploitability key in JSON/YAML (tiers serialize as their names).
--chain emits a labelled hypothesis exploit chain (never presented as fact).
CI gating via --fail-if exploit.viable and --fail-if exploit.technique=<id>.
Static, offline, no new dependencies; multi-arch for the posture/import path.

2. LLM-ready output (`-o llm`)

A self-grounding Markdown format meant to be pasted into an LLM assistant so it reasons from the tool, not from stale training data.

New pkg/knowledge/ semantic layer: each check id → meaning + authoritative fix (covers every FileFields key; enforced by a coverage test).
Output = a grounding directive (prefer these findings; cite check ids) + a knowledge block emitted once per run (O(checks), not O(checks×targets)) + terse per-target rows + inline --exploit verdicts.
--llm-no-preamble strips the directive; works across file / dir / proc / procAll / kernel modes.
Composes with --exploit; default output for existing formats is unchanged.

Testing

404 tests pass; go build ./... and go vet ./... clean.
Every task followed red → green TDD; each feature got a final whole-branch review, and the reviews' blocking findings were fixed:
- exploit: removed an "is exploitable" over-claim in the fail-if message; ret2libc/ret2dlresolve now cite the gating import (not posture alone).
- llm: renderer now honors the Fields override so proc mode renders seccomp; --llm-no-preamble threaded to proc/kernel.
Default output for table/json/yaml/xml/csv is byte-identical when the new flags are absent.

Docs

Full usage documented in docs/: usage.md (all new flags, exploitability + LLM sections, exploit.* fail-if predicates), new docs/checks/exploitability.md and docs/llm.md reference pages with worked examples, and index.md updated. mkdocs nav updated.

Deferred (intentional follow-ups, non-blocking)

Flagged by the final reviews and logged for a follow-up PR: XML format doesn't embed exploitability yet; --exploit no-ops on proc commands; collectGOT handles 64-bit .rela.plt only; the --fail-if flag help string doesn't yet list the exploit.* predicates.

Notes for reviewers

This branch is stacked — it contains both features. Phases B (disassembly-enriched evidence) and C (taint/reachability) of the exploitability design are specified but intentionally not built here.

- Tier.String() now returns REQUIRES-INPUT-CONTROL / REQUIRES-LEAK (hyphens, not underscores), matching the task brief. - hasControlFlowHijackPrimitive checks only CatUnboundedCopy, dropping the extra CatCommandExec check that wasn't in spec; restored its Phase-A doc comment. - Restored dropped Phase B/C comments on CanaryPartial, FieldFunctions, FieldTaint.

Adds FileReport.Exploitability (omitempty), utils.Option/WithExploit, and threads opts through RunFileChecks, RunListChecks(Parallel), and the file/dir/listfile commands so exploit.Assess only runs when --exploit or --chain is passed. Default output is unchanged.

Add RenderExploitTable to print verdicts (tier, rule, technique, citations) plus the attacker's-bar summary, and wire it into writeTable's report loop so `--exploit` table output shows the exploitability section right after each binary's name line.

Tier is an int enum, so encoding/json and yaml previously emitted raw integers (e.g. "Tier": 0) instead of readable names. Add MarshalText so Tier round-trips through JSON/YAML as VIABLE, BLOCKED, etc.

RenderChain synthesizes a labeled hypothesis chain from a VIABLE stack-bof-overwrite hijack plus the first VIABLE payload technique (ret2plt, got-overwrite, shellcode-injection, or ret2libc), falling back to a "no consistent chain" line when either step is missing. Wired into writeTable via PrintOptions.Chain, set from the existing --chain flag across file/dir/listfile commands.

EvaluateFailIf now recognizes two special --fail-if keys evaluated against FileReport.Exploitability instead of Checks: - exploit.viable: fails when any verdict reaches TierViable. - exploit.technique=<rule id>: fails when a verdict for that rule reaches TierRequiresLeak or above. Both keys are accepted in the known-key validation step; a nil Exploitability report never matches. Existing check-key behavior is unchanged. Tests added in failif_exploit_test.go (pre-commit hook enforces a green suite, so the RED test and its GREEN implementation land in one commit rather than two).

… in ret2libc/ret2dlresolve

Maps checksec.Status to compact glyphs (ok/~/!/i) for LLM-oriented output.

writeLLMTarget prints a "## Target: <name>" header followed by one terse row per present FileFields check, in canonical field order.

Wires the LLM-oriented output format end to end: writeLLM assembles the grounding preamble, shared knowledge block, and per-target sections (with inline exploitability when present) built in earlier tasks. Registers "llm" in FilePrinter and threads --llm-no-preamble through file/dir/listfile.

…eamble to proc/kernel The LLM renderer ignored PrintOptions.Fields and always iterated the global FileFields registry, so -o llm never rendered the seccomp column in proc/procAll mode even though ProcFields was passed through. LLMNoPreamble was also not wired into proc/procAll/kernel, so --llm-no-preamble had no effect for those commands.

…tes in full usage

augmentcode · 2026-07-04T11:56:48Z

🤖 Augment PR Summary

Summary: This PR extends checksec with two research features: (1) static exploitability mitigation-obstruction reasoning via --exploit/--chain, and (2) a new -o llm output format that emits a self-grounding Markdown report for LLM-assisted analysis.

Changes:

Added pkg/exploit/ evidence collection + rule engine + 7 technique rules, producing tiered verdicts and a synthesized “Bar” line.
Plumbed exploitability collection into file/dir/listfile execution via an options pattern (utils.Option / WithExploit) and embeds results into JSON/YAML reports.
Added --fail-if predicates exploit.viable and exploit.technique=<id> for CI gating.
Implemented human-readable exploitability rendering and optional hypothesis chain rendering in table output.
Introduced pkg/knowledge/ check metadata registry and an -o llm renderer that emits a directive + shared knowledge block + per-target findings (and inline exploitability).
Threaded --llm-no-preamble through file/dir/proc/procAll/kernel printing paths and added MkDocs documentation pages/nav updates.
Added extensive unit tests covering engine abstention, invariants, printers, knowledge coverage, and new fail-if semantics.

Technical Notes: Exploitability analysis is explicitly framed as evidence-cited posture reasoning (never “exploitable”), tiers serialize by name in machine formats, and -o llm is designed to keep grounding cost O(checks) per run (knowledge block emitted once).

_{🤖 Was this summary useful? React with 👍 or 👎}

augmentcode

Review completed. 4 suggestions posted.

Comment augment review to trigger a new review at any time.

…validation, drop dead WithExploit arg - postureFromChecks: treat indeterminate posture (NX 'No GNU_STACK'/Warn, PIE 'DSO'/Info, RELRO N/A) as protected so uncertain evidence never yields a false VIABLE verdict (honesty invariant). - --fail-if exploit.technique=<id>: reject unknown/empty ids (via exploit.IsKnownRuleID) so a typo'd predicate can't silently no-op a CI gate. - WithExploit(): drop the unused chain arg; chain is render-only via PrintOptions.Chain.

slimm609 added 30 commits July 3, 2026 19:32

feat(exploit): import categorization

d808be2

feat(exploit): core Evidence, Verdict, and Tier types

d330bbf

feat(exploit): static ELF-fact collectors (imports, segments, GOT)

dd31f08

feat(exploit): assemble Evidence from ELF + posture checks

c3c305e

feat(exploit): rule engine with abstain-on-missing-evidence

0889a6d

feat(exploit): stack-bof-overwrite rule

4e92652

feat(exploit): ret2plt rule

6d5a904

feat(exploit): got-overwrite rule

54e6db0

feat(exploit): shellcode-injection rule

ad00abf

feat(exploit): ret2libc, format-string, ret2dlresolve rules

84cbeee

feat(exploit): exploitation-bar corpus

1f4de9e

feat(exploit): serialize Tier as its name in JSON/YAML

6d24030

Tier is an int enum, so encoding/json and yaml previously emitted raw integers (e.g. "Tier": 0) instead of readable names. Add MarshalText so Tier round-trips through JSON/YAML as VIABLE, BLOCKED, etc.

test(exploit): honesty and mitigation-obstruction invariants

ef37c66

docs: exploitability (--exploit) reference page

d1356cf

fix(exploit): obstruction-framed fail-if message + cite gating import…

db74d89

… in ret2libc/ret2dlresolve

feat(knowledge): semantic check-knowledge registry

15f9795

feat(utils): add sevGlyph severity glyph mapping

9d46f03

Maps checksec.Status to compact glyphs (ok/~/!/i) for LLM-oriented output.

feat(llm): grounding preamble with directive + exploit epistemics

6dfd469

feat(llm): ground-once knowledge block (fix only for non-good checks)

110400f

feat(llm): add per-target block renderer

fd2e44f

writeLLMTarget prints a "## Target: <name>" header followed by one terse row per present FileFields check, in canonical field order.

feat(llm): inline exploitability sub-block

5ad4727

feat(llm): kernel-mode llm rendering grouped by type

20cf4a4

docs: -o llm self-grounding output format reference

27cf6e6

docs: document --exploit/--chain, -o llm, and exploit fail-if predica…

efe1fb9

…tes in full usage

augmentcode Bot reviewed Jul 4, 2026

View reviewed changes

Comment thread pkg/exploit/collector_static.go Outdated

Comment thread pkg/exploit/collector_static.go

Comment thread pkg/utils/failif.go

Comment thread pkg/utils/report.go Outdated

Navigation Menu

feat: exploitability reasoning (--exploit) + LLM-ready output (-o llm) - #348

feat: exploitability reasoning (--exploit) + LLM-ready output (-o llm)#348
slimm609 wants to merge 32 commits into
mainfrom
feat/llm-output

slimm609 commented Jul 4, 2026

Uh oh!

augmentcode Bot commented Jul 4, 2026

Uh oh!

augmentcode Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

slimm609 commented Jul 4, 2026

Summary

1. Exploitability reasoning (--exploit / --chain)

2. LLM-ready output (-o llm)

Testing

Docs

Deferred (intentional follow-ups, non-blocking)

Notes for reviewers

Uh oh!

augmentcode Bot commented Jul 4, 2026

Uh oh!

augmentcode Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

1. Exploitability reasoning (`--exploit` / `--chain`)

2. LLM-ready output (`-o llm`)