Repository navigation
feat: pattern-specific definitions and severity vectors for all 200 records - #10
Merged
Merged
Conversation
…ecords Every record's definition was one template sentence, and every record carried the same six-axis severity vector regardless of its score, so the vector explained nothing and 175 scores were flat per-domain defaults. Each of the 200 patterns now has its own definition stating the mechanism, the trust boundary it crosses, and the unsafe result, and its own vector on the six v1.0 axes, each scored 1 to 5. The baseline score is derived from the vector by a documented method in docs/STANDARD.md, weighting impact and exploitability double. The 25 v1.0 reference scores are unchanged and marked caap-v1.0-baseline; their vectors land within 0.2 of the preserved score and the generator refuses anything beyond 0.5. All other scores are vector-derived. Ratings gain a low band below 4.0. Pattern pages show the severity. Repository validation now checks that definitions are unique and not the template, vectors are complete and in range, ratings match scores, non-reference scores equal their vector, and reference scores are the v1.0 baseline within tolerance. Tests pin the 25 v1.0 scores as a regression guard. Distribution after the change: 13 critical, 166 high, 21 medium, 131 distinct vectors, scores 6.0 to 9.4. Signed-off-by: requie <tarique.smith@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Every record's definition was one template sentence ("Tests whether X can cross an agent trust boundary..."), and every record carried the same six-axis severity vector regardless of its score, so the vector explained nothing and the 175 non-reference scores were flat per-domain defaults. Each of the 200 patterns now has its own definition stating the mechanism, the trust boundary it crosses, and the unsafe result, and its own vector on the six CAAP axes (impact, exploitability, privilege, autonomy, persistence, propagation), each scored 1 to 5.
This is CAAP's own six-axis vector as defined in v1.0, not CVSS. No CVSS strings are introduced.
Pattern or implementation impact
Standards review: this changes the definition and severity of every record. Stable IDs, titles, families, maturity, mappings, and relationships are unchanged. The 25 v1.0 reference baseline scores are unchanged and marked
score_source: caap-v1.0-baseline.scripts/generate_catalog.py: newPATTERN_DETAILStable with a definition and vector per pattern, keyed by name; the generator refuses to build if any pattern is missing or any entry is orphaned.severity_from_vectorderives the baseline score as10 * (2*impact + 2*exploitability + privilege + autonomy + persistence + propagation) / 40, rounded to one decimal.severity_ratingadds alowband below 4.0. Reference vectors must land within 0.5 of the preserved v1.0 score or generation fails; all 25 land within 0.2. Pattern pages gain a Severity line.scripts/validate_repository.py: checks definitions are unique and non-template, vectors complete and in range, ratings match scores, non-reference scores equal their vector, and reference scores are the v1.0 baseline within tolerance.tests/test_taxonomy.py: pins the 25 v1.0 scores as a regression guard; checks vector derivation, range, and distinctness; checks definitions are unique, specific, and at least 80 characters.docs/STANDARD.md: new Severity section defining each axis, the formula, the rating bands (which match the 2.0.0-draft.1 standard document: critical 9.0+, high 7.0 to 8.9, medium 4.0 to 6.9, low below 4.0), the reference-score rule, and the caveat that baseline scores are illustrative estimates.CHANGELOG.md: Changed entry.data/taxonomy/caap-200.jsonand.yaml,docs/TAXONOMY.md, 200 pattern pages, 200 case files (severity is copied into each case), and the packaged copies. The diff is 433 files for that reason; only the generator, validator, tests, standard, and changelog are hand-edited.Distribution after the change: 13 critical, 166 high, 21 medium, 0 low; 131 distinct vectors; scores 6.0 to 9.4. Before: 9 critical, 190 high, 1 medium; 1 vector.
Safety impact
Not applicable. No fixture, capability, sink, network boundary, persistence boundary, or public procedure changes. Definitions describe mechanisms at the level of the existing pattern names and carry no payloads or procedures.
Validation
ruff check src tests scripts examplesis cleanReview focus
The definitions and axis judgments are the substance of this PR and were drafted for domain-editor review, not just CI. Worth a second opinion: the four new critical ratings outside the reference set (Dependency Confusion, Build-Script Injection, CI Command Injection, Dependency Installation Hijack), the embodied-agent domain where physical harm is treated as high persistence, and the human-trust domain, which scores lowest on average because those mechanisms need a human to act. Correcting any vector is an edit to the generator's table plus regeneration.