Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
88 commits
Select commit Hold shift + click to select a range
677b52d
Implement token-saving skill and related evaluation files; update REA…
miroslavpojer May 20, 2026
e709f12
Add title parameter to release notes presence check in PR workflow
miroslavpojer May 20, 2026
f653b27
Update documentation for token-saving skill; add contribution guideli…
miroslavpojer May 21, 2026
fd9f957
Add token-saving skill documentation with response formatting rules
miroslavpojer May 21, 2026
b0eb566
Initial design, review of skill.md files not finished.
miroslavpojer May 21, 2026
dcbfcb5
Update links to living-doc-glossary in various skill documents and ag…
miroslavpojer May 22, 2026
15b5cec
Update links to living-doc-glossary in SKILL.md files and remove obso…
miroslavpojer May 22, 2026
9aab9d1
Add living doc automation scripts for ID assignment, gap detection, i…
miroslavpojer May 22, 2026
c172270
feat: add script to scan .feature files for AC link compliance
miroslavpojer May 22, 2026
ae2fa98
feat: enhance documentation with new skills and update references
miroslavpojer May 22, 2026
313385e
feat: remove outdated implementation roadmap from the repository
miroslavpojer May 22, 2026
8e33c87
Refactor Gherkin step definitions and Living Doc skills
miroslavpojer May 22, 2026
c63f2dd
feat: update Gherkin step definitions and enhance living documentatio…
miroslavpojer May 22, 2026
b983ef6
Add evaluation configurations for Gherkin and PageObject skills
miroslavpojer May 22, 2026
21503cd
feat: enhance Gherkin-Living Doc sync skill with improved description…
miroslavpojer May 22, 2026
ea9b7c8
Refactor living documentation skills for improved clarity and functio…
miroslavpojer May 22, 2026
d13a22f
feat: add evals and trigger evaluations for living-doc and living-doc…
miroslavpojer May 22, 2026
4d94d28
chore: remove roadmap.md as part of project restructuring
miroslavpojer May 22, 2026
8ad9e59
tmp
miroslavpojer May 25, 2026
6ac9ff3
Remove tmp data.
miroslavpojer May 26, 2026
949234b
Updated form Unify project integration.
miroslavpojer May 27, 2026
337bcfd
Updated evals and finished round 1 of trigger evals testing.
miroslavpojer May 27, 2026
e4cef54
Update living documentation agent description and triggers for clarit…
miroslavpojer May 27, 2026
9edf2eb
Update .gitignore and enhance living documentation skills with new co…
miroslavpojer May 27, 2026
8e7ba76
Enhance evals with improved output formatting and additional queries …
miroslavpojer May 27, 2026
17baf4e
Backup from integration
miroslavpojer May 30, 2026
3bdc5c2
partial tune status backup
miroslavpojer May 30, 2026
3fd7e50
Skill gap reduction.
miroslavpojer May 30, 2026
b45c00a
Final fix of gaps and issues.
miroslavpojer May 30, 2026
4f825bc
Test case review.
miroslavpojer May 31, 2026
fafcd64
Tested agent and improved testing doc.
miroslavpojer May 31, 2026
d4462a7
Skill test - round 1.
miroslavpojer May 31, 2026
c615596
Update SKILL.md files to enhance trigger phrases and improve descript…
miroslavpojer May 31, 2026
c70f8ca
Worked in changed from weakness review.
miroslavpojer May 31, 2026
f3c88e3
Testing round 2.
miroslavpojer May 31, 2026
2c2d495
Update skills from adoption pilot.
miroslavpojer Jul 13, 2026
c78364d
improved evals
miroslavpojer Jul 13, 2026
14f5b6a
Minimize skill text size — preserve information, reduce verbosity
miroslavpojer Jul 13, 2026
46447da
tuned skills according exising evals
miroslavpojer Jul 13, 2026
561c1ba
Merge master into feature/7-living-doc-copilot
miroslavpojer Jul 14, 2026
40a13a4
Merge branch 'master' into feature/7-living-doc-copilot
miroslavpojer Jul 14, 2026
9dd739c
Refactor living documentation and BDD copilot integration; update REA…
miroslavpojer Jul 14, 2026
ac52bfe
Update living documentation and BDD copilot integration; enhance READ…
miroslavpojer Jul 14, 2026
f9aaaf0
Add living documentation schemas and glossary for BDD framework
miroslavpojer Jul 14, 2026
39c0299
Refactor schema paths in JSON files and update file handling in Pytho…
miroslavpojer Jul 14, 2026
74d7bf2
Update schema paths and enhance manifest description for clarity
miroslavpojer Jul 14, 2026
0dc583c
Enhance regex patterns for AC IDs and feature IDs; update living docu…
miroslavpojer Jul 14, 2026
6a354b9
Enhance regex for AC IDs in coverage report; improve ID extraction lo…
miroslavpojer Jul 14, 2026
aa1081a
Update ID format in coverage report; change acceptance criteria ID to…
miroslavpojer Jul 14, 2026
7f9d7c7
Update trigger phrases and documentation for clarity; fix script usag…
miroslavpojer Jul 14, 2026
05b47b0
Refine descriptions for living documentation and BDD automation; stan…
miroslavpojer Jul 14, 2026
976b90d
Normalize acceptance criteria state values and enhance documentation …
miroslavpojer Jul 14, 2026
04cd211
Clarify agent invocation instructions and update documentation for ag…
miroslavpojer Jul 14, 2026
ccd9df7
Enhance AC ID handling in coverage report; add canonicalization for n…
miroslavpojer Jul 14, 2026
acb2cf9
Refine AC ID format handling in scripts; update regex patterns for co…
miroslavpojer Jul 14, 2026
cbbe72f
Enhance token-saving skill and documentation
miroslavpojer Jul 14, 2026
5e9b25b
Refactor Feature ID handling to enforce numeric-only convention
miroslavpojer Jul 14, 2026
30ec29e
Standardize AC state terminology to Title-case across documentation a…
miroslavpojer Jul 14, 2026
c8d551b
Refactor AC state handling to use lowercase with underscores across t…
miroslavpojer Jul 14, 2026
397d527
Update references to living-doc schemas and glossary across multiple …
miroslavpojer Jul 14, 2026
8083c10
Update README and schema files for clarity; refine feature ID pattern…
miroslavpojer Jul 15, 2026
555979a
Update feature IDs to numeric-only format across various skills and d…
miroslavpojer Jul 15, 2026
8922aa0
Enhance living_doc_id.py documentation to clarify ID conventions; enf…
miroslavpojer Jul 15, 2026
4b2a288
Refactor file search in dead-code detectors to use recursive globbing…
miroslavpojer Jul 15, 2026
df55e38
Update slug-based FEAT ID tests to improve error message clarity and …
miroslavpojer Jul 15, 2026
e035326
Final adjustment using local pr-review skill.
miroslavpojer Jul 15, 2026
4780ad7
Enhance documentation for system-level preconditions and exclusions a…
miroslavpojer Jul 16, 2026
c114cb7
Enhance living documentation by adding optional `functionalities` fie…
miroslavpojer Jul 16, 2026
5addc27
Merge branch 'master' into feature/7-living-doc-copilot
miroslavpojer Aug 18, 2026
ed812bc
Removed file duplicate.
miroslavpojer Aug 18, 2026
6146d89
Merge branch 'master' into feature/7-living-doc-copilot
miroslavpojer Aug 18, 2026
af0fceb
Add living documentation skills for feature creation, functionality d…
miroslavpojer Aug 18, 2026
778a992
Remove unnecessary .DS_Store files from living documentation scripts
miroslavpojer Aug 19, 2026
bd004b3
fix: remove trailing commas in expected_output fields across evals
miroslavpojer Aug 19, 2026
60bfa8e
feat: add AC-ID pattern matching and corresponding tests for trace_en…
miroslavpojer Aug 19, 2026
e5230c5
refactor: remove compatibility field from SKILL.md files
miroslavpojer Aug 19, 2026
220bef0
feat: enhance test classification logic to avoid misclassifying domai…
miroslavpojer Aug 19, 2026
de24bb1
feat: improve AC_TAG regex to prevent matching malformed tags and add…
miroslavpojer Aug 19, 2026
48acbc8
feat: implement shared AC tag grammar and validation functions in ac_…
miroslavpojer Aug 19, 2026
d3b1096
feat: update AC ID assignment format and validation; add regression t…
miroslavpojer Aug 19, 2026
11334ee
feat: enhance method call collection in find_unused_po_methods.py; ad…
miroslavpojer Aug 19, 2026
0bff2b8
feat: add shared library installation instructions and error handling…
miroslavpojer Aug 19, 2026
b77714c
fix: update links to 'Getting Started' in documentation for consistency
miroslavpojer Aug 19, 2026
64e7a9f
feat: update expected outputs and prompts in evals and skills documen…
miroslavpojer Aug 19, 2026
0a62cdc
feat: enhance find_unused_po_components.py to scan fixtures and compo…
miroslavpojer Aug 19, 2026
5980d2b
Merge branch 'master' into feature/7-living-doc-copilot
miroslavpojer Aug 19, 2026
595528b
feat: add 'Create Repository' guide to documentation and remove dupli…
miroslavpojer Aug 19, 2026
d00ca78
feat: add dependency installation step for pyyaml in test scripts wor…
miroslavpojer Aug 19, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
355 changes: 355 additions & 0 deletions .github/agents/evals/living-doc-bdd-copilot/evals.json

Large diffs are not rendered by default.

27 changes: 27 additions & 0 deletions .github/agents/evals/living-doc-bdd-copilot/fixture-map.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Fixture Map — living-doc-bdd-copilot agent evals

## Eval coverage summary

| Eval ID | Category | Description | Fixture files |
|---------|----------|-------------|---------------|
| 1 | happy-path | Business Seed assembly — seed.yaml structure | — |
| 2 | happy-path | Create mode: PageObject generation from crawled surface | — |
| 3 | happy-path | Scenario generation from US ACs | — |
| 4 | regression | RE-SCAN mode — selector drift detection and repair | — |
| 5 | regression | HEALING mode — broken step definitions | — |
| 6 | negative | Create User Story request stays in this agent — no handoff | — |
| 7 | paraphrase | "fix failing tests" → HEALING mode trigger | — |
| 8 | regression | REMOVE mode — full feature removal with pre-deletion checklist | — |
| 9 | regression | Partial state rule: seed.yaml present, manifest.json absent → first run | — |
| 10 | regression | Credential safety — literal credentials in seed.yaml rejected | — |
| 11 | edge-case | Source E guided traversal — blocked crawl, unknown field value | — |
| 12 | output-format | manifest.json entry structure for a scanned route | — |
| 25 | negative | Unit test request → decline + point to test-unit-write skill | — |

## Trigger eval summary

| Count | Triggers (should_trigger=true) | Non-triggers (should_trigger=false) |
|-------|-------------------------------|--------------------------------------|
| 44 total | 22 true | 22 false |

> No fixture files — all evals use inline prompt/expected_output; agent behavior is assessed against the agent.md operating rules and skill definitions.
360 changes: 360 additions & 0 deletions .github/agents/living-doc-bdd-copilot.agent.md

Large diffs are not rendered by default.

8 changes: 8 additions & 0 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# Copilot Instructions

## Always Active

The **token-saving** skill is always active in this workspace. Every response applies concise formatting rules and removes noise (no filler, no closing platitudes, structured output).

See [Token-Saving Skill](../docs/guides/token-saving.md) for details, including override behavior when you explicitly request depth.

---

## PR Review

When reviewing a pull request, load and apply:
Expand Down
60 changes: 60 additions & 0 deletions .github/workflows/test-scripts.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
name: Test Operational Scripts

on:
pull_request:
paths:
- "skills/**/*.py"
- ".github/workflows/test-scripts.yml"

permissions:
contents: read

jobs:
run-tests:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.11"

- name: Install dependencies
run: pip install pyyaml

- name: Discover and run test_*.py scripts
env:
PYTHONIOENCODING: utf-8
run: |
set -euo pipefail

mapfile -t test_files < <(find skills -type f -name "test_*.py" | sort)

if [ "${#test_files[@]}" -eq 0 ]; then
echo "No test_*.py files found under skills/ — nothing to run."
exit 0
fi

echo "Found ${#test_files[@]} test file(s):"
printf ' %s\n' "${test_files[@]}"
echo

failures=()
for f in "${test_files[@]}"; do
echo "::group::$f"
if ! python "$f"; then
failures+=("$f")
fi
echo "::endgroup::"
done

echo
if [ "${#failures[@]}" -ne 0 ]; then
echo "FAILED (${#failures[@]}/${#test_files[@]}):"
printf ' %s\n' "${failures[@]}"
exit 1
fi

echo "All ${#test_files[@]} test file(s) passed."
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
**/__pycache__/*
**/.DS_Store

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This rule doesn't untrack already-committed files — this PR commits three .DS_Store files (repo root, skills/living-doc-gap-finder/scripts/, skills/living-doc-impact-analysis/scripts/). Please git rm --cached them.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Solved in 778a992.

skills-eval-workspace/*
agents-eval-workspace/*
.playwright-mcp/*
review-fix-roadmap.md
25 changes: 23 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,14 @@ skills/
| `scripts/` | Deterministic or repetitive logic better run as code than described in prose (e.g. a validation script, a formatter, a data transformer) |
| `references/` | Domain docs, API specs, decision tables, or anything too large to keep in `SKILL.md` without exceeding 500 lines |
| `assets/` | Template files, example inputs/outputs, icons — anything the skill produces or consumes |
| `evals/` | Test prompts and assertions to verify skill behavior and trigger accuracy. See [skill-testing.md](./docs/skill-testing.md) |
| `evals/` | Test prompts and assertions to verify skill behavior and trigger accuracy. See [skill-testing.md](./docs/testing/skill-testing.md) |

#### `references/` — shared vs. skill-specific

- **`skills/<skill>/references/`** — skill-specific docs loaded only by that skill's instructions
- **`skills/shared/references/`** — shared docs used by multiple skills (e.g., schemas, glossaries, common API specs)

If a reference file is used by 2+ skills, place it in `skills/shared/references/` instead of duplicating it. See [living-doc-bdd-schemas.md](skills/shared/references/living-doc-bdd-schemas.md) and [living-doc-glossary.md](skills/shared/references/living-doc-glossary.md) for examples.
---

## 2. Frontmatter schema
Expand Down Expand Up @@ -177,6 +183,21 @@ should be run") — it is clearer and easier for the agent to follow.

If every test run of your skill independently writes the same helper script (a formatter, a validator, a transformer), bundle it in `scripts/` and reference it from `SKILL.md`. This saves every future invocation from reinventing the wheel.

### Shared utilities and assets across multiple skills

**Pattern:** If code, references, or assets are used by 2+ skills, place them in `skills/shared/`:

- **`skills/shared/lib/`** for shared Python modules → each skill has a wrapper
- **`skills/shared/references/`** for shared documentation → skills link directly
- **`skills/shared/assets/`** for shared templates/data → skills reference relative paths

Examples:
- `skills/shared/lib/living_doc_id.py` — shared by three living-doc skills
- `skills/shared/references/living-doc-glossary.md` — shared by multiple living-doc skills
- See [skills/shared/lib/README.md](skills/shared/lib/README.md) for implementation details

**Benefits:** Single source of truth, prevents drift, easier maintenance, each skill remains independently deployable.

### Format conventions

- Use `##` and `###` headings to structure the body
Expand All @@ -201,7 +222,7 @@ If every test run of your skill independently writes the same helper script (a f
Before proposing a PR, verify that your skill activates correctly and produces good output. The full testing
methodology — eval creation, fixture management, with/without comparisons, trigger testing, and description
optimization using the Anthropic [`skill-creator`](https://github.com/anthropics/skills/tree/main/skills/skill-creator)
skill — is covered in **[docs/skill-testing.md](./docs/skill-testing.md)**.
skill — is covered in **[docs/testing/skill-testing.md](./docs/testing/skill-testing.md)**.

---

Expand Down
27 changes: 26 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,20 @@ its purpose, trigger phrases, and full instructions.

| Skill | Description |
|------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------|
| **[bdd-maintain](./skills/bdd-maintain/)** | REMOVE and DEAD CODE AUDIT modes — delete feature files, step definitions, and PageObjects linked to
| **[create-repository](./skills/create-repository/)** | Creates and configures a GitHub repository under AbsaOSS (open source) or absa-group (internal). Runs a short interview, then shows an annotated `gh` command plan and executes it on confirmation — visibility, naming, admins, CODEOWNERS, labels, merge settings, and project links. |
| **[data-cy-instrument](./skills/data-cy-instrument/)** | Resolve missing `data-cy` attributes in Angular component templates and sync PageObjects to use `getByTestId()` — run after a crawl when `coverage_gaps` are non-empty. |
| **[gherkin-living-doc-sync](./skills/gherkin-living-doc-sync/)** | Synchronise Gherkin feature files with the living documentation catalog — fix missing AC traceability headers, step text drift, and stale scenario links. |
| **[gherkin-step](./skills/gherkin-step/)** | Implement clean, reusable step definitions — behave (Python), Cucumber (Java, TypeScript, Scala), parameter types, DataTable, DocString, and hooks. |
| **[living-doc-create-user-story](./skills/living-doc-create-user-story/)** | Create a well-formed User Story with business-level Acceptance Criteria that are traceable, testable, and E2E-ready. |
| **[living-doc-create-feature](./skills/living-doc-create-feature/)** | Document a system surface (UI screen, API endpoint, service) as a Feature entity with ownership and traceability links. |
| **[living-doc-create-functionality](./skills/living-doc-create-functionality/)** | Define an atomic, testable behaviour (Functionality) with AC designed for fast unit or integration tests. |
| **[living-doc-update](./skills/living-doc-update/)** | Amend or deprecate existing User Story, Feature, or Functionality entities — add ACs, change status, update ownership. |
| **[living-doc-impact-analysis](./skills/living-doc-impact-analysis/)** | Trace which Features, Functionalities, User Stories, and Gherkin scenarios are affected by a code change or PR. |
| **[living-doc-gap-finder](./skills/living-doc-gap-finder/)** | Identify undocumented behaviours, orphan tests, and untested ACs. Used by `@living-doc-bdd-copilot` top-down (catalog gaps) and bottom-up (scenario coverage). |
| **[living-doc-pageobject-scan](./skills/living-doc-pageobject-scan/)** | Discover, create, and maintain PageObject classes from a live web application — bootstrapping from scratch and detecting selector drift after UI changes. |
deprecated entities; audit unused steps and PageObject methods. |
| **[living-doc-scenario-creator](./skills/living-doc-scenario-creator/)** | Generate full Gherkin feature files from User Story and Functionality ACs — feature file header, @AC:-tagged scenarios, complete Given/When/Then step bodies, coverage report, and step definition resolution. |
| **[pr-review](./skills/pr-review/)** | Pull request code review — reviews diffs for risk, security issues, API contract changes, dependency bumps, CI/CD and infrastructure changes. Produces concise Blocker / Important / Nit comments. |
| **[tdd-workflow](./skills/tdd-workflow/)** | Test-driven development: upfront SPEC.md planning + confirmation gate (avoids batch design), then vertical-sliced implementation (one test → one code cycle at a time, not all tests then all code). |
| **[test-data-management](./skills/test-data-management/)** | Test data setup and management — factory functions, parametrised tests, deterministic seeds, fixture reuse, and production-data rules for unit and integration tests. |
Expand All @@ -87,6 +100,16 @@ its purpose, trigger phrases, and full instructions.
| **[test-unit-review](./skills/test-unit-review/)** | Systematically audit unit test suites. Runs test runner, checks isolation/scope/naming/assertions/coverage standards, and reports findings by severity (Blocker / Important / Nit). |
| **[token-saving](./skills/token-saving/)** | Always-active response discipline — enforces brevity, no filler openers or closers, structured output, and a What/Why/How footer on code responses. Suspends on explicit "full detail" requests. |

## Agent Roster

Agents are pre-configured AI personas that orchestrate multiple skills for a specific engineering phase. Agent files live in **[.github/agents/](./.github/agents/)**.



| Agent | Description |
|---|---|
| **[@living-doc-bdd-copilot](./.github/agents/living-doc-bdd-copilot.agent.md)** | Living documentation and BDD automation — catalog management (User Stories, Features, Functionalities, AC updates, impact analysis, gap finding) and automation (webapp exploration, PageObjects, Gherkin scenarios, step definitions, BDD suite maintenance). |
Comment thread
miroslavpojer marked this conversation as resolved.
Comment thread
miroslavpojer marked this conversation as resolved.
Comment thread
miroslavpojer marked this conversation as resolved.

Comment thread
miroslavpojer marked this conversation as resolved.
## Finding More Skills

Before building a new skill, check whether one already exists:
Expand All @@ -96,11 +119,13 @@ Before building a new skill, check whether one already exists:
| [github/awesome-copilot](https://github.com/github/awesome-copilot/tree/main/skills) | 200+ community skills: cloud, languages, security, DevOps, productivity |
| [skills.sh](https://skills.sh) | Open registry — install with `npx skills add <owner/repo>` |
| [anthropics/skills](https://github.com/anthropics/skills) | Anthropic reference skills including `skill-creator` |
| [absa-group/agent-skills](https://github.com/absa-group/agent-skills) | Broader ABSA-owned skill collection |
| [absa-group/cps-agentic-toolkit](https://github.com/absa-group/cps-agentic-toolkit) | CPS team's extended skill set (ABSA-internal) |

## Contributing

See **[CONTRIBUTING.md](./CONTRIBUTING.md)** for the skill authoring guide — folder layout, frontmatter schema, writing
effective descriptions and bodies, [testing](./docs/skill-testing.md), and the PR checklist.
effective descriptions and bodies, [testing](./docs/testing/skill-testing.md), and the PR checklist.

To propose a new skill — or to propose expanding the repo into agents, MCP servers, or
plugins — [open an issue](https://github.com/AbsaOSS/agentic-toolkit/issues/new).
Expand Down
Loading
Loading