Skip to content

Commit a7a74a0

Browse files
committed
docs(v0.20.0): add documentation for --failed flag and unknown argument detection
v0.20.0 introduces two CLI improvements that enhance the developer workflow: 1. --failed (-F) flag: Re-run only scenarios that failed in the previous run. This accelerates the TDD cycle by skipping passing tests during iteration. State is persisted to .probitas/last-run.json between runs. 2. Unknown argument detection: Provides contextual hints for common mistakes (--tag → -s "tag:value") and suggests similar options for typos using Levenshtein distance (--verbos → --verbose). Added comprehensive documentation in: - docs/configuration.md: Two new sections with detailed usage examples - docs/overview.md: Quick reference in "Running Scenarios" section These features improve discoverability and reduce friction when writing and debugging tests.
1 parent 211b446 commit a7a74a0

2 files changed

Lines changed: 105 additions & 0 deletions

File tree

docs/configuration.md

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -156,6 +156,108 @@ probitas run --max-failures 1
156156
probitas run -s tag:smoke
157157
```
158158

159+
## Re-running Failed Scenarios
160+
161+
Use the `--failed` (`-F`) flag to re-run only scenarios that failed in the
162+
previous run. This is useful for iterating on failing tests without waiting for
163+
all scenarios to complete.
164+
165+
```bash
166+
# Run all scenarios
167+
probitas run
168+
169+
# Re-run only scenarios that failed
170+
probitas run --failed
171+
172+
# Short form
173+
probitas run -F
174+
```
175+
176+
### How It Works
177+
178+
1. After each `probitas run`, the CLI saves the list of failed scenarios to
179+
`.probitas/last-run.json`
180+
2. When `--failed` is specified, only scenarios matching the saved list are
181+
executed
182+
3. The filter is applied after selectors (AND logic)
183+
184+
```bash
185+
# Run failed scenarios that also have @api tag
186+
probitas run -F -s tag:api
187+
188+
# Run failed scenarios excluding @slow tag
189+
probitas run -F -s "!tag:slow"
190+
```
191+
192+
### State File
193+
194+
The `.probitas/` directory contains machine-specific state and should be added
195+
to `.gitignore`:
196+
197+
```gitignore
198+
# Probitas state directory
199+
.probitas/
200+
```
201+
202+
The state file (`last-run.json`) contains:
203+
204+
- Schema version for forward compatibility
205+
- Timestamp of when the run completed
206+
- List of failed scenarios (name, file path, and error message)
207+
208+
## Unknown Argument Detection
209+
210+
Probitas CLI provides helpful error messages when you use unknown options. This
211+
helps catch typos and guides you toward the correct syntax.
212+
213+
### Common Mistakes
214+
215+
The CLI recognizes common mistakes and provides contextual hints:
216+
217+
```bash
218+
# Mistake: --tag instead of -s "tag:value"
219+
$ probitas run --tag api
220+
Unknown option: --tag
221+
Did you mean '-s "tag:api"'? Use the selector option to filter by tag.
222+
223+
# Mistake: --name instead of -s "name:value"
224+
$ probitas run --name Login
225+
Unknown option: --name
226+
Did you mean '-s "name:Login"'? Use the selector option to filter by name.
227+
228+
# Mistake: --filter instead of -s
229+
$ probitas run --filter smoke
230+
Unknown option: --filter
231+
Did you mean '-s "smoke"'? Use the selector option to filter scenarios.
232+
```
233+
234+
### Typo Detection
235+
236+
For other unknown options, the CLI suggests similar known options using
237+
Levenshtein distance:
238+
239+
```bash
240+
# Typo: --verbos instead of --verbose
241+
$ probitas run --verbos
242+
Unknown option: --verbos
243+
Did you mean '--verbose'?
244+
245+
# Typo: --time-out instead of --timeout
246+
$ probitas run --time-out 10s
247+
Unknown option: --time-out
248+
Did you mean '--timeout'?
249+
```
250+
251+
### Fallback Help
252+
253+
When no similar option is found, you'll be directed to the help command:
254+
255+
```bash
256+
$ probitas run --unknown-flag
257+
Unknown option: --unknown-flag
258+
Run 'probitas run --help' for available options.
259+
```
260+
159261
## Scenario Options
160262

161263
Configure scenarios using the options parameter of

docs/overview.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,6 +99,9 @@ probitas run -s tag:example
9999

100100
# Run with different reporter
101101
probitas run --reporter json
102+
103+
# Re-run only failed scenarios from previous run
104+
probitas run --failed
102105
```
103106

104107
## Code Quality Commands

0 commit comments

Comments
 (0)