|
1 | 1 | --- |
2 | | -llms_description: "Plugin system for extending Testo: event listeners, middleware, custom discovery (documentation in progress)" |
| 2 | +faqLevel: 2 |
| 3 | +outline: [2, 3] |
| 4 | +llms: true |
| 5 | +llms_description: "PluginConfigurator interface, creating plugins with interceptors and event listeners, FallbackInterceptor for attribute-based extensions" |
3 | 6 | --- |
4 | 7 |
|
5 | 8 | # Plugins |
6 | 9 |
|
7 | | -Testo's plugin system allows you to extend and customize the testing framework behavior. |
| 10 | +A plugin in Testo is an independent module responsible for a specific framework feature. Test discovery, assertions (<plugin>Assert</plugin>), lifecycle hooks (<plugin>Lifecycle</plugin>), benchmarks (<plugin>Bench</plugin>), filtering (<plugin>Filter</plugin>) — these are all separate plugins. The more plugins you enable, the more capabilities you get. Any of them can be disabled, replaced, or extended with your own. |
8 | 11 |
|
9 | | -::: tip Coming Soon |
10 | | -This documentation section is under development. In the meantime, you can explore the [Events](./events.md) system for extending test behavior. |
| 12 | +A plugin can consist of a configurator, interceptors, attributes, event listeners — in any combination. For example, the <plugin>Assert</plugin> plugin uses a configurator to register interceptors, while <plugin>Retry</plugin> has no configurator and works entirely through an attribute or interceptor. |
| 13 | + |
| 14 | +## Plugin configurator |
| 15 | + |
| 16 | +A configurator is a class implementing the <class>\Testo\Common\PluginConfigurator</class> interface: |
| 17 | + |
| 18 | +```php |
| 19 | +interface PluginConfigurator |
| 20 | +{ |
| 21 | + public function configure(Container $container): void; |
| 22 | +} |
| 23 | +``` |
| 24 | + |
| 25 | +When a plugin is loaded, Testo calls `configure()` and passes its internal DI container. The configurator uses the container to access the framework API. Configurators are registered at two levels: application and Test Suite — see [Configuration — Plugins](./configuration.md#plugins) for details. |
| 26 | + |
| 27 | +The container provides three main extension points: |
| 28 | + |
| 29 | +### Interceptors |
| 30 | + |
| 31 | +Interceptors are middleware that hook into test discovery and execution pipelines. See the [interceptors](./interceptors.md) page for details. |
| 32 | + |
| 33 | +Interceptors are registered via <class>\Testo\Pipeline\InterceptorCollector</class>: |
| 34 | + |
| 35 | +```php |
| 36 | +$container->get(InterceptorCollector::class)->addInterceptor(new MyInterceptor()); |
| 37 | +``` |
| 38 | + |
| 39 | +### Event listeners |
| 40 | + |
| 41 | +Testo emits [events](./events.md) at every stage of execution: session start and finish, Test Suite, Test Case, and individual tests. A configurator can subscribe to any of these events via <class>\Testo\Common\EventListenerCollector</class>: |
| 42 | + |
| 43 | +```php |
| 44 | +$container->get(EventListenerCollector::class)->addListener( |
| 45 | + TestFinished::class, |
| 46 | + function (TestFinished $event) { |
| 47 | + // React to test completion |
| 48 | + }, |
| 49 | +); |
| 50 | +``` |
| 51 | + |
| 52 | +This mechanism follows the [PSR-14](https://www.php-fig.org/psr/psr-14/) standard with one restriction: events are **always** immutable. The same applies to any custom events you create. |
| 53 | + |
| 54 | +See the [Events](./events.md) page for the full list. |
| 55 | + |
| 56 | +### Container bindings |
| 57 | + |
| 58 | +A configurator can register services in the DI container that will be available to interceptors and other framework components. |
| 59 | + |
| 60 | +::: info |
| 61 | +Each Test Suite runs in its own container scope. This means that bindings and cached services from a Test Suite-level configurator are isolated — different Test Suites won't share state. |
11 | 62 | ::: |
12 | 63 |
|
13 | | -## Overview |
| 64 | +```php |
| 65 | +// Factory — service is created lazily on first access |
| 66 | +$container->bind(MyService::class, static fn(Container $c) => new MyService( |
| 67 | + $c->get(EventDispatcherInterface::class), |
| 68 | +)); |
| 69 | + |
| 70 | +// Ready instance — immediately available via get() |
| 71 | +$container->set(new MyConfig(timeout: 30)); |
| 72 | + |
| 73 | +// Retrieving a service from the container |
| 74 | +$dispatcher = $container->get(EventDispatcherInterface::class); |
| 75 | + |
| 76 | +// Creating an instance without storing it in the container |
| 77 | +$handler = $container->make(MyHandler::class, ['verbose' => true]); |
| 78 | +``` |
| 79 | + |
| 80 | +## Creating a plugin |
| 81 | + |
| 82 | +### Failed test logger |
| 83 | + |
| 84 | +Suppose you want to log every failed test to a file. Here's a configurator that listens to the <class>\Testo\Event\Test\TestPipelineFinished</class> event and writes the result: |
| 85 | + |
| 86 | +```php |
| 87 | +use Internal\Container\Container; |
| 88 | +use Testo\Common\EventListenerCollector; |
| 89 | +use Testo\Common\PluginConfigurator; |
| 90 | +use Testo\Event\Test\TestPipelineFinished; |
14 | 91 |
|
15 | | -Plugins in Testo provide a way to: |
16 | | -- Register custom event listeners |
17 | | -- Add middleware to test execution pipeline |
18 | | -- Integrate external tools and services |
19 | | -- Customize test discovery and execution |
| 92 | +final readonly class FailureLoggerPlugin implements PluginConfigurator |
| 93 | +{ |
| 94 | + public function __construct( |
| 95 | + private string $logFile = 'test-failures.log', |
| 96 | + ) {} |
20 | 97 |
|
21 | | -## Creating a Plugin |
| 98 | + #[\Override] |
| 99 | + public function configure(Container $container): void |
| 100 | + { |
| 101 | + $container->get(EventListenerCollector::class)->addListener( |
| 102 | + TestPipelineFinished::class, |
| 103 | + $this->onTestFinished(...), |
| 104 | + ); |
| 105 | + } |
22 | 106 |
|
23 | | -Documentation coming soon. |
| 107 | + private function onTestFinished(TestPipelineFinished $event): void |
| 108 | + { |
| 109 | + if (!$event->testResult->status->isFailure()) { |
| 110 | + return; |
| 111 | + } |
24 | 112 |
|
25 | | -## Registering Plugins |
| 113 | + $line = \sprintf( |
| 114 | + "[%s] %s %s::%s: %s\n", |
| 115 | + \date('Y-m-d H:i:s'), |
| 116 | + \strtoupper($event->testResult->status->name), |
| 117 | + $event->testInfo->caseInfo->definition->reflection?->getName(), |
| 118 | + $event->testInfo->testDefinition->reflection->getName(), |
| 119 | + \str_replace("\n", ' ', $event->testResult->failure?->getMessage() ?? 'unknown'), |
| 120 | + ); |
26 | 121 |
|
27 | | -Documentation coming soon. |
| 122 | + \file_put_contents($this->logFile, $line, \FILE_APPEND); |
| 123 | + } |
| 124 | +} |
| 125 | +``` |
28 | 126 |
|
29 | | -## Built-in Plugins |
| 127 | +Things to note: |
30 | 128 |
|
31 | | -Documentation coming soon. |
| 129 | +- The configurator accepts `$logFile` in its constructor, so you can customize it per registration. |
| 130 | +- <class>\Testo\Event\Test\TestPipelineFinished</class> fires after all interceptors (including retries), so it carries the final result. |
| 131 | +- `$event->testResult->status->isFailure()` returns `true` for both `Failed` and `Error`. |
| 132 | + |
| 133 | + |
| 134 | +### Flaky report in Pull Request |
| 135 | + |
| 136 | +Let's build a plugin that collects flaky tests (ones that only passed after a <plugin>Retry</plugin>) and posts a comment on the GitHub PR: |
| 137 | + |
| 138 | +```php |
| 139 | +use Internal\Container\Container; |
| 140 | +use Testo\Common\EventListenerCollector; |
| 141 | +use Testo\Common\PluginConfigurator; |
| 142 | +use Testo\Core\Value\Status; |
| 143 | +use Testo\Event\Framework\SessionFinished; |
| 144 | +use Testo\Event\Test\TestPipelineFinished; |
| 145 | + |
| 146 | +final class FlakyPRCommentPlugin implements PluginConfigurator |
| 147 | +{ |
| 148 | + #[\Override] |
| 149 | + public function configure(Container $container): void |
| 150 | + { |
| 151 | + // Check that we're in GitHub Actions and this is a PR |
| 152 | + $token = \getenv('GITHUB_TOKEN'); |
| 153 | + $repo = \getenv('GITHUB_REPOSITORY'); // owner/repo |
| 154 | + $ref = (string) \getenv('GITHUB_REF'); // refs/pull/123/merge |
| 155 | + if (!$token || !$repo || !\preg_match('#^refs/pull/(\d+)/#', $ref, $m)) { |
| 156 | + return; // not in CI or not a PR — do nothing |
| 157 | + } |
| 158 | + |
| 159 | + $prNumber = $m[1]; |
| 160 | + |
| 161 | + $listeners = $container->get(EventListenerCollector::class); |
| 162 | + $listeners->addListener(TestPipelineFinished::class, $this->onTestFinished(...)); |
| 163 | + $listeners->addListener( |
| 164 | + SessionFinished::class, |
| 165 | + fn(SessionFinished $e) => $this->postComment($token, $repo, $prNumber), |
| 166 | + ); |
| 167 | + } |
| 168 | + |
| 169 | + /** @var list<string> */ |
| 170 | + private array $flakyTests = []; |
| 171 | + |
| 172 | + private function onTestFinished(TestPipelineFinished $event): void |
| 173 | + { |
| 174 | + if ($event->testResult->status !== Status::Flaky) { |
| 175 | + return; |
| 176 | + } |
| 177 | + |
| 178 | + $case = $event->testInfo->caseInfo->definition->reflection?->getShortName(); |
| 179 | + $test = $event->testInfo->testDefinition->reflection->getName(); |
| 180 | + $this->flakyTests[] = $case === null ? "{$test}()" : "{$case}::{$test}()"; |
| 181 | + } |
| 182 | + |
| 183 | + private function postComment(string $token, string $repo, string $prNumber): void |
| 184 | + { |
| 185 | + if ($this->flakyTests === []) { |
| 186 | + return; |
| 187 | + } |
| 188 | + |
| 189 | + $list = \implode("\n", \array_map( |
| 190 | + static fn(string $name) => "- `{$name}`", |
| 191 | + $this->flakyTests, |
| 192 | + )); |
| 193 | + |
| 194 | + @\file_get_contents( |
| 195 | + "https://api.github.com/repos/{$repo}/issues/{$prNumber}/comments", |
| 196 | + context: \stream_context_create([ |
| 197 | + 'http' => [ |
| 198 | + 'method' => 'POST', |
| 199 | + 'header' => \implode("\r\n", [ |
| 200 | + "Authorization: Bearer {$token}", |
| 201 | + 'Content-Type: application/json', |
| 202 | + 'User-Agent: Testo', |
| 203 | + ]), |
| 204 | + 'content' => \json_encode([ |
| 205 | + 'body' => "⚠️ **Flaky tests detected**\n\n{$list}", |
| 206 | + ]), |
| 207 | + ], |
| 208 | + ]), |
| 209 | + ); |
| 210 | + } |
| 211 | +} |
| 212 | +``` |
| 213 | + |
| 214 | +This configurator subscribes to two events: <class>\Testo\Event\Test\TestPipelineFinished</class> to collect flaky tests, and <class>\Testo\Event\Framework\SessionFinished</class> to post the comment once the session ends. When run locally, the environment variables are missing, so `configure()` returns early — no listeners, no side effects. |
| 215 | + |
| 216 | +::: info |
| 217 | +Register this plugin at the application level (`ApplicationConfig::$plugins`), not per Test Suite — the <class>\Testo\Event\Framework\SessionFinished</class> event fires outside the Test Suite scope. |
| 218 | + |
| 219 | +```php |
| 220 | +return new ApplicationConfig( |
| 221 | + suites: [...], |
| 222 | + plugins: [ |
| 223 | + new FlakyPRCommentPlugin(), |
| 224 | + ], |
| 225 | +); |
| 226 | +``` |
| 227 | +::: |
| 228 | + |
| 229 | +See this plugin in action: [php-testo/testo#107](https://github.com/php-testo/testo/pull/107). |
| 230 | + |
| 231 | +::: question How to configure GitHub Actions for this plugin? |
| 232 | +Give the workflow permission to write to PRs and pass the token in the test step: |
| 233 | + |
| 234 | +```yaml |
| 235 | +on: [ 'pull_request' ] |
| 236 | + |
| 237 | +permissions: |
| 238 | + pull-requests: write |
| 239 | + |
| 240 | +jobs: |
| 241 | + tests: |
| 242 | + steps: |
| 243 | + - run: vendor/bin/testo |
| 244 | + env: |
| 245 | + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} |
| 246 | +``` |
| 247 | +::: |
32 | 248 |
|
33 | | -## Examples |
| 249 | +## Plugins without a configurator |
34 | 250 |
|
35 | | -Documentation coming soon. |
| 251 | +Not all plugins need a configurator. Some, like <plugin>Retry</plugin> and <plugin>Data</plugin>, work entirely through PHP attributes with automatic interceptor activation. See [interceptors](./interceptors.md) for details. |
0 commit comments