Repository navigation
chatcops test: JUnit reporter and --output #416
Description
CI systems (GitLab, Jenkins, Azure, the GitHub test report actions) read JUnit XML. This adds --reporter junit.
How reporters work
Reporters are registered in REPORTERS (packages/cli/src/test/reporters/index.ts:32). A Reporter has onResult(result), called as each test finishes, and onEnd(results, summary), called once with all of them and a RunSummary (mode, judge, cost). reporters/json.ts is the shortest example and reporters/pretty.ts the default. They get a ReporterOutput to write to, which is stdout today.
What to add
reporters/junit.ts: one<testsuite>per eval file and one<testcase>per test, withclassnamethe file andnamethe test name, the time in seconds from the turns'latencyMs, and a<failure>with the failed reasons and the reply for failed tests. Escape XML properly.--output <file>inpackages/cli/src/main.ts(next toreporter, around line 136). With one--reporter, write to the file instead of stdout. With several,--reporter junit=evals.xmlstyle is clearer, so supportname=fileper reporter and make--outputthe shortcut for a single one. Pick what you think reads best and explain it in the PR.
Acceptance criteria
- Tests: the XML for passing, failing and errored tests, escaping of
<,&and quotes, writing to a file - Documented in
website/src/content/docs/core/testing.mdx, with a changeset for@chatcops/cli(minor) -
pnpm test:coverageandpnpm lintpass
Activity
kachamsiddarth commentedon Oct 28, 2026
Hi @anurag629, i would like to work on this issue can you assign me?
anurag2787 commentedon Oct 28, 2026
Hi @anurag629 can i work on it
anurag629 commentedon Oct 28, 2026
@kachamsiddarth it's yours. Reporters are registered in
REPORTERS(packages/cli/src/test/reporters/index.ts:32), andreporters/json.tsis the shortest one to copy from.@anurag2787 kachamsiddarth asked first; you've got #415.
ashutoshpalhare commentedon Oct 29, 2026
Hi! I'd like to work on this issue for Hacktoberfest 2026. I plan to add the JUnit XML reporter, support output files, cover XML escaping and pass/fail/error cases with tests, and update the testing docs and changeset. Could you please assign this issue to me before I start?
anurag629 commentedon Oct 29, 2026
@ashutoshpalhare thanks, but this one is already assigned to @kachamsiddarth. #452, which you filed, is free and I've assigned it to you, see my note there.
anurag629 commentedon Nov 5, 2026
@kachamsiddarth checking in on this one. It's been eight days since I assigned it and there's no PR yet. If you're working on it locally, push your branch and open a PR, even as a draft, so I can see where it stands. If there's no update by Nov 8, I'll free it so someone else can pick it up. You're welcome to ask for it again later.
bhushan4work commentedon Nov 5, 2026
I would like to work on this one , if theres no response till 8 nov then willl self assign and write a fix .
anurag629 commentedon Nov 6, 2026
@bhushan4work kachamsiddarth opened #590 for this one, so it stays with them. #417 still opens up on Nov 8 if there's no update there.
bhushan4work commentedon Nov 6, 2026
works