Skip to content

Update sdk-tests skill matcher and assertion rules - #4108

Merged
ambushwork merged 1 commit into
developfrom
ylu/update-sdk-tests-skill-rules-36cad118
Nov 5, 2026
Merged

ambushwork merged 1 commit into
developfrom
ylu/update-sdk-tests-skill-rules-36cad118

Conversation

@ambushwork

Copy link
Copy Markdown
Member

What does this PR do?

Updates the sdk-tests skill (ai/skills/sdk-tests/SKILL.md):

  • Removes the "no region comments" and "no inline comments in test bodies" rules.
  • Adds a rule to use argumentCaptor and assert individual fields, never argThat { … }.
  • Adds a rule to import Mockito/AssertJ symbols instead of writing fully qualified names inline.
  • Adds "omit defaults" to the matcher guidance: no times(1), no eq() when every matcher would be eq, and never() instead of times(0).
  • Adds a rule to use the most specific assertion (isTrue()/isFalse(), isEqualTo(list) for deterministic order).
  • Adds a rule to assert the message, not only the level and target, when verifying log calls.

Motivation

An audit of the skill against existing tests showed the region and inline-comment rules conflict with thousands of existing lines. The new rules record conventions reviewers keep enforcing.

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • Make sure you discussed the feature or bugfix with the maintaining team in an Issue
  • Make sure each commit and the PR mention the Issue number (cf the CONTRIBUTING doc)

@ambushwork
ambushwork marked this pull request as ready for review November 5, 2026 13:20
@ambushwork
ambushwork requested review from a team as code owners November 5, 2026 13:20
@ambushwork ambushwork added the ready-for-human-review All agentic review feedback was addressed and PR is ready for the human review. label Nov 5, 2026
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-05T13:51:14.575697Z d175e79 New commits
🔒 Security Review ✅ Completed 2026-11-05T13:51:53.361943Z d175e79 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@datadog-datadog-prod-us1

datadog-datadog-prod-us1 Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

Pipelines  Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 72.12% (-0.04%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: d175e79 | Docs | View more details | Give us feedback!

@datadog-datadog-prod-us1 datadog-datadog-prod-us1 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

The captor, matcher, and logger-message assertion guidance is consistent with existing test utilities and repository conventions.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit 9cdb568

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9cdb568733

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread ai/skills/sdk-tests/SKILL.md Resolved
kikoveiga
kikoveiga previously approved these changes Nov 5, 2026

@kikoveiga kikoveiga left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is nice! Both the imports rule and argumentCaptor vs argThat are important for AI to follow.

As more and more code is generated, we can think about improving or transferring these guidelines into hard detekt rules. This reminds me of PreferTimeProvider or other custom detekt rules we have, also working on enabling an UnnecessaryFullyQualifiedName detekt rule in #4105.

@ambushwork
ambushwork force-pushed the ylu/update-sdk-tests-skill-rules-36cad118 branch from 9cdb568 to d175e79 Compare November 5, 2026 13:49
verify(mockInstrumentation).onResponseSucceeded(any(), capture())
```

### Use `argumentCaptor`, never `argThat`

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is a certain repetition, because it is already mentioned in AGENTS.md. I think we can restructure AGENTS.md to cut the stuff which is in the skills. We need to have proper forwarding.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True, seems there is a whole section overlaps with this file

@ambushwork
ambushwork merged commit b2d2deb into develop Nov 5, 2026
26 checks passed
@ambushwork
ambushwork deleted the ylu/update-sdk-tests-skill-rules-36cad118 branch November 5, 2026 14:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human-review All agentic review feedback was addressed and PR is ready for the human review.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants