Add `assertDoesNotAddMessages` to `CheckerTestCase` by ShehabSherif0 · Pull Request #10930 · pylint-dev/pylint

ShehabSherif0 · 2026-03-20T10:38:33Z

Summary

Adds assertDoesNotAddMessages(*message_ids: str) to CheckerTestCase as a context manager that asserts specific message IDs are not emitted during a checker run, while allowing other messages to be present.

This fills the gap between assertNoMessages() (no messages at all) and assertAddsMessages() (exact set of messages). Now you can write tests like:

with self.assertDoesNotAddMessages("missing-module-docstring"):
    self.checker.visit_module(node)

Motivation

Closes #9598. When testing checkers, it is useful to verify a specific message is not raised for a given input without requiring that zero messages are raised. For example, confirming a checker no longer emits a false positive for a pattern, even if it still correctly emits other warnings.

Changes

pylint/testutils/checker_test_case.py: Added assertDoesNotAddMessages(*message_ids: str) context manager.
tests/testutils/test_checker_test_case.py: New test file covering all six scenarios from the issue plus edge cases.
doc/whatsnew/fragments/9598.internal: Towncrier news fragment.

Test Scenarios

Following the scenarios from @Pierre-Sassoulas in the issue:

#	Scenario	Method	Expected
1	expected raised / actual raised	`assertAddsMessages`	pass
2	expected raised / actual not raised	`assertAddsMessages`	fail
3	expected raised / actual not raised but another raised	`assertAddsMessages`	fail
4	expected not raised / actual raised	`assertDoesNotAddMessages`	fail
5	expected not raised / actual not raised	`assertDoesNotAddMessages`	pass
6	expected not raised / actual not raised but another raised	`assertDoesNotAddMessages`	pass

Plus edge cases: no arguments raises TypeError, multiple unwanted IDs fail if any match, and exceptions inside the body drain the message buffer cleanly.

Add a new context manager method to CheckerTestCase that asserts specific messages are NOT emitted, while allowing other messages to be present. This complements assertNoMessages which asserts that no messages at all are emitted. Also adds a _messages_match static helper for comparing two MessageTest instances field-by-field. Includes tests covering all six scenarios outlined in the issue plus additional edge cases for ignore_position and multiple unwanted messages. Closes pylint-dev#9598

for more information, see https://pre-commit.ci

Copilot

Pull request overview

Adds a new negative-assertion helper to Pylint’s checker unit-test infrastructure, enabling tests to assert that specific messages are not emitted while still allowing other messages.

Changes:

Added CheckerTestCase.assertDoesNotAddMessages(*messages, ignore_position=False) plus a helper for MessageTest comparisons.
Added unit tests covering the expected pass/fail scenarios and ignore_position behavior.
Added a towncrier news fragment documenting the feature.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File	Description
`pylint/testutils/checker_test_case.py`	Introduces `assertDoesNotAddMessages` and message-matching helper used by test cases.
`tests/testutils/test_checker_test_case.py`	Adds tests validating the new assertion’s behavior across multiple scenarios.
`doc/whatsnew/fragments/9598.feature`	Documents the new test utility in the changelog fragments.

- Use try/except/else instead of bare yield so messages are always drained and never leak into subsequent tests when the with-block raises an exception - Raise TypeError when called with no arguments (vacuous call) - Fix double space in docstring - Reformat news fragment: wrap long line, use Refs instead of Closes

for more information, see https://pre-commit.ci

codecov · 2026-03-20T16:34:13Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.17%. Comparing base (d796007) to head (9f088f4).
⚠️ Report is 8 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main   #10930   +/-   ##
=======================================
  Coverage   96.16%   96.17%           
=======================================
  Files         178      178           
  Lines       19627    19649   +22     
=======================================
+ Hits        18875    18898   +23     
+ Misses        752      751    -1

Files with missing lines	Coverage Δ
pylint/testutils/checker_test_case.py	`98.38% <100.00%> (+0.56%)`	⬆️

... and 3 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

…rage Add targeted unit tests for each early-return False branch in _messages_match: node, args, confidence, col_offset, end_line, and end_col_offset mismatches. This brings patch coverage from 83.33% to 100%.

for more information, see https://pre-commit.ci

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

Pierre-Sassoulas

Thank you for contributing to pylint. Sorry fr the delay I didn't send the review I thought I sent.

Pierre-Sassoulas · 2026-03-22T15:09:55Z

+        if expected.node != actual.node:
+            return False
+        if expected.args != actual.args:
+            return False
+        if expected.confidence != actual.confidence:
+            return False
+        if not ignore_position:
+            if expected.line != actual.line:
+                return False
+            if expected.col_offset != actual.col_offset:
+                return False
+            if expected.end_line != actual.end_line:
+                return False
+            if expected.end_col_offset != actual.end_col_offset:
+                return False
+        return True


Don't we only care about the msgid ?

You are right. The method now accepts plain message ID strings instead of full MessageTest objects, matching the intended use pattern from the issue. The check is just: if that message ID appears anywhere in the emitted messages, fail. Position, args, node and confidence are all irrelevant to the "did this message fire at all" question. The _messages_match helper and ignore_position parameter have been removed.

- assertDoesNotAddMessages now accepts message ID strings instead of full MessageTest objects, matching the intended use case described in the issue: checking that a specific message was never triggered regardless of position, args, or node - Remove _messages_match helper and ignore_position parameter as they are no longer needed with the simpler msg_id-only matching - Rename news fragment from 9598.feature to 9598.internal since this change touches the plugin developer API, not an end-user feature - Simplify tests to use plain string IDs

for more information, see https://pre-commit.ci

Replace try/except/else with try/except/finally to avoid R1720. The exception_raised flag tracks whether the with-block raised, so the finally block skips message checks when propagating an exception from the test body.

Pierre-Sassoulas

Thank you, let's add some assertion about the message raised and let's merge.

Co-authored-by: Pierre Sassoulas <pierre.sassoulas@gmail.com>

for more information, see https://pre-commit.ci

github-actions · 2026-03-26T06:22:36Z

🤖 According to the primer, this change has no effect on the checked open source code. 🤖🎉

This comment was generated for commit 9f088f4

Copilot AI review requested due to automatic review settings March 20, 2026 10:38

Copilot started reviewing on behalf of ShehabSherif0 March 20, 2026 10:38 View session

[pre-commit.ci] auto fixes from pre-commit.com hooks

3299943

for more information, see https://pre-commit.ci

Copilot AI reviewed Mar 20, 2026

View reviewed changes

Comment thread pylint/testutils/checker_test_case.py Outdated

Comment thread pylint/testutils/checker_test_case.py Outdated

Comment thread pylint/testutils/checker_test_case.py Outdated

Comment thread doc/whatsnew/fragments/9598.feature Outdated

ShehabSherif0 and others added 2 commits March 20, 2026 12:49

[pre-commit.ci] auto fixes from pre-commit.com hooks

4c98679

for more information, see https://pre-commit.ci

This comment has been minimized.

Sign in to view

ShehabSherif0 and others added 2 commits March 20, 2026 21:32

test: cover all _messages_match mismatch branches for 100% patch cove…

faedc02

…rage Add targeted unit tests for each early-return False branch in _messages_match: node, args, confidence, col_offset, end_line, and end_col_offset mismatches. This brings patch coverage from 83.33% to 100%.

[pre-commit.ci] auto fixes from pre-commit.com hooks

0f15cc2

for more information, see https://pre-commit.ci

This comment has been minimized.

Sign in to view

ShehabSherif0 requested a review from Copilot March 21, 2026 06:43

Copilot started reviewing on behalf of ShehabSherif0 March 21, 2026 06:43 View session

Copilot AI reviewed Mar 21, 2026

View reviewed changes

Comment thread tests/testutils/test_checker_test_case.py Outdated

ShehabSherif0 marked this pull request as draft March 21, 2026 06:56

Pierre-Sassoulas added Enhancement ✨ Improvement to a component Maintenance Discussion or action around maintaining pylint or the dev workflow labels Mar 22, 2026

Pierre-Sassoulas added this to the 4.1.0 milestone Mar 22, 2026

Pierre-Sassoulas reviewed Mar 25, 2026

View reviewed changes

ShehabSherif0 and others added 2 commits March 26, 2026 01:44

[pre-commit.ci] auto fixes from pre-commit.com hooks

8a3aaec

for more information, see https://pre-commit.ci

ShehabSherif0 marked this pull request as ready for review March 25, 2026 23:52

Fix no-else-raise lint warning in assertDoesNotAddMessages

9f088f4

Replace try/except/else with try/except/finally to avoid R1720. The exception_raised flag tracks whether the with-block raised, so the finally block skips message checks when propagating an exception from the test body.

Pierre-Sassoulas requested changes Mar 26, 2026

View reviewed changes

Pierre-Sassoulas and others added 2 commits March 26, 2026 07:09

Less LLM comments, skeleton of raise match assertion

12d10c4

Co-authored-by: Pierre Sassoulas <pierre.sassoulas@gmail.com>

[pre-commit.ci] auto fixes from pre-commit.com hooks

7d08354

for more information, see https://pre-commit.ci

Uh oh!

Conversation

ShehabSherif0 commented Mar 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Motivation

Changes

Test Scenarios

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

codecov Bot commented Mar 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

This comment has been minimized.

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Pierre-Sassoulas left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Pierre-Sassoulas Mar 22, 2026

Choose a reason for hiding this comment

Uh oh!

ShehabSherif0 Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Pierre-Sassoulas left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Mar 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ShehabSherif0 commented Mar 20, 2026 •

edited

Loading

codecov Bot commented Mar 20, 2026 •

edited

Loading