Skip to content

conformance: bump type checker versions #2254

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
JelleZijlstra merged 1 commit into from
Apr 8, 2026
Merged

conformance: bump type checker versions #2254

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 5 additions & 6 deletions conformance/results/mypy/callables_kwargs.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ notes = """
Allows callable without kwargs to be assigned to callable with unpacked kwargs
"""
output = """
callables_kwargs.py:22: note: "func1" defined here
callables_kwargs.py:46: error: Missing named argument "v1" for "func1" [call-arg]
callables_kwargs.py:46: error: Missing named argument "v3" for "func1" [call-arg]
callables_kwargs.py:51: error: Unexpected keyword argument "v4" for "func1" [call-arg]
Expand All @@ -15,14 +14,14 @@ callables_kwargs.py:63: error: "func1" gets multiple values for keyword argument
callables_kwargs.py:64: error: "func2" gets multiple values for keyword argument "v3" [misc]
callables_kwargs.py:64: error: Argument 1 to "func2" has incompatible type "int"; expected "str" [arg-type]
callables_kwargs.py:65: error: "func2" gets multiple values for keyword argument "v1" [misc]
callables_kwargs.py:101: error: Incompatible types in assignment (expression has type "def func1(**kwargs: Unpack[TD2]) -> None", variable has type "TDProtocol3") [assignment]
callables_kwargs.py:101: error: Incompatible types in assignment (expression has type "def func1(**kwargs: **TD2) -> None", variable has type "TDProtocol3") [assignment]
callables_kwargs.py:101: note: "TDProtocol3.__call__" has type "def __call__(self, *, v1: int, v2: int, v3: str) -> None"
callables_kwargs.py:102: error: Incompatible types in assignment (expression has type "def func1(**kwargs: Unpack[TD2]) -> None", variable has type "TDProtocol4") [assignment]
callables_kwargs.py:102: error: Incompatible types in assignment (expression has type "def func1(**kwargs: **TD2) -> None", variable has type "TDProtocol4") [assignment]
callables_kwargs.py:102: note: "TDProtocol4.__call__" has type "def __call__(self, *, v1: int) -> None"
callables_kwargs.py:103: error: Incompatible types in assignment (expression has type "def func1(**kwargs: Unpack[TD2]) -> None", variable has type "TDProtocol5") [assignment]
callables_kwargs.py:103: error: Incompatible types in assignment (expression has type "def func1(**kwargs: **TD2) -> None", variable has type "TDProtocol5") [assignment]
callables_kwargs.py:103: note: "TDProtocol5.__call__" has type "def __call__(self, v1: int, v3: str) -> None"
callables_kwargs.py:111: error: Overlap between argument names and ** TypedDict items: "v1" [misc]
callables_kwargs.py:122: error: Unpack item in ** argument must be a TypedDict [misc]
callables_kwargs.py:111: error: Overlap between parameter names and ** TypedDict items: "v1" [misc]
callables_kwargs.py:122: error: Unpack item in ** parameter must be a TypedDict [misc]
"""
conformance_automated = "Fail"
errors_diff = """
Expand Down
16 changes: 8 additions & 8 deletions conformance/results/mypy/constructors_callable.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,33 +17,33 @@ Line 128: Unexpected errors ['constructors_callable.py:128: error: Expression is
Line 145: Unexpected errors ['constructors_callable.py:145: error: Expression is of type "Class6Any", not "Any" [assert-type]', 'constructors_callable.py:145: error: Too few arguments [call-arg]']
"""
output = """
constructors_callable.py:36: note: Revealed type is "def (x: builtins.int) -> constructors_callable.Class1"
constructors_callable.py:36: note: Revealed type is "def (x: int) -> constructors_callable.Class1"
constructors_callable.py:38: error: Too few arguments [call-arg]
constructors_callable.py:39: error: Unexpected keyword argument "y" [call-arg]
constructors_callable.py:49: note: Revealed type is "def () -> constructors_callable.Class2"
constructors_callable.py:51: error: Too many arguments [call-arg]
constructors_callable.py:64: note: Revealed type is "def (x: builtins.int) -> constructors_callable.Class3"
constructors_callable.py:64: note: Revealed type is "def (x: int) -> constructors_callable.Class3"
constructors_callable.py:66: error: Too few arguments [call-arg]
constructors_callable.py:67: error: Unexpected keyword argument "y" [call-arg]
constructors_callable.py:68: error: Too many arguments [call-arg]
constructors_callable.py:74: error: Incompatible return type for "__new__" (returns "int", but must return a subtype of "Class4") [misc]
constructors_callable.py:79: note: Revealed type is "def (x: builtins.int) -> constructors_callable.Class4"
constructors_callable.py:79: note: Revealed type is "def (x: int) -> constructors_callable.Class4"
constructors_callable.py:80: error: Expression is of type "Class4", not "int" [assert-type]
constructors_callable.py:81: error: Too few arguments [call-arg]
constructors_callable.py:82: error: Unexpected keyword argument "y" [call-arg]
constructors_callable.py:99: note: Revealed type is "def (*args: Any, **kwargs: Any) -> constructors_callable.Class5"
constructors_callable.py:102: error: Expression is of type "Class5", not "Never" [assert-type]
constructors_callable.py:107: error: Expression is of type "Class5", not "Never" [assert-type]
constructors_callable.py:118: error: Incompatible return type for "__new__" (returns "Class6Proxy", but must return a subtype of "Class6") [misc]
constructors_callable.py:127: note: Revealed type is "def (x: builtins.int) -> constructors_callable.Class6"
constructors_callable.py:127: note: Revealed type is "def (x: int) -> constructors_callable.Class6"
constructors_callable.py:128: error: Expression is of type "Class6", not "Class6Proxy" [assert-type]
constructors_callable.py:128: error: Too few arguments [call-arg]
constructors_callable.py:144: note: Revealed type is "def (x: builtins.int) -> constructors_callable.Class6Any"
constructors_callable.py:144: note: Revealed type is "def (x: int) -> constructors_callable.Class6Any"
constructors_callable.py:145: error: Expression is of type "Class6Any", not "Any" [assert-type]
constructors_callable.py:145: error: Too few arguments [call-arg]
constructors_callable.py:164: note: Revealed type is "Overload(def (x: builtins.int) -> constructors_callable.Class7[builtins.int], def (x: builtins.str) -> constructors_callable.Class7[builtins.str])"
constructors_callable.py:184: note: Revealed type is "def [T] (x: builtins.list[T`1], y: builtins.list[T`1]) -> constructors_callable.Class8[T`1]"
constructors_callable.py:164: note: Revealed type is "Overload(def (x: int) -> constructors_callable.Class7[int], def (x: str) -> constructors_callable.Class7[str])"
constructors_callable.py:184: note: Revealed type is "def [T] (x: list[T], y: list[T]) -> constructors_callable.Class8[T]"
constructors_callable.py:186: error: Cannot infer function type argument [misc]
constructors_callable.py:195: note: Revealed type is "def [T] (x: builtins.list[T`-1], y: builtins.list[T`-1]) -> constructors_callable.Class9"
constructors_callable.py:195: note: Revealed type is "def [T] (x: list[T], y: list[T]) -> constructors_callable.Class9"
constructors_callable.py:197: error: Cannot infer function type argument [misc]
"""
4 changes: 4 additions & 0 deletions conformance/results/mypy/dataclasses_descriptors.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,15 @@ notes = """
Does not correctly evaluate type of descriptor access.
"""
output = """
dataclasses_descriptors.py:61: error: Cannot access instance-only attribute "x" on class object [misc]
dataclasses_descriptors.py:62: error: Cannot access instance-only attribute "y" on class object [misc]
Comment on lines +6 to +7
Copy link
Copy Markdown
Member

@AlexWaygood AlexWaygood Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these new mypy diagnostics seem like they might be worth describing in the notes field? Especially because pyrefly and ty also object to the assertions on these lines, for basically the same reason: descriptors resolve differently when accessed on the class object than when accessed on instances, and the assertions on these lines don't seem fully cognisant of that aspect of the runtime behaviour

dataclasses_descriptors.py:66: error: Expression is of type "Desc2[int]", not "int" [assert-type]
dataclasses_descriptors.py:67: error: Expression is of type "Desc2[str]", not "str" [assert-type]
"""
conformance_automated = "Fail"
errors_diff = """
Line 61: Unexpected errors ['dataclasses_descriptors.py:61: error: Cannot access instance-only attribute "x" on class object [misc]']
Line 62: Unexpected errors ['dataclasses_descriptors.py:62: error: Cannot access instance-only attribute "y" on class object [misc]']
Line 66: Unexpected errors ['dataclasses_descriptors.py:66: error: Expression is of type "Desc2[int]", not "int" [assert-type]']
Line 67: Unexpected errors ['dataclasses_descriptors.py:67: error: Expression is of type "Desc2[str]", not "str" [assert-type]']
"""
2 changes: 1 addition & 1 deletion conformance/results/mypy/directives_deprecated.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ output = """
directives_deprecated.py:18: error: class _directives_deprecated_library.Ham is deprecated: Use Spam instead [deprecated]
directives_deprecated.py:24: error: function _directives_deprecated_library.norwegian_blue is deprecated: It is pining for the fjords [deprecated]
directives_deprecated.py:25: error: function _directives_deprecated_library.norwegian_blue is deprecated: It is pining for the fjords [deprecated]
directives_deprecated.py:30: error: overload def (x: builtins.int) -> builtins.str of function _directives_deprecated_library.foo is deprecated: Only str will be allowed [deprecated]
directives_deprecated.py:30: error: overload def (x: int) -> str of function _directives_deprecated_library.foo is deprecated: Only str will be allowed [deprecated]
directives_deprecated.py:41: error: function _directives_deprecated_library.Spam.__add__ is deprecated: There is enough spam in the world [deprecated]
directives_deprecated.py:42: error: function _directives_deprecated_library.Spam.__add__ is deprecated: There is enough spam in the world [deprecated]
directives_deprecated.py:44: error: function _directives_deprecated_library.Spam.greasy is deprecated: All spam will be equally greasy [deprecated]
Expand Down
4 changes: 2 additions & 2 deletions conformance/results/mypy/directives_reveal_type.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
conformant = "Pass"
output = """
directives_reveal_type.py:14: note: Revealed type is "builtins.int | builtins.str"
directives_reveal_type.py:15: note: Revealed type is "builtins.list[builtins.int]"
directives_reveal_type.py:14: note: Revealed type is "int | str"
directives_reveal_type.py:15: note: Revealed type is "list[int]"
directives_reveal_type.py:16: note: Revealed type is "Any"
directives_reveal_type.py:17: note: Revealed type is "directives_reveal_type.ForwardReference"
directives_reveal_type.py:19: error: "reveal_type" expects 1 argument [misc]
Expand Down
2 changes: 1 addition & 1 deletion conformance/results/mypy/generics_defaults_referential.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ output = """
generics_defaults_referential.py:23: error: Expression is of type "type[slice[StartT, StopT, StepT]]", not "type[slice[int, int, int | None]]" [assert-type]
generics_defaults_referential.py:38: error: Argument 1 to "Foo" has incompatible type "str"; expected "int" [arg-type]
generics_defaults_referential.py:54: error: Type parameter "Start2T" has a default type that refers to one or more type variables that are out of scope [misc]
generics_defaults_referential.py:61: error: Type variable S1 referenced in the default of S2 is unbound [misc]
generics_defaults_referential.py:75: error: TypeVar default must be one of the constraint types [misc]
generics_defaults_referential.py:78: error: TypeVar default must be one of the constraint types [misc]
generics_defaults_referential.py:79: error: TypeVar default must be one of the constraint types [misc]
Expand All @@ -14,7 +15,6 @@ generics_defaults_referential.py:104: error: Expression is of type "Bar[int, lis
conformance_automated = "Fail"
errors_diff = """
Line 37: Expected 1 errors
Line 61: Expected 1 errors
Line 69: Expected 1 errors
Line 23: Unexpected errors ['generics_defaults_referential.py:23: error: Expression is of type "type[slice[StartT, StopT, StepT]]", not "type[slice[int, int, int | None]]" [assert-type]']
Line 78: Unexpected errors ['generics_defaults_referential.py:78: error: TypeVar default must be one of the constraint types [misc]']
Expand Down
4 changes: 0 additions & 4 deletions conformance/results/mypy/historical_positional.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,9 @@ Treats keyword-only parameter as positional-only.
Applies legacy positional-only rules when PEP 570 syntax is used.
"""
output = """
historical_positional.py:13: note: "f1" defined here
historical_positional.py:18: error: Unexpected keyword argument "__x" for "f1" [call-arg]
historical_positional.py:45: note: "f3" defined here
historical_positional.py:48: error: Unexpected keyword argument "__y" for "f3" [call-arg]
historical_positional.py:52: note: "m1" of "A" defined here
historical_positional.py:59: error: Unexpected keyword argument "__x" for "m1" of "A" [call-arg]
historical_positional.py:66: note: "f4" defined here
historical_positional.py:69: error: Unexpected keyword argument "__y" for "f4" [call-arg]
"""
conformance_automated = "Fail"
Expand Down
2 changes: 1 addition & 1 deletion conformance/results/mypy/overloads_consistency.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,5 @@ errors_diff = """
"""
output = """
overloads_consistency.py:28: error: Overloaded function implementation cannot produce return type of signature 2 [misc]
overloads_consistency.py:44: error: Overloaded function implementation does not accept all possible arguments of signature 2 [misc]
overloads_consistency.py:44: error: Overloaded function implementation does not accept all possible parameters of signature 2 [misc]
"""
6 changes: 3 additions & 3 deletions conformance/results/mypy/tuples_type_compat.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,8 @@ tuples_type_compat.py:62: error: Incompatible types in assignment (expression ha
tuples_type_compat.py:76: error: Expression is of type "tuple[int]", not "tuple[int] | tuple[str, str] | tuple[int, *tuple[str, ...], int]" [assert-type]
tuples_type_compat.py:81: error: Expression is of type "tuple[str, str] | tuple[int, int]", not "tuple[int] | tuple[str, str] | tuple[int, *tuple[str, ...], int]" [assert-type]
tuples_type_compat.py:86: error: Expression is of type "tuple[int, str, int]", not "tuple[int] | tuple[str, str] | tuple[int, *tuple[str, ...], int]" [assert-type]
tuples_type_compat.py:101: error: Expression is of type "tuple[int] | tuple[str, str] | tuple[int, *tuple[str, ...], int]", not "tuple[int]" [assert-type]
tuples_type_compat.py:106: error: Expression is of type "tuple[int] | tuple[str, str] | tuple[int, *tuple[str, ...], int]", not "tuple[str, str] | tuple[int, int]" [assert-type]
tuples_type_compat.py:111: error: Expression is of type "tuple[int] | tuple[str, str] | tuple[int, *tuple[str, ...], int]", not "tuple[int, str, int]" [assert-type]
tuples_type_compat.py:102: error: Expression is of type "tuple[int]", not "tuple[int] | tuple[str, str] | tuple[int, *tuple[str, ...], int]" [assert-type]
tuples_type_compat.py:107: error: Expression is of type "tuple[str, str] | tuple[int, int]", not "tuple[int] | tuple[str, str] | tuple[int, *tuple[str, ...], int]" [assert-type]
tuples_type_compat.py:127: error: Expression is of type "tuple[int | str, str]", not "tuple[int | str, int | str]" [assert-type]
tuples_type_compat.py:130: error: Expression is of type "tuple[int | str, int]", not "tuple[int | str, int | str]" [assert-type]
tuples_type_compat.py:149: error: Expression is of type "Sequence[object]", not "Sequence[complex | list[int]]" [assert-type]
Expand All @@ -32,6 +31,7 @@ tuples_type_compat.py:188: error: Incompatible types in assignment (expression h
"""
conformance_automated = "Fail"
errors_diff = """
Lines 111, 112: Expected error (tag 'func6_3')
Copy link
Copy Markdown
Member

@AlexWaygood AlexWaygood Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this also be mentioned in the notes field?

Copy link
Copy Markdown
Collaborator Author

@rchen152 rchen152 Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For neither assert_type to error, I think mypy must be marking this match case as unreachable. Amusingly enough, this is the third time this has happened - I previously filed bugs against zuban and pyright for this.

Line 149: Unexpected errors ['tuples_type_compat.py:149: error: Expression is of type "Sequence[object]", not "Sequence[complex | list[int]]" [assert-type]']
Line 152: Unexpected errors ['tuples_type_compat.py:152: error: Expression is of type "Sequence[object]", not "Sequence[int | str]" [assert-type]']
"""
2 changes: 0 additions & 2 deletions conformance/results/mypy/typeddicts_extra_items.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -99,8 +99,6 @@ typeddicts_extra_items.py:124: error: Unexpected keyword argument "extra_items"
typeddicts_extra_items.py:128: error: Key "name" of TypedDict "MovieEI" cannot be deleted [misc]
typeddicts_extra_items.py:129: error: TypedDict "MovieEI" has no key "year" [typeddict-item]
typeddicts_extra_items.py:137: error: Unexpected keyword argument "extra_items" for "__init_subclass__" of "TypedDict" [call-arg]
typeddicts_extra_items.py:140: note: "unpack_no_extra" defined here
typeddicts_extra_items.py:141: note: "unpack_extra" defined here
typeddicts_extra_items.py:143: error: Unexpected keyword argument "year" for "unpack_no_extra" [call-arg]
typeddicts_extra_items.py:144: error: Unexpected keyword argument "year" for "unpack_extra" [call-arg]
typeddicts_extra_items.py:149: error: Unexpected keyword argument "extra_items" for "__init_subclass__" of "TypedDict" [call-arg]
Expand Down
2 changes: 1 addition & 1 deletion conformance/results/mypy/version.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1 @@
version = "mypy 1.19.1"
version = "mypy 1.20.0"
4 changes: 2 additions & 2 deletions conformance/results/pyrefly/callables_annotation.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,10 @@ ERROR callables_annotation.py:29:8-9: Unexpected keyword argument `a` [unexpecte
ERROR callables_annotation.py:29:8-11: Expected 2 more positional arguments [bad-argument-count]
ERROR callables_annotation.py:29:13-14: Unexpected keyword argument `b` [unexpected-keyword]
ERROR callables_annotation.py:35:8-9: Expected 0 positional arguments, got 1 [bad-argument-count]
ERROR callables_annotation.py:55:5-18: `Callable` requires exactly two arguments but 1 was found [bad-specialization]
ERROR callables_annotation.py:55:5-18: Expected 2 arguments for `Callable`, got 1 [bad-specialization]
ERROR callables_annotation.py:56:14-17: Callable types can only have `ParamSpec` in this position, got `int` [bad-specialization]
ERROR callables_annotation.py:57:18-23: Expected a type form, got instance of `list[type[int]]` [not-a-type]
ERROR callables_annotation.py:58:5-28: `Callable` requires exactly two arguments but 3 was found [bad-specialization]
ERROR callables_annotation.py:58:5-28: Expected 2 arguments for `Callable`, got 3 [bad-specialization]
ERROR callables_annotation.py:59:15-18: Invalid position for `...` [invalid-argument]
ERROR callables_annotation.py:91:7-15: `() -> str` is not assignable to variable `cb3` with type `(int, ...) -> str` [bad-assignment]
ERROR callables_annotation.py:93:7-15: `(*, a: int) -> str` is not assignable to variable `cb3` with type `(int, ...) -> str` [bad-assignment]
Expand Down
2 changes: 0 additions & 2 deletions conformance/results/pyrefly/directives_no_type_check.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,6 @@ errors_diff = """
"""
output = """
ERROR directives_no_type_check.py:15:14-16: `Literal['']` is not assignable to `int` [bad-assignment]
ERROR directives_no_type_check.py:29:7-17: Argument `Literal[b'invalid']` is not assignable to parameter `a` with type `int` in function `func1` [bad-argument-type]
ERROR directives_no_type_check.py:29:19-31: Argument `Literal[b'arguments']` is not assignable to parameter `b` with type `str` in function `func1` [bad-argument-type]
ERROR directives_no_type_check.py:32:6-8: Missing argument `a` in function `func1` [missing-argument]
ERROR directives_no_type_check.py:32:6-8: Missing argument `b` in function `func1` [missing-argument]
"""
5 changes: 1 addition & 4 deletions conformance/results/pyrefly/enums_member_values.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,9 @@ conformance_automated = "Pass"
errors_diff = """
"""
output = """
ERROR enums_member_values.py:21:12-43: assert_type(int, Literal[1]) failed [assert-type]
ERROR enums_member_values.py:22:12-41: assert_type(int, Literal[1]) failed [assert-type]
ERROR enums_member_values.py:26:16-50: assert_type(int, Literal[1, 3]) failed [assert-type]
ERROR enums_member_values.py:30:16-51: assert_type(int, Literal[1, 2, 3]) failed [assert-type]
ERROR enums_member_values.py:54:12-46: assert_type(Any, Literal[1]) failed [assert-type]
ERROR enums_member_values.py:68:12-42: assert_type(int, Literal[1]) failed [assert-type]
ERROR enums_member_values.py:78:5-10: Enum member `GREEN` has type `str`, must match the `_value_` attribute annotation of `int` [bad-assignment]
ERROR enums_member_values.py:78:5-10: Enum member `GREEN` has type `Literal['green']`, must match the `_value_` attribute annotation of `int` [bad-assignment]
ERROR enums_member_values.py:85:24-29: `int` is not assignable to attribute `_value_` with type `str` [bad-assignment]
"""
Loading