Skip to content

Commit d152fec

Browse files
Remove Basilisk from conformance results
Remove Basilisk from conformance results Reverts #2316, at the request of Basilisk's author. Some of Basilisk's rules match raw source text and hard-coded typing symbol names rather than resolved symbols on the AST, so its published score reflects the spelling of the test files rather than conformance to the specification. Semantics-preserving changes to the suite — renaming imported typing symbols, adjusting whitespace — collapse the result. Removes the basilisk result files, the BasiliskTypeChecker adapter, the dependency entries, and the README reference, and regenerates results.html from the remaining checkers. No other type checker's results are affected.
1 parent a490662 commit d152fec

147 files changed

Lines changed: 24 additions & 2302 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎conformance/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,7 @@ Note that some type checkers may not run on some platforms. If a type checker fa
8181

8282
Different type checkers report errors in different ways (with different wording in error messages and different line numbers or character ranges for errors). This variation makes it difficult to fully automate test validation given that tests will want to check for both false positive and false negative type errors. Some level of manual inspection will therefore be needed to determine whether a type checker is fully conformant with all tests in any given test file. This "scoring" process is required only when the output of a test changes — e.g. when a new version of that type checker is released and the tests are rerun. We assume that the output of a type checker will be the same from one run to the next unless/until a new version is released that fixes or introduces a bug. In this case, the output will need to be manually inspected and the conformance results re-scored for those tests whose output has changed.
8383

84-
[Conformance results](https://htmlpreview.github.io/?https://github.com/python/typing/blob/main/conformance/results/results.html) are reported and summarized for each supported type checker. Currently, results are reported for basilisk, mypy, pyrefly, pyright, zuban, ty, and pycroscope. It is the goal and desire to add additional type checkers over time.
84+
[Conformance results](https://htmlpreview.github.io/?https://github.com/python/typing/blob/main/conformance/results/results.html) are reported and summarized for each supported type checker. Currently, results are reported for mypy, pyrefly, pyright, zuban, ty, and pycroscope. It is the goal and desire to add additional type checkers over time.
8585

8686
## Adding a New Test Case
8787

‎conformance/pyproject.toml‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,6 @@ name = "typing-conformance"
33
version = "0.1.0"
44
requires-python = "==3.12.*"
55
dependencies = [
6-
"basilisk-python",
76
"jinja2",
87
"markdown",
98
"mypy",

‎conformance/results/basilisk/aliases_explicit.toml‎

Lines changed: 0 additions & 27 deletions
This file was deleted.

‎conformance/results/basilisk/aliases_implicit.toml‎

Lines changed: 0 additions & 35 deletions
This file was deleted.

‎conformance/results/basilisk/aliases_newtype.toml‎

Lines changed: 0 additions & 21 deletions
This file was deleted.

‎conformance/results/basilisk/aliases_recursive.toml‎

Lines changed: 0 additions & 18 deletions
This file was deleted.

‎conformance/results/basilisk/aliases_type_statement.toml‎

Lines changed: 0 additions & 32 deletions
This file was deleted.

‎conformance/results/basilisk/aliases_typealiastype.toml‎

Lines changed: 0 additions & 29 deletions
This file was deleted.

‎conformance/results/basilisk/aliases_variance.toml‎

Lines changed: 0 additions & 10 deletions
This file was deleted.

‎conformance/results/basilisk/annotations_coroutines.toml‎

Lines changed: 0 additions & 6 deletions
This file was deleted.

0 commit comments

Comments
 (0)