Add basic typechecking by MarcellPerger1 · Pull Request #72 · MarcellPerger1/programming_lang_2

MarcellPerger1 · 2025-06-15T21:03:03Z

Added basic typechecking
Resolve TODOs in code
Split into several modules
Add a few unittests
Add e2e test using example 2 (will require some modifications though)
Add fuzzing

For now the typechecking is very basic and proof-of-concept. We may need to create a typed AST.

…cific one

…nstead of mro() (Improvement of ~15% in name resolution)

…better anyway

…ng type info to nodes (So I don't have to create yet another tree of node classes and another set of converters)

TODO: AST types, return AST rather than True/False

`__notes__` isn't present any more

MarcellPerger1 · 2026-04-23T23:50:27Z

Ok, so need to fix the snapshot-fmt to be same as before for the untyped AST.

… or all versions)

TODO: should stop supporting 3.10 soon, by November at the latest

(Note: this and last commit were TDD'd together - well the TDD was used for bug fixes)

…error.py)

…ems to be quite effective

MarcellPerger1 · 2026-07-14T14:35:16Z

Gah, I can't wait for 3.10 to go end-of-life so I can do typing properly (well as properly as you can in the current state of the type system)

(More 3.10 weirdness...)

How on Earth did fuzzing not catch this?

FAILING(typecheck(errors)): on error region when multiple

…ehaviour not crashing

MarcellPerger1 · 2026-07-14T16:51:26Z

Look at that diff! Well it's only +2000 -800 coz snapshots but still!

Because Pycharm now somewhat supports `TypeVar` defaults (I mean that type-checking doesn't completely die because of the presence of `default`). Also, added `typing_extensions` up to 3.12 for `TypeVar` with default to requirements-dev.txt (dev because only needed for type checking)

MarcellPerger1

Lots of comments.

MarcellPerger1 · 2026-07-15T18:23:49Z

+        return result
+
+    @classmethod
+    def _get_funcs(cls, mapping: dict[type[WT] | type, list[VT]], tp: type[WT]) -> list[VT]:


type[WT] | type - why can't this simply be type[WT]? Do we have to replace type[WT] with WTT here? Hopefully not, hopefully type[WT] will just work but Pycharm bugs.

MarcellPerger1 · 2026-07-15T18:24:19Z

+
+class FilteredWalker(BasicFilteredWalker[WT], Generic[WT]):
+    def __init__(self):
+        # Can then add studd to specific instances


Suggested change

# Can then add studd to specific instances

# Can then add stuff to specific instances

MarcellPerger1 · 2026-07-15T18:26:37Z

+        The restrictions on name are because we have no other way of detecting
+         it (without metaclass dark magic) as we can't refer to the class while
+         its namespace is being evaluated


We could just do __init_subclass__ with some slight dark magic though... like iterating over the newly defined attributes and seeing which one is of our new fancy custom type that can only be created via the decorator? Or like setting an attribute on the functions a la @abstractmethod

MarcellPerger1 · 2026-07-15T18:27:29Z

 class HasRegion:
    region: StrRegion


Should this not be a Protocol? PR does Pycharm not support that properly?

MarcellPerger1 · 2026-07-17T10:35:03Z


 from parser.cst.cstgen import CstGen
 from parser.lexer.tokenizer import Tokenizer
+from scripts.fuzz import fuzz


Are we sure we want to have tests importing scripts while some scripts use tests?

MarcellPerger1 · 2026-07-17T10:36:40Z

+        decl: AstDeclNode[TypeMetadata]
+        self.assertTypecheckedTo(decl.value, ValType())
+        self.assertTypecheckedTo(decl.ident, ValType())
+        # self.assertMatchesSnapshot(prog)


Should we un-comment these?

I'm not sure there's much point because all we're changing in the typechecker is the types, so snapshots would only test the astgen as extra.

MarcellPerger1 · 2026-07-17T10:39:37Z

 from parser.common.error import BaseLocatedError


+class _PatchRegionDisplay:


A comment on why we need this would be good. Also, I recall there's some BoundMock in some other file. Would that work here?

MarcellPerger1 · 2026-07-17T10:40:46Z

+    def __init__(self):
+        def _mocked_display(*args, **kwargs):
+            return self._mock(*args, **kwargs)
+        self._mock = MagicMock()


A comment about what this mock is etc would be good (this bit is quite non-obvious)

MarcellPerger1 · 2026-07-17T10:48:22Z

+# (Note: dataclasses._MISSING_TYPE isn't actually a runtime thing, it's just
+# for type checkers to recognise dataclasses.MISSING)


Comment in wrong place, should be 3 lines up.

MarcellPerger1 added 6 commits June 14, 2025 18:24

refactor(ast): Move ast node subclasses into ast_nodes.py

f919750

refactor(ast): Move FilteredWalker into own module

4d499d8

feat(ast-walker): Don't call more general walkers if skipped by a spe…

1af4021

…cific one

perf(filtered-walker): Replace generator with listcomp, use __mro__ i…

664fe3e

…nstead of mro() (Improvement of ~15% in name resolution)

feat(typecheck): Start typechecking (add statements this commit)

fff6268

feat(typecheck): Finish typechecker (add expressions)

5e37ec0

MarcellPerger1 added T-feature A new feature A-TypeChecking Area: Type checking/name resolution labels Jun 15, 2025

MarcellPerger1 added 9 commits April 11, 2026 16:01

feat(typecheck): Better type formatting in error messages

878bee3

refactor(cst): Remove unneeded cast

22a8a5b

refactor(ast): Replace code with Pycharm-friendly version

fa6dee3

refactor(cst): Improve type annotations a bit

188fa54

refactor(ast): Improve type alias placement?? Well, Pycharm likes it …

4263415

…better anyway

refactor(ast): Generic-based AstNode metadata in preparation for addi…

4a95006

…ng type info to nodes (So I don't have to create yet another tree of node classes and another set of converters)

feat(typecheck): Add types to AST in a rather clean way

32db947

TODO: AST types, return AST rather than True/False

refactor(types): Fix outdated type annotation

b545f05

`__notes__` isn't present any more

feat(typecheck+types): Add proper types for typed AST

b26f9a5

MarcellPerger1 added 12 commits April 24, 2026 19:20

refactor(cst): Remove no-longer-needed cast

6f2092b

ci: Don't fail-fast (we need to know if it fails on just old versions…

af0257c

… or all versions)

ci: Add Python 3.13 and 3.14 to testing matrix

4505ce0

TODO: should stop supporting 3.10 soon, by November at the latest

feat: Fix tree printing, massive refactor

5f20c19

test: Tests for tree printing

d51d535

(Note: this and last commit were TDD'd together - well the TDD was used for bug fixes)

chore: Add note about broken recursive_eq function

cbb23c7

refactor: Split monolithic typecheck.py into multiple modules

5845f28

chore: Update fuzzing corpus

90341f6

feat(tree-print): Make removing defaults more general

aaa8b34

chore: Update fuzzing corpus

a9562ef

refactor: Move region display to str_region.py (rather than being in …

294b5e7

…error.py)

test: Fix mocks that tried to replace old method

e026641

MarcellPerger1 added 7 commits July 14, 2026 14:11

chore(test): Update snapshot for main_example_3 test

ddae580

ci: Better fuzzing

a800877

chore(fuzz): Do a light 100K fuzzing run with the new fuzzer which se…

f9882de

…ems to be quite effective

chore(fuzz): Do a 1M fuzzing run

2a979db

fix(types): Add __future__ annotations to allow use of TypeIs

9f3f24b

fix(test): Update fuzz-based tests to work with new fuzzer

af24f14

fix(test): Fix import error being caught as type error

21c066f

MarcellPerger1 added 11 commits July 14, 2026 15:36

fix(ci): Include requirements.txt in requirements-dev.txt

f4e0236

fix(fuzz): Add byteorder (because 3.10 requires it explicitly)

211609a

(More 3.10 weirdness...)

test(typecheck): Add test for aug-assign [FAILING]

7ae52e4

How on Earth did fuzzing not catch this?

fix(typecheck): Fix aug-assign

b2a7d72

fix(typecheck): Nore robust assignment type-checking

c3679ab

test(typecheck): Add some tests [FAILING]

cc00f64

FAILING(typecheck(errors)): on error region when multiple

fix(typecheck): Improve error region for many extraneous call args

af4b6d6

test(typecheck): More error-tests

56f3112

test(typecheck): Test that attributes don't cause violent death

a66bf2d

test(typecheck): Add more tests for calling, getitem, unimplemented b…

74617b4

…ehaviour not crashing

fix(typecheck+types): Fix Pycharm type warning

a880f51

MarcellPerger1 marked this pull request as ready for review July 14, 2026 16:47

MarcellPerger1 added 8 commits July 15, 2026 12:35

refactor(ast+types): Try improve types a bit but Pycharm bug

cf3b78b

test(ast): Add tests for a bug in AstGen [FAILING]

1347a4d

test(ast): Fix test [FAILING]

b0dc66b

fix(ast): Fix major bug in else if handling

8a9f886

test(ast): Update snapshots

9b35a47

fix(types): Small type improvements

cd64f73

chore: Remove some outdated TODOs

a783c33

MarcellPerger1 commented Jul 17, 2026

View reviewed changes

	# Can then add studd to specific instances
	# Can then add stuff to specific instances

		from parser.common.error import BaseLocatedError


		class _PatchRegionDisplay:

		# (Note: dataclasses._MISSING_TYPE isn't actually a runtime thing, it's just
		# for type checkers to recognise dataclasses.MISSING)

Conversation

MarcellPerger1 commented Jun 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MarcellPerger1 commented Apr 23, 2026

Uh oh!

MarcellPerger1 commented Jul 14, 2026

Uh oh!

MarcellPerger1 commented Jul 14, 2026

Uh oh!

MarcellPerger1 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

MarcellPerger1 commented Jun 15, 2025 •

edited

Loading