Repository navigation
187 structure - #318
Draft
pjljvandelaar wants to merge 45 commits into
Draft
187 structure#318pjljvandelaar wants to merge 45 commits into
pjljvandelaar wants to merge 45 commits into
Conversation
The dependency rules of the repository structure now live only in [tool.importlinter] of pyproject.toml, including the order of the core sub-packages and the independence of the parser bindings. Unit tests follow the rules of the unit they test; test helpers are imported through the test package so import-linter sees them. Known violations are ignore_imports entries with their reason.
- Accept repository-structure.md, mark the code repository decision as superseded, and capture: extractors are recipes, parser-specific recipes declare their parser bindings, the public API rules, the test layout (test/unit, test/features, test/tools) and that only the developer docs describe the tools. - Add tools/check_layout.py, configured in [tool.check_layout], which checks that the tests and the docs mirror the code; run in CI. - Make tools a package, run every tool as python -m tools.<name>, and let the tests import the tools. - Add Import Linter contracts that keep the tools and their tests independent of the rest of the repository. - Move the ruff/pyright runner shared by lint_budget and ruff_stats into tools/shared/linters.py. - Remove the unanchored lib/, lib64/, var/, parts/, eggs/, downloads/ and environment entries from .gitignore; keep /venv/. - Lower the lint budget to the new counts.
save_budget wrote the platform line ending, so on Windows the budget file got CRLF, which .gitattributes does not allow. It now always writes LF; a test checks the written bytes.
Steps with substeps src, test, docs and remaining, a gate per step, and the placement of the test helpers. Part of issue 187.
Signed-off-by: pjljvandelaar <pierre.vandelaar@tno.nl>
…s.py It ran only as part of pytest; it now has its own CI step, test and docs page. Resolves the open decision in the migration plan. Part of issue 187.
Main removed TautToPythonUnittest, SimplifyRenaissance and taut2test (#299); Import Linter fails on ignore_imports entries that no longer match. Part of issue 187.
The docs workflow runs codespell without the ignore list, so the plan's gate, which quoted it, failed. Both workflows now read [tool.codespell]. Part of issue 187.
… is removed Part of issue 187.
pjljvandelaar
marked this pull request as draft
October 29, 2026 15:57
tools/check_references.py reads the docs, the comments and docstrings, and the feature files. It skips Markdown link targets (checked by mkdocs) and examples (paths with the word 'example'). [tool.check_references] excludes the decision records and the migration page, lists the type parameter files as known issues and the paths that do not exist yet as stale. list_files and with_directories move to tools/shared/repository_files.py.
Conflicts: lint-budget.json takes the lower budgets; the test/search_strategies tests keep importing through the test package; the test/lst, test/search_strategies and test/utils docstrings describe what those packages hold. From main: ast_utils now imports the rewriter's Rewritable, and test/utils/test_ast_utils.py parses C code; both are recorded as known violations of the layers contract. test/tree_sitter/ no longer exists, so check-layering.md no longer mentions test directories without an __init__.py.
… the rewriter in step 1.4
Every importer of the 21 re-exported names imports from the defining module, which removes the late import that worked around the circular import with integrations.clang. The re-exports hid that test.utils_for_tests imports PatternMatch, so it moves to the core.matching layer.
Collaborator
Author
|
Problem with this Pull Request. |
src/renaissance/core/ exports nothing from its root; test/unit/ and test/unit/core/ are packages. Four mirrors of the core (tests, module pages, API pages, user pages) are in [tool.check_layout], with no units yet. test/unit/ exists now, so it leaves the stale list of [tool.check_references], and the check-references page uses another not-yet-existing path as its example.
semantic_kind.py and node_protocol.py move to src/renaissance/core/model/, whose __init__ exports NodeProtocol and SemanticKind; every importer imports from renaissance.core.model. test_node_protocol.py moves to test/unit/core/model/; its import of match_finder remains a known violation. The protected contract core-model-api allows only core.model and its unit tests to import its modules. The mirror pages of core.model are in the developer modules, the API reference and the user documentation; docs/user/core/ leaves the stale list.
preceding_sibling and next_sibling go to the public API of core.model in step 1.4, not to core.model.for_bindings.
TextSegment, SyntaxNode and Siblings move from syntax_tree to core.model and join its public API. Their tests move to test/unit/core/model/, and the test doubles infra_text_segment.py and infra_syntax_node.py, which only these tests use, to test/unit/core/model/shared/, an extra of the test mirror.
# Conflicts: # src/renaissance/syntax_tree/__init__.py # src/renaissance/syntax_tree/ast_finder.py # src/renaissance/syntax_tree/ast_refactor_actions.py # src/renaissance/syntax_tree/ast_rewriter.py # src/renaissance/syntax_tree/match_finder.py # test/c_cpp/test_ast_factory.py # test/c_cpp/test_ast_finder.py # test/c_cpp/test_ast_references.py
TextUtils moves from utils/text_utils.py to core.model and joins its public API. fix_indent, which has no users, moves with it but stays internal. signature_to_id leaves the core for its only user, integrations/tree_sitter/visualizer.py, and snake_case for its only user, recipes/python_refactoring.py. Test decision: test/utils/test_text_utils.py tests only snake_case, so its test class TestSnakeCase moves with snake_case into test/recipes/test_python_refactoring.py. In test/unit/core/model/ it would import a recipe and break the layers contract.
Resolve the conflicts with the structure of this branch: imports from renaissance.core.model and the defining modules, no re-exports of renaissance.syntax_tree, and uv.lock taken from main and re-locked for import-linter. Record main's new known violations in the import contracts, and remove the entries of the deleted examples and test_removed_legacy_api. Update the plan and the repository structure for the deleted examples, DEFAULT_INDENT and fix_indent.
run_ruff_import_cleanup removes the imports that a recipe left unused. Step 1.6 moves utils/ruff_utils.py to the recipes, before step 2.1 checks that src/renaissance/utils/ no longer exists; phase 7 makes it a package like the other recipes.
A binding provides offsets into the UTF-8 encoded code, so turning a line and column into an offset is internal to the binding. Remove the stale violation of cst_node importing convert, which main removed. Add a TODO that convert may be off by one per line with CRLF line endings.
traverse, preceding_sibling and next_sibling become public API of core.model, defined in core/model/node_utils.py with the internal process_node and display_context. match_props, match_children and format_node move to core/model/for_bindings.py. is_ancestor_of moves to the rewriter, its only user, and to_str to python_refactoring. The test of is_ancestor_of becomes test/syntax_tree/test_is_ancestor_of.py; its sibling tests move to test/unit/core/model/test_node_utils.py. The protected contract of core.model lists its modules, without for_bindings, which gets its own protected contract. That ast_node and ast_shower import internals of core.model is a known violation until step 1.5.
cst_utils analyses the names in a libcst tree, which is the reference analysis that a binding provides; a recipe uses it as a step. Step 6.4 moves it with cst_node to parser_bindings/python/libcst/.
The tests of module x are in test_x.py, or in test_x___<aspect>.py when there are several files. Module names therefore contain no triple underscore. The layout check does not enforce the rule yet.
Repository structure decides that file and directory names use a single _ throughout the repository.
The documentation pages, images and their directories, the feature files, the workflows,
IMPROVEMENT_PLAN.md and lint_budget.json are renamed, and every reference to them follows.
The image policy refers to the rule. A page now has the name of the unit it mirrors, so the
layout check loses its {kebab_path} placeholder.
…check The layout check accepts test_x___<aspect>.py as a counterpart of module x in a mirror with aspects = true. The test mirrors of tools/ and tools/shared/ allow aspects, and a new mirror maps the modules of core.model onto their test files; for_bindings, semantic_kind, siblings and text_utils are exempt, as they have no unit tests of their own yet. The new tools/check_names.py checks names only: words are separated by a single _, or by ___ in names that start with test_, and never by -.
The layout check reports a module that has both test_x.py and test_x___<aspect>.py, as repository_structure.md decides: one file when the tests fit in it, one file per aspect when they do not.
The layout check reports a module whose only test file is test_x___<aspect>.py: a single file is test_x.py.
The name check now also requires a name to be an optional leading . (a hidden file such as .gitignore), a stem of ASCII letters, digits and _, and an optional extension. This subsumes the earlier check for -. pyrightconfig.strict.json is renamed to pyrightconfig_strict.json.
core.navigate is the lowest core layer, below core.model. It has two modules: - protocols.py: HasParent, HasChildren and NodeNavigateProtocol. - navigate.py: is_ancestor_of, is_proper_ancestor_of, ancestors, proper_ancestors, nearest_ancestor, nearest_proper_ancestor, next_sibling and previous_sibling. These functions compare nodes by identity. NodeProtocol extends NodeNavigateProtocol, and Rewritable extends HasParent. Behaviour changes: - preceding_sibling is renamed to previous_sibling: the function, and the properties of ASTNode, PythonCstNode, PythonRstNode and the tree-sitter node. - ASTNode.is_ancestor_of, which compared by equality, and is_descendant_of are removed. - ASTNode.get_ancestor delegates to nearest_proper_ancestor. - The rewriter calls is_proper_ancestor_of and behaves as before. The protocols type parent and children with Self. PythonPattern and PythonRstNode now do too, and create their children with type(self) instead of their class name. Without subclasses, type(self) is the class, so their behaviour is unchanged. Tests: 2802 -> 2811 passed, 53 -> 52 xfailed. - The 9 tests of test_is_ancestor_of.py and test_node_utils.py are replaced by the 17 tests of test/unit/core/navigate/. - test_get_ancestor_finds_enclosing_function now passes.
node can also be an ast.AST, a PythonCstNode, an LSTNode, or a str.
Ruff TD003 requires a link; without it the lint budget check fails.
AbstractSet[str] (collections.abc.Set) states that the functions only read them, and accepts the set constants of the bindings; frozenset[str] would reject a set.
…(step 1.9) Parser bindings share helpers in parser_bindings/shared/: it imports nothing from Renaissance.Py, and only the parser bindings and its unit tests import it. - Step 1.9, done before 1.5: match_props and match_children move to integrations/shared/equal.py; core/model/for_bindings.py becomes format.py, because format_node reads DISPLAY_PARSER_KIND and ASTNode uses it. - The for_bindings modules of core.pattern and core.matching are dropped: the placeholder markers and the dollar notation are public, in core/pattern/placeholders.py (step 2.1); find_in_list is public and pattern_kind is internal to core.matching (step 3.1). - Step 6.2 moves integrations/shared/ to parser_bindings/shared/.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.