Add project specific configuration via local or remote pyproject.toml (#30)

* Add rudimentary format configuration via pyproject.toml

The idea is to have three sources of configuration, in order of
priority:
- Local TOML file specified with --config option
- Remote pyproject.toml at the last given revision
- Default configuration in _config.py

* Move default configuration into TOML file

* Add missing newline to intro_template

* Add documentation to configuration options

* Use scientific-python/changelist as example

* Document configuration options in README

and tweak or extend other sections accordingly.

* List features in README

and also hint that changelist is not intended to replace the human part
in documenting a release.

Author: lagru

MANIFEST.in

@@ -0,0 +1 @@
+include src/changelist/*.toml

README.md

@@ -1,8 +1,20 @@
 # changelist
 
-Prepare an automatic changelog from GitHub pull requests. For example, see
+Changelist helps you write better release notes by automating as much of the
+process as possible. For example, see
 https://github.com/scientific-python/changelist/blob/main/CHANGELOG.md.
 
+**Features**
+
+- Compile a list of pull requests, code authors and reviewers between any two
+  valid Git objects (refs).
+- Categorize pull requests into sections based on GitHub labels.
+- Point it at any repository on GitHub. No need to clone or checkout a
+  repository locally, a Python package is all that's needed.
+
+We recommend to treat the generated document as a first draft to build
+on and not as an already perfect documentation of the release.
+
 _This project is currently in its alpha stage and might be incomplete or change a lot!_
 
 ## Installation
@@ -11,26 +23,101 @@ _This project is currently in its alpha stage and might be incomplete or change
 pip install changelist
 ```
 
-## Set up your repository
+## Usage
+
+The script requires a [GitHub personal access
+token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens).
+The token does not need any permissions, since it is used only to
+increase query limits.
+
+```sh
+export GH_TOKEN='...'
+changelist scientific-python/changelist v0.2.0 main
+```
 
-To categorize merged PRs in the changelist, each PR
-must have have one of the following labels:
+This will list all pull requests, authors and reviewers that touched commits
+between `v0.2.0` and `main` (excluding `v0.2.0`).
+Pull requests are sorted into section according to the configuration in
+`tool.changelist.label_section_map`.
+
+## Configuration
+
+changelist can be configured from two sources, in order of precedence:
+
+- A local TOML file specified with the `--config` option
+- A remote `pyproject.toml` at `stop_rev`
+
+If a configuration option is not specified in either file above, changelist
+falls back to the following configuration:
+
+````toml
+# Default changelist configuration as supported in pyproject.toml
+[tool.changelist]
+
+# A template string that is included as the title of the generated notes.
+# "{repo_name}" and "{version}", if given, are replaced by the respective
+# values given in the command line.
+title_template = "{repo_name} {version}"
+
+# A template string that is included as introductory text after the title.
+# "{repo_name}" and "{version}", if given, are replaced by the respective
+# values given in the command line.
+intro_template = """
+We're happy to announce the release of {repo_name} {version}!
+"""
+
+# A template string that is included at the end of the generated notes.
+# "{repo_name}" and "{version}", if given, are replaced by the respective
+# values given in the command line.
+outro_template = """
+_These lists are automatically generated, and may not be complete or may contain
+duplicates._
+"""
+
+# Profiles that are excluded from the contributor list.
+ignored_user_logins = [
+    "web-flow",
+]
+
+# If this regex matches a pull requests description, the captured content
+# is included instead of the pull request title.
+# E.g. the default regex below is matched by
+#
+# ```release-note
+# An ideally expressive description of the change that is included as a single
+# bullet point. Newlines are removed.
+# ```
+#
+# If you modify this regex, make sure to match the content with a capture
+# group named "summary".
+pr_summary_regex = "^```release-note\\s*(?P<summary>[\\s\\S]*?\\w[\\s\\S]*?)\\s*^```"
+
+# If any of a pull request's labels matches one of the regexes on the left side
+# its summary will appear in the appropriate section with the title given on
+# the right side. If a pull request doesn't match one of these categories it is
+# sorted into a section titled "Other". Pull request can appear in multiple
+# sections as long as their labels match.
+[tool.changelist.label_section_map]
+".*Highlight.*" = "Highlights"
+".*New feature.*" = "New Features"
+".*API.*" = "API Changes"
+".*Enhancement.*" = "Enhancements"
+".*Performance.*" = "Performance"
+".*Bug fix.*" = "Bug Fixes"
+".*Documentation.*" = "Documentation"
+".*Infrastructure.*" = "Infrastructure"
+".*Maintenance.*" = "Maintenance"
+````
 
-- `type: Highlights`
-- `type: New features`
-- `type: Enhancements`
-- `type: Performance`
-- `type: Bug fix`
-- `type: API`
-- `type: Maintenance`
-- `type: Documentation`
-- `type: Infrastructure`
+## Set up your repository
 
-This list will soon be configurable.
+To categorize merged PRs in the changelist with the default configuration, each
+PR must have a label that matches one of the regexes on the left side of the
+`label_section_map` table, e.g. `type: Highlights`.
 
 ### Label checking
 
-To ensure that each PR has an associated `type: ` label,
+You may want to ensure that each PR has an associated `type: ` label,
 we recommend adding an action that fails CI if the label is missing.
 
 To do so, place the following in `.github/workflows/label-check.yaml`:
@@ -89,15 +176,3 @@ jobs:
 ```
 
 See https://github.com/scientific-python/attach-next-milestone-action for more information.
-
-## Usage
-
-```sh
-export GH_TOKEN='...'
-changelist scikit-image/scikit-image v0.21.0 main
-```
-
-The script requires a [GitHub personal access
-token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens).
-The token does not need any permissions, since it is used only to
-increase query limits.

src/changelist/_cli.py

@@ -1,6 +1,7 @@
 import argparse
 import logging
 import os
+import re
 import sys
 import tempfile
 from pathlib import Path
@@ -10,6 +11,7 @@
 from github import Github
 from tqdm import tqdm
 
+from ._config import add_config_defaults, local_config, remote_config
 from ._format import MdFormatter, RstFormatter
 from ._query import commits_between, contributors, pull_requests_from_commits
 
@@ -44,7 +46,7 @@ def parse_command_line(func: Callable) -> Callable:
     parser.add_argument(
         "org_repo",
         help="Org and repo name of a repository on GitHub (delimited by a slash), "
-        "e.g. 'numpy/numpy'",
+        "e.g. 'scientific-python/changelist'",
     )
     parser.add_argument(
         "start_rev",
@@ -57,7 +59,7 @@ def parse_command_line(func: Callable) -> Callable:
     )
     parser.add_argument(
         "--version",
-        default="0.0.0",
+        default="x.y.z",
         help="Version you're about to release, used title and description of the notes",
     )
     parser.add_argument("--out", help="Write to file, prints to STDOUT otherwise")
@@ -72,6 +74,12 @@ def parse_command_line(func: Callable) -> Callable:
         action="store_true",
         help="Clear cached requests to GitHub's API before running",
     )
+    parser.add_argument(
+        "--config",
+        dest="config_path",
+        help="Path to local TOML configuration (falls back on remote "
+        "pyproject.toml or default config if not given)",
+    )
     parser.add_argument(
         "-v",
         "--verbose",
@@ -98,6 +106,7 @@ def main(
     out: str,
     format: str,
     clear_cache: bool,
+    config_path: str,
     verbose: int,
 ):
     """Main function of the script.
@@ -125,6 +134,12 @@ def main(
         )
     gh = Github(gh_token)
 
+    if config_path is None:
+        config = remote_config(gh, org_repo, rev=stop_rev)
+    else:
+        config = local_config(Path(config_path))
+    config = add_config_defaults(config)
+
     print("Fetching commits...", file=sys.stderr)
     commits = commits_between(gh, org_repo, start_rev, stop_rev)
     pull_requests = pull_requests_from_commits(
@@ -144,6 +159,12 @@ def main(
         authors=authors,
         reviewers=reviewers,
         version=version,
+        title_template=config["title_template"],
+        intro_template=config["intro_template"],
+        outro_template=config["outro_template"],
+        label_section_map=config["label_section_map"],
+        pr_summary_regex=re.compile(config["pr_summary_regex"], flags=re.MULTILINE),
+        ignored_user_logins=config["ignored_user_logins"],
     )
 
     if out:

src/changelist/_config.py

@@ -0,0 +1,49 @@
+import logging
+from pathlib import Path
+
+from github import Github, UnknownObjectException
+
+try:
+    import tomllib
+except ModuleNotFoundError:
+    import tomli as tomllib
+
+logger = logging.getLogger(__name__)
+
+here = Path(__file__)
+
+
+DEFAULT_CONFIG_PATH = here.parent / "default_config.toml"
+
+
+def remote_config(gh: Github, org_repo: str, *, rev: str):
+    repo = gh.get_repo(org_repo)
+    try:
+        file = repo.get_contents("pyproject.toml", ref=rev)
+        logger.debug("found pyproject.toml in %s@%s", org_repo, rev)
+        content = file.decoded_content.decode()
+    except UnknownObjectException:
+        content = ""
+    config = tomllib.loads(content)
+    config = config.get("tool", {}).get("changelist", {})
+    return config
+
+
+def local_config(path: Path) -> dict:
+    with path.open("rb") as fp:
+        config = tomllib.load(fp)
+    config = config.get("tool", {}).get("changelist", {})
+    return config
+
+
+def add_config_defaults(
+    config: dict, *, default_config_path: Path = DEFAULT_CONFIG_PATH
+) -> dict:
+    with default_config_path.open("rb") as fp:
+        defaults = tomllib.load(fp)
+    defaults = defaults["tool"]["changelist"]
+    for key, value in defaults.items():
+        if key not in config:
+            config[key] = value
+            logger.debug("using default config value for %s", key)
+    return config

src/changelist/_format.py

@@ -20,31 +20,15 @@ class MdFormatter:
     authors: set[Union[NamedUser]]
     reviewers: set[NamedUser]
 
-    version: str = "x.y.z"
-    title_template: str = "{repo_name} {version}"
-    intro_template: str = """
-We're happy to announce the release of {repo_name} {version}!
-"""
-    outro_template: str = (
-        "_These lists are automatically generated, and may not be complete or may "
-        "contain duplicates._\n"
-    )
+    version: str
+    title_template: str
+    intro_template: str
+    outro_template: str
+
     # Associate regexes matching PR labels to a section titles in the release notes
-    regex_section_map: tuple[tuple[str, str], ...] = (
-        (".*Highlight.*", "Highlights"),
-        (".*New feature.*", "New Features"),
-        (".*Enhancement.*", "Enhancements"),
-        (".*Performance.*", "Performance"),
-        (".*Bug fix.*", "Bug Fixes"),
-        (".*API.*", "API Changes"),
-        (".*Maintenance.*", "Maintenance"),
-        (".*Documentation.*", "Documentation"),
-        (".*Infrastructure.*", "Infrastructure"),
-    )
-    ignored_user_logins: tuple[str] = ("web-flow",)
-    pr_summary_regex = re.compile(
-        r"^```release-note\s*(?P<summary>[\s\S]*?\w[\s\S]*?)\s*^```", flags=re.MULTILINE
-    )
+    label_section_map: dict[str, str]
+    pr_summary_regex: re.Pattern
+    ignored_user_logins: tuple[str]
 
     def __str__(self) -> str:
         """Return complete release notes document as a string."""
@@ -80,10 +64,10 @@ def _prs_by_section(self) -> OrderedDict[str, set[PullRequest]]:
         """
         label_section_map = {
             re.compile(pattern): section_name
-            for pattern, section_name in self.regex_section_map
+            for pattern, section_name in self.label_section_map.items()
         }
         prs_by_section = OrderedDict()
-        for _, section_name in self.regex_section_map:
+        for _, section_name in self.label_section_map.items():
             prs_by_section[section_name] = set()
         prs_by_section["Other"] = set()
 
@@ -182,7 +166,9 @@ def _format_intro(self):
         yield from (f"{line}\n" for line in intro.split("\n"))
 
     def _format_outro(self) -> Iterable[str]:
-        outro = self.outro_template
+        outro = self.outro_template.format(
+            repo_name=self.repo_name, version=self.version
+        )
         # Make sure to return exactly one line at a time
         yield from (f"{line}\n" for line in outro.split("\n"))