known_good.json maybe use "version" instead of "hash" (#471)

Author: a-zw

docs/internals/extensions/source_code_linker.md

@@ -12,6 +12,7 @@ The feature is split into **two layers**:
 
 1. **Bazel pre-processing (before Sphinx runs)**
    Generates and aggregates JSON caches containing the raw `source_code_link` findings (and, if available, repository metadata like `repo_name/hash/url`).
+   Note that "hash" might also be a "version" tag.
 
 2. **Sphinx extension (during the Sphinx build)**
    Reads the aggregated JSON, validates and adapts it, and finally injects the links into Sphinx needs in the final layout (**RepoSourceLink**).
@@ -36,6 +37,7 @@ one JSON cache per repository.
 It also adds metadata to each needlink that is needed in further steps.
 
 Example of requirement tags:
+
 ```python
 # Choose one or the other, both mentioned here to avoid detection
 # req-Id/req-traceability: <NEED_ID>
@@ -83,20 +85,23 @@ This step also fills in url & hash if there is a known_good_json provided (e.g.
 
 (repo-metadata-rules)=
 #### Repo metadata rules
+
 Here are some basic rules regarding the MetaData information
 
 In a combo build, a known_good_json **must** be provided.
 If known_good_json is found then the hash & url will be filled for each need by the script.
 
 Combo build:
-  - `repo_name`: repository name (parsed from filepath in step 1)
-  - `hash`: revision/commit hash (as provided by the known_good_json)
-  - `url`: repository remote URL (as provided by the known_good_json)
+
+- `repo_name`: repository name (parsed from filepath in step 1)
+- `hash`: revision/commit hash (as provided by the known_good_json)
+- `url`: repository remote URL (as provided by the known_good_json)
 
 Local build:
-  - `repo_name`: 'local_repo'
-  - `hash`: will be empty at this point, and later filled via parsing the git commands
-  - `url`: will be empty at this point, and later filled via parsing the git commands
+
+- `repo_name`: 'local_repo'
+- `hash`: will be empty at this point, and later filled via parsing the git commands
+- `url`: will be empty at this point, and later filled via parsing the git commands
 
 ---
 
@@ -140,6 +145,7 @@ def test_feature():
     ...
 
 ```
+
 > Note: If you use the decorator, it will check that you have specified a docstring inside the function.
 
 #### Data Flow
@@ -153,7 +159,7 @@ def test_feature():
      - Result (e.g. passed, failed, skipped)
      - Result text (if failed/skipped will check if message was attached to it)
      - Verifications (`PartiallyVerifies`, `FullyVerifies`)
-    - Also parses metadata according to the [metadata rules](#repo-metadata-rules)
+     - Also parses metadata according to the [metadata rules](#repo-metadata-rules)
 
    - Cases without metadata are logged out as info (not errors).
    - Test cases with metadata are converted into:
@@ -168,7 +174,9 @@ def test_feature():
    - Warns on missing need IDs.
 
 #### Example JSON Cache (DataFromTestCase)
+
 The DataFromTestCase depicts the information gathered about one testcase.
+
 ```json
 [
   {
@@ -246,6 +254,7 @@ Instead of repeating `repo_name/hash/url` for every single link entry, the final
 - All links belonging to that repository are stored beneath it
 
 This somewhat looks like this:
+
 ```{code-block} json
 [
   {
@@ -296,20 +305,22 @@ During the Sphinx build process, the extension applies the computed links to nee
 ## Known Limitations
 
 ### General
--  In combo builds, known_good_json is required.
--  inefficiencies in creating the links and saving the JSON caches
--  Not compatible with **Esbonio/Live_preview**
+
+- In combo builds, known_good_json is required.
+- inefficiencies in creating the links and saving the JSON caches
+- Not compatible with **Esbonio/Live_preview**
 
 ### Codelinks
--  GitHub links may 404 if the commit isn’t pushed
--  Tags must match exactly (e.g. #<!-- comment prevents parsing this occurance --> req-Id)
--  `source_code_link` isn’t visible until the full Sphinx build is completed
+
+- GitHub links may 404 if the commit isn’t pushed
+- Tags must match exactly (e.g. #<!-- comment prevents parsing this occurance --> req-Id)
+- `source_code_link` isn’t visible until the full Sphinx build is completed
 
 ### TestLink
 
--  GitHub links may 404 if the commit isn’t pushed
--  XML structure must be followed exactly (e.g. `properties & attributes`)
--  Relies on test to be executed first
+- GitHub links may 404 if the commit isn’t pushed
+- XML structure must be followed exactly (e.g. `properties & attributes`)
+- Relies on test to be executed first
 
 ---
 
@@ -332,16 +343,19 @@ rm -rf _build/
 ## Internal  Overview
 
 The bazel part:
-```
+
+```text
 scripts_bazel/
 ├── BUILD   # Declare libraries and filegroups needed for bazel
 ├── generate_sourcelinks_cli.py # Bazel step 1 => Parses sourcefiles for tags
 ├── merge_sourcelinks.py
 └── tests
 │   └── ...
 ```
+
 The Sphinx extension
-```
+
+```text
 score_source_code_linker/
 ├── __init__.py                   # Main Sphinx extension; combines CodeLinks + TestLinks
 ├── generate_source_code_links_json.py  # Most functionality moved to 'scripts_bazel/generate_sourcelinks_cli'
@@ -356,6 +370,7 @@ score_source_code_linker/
 ```
 
 ---
+
 ## Clearing Cache Manually
 
 To clear the build cache, run:
@@ -364,7 +379,8 @@ To clear the build cache, run:
 rm -rf _build/
 ```
 
-## Examples:
+## Examples
+
 To see working examples for CodeLinks & TestLinks, take a look at the Docs-As-Code documentation.
 
 [Example CodeLink](https://eclipse-score.github.io/docs-as-code/main/requirements/requirements.html#tool_req__docs_common_attr_id_scheme)

src/extensions/score_source_code_linker/helpers.py

@@ -107,5 +107,10 @@ def parse_info_from_known_good(
     for category in kg_json["modules"].values():
         if repo_name in category:
             m = category[repo_name]
-            return (m["hash"], m["repo"].removesuffix(".git"))
+            hash_or_version = m.get("hash") or m.get("version")
+            if hash_or_version is None:
+                raise KeyError(
+                    f"Module {repo_name} has neither 'hash' nor 'version' key."
+                )
+            return (hash_or_version, m["repo"].removesuffix(".git"))
     raise KeyError(f"Module {repo_name} not found in known_good_json.")

src/extensions/score_source_code_linker/tests/test_helpers.py

@@ -189,6 +189,76 @@ def test_parse_info_from_known_good_empty_repo_dict_in_json(tmp_path: Path):
         parse_info_from_known_good(json_file, "any_repo")
 
 
+VALID_KNOWN_GOOD_WITH_VERSION = {
+    "modules": {
+        "target_sw": {
+            "score_baselibs": {
+                "repo": "https://github.com/eclipse-score/baselibs.git",
+                "version": "v1.2.3",
+            },
+        },
+        "tooling": {
+            "score_docs_as_code": {
+                "repo": "https://github.com/eclipse-score/docs-as-code.git",
+                "version": "4.3-alpha",
+            }
+        },
+    }
+}
+
+
+@pytest.fixture
+def known_good_json_with_version(tmp_path: Path):
+    """Providing a known_good.json file that uses 'version' instead of 'hash'."""
+    json_file = tmp_path / "known_good_version.json"
+    _ = json_file.write_text(json.dumps(VALID_KNOWN_GOOD_WITH_VERSION))
+    return json_file
+
+
+def test_parse_info_from_known_good_with_version(known_good_json_with_version: Path):
+    """Test that 'version' is accepted as a fallback when 'hash' is absent."""
+    hash_result, repo_result = parse_info_from_known_good(
+        known_good_json_with_version, "score_baselibs"
+    )
+
+    assert hash_result == "v1.2.3"
+    assert repo_result == "https://github.com/eclipse-score/baselibs"
+
+
+def test_parse_info_from_known_good_with_version_different_category(
+    known_good_json_with_version: Path,
+):
+    """Test that 'version' works for a module in a different category."""
+    hash_result, repo_result = parse_info_from_known_good(
+        known_good_json_with_version, "score_docs_as_code"
+    )
+
+    assert hash_result == "4.3-alpha"
+    assert repo_result == "https://github.com/eclipse-score/docs-as-code"
+
+
+def test_parse_info_from_known_good_neither_hash_nor_version(tmp_path: Path):
+    """Test that KeyError is raised when neither 'hash' nor 'version' is present."""
+    json_file = tmp_path / "broken.json"
+    _ = json_file.write_text(
+        json.dumps(
+            {
+                "modules": {
+                    "target_sw": {
+                        "score_baselibs": {
+                            "repo": "https://github.com/eclipse-score/baselibs.git",
+                        }
+                    }
+                }
+            }
+        )
+    )
+
+    msg = "score_baselibs has neither 'hash' nor 'version'"
+    with pytest.raises(KeyError, match=msg):
+        parse_info_from_known_good(json_file, "score_baselibs")
+
+
 # Tests for get_github_link_from_json
 def test_get_github_link_from_json_happy_path():
     """

src/extensions/score_source_code_linker/tests/test_xml_parser.py

@@ -21,6 +21,8 @@
 Once we enable those we will need to change the tests
 """
 
+import json
+import os
 import xml.etree.ElementTree as ET
 from collections.abc import Callable
 from pathlib import Path
@@ -382,3 +384,85 @@ def test_clean_test_file_name_empty_path_raises_error():
         ValueError, match="Filepath does not have 'bazel-testlogs' nor 'tests-report'"
     ):
         xml_parser.clean_test_file_name(raw_path)
+
+
+#              ╭──────────────────────────────────────────────────────────╮
+#              │           Tests for get_metadata_from_test_path          │
+#              ╰──────────────────────────────────────────────────────────╯
+
+_KNOWN_GOOD_WITH_HASH = {
+    "modules": {
+        "tooling": {
+            "score_docs_as_code": {
+                "repo": "https://github.com/eclipse-score/docs-as-code.git",
+                "hash": "abc123hashvalue",
+            }
+        }
+    }
+}
+
+_KNOWN_GOOD_WITH_VERSION = {
+    "modules": {
+        "tooling": {
+            "score_docs_as_code": {
+                "repo": "https://github.com/eclipse-score/docs-as-code.git",
+                "version": "v2.1.0",
+            }
+        }
+    }
+}
+
+_COMBO_TEST_PATH = Path(
+    "/root/bazel-testlogs/external/score_docs_as_code+/src/ext/score_foo/test.xml"
+)
+
+
+def test_get_metadata_from_test_path_local():
+    """Local builds produce empty hash/url without reading known_good.json."""
+    local_path = Path(
+        "/home/root/docs-as-code/bazel-testlogs/src/extensions/foo/test.xml"
+    )
+    md = xml_parser.get_metadata_from_test_path(local_path)
+    assert md["repo_name"] == "local_repo"
+    assert md["hash"] == ""
+    assert md["url"] == ""
+
+
+def test_get_metadata_from_test_path_combo_with_hash(tmp_path: Path):
+    """Combo builds with 'hash' in known_good.json populate metadata correctly."""
+    json_file = tmp_path / "known_good.json"
+    json_file.write_text(json.dumps(_KNOWN_GOOD_WITH_HASH))
+
+    old = os.environ.get("KNOWN_GOOD_JSON")
+    try:
+        os.environ["KNOWN_GOOD_JSON"] = str(json_file)
+        md = xml_parser.get_metadata_from_test_path(_COMBO_TEST_PATH)
+    finally:
+        if old is None:
+            os.environ.pop("KNOWN_GOOD_JSON", None)
+        else:
+            os.environ["KNOWN_GOOD_JSON"] = old
+
+    assert md["repo_name"] == "score_docs_as_code"
+    assert md["hash"] == "abc123hashvalue"
+    assert md["url"] == "https://github.com/eclipse-score/docs-as-code"
+
+
+def test_get_metadata_from_test_path_combo_with_version(tmp_path: Path):
+    """Combo builds with 'version' in known_good.json populate metadata correctly."""
+    json_file = tmp_path / "known_good.json"
+    json_file.write_text(json.dumps(_KNOWN_GOOD_WITH_VERSION))
+
+    old = os.environ.get("KNOWN_GOOD_JSON")
+    try:
+        os.environ["KNOWN_GOOD_JSON"] = str(json_file)
+        md = xml_parser.get_metadata_from_test_path(_COMBO_TEST_PATH)
+    finally:
+        if old is None:
+            os.environ.pop("KNOWN_GOOD_JSON", None)
+        else:
+            os.environ["KNOWN_GOOD_JSON"] = old
+
+    assert md["repo_name"] == "score_docs_as_code"
+    assert md["hash"] == "v2.1.0"
+    assert md["url"] == "https://github.com/eclipse-score/docs-as-code"