Skip to content

feat: add consolidated module-index JSON files and auto-update workflow#2762

Closed
jaredfholgate wants to merge 1 commit into
mainfrom
feature/module-indexes-json
Closed

feat: add consolidated module-index JSON files and auto-update workflow#2762
jaredfholgate wants to merge 1 commit into
mainfrom
feature/module-indexes-json

Conversation

@jaredfholgate

Copy link
Copy Markdown
Member

Summary

Adds three consolidated module-index JSON files under docs/static/module-indexes/, plus the tooling and CI to keep them in sync with the existing CSV indexes.

Generated JSON files

File Shape Purpose
modules.json sorted array Merged Bicep + Terraform module data. Resource modules are matched on ProviderNamespace+ResourceType so each resource appears once with both ecosystems nested under it. Pattern and utility modules are listed per ecosystem.
bicep.json map keyed by ModuleName Minimal { moduleType, displayName, status } — designed to be consumed by the Bicep VS Code extension AVM provider (PR #18958).
terraform.json map keyed by ModuleName Terraform equivalent of bicep.json.

All files are deterministic, UTF-8 (no BOM), LF line endings, sorted, trailing newline.

New tooling

  • utilities/tools/platform/Set-ModuleIndexesJson.ps1 — generator. Reads the six CSVs (BicepResourceModules.csv, BicepPatternModules.csv, BicepUtilityModules.csv and the Terraform equivalents), normalises the rows (drops the ModuleOwnersGHTeam column), matches resource modules across ecosystems, and writes the three JSONs.
  • utilities/pipelines/sharedScripts/New-AutoMergePullRequest.ps1 — generic shared helper for any future "regenerate file → open & auto-merge PR" workflow. Takes a file list, branch prefix, title and a body with a {{CHANGED_FILES}} placeholder; uses git status --porcelain (handles untracked-on-first-run); creates a timestamped branch, opens a PR via gh and merges with --admin (or --auto if -Admin is omitted). Validated with -WhatIf.

New workflow

.github/workflows/platform.updateModuleIndexesJson.yml:

  • Triggers on push to main when any of the source CSVs, the generator script, the shared PR helper or this workflow itself change, plus workflow_dispatch. (No cron — all inputs are in-repo, so there is nothing to drift between pushes.)
  • Authenticates as the AVM GitHub App using AVM_APP_CLIENT_ID / AVM_APP_PRIVATE_KEY (same secrets as avm-terraform-governance).
  • Configures git identity dynamically as <app-slug>[bot] with the correct <id>+<app-slug>[bot]@users.noreply.github.com email (governance pattern).
  • Regenerates the JSONs and calls New-AutoMergePullRequest to publish them via an auto-merged PR.
  • All steps run in pwsh; permissions: contents: read (writes go through the App token).

Notes / open questions

  • Workflow uses environment: platform (this repo's existing convention). avm-terraform-governance uses environment: avm — happy to switch if preferred.
  • Merge mode is --admin for an immediate auto-merge. If branch protection is configured to reject this for App identities, swap to --auto in the workflow's New-AutoMergePullRequest call (just drop the -Admin switch).

Adds three JSON files under docs/static/module-indexes/ generated from the
existing CSV indexes:

* modules.json  - merged Bicep + Terraform module data (resource modules
                  matched on ProviderNamespace+ResourceType; pattern and
                  utility modules listed per ecosystem). Sorted array.
* bicep.json    - minimal map keyed by ModuleName (moduleType, displayName,
                  status) for consumption by the Bicep VS Code extension.
* terraform.json - Terraform equivalent of bicep.json.

Tooling:

* utilities/tools/platform/Set-ModuleIndexesJson.ps1 - generator (parses the
  CSVs, normalises rows, emits deterministic UTF-8 (no BOM) + LF output).
* utilities/pipelines/sharedScripts/New-AutoMergePullRequest.ps1 - generic
  helper that detects changes for a given file list, creates a timestamped
  branch, opens a PR (with a {{CHANGED_FILES}} placeholder in the body)
  using the GitHub App and auto-merges it.
* .github/workflows/platform.updateModuleIndexesJson.yml - runs on push to
  main (when CSVs/scripts/workflow change) or workflow_dispatch. Uses the
  AVM GitHub App (AVM_APP_CLIENT_ID / AVM_APP_PRIVATE_KEY) and opens an
  auto-merged PR via the shared helper.
@microsoft-github-policy-service microsoft-github-policy-service Bot added the Needs: Triage 🔍 Maintainers need to triage still label Jun 10, 2026
@microsoft-github-policy-service

Copy link
Copy Markdown

Important

The "Needs: Triage 🔍" label must be removed once the triage process is complete!

Tip

For additional guidance on how to triage this issue/PR, see the AVM Issue Triage documentation.

@jtracey93

Copy link
Copy Markdown
Contributor

@jaredfholgate are we going to close this in favour of our other approach we have planned to flip this around?

@jaredfholgate

Copy link
Copy Markdown
Member Author

@jaredfholgate are we going to close this in favour of our other approach we have planned to flip this around?

Yes. I'll close this for now. This was triggered by the Bicep VSCode extension PR, but I think we can solve that by adding what they need to the existing metadata endpoint instead?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Needs: Triage 🔍 Maintainers need to triage still

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants