Skip to content

feat(projectconfig): add RPM repo resources and repo-set templates#202

Merged
reubeno merged 3 commits into
microsoft:mainfrom
reubeno:repos-in-toml
May 22, 2026
Merged

feat(projectconfig): add RPM repo resources and repo-set templates#202
reubeno merged 3 commits into
microsoft:mainfrom
reubeno:repos-in-toml

Conversation

@reubeno
Copy link
Copy Markdown
Member

@reubeno reubeno commented May 20, 2026

Introduce a [resources] section that lets projects declare RPM repos and reusable layout templates in TOML, replacing per-consumer URL hard-coding. Distro versions select inputs per use-case (rpm-build vs image-build), with sets expanded lazily during validation.

Ship two built-in templates: azl-standard (base/sdk x main/debuginfo/srpms) and koji-dist-repo (per-arch trees + src). Adds reference + explanation docs with an end-to-end TOML example, regenerates the JSON schema, and rejects local gpg-key paths for rpm-build (mock evaluates URIs inside the chroot).

NOTE: This does not wire up any consumers of the new data; this is primarily focused on the schema extension and being able to process the new data.

Copilot AI review requested due to automatic review settings May 20, 2026 19:16
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new [resources] configuration section to centralize RPM repository definitions and reusable “repo set template” layouts, then lets distro versions select per-use-case repo inputs (rpm-build vs image-build). This moves repo URL/layout knowledge out of individual consumers (mock templates / kiwi configs) and into projectconfig + schema/docs.

Changes:

  • Introduce ResourcesConfig with RPM repos, repo-set templates, and repo-set instantiations (including expansion and validation).
  • Add inputs to DistroVersionDefinition to reference repos / repo sets per build use-case, with validation (including rejecting local gpg-key for rpm-build).
  • Regenerate JSON schema + scenario snapshots, and add user documentation plus default built-in templates.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
schemas/azldev.schema.json Schema update for [resources] and distros.*.versions.*.inputs.
scenario/__snapshots__/TestSnapshotsContainer_config_generate-schema_stdout_1.snap Snapshot update for schema generation output.
scenario/__snapshots__/TestSnapshots_config_generate-schema_stdout_1.snap Snapshot update for schema generation output.
internal/projectconfig/resources.go New resources model, repo-set expansion logic, and repo/GPG/URI validation helpers.
internal/projectconfig/resources_internal_test.go Unit tests for resources validation and expansion helpers.
internal/projectconfig/project.go Wire resources + distro input validation into ProjectConfig.Validate().
internal/projectconfig/loader.go Merge [resources] across config files with per-file relative-path resolution.
internal/projectconfig/distro.go Add DistroVersionDefinition.Inputs + DistroVersionInputs types.
internal/projectconfig/configfile.go Allow config files to declare [resources] (schema + load structure).
docs/user/reference/config/resources.md Reference documentation for the new [resources] section.
docs/user/reference/config/distros.md Document distros.*.versions.*.inputs.
docs/user/reference/config/config-file.md Add resources to root config layout.
docs/user/README.md Link new explanation/reference docs.
docs/user/explanation/repos.md Explanation doc for repo resources, templates, and sets.
defaultconfigs/content/defaults.toml Ship built-in templates (azl-standard, koji-dist-repo).
Comments suppressed due to low confidence (1)

internal/projectconfig/project.go:175

  • Same wrapping style issue as above: this %w wrap would be clearer with the repo’s standard "context:\n%w" formatting to keep nested errors readable.
			imageBuild, err := version.EffectiveImageBuildRepos(resources)
			if err != nil {
				return fmt.Errorf("distro %q version %q: %w", distroName, versionName, err)
			}

Comment thread internal/projectconfig/resources.go Outdated
Comment thread internal/projectconfig/resources.go Outdated
Comment thread internal/projectconfig/resources.go Outdated
Comment thread internal/projectconfig/project.go
Comment thread internal/projectconfig/project.go
@reubeno reubeno force-pushed the repos-in-toml branch 2 times, most recently from c35c81b to f5f2d71 Compare May 20, 2026 22:07
Copilot AI review requested due to automatic review settings May 20, 2026 22:07
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (2)

internal/projectconfig/resources.go:763

  • The undefined-template error suggests defining the template under [resources.rpm-repo-set-templates.%s], but template names are allowed to include '.'/':' etc (per the name regex) which would require quoting in TOML keys. Consider rewording to avoid implying an unquoted table header (e.g., “define a key under [resources.rpm-repo-set-templates] named …”), or render the example using a quoted key form.
		return nil, fmt.Errorf(
			"rpm-repo-set %#q references undefined `template` %#q; "+
				"define it under [resources.rpm-repo-set-templates.%s]",
			setName, set.Template, set.Template,
		)

internal/projectconfig/resources.go:1090

  • This error message formats repo/set values with %q. Elsewhere in this file errors tend to use %#q for strings (and the repo has a convention for that), which makes messages unambiguous when values contain escapes. Consider switching these to %#q for consistency/readability.
		return fmt.Errorf(
			"`inputs.%s` entry %d sets both `repo` (%q) and `set` (%q); exactly one is required",
			useCase, index, entry.Repo, entry.Set,
		)

Comment thread internal/projectconfig/resources.go Outdated
Comment thread internal/projectconfig/resources.go Outdated
@reubeno
Copy link
Copy Markdown
Member Author

reubeno commented May 20, 2026

Addressed the latest Copilot findings in d29d4a8:

  • resources.go:763 / 727 / 1087 (TOML key examples): rephrased the undefined-template and collision error messages to reference the parent table ([resources.rpm-repo-set-templates] / [resources.rpm-repos]) rather than synthesizing a child table header that may need quoting.
  • resources.go:1090 (%q%#q): switched the XOR error in validateDistroVersionInput to use %#q for repo/set values to match the rest of the file.
  • resources.go:685 (EffectiveRpmRepos doc): clarified that path absolutization is a precondition handled by WithAbsolutePaths during load/merge — the method itself only joins URIs and validates.

@reubeno reubeno marked this pull request as ready for review May 20, 2026 23:11
Copilot AI review requested due to automatic review settings May 20, 2026 23:11
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated 3 comments.

Comment thread internal/projectconfig/resources.go Outdated
Comment thread internal/projectconfig/resources.go Outdated
Comment thread internal/projectconfig/resources.go
Introduce a [resources] section that lets projects declare RPM repos and
reusable layout templates in TOML, replacing per-consumer URL hard-coding.
Distro versions select inputs per use-case (rpm-build vs image-build), with
sets expanded lazily during validation.

Ship two built-in templates: azl-standard (base/sdk x main/debuginfo/srpms)
and koji-dist-repo (per-arch trees + src). Adds reference + explanation docs
with an end-to-end TOML example, regenerates the JSON schema, and rejects
local gpg-key paths for rpm-build (mock evaluates URIs inside the chroot).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (1)

internal/projectconfig/resources.go:985

  • Template sub-repo subpath validation rejects leading slashes but does not reject .. path segments, even though joinSetBaseURI rejects .. during set expansion. This means an invalid template can pass validation and only fail later during expansion (with less direct context). Consider rejecting .. segments here as well to keep template validation self-contained and consistent.
		if strings.HasPrefix(sub.Subpath, "/") {
			return fmt.Errorf(
				"rpm-repo-set-template %#q sub-repo %#q `subpath` must be relative (no leading slash)",
				name, sub.Name,
			)

Comment thread internal/projectconfig/resources.go Outdated
Comment thread internal/projectconfig/resources.go
Comment thread internal/projectconfig/resources.go
Comment thread internal/projectconfig/resources.go
Comment thread internal/projectconfig/loader.go
- validateRpmRepoSetTemplate: report sub-repo index as 1-based for
  consistency with other inputs.* diagnostics.
- joinSetBaseURI: reject `?` and `#` in sub-repo `subpath` so a
  subpath cannot reintroduce a query string or fragment into the
  synthesized repo URL (base-uri is already checked for both).
- ResourcesConfig.MergeUpdatesFrom: drop the always-nil error return;
  the method only does map overlays and has no fallible operations,
  unlike sibling MergeUpdatesFrom methods that delegate to mergo.
- loader.mergeResources: drop the now-unreachable error wrap.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

@Tonisal-byte Tonisal-byte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good :)

Comment thread internal/projectconfig/project.go
…mRepo

validateRpmRepos already calls validateRpmRepoName on each map key
immediately before invoking validateRpmRepo, so the inner re-check was
redundant. Make validateRpmRepo's contract purely 'per-resource
structural validity' and leave key validation to the caller, matching
the pattern used by validateRpmRepoSetTemplate / validateRpmRepoSet.

Reported by @Tonisal-byte on PR microsoft#202.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 22, 2026 18:34
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated 1 comment.

Comment on lines +984 to +993
if strings.HasPrefix(sub.Subpath, "/") {
return fmt.Errorf(
"rpm-repo-set-template %#q sub-repo %#q `subpath` must be relative (no leading slash)",
name, sub.Name,
)
}

if err := validateNoUnsafeChars("rpm-repo-set-template", "subpath", sub.Name, sub.Subpath); err != nil {
return fmt.Errorf("rpm-repo-set-template %#q:\n%w", name, err)
}
@reubeno reubeno merged commit 73f47fd into microsoft:main May 22, 2026
15 checks passed
@reubeno reubeno deleted the repos-in-toml branch May 22, 2026 20:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants