feat(creds): wire network and firstboot system credential consumers

docs/skills/architecture-roadmap.md

@@ -25,10 +25,9 @@ Priorities are derived from [gap-analysis-distros.md](gap-analysis-distros.md).
 | 1 | A/B dual-slot root partitions with matching ESP/UKI slots | Root fs only has slot A today; sysupdate already names slots A+B. |
 | 2 | Mount `/usr` read-only and enforce the state model | DDI is currently booted `rw`; sysext-first design assumes immutable `/usr`. |
 | 3 | Boot-time selection / automatic rollback of a failed update | No previous OS version is kept once a root update overwrites the slot. |
-| 4 | Broader `systemd-creds` integration for SSH keys and network configuration | Only `root` password credential path is shipped in `os-creds-prov.bst`. |
-| 5 | TPM2-bound credential delivery at first boot | Documented but not wired into the installed OS image. |
-| 6 | Native reboot coordination for non-Kubernetes and single-node hosts | Kured only covers Kubernetes nodes; no FleetLock/locksmith equivalent. |
-| 7 | Staged rollout behavior for larger fleets | Future after items 1-3 are implemented. |
+| 4 | Credential provisioning smoke tests on real hardware | SSH keys, static network, and firstboot settings are wired through systemd credentials; TPM2-sealed credential decryption still needs hardware proof. |
+| 5 | Native reboot coordination for non-Kubernetes and single-node hosts | Kured only covers Kubernetes nodes; no FleetLock/locksmith equivalent. |
+| 6 | Staged rollout behavior for larger fleets | Future after items 1-3 are implemented. |
 
 ## Status notes
 

docs/skills/ddi-installer.md

@@ -47,11 +47,21 @@ terminal-based interactive installation that:
 - Reboots into the installed system.
 
 User provisioning is handled on the target system's first boot via systemd
-system credentials (`systemd-sysusers`, `systemd-tmpfiles`) so the base image
-remains stateless. `systemd-firstboot.service` is masked on the target image
-(`/etc/systemd/system/systemd-firstboot.service -> /dev/null`) to guarantee
-unattended, prompt-free startup. The target DDI also pre-stages the extracted CA
-certificate bundle (`/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem` and
+system credentials so the base image remains stateless. `systemd-sysusers`
+consumes root account records, `systemd-tmpfiles` consumes `tmpfiles.extra`
+for arbitrary first-boot files, and `systemd-network-generator` consumes
+`network.conf.*` / `network.link.*` / `network.netdev.*` /
+`network.network.*` credentials before networkd starts. The stock interactive
+`systemd-firstboot.service` stays masked, but
+`bluefin-firstboot-credentials.service` runs `systemd-firstboot`
+non-interactively when `firstboot.locale`, `firstboot.timezone`,
+`firstboot.hostname`, or related credentials are present. When
+`firstboot.hostname` is supplied, the service also applies the live kernel
+hostname before `systemd-networkd` starts so first-boot DHCP uses it. With no
+credentials, the target keeps the default DHCP network and does not prompt. The
+target DDI
+also pre-stages the extracted CA certificate bundle
+(`/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem` and
 `/etc/ssl/certs/ca-certificates.crt`) and a standard `/etc/hosts` file for
 container runtime pod sandboxes.
 

docs/skills/gap-analysis-distros.md

@@ -97,7 +97,7 @@ Bluefin Server is a BuildStream 2-based, image-based Linux server OS built from
 | **Philosophy** | Systemd-native, minimal, image-based server OS appliance; base DDI includes bash for login and bring-up while heavy developer/debug tools live in sysexts or system containers; intended to run container workloads and Kubernetes via optional sysexts. Sources: [AGENTS.md](../../AGENTS.md), [factory-integration.md](factory-integration.md). |
 | **State model** | Target OS DDI is an XFS filesystem image. A separate persistent `/var` partition is created by the installer. There is no second root slot provisioned today, and the UKI cmdline currently uses `rw`, so the root is not mounted read-only at runtime. Sources: [bluefin-server-ddi.bst](../../elements/oci/bluefin-server-ddi.bst), [20-root-a.conf](../../files/installer/repart.d/20-root-a.conf), [bluefin-server-installer.bst](../../elements/oci/bluefin-server-installer.bst). |
 | **Updates** | `systemd-sysupdate` reads root/UKI transfers from `files/os/sysupdate.d/` and the optional k0s transfer from the `k0s` component directory. Assets are published to GitHub Releases, and the combined `SHA256SUMS` manifest is signed in CI with a GPG key. `Verify=yes` is the default. Sources: [systemd-sysupdate-verification.md](systemd-sysupdate-verification.md), [50-root.transfer](../../files/os/sysupdate.d/50-root.transfer), [60-uki.transfer](../../files/os/sysupdate.d/60-uki.transfer), [70-k0s.transfer](../../files/os/sysupdate.k0s.d/70-k0s.transfer), also `systemd-sysupdate(8)`. |
-| **Provisioning** | The installer is an offline `systemd-sysinstall` image that embeds the DDI as a data partition. First-boot configuration is intended to be delivered via `systemd-creds` through the ESP or hypervisor metadata. Today only `passwd.hashed-password.root` is consumed via `systemd-sysusers.d`; the documented `tmpfiles.extra` path for SSH keys and similar files is not implemented. Sources: [bluefin-server-installer.bst](../../elements/oci/bluefin-server-installer.bst), [10-root-creds.conf](../../files/os/sysusers.d/10-root-creds.conf), [os-creds-prov.bst](../../elements/bluefin-server/os-creds-prov.bst), [systemd-creds(1)](https://www.freedesktop.org/software/systemd/man/latest/systemd-creds.html). |
+| **Provisioning** | The installer is an offline `systemd-sysinstall` image that embeds the DDI as a data partition. First-boot configuration is delivered via `systemd-creds` through the ESP or hypervisor metadata: sysusers credentials cover root, `tmpfiles.extra` can write arbitrary first-boot files such as SSH `authorized_keys`, `network.*` credentials seed networkd, and `firstboot.*` credentials configure locale/timezone/hostname non-interactively. Sources: [bluefin-server-installer.bst](../../elements/oci/bluefin-server-installer.bst), [os-creds-prov.bst](../../elements/bluefin-server/os-creds-prov.bst), [tpm2-credential-sealing.md](tpm2-credential-sealing.md), [systemd-creds(1)](https://www.freedesktop.org/software/systemd/man/latest/systemd-creds.html). |
 | **Customization** | Adds software through `systemd-sysext` (overlay `/usr`) and `systemd-confext` (overlay `/etc`) images. The base OS `os-release` advertises `ID=flatcar` and a matching `VERSION_ID` so pre-built Flatcar Bakery extensions load. k0s is shipped as a separately built, optionally enabled sysext. Sources: [systemd-sysext-extensions.md](systemd-sysext-extensions.md), [k0s-sysext.md](k0s-sysext.md), [os-release-flatcar.bst](../../elements/bluefin-server/os-release-flatcar.bst), [systemd-sysext(8)](https://www.freedesktop.org/software/systemd/man/latest/systemd-sysext.html). |
 | **Reboot coordination** | `systemd-sysupdate.service` has an `ExecStartPost` that touches `/run/reboot-required`. Rolling reboots across Kubernetes nodes rely on Kured reading that file. Sources: [os-kured-hook.bst](../../elements/bluefin-server/os-kured-hook.bst), [kured-hook.conf](../../files/os/systemd/systemd-sysupdate.service.d/kured-hook.conf), [Kured project](https://github.com/weaveworks/kured). |
 
@@ -112,10 +112,9 @@ Bluefin Server is a BuildStream 2-based, image-based Linux server OS built from
 
 ### Provisioning
 
-- **Gap:** The documented `systemd-creds` first-boot provisioning for SSH keys (`tmpfiles.extra`) is not present in the build (`os-creds-prov.bst` only ships `sysusers.d`).
-  Operators can only pre-seed the root password today.
-- **Gap:** `systemd-firstboot` is masked in the installer environment, so interactive first-boot questions are skipped; any further user/network/timezone configuration must be supplied through credentials that the build does not yet consume.
-- **Gap:** TPM2 sealing for credentials is documented in `docs/skills/tpm2-credential-sealing.md`, but there is no evidence in the OS build that sealed credentials are generated, shipped, or decrypted automatically during first boot.
+- **Remaining gap:** The credential consumers are wired through systemd-native
+  facilities, but TPM2-sealed credential decryption still needs a hardware boot
+  proof on a TPM2-equipped host.
 
 ### Update delivery
 
@@ -134,7 +133,7 @@ Bluefin Server is a BuildStream 2-based, image-based Linux server OS built from
 
 1. **A/B dual-slot rollback is not wired end-to-end.** The `sysupdate` transfer names `root-a`/`root-b`, but the installer only provisions `root-a`, so Bluefin cannot atomically stage and roll back a new root image today.
 2. **Root filesystem immutability is not enforced.** The DDI is built and booted read-write; the intended read-only `/usr` + overlay model depends entirely on optional sysext behavior rather than runtime policy.
-3. **First-boot credential provisioning is incomplete.** Only the `root` password credential path is shipped; SSH keys, network configuration, and other `systemd-creds`-based provisioning remain documented but not implemented.
+3. **Credential provisioning hardware proof is incomplete.** SSH keys, network files, and firstboot settings are wired through `systemd-creds`; TPM2-sealed decryption still needs a hardware boot proof.
 
 These gaps drive the priorities in [architecture-roadmap.md](architecture-roadmap.md).
 

docs/skills/tpm2-credential-sealing.md

@@ -14,12 +14,22 @@ To secure sensitive provisioning credentials (such as hashed root passwords or
 SSH keys) against physical tampering or unauthorized extraction, bind them to
 the TPM2 and the UKI boot state using `systemd-creds`.
 
-## Current scope
+## Supported provisioning credentials
 
-This skill is the canonical home for sealed credentials such as hashed root
-passwords and similar provisioning secrets. Broader `systemd-creds` integration
-for SSH keys and network configuration is a future roadmap item in
-[architecture-roadmap.md](architecture-roadmap.md).
+Bluefin Server consumes the same system credential names that systemd already
+decrypts and routes at boot:
+
+| Credential | Consumer | Purpose |
+|---|---|---|
+| `passwd.hashed-password.root` | `systemd-sysusers` / `systemd-firstboot` | Optional root password hash for break-glass provisioning. |
+| `tmpfiles.extra` | `systemd-tmpfiles` | Extra tmpfiles rules, such as writing an operator's SSH `authorized_keys`. |
+| `network.network.*`, `network.netdev.*`, `network.link.*`, `network.conf.*` | `systemd-network-generator` | Static network, routes, virtual devices, and networkd config. |
+| `network.dns`, `network.search_domains` | `systemd-resolved` | DNS resolver defaults. |
+| `firstboot.locale`, `firstboot.locale-messages`, `firstboot.keymap`, `firstboot.timezone`, `firstboot.hostname` | `bluefin-firstboot-credentials.service` | Non-interactive locale, keymap, timezone, and hostname setup. |
+
+The stock DHCP network remains installed in `/usr/lib/systemd/network/20-wired.network`.
+Credential-generated network files are emitted under `/run/systemd/network/` and
+can use lower numeric prefixes such as `10-static.network` to override DHCP.
 
 ## Verify TPM2 device availability
 
@@ -44,8 +54,9 @@ systemd-creds encrypt \
   /path/to/secured_credential.cred
 ```
 
-- `--name=` must match the credential name the consumer expects (for example,
-  `passwd.hashed-password.root` is read by `systemd-sysusers`).
+- `--name=` must match the credential name the consumer expects. Examples:
+  `passwd.hashed-password.root`, `tmpfiles.extra`,
+  `network.network.10-static`, or `firstboot.hostname`.
 - `--with-key=tpm2` forces a TPM2-bound credential. The default `auto` also uses
   the host key if `/var/lib/systemd/` is on persistent media; omit the switch
   if you want both bindings.
@@ -60,12 +71,35 @@ container/hypervisor mechanism:
 ```bash
 # ESP delivery
 mkdir -p /loader/credentials/
-cp /path/to/secured_credential.cred /loader/credentials/passwd.hashed-password.root.cred
+cp /path/to/secured_credential.cred \
+  /loader/credentials/passwd.hashed-password.root.cred
 
 # Or via a container/hypervisor argument
 --set-credential=passwd.hashed-password.root:/path/to/secured_credential.cred
 ```
 
+Example `tmpfiles.extra` payload writing SSH `authorized_keys` for the `core`
+operator account. The account must already be defined by `systemd-sysusers`,
+and the payload must create the parent directories before writing the file:
+
+```text
+d /var/home/core 0700 core core -
+d /var/home/core/.ssh 0700 core core -
+f~ /var/home/core/.ssh/authorized_keys 0600 core core - c3NoLWVkMjU1MTkgQUFBQUMzTnphQzFsWkRJMU5URTEAAAA...
+```
+
+Example static network credential name: `network.network.10-static`.
+
+```ini
+[Match]
+Name=en*
+
+[Network]
+Address=192.0.2.10/24
+Gateway=192.0.2.1
+DNS=192.0.2.53
+```
+
 ## See also
 
 - [CONTEXT.md](../../CONTEXT.md) — canonical project domain glossary.

elements/bluefin-server/os-creds-prov.bst

@@ -1,9 +1,31 @@
-kind: import
-description: Install first-boot systemd credentials provisioning helpers into target OS.
+kind: manual
+
+description: |
+  Install first-boot systemd credential provisioning contracts into the target OS.
+  Supports sysusers credentials, systemd-network-generator network.* credentials,
+  and a non-interactive systemd-firstboot credential gate.
+
+build-depends:
+  - base/base-stack.bst
 
 sources:
   - kind: local
     path: files/os/sysusers.d
+    directory: sysusers-src
+  - kind: local
+    path: files/os/creds/systemd/system
+    directory: systemd-src
+
+# Disable binary stripping; this element only installs text files and the
+# runtime-minimal sandbox does not include the stripper toolchain.
+variables:
+  strip-binaries: ""
 
 config:
-  target: /usr/lib/sysusers.d
+  install-commands:
+    - |
+      set -euo pipefail
+
+      install -Dm0644 sysusers-src/*.conf -t "%{install-root}/usr/lib/sysusers.d/"
+      mkdir -p "%{install-root}/usr/lib/systemd/system"
+      cp -a systemd-src/. "%{install-root}/usr/lib/systemd/system/"

files/os/creds/systemd/system/bluefin-firstboot-credentials.service

@@ -0,0 +1,28 @@
+[Unit]
+Description=Apply Bluefin first-boot system credentials
+Documentation=man:systemd-firstboot(1) man:systemd.system-credentials(7)
+ConditionPathIsReadWrite=/etc
+DefaultDependencies=no
+After=systemd-remount-fs.service systemd-sysusers.service systemd-tmpfiles-setup.service
+Before=sysinit.target systemd-networkd.service
+ConditionPathExists=!/etc/.bluefin-firstboot-credentials
+ConditionCredential=|firstboot.locale
+ConditionCredential=|firstboot.locale-messages
+ConditionCredential=|firstboot.keymap
+ConditionCredential=|firstboot.timezone
+ConditionCredential=|firstboot.hostname
+
+[Service]
+Type=oneshot
+ImportCredential=firstboot.locale
+ImportCredential=firstboot.locale-messages
+ImportCredential=firstboot.keymap
+ImportCredential=firstboot.timezone
+ImportCredential=firstboot.hostname
+ExecStart=systemd-firstboot --force --welcome=no
+ExecStartPost=/bin/bash -eu -c 'if [ -s "$CREDENTIALS_DIRECTORY/firstboot.hostname" ]; then hostname="$(<"$CREDENTIALS_DIRECTORY/firstboot.hostname")"; /usr/bin/hostname "$hostname"; fi'
+ExecStartPost=/usr/bin/touch /etc/.bluefin-firstboot-credentials
+RemainAfterExit=yes
+
+[Install]
+WantedBy=sysinit.target

files/os/creds/systemd/system/systemd-network-generator.service.d/10-bluefin-credentials.conf

@@ -0,0 +1,5 @@
+[Service]
+ImportCredential=network.conf.*
+ImportCredential=network.link.*
+ImportCredential=network.netdev.*
+ImportCredential=network.network.*

files/os/systemd/system-preset/zz-enable-bluefin-firstboot-credentials.preset

@@ -0,0 +1 @@
+enable bluefin-firstboot-credentials.service

files/os/systemd/system-preset/zz-enable-systemd-network-generator.preset

@@ -0,0 +1 @@
+enable systemd-network-generator.service

tests/unit/test_credentials_provisioning.py

@@ -0,0 +1,134 @@
+"""Contracts for first-boot systemd credential provisioning."""
+
+from __future__ import annotations
+
+from pathlib import Path
+
+import yaml
+
+REPO_ROOT = Path(__file__).resolve().parents[2]
+ELEMENT = REPO_ROOT / "elements" / "bluefin-server" / "os-creds-prov.bst"
+STACK = REPO_ROOT / "elements" / "bluefin-server" / "os-stack.bst"
+FIRSTBOOT = (
+    REPO_ROOT
+    / "files"
+    / "os"
+    / "creds"
+    / "systemd"
+    / "system"
+    / "bluefin-firstboot-credentials.service"
+)
+FIRSTBOOT_PRESET = (
+    REPO_ROOT
+    / "files"
+    / "os"
+    / "systemd"
+    / "system-preset"
+    / "zz-enable-bluefin-firstboot-credentials.preset"
+)
+NETWORK_GENERATOR_PRESET = (
+    REPO_ROOT
+    / "files"
+    / "os"
+    / "systemd"
+    / "system-preset"
+    / "zz-enable-systemd-network-generator.preset"
+)
+NETWORK_GENERATOR_DROPIN = (
+    REPO_ROOT
+    / "files"
+    / "os"
+    / "creds"
+    / "systemd"
+    / "system"
+    / "systemd-network-generator.service.d"
+    / "10-bluefin-credentials.conf"
+)
+NETWORK = REPO_ROOT / "files" / "os" / "systemd" / "network" / "20-wired.network"
+TPM2_SKILL = REPO_ROOT / "docs" / "skills" / "tpm2-credential-sealing.md"
+
+
+def test_creds_provisioning_element_stages_all_credential_consumers() -> None:
+    data = yaml.safe_load(ELEMENT.read_text(encoding="utf-8"))
+
+    assert data["kind"] == "manual"
+    assert "base/base-stack.bst" in data["build-depends"]
+    assert "bluefin-server/os-creds-prov.bst" in STACK.read_text(encoding="utf-8")
+
+    sources = {source["directory"]: source["path"] for source in data["sources"]}
+    assert sources == {
+        "sysusers-src": "files/os/sysusers.d",
+        "systemd-src": "files/os/creds/systemd/system",
+    }
+
+    commands = "\n".join(data["config"]["install-commands"])
+    assert "/usr/lib/sysusers.d/" in commands
+    assert "cp -a systemd-src/." in commands
+
+
+def test_firstboot_credentials_are_noninteractive_and_presence_gated() -> None:
+    unit = FIRSTBOOT.read_text(encoding="utf-8")
+
+    for credential in (
+        "firstboot.locale",
+        "firstboot.locale-messages",
+        "firstboot.keymap",
+        "firstboot.timezone",
+        "firstboot.hostname",
+    ):
+        assert f"ConditionCredential=|{credential}" in unit
+        assert f"ImportCredential={credential}" in unit
+
+    assert "ExecStart=systemd-firstboot --force --welcome=no" in unit
+    assert "/usr/bin/hostname \"$hostname\"" in unit
+    assert unit.index("/usr/bin/hostname") < unit.index("/usr/bin/touch /etc/.bluefin-firstboot-credentials")
+    assert "--prompt" not in unit
+    assert "ConditionPathIsReadWrite=/etc" in unit
+    assert "ConditionPathExists=!/etc/.bluefin-firstboot-credentials" in unit
+    assert (
+        "After=systemd-remount-fs.service systemd-sysusers.service "
+        "systemd-tmpfiles-setup.service"
+        in unit
+    )
+    assert "WantedBy=sysinit.target" in unit
+    assert FIRSTBOOT_PRESET.read_text(encoding="utf-8") == (
+        "enable bluefin-firstboot-credentials.service\n"
+    )
+
+
+def test_network_credentials_override_dhcp_without_removing_fallback() -> None:
+    network = NETWORK.read_text(encoding="utf-8")
+    skill = TPM2_SKILL.read_text(encoding="utf-8")
+
+    assert network == "[Match]\nName=e*\n\n[Network]\nDHCP=ipv4\n"
+    assert "network.network.*" in skill
+    assert "network.netdev.*" in skill
+    assert NETWORK_GENERATOR_DROPIN.read_text(encoding="utf-8") == (
+        "[Service]\n"
+        "ImportCredential=network.conf.*\n"
+        "ImportCredential=network.link.*\n"
+        "ImportCredential=network.netdev.*\n"
+        "ImportCredential=network.network.*\n"
+    )
+    assert "systemd-network-generator" in skill
+    assert "/run/systemd/network/" in skill
+    assert "10-static.network" in skill
+    assert NETWORK_GENERATOR_PRESET.read_text(encoding="utf-8") == (
+        "enable systemd-network-generator.service\n"
+    )
+
+
+def test_tpm2_sealing_docs_cover_all_supported_credential_names() -> None:
+    skill = TPM2_SKILL.read_text(encoding="utf-8")
+
+    for credential in (
+        "passwd.hashed-password.root",
+        "tmpfiles.extra",
+        "network.network.10-static",
+        "firstboot.hostname",
+    ):
+        assert credential in skill
+
+    assert "--with-key=tpm2" in skill
+    assert "--tpm2-pcrs=7+11" in skill
+    assert "/loader/credentials/" in skill