Skip to content

auto-updates fail with opendir(boot): Operation not permitted when /boot is a systemd automount that has idled out #2402

Description

@stavros-k

⚠️ DISCLAIMER. Report is generated by Claude Code while it was troubleshooting one of my bench systems.
I've read the report and indeed seems to be true.
If you don't want to read an LLM report, please stop here. Feel free to close the report.


Summary

bootc-fetch-apply-updates.service does not declare RequiresMountsFor=/boot. On systems where /boot is a systemd-gpt-auto-generator automount with TimeoutIdleSec=120, /boot unmounts ~2 minutes after boot and the update timer then fires against an unmounted /boot, failing with:

error: Initializing storage: opendir(boot): Operation not permitted

The same failure hits bootc status and bootc upgrade run manually.

ostree hardened its own units against this exact hazard in ostreedev/ostree#2544; bootc's updater unit never got the equivalent.

Environment

  • bootc 1.16.7-1.fc44
  • ostree 2026.3-1.fc44
  • systemd 259.8-1.fc44
  • Base image quay.io/fedora/fedora-bootc:44
  • BIOS/GRUB boot: sda1 BIOS boot, sda2 vfat ESP (unused), sda3 xfs root
  • / is a composefs overlay; no /etc/fstab

Symptom

Aug 21 18:40:55 systemd[1]: Starting bootc-fetch-apply-updates.service - Apply bootc updates...
Aug 21 18:40:56 bootc[2194]: error: Initializing storage: opendir(boot): Operation not permitted
Aug 21 18:40:56 systemd[1]: bootc-fetch-apply-updates.service: Failed with result 'exit-code'.

Repeats on every timer firing. opendir(...) is libostree's glnx_throw_errno_prefix style, i.e. EPERM from openat(sysroot_fd, "boot").

Reproducer

Deterministic on an affected system:

$ sudo umount /boot          # simulate the automount idling out
$ sudo bootc status
error: Status: opendir(boot): Operation not permitted

Note bootc's access does not trigger the automount — /boot remains unmounted afterwards. (I suspect this is because bootc re-execs under unshare -m and autofs won't service the request from that private mount namespace, but I haven't confirmed that.)

Touch /boot by any other means and bootc works again:

$ ls /boot >/dev/null && sudo bootc status   # succeeds

Root cause

Two generators both claim /boot:

  • ostree-system-generator -> /run/systemd/generator/boot.mount (What=/sysroot/boot, Options=bind) — the real mount
  • systemd-gpt-auto-generator -> /run/systemd/generator.late/boot.automount (Where=/boot, TimeoutIdleSec=120, described as "EFI System Partition Automount") — targeting the otherwise-unused ESP

When the autofs idle timer fires it tears down ostree's /boot bind mount. The ostree units survive this because they declare the dependency:

$ systemctl show boot.mount -p RequiredBy
RequiredBy=ostree-finalize-staged.service ostree-boot-complete.service \
           ostree-finalize-staged-hold.service rpm-ostree-fix-shadow-mode.service local-fs.target

$ systemctl show bootc-fetch-apply-updates.service -p RequiresMountsFor
RequiresMountsFor=

Why this is newly visible

This system ran fine for two weeks on ostree 2026.2 and broke on upgrade to 2026.3, with bootc, systemd and the kernel at identical versions across both deployments. The behaviour change is ostreedev/ostree@62109dca ("prepare-root: create /run/systemd/volatile-root for composefs"), whose commit message states:

With composefs enabled both are overlayfs mounts with an anonymous st_dev … As a result systemd-gpt-auto-generator silently skips all partition discovery: the ESP is never automounted on /boot

Before 2026.3, gpt-auto could not resolve the composefs root's block device and skipped partition discovery entirely, so boot.automount was never generated. Journal evidence — Set up automount boot.automount appears only in deployments running 2026.3:

boot -3: ostree 2026.2  automount=0
boot -2: ostree 2026.3  automount=1   <- /boot unmounted 2m41s after boot
boot -1: ostree 2026.2  automount=0
boot  0: ostree 2026.3  automount=1

So ostree 2026.3 makes gpt-auto work as intended, which in turn exposes the missing dependency in bootc. I'd expect this to affect any composefs-based bootc system with a discoverable ESP as 2026.3 rolls out.

Proposed fix

Mirror what ostree does — add to bootc-fetch-apply-updates.service:

[Unit]
RequiresMountsFor=/boot

Verified as effective here via a drop-in: systemd then pulls in boot.mount and orders the service after it.

Two secondary suggestions:

  1. opendir(boot): Operation not permitted is opaque for what is really "/boot is not mounted". A dedicated check with a clearer message would save a lot of digging.
  2. Any other bootc command touching /boot (bootc status, bootc upgrade, bootc switch) has the same exposure when run from an idle shell, and can't be fixed by unit dependencies. Triggering the automount explicitly, or reporting a clear error, would help.

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions