feat: add support for ephemeral-encryption-keys

[piyush-jena]

Description of changes:

• Adds support for ephemeral encryption keys. The encryption keys are placed in tmpfs (/run) instead of the keystore and deleted after unlock.
• As part of this feature, we encrypt the private partition with LUKS2 cipher. Because the encryption key is temporary, we format the partition every boot (with the same parameters) and re-encrypt every boot.
• The datastore inside of private partition is further encrypted with fscrypt. This requires an additional fs feature to be enabled (encrypt) which we pass in the previous step (or using tune2fs in case of data partition).
• Data Partition is formatted with the default parameters which is why we use systemd-makefs. We force the reformat by running wipefs first. This deletes all fs related signatures tricking systemd-makefs into reformatting and avoiding the additional overhead of writing 0s to the entire partition.
• Additionally we use image features environment variables in various places to choose keystore path or trick early-boot-config into populating the datastore every boot (because we format the private partition every boot).

Testing done:

• The tmpfs keystore has SELinux label private_t

bash-5.2# ls -lZ /run/rottweiler
total 0
bash-5.2# ls -lZ /run | grep rottweiler
drwx------.  2 root   root   system_u:object_r:private_t:s0     40 Aug 12 16:22 rottweiler

• Verified the new keystore doesn't contain the keys.

[root@admin]# sheltie
bash-5.2# ls -lia /run/rottweiler/
total 0
918 drwx------.  2 root root  40 Aug 10 23:13 .
  1 drwxr-xr-x. 21 root root 520 Aug 10 23:13 ..

• Verified the filesystems and the directories are encrypted.

bash-5.2# lsblk -o NAME,TYPE,FSTYPE,MOUNTPOINT
NAME                     TYPE  FSTYPE         MOUNTPOINT
nvme1n1                  disk
`-nvme1n1p1              part  crypto_LUKS
  `-BOTTLEROCKET-DATA    crypt xfs            /local
nvme0n1                  disk
|-nvme0n1p1              part
|-nvme0n1p2              part  vfat
|-nvme0n1p3              part  vfat           /boot
|-nvme0n1p4              part  erofs
|-nvme0n1p5              part  DM_verity_hash
|-nvme0n1p6              part
|-nvme0n1p7              part  crypto_LUKS
| `-BOTTLEROCKET-PRIVATE crypt ext4           /var/lib/bottlerocket
`-nvme0n1p8              part
bash-5.2# rottweiler check directory /.bottlerocket/datastore encrypted
directory '/.bottlerocket/datastore' is encrypted

• Reboot (with user-data) works - used bootstrap commands as user data to change settings.motd and in every boot its the updated value
  user-data

[settings.bootstrap-commands.mybootstrap]
commands = [["apiclient", "set", "settings.motd=hello"]]
essential = true
mode = "always"

output:

bash-5.2# apiclient get settings.motd
{
  "settings": {
    "motd": "hello"
  }
}

A second consequence of reformatting data partition is that the journal for previous boot is lost which was also checked.

• Moving the EBS volume to a different instance makes it unreadable
• lockdown mode works

bash-5.2# apiclient get settings.motd
{
  "settings": {
    "motd": "hello"
  }
}
bash-5.2# apiclient lockdown
01:21:33 [INFO] Lockdown completed
bash-5.2# apiclient get settings.motd
Failed to get settings: Failed GET request to '/?prefix=settings.motd': Status 500 when GETing /?prefix=settings.motd: Data store error during get_prefix 'settings.motd' for Live: Data store integrity violation at /var/lib/bottlerocket/datastore/current/live: Live datastore missing

• (A)I went through journal and it didn't find any errors during all this experiments
  Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

[ginglis13]

Suggested change

	Before=encrypt-datastore.service encrypt-localfs.service encrypt-privatefs.service
	Before=encrypt-datastore.service encrypt-local-fs.service encrypt-private-fs.service

[vigh-m]

Similar error in the [Install] section below

[ginglis13]

The comment doesn't match the logic here -

We do not write the marker file in case ephemeral encryption keys feature is enabled.

if is_ephemeral_encryption_keys_enabled -> write the file

[ginglis13]

This file doesn't appear to be used

[vigh-m]

⬆️ Updated following the comment and retested:

[root@admin]# sheltie
bash-5.2# ls -lZ /run/rottweiler
total 0
bash-5.2# ls -lZ /run | grep rottweiler
drwx------.  2 root   root   system_u:object_r:private_t:s0     40 Aug 15 04:17 rottweiler
bash-5.2# ls -lia /run/rottweiler/
total 0
1 drwx------.  2 root root  40 Aug 15 04:17 .
1 drwxr-xr-x. 21 root root 500 Aug 15 04:17 ..
bash-5.2# lsblk -o NAME,TYPE,FSTYPE,MOUNTPOINT
NAME                     TYPE  FSTYPE         MOUNTPOINT
nvme1n1                  disk
`-nvme1n1p1              part  crypto_LUKS
  `-BOTTLEROCKET-DATA    crypt xfs            /local
nvme0n1                  disk
|-nvme0n1p1              part
|-nvme0n1p2              part  vfat
|-nvme0n1p3              part  vfat           /boot
|-nvme0n1p4              part  erofs
|-nvme0n1p5              part  DM_verity_hash
|-nvme0n1p6              part
|-nvme0n1p7              part  crypto_LUKS
| `-BOTTLEROCKET-PRIVATE crypt ext4           /var/lib/bottlerocket
`-nvme0n1p8              part
bash-5.2# apiclient lockdown
04:19:26 [INFO] Lockdown completed
bash-5.2# apiclient get settings.motd
Failed to get settings: Failed GET request to '/?prefix=settings.motd': Status 500 when GETing /?prefix=settings.motd: Data store error during get_prefix 'settings.motd' for Live: Data store integrity violation at /var/lib/bottlerocket/datastore/current/live: Live datastore missing
bash-5.2# systemctl --failed
  UNIT LOAD ACTIVE SUB DESCRIPTION

0 loaded units listed.