Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Breaking Changes

* **role:gitlab_ce**: `gitlab-dump.service` runs `gitlab-backup create STRATEGY=copy`, as the GitLab documentation recommends for a running instance, so the dump no longer fails with "file changed as we read it". GitLab copies each of its file stores (uploads, CI artifacts, LFS objects, the container registry, ...) to the backup path before archiving it, one after the other, so `gitlab_ce__rb_gitlab_rails_backup_path` needs free space for the largest of them on top of the archive. Grow the filesystem of the backup path if necessary.
* **role:sshd, playbook:setup_basic**: sshd turns new connections away earlier while many logins are pending (`MaxStartups 10:30:60` instead of the OpenSSH default `10:30:100`), as the CIS Benchmarks require, and `MaxStartups` is configurable with `sshd__max_startups__host_var` / `sshd__max_startups__group_var`. A `MaxStartups` set through `sshd__raw` no longer takes effect, move it to the variable. Hosts with many simultaneous SSH logins need a higher value, for example `'100:30:200'` on GitLab hosts ([#723](https://github.com/Linuxfabrik/lfops/issues/723)).
* **role:grav, playbook:setup_grav**: The code of Grav, its configuration, pages, plugins and themes belong to `root`, so a hole in Grav or a plugin can no longer plant code in the site. The web server only writes `assets`, `backup`, `cache`, `images`, `logs`, `tmp`, `user/accounts` and `user/data`, and only these keep the SELinux type `httpd_sys_rw_content_t`. Grav, its plugins and themes are updated as `root` by `grav-selfupgrade-<instance>.timer` and `grav-update-<instance>.timer`, which replace `grav-selfupgrade.timer` and `grav-update.timer`, so several Grav sites on one host are each updated. Run other `bin/gpm` commands with `grav-gpm <installation directory> <arguments>` instead of `sudo -u apache bin/gpm`, which now fails, and deploy content below `user/` as `root`. The role no longer installs the Admin Panel, which could not save anything anymore, and drops `grav__skip_admin` and `grav__users`; it lists the Admin Panel plugins still installed at the end of the play, for you to remove. The vHost no longer accepts `POST`, and `setup_grav` no longer sets the SELinux boolean `httpd_can_network_connect`, which keeps its current value.
* **role:apache_httpd**: The `proxy` vHost template waits 30 instead of 5 seconds for the backend, so slower responses no longer end in a 502, and a hanging backend holds the connection longer. Set `conf_proxy_timeout: 5` on the vHost to keep the previous value.
Expand Down Expand Up @@ -82,6 +83,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Added

* **role:fangfrisch**: `fangfrisch__urlhaus_auth_key` sets the abuse.ch Auth-Key and enables the URLhaus signatures.
* **role:mount**: The `mode` subkey of `mount__mounts__*_var` sets the mode of the root of a mounted filesystem.
* **role:php**: `php__opcache_blacklist__host_var` / `php__opcache_blacklist__group_var` exclude files from OPcache, and the role creates the blacklist file that `opcache.blacklist_filename` already pointed to.
* **role:libreoffice**: `libreoffice__client_apache: true` also works on Debian and Ubuntu.
* **plugin:bitwarden_item, module:bitwarden_item**: Reach `bw serve` through a Unix socket instead of the loopback port, so its file permissions decide which local users get at the vault. Needs bw CLI 2025.9.0 or newer. Set `LFOPS_BITWARDEN_UNIX_SOCKET` to the path of the socket.
Expand Down Expand Up @@ -251,6 +254,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Fixed

* **role:php**: On the Red Hat family, PHP modules from the inventory that are already installed as a dependency of another package are marked as user-installed, so `dnf autoremove` no longer removes them once that package is gone.
* **role:sshd**: `sshd__sftp_subsystem` from the inventory takes effect, where the role always deployed its own default.
* **role:fangfrisch**: Without `fangfrisch__urlhaus_auth_key`, Fangfrisch no longer tries to download the URLhaus signatures, which failed with `401 Unauthorized` on every refresh.
* **role:wordpress, playbook:setup_wordpress**: Installs `php-mysqlnd` instead of `php-pecl-mysql`, the obsolete `mysql_*` extension that WordPress does not use, and removes `php-pecl-mysql` from existing hosts. If another application on the host still needs it, add `php-pecl-mysql` with `state: 'present'` to `php__modules__host_var`.
* **role:icinga2_agent**: The run fails if the host cannot be added to the Icinga Director, for example because of a wrong login, instead of continuing as if it had been added.
* **role:mount**: The role no longer resets the owner and mode of the root of a mounted filesystem to `root:root` and `0755` on every run, and no longer fails on an NFS export with `root_squash`. `owner` and `group` are only applied where a mount sets them, and reach the mounted filesystem in the first run.
* **role:gitlab_ce**: `gitlab-dump.service` fails when the backup fails, so `systemctl --failed` and the monitoring notice it, instead of always reporting success.
* **role:duplicity**: `duba` no longer removes old backups of a path whose backup failed in the same run.
* **role:duplicity**: `duba` reports a missing or invalid `/etc/duba/duba.json` with an error message and exit code 201 or 203 instead of a Python traceback.
* **role:grav, playbook:setup_grav**: OPcache no longer fills up with recompiled copies of Grav's file list caches and no longer throws away its whole cache every few days.
* **role:repo_epel**: On Rocky 10 and RHEL 10, EPEL follows the minor release of the host, so the daily system update no longer fails on EPEL packages built for a RHEL minor release the host does not have yet.
* **role:mariadb_server**: With `skip_name_resolve`, the `proxies_priv` warning in the MariaDB error log also goes away on servers that carry the entry for `root` instead of `mysql`.
Expand Down
42 changes: 41 additions & 1 deletion extensions/molecule/duplicity/verify.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,37 @@
that: '"usage: duba" in __molecule__duba_help_result["stdout"]'
fail_msg: '{{ __molecule__duba_help_result | to_json }}'

# A missing or invalid config file is reported with an error message and duba's own exit
# code, not with a Python traceback (exit code 222).
- name: 'duba --config=/nonexistent'
ansible.builtin.command: '/usr/local/bin/duba --config=/nonexistent'
changed_when: false
failed_when: false
register: '__molecule__duba_missing_config_result'

- name: 'Deploy /tmp/molecule-duba-invalid.json'
ansible.builtin.copy:
dest: '/tmp/molecule-duba-invalid.json'
content: '{invalid'
mode: 0o600

- name: 'duba --config=/tmp/molecule-duba-invalid.json'
ansible.builtin.command: '/usr/local/bin/duba --config=/tmp/molecule-duba-invalid.json'
changed_when: false
failed_when: false
register: '__molecule__duba_invalid_config_result'

- name: 'Assert that duba reports a missing and an invalid config file'
ansible.builtin.assert:
that:
- '__molecule__duba_missing_config_result["rc"] == 201'
- '"ERROR: duba - I/O error: No such file or directory while opening or reading /nonexistent" in __molecule__duba_missing_config_result["stderr"]'
- '__molecule__duba_invalid_config_result["rc"] == 203'
- '"ERROR: duba - JSON decode error in /tmp/molecule-duba-invalid.json" in __molecule__duba_invalid_config_result["stderr"]'
- '"Traceback" not in __molecule__duba_missing_config_result["stdout"] ~ __molecule__duba_missing_config_result["stderr"]'
- '"Traceback" not in __molecule__duba_invalid_config_result["stdout"] ~ __molecule__duba_invalid_config_result["stderr"]'
fail_msg: '{{ [__molecule__duba_missing_config_result, __molecule__duba_invalid_config_result] | to_json }}'

# Ask gpg whether the master key is imported and trusted ultimately ("u" in field 2 of the
# colon listing), since the role encrypts every backup to it.
- name: 'gpg --list-keys --with-colons {{ duplicity__gpg_encrypt_master_key }}'
Expand Down Expand Up @@ -350,7 +381,8 @@

# One failed source has to fail the whole run, also if the other sources finish after it.
# A regular file in place of the backup directory of the divided entry "broken" breaks its
# backup, the other sources still back up fine.
# backup, the other sources still back up fine. duba must not remove old backups of the
# path whose backup failed, while it still cleans up the others.
- name: 'mkdir /srv/duba-molecule/divided/broken'
ansible.builtin.file:
path: '/srv/duba-molecule/divided/broken'
Expand Down Expand Up @@ -381,6 +413,14 @@
- '__molecule__duba_journal_result["stdout_lines"] | select("search", "ERROR: duba - Exit code [0-9]+: .* file:///var/backups/duba-molecule/srv/duba-molecule/divided/broken$") | list | length > 0'
fail_msg: '{{ __molecule__duba_journal_result["stdout_lines"] }}'

- name: 'Assert that duba skipped the cleanup of the broken source only'
ansible.builtin.assert:
that:
- '__molecule__duba_journal_result["stdout_lines"] | select("search", "WARNING: duba - Skipping the cleanup of /srv/duba-molecule/divided/broken, since its backup failed.$") | list | length > 0'
- '__molecule__duba_journal_result["stdout_lines"] | select("search", "Running: .* remove-older-than .* file:///var/backups/duba-molecule/srv/duba-molecule/divided/broken$") | list | length == 0'
- '__molecule__duba_journal_result["stdout_lines"] | select("search", "Running: .* remove-older-than .* file:///var/backups/duba-molecule/srv/duba-molecule/plain$") | list | length > 0'
fail_msg: '{{ __molecule__duba_journal_result["stdout_lines"] }}'

- name: 'rm -rf /srv/duba-molecule/divided/broken /var/backups/duba-molecule/srv/duba-molecule/divided/broken'
ansible.builtin.file:
path: '{{ item }}'
Expand Down
72 changes: 71 additions & 1 deletion extensions/molecule/gitlab_ce/verify.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
# Checks that GitLab runs and that its bundled NGINX uses the settings of gitlab.rb: the
# listen port, the registry and the real IP of a request from a trusted proxy.
# listen port, the registry and the real IP of a request from a trusted proxy. Also checks that
# gitlab-dump.service writes a backup and fails when the dump fails.
- name: 'Verify GitLab CE'
hosts: 'systems_under_test'
gather_facts: false
Expand Down Expand Up @@ -69,3 +70,72 @@
retries: 5
delay: 2
changed_when: false

# A dump of the running instance has to succeed and leave an archive in the backup path.
- name: 'systemctl start gitlab-dump.service' # noqa command-instead-of-module (a oneshot run, not a state)
ansible.builtin.command: 'systemctl start gitlab-dump.service'
changed_when: false
failed_when: false
register: '__molecule__gitlab_dump_result'

- name: 'journalctl --unit=gitlab-dump.service --lines=60'
ansible.builtin.command: 'journalctl --unit=gitlab-dump.service --no-pager --lines=60'
changed_when: false
register: '__molecule__gitlab_dump_journal_result'

- name: 'find /backup/gitlab -name *_gitlab_backup.tar'
ansible.builtin.find:
paths: '/backup/gitlab'
patterns: '*_gitlab_backup.tar'
register: '__molecule__gitlab_dump_find_result'

- name: 'Assert that the dump succeeded and wrote an archive'
ansible.builtin.assert:
that:
- '__molecule__gitlab_dump_result["rc"] == 0'
- '__molecule__gitlab_dump_find_result["matched"] > 0'
fail_msg: '{{ __molecule__gitlab_dump_journal_result["stdout_lines"] }}'

# Without PostgreSQL the database dump fails, and the unit has to fail with it, so that
# `systemctl --failed` and the monitoring notice a broken backup.
- name: 'gitlab-ctl stop postgresql'
ansible.builtin.command: 'gitlab-ctl stop postgresql'
changed_when: false

- name: 'systemctl start gitlab-dump.service # without PostgreSQL' # noqa command-instead-of-module (a oneshot run, not a state)
ansible.builtin.command: 'systemctl start gitlab-dump.service'
changed_when: false
failed_when: false
register: '__molecule__gitlab_dump_failed_result'

- name: 'systemctl is-failed gitlab-dump.service' # noqa command-instead-of-module (read-only state query)
ansible.builtin.command: 'systemctl is-failed gitlab-dump.service'
changed_when: false
failed_when: false
register: '__molecule__gitlab_dump_is_failed_result'

- name: 'gitlab-ctl start postgresql'
ansible.builtin.command: 'gitlab-ctl start postgresql'
changed_when: false

- name: 'systemctl reset-failed gitlab-dump.service' # noqa command-instead-of-module (clears the state the test left behind)
ansible.builtin.command: 'systemctl reset-failed gitlab-dump.service'
changed_when: false
failed_when: false

- name: 'Assert that gitlab-dump.service failed without PostgreSQL'
ansible.builtin.assert:
that:
- '__molecule__gitlab_dump_failed_result["rc"] != 0'
- '__molecule__gitlab_dump_is_failed_result["stdout"] == "failed"'
fail_msg: '{{ [__molecule__gitlab_dump_failed_result, __molecule__gitlab_dump_is_failed_result] | to_json }}'

# the following verify run and the idempotence step need a GitLab that answers again
- name: 'curl http://127.0.0.1:8090/users/sign_in (after the PostgreSQL restart)'
ansible.builtin.uri:
url: 'http://127.0.0.1:8090/users/sign_in'
status_code: 200
register: '__molecule__gitlab_sign_in_after_result'
until: '__molecule__gitlab_sign_in_after_result is not failed'
retries: 30
delay: 10
25 changes: 25 additions & 0 deletions extensions/molecule/mount/converge.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,31 @@
creates: '/var/tmp/lfops-molecule.img'
executable: '/bin/bash'

# the filesystem of the mount that sets owner, group and mode in group_vars
- name: 'truncate --size=300M /var/tmp/lfops-molecule-owned.img; mkfs.xfs /var/tmp/lfops-molecule-owned.img'
ansible.builtin.shell:
cmd: |
set -o errexit
truncate --size=300M /var/tmp/lfops-molecule-owned.img
mkfs.xfs -q /var/tmp/lfops-molecule-owned.img
creates: '/var/tmp/lfops-molecule-owned.img'
executable: '/bin/bash'


- name: 'Converge mount playbook'
ansible.builtin.import_playbook: 'linuxfabrik.lfops.mount'


# What the admin of a share does: give its root an owner and a mode of its own. The mount sets
# neither in group_vars, so the role has to leave both alone. In the idempotence step the role
# runs before this play again, and would report a change if it reset them.
- name: 'Change owner and mode of the root of the mounted filesystem'
hosts: 'systems_under_test'
gather_facts: false
tasks:

- name: 'chown nobody /srv/lfops-molecule; chmod 1777 /srv/lfops-molecule'
ansible.builtin.file:
path: '/srv/lfops-molecule'
owner: 'nobody'
mode: 0o1777
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,12 @@ mount__mounts__group_var:
fstype: 'xfs'
opts: 'loop'
state: 'mounted'
# owner, group and mode from the inventory reach the root of the mounted filesystem
- path: '/srv/lfops-molecule-owned'
src: '/var/tmp/lfops-molecule-owned.img'
fstype: 'xfs'
opts: 'loop'
owner: 'nobody'
group: 'daemon'
mode: '0750'
state: 'mounted'
24 changes: 23 additions & 1 deletion extensions/molecule/mount/verify.yml
Original file line number Diff line number Diff line change
@@ -1,8 +1,30 @@
- name: 'Verify mount relabels a freshly created filesystem'
- name: 'Verify mount relabels a freshly created filesystem and only sets the owner and mode it is given'
hosts: 'systems_under_test'
gather_facts: false
tasks:

# Both paths have to be mount points, otherwise the checks below look at plain directories.
- name: 'findmnt --noheadings --output=TARGET /srv/lfops-molecule; findmnt ... /srv/lfops-molecule-owned'
ansible.builtin.command: 'findmnt --noheadings --output=TARGET {{ item }}'
loop:
- '/srv/lfops-molecule'
- '/srv/lfops-molecule-owned'
changed_when: false

- name: 'stat --format="%U %G %a" /srv/lfops-molecule /srv/lfops-molecule-owned'
ansible.builtin.command: 'stat --format="%U %G %a" /srv/lfops-molecule /srv/lfops-molecule-owned'
changed_when: false
register: '__molecule__owner_mode_result'

# The first mount sets no owner, group or mode, so it keeps what converge.yml gave the root
# of its filesystem after mounting. The second one gets what group_vars sets.
- name: 'Assert the owner, group and mode of the roots of the mounted filesystems'
ansible.builtin.assert:
that:
- '__molecule__owner_mode_result["stdout_lines"][0] == "nobody root 1777"'
- '__molecule__owner_mode_result["stdout_lines"][1] == "nobody daemon 750"'
fail_msg: '{{ __molecule__owner_mode_result["stdout_lines"] }}'

# What the relabelling is for: the mount point and what is created on it carry the context
# the policy assigns to the path, not unlabeled_t, which confined services may not access.
- name: 'touch /srv/lfops-molecule/probe'
Expand Down
21 changes: 21 additions & 0 deletions extensions/molecule/setup_wordpress/converge.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,27 @@
- name: 'Converge setup_wordpress playbook'
ansible.builtin.import_playbook: 'linuxfabrik.lfops.setup_wordpress'

# What a host installed by an older release looks like: `php-mysql` resolved to php-pecl-mysql,
# which pulled in php-mysqlnd as a dependency. The following plays have to migrate it. Only once,
# so that the idempotence run does not start over.
- name: 'Put the PHP MySQL packages into the state an older release left behind'
hosts: 'systems_under_test'
become: true
tasks:

- name: 'dnf install php-mysql; dnf mark remove php-mysqlnd'
ansible.builtin.shell: |-
set -o errexit
dnf --assumeyes install php-mysql
dnf mark remove php-mysqlnd
rpm --query php-pecl-mysql
touch /var/lib/lfops-molecule-legacy-php-mysql
args:
creates: '/var/lib/lfops-molecule-legacy-php-mysql'
when:
- 'ansible_facts["os_family"] == "RedHat"'


# Further instances on the same host, as pseudo hosts in the inventory would deploy them: the
# same playbook again, with the variables of the instance overriding those from group_vars.
# Together with the instance at the root of wordpress.example.com they cover an instance in a
Expand Down
Loading
Loading