Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/pr.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ jobs:
chmod +x ./kind
sudo mv ./kind /usr/local/bin/kind
- name: Install containerlab
run: bash -c "$(curl -sL https://get.containerlab.dev)" -- -v 0.78.2
run: bash -c "$(curl -sL https://get.containerlab.dev)" -- -v 0.81.1
- name: Install chainsaw
uses: kyverno/action-install-chainsaw@v0.2.12

Expand Down
6 changes: 5 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,9 @@ TESTS := \
10-bmcversion \
12-force-power-off \
11-bmcuser \
13-event-subscriptions
13-event-subscriptions \
14-biosversion-upgrade \
15-bmcversion-upgrade

.PHONY: help
help: ## Show available targets
Expand All @@ -45,6 +47,8 @@ help: ## Show available targets
@echo " test/12-force-power-off (forced power off after a graceful shutdown is ignored)"
@echo " test/11-bmcuser (BMCUser, BMC account create/rotate/delete)"
@echo " test/13-event-subscriptions (Redfish event subscriptions via metal-maintenance-operator telemetry)"
@echo " test/14-biosversion-upgrade (BIOSVersion, BIOS firmware upgrade and downgrade)"
@echo " test/15-bmcversion-upgrade (BMCVersion, BMC firmware upgrade and downgrade)"
@echo ""

.PHONY: require-values
Expand Down
10 changes: 10 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,16 @@ biosVersion: "1.0.0"
testMessageId: "TestEventId" # a MessageId your BMC accepts in SubmitTestEvent, e.g. "SYS1000" on Dell iDRAC
testSeverity: "OK"
testOriginOfCondition: "/redfish/v1/Managers/1"
14-biosversion-upgrade:
transferProtocol: HTTP
baselineImageURI: "http://repo.example.com/bios-1.0.0.bin" # The URL to the current BIOS image. Will be flashed at least once during the test
version: "1.1.0"
imageURI: "http://repo.example.com/bios-1.1.0.bin" # The URL to the new BIOS version that will be flashed during the test
15-bmcversion-upgrade:
transferProtocol: HTTP
baselineImageURI: "http://repo.example.com/bmc-1.0.0.bin" # The URL to the current BMC firmware image. Will be flashed at least once during the test
version: "1.1.0"
imageURI: "http://repo.example.com/bmc-1.1.0.bin" # The URL to the new BMC firmware image
```

Adjust the credentials and expectations to your respective values and point the tests to it:
Expand Down
4 changes: 2 additions & 2 deletions capability-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,9 +76,9 @@ Which Chainsaw tests cover each capability is tracked in the [coverage document]

| Capability id | `BMC` method | Level | Redfish API | Redfish request body |
|--------------------------|-----------------------------------|:--------:|-----------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------|
| `firmware.bios.upgrade` | `UpgradeBiosVersion` | OPTIONAL | `POST` `#UpdateService.SimpleUpdate` | `{"ImageURI": "<uri>", "TransferProtocol": "<proto>", "Targets": ["<bios-inventory-uri>"], "@Redfish.OperationApplyTime": "Immediate"}` |
| `firmware.bios.upgrade` | `UpgradeBiosVersion` | OPTIONAL | `POST` `#UpdateService.SimpleUpdate` | `{"ImageURI": "<uri>", "TransferProtocol": "<proto>"}` - see below |
| `firmware.bios.task` | `GetBiosUpgradeTask` | OPTIONAL | `GET` task monitor / `TaskService` `Task` (`TaskState`, `TaskStatus`) | - |
| `firmware.bmc.upgrade` | `UpgradeBMCVersion` | OPTIONAL | `POST` `#UpdateService.SimpleUpdate` | `{"ImageURI": "<uri>", "TransferProtocol": "<proto>", "Targets": ["<bmc-inventory-uri>"], "@Redfish.OperationApplyTime": "Immediate"}` |
| `firmware.bmc.upgrade` | `UpgradeBMCVersion` | OPTIONAL | `POST` `#UpdateService.SimpleUpdate` | `{"ImageURI": "<uri>", "TransferProtocol": "<proto>"}` - see below |
| `firmware.bmc.task` | `GetBMCUpgradeTask` | OPTIONAL | `GET` task monitor / `Task` | - |
| `firmware.pending-check` | `CheckBMCPendingComponentUpgrade` | OPTIONAL | `GET` `UpdateService/FirmwareInventory` | - |

Expand Down
86 changes: 44 additions & 42 deletions coverage.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,51 +34,53 @@ These rows/capabilities include the following placeholders in their `test` field
| `tests/11-bmcuser` | `BMC Registration`, `account.service`, `account.list`, `account.create-update`, `account.delete` |
| `tests/12-force-power-off` | `BMC Registration`, `Discovery`, `power.on`, `power.off-graceful`, `power.off-force`, `power.wait-state`, `boot.override-pxe` |
| `tests/13-event-subscriptions` | `BMC Registration`, `events.subscribe`, `events.unsubscribe`, `events.list`, `events.test-event`, `events.delivery` |
| `tests/14-biosversion-upgrade` | `BMC Registration`, `Discovery`, `bios.version`, `manager.reset`, `firmware.pending-check`, `firmware.bios.upgrade`, `firmware.bios.task` |
| `tests/15-bmcversion-upgrade` | `BMC Registration`, `Discovery`, `bmc.version`, `manager.reset`, `firmware.pending-check`, `firmware.bmc.upgrade`, `firmware.bmc.task` |

## Coverage by capability

| Capability ID | Covered by | Verified |
|------------------------------|-------------------------------------------------------------------------------|:--------:|
| `power.on` | `Discovery` | yes |
| `power.off-graceful` | `Discovery` | yes |
| `power.off-force` | `tests/12-force-power-off`, `tests/03-power-annotation` See **[0]** | yes |
| `power.reset` | `tests/03-power-annotation` | no |
| `power.wait-state` | `Discovery` | yes |
| `boot.override-pxe` | `Discovery` | yes |
| `boot.order-get` | `tests/09-persistent-boot-order` | yes |
| `boot.order-set` | `tests/09-persistent-boot-order` | yes |
| `inventory.systems` | `BMC Registration` | yes |
| `inventory.system-info` | `BMC Registration` | yes |
| `inventory.processors` | `Discovery` | yes |
| `inventory.storages` | `Discovery` | no |
| `bios.version` | `Discovery`, `tests/06-biossettings-noreboot`, `tests/07-biossettings-reboot` | yes |
| `bios.attr-get` | `tests/06-biossettings-noreboot`, `tests/07-biossettings-reboot` | yes |
| `bios.attr-pending` | `tests/06-biossettings-noreboot`, `tests/07-biossettings-reboot` | yes |
| `bios.attr-set-on-reset` | `tests/06-biossettings-noreboot`, `tests/07-biossettings-reboot` | yes |
| `bios.attr-check` | `tests/06-biossettings-noreboot`, `tests/07-biossettings-reboot` | yes |
| `bmc.version` | `tests/10-bmcversion` | yes |
| `bmc-settings.attr-get` | `tests/08-bmcsettings` | yes |
| `bmc-settings.attr-pending` | `tests/08-bmcsettings` | no |
| `bmc-settings.set-immediate` | `tests/08-bmcsettings` | yes |
| `bmc-settings.attr-check` | `tests/08-bmcsettings` | no |
| `firmware.bios.upgrade` | uncovered | - |
| `firmware.bios.task` | uncovered | - |
| `firmware.bmc.upgrade` | uncovered | - |
| `firmware.bmc.task` | uncovered | - |
| `firmware.pending-check` | uncovered | - |
| `manager.get` | `BMC Registration` | yes |
| `manager.discover` | uncovered | - |
| `manager.reset` | `tests/04-bmc-reset` | no |
| `account.create-update` | `tests/11-bmcuser` | yes |
| `account.delete` | `tests/11-bmcuser` | yes |
| `account.list` | `tests/11-bmcuser` | yes |
| `account.service` | `tests/11-bmcuser` | yes |
| `indicator.set-led` | `tests/05-indicator-led` | yes |
| `events.subscribe` | `tests/13-event-subscriptions` | yes |
| `events.unsubscribe` | `tests/13-event-subscriptions` | yes |
| `events.list` | `tests/13-event-subscriptions` | yes |
| `events.test-event` | `tests/13-event-subscriptions` | yes |
| `events.delivery` | `tests/13-event-subscriptions` | yes |
| Capability ID | Covered by | Verified |
|------------------------------|---------------------------------------------------------------------------------------------------------------|:--------:|
| `power.on` | `Discovery` | yes |
| `power.off-graceful` | `Discovery` | yes |
| `power.off-force` | `tests/12-force-power-off`, `tests/03-power-annotation` See **[0]** | yes |
| `power.reset` | `tests/03-power-annotation` | no |
| `power.wait-state` | `Discovery` | yes |
| `boot.override-pxe` | `Discovery` | yes |
| `boot.order-get` | `tests/09-persistent-boot-order` | yes |
| `boot.order-set` | `tests/09-persistent-boot-order` | yes |
| `inventory.systems` | `BMC Registration` | yes |
| `inventory.system-info` | `BMC Registration` | yes |
| `inventory.processors` | `Discovery` | yes |
| `inventory.storages` | `Discovery` | no |
| `bios.version` | `Discovery`, `tests/06-biossettings-noreboot`, `tests/07-biossettings-reboot`, `tests/14-biosversion-upgrade` | yes |
| `bios.attr-get` | `tests/06-biossettings-noreboot`, `tests/07-biossettings-reboot` | yes |
| `bios.attr-pending` | `tests/06-biossettings-noreboot`, `tests/07-biossettings-reboot` | yes |
| `bios.attr-set-on-reset` | `tests/06-biossettings-noreboot`, `tests/07-biossettings-reboot` | yes |
| `bios.attr-check` | `tests/06-biossettings-noreboot`, `tests/07-biossettings-reboot` | yes |
| `bmc.version` | `tests/10-bmcversion`, `tests/15-bmcversion-upgrade` | yes |
| `bmc-settings.attr-get` | `tests/08-bmcsettings` | yes |
| `bmc-settings.attr-pending` | `tests/08-bmcsettings` | no |
| `bmc-settings.set-immediate` | `tests/08-bmcsettings` | yes |
| `bmc-settings.attr-check` | `tests/08-bmcsettings` | no |
| `firmware.bios.upgrade` | `tests/14-biosversion-upgrade` | yes |
| `firmware.bios.task` | `tests/14-biosversion-upgrade` | no |
| `firmware.bmc.upgrade` | `tests/15-bmcversion-upgrade` | yes |
| `firmware.bmc.task` | `tests/15-bmcversion-upgrade` | no |
| `firmware.pending-check` | `tests/14-biosversion-upgrade`, `tests/15-bmcversion-upgrade` | no |
| `manager.get` | `BMC Registration` | yes |
| `manager.discover` | uncovered | - |
| `manager.reset` | `tests/04-bmc-reset`, `tests/14-biosversion-upgrade`, `tests/15-bmcversion-upgrade` | no |
| `account.create-update` | `tests/11-bmcuser` | yes |
| `account.delete` | `tests/11-bmcuser` | yes |
| `account.list` | `tests/11-bmcuser` | yes |
| `account.service` | `tests/11-bmcuser` | yes |
| `indicator.set-led` | `tests/05-indicator-led` | yes |
| `events.subscribe` | `tests/13-event-subscriptions` | yes |
| `events.unsubscribe` | `tests/13-event-subscriptions` | yes |
| `events.list` | `tests/13-event-subscriptions` | yes |
| `events.test-event` | `tests/13-event-subscriptions` | yes |
| `events.delivery` | `tests/13-event-subscriptions` | yes |

#### Footnotes
**[0]**: The test is using the `Reset`-method of the `BMC` interface, but the result is the same capability being tested as using the `ForcePowerOff`-method.
18 changes: 18 additions & 0 deletions tests/14-biosversion-upgrade/biosversion.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
apiVersion: system.metal.ironcore.dev/v1alpha1
kind: BIOSVersion
metadata:
name: (join('-', [$bmcName, $role]))
annotations:
# Set so chainsaw can delete the resoruce when a ServerMaintenance is still
# active, after a potential failed upgrade.
metal.ironcore.dev/operation: allow-in-progress-delete
spec:
serverRef:
name: ($serverName)
version: ($version)
image:
URI: ($imageURI)
transferProtocol: ($values."14-biosversion-upgrade".transferProtocol)
serverMaintenancePolicy: Enforced
retryPolicy:
maxAttempts: 0
Loading
Loading