[matthewgrossman]

Summary

A worked example of a test that needs a specific deployment feature, Keycloak. tmachine declares an environment that runs Keycloak and publishes how to reach it as OPENSHELL_TEST_* variables. The runner passes those variables to the tests, and a suite-local fixture reads them. The Keycloak provider-refresh test is ported to testkit on top of this.

Stacked on #4773. Draft for discussion.

Related Issue

Part of #3954

How it fits together

1. tmachine declares the environment. ubuntu-docker-rootful-keycloak in tests/tmachine/config.nix runs the usual Docker setup plus ansible/playbooks/keycloak.yaml. That playbook starts Keycloak with --restart unless-stopped and writes /etc/openshell-test/env.d/keycloak.env. Setup is cached as a disk image, so the Keycloak image is cached too, and the container comes back on every boot.
2. The runner passes the variables through. After boot, test_env.rs reads every /etc/openshell-test/env.d/*.env file and rejects names outside the OPENSHELL_TEST_ prefix (errors never print values). guest_env adds the variables to the testkit-ready and guest nextest commands. The runner has no idea what Keycloak is.
3. The test helper reads them. KeycloakContext::new(cx).await reads OPENSHELL_TEST_KEYCLOAK_{ISSUER,CLIENT_ID,USERNAME,PASSWORD} and waits for the realm to answer. Set the same variables to run the suite against your own deployment.

Run it locally:

nix run .#openshell-test-runner -- \
  --environment ubuntu-docker-rootful-keycloak --installer deb \
  --family features --package keycloak

Combinations such as Podman + Keycloak are more tmachine environments. They can be generated as a cross product later without changing the runner or the tests.

Changes

• testkit: Cli::run_with_env and Context::cli_with_env, so secrets reach the CLI through env vars rather than argv.
• tmachine: the ubuntu-docker-rootful-keycloak environment. Environment setup can now take inputs, used for the realm file; environments without inputs keep their cache keys. The keycloak role adds a restart policy.
• Runner: test_env.rs, and guest_env takes the published variables.
• Keycloak fixture (tests/suites/features/keycloak): KeycloakContext derefs to testkit::Context and adds password_grant(), revoke() and token_endpoint(). A missing variable fails the test.
• Ported test: provider_refresh.rs, with the same CLI calls and checks as the old one and run-unique provider and profile names.
• Lockfile: tests/suites/features/Cargo.lock is aligned with the root lock and added to the alignment check.
• CI: an E2E Suites matrix row for features/keycloak on the new environment; the tmachine disk cache key includes the realm file.
• Docs: an "Environments with services" paragraph in TESTING.md.

The old tests/suites/features/provider-refresh/keycloak crate and its tmachine testsuite are unchanged.

Open questions

• Env vars versus a config file the fixture reads directly, and the exact variable names. For now tests read env vars only; letting testkit read env.d itself is deferred.
• The variables, including the throwaway realm password, reach the guest as env arguments over SSH.
• Provider and profile cleanup runs whether the test passes or fails, but a panic or a killed process skips it.

Testing

• Runner, tmachine, testkit and features workspace unit tests and clippy pass
• Lockfile alignment check and its tests pass
• cargo nextest list -p keycloak lists keycloak::provider_refresh (host target)
• The Keycloak environment runs green in a tmachine VM (Branch E2E)

Checklist

• Follows Conventional Commits
• Commits are signed off (DCO)

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[github-actions]

Label test:e2e applied, but pull-request/4891 does not exist yet. A maintainer needs to comment /ok to test df2bcbd332a27e575dba7413b368b82abbd0fdc3 to mirror this PR. Once the mirror exists, re-apply the label or re-run Branch E2E Checks from the Actions tab.

[matthewgrossman]

/ok to test df2bcbd