diff --git a/skills/maya-plugin-e2e/SKILL.md b/skills/maya-plugin-e2e/SKILL.md new file mode 100644 index 0000000..0f2ea45 --- /dev/null +++ b/skills/maya-plugin-e2e/SKILL.md @@ -0,0 +1,151 @@ +--- +name: maya-plugin-e2e +description: "Maya plugin UI/e2e proof workflow: use when testing Maya plugins through Maya Stall, Windows Maya hosts, pluginArtifacts, screenshots/recordings, Scenario Result JSON, untrusted-plugin popups, UI clicking, or PR closeout gates that need real Maya evidence." +--- + +# Maya Plugin E2E + +## Overview + +Run real Maya plugin proof without rediscovering the same traps. Prefer deterministic setup, trusted plugin staging, explicit UI actions, and evidence-backed closeout over manual smoke claims. + +## Fast Path + +1. Confirm target commit, CI artifact job, artifact size, and plugin hash before touching the live host. +2. Confirm the Maya Stall host is free, the run command, host config, and target profile. +3. Verify `pluginArtifacts` are declared by the scenario/project and the host config has a trusted plugin artifact root. +4. Start from a clean run workspace; do not reuse stale screenshots, recordings, or Scenario Results. +5. Run the real gate through `maya-stall run`, not manual Maya unless explicitly debugging. +6. As soon as Maya is expected to load the `.mll`, capture/inspect a screenshot or equivalent UI state before waiting for long scenario completion. +7. If that early checkpoint shows `Untrusted Plugin Loading`, stop the wait and classify trusted staging/host baseline as failed. +8. Inspect Scenario Result JSON first, then screenshots/recordings, logs, and saved scene. +9. Close only with exact-head proof: commit, CI artifact job/id/hash/size, run id, screenshot/recording paths/sizes, Scenario Result fields, confidentiality pass. + +## Preflight + +Use exact queries; do not print secrets or full host configs. + +```bash +maya-stall status +python3 - <<'PY' +import json, pathlib, zipfile, hashlib +artifact = pathlib.Path("artifacts.zip") +print("artifact:", artifact.resolve()) +print("artifact_size:", artifact.stat().st_size) +print("artifact_sha256:", hashlib.sha256(artifact.read_bytes()).hexdigest()) +with zipfile.ZipFile(artifact) as z: + for name in z.namelist(): + if name.lower().endswith((".mll", ".so", ".bundle", ".dll")): + data = z.read(name) + print("plugin:", name) + print("plugin_size:", len(data)) + print("plugin_sha256:", hashlib.sha256(data).hexdigest()) +PY +``` + +For Maya Stall host config, report only safe shape: + +```bash +python3 - <<'PY' +import json, pathlib +try: + import yaml +except Exception: + raise SystemExit("Install/read yaml another way; do not dump host config.") +p = pathlib.Path("/path/to/hosts.yaml") +data = yaml.safe_load(p.read_text()) +hosts = data.get("hosts") or [] +for i, h in enumerate(hosts): + safe = sorted(k for k in h if k.lower() not in {"host","hostname","user","username","identityfile","password","privatekey"}) + print(f"host[{i}] keys:", safe) + print(f"host[{i}] trustedPluginArtifactsRoot:", bool(h.get("trustedPluginArtifactsRoot"))) +PY +``` + +If `trustedPluginArtifactsRoot` is missing and the plugin loads from a transient workspace path, expect Maya's security modal. Fix trusted staging before rerun; clicking `Allow` is a diagnostic escape hatch, not accepted proof. + +## Running + +Use the project-provided command exactly when a coordinator gives one: + +```bash +/tmp/maya-stall-latest run --host-config /tmp/hosts.yaml --target-profile default --stop-after never product_ui_e2e +``` + +During a live run: + +- Do not start a second live run against a shared host. +- Watch for timeout text, Scenario Result path, run id, artifact bundle root, screenshot path, and recording path. +- Immediately after plugin load, or after the first log/event that means Maya should load the `.mll`, inspect a screenshot or equivalent UI state. +- If the early screenshot shows `Untrusted Plugin Loading`, stop waiting for scenario completion and fix trusted staging or host baseline before rerun. +- If the command times out, immediately inspect the captured failure screenshot before editing code. +- If the screenshot shows a Maya modal, identify the modal; do not assume the scenario is slow. +- If the screenshot shows no modal and Maya is active, inspect script logs and outputs for waiting loops, expensive meshes, missing file writes, or stuck UI callbacks. + +## UI Proof + +For behavior-changing plugin PRs, require real UI operations that would fail if controls or windows are broken: + +- Create representative geometry, not a trivial plane, when the behavior is surface-dependent. +- Open the plugin's real UI windows in the same Maya session. +- Position windows before screenshot/recording; verify bounding boxes do not overlap. +- Click or call through the actual UI control callbacks used by artists. +- Assert resulting node attributes or state changed to the expected values. +- Sample geometry before/after deformation; fail on zero or tiny displacement. +- Save the scene and include the saved path in Scenario Result JSON. + +Common KLV Push/Dynamics expectations: + +- Curved high-resolution target, e.g. `polySphere` or body-like mesh. +- KLV Push UI controls: firmness/depth/bulge or project-equivalent real artist controls. +- KLV Dynamics window in the same session when the PR touches shared product UI. +- Scenario fields: `targetSurface`, `windowsNonOverlapping`, `deformationSamples`, `savedScene`. + +## Screenshot/Recording Points + +Capture or verify evidence at these points: + +- Startup failure: immediate failure screenshot, useful for modals/popups. +- Immediately after plugin load or expected `.mll` load: no `Untrusted Plugin Loading` modal before any long wait. +- After plugin load: UI windows visible, no security modal. +- After control edits: changed controls visible where possible. +- After deformation: viewport shows changed target. +- Final closeout: Scenario Result references screenshot/recording/saved scene or the evidence bundle contains them. + +Before publishing evidence paths or attaching media, inspect for private desktop content, hostnames, secrets, chat/browser tabs, or personal files. If screenshots include private content, keep them local and report only non-sensitive metadata unless the coordinator explicitly approves publication. + +## Popup Handling + +Treat recurring popups as setup failures first: + +- `Untrusted Plugin Loading`: stop the scenario wait immediately, verify `pluginArtifacts` and `trustedPluginArtifactsRoot`, then rerun from trusted staging. Clicking `Allow` is allowed only to diagnose, teach, or clear the host baseline; accepted proof must come from a fresh command rerun. +- Missing plugin/load failure: verify downloaded artifact commit/job, extracted plugin extension, `MAYA_PLUG_IN_PATH`, and Scenario Result logs. +- License/update/welcome dialogs: close only if they are host baseline noise; then rerun from a clean state and record the action. +- File overwrite/save dialogs: make scenario write to a unique run path or remove the prompt condition in code. + +Only use manual clicks to diagnose or clear host baseline state. Accepted proof must be repeatable from the command line. + +## Failure Triage + +Classify before patching: + +- Scenario-owned: too slow mesh, waits on missing output, no timeout budget awareness, non-unique files, UI overlap, callback not invoked, no final JSON. +- Host/Maya Stall owned: SSH/session timeout before scenario starts, broken screenshot/recording transport, missing trusted root in host config, broker unreachable. +- Artifact-owned: stale commit artifact, wrong Maya version, missing plugin binary, bad hash/size mismatch. + +If scenario-owned, patch the same PR/MR and rerun local tests, autoreview, CI, artifact download, then real live gate. If host/Maya Stall owned and independent of the scenario, open/propose separate infra work and do not merge on waived proof. + +## Closeout Checklist + +Record concise proof: + +- target commit and target branch +- CI pipeline/job id, artifact size, artifact sha256, plugin sha256 +- Maya Stall command, run id, Maya version/profile +- Scenario Result path and key fields +- screenshot path/size and recording path/size +- saved scene path/size +- local tests and autoreview result +- Public Artifact Confidentiality: PASS/FAIL with reason + +Never close a behavior-changing Maya plugin PR with fake/local-only, stale-head, plane-only, skipped, manual-only, or screenshot-only proof. diff --git a/skills/maya-plugin-e2e/agents/openai.yaml b/skills/maya-plugin-e2e/agents/openai.yaml new file mode 100644 index 0000000..fbf7412 --- /dev/null +++ b/skills/maya-plugin-e2e/agents/openai.yaml @@ -0,0 +1,4 @@ +interface: + display_name: "Maya Plugin E2E" + short_description: "Reliable Maya plugin UI proof runs" + default_prompt: "Use $maya-plugin-e2e to run a Maya plugin UI e2e gate with trusted plugin staging and screenshot/recording proof."