Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/desktop-test-env-secret-length.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@fluentui-react-native/desktop-test': patch
---

Stop redacting short environment values such as `true` or `0` from command output. Variables named like a token, secret, password, nonce, or authorization header are only registered as secrets when their value is at least 8 characters long, so machine-readable output like `"delegated":true` is no longer rewritten to `[REDACTED]`.
8 changes: 8 additions & 0 deletions packages/agentic/desktop-test/references/reporting.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,14 @@ Each run writes `artifacts/desktop/<run-id>/` by default:
| `junit.xml` | Test outcomes, infrastructure errors, and incomplete work |
| `commands/` | Bounded, redacted stdout/stderr logs with source attribution |

Command output is redacted before it is logged or reported. Redaction covers the
secrets the harness registers and the values of environment variables whose
names contain `nonce`, `token`, `password`, `secret`, or `authorization`, when
the value is at least 8 characters long. Shorter values such as `true` or `0`
are flags, not credentials, and redacting them would rewrite machine-readable
output. Register a shorter credential explicitly with the reporter's
`addSecret`.

Stderr and test progress are live without verbose mode. Failed commands replay
the collected output. GitHub Actions grouping and annotations are rendered by
the supervisor; child output cannot issue workflow commands. Upload the full
Expand Down
7 changes: 6 additions & 1 deletion packages/agentic/desktop-test/src/commands.ts
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,10 @@ export type NodeDesktopCommandRunnerOptions = {
maxLogBytes?: number;
};

// Credentials are long. Flags and counters such as `true` or `0` are not, and redacting them rewrites machine-readable
// output (`"delegated":true` becomes `"delegated":[REDACTED]`). Shorter secrets must be registered explicitly with addSecret.
const MIN_ENVIRONMENT_SECRET_LENGTH = 8;

export class NodeDesktopCommandRunner implements DesktopCommandRunner {
private reporterValue?: DesktopReportWriter;
private ownedReporter?: DesktopRunReporter;
Expand Down Expand Up @@ -85,7 +89,8 @@ export class NodeDesktopCommandRunner implements DesktopCommandRunner {
verbose: this.options.verbose,
}));
for (const [key, value] of Object.entries({ ...process.env, ...command.env, ...this.options.environment })) {
if (/nonce|token|password|secret|authorization/i.test(key) && value) reporter.addSecret(value);
if (/nonce|token|password|secret|authorization/i.test(key) && value && value.length >= MIN_ENVIRONMENT_SECRET_LENGTH)
reporter.addSecret(value);
}
const id = `command-${randomUUID()}`;
const emit = reporter.source(id);
Expand Down
23 changes: 23 additions & 0 deletions packages/agentic/desktop-test/src/contracts.contract.cjs
Original file line number Diff line number Diff line change
Expand Up @@ -323,6 +323,29 @@ async function main() {
.slice(out.text().indexOf('::group::one'), out.text().indexOf('::endgroup::', out.text().indexOf('::group::one')));
assert(groupOne.includes('::stop-commands::'));
assert(!groupOne.includes('::warning::two'));
} else if (mode === 'command-secrets') {
const run = reporter({ kind: 'lifecycle' });
const runner = new NodeDesktopCommandRunner({ reporter: run });
// Secret-named variables with short values (flags, counters) must not rewrite machine output; 8 characters and up are redacted.
await runner.run({
...command(
"console.log(JSON.stringify({ delegated: true, exitCode: 0 })); console.log('seven ' + process.env.FIXTURE_SEVEN_NONCE); console.log('eight ' + process.env.FIXTURE_EIGHT_SECRET);",
'secret-environment',
),
env: { FIXTURE_FLAG_TOKEN: 'true', FIXTURE_SEVEN_NONCE: '1234567', FIXTURE_EIGHT_SECRET: 'abcdefgh' },
});
assert.equal((await run.finish()).status, 'passed');
const output = fs
.readFileSync(path.join(run.root, 'events.jsonl'), 'utf8')
.trim()
.split('\n')
.map((line) => JSON.parse(line))
.filter((event) => event.kind === 'output' && event.payload.stream === 'stdout')
.map((event) => event.payload.text);
assert.deepEqual(JSON.parse(output.find((text) => text.includes('"delegated"'))), { delegated: true, exitCode: 0 });
assert(output.includes('seven 1234567\n'), JSON.stringify(output));
assert(output.includes('eight [REDACTED]\n'), JSON.stringify(output));
assert(!output.join('').includes('abcdefgh'));
} else if (mode === 'command-arguments') {
const run = reporter({ kind: 'lifecycle' });
const runner = new NodeDesktopCommandRunner({ reporter: run });
Expand Down
1 change: 1 addition & 0 deletions packages/agentic/desktop-test/src/contracts.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ describe('neutral desktop harness contracts', () => {
'adapters',
'reporting',
'commands',
'command-secrets',
'command-arguments',
'transport',
'delegated-cli',
Expand Down
Loading