Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
107 commits
Select commit Hold shift + click to select a range
bd011a2
Add OpenShell and Sentry audit workspace
omwei-org Oct 17, 2026
63bc59a
Record initial OpenShell audit findings
omwei-org Oct 17, 2026
c6a07f7
Add OpenShell audit runtime test matrix
omwei-org Oct 17, 2026
2a4fae1
Pin OpenShell source evidence baseline
omwei-org Oct 17, 2026
a912a38
Tighten evidence labels and EABC mapping in findings
omwei-org Oct 17, 2026
3afcbee
Expand audit runtime matrix and evidence criteria
omwei-org Oct 17, 2026
f2a29cf
Clarify evidence classes and runtime result vocabulary
omwei-org Oct 17, 2026
f3e5ffe
Pin audit evidence links and distinguish docs from source
omwei-org Oct 17, 2026
7834422
Add line-pinned links to findings register
omwei-org Oct 17, 2026
74b6d43
Tighten source permalinks to relevant line ranges
omwei-org Oct 17, 2026
01283ce
Clarify runtime result-state wording in test matrix
omwei-org Oct 17, 2026
c27d8cb
Link VM mount claims to option-default implementation
omwei-org Oct 17, 2026
a3af0dc
Add NVIDIA Sentry public claims and provisional EABC criteria
omwei-org Oct 17, 2026
610e6f8
Record Sentry public claims as documentation-only evidence
omwei-org Oct 17, 2026
a1ac159
Correct source line ranges and complete OS-07 EABC mapping
omwei-org Oct 17, 2026
44e6c16
Tighten test matrix source permalinks and EABC mappings
omwei-org Oct 17, 2026
a766cce
Record upstream baseline drift as a scoped audit finding
omwei-org Oct 17, 2026
acff995
Add explicit baseline-drift gate to runtime matrix
omwei-org Oct 17, 2026
c9aeb73
Record explicit outer-fence contract change in newer upstream
omwei-org Oct 17, 2026
3ffd580
Add runtime tests for newer provisioning-evidence contract
omwei-org Oct 17, 2026
3202769
Add runtime tests for newer provisioning-evidence contract
omwei-org Oct 17, 2026
183ff51
Document newer upstream assurance-contract review scope
omwei-org Oct 17, 2026
639e257
Reconcile baseline finding with Docker and Podman native fence checks
omwei-org Oct 17, 2026
8d2a28d
Strengthen baseline tests for actual Docker and Podman fence drift
omwei-org Oct 17, 2026
b205db2
Record VM provisioning-evidence binding question
omwei-org Oct 17, 2026
8d0c848
Add VM evidence-to-effective-launch binding test
omwei-org Oct 17, 2026
758ed27
Record delegated backend assurance boundary
omwei-org Oct 17, 2026
35f12ed
Add delegated backend assurance-boundary test
omwei-org Oct 17, 2026
036bb81
Add QEMU and libkrun no-NIC source counter-evidence
omwei-org Oct 17, 2026
a8050b3
Add VM vsock and no-direct-egress verification
omwei-org Oct 17, 2026
85951bc
Reflect VM and container source counter-evidence in audit scope
omwei-org Oct 17, 2026
795a12e
Refine delegated backend finding with admission controls
omwei-org Oct 17, 2026
c8ebee8
Align delegated backend test with trusted admission boundary
omwei-org Oct 17, 2026
8a0be83
Add credentialed L4 fallback assurance question
omwei-org Oct 17, 2026
889605c
Add upstream canary test for credentialed L4 fallback
omwei-org Oct 17, 2026
8f7ed7a
Reconcile generation finding with newer source guards and tests
omwei-org Oct 17, 2026
f092352
Add deterministic mixed-generation and credential canary test
omwei-org Oct 17, 2026
8c020af
Add current-source counter-evidence for credential guard
omwei-org Oct 17, 2026
cd31e6a
Align credential fallback test with source counter-evidence
omwei-org Oct 17, 2026
809371c
Record OpenShell follow-up source evidence at b58e24c
omwei-org Oct 17, 2026
a71f42f
Link newer OpenShell source review and clarify scope
omwei-org Oct 17, 2026
88b8d8e
Use canonical current policy schema citation
omwei-org Oct 17, 2026
32786e1
Add OS-06 generation consistency follow-up
omwei-org Oct 17, 2026
46d55b0
Record OS-06 generation review findings
omwei-org Oct 17, 2026
d0671dd
Link OS-06 source review from runtime matrix
omwei-org Oct 17, 2026
d324c0f
Add deterministic runtime protocol for generation race tests
omwei-org Oct 17, 2026
80b6686
Link deterministic generation runtime protocol
omwei-org Oct 17, 2026
5403873
Map GEN runtime test seams to pinned OpenShell source
omwei-org Oct 17, 2026
db1645b
Link pinned-source GEN seam map
omwei-org Oct 17, 2026
6555078
Refine OS-06 based on OPA lock and snapshot inspection
omwei-org Oct 17, 2026
c5cdbf9
Refine GEN seams after OPA serialization review
omwei-org Oct 17, 2026
7043ceb
Narrow OS-06 concern after OPA serialization inspection
omwei-org Oct 17, 2026
af329f0
Inventory existing generation tests and GEN-01 evidence gaps
omwei-org Oct 17, 2026
b06410d
Document existing GEN-01 regression coverage and narrow test gap
omwei-org Oct 17, 2026
617edba
Link GEN-01 regression coverage review
omwei-org Oct 17, 2026
5e1c741
Compare RFC 0005 Phase 3 with pinned CONNECT implementation
omwei-org Oct 17, 2026
12e273c
Record RFC 0005 conformance status for OS-06
omwei-org Oct 17, 2026
2e4cd09
Link RFC 0005 conformance review in test matrix
omwei-org Oct 17, 2026
b4f43e1
Clarify RFC Phase 3 cutover status from CONNECT call-site review
omwei-org Oct 17, 2026
81202e0
Add focused RFC 0005 Phase 3 CONNECT call-path audit
omwei-org Oct 17, 2026
13358d6
Refine RFC 0005 audit: distinguish snapshot projections from credenti…
omwei-org Oct 17, 2026
36628f3
Correct OS-06 assessment with refined CONNECT source trace
omwei-org Oct 17, 2026
3aa3053
Refine OS-06 test focus to credential-guard reload boundary
omwei-org Oct 17, 2026
a9988d4
Record upstream CI evidence without overstating runtime coverage
omwei-org Oct 17, 2026
bd1b331
Add Sentry composition evidence gap assessment
omwei-org Oct 17, 2026
1a5a089
Link Sentry and composition findings to public-claims evidence gap
omwei-org Oct 17, 2026
9985141
Align RFC 0005 review with refined source trace
omwei-org Oct 17, 2026
47638bc
Clarify OS-06 baseline, RFC phase status, credential guard generation…
omwei-org Oct 17, 2026
fc11c3c
Clarify OS-06 generation evidence and separate source-review baseline
omwei-org Oct 17, 2026
d469230
Document follow-up generation test gap without changing matrix baseline
omwei-org Oct 17, 2026
d7b7b6b
Unify Sentry test status and date rolling documentation access
omwei-org Oct 17, 2026
16e2243
Tighten evidence claims and baseline/status distinctions
omwei-org Oct 17, 2026
f6c0e93
Tighten evidence claims and baseline/status distinctions
omwei-org Oct 17, 2026
c5a7999
Unify matrix status vocabulary and distinguish scope prerequisites
omwei-org Oct 17, 2026
9842eb4
Polish RFC Phase 3 audit verification checklist
omwei-org Oct 17, 2026
7f77f1c
Reconcile finding commits, summary coverage, outer-fence evidence gap…
omwei-org Oct 17, 2026
7064abc
Record dated commit-status evidence and cross-link fallback findings
omwei-org Oct 17, 2026
b7111fe
Record dated commit-status evidence and cross-link fallback findings
omwei-org Oct 17, 2026
b346d2f
Align matrix test IDs with renamed VM and delegated-backend findings
omwei-org Oct 17, 2026
10a529e
Add focused revocation and controller-loss evidence review
omwei-org Oct 17, 2026
d035589
Cross-link focused revocation and controller-loss source review
omwei-org Oct 17, 2026
0f19c54
Add explicit revocation and controller-loss effect-side tests
omwei-org Oct 17, 2026
eddc790
Reconcile outer-fence finding with RFC 0012 and revocation source evi…
omwei-org Oct 17, 2026
c214bde
Record inspected upstream Rust CI commands and evidence limits
omwei-org Oct 17, 2026
24c76e6
Clarify evidence classes and record backend stop-bound documentation gap
omwei-org Oct 17, 2026
30d3fe4
Split static and dynamic revocation tests and enumerate controller-lo…
omwei-org Oct 17, 2026
b39a969
Record artifact inspection and precise-SHA test execution limitation
omwei-org Oct 17, 2026
7156ab6
Record per-backend stopping-bound documentation gap as DOC-CLAIM only
omwei-org Oct 17, 2026
a520d62
Label named source regression tests as TEST-CONTRACT evidence
omwei-org Oct 17, 2026
8ea05d5
Normalize matrix table, pin target SHA per row, resolve IDs and clari…
omwei-org Oct 17, 2026
f219440
Repair matrix table structure, target commits, IDs, overlaps and cont…
omwei-org Oct 17, 2026
9e3ecc2
Add pinned GitHub Actions workflow for targeted OpenShell regression …
omwei-org Oct 17, 2026
656e872
Ensure targeted test workflow continues after an individual JUnit val…
omwei-org Oct 17, 2026
62f354d
Rewrite pinned OpenShell test workflow with per-test JUnit and exit-c…
omwei-org Oct 17, 2026
f2b5f18
Fix JUnit validation expression in targeted test workflow
omwei-org Oct 17, 2026
0ceb591
Align backend IDs and remove overlapping VM/network-loss checks
omwei-org Oct 17, 2026
bedd6af
Fix relative link to targeted test workflow
omwei-org Oct 17, 2026
8c0d924
Document three distinct targeted tests and the reproducible CI workflow
omwei-org Oct 17, 2026
71ecece
Record cargo-nextest version in reproducibility artifact
omwei-org Oct 17, 2026
3e6cb1c
Fix OpenShell targeted nextest reporting and test discovery
omwei-org Oct 17, 2026
c8b198e
Pin GitHub Actions dependencies for OpenShell audit workflow
omwei-org Oct 17, 2026
1e6f820
Harden OpenShell audit workflow triggers and evidence metadata
omwei-org Oct 17, 2026
a530a10
Correct nextest exact-name regex and preserve in-flight evidence runs
omwei-org Oct 17, 2026
9acfb25
Record OpenShell targeted regression Run 12 evidence
omwei-org Oct 18, 2026
a290626
docs: add OpenShell runtime evidence template
omwei-org Oct 18, 2026
0dddfd6
docs: record Sentry target identification gaps
omwei-org Oct 18, 2026
9b97ef9
docs: reconcile OpenShell test and runtime commit pins
omwei-org Oct 18, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
214 changes: 214 additions & 0 deletions .github/workflows/openshell-targeted-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,214 @@
name: OpenShell targeted regression tests

on:
workflow_dispatch:
push:
branches:
- audit/openshell-sentry-2026-10
paths:
- .github/workflows/openshell-targeted-tests.yml

concurrency:
group: openshell-targeted-tests-${{ github.ref }}
cancel-in-progress: false

permissions:
contents: read

jobs:
targeted-regressions:
name: Run named tests at pinned OpenShell SHA
runs-on: ubuntu-24.04
timeout-minutes: 60
env:
OPEN_SHELL_SHA: b58e24c9dabd6d4bcf2b7186592d3ad50646541b
CARGO_TERM_COLOR: always
RUST_BACKTRACE: 1
steps:
- name: Checkout audit repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2

- name: Checkout exact OpenShell commit
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
repository: NVIDIA/OpenShell
ref: b58e24c9dabd6d4bcf2b7186592d3ad50646541b
path: OpenShell
persist-credentials: false

- name: Verify exact source SHA
shell: bash
run: |
set -euo pipefail
cd OpenShell
actual="$(git rev-parse HEAD)"
echo "expected_sha=$OPEN_SHELL_SHA"
echo "actual_sha=$actual"
test "$actual" = "$OPEN_SHELL_SHA"
mkdir -p ../artifacts
{
echo "audit_repository=omwei-org/eabc-agentrust-interoperability"
echo "audit_repository_sha=$GITHUB_SHA"
echo "workflow_ref=$GITHUB_WORKFLOW_REF"
echo "target_repository=NVIDIA/OpenShell"
echo "target_sha=$actual"
echo "checkout_utc=$(date -u +%Y-%m-%dT%H:%M:%SZ)"
} > ../artifacts/source-checkout.txt

- name: Install Rust toolchain from pinned OpenShell source
uses: dtolnay/rust-toolchain@e081816240890017053eacbb1bdf337761dc5582 # pinned Rust toolchain action
with:
toolchain: 1.95.0

- name: Install system build dependencies
run: |
sudo apt-get update
sudo apt-get install -y build-essential clang cmake pkg-config libz3-dev

- name: Install pinned cargo-nextest
run: cargo install cargo-nextest --version 0.9.143 --locked

- name: List and run each named regression test
shell: bash
run: |
set -uo pipefail
cd OpenShell
mkdir -p ../artifacts
overall_rc=0

run_one() {
name="$1"
filter="$2"
junit="$GITHUB_WORKSPACE/artifacts/$name.junit.xml"
log="../artifacts/$name.log"
meta="../artifacts/$name.meta.txt"
listed="../artifacts/$name.list.txt"
tool_config="/tmp/nextest-$name.toml"

{
echo "test_name=$filter"
echo "audit_repository=omwei-org/eabc-agentrust-interoperability"
echo "audit_repository_sha=$GITHUB_SHA"
echo "workflow_ref=$GITHUB_WORKFLOW_REF"
echo "target_sha=$(git rev-parse HEAD)"
echo "cargo_nextest_version=$(cargo nextest --version)"
echo "started_utc=$(date -u +%Y-%m-%dT%H:%M:%SZ)"
} > "$meta"

set +e
cargo nextest list --locked --profile ci --workspace --features openshell-server/test-support > "$listed" 2>&1
list_rc=$?
set -e
cat "$listed"

if [ "$list_rc" -ne 0 ]; then
echo "result=HARNESS ERROR" | tee -a "$meta"
echo "nextest_list_exit_code=$list_rc" | tee -a "$meta"
overall_rc=1
return
fi

if ! grep -F -- "$filter" "$listed" >/dev/null; then
echo "result=NOT FOUND" | tee -a "$meta"
echo "nextest_list_exit_code=0" | tee -a "$meta"
overall_rc=1
return
fi

# Nextest writes JUnit via profile configuration, not a --junit CLI flag.
# Use a tool-specific overlay so upstream OpenShell files remain untouched.
printf '[profile.ci.junit]\npath = "%s"\nstore-success-output = true\n' "$junit" > "$tool_config"

command="cargo nextest run --locked --profile ci --workspace --features openshell-server/test-support -E 'test(/(^|::)${filter}$/)' --tool-config-file openshell-audit:$tool_config"
{
echo "listed_test_match=$(grep -F -- "$filter" "$listed" | head -n 1)"
echo "command=$command"
} >> "$meta"

set +e
cargo nextest run --locked --profile ci --workspace --features openshell-server/test-support -E "test(/(^|::)${filter}$/)" --tool-config-file "openshell-audit:$tool_config" > "$log" 2>&1
rc=$?
set -e
cat "$log"
echo "cargo_nextest_exit_code=$rc" | tee -a "$meta"

if [ ! -s "$junit" ]; then
echo "result=HARNESS ERROR" | tee -a "$meta"
echo "reason=JUnit report missing or empty" | tee -a "$meta"
overall_rc=1
return
fi

set +e
python3 - "$junit" "$filter" "$rc" >> "$meta" <<'PY'
import sys
import xml.etree.ElementTree as ET

report, target, run_rc = sys.argv[1], sys.argv[2], int(sys.argv[3])
try:
root = ET.parse(report).getroot()
except Exception as exc:
print("result=HARNESS ERROR")
print(f"junit_parse_error={exc}")
raise SystemExit(2)

cases = list(root.iter("testcase"))
selected = [
case for case in cases
if target == case.attrib.get("name", "")
or target in case.attrib.get("name", "")
or target in case.attrib.get("classname", "")
]
print(f"junit_testcases_total={len(cases)}")
print(f"junit_matching_testcases={len(selected)}")
if len(cases) != 1 or len(selected) != 1:
print("result=HARNESS ERROR")
print("reason=Expected exactly one JUnit testcase for the exact named-test filter")
raise SystemExit(2)

failures = [node for case in selected for node in list(case.findall("failure")) + list(case.findall("error"))]
skipped = [node for case in selected for node in case.findall("skipped")]
if failures:
print("result=FAIL")
print(f"junit_failures_or_errors={len(failures)}")
raise SystemExit(1)
if skipped:
print("result=NOT TESTED")
print(f"junit_skipped={len(skipped)}")
raise SystemExit(2)
if run_rc != 0:
print("result=HARNESS ERROR")
print("reason=nextest returned nonzero but matching JUnit testcase has no failure/error")
raise SystemExit(2)
print("result=PASS")
PY
junit_rc=$?
set -e
if [ "$junit_rc" -ne 0 ]; then overall_rc=1; fi
}

run_one stale_generation_fails_before_relay_context_is_created stale_generation_fails_before_relay_context_is_created
run_one test_forward_relay_guard_blocks_stale_generation_before_upstream_write test_forward_relay_guard_blocks_stale_generation_before_upstream_write
run_one credential_guard_does_not_shadow_inspected_endpoint_config credential_guard_does_not_shadow_inspected_endpoint_config

{
echo "audit_repository=omwei-org/eabc-agentrust-interoperability"
echo "audit_repository_sha=$GITHUB_SHA"
echo "workflow_ref=$GITHUB_WORKFLOW_REF"
echo "target_repository=NVIDIA/OpenShell"
echo "target_sha=$(git rev-parse HEAD)"
echo "toolchain=$(rustc --version)"
echo "cargo_nextest=$(cargo nextest --version)"
echo "finished_utc=$(date -u +%Y-%m-%dT%H:%M:%SZ)"
} > ../artifacts/run-summary.txt
for f in ../artifacts/*.meta.txt; do cat "$f" >> ../artifacts/run-summary.txt; done
exit "$overall_rc"

- name: Upload logs, metadata, test listings and JUnit results
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: openshell-targeted-regressions-${{ github.run_id }}-${{ github.run_attempt }}
path: artifacts/
if-no-files-found: error
retention-days: 30
55 changes: 55 additions & 0 deletions audits/nvidia-openshell-sentry/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# NVIDIA OpenShell / Sentry — EABC audit

## Purpose

Track an evidence-based review of NVIDIA OpenShell and NVIDIA Sentry against the EABC execution-boundary model. This audit is separate from the repository's AgenTrust/cMCP interoperability experiment; it does not change that experiment's scope or conclusions.

## Scope

1. OpenShell standalone.
2. Sentry standalone.
3. OpenShell + Sentry composition, where supported and evidenced.
4. EABC-relevant properties: non-bypassability, boundary integrity, integrity of state/context, and exclusivity; record enforcement-substrate assumptions separately as EBP assurance constraints.

## Evidence discipline

Use the following status labels consistently:

- **DOC-CLAIM**: a vendor/upstream documentation statement; not independently verified by runtime testing.
- **SOURCE**: behavior or control observed in pinned implementation source.
- **TEST-CONTRACT**: a test exists and asserts the stated behavior; this does not mean it was run.
- **RUNTIME**: evidence from an executed test in a recorded environment with commands, versions, observed results and artifacts.
- **NOT ASSESSED**: evidence has not yet been collected to evaluate the claim.

Keep evidence class separate from test result. Runtime result vocabulary: **NOT TESTED**, **INCONCLUSIVE**, **PASS**, **FAIL**. Use **NOT REPRODUCED** only after an actual reproduction attempt was executed and its conditions/results recorded.

Do not convert a source-level test into a claim of runtime validation. Do not report a vulnerability solely because a control was not found in the files reviewed. Record pinned commits and platform/driver scope for every conclusion. Every finding must include its evidence class, an explicit EABC criterion mapping (or state that mapping is not yet assessed), and permalink(s) to the pinned source/documentation with line ranges wherever available. A candidate EABC mapping is not a conformance result.

## Current documents

- [Findings register](findings.md)
- [Runtime test matrix](test-matrix.md)
- [OpenShell pinned-source evidence — original baseline `a4c848a`](evidence/openshell-a4c848a.md)
- [OpenShell follow-up source review — newer commit `b58e24c`](evidence/openshell-b58e24c.md)

## Initial baseline

OpenShell source review is pinned to [`a4c848adbddd0b4732fd70ddcf8a563a5b1e9317`](https://github.com/NVIDIA/OpenShell/commit/a4c848adbddd0b4732fd70ddcf8a563a5b1e9317). The review environment could not resolve GitHub/raw GitHub hosts for local checkout/runtime testing. Source/documentation inspection was performed through repository connector access; no runtime tests were executed. Therefore, runtime status remains NOT TESTED unless a later dated execution record is added.

## Upstream drift — separate review required

The initial findings and original matrix remain scoped to `a4c848adbddd0b4732fd70ddcf8a563a5b1e9317`. A later comparison checked upstream commit `b58e24c9dabd6d4bcf2b7186592d3ad50646541b` (13 commits ahead at the time of comparison). That range materially changed the isolation interface, provisioning-evidence contract, RPC runtime and conformance tests. See [BASELINE-01 and BASELINE-02 in the findings register](findings.md) and the [baseline-drift gate and added tests](test-matrix.md).

The source review found that the older common contract explicitly required four outer-fence guarantees, while the newer common contract binds native evidence to a generation and explicitly warns that the digest is not independent proof of enforcement. This is an **assurance-contract difference, not a confirmed vulnerability**. Partial counter-evidence has now been inspected: Docker and Podman verify actual container network configuration, while the VM source uses QEMU `-nic none` and configures libkrun with vsock/virtiofs rather than an IP NIC. These source-level controls reduce the basis for a claim of a direct-egress defect, but they do not replace runtime evidence or settle revocation/controller-loss behavior, all platform variants, or delegated-backend assurance.

Do not silently replace the original pinned baseline. For any new runtime result, state the exact commit, driver, environment and evidence artifacts. A newer-upstream review should use a distinct execution record and revise scope only after examining the new validation paths.

## Follow-up source review

A targeted follow-up review was recorded for OpenShell commit `b58e24c9dabd6d4bcf2b7186592d3ad50646541b`. It adds source counter-evidence for generation guards, credential guard behavior, driver-level isolation checks and delegated-backend admission. It is **not** a complete re-audit of the newer commit, and no runtime tests were executed. The original findings and runtime matrix remain pinned to `a4c848adbddd0b4732fd70ddcf8a563a5b1e9317`; do not transfer findings or test results across commits without a commit-specific review and execution record.

The current policy schema documents `allow_uninspected_credentials` as a security-sensitive explicit opt-in for credentialed traffic that cannot be inspected. See [NVIDIA policy schema](https://docs.nvidia.com/openshell/latest/reference/policy-schema). Documentation is `DOC-CLAIM`, not runtime evidence.

## Working rule

Every finding must state: claim, EABC relevance, evidence and pinned reference, counter-evidence/limits, status, and next falsification test. Keep vendor-specific features distinct from EABC conformance claims.
Loading
Loading