Skip to content

Support reviewed workbook updates with clean public artifacts and local audit evidence - #9

Merged
oferInbar merged 3 commits into
oferInbar:Solution-toolkit-masterfrom
kingwil:kingwil/toolkit-reviewed-workbook-updates
Oct 16, 2026
Merged

oferInbar merged 3 commits into
oferInbar:Solution-toolkit-masterfrom
kingwil:kingwil/toolkit-reviewed-workbook-updates

Conversation

@kingwil

@kingwil kingwil commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator

Maintainer-authorized scope

Support explicitly reviewed post-import/post-migration workbook corrections while keeping published authored JSON free of personal review and audit history. Toolkit development only: no content branches, actual workbooks, real content packages, or cloud resources were modified.

This replaces the earlier embedded-audit design in this PR. There is no anonymized reviewer, renamed acceptance field, or relocated audit payload in published workbook metadata.

Clean public contract

The actual Notebook and existing xdrMetadata identity/version remain unchanged. Public contentProvenance contains only:

  • format: "public-workbook-v1" and factual native/migrated origin;
  • required workloads, connector/table prerequisites, and exact workspace-function source paths/hashes.

No author/reviewer accounts, acceptance decisions, references, sessions, conversion/native/update receipts, per-query inventories, reverse deltas, or audit histories are published. Public dependency metadata does not claim personal approval. No duplicate full native source artifact is required in the repository/package.

Both schema mirrors and the normalized vendored schema manifest hash are synchronized. Legacy proof schemas remain readable for importing and independently validating historical evidence; V3.1 publication rejects legacy authored files until explicitly cleaned.

Explicit local review and preserved evidence

  • workbook-update-plan/review/apply retain original-baseline validation, whole-document and changed/added/removed-query review, stable-ID checks, and strict current lineage/dependencies. Apply now writes clean public JSON and a private receipt instead of embedding authoredReview.
  • workbook-publication-plan inspects an already reviewed legacy artifact, including old migrated/native provenance and authoredReview. It validates historical proof, exposes complete current dependencies, and reports missing connector coverage without inventing it.
  • workbook-publication-clean --expected-sha256 ... --review ... requires explicit current local authorization and original raw-file hash agreement. It preserves the original artifact bytes, available source bytes, and authorization in untracked/gitignored Logs/workbook-reviews; only public provenance changes.
  • Existing historical evidence is not deleted. Reauthorization archives the prior local receipt. Local input/baseline/review/current-document hashes remain bound; rekeying a receipt cannot reuse stale authorization for new payload/settings/dependencies.
  • Clean published artifacts can serve as public baselines for subsequent explicit update reviews. A new edit needs fresh review, not the old approval.

Review hashes detect staleness; they are not signatures or proof of human identity. Git history/review and accurate explicit authorization remain the trust boundary.

Validation, dependencies, and reproducible packaging

workbook-publication-validate --mode structural, the schema, Python structural validation, and the static adapter work with public content/dependencies without private approvals. They do not approve authoring.

--mode authoring, native-workbook-validate, authoring workflow checks, and normal Python/PowerShell packaging additionally require the current hash-bound local receipt. The PowerShell adapter checks the actual in-memory document. There is no silent HEAD or approval fallback.

An explicitly selected SENTINEL_XDR_WORKBOOK_PUBLIC_REF=<published commit> permits standalone reproduction, not authoring/workflow acceptance. Every selected in-memory authored workbook must match its exact file at that commit; public table/function/source validation still applies. Missing refs or changed content fail. This allows a clean checkout to reproduce without shipping private receipts or raw native source copies.

Unknown tables/functions, unsupported dynamic/cross-resource inputs, exact workspace-function hashes, unconditional emitted savedSearch prerequisites, native-only restrictions, and built-in ASIM behavior remain enforced. SecurityIncident is classified as a Sentinel dependency. Historical incomplete nested-source exceptions remain valid for importing old proof, but are not public packaging waivers: missing current workload/connector coverage requires explicit local completion and review.

Installed Notebook provenance remains the deployment identity/version quartet. Registration retains public dependencies. Notebook/1.0, canonical/instance IDs, authored content versions, solution version policy, default-false deployment switches, V3.1 candidate/official validation, installation/registration qualification, and environment authorization gates remain unchanged. Same-version cleanup/rebuild does not increment authored content versions; deployment contentVersion remains the selected solution release.

Attribution automatic lookup and its fail-closed guard are unchanged. Documentation no longer suggests skipping attribution for this maintenance route.

Usage and verification

See the README sections Reviewed post-import workbook corrections (offline) and Clean public workbooks and private authoring receipts for plan/review/cleanup, explicit structural versus authoring validation, standalone no-bump V3.1 packaging, public-checkout reproduction, and official validation-only wrapper usage. The canonical workflow documents the same publication boundary without relaxing existing stages/gates.

Python 3.12, existing environment; no dependency installation, full content suite, real content build, live KQL/UI, deployment, or runtime qualification.

  • Final focused offline update/publication/original-migrated/native selections and vendored manifest integrity: 237 passed, five real V3.1 fixture-build cases deselected.
  • Tests include clean legacy migration/native/update outputs, verbatim retained local proof/authorization, stale and rekeyed receipts, metadata/payload/dependency edits, source-free portable structure, explicit Git-ref reproduction through the PowerShell adapter, ASIM and workspace-function prerequisites, missing Sentinel connector coverage, and clean installation/registration resources plus a fixture archive.
  • Read-only cleanup plans were checked against the five existing authored inputs. Four have complete current prerequisites; the historical hybrid Office 365 artifact correctly blocks on missing SecurityIncident connector coverage until explicitly reviewed. No actual content file was written.
  • Separate attribution failure-path tests passed. The unchanged attribution success-path test hit Windows metadata atomic-replacement failures in several subtests on both attempts; those errors remained fail-closed with no marker. This unrelated backend was not altered or waived.
  • git diff --check passed.

Subsequent content integration (16 October 2026)

The implementation-only scope above describes the toolkit development task. Afterward, the coordinator used these commands to clean and publish all five actual workbook artifacts in separate content PRs:

All five cleanup operations passed authoring validation with current local receipts. Original Notebook queries, visuals, settings, stable IDs and versions were preserved. Reviewer identities, decisions, session references and historical proof remain only in ignored local evidence; none are shipped in the authored JSON or package.

For the hybrid Detections and Insights workbook, the existing MicrosoftThreatProtection.json connector explicitly lists both SecurityAlert and SecurityIncident. Current public prerequisites were completed against that source under explicit local review, without changing historical receipts or relaxing the missing-coverage gate.

Both same-version V3.1 candidates were regenerated and inspected. The XDR generated ARM/ZIP carries the completed dependency declaration. The O365 deployed JSON was unchanged, so timestamp-only ZIP churn was discarded. The intervening remote O365 API-version correction was preserved.

This integration is not clean official validation: Defender ARM-TTK still fails inner-template extraction; O365 still reports 47 passes and six failures plus the existing trailing null-Path diagnostic. No full suite rerun, new live-query/UI test, deployment or runtime qualification was performed during cleanup. User-supplied prior runtime observations are described separately in the content PRs.

kingwil and others added 2 commits October 15, 2026 15:20
Preserve original conversion and native review receipts, reconstruct and validate the historical baseline, and bind current payload/query dependency reviews without changing IDs or existing prerequisites. Integrate offline plan/review/apply with authored schemas and V3.1 validation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Handle accepted migrated baselines with incomplete nested Sentinel lineage without rewriting receipts. Require exact per-query inherited table sets and explicit Sentinel connector coverage; keep native gates strict and feed validated current tables into PowerShell prerequisite scanning. Document standalone V3.1 content maintenance without changing frozen workflow gates.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Preserve legacy proof locally, require fresh hash-bound authoring receipts, and support explicit public Git-ref reproduction without private approvals.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@kingwil kingwil changed the title Support explicitly reviewed post-import workbook corrections Support reviewed workbook updates with clean public artifacts and local audit evidence Oct 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants