Skip to content

Sentinel to XDR solution migrator V1 PREVIEW - #2

Open
oferInbar wants to merge 5 commits into
masterfrom
preview/sentinel-to-xdr-migrator-v1-20261012
Open

oferInbar wants to merge 5 commits into
masterfrom
preview/sentinel-to-xdr-migrator-v1-20261012

Conversation

@oferInbar

Copy link
Copy Markdown
Owner

Required items, please complete

Change(s):

  • Introduce the Sentinel to XDR solution migrator V1 PREVIEW toolkit and its agent-neutral workflow, CLI, schemas, tests, documentation, and editor/agent adapters.
  • Include deterministic Microsoft Sentinel analytic-rule conversion to Defender XDR Custom Detection YAML, source provenance, explicit mapping reviews, disabled-by-default content, current-evidence validation, and V3.1 hybrid solution packaging.
  • Include supporting ingestion-contract tools, solution migration utilities, and scoped local/CI validation integrations required by the toolkit.
  • Add explicit V3.1 target-version selection, full-inventory offline Authoring completion, receipt-bound conversion-evidence recovery, release-notes-only refresh, and guarded same-version content rebuilds. Rebuild approval must match the complete actual changed-file set; undeclared edits and ReleaseNotes changes are rejected, including at dispatch, acceptance, and retries.
  • Publish an isolated toolkit-only snapshot from master, not the development branch's unrelated ancestry. No top-level solution content, generated solution ZIPs, operational .env, local logs/evidence, credential caches, ignored development candidates, or generated compiler maps are included. The separate solution-content draft is Azure/Azure-Sentinel#15308; fork PR DO NOT MERGE draft PR #1 was an earlier external contribution and is already merged.

Reason for Change(s):

  • Provide a preview, resumable toolkit for authors to produce reviewable Sentinel/XDR solution content without requiring a live tenant as a prerequisite for cloud-free Authoring.
  • Authoring follows Discovery -> Conversion -> Validation -> Packaging -> Report. Offline completion requires current full-inventory conversion, structural validation, and resolved technical and hash-bound review findings. It records runtime validation as not run; it is not live Qualification.
  • Qualification remains explicitly selected and target-bound. Runtime-query consent, exact query identities, deployment, reviewed shared mock ingestion, and alert parity retain their separate gates. Environment writes require fresh exact-scope approval; neither conversion nor review acceptance authorizes them.
  • Persist workflow state and receipt/artifact hashes so retries cannot silently relabel stale packages or consume release actions twice. Keep uncertainty, exclusions, failures, skipped checks, and schedule/entity conversion differences visible.

Version Updated:

  • This PR is labeled V1 PREVIEW, not a production-qualified release or a solution package version bump.
  • Python package metadata versions remain as implemented. V3.1 packaging remains the supported solution publication route; this PR does not publish the migrated solutions themselves.

Testing Completed:

  • Current focused regression command:
    cd Tools/SentinelToXDRMigration/tests &&
    PYTHONDONTWRITEBYTECODE=1 /opt/anaconda3/bin/python3.12 -m unittest test_workflow test_packaging test_packaging_selection test_content_deployment_parameters test_version_policy test_customer_usage_attribution test_report_dashboard
    Passed: 244 tests, exit 0. Covers the latest release/version, workflow, packaging, native and encoded-list path-containment regressions, content parameters, attribution, and report changes.
  • Earlier reviewed toolkit baseline verification: 764 migration tests ran, 756 passed and eight installed-browser tests were deliberately skipped; ten validation-binding tests passed with two actual-content integrations skipped; six ingestion-resolver tests passed. Those results predate the latest feature changes and are not represented as a fresh full-suite run for this PR.
  • The earlier full suite used a separately reviewed session-local software-guarded runner with isolated state and mocked credentials; OS sandboxing was explicitly waived. Software guards are not native-code or filesystem confinement. The session runner and private evidence are not published.
  • No cloud deployment, runtime-query execution, mock ingestion, live registration acceptance, or alert parity was performed for this publication. Synthetic packaging tests are not genuine ARM-TTK/live deployment qualification.

Checked that the validations are passing and have addressed any issues that are present:

  • The focused regressions passed. This preview does not claim all possible validation or runtime behavior is qualified.
  • Privacy/publication-hygiene review completed: the original snapshot and successive changed-file audits are bound to the final 212-file candidate manifest. No literal private operational data or credentials were identified in the publishable snapshot. An unrelated ASIM-only instruction-file delta was excluded from this toolkit PR; another flagged path was unchanged generic CI text, not a private user profile.
  • LOW runtime privacy limitation: connector diagnostics and symlink-error messages can include resolved absolute filesystem paths. Captured local logs may therefore expose operator home/workspace locations; sanitize them before sharing. This is a log-minimization follow-up, not an exposed credential or private identifier embedded in this PR.
  • Static security review and fix re-review completed: the reported HIGH path-traversal issue is resolved on the reviewed V3.1 packaging entry point. Python and PowerShell validate native and legacy JSON-encoded content lists; connector and metadata reads use contained, literal filesystem paths. Regression tests cover the original sink and the encoded-list bypass identified during re-review.
  • The security assessment was a bounded static review, not a line-by-line audit of all 212 files. The fix re-review covered the changed containment helpers and their V3.1 call path; unchanged readers and all possible direct invocation routes are not asserted to be exhaustively audited. No absolute safety or live Qualification claim is made.
  • The full inherited snapshot's git diff --cached --check reports existing trailing-whitespace/CRLF warnings. The latest 15-file feature delta passes that check. No implementation changes were made solely to suppress inherited whitespace findings.
  • Known limitations remain explicit: source NRT timing can be converted to a scheduled XDR frequency, runtime output/entity bindings require live verification, and generated resources are disabled. The Azure Activity ARM-TTK extraction failure documented in the separate solution-content draft Azure/Azure-Sentinel#15308 remains unresolved and is not hidden or repaired by this toolkit publication.

Known limitations and follow-up work

The preview's follow-up backlog is tracked in Tools/SentinelToXDRMigration/FOLLOW_UPS.md. These items are not claims of completed functionality or commitments to a release date. Publication-blocking security findings must be resolved separately; they are not optional roadmap items.

  • Strengthen File entity identity validation and preserve YAML version spelling during packaging.
  • Reconcile and improve exact-target authentication/access preflight, deployment/ingestion permission readiness, and private-configuration/support-export safeguards.
  • Improve user-facing progress, preflight visualization, and concise watchdog messages; suppress repetitive no-change alerts.
  • Route explicit user decisions promptly to the accountable owner, preserving run/query bindings and read-only watchdog boundaries.
  • Improve appropriate stage delegation without duplicate executors, and default release-note dates to today unless overridden.
  • Define the proposed Author and Validate deployment-only mode without conflating a successful disabled-resource deployment with functional runtime validation or strict Qualification.

oferInbar and others added 2 commits October 12, 2026 20:31
Include the toolkit, agent-neutral Authoring and Qualification workflows,
V3.1 packaging integrations, reviewed release rebuild guards, schemas,
regression tests, documentation, and supporting ingestion tools.

Isolate the toolkit snapshot from unrelated development-branch content.
Exclude migrated solutions, private configuration and runtime evidence,
ignored development candidates, and generated compiler maps.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: f8553aac-d21b-4e0c-a24a-6dbb12de8a9e
Update the isolated publication snapshot with the reviewed packaging containment fix and curated follow-ups.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f8553aac-d21b-4e0c-a24a-6dbb12de8a9e
Comment thread .script/tests/KqlvalidationsTests/KqlValidationTests.cs Dismissed
Comment thread .script/tests/KqlvalidationsTests/KqlValidationTests.cs Dismissed
Comment thread .script/tests/KqlvalidationsTests/Utils.cs Dismissed
Comment thread .script/tests/KqlvalidationsTests/Utils.cs Dismissed
Comment thread .script/tests/detectionTemplateSchemaValidation/DetectionsYamlFilesTestData.cs Dismissed
Comment thread .script/tests/detectionTemplateSchemaValidation/DetectionsYamlFilesTestData.cs Dismissed
Comment thread .script/tests/detectionTemplateSchemaValidation/DetectionsYamlFilesTestData.cs Dismissed
Comment thread .script/tests/detectionTemplateSchemaValidation/DetectionsYamlFilesTestData.cs Dismissed
Comment thread Tools/AzureMonitorLogsIngestion/tests/test_registry.py Dismissed
Comment thread Tools/SentinelToXDRMigration/sentinel_xdr_migration/alert_parity.py Fixed
Comment thread Tools/SentinelToXDRMigration/sentinel_xdr_migration/alert_parity.py Fixed
Comment thread Tools/SentinelToXDRMigration/sentinel_xdr_migration/alert_parity.py Fixed
Comment thread Tools/SentinelToXDRMigration/sentinel_xdr_migration/alert_parity.py Fixed
Comment thread Tools/SentinelToXDRMigration/sentinel_xdr_migration/alert_parity.py Fixed
Move shared ARM request helpers into a dependency-leaf module so analytic deployment no longer depends on alert parity, and remove the unused import. Add isolated import-order regression coverage.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f8553aac-d21b-4e0c-a24a-6dbb12de8a9e
oferInbar and others added 2 commits October 12, 2026 22:41
Move analytic rule payload construction into a dependency-leaf module and keep duration parsing in a shared leaf utility. Deployment planning and deployment now reuse the same payload builder without importing one another, eliminating the remaining reported cyclic-import paths while preserving behavior.

Add static cycle detection and isolated import-order coverage.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f8553aac-d21b-4e0c-a24a-6dbb12de8a9e
Fetch PR base and head branches into distinct local refs (pr_base_branch and pr_head_branch) so git diff validates detection ID stability without failing on non-fast-forward master:master fetches.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c2e42e4a-1b3f-4241-aae3-b07d52d99a5d
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants