[Chencheng Xie (Microsoft) (melmarsezio)]

Required items, please complete

Change(s):

• Decoupled required ITP validation from raw-table replay:
  • Builds a typed parser-input snapshot directly from the prepared fixture.
  • Restores fixture-only fields omitted from table schemas, such as Type, TenantId, and Title, while excluding non-replayable Azure-generated fields.
  • Runs ParserFilterQuery against an inline typed KQL datatable.
  • Uploads fixture rows directly through the parser DCR.
  • Completes required parser ingestion and ASimDataTester validation before optional QTP replay.
• Made QTP source replay optional:
  • Creates the raw DCR only when a mapped QTP is eligible.
  • Uses the ITP YAML InputStream as the raw DCR outputStream.
  • Treats raw table, DCR, upload, or visibility failures as warnings.
  • Blocks QTP execution and parity after replay warnings without failing required ITP validation.
• Improved parity validation and artifacts:
  • Persists complete qtp_schema, qtp_rows, itp_schema, and itp_rows.
  • Queries the complete ASIM target-table row shape.
  • Builds intersect_schema by shared column name and records QTP/ITP types.
  • Fails schema parity when a shared column has inconsistent types.
  • Persists qtp_rows_filtered and itp_rows_filtered.
  • Runs the existing typed, unordered, duplicate-aware row comparison only after shared schema consistency passes.
  • Preserves all parity datasets in the downloadable artifact when parity fails.
• Runs ASimDataTester once per target; successful QTP replay invokes a parity-only validation path.
• Exposes actionable diagnostics in report.json, Markdown summaries, and GitHub annotations:
  • Includes nested Azure/provider error codes, messages, and targets.
  • Reports optional QTP replay failures as warnings.
  • Surfaces target-output failures under parity.
  • Shows cleanup deletion errors.
• Removed obsolete compatibility code from the old required raw-table architecture, including Azure-query source snapshot construction and physical-table filter fallback.
• Updated the ingestion validation README and regression suite for the new flow.

Reason for Change(s):

• Workflow run 39823143359 for [ASIM] Add Salesforce Audit Trail and Login History ingestion-time parsers #15390 showed that required parser execution could be blocked by raw source-table replay limitations unrelated to the parser DCR itself.
• Legacy Salesforce routes require connector-specific Microsoft stream names rather than a generated Microsoft-{SourceRawTable} stream.
• Required parser transformation, target ingestion, schema validation, and ASIM logical-value testing should run independently of optional QTP source replay.
• QTP/ITP parity needs complete downloadable datasets and explicit shared-column type consistency to diagnose mapping differences.
• Reviewers need the exact Azure rejection or semantic error in the stage summary rather than only a generic HTTP failure.

Version Updated:

• Not applicable. No Detections or Analytic Rule templates are changed.

Testing Completed:

• python -m unittest discover -s ".script\tests\asimParsersTest\ingestTimeParser\tests" -p "test_asim_*.py" — Passed, 138 active tests.
• python -m compileall -q .script\tests\asimParsersTest\ingestTimeParser — Passed.
• python -m pyflakes .script\tests\asimParsersTest\ingestTimeParser — Passed.
• git diff --cached --check and git diff --check — Passed before commit.
• Dry-ran typed parser-input snapshot construction against all four Salesforce artifacts from run 39823143359 — Passed; fixture rows, inferred parser-only fields, and YAML InputStream values were preserved.
• Replayed the legacy Salesforce Audit Trail report artifact — Passed; nested InvalidStream and output-stream target details are exposed in the stage summary.
• GitHub Actions workflow lint was not rerun because this PR does not modify workflow YAML.
• A new live Azure validation run was not performed from this branch.

Checked that the validations are passing and have addressed any issues that are present:

• Yes. All local regression, compilation, lint, artifact, and patch-integrity checks listed above pass.