Skip to content

[ASIM] Add Salesforce Audit Trail and Login History ingestion-time parsers - #15390

Open
Chencheng Xie (Microsoft) (melmarsezio) wants to merge 6 commits into
masterfrom
chenchengxie/SalesforceIngestTimeParser
Open

Chencheng Xie (Microsoft) (melmarsezio) wants to merge 6 commits into
masterfrom
chenchengxie/SalesforceIngestTimeParser

Conversation

@melmarsezio

Copy link
Copy Markdown
Contributor

Purpose

Add ingestion-time ASIM normalization for Salesforce Setup Audit Trail and Login History data across both the Microsoft native-table routes and the newer CCF custom-table routes. Each source table receives its own ITP and source-specific validation fixture, while reusing the established query-time parser as the optional parity reference.

Required items, please complete

Change(s):

  • Added four Salesforce ingestion-time parsers:
ASIM schema Source table Input stream Target table Optional QTP counterpart
AuditEvent SalesforceAuditTrail Microsoft-SentinelSalesforceAuditTrail ASimAuditEventLogs ASimAuditEventSalesforceAuditTrail
AuditEvent SalesforceAuditTrailV2_CL Custom-SalesforceAuditTrailV2_CL ASimAuditEventLogs ASimAuditEventSalesforceAuditTrail
Authentication SalesforceLoginHistory Microsoft-SentinelSalesforceLoginHistory ASimAuthenticationEventLogs ASimAuthenticationSalesforceLoginHistory
Authentication SalesforceLoginHistoryV2_CL Custom-SalesforceLoginHistoryV2_CL ASimAuthenticationEventLogs ASimAuthenticationSalesforceLoginHistory
  • Added one source-table-specific CSV fixture for each ITP using the required naming contract:
    {EventVendor}_{EventProduct}_{Schema}_{SourceRawTable}_IngestedLogs.csv.
  • Reused the existing authoritative Salesforce sample events for the native-table routes.
  • Adapted the same event values to the repository schemas for the V2 custom tables, removing legacy workspace metadata fields and setting the correct V2 Type.
  • Preserved pass-through ParserFilterQuery: source behavior for all four routes.

Reason for Change(s):

  • Salesforce Audit Trail and Login History currently have query-time ASIM normalization, but no ingestion-time parser coverage for the native and CCF source tables.
  • These parsers enable normalization into native ASIM tables during ingestion and prepare all four routes for the new automated ITP validation workflow.
  • Related project: ADO 5596535.

Version Updated:

  • Not applicable; no Detections/Analytic Rule templates are changed.

Testing Completed:

  • python -m unittest discover -s ".script\tests\asimParsersTest\ingestTimeParser\tests" -p "test_asim_*.py" — Passed, 124 tests.
  • Prepared all four CSV fixtures through prepare_fixture — Passed; each fixture produced 10 typed rows using the repository table schema.
  • Discovery and counterpart resolution — Passed; all four ITPs resolved the expected source-specific fixture and QTP counterpart.
  • Optional QTP eligibility — Passed for all four routes.
  • git diff --check origin/master...HEAD — Passed.

Known static-validation findings

The fixtures resolve and prepare correctly, but the current static validator reports these parser metadata/stream findings that still need resolution before merge:

  • META005 / META006 for all four files because ParserName currently uses the _ASim_... built-in-function alias format rather than the validator's ASim<Schema>... parser-name format.
  • SOURCE001 / SOURCE002 for the two native-table routes because their connector stream names are Microsoft-SentinelSalesforceAuditTrail and Microsoft-SentinelSalesforceLoginHistory, while the validator currently expects Microsoft-{SourceRawTable} and searches connector DCR JSON evidence.

Checked that the validations are passing and have addressed any issues that are present:

  • Need Help. Fixture preparation, discovery, QTP eligibility, regression tests, and diff checks pass. The parser-name and native-stream static-validation findings above require a parser-contract or validator-contract decision before merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants