[JamesAde11]

Change(s)

Add a query-time ASIM EmailEvent 1.0.0 normalization parser for the native EmailEvents source table, with regular and filtering variants.

Replacement for closed PR #15354. These are alternative implementations, not separate additions to merge together.

The contribution includes:

• Regular and filtering parser YAML definitions.
• Registration in both EmailEvent unifiers, including parameter forwarding and source/global exclusions.
• Four parser/unifier changelogs and generated ARM deployment templates.
• Repository-required source-table definition and sample assets.

Development helpers, local test adaptations, raw execution evidence and environment configuration are not included.

Reason for Change(s)

Normalize Microsoft Defender for Office 365 delivery records into the common EmailEvent contract so queries can use normalized email fields instead of product-specific names.

This is query-time normalization. The contribution does not create a destination table or deploy functions into customer workspaces.

The implementation follows the official ASIM parser creation workflow. The skill files were followed directly rather than invoked through a registered skill. Read-only validation adaptations are described below.

Mapping decisions

• Classify records as EmailDelivery, preserving one output row per source record.
• Map native TimeGenerated to event start/end time.
• Keep header From and SMTP envelope sender separate.
• Map the actual recipient to Dst and retain To/Cc arrays.
• Keep delivery outcomes separate from threat verdicts. Successful delivery does not mean a message is safe.
• Map Delivered/Junked to Delivered/Success, Replaced to Modified/Success, and Blocked or Failed/Dropped locations to Failure.
• Normalize only supported delivery locations, retaining the original location separately.
• Preserve missing or unknown information rather than inventing values or policy causes.
• Prefer a valid IPv4 address, otherwise a valid IPv6 address. Reject network prefixes as individual source addresses.
• Support disabled, optional unmapped-field packing and the standard EmailEvent filters.
• Do not join other event tables, expand records, aggregate or deduplicate.

Version Updated

• New source parsers: 0.1.0.
• EmailEvent unifiers: 1.0.0 to 1.0.1.
• Target EmailEvent schema remains 1.0.0.

Testing Completed

Read-only KQL validation used records already present in a Log Analytics workspace, explicitly scoped to the native EmailEvents table and a fixed 24-hour window.

No records were ingested and no parser or tester functions were deployed. The repository sample asset is separate from this tenant-data validation.

Check	Result
Source-to-output record preservation	All 1,636 records preserved
Direct source-field mappings	Zero mismatches across 23 fields, including duplicate multiplicities
Delivery, direction and authentication checks	Zero mismatches in the observed cases
Parser filtering, time boundaries, packing, disabled behavior and full-output variant equivalence	35 controls passed
Both unifiers, parameter forwarding and source/global exclusions	26 controls passed
Separate IPv4/IPv6 prefix checks through the filtering parser and unifier	4 controls passed
Official schema-tester logic, both variants	Zero errors; 11 recommended-field warnings per variant
Unchanged official data-tester logic, 1,000 records per variant	Two recipient-array metadata errors per variant, retained
Separately corrected recipient metadata with official data-tester logic	Zero errors on 1,000 records per variant
Separate tester type controls	String/empty arrays accepted; numeric, mixed and Boolean arrays and dictionary-shaped values rejected
Existing repository parser-template checks	30 passed
Generated deployment assets	Six ARM JSON files parsed; four query/parameter definitions matched YAML

The 65 behavioral controls compare expected results with actual event membership and multiplicities, rather than checking row counts alone.

Workflow adaptations and retained limitations

• The exact packaged parser bodies were evaluated as query-local functions against the workspace source.

The observed records covered inbound, outbound, internal and unknown directions, delivered and blocked outcomes, recipient arrays, authentication results, and IPv4/IPv6 addresses.

The window contained no populated threat classifications, quarantine events, Junked/Replaced actions or unknown delivery actions. Those cases are not covered by this tenant-data run.

No parser-code changes were required following the tenant-data validation.

Review requested

Review requested from Derrick Lee (@yummyblabla) and Ofer Shezaf (@oshezaf).