Repository navigation
ASIM: Add Proofpoint POD Message EmailEvent parsers - #15409
Open
adbhogeGit wants to merge 11 commits into
Open
adbhogeGit wants to merge 11 commits into
adbhogeGit wants to merge 11 commits into
Conversation
Add parameterless and filtering parsers, unifier registrations, fixtures, focused offline validation, and generated ARM assets. Live validation is deferred by user direction. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 723d3767-4cf2-432d-8609-3cff81330f86
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 723d3767-4cf2-432d-8609-3cff81330f86
Remove generated ARM changes and retain source-only offline checks and documentation. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 723d3767-4cf2-432d-8609-3cff81330f86
…roofpoint-message-parser
|
ASIM parsers have been changed. ARM templates were regenerated from the updated KQL function YAML files. |
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
The reported KQL catalog failures, deferred live validation, and unverified source mappings require maintainer review before merging.
No open issues 🎉
What changed in this PR
Adds Proofpoint POD message support to ASIM EmailEvent 1.0.0, making ProofpointPODMessage_CL available through the standard EmailEvent unifiers.
Changes:
- Adds parameterless and filtering parsers that preserve one event per source message, with optional source details.
- Registers both parsers in the unifiers and includes generated ARM deployment artifacts.
- Adds synthetic fixtures, offline validation checks, changelogs, and documentation of mapping limitations and deferred live validation.
| File | Description |
|---|---|
| Sample Data/ASIM/ProofpointPODMessage_CL_Schema.csv | Defines the source fixture schema. |
| Sample Data/ASIM/Proofpoint_POD_EmailEvent_IngestedLogs.csv | Adds four synthetic message scenarios. |
| Parsers/ASimEmailEvent/Tests/ProofpointPODMessage.Tests.ps1 | Checks contracts, KQL types, filter bindings, and registrations offline. |
| Parsers/ASimEmailEvent/README.md | Documents mappings, filters, limitations, and validation status. |
| Parsers/ASimEmailEvent/Parsers/vimEmailEventProofpointPODMessage.yaml | Adds the filtering Proofpoint parser. |
| Parsers/ASimEmailEvent/Parsers/imEmailEvent.yaml | Registers the filtering parser and forwards parameters. |
| Parsers/ASimEmailEvent/Parsers/ASimEmailEventProofpointPODMessage.yaml | Adds source-to-EmailEvent normalization. |
| Parsers/ASimEmailEvent/Parsers/ASimEmailEvent.yaml | Registers the parameterless parser. |
| Parsers/ASimEmailEvent/CHANGELOG/vimEmailEventProofpointPODMessage.md | Records the filtering parser’s initial release. |
| Parsers/ASimEmailEvent/CHANGELOG/imEmailEvent.md | Records the filtering unifier update. |
| Parsers/ASimEmailEvent/CHANGELOG/ASimEmailEventProofpointPODMessage.md | Records the parameterless parser’s initial release. |
| Parsers/ASimEmailEvent/CHANGELOG/ASimEmailEvent.md | Records the parameterless unifier update. |
| Parsers/ASimEmailEvent/ARM/vimEmailEventProofpointPODMessage/vimEmailEventProofpointPODMessage.json | Adds the filtering parser deployment template. |
| Parsers/ASimEmailEvent/ARM/vimEmailEventProofpointPODMessage/README.md | Provides filtering parser deployment links. |
| Parsers/ASimEmailEvent/ARM/imEmailEvent/imEmailEvent.json | Updates the deployed filtering unifier. |
| Parsers/ASimEmailEvent/ARM/FullDeploymentEmailEvent.json | Includes both source parsers in full deployment. |
| Parsers/ASimEmailEvent/ARM/ASimEmailEventProofpointPODMessage/README.md | Provides parameterless parser deployment links. |
| Parsers/ASimEmailEvent/ARM/ASimEmailEventProofpointPODMessage/ASimEmailEventProofpointPODMessage.json | Adds the parameterless parser deployment template. |
| Parsers/ASimEmailEvent/ARM/ASimEmailEvent/ASimEmailEvent.json | Updates the deployed parameterless unifier. |
Review effort: Balanced
Register the exact CCP custom-table schema and normalize JSON-encoded dynamic columns so CI fixture ingestion produces typed EmailEvent arrays. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 723d3767-4cf2-432d-8609-3cff81330f86
Document that checked-in parser ARM templates are generated and committed by the repository GitHub action, rather than manually edited. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 723d3767-4cf2-432d-8609-3cff81330f86
Keep the parser mappings native to their dynamic connector columns; the hosted recipient-array error comes from missing ArrayValuesType metadata, and the filter validator lacks EmailEvent support. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 723d3767-4cf2-432d-8609-3cff81330f86
added 2 commits
October 23, 2026 19:02
…roofpoint-message-parser
Derrick Lee (yummyblabla)
left a comment
Collaborator
There was a problem hiding this comment.
Don't
| @@ -0,0 +1,216 @@ | |||
| $ErrorActionPreference = 'Stop' | |||
Collaborator
There was a problem hiding this comment.
We don't need this file
| - [Normalization and the Advanced Security Information Model (ASIM)](https://aka.ms/AboutASIM) | ||
| - [Deploy all of ASIM](https://aka.ms/DeployASIM) | ||
| - [ASIM Email Event normalization schema reference](https://aka.ms/ASimEmailEventDoc) No newline at end of file | ||
| - [ASIM Email Event normalization schema reference](https://aka.ms/ASimEmailEventDoc) |
Collaborator
There was a problem hiding this comment.
Changes not needed in this README
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Required items, please complete
Change(s):
ProofpointPODMessage_CLschema from the CCP connector definition in the repository KQL-validation table catalog.Reason for Change(s):
KS204caused by the missing custom-table definition. The remaining hosted ASIM data/filter failures are validator catalog/support gaps, not an evidence-backed parser mapping defect. No validation was weakened.Version Updated:
Testing Completed:
89a900d: KQL Validations, ASIM template/ingest-time validations, ARM generation, CodeQL, syntax, solution, connector, detection, and other checks.ProofpointPODMessage_CL. The repository's standard shared-canary CI ingestion completed.Checked that the validations are passing and have addressed any issues that are present:
89a900d; this resolves the earlierKS204missing-table-schema failure. The catalog fixture mirrors the connector's exact current CCP table schema.EmailToRecipientsarrays with “All inner values must be of type []”:ASIM/dev/ASimTester/ASimTester.csvdeclaresDynamicType=arraybut leavesArrayValuesTypeblank, although source recipients are string arrays. The ASIM tester catalog was not changed.Schema: EmailEvent - Not an existing schema or not supported by the validations script, before testing parser parameters.ASimFilteringTest.py's supported-schema catalog omits EmailEvent. Shared EmailEvent validator support is being implemented in a separate PR; this PR does not modify that code.89a900d69164ca3cc5c646cd6711b2a078027e8ehave passed. The PR remains open and ready for review, not merged. Live user-workspace validation remains deferred.