Skip to content

New solution: PolySwarm for Microsoft Sentinel (3.0.0) - #15388

Open
Erick Ingleby (erickingleby-polyswarm) wants to merge 1 commit into
Azure:masterfrom
polyswarm:polyswarm-solution
Open

Erick Ingleby (erickingleby-polyswarm) wants to merge 1 commit into
Azure:masterfrom
polyswarm:polyswarm-solution

Conversation

@erickingleby-polyswarm

Copy link
Copy Markdown

Change(s):

  • Adds a new solution, Solutions/PolySwarm, version 3.0.0:
    • Two codeless (CCF) data connectors that bring PolySwarm indicators into ThreatIntelIndicators: PolySwarm Hunting (files matched by the customer's PolySwarm hunts) and PolySwarm Curated Feed (TAXII 2.1).
    • Three analytics rules, shipped disabled, mapped to MITRE ATT&CK with entity mappings.
    • Four hunting queries (file hash, IP, domain, URL), mapped to MITRE ATT&CK.
    • Fifteen playbooks: file hash, URL, IP and domain enrichment (incident- and entity-triggered), sandbox detonation for URLs and Azure Storage blobs, a quota check, and two playbooks marked preview (Outlook attachment detonation and Microsoft Defender for Endpoint file detonation).
    • A status workbook that reads only the customer's workspace.
    • A getting-started guide with screenshots, plus deployment and permissions documentation, under Solutions/PolySwarm/docs.
  • Adds the PolySwarm logo (Logos/PolySwarm.svg), the workbook at Workbooks/PolySwarmStatus.json with light and dark previews, and its WorkbooksMetadata.json entry.
  • Adds PolySwarmHuntingCCFDefinition and PolySwarmFeedCCFDefinition to ValidConnectorIds.json for the detection template schema check.

Reason for Change(s):

  • New partner solution from PolySwarm, worked on with Microsoft App Assure. Partner Center enrollment is in progress; the publisherId in SolutionMetadata.json is the ID PolySwarm will register.

Version Updated:

  • Yes. New solution at 3.0.0; all analytics rules and hunting queries carry a version.

Testing Completed:

  • Yes. Installed the packaged solution in a Microsoft Sentinel workspace onboarded to the Microsoft Defender portal. Connected both data connectors and confirmed data arriving in ThreatIntelIndicators. Deployed all fifteen playbooks and ran the enrichment, URL and storage blob sandbox and quota playbooks on a live incident, both incident- and entity-triggered. Ran every analytics rule, hunting query, workbook query and connector page query against the workspace without errors. The two preview playbooks deploy cleanly but have not yet been run against a live Defender for Endpoint tenant or Exchange Online mailbox, which is why they are labelled preview.

Checked that the validations are passing and have addressed any issues that are present:

  • Yes. Ran .script/local-validation and ARM-TTK locally. YAML, JSON, logo, documents link, ID change, data connector, content branding, playbook, workbook template, solution, non-ASCII and ARM-TTK (48 of 48) checks pass for this solution. The detection template schema run passes for every PolySwarm template; its only failures are 14 existing templates elsewhere in the repository. KQL validation could not be run locally because it needs Microsoft's internal Kusto package.

Branch: polyswarm-solution, from a fork at polyswarm/Azure-Sentinel, into Azure/Azure-Sentinel master.

@erickingleby-polyswarm

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree company="Swarm Technologies Inc"

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved query correctness, entity mapping, workbook reporting, and validation issues block approval.

7 open issues
What changed in this PR

Introduces the PolySwarm 3.0.0 solution for Microsoft Sentinel, integrating threat intelligence, incident enrichment, sandbox detonation, and workspace monitoring.

Changes:

  • Adds two codeless connectors, three disabled analytics rules, and four hunting queries.
  • Adds fifteen playbooks with deployment, permissions, and migration documentation.
  • Adds status workbooks, solution metadata, branding, and connector-ID registrations.
File Description and review findings
Workbooks/​WorkbooksMetadata.json Registers the workbook. Moderate: Referenced logo is missing from Workbooks/Images/Logos, causing metadata validation to fail.
Workbooks/​PolySwarmStatus.json Adds the gallery dashboard. Moderate: Exact-name filtering excludes dynamically named malicious-file alerts.
Solutions/​PolySwarm/​Workbooks/​PolySwarmStatus.json Adds the packaged dashboard. Moderate: The alert tile likewise excludes dynamically named malicious-file alerts.
Solutions/​PolySwarm/​SolutionMetadata.json Defines publisher, support, and categories.
Solutions/​PolySwarm/​ReleaseNotes.md Records the initial release.
Solutions/​PolySwarm/​Playbooks/​PolySwarm-Sandbox-Url/​readme.md Documents URL detonation and safeguards.
Solutions/​PolySwarm/​Playbooks/​PolySwarm-Sandbox-StorageBlob/​readme.md Documents blob detonation and storage access.
Solutions/​PolySwarm/​Playbooks/​PolySwarm-Sandbox-OutlookAttachment/​readme.md Documents preview attachment detonation.
Solutions/​PolySwarm/​Playbooks/​PolySwarm-Enrich-Url/​readme.md Documents URL enrichment.
Solutions/​PolySwarm/​Playbooks/​PolySwarm-Enrich-Ip/​readme.md Documents IP enrichment and result interpretation.
Solutions/​PolySwarm/​Playbooks/​PolySwarm-Enrich-FileHash/​readme.md Documents hash enrichment and authentication.
Solutions/​PolySwarm/​Playbooks/​PolySwarm-Enrich-Domain/​readme.md Documents domain enrichment.
Solutions/​PolySwarm/​Playbooks/​PolySwarm-CheckQuota/​readme.md Documents quota checks and thresholds.
Solutions/​PolySwarm/​Package/​testParameters.json Listed generated-package path; excluded from review.
Solutions/​PolySwarm/​Hunting Queries/​PolySwarmThreatHuntUrl.yaml Correlates URLs with web sessions. Moderate: Whole-URL lowercasing conflates case-sensitive resources and changes the observed URL. Moderate: Metadata extraction reads event AdditionalFields, not indicator AdditionalFields1.
Solutions/​PolySwarm/​Hunting Queries/​PolySwarmThreatHuntIp.yaml Correlates IP indicators with network sessions. Moderate: Metadata extraction reads event fields instead of the indicator’s malware family and report link.
Solutions/​PolySwarm/​Hunting Queries/​PolySwarmThreatHuntHash.yaml Correlates hashes with file events. Moderate: Algorithm and Value must share one FileHash entity mapping. Moderate: Metadata extraction reads event fields rather than indicator metadata.
Solutions/​PolySwarm/​Hunting Queries/​PolySwarmThreatHuntDomain.yaml Correlates domains with DNS activity. Moderate: Metadata extraction reads DNS-event fields rather than indicator metadata.
Solutions/​PolySwarm/​docs/​upgrading.md Explains migration from earlier playbooks.
Solutions/​PolySwarm/​docs/​permissions.md Documents Azure and API permissions. Nit: Distinguish RBAC-only role assignments from the secret Get access policy needed for access-policy vaults.
Solutions/​PolySwarm/​Data/​Solution_PolySwarm.json Defines solution packaging inputs. Moderate: Descriptions on lines 5 and 44 exceed the required 1,000-character limit.
Solutions/​PolySwarm/​Data Connectors/​PolySwarmSandboxResults/​readme.md Listed connector documentation; excluded from review.
Solutions/​PolySwarm/​Data Connectors/​PolySwarmHunting_CCF/​readme.md Listed hunting-connector documentation; excluded from review.
Solutions/​PolySwarm/​Data Connectors/​PolySwarmHunting_CCF/​PolySwarmHunting_CCF_PollerConfig.json Listed hunting poller configuration; excluded from review.
Solutions/​PolySwarm/​Data Connectors/​PolySwarmHunting_CCF/​PolySwarmHunting_CCF_DataConnectorDefinition.json Listed hunting connector definition; excluded from review.
Solutions/​PolySwarm/​Data Connectors/​PolySwarmFeed_CCF/​readme.md Listed feed-connector documentation; excluded from review.
Solutions/​PolySwarm/​Data Connectors/​PolySwarmFeed_CCF/​PolySwarmFeed_CCF_PollerConfig.json Listed feed poller configuration; excluded from review.
Solutions/​PolySwarm/​Data Connectors/​PolySwarmFeed_CCF/​PolySwarmFeed_CCF_DCR.json Listed feed data-collection rule; excluded from review.
Solutions/​PolySwarm/​Data Connectors/​PolySwarmFeed_CCF/​PolySwarmFeed_CCF_DataConnectorDefinition.json Listed feed connector definition; excluded from review.
Solutions/​PolySwarm/​Analytic Rules/​PolySwarm_SandboxDetonationMalicious.yaml Detects malicious sandbox results.
Solutions/​PolySwarm/​Analytic Rules/​PolySwarm_MaliciousFileObserved.yaml Correlates hashes with security telemetry. Moderate: Strict unions fail when source tables are absent. Moderate: innerunique arbitrarily drops indicators sharing a hash; use inner to preserve their evidence.
Solutions/​PolySwarm/​Analytic Rules/​PolySwarm_HighConfidenceHuntHit.yaml Detects high-confidence hunt indicators. Moderate: Filtering before selecting the latest row per Id can alert on obsolete active/high-confidence states.
Logos/​PolySwarm.svg Adds solution branding.
.script/​tests/​detectionTemplateSchemaValidation/​ValidConnectorIds.json Registers both PolySwarm connector IDs.

Review effort: Balanced

"Name": "PolySwarm",
"Author": "PolySwarm - support@polyswarm.io",
"Logo": "<img src=\"https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Logos/PolySwarm.svg\" width=\"75px\" height=\"75px\">",
"Description": "[PolySwarm](https://polyswarm.io) is a threat intelligence platform built on a marketplace of competing detection engines. Files, URLs, IP addresses and domains are analyzed by multiple engines in parallel, their consensus is scored as a single PolyScore, PolyUnite identifies the malware family, and sandbox detonation adds behavioral analysis. This solution enriches Microsoft Sentinel incident entities (file hashes, URLs, IP addresses and domains) with PolySwarm's multi-engine malware verdicts, naming every engine that agreed, and lets an analyst detonate a URL, an email attachment, an Azure Storage blob, or a file retrieved directly from a device through Microsoft Defender for Endpoint, in the PolySwarm sandbox, returning a score-derived verdict, malware family, MITRE ATT&CK techniques and a report link on the incident. Data connectors bring PolySwarm threat intelligence, from an organization's own PolySwarm hunt results and from the PolySwarm curated feed, into Microsoft Sentinel's native threat intelligence table, so Microsoft's built-in indicator matching runs against PolySwarm data alongside the analytics rules and hunting queries this solution provides.\n\nCoverage by indicator type: MD5, SHA-1, SHA-256, URL, domain and IP are answered. Email addresses are not supported, because PolySwarm has no endpoint for them. A small PolySwarm status workbook shows what has arrived from each connector, reading only your own workspace.\n\nUnderlying Microsoft Technologies used:\nThis solution depends on underlying Microsoft technologies. Some of these dependencies either may be in Preview state or might result in additional ingestion or operational costs:\n* [Logic Apps](https://learn.microsoft.com/en-us/azure/logic-apps/logic-apps-pricing)\n* [Microsoft Sentinel Codeless Connector Framework](https://learn.microsoft.com/en-us/azure/sentinel/create-codeless-connector)\n* [Threat Indicators](https://learn.microsoft.com/en-us/azure/sentinel/upload-indicators-api)\n* [Key Vault](https://learn.microsoft.com/en-us/azure/key-vault/general/pricing)",
Comment on lines +48 to +51
- entityType: FileHash
fieldMappings:
- identifier: Algorithm
columnName: HashType
query: |
_Im_WebSession
| where isnotempty(Url)
| extend Url = tolower(Url)
"type": 3,
"content": {
"version": "KqlItem/1.0",
"query": "union isfuzzy=true SecurityAlert, (datatable(TimeGenerated:datetime, AlertName:string, AlertSeverity:string, Status:string)[])\n| where TimeGenerated {TimeRange}\n| where AlertName in~ ('PolySwarm high-confidence hunt hit', 'PolySwarm malicious file observed', 'PolySwarm sandbox detonation malicious')\n| summarize Alerts = count(), ['Last alert'] = max(TimeGenerated) by AlertName, AlertSeverity",
"type": 3,
"content": {
"version": "KqlItem/1.0",
"query": "union isfuzzy=true SecurityAlert, (datatable(TimeGenerated:datetime, AlertName:string, AlertSeverity:string, Status:string)[])\n| where TimeGenerated {TimeRange}\n| where AlertName in~ ('PolySwarm high-confidence hunt hit', 'PolySwarm malicious file observed', 'PolySwarm sandbox detonation malicious')\n| summarize Alerts = count(), ['Last alert'] = max(TimeGenerated) by AlertName, AlertSeverity",
},
{
"workbookKey": "PolySwarmStatusWorkbook",
"logoFileName": "PolySwarm.svg",
| Component | Role | Scope | Why this role and not a broader one |
| --- | --- | --- | --- |
| Every enrichment playbook (file hash, URL, IP, domain; both incident and entity trigger) | Microsoft Sentinel Responder | Resource group or Sentinel workspace | Lets the playbook read incident entities and add comments. Does not grant it the ability to change analytics rules, automation rules or workspace configuration, which **Microsoft Sentinel Contributor** would. |
| Every playbook that reads the PolySwarm API key from Key Vault | Key Vault Secrets User | The Key Vault, or the individual secret if the vault uses access-policy scoping | Read-only access to secret values. Does not grant the ability to create, delete or manage secrets, keys or certificates, which **Key Vault Administrator** or **Key Vault Secrets Officer** would. |
@v-maheshbh

Copy link
Copy Markdown
Contributor

Hi Erick Ingleby (@erickingleby-polyswarm)

Kindly check the Copilot comments. If the feedback has been addressed, please mark the corresponding comments as resolved.

Kindly add both black‑and‑white preview images inside solution workbook preview images folder, and also add the images and logo under the following path as well
https://github.com/Azure/Azure-Sentinel/tree/master/Workbooks/Images/Preview

Kindly attach a screenshot showing the CCF connector in a connected state for validation and review purposes
and relevant invocation logs demonstrating successful connector execution and data ingestion related to the changes introduced in this PR.

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Codeless Connector Framework (CCF) Connector New Solution For new Solutions which are new to Microsoft Sentinel

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants