Skip to content

feat: enable data filtering for batches#1245

Merged
jaissica12 merged 16 commits intodevelopmentfrom
feat/add-batch-filtering-before-kit-forwarding
Apr 13, 2026
Merged

feat: enable data filtering for batches#1245
jaissica12 merged 16 commits intodevelopmentfrom
feat/add-batch-filtering-before-kit-forwarding

Conversation

@jaissica12
Copy link
Copy Markdown
Contributor

@jaissica12 jaissica12 commented Apr 9, 2026

Background

  • Kits that implement processBatch receive the full unfiltered batch from the SDK. Unlike individual events (which go through sendEventToForwarders with 5 types of data filtering), batches were forwarded without any filtering applied. This meant kits receiving batches could see events, user attributes, user identities, and event attributes that should have been blocked or stripped based on dashboard-configured filter rules.

What Has Changed

  • Applied the same 5 data filters to sendBatchToForwarders that already exist in sendEventToForwarders: attribute forwarding rules, event name/type filters, event attribute filters, user identity filters, and user attribute filters
  • Extracted shared filter helper functions (isBlockedByForwardingRule, isBlockedByEventFilter, filterEventAttributes, filterUserIdentities) from inline code in sendEventToForwarders so both event and batch paths reuse the same logic
  • Added reverse-mapping functions in sdkToEventsApiConverter.ts (getMessageTypeFromEventType, getEventCategoryFromCustomEventType, getIdentityTypeFromBatchKey, getEventNameFromBatchEvent) to translate batch snake_case fields back to SDK numeric types needed by the filter functions
  • If all events in a batch are filtered out, processBatch is skipped entirely for that forwarder

Screenshots/Video

1 2 3 4 5

Checklist

  • I have performed a self-review of my own code.
  • I have made corresponding changes to the documentation.
  • I have added tests that prove my fix is effective or that my feature works.
  • I have tested this locally.

Additional Notes

  • {Any additional information or context relevant to this PR}

Reference Issue (For employees only. Ignore if you are an outside contributor)

  • Closes NA

@jaissica12 jaissica12 marked this pull request as ready for review April 9, 2026 16:15
@jaissica12 jaissica12 requested a review from a team as a code owner April 9, 2026 16:15
@cursor
Copy link
Copy Markdown

cursor bot commented Apr 9, 2026

PR Summary

Medium Risk
Changes the kit forwarding pipeline to filter and mutate processBatch payloads (events, attributes, identities) similarly to single-event forwarding, which could alter what downstream kits receive. Risk is moderate due to behavioral changes in core forwarding logic and new mapping/filter utilities.

Overview
Adds batch-level kit filtering so sendBatchToForwarders now removes blocked events and strips filtered event/user data (attribute forwarding rules, event name/type filters, event/screen attribute filters, and user identity/attribute filters), and skips calling processBatch when a batch is fully filtered out.

Refactors the single-event forwarder path to reuse new shared helpers in forwarder-utils.ts, adds reverse-mapping helpers in types.ts/sdkToEventsApiConverter.ts to interpret batch event payloads for filtering, and moves the SDK-wide extend implementation into utils.ts (updating call sites and tests accordingly).

Reviewed by Cursor Bugbot for commit 3fb2754. Bugbot is set up for automated code reviews on this repo. Configure here.

Comment thread src/forwarders.js Outdated
Comment thread src/sdkToEventsApiConverter.ts Outdated
Comment thread src/sdkToEventsApiConverter.ts Outdated
Comment thread src/forwarders.js Outdated
Comment thread dist/types/src/forwarders.d.ts Outdated
Comment thread src/forwarders.js Outdated
Comment thread src/forwarders.js Outdated
Comment thread test/src/tests-forwarders.ts
Comment thread src/forwarder-utils.ts
Comment thread src/sdkToEventsApiConverter.ts Outdated
Comment thread src/forwarders.js
Comment thread src/forwarder-utils.ts
Comment thread src/forwarder-utils.ts
Copy link
Copy Markdown

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 49e0cc8. Configure here.

Comment thread src/forwarder-utils.ts Outdated
Comment thread src/forwarder-utils.ts Outdated
Comment thread src/forwarder-utils.ts Outdated
Comment thread src/forwarders.js Outdated
Comment thread src/forwarders.js Outdated
Comment thread src/types.ts
@rmi22186
Copy link
Copy Markdown
Member

Overall this is such a great improvement vs the mess that forwarders.js was prior. Great work not just migrating to TS, but cleaning and refactoring the logic to make it more understandable.

@jaissica12 jaissica12 requested a review from rmi22186 April 13, 2026 13:26
Comment thread test/src/tests-forwarders.ts Outdated
Comment thread test/src/tests-forwarders.ts Outdated
Comment thread test/src/tests-forwarders.ts Outdated
@sonarqubecloud
Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
31.0% Duplication on New Code (required ≤ 3%)

See analysis details on SonarQube Cloud

@jaissica12 jaissica12 merged commit ce56ddb into development Apr 13, 2026
50 of 55 checks passed
github-actions bot pushed a commit that referenced this pull request Apr 13, 2026
# [2.64.0](v2.63.0...v2.64.0) (2026-04-13)

### Features

* enable data filtering for batches ([#1245](#1245)) ([ce56ddb](ce56ddb))
@mparticle-automation
Copy link
Copy Markdown
Collaborator

🎉 This PR is included in version 2.64.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants