[Salesforce] Support batching mechanism to process large amount of events
- Dominant language
- Go
- Stars
- 12.7k
- Forks
- 5k
- Avg merge
- 2d 54m
- Merged PRs (30d)
- 381
Description
When dealing with high-volume Salesforce environments, the LoginEvents (and others) query may fail to return data and eventually time out (on the Salesforce side). This occurs because Salesforce must accumulate, sort, and batch a large number of rows before returning results to the client — placing substantial load on the Salesforce backend.
## Root Cause
- Salesforce does not support ORDER BY ASC for Login and Logout events
- While sorting by EventDate is supported, only descending order is allowed
- Configuring a shorter initial_interval does not resolve the issue, as Salesforce still processes the full dataset before returning results
## Proposed Enhancement
Introduce a more efficient batching strategy in the Salesforce module that minimizes server-side load and prevents query timeouts.
## Expected Impact
- Improved reliability when collecting large volumes of Login and Logout events
- Reduced query timeouts from Salesforce API
- Better scalability for high-volume environments
Contributor guide
Assessment
This issue has not been assessed yet.