elastic / elastic/beats

[Salesforce] Support batching mechanism to process large amount of events

Open
#48,196 0 comments 0 reactions 0 assignees View on GitHub
enhancement Team:Obs-InfraObs
Dominant language
Go
Stars
12.7k
Forks
5k
Avg merge
2d 54m
Merged PRs (30d)
381

Description

When dealing with high-volume Salesforce environments, the LoginEvents (and others) query may fail to return data and eventually time out (on the Salesforce side). This occurs because Salesforce must accumulate, sort, and batch a large number of rows before returning results to the client — placing substantial load on the Salesforce backend.

## Root Cause

- Salesforce does not support ORDER BY ASC for Login and Logout events
- While sorting by EventDate is supported, only descending order is allowed
- Configuring a shorter initial_interval does not resolve the issue, as Salesforce still processes the full dataset before returning results

## Proposed Enhancement

Introduce a more efficient batching strategy in the Salesforce module that minimizes server-side load and prevents query timeouts.

## Expected Impact

- Improved reliability when collecting large volumes of Login and Logout events
- Reduced query timeouts from Salesforce API
- Better scalability for high-volume environments

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.