`chunk()` returns a sparse array when one event exceeds `maxKB`, stalling the event queue (regression in 2.23.0)
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 78/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Quiet
- Tech stack
- react-native, typescript
Research direction
Start in packages/core/src/util.ts with chunk and reproduce the sparse-array case using the standalone example; then inspect batching and error aggregation in packages/core/src/plugins/SegmentDestination.ts. Done means chunk returns dense batches, resets its size accumulator for each batch, preserves the count limit, and allows oversized-event responses to be processed so the queue drains.
Written by the indexing model from the issue text.
Description
analytics-react-nativeversion: 2.24.0 (regression introduced in 2.23.0; 2.21.3 and 2.22.0 unaffected)- Integrations versions (if used):
@segment/analytics-react-native-plugin-advertising-id,@segment/analytics-react-native-plugin-idfa - React Native version: 0.83.10 (Hermes, New Architecture, Expo SDK 55)
- iOS or Android or both? Both
chunk() in packages/core/src/util.ts can return a sparse array. Since 2.23.0 the upload path iterates its results with for...of, which does not skip holes, so a single oversized event now makes every flush throw and the event queue stops draining.
chunk assigns into an index rather than appending:
if (maxKB !== undefined) {
rollingKBSize += sizeOf(item);
if (rollingKBSize >= maxKB) {
chunks[++currentChunk] = [item]; // on index 0 this skips chunks[0]
return chunks;
}
}
When the first item alone is >= maxKB, currentChunk goes 0 -> 1 and chunks[0] is never created. MAX_PAYLOAD_SIZE_IN_KB is 500, so any single queued event serialising to 500KB or more triggers it.
Array.prototype.map preserves the hole, Promise.all resolves it to undefined, and aggregateErrors in packages/core/src/plugins/SegmentDestination.ts then reads result.status off undefined:
const results: BatchResult[] = await Promise.all(batches.map((batch) => this.uploadBatch(batch)));
const aggregation = this.aggregateErrors(results); // for (const result of results) { switch (result.status)
This was harmless before 2.23.0. 2.21.x used chunkedEvents.map(async (batch) => { ... }) and never iterated the results, so the hole was silently skipped and the oversized event simply never uploaded. 2.23.0 added aggregateErrors with for (const result of results), which made the same sparse array fatal.
Steps to reproduce
chunk and sizeOf are pure, so the sparse array reproduces standalone (both copied verbatim from packages/core/src/util.ts):
const sizeOf = (obj) => (encodeURI(JSON.stringify(obj)).split(/%..|./).length - 1) / 1024;
const chunk = (array, count, maxKB) => {
if (!array.length || !count) return [];
let currentChunk = 0, rollingKBSize = 0;
return array.reduce((chunks, item, index) => {
if (maxKB !== undefined) {
rollingKBSize += sizeOf(item);
if (rollingKBSize >= maxKB) { chunks[++currentChunk] = [item]; return chunks; }
}
if (index !== 0 && index % count === 0) { chunks[++currentChunk] = [item]; }
else { if (chunks[currentChunk] === undefined) chunks[currentChunk] = []; chunks[currentChunk].push(item); }
return chunks;
}, []);
};
// one event over MAX_PAYLOAD_SIZE_IN_KB (500), then two normal ones
const big = { messageId: 'a', properties: { blob: 'x'.repeat(520 * 1024) } };
const batches = chunk([big, { messageId: 'b' }, { messageId: 'c' }], 100, 500);
// [ <1 empty item>, [ {messageId:'a'} ], [ {messageId:'b'} ], [ {messageId:'c'} ] ]
// ^ the hole ^ 'b' and 'c' should have shared a batch;
// rollingKBSize is never reset, so they don't
console.log(batches);
console.log(0 in batches); // false <-- hole
Promise.all(batches.map((b) => ({ status: 'success', messageIds: [] }))).then((results) => {
for (const result of results) { void result.status; } // TypeError
});
In-app, track any event whose serialised size is >= 500KB while it is the only or first entry in the persisted queue, then let a flush policy fire.
Expected behavior
chunk returns a dense array of non-empty batches. An item that alone exceeds maxKB gets its own batch; the server rejects it with a 4xx, default4xxBehavior: 'drop' drops it, and the queue continues to drain.
Actual behavior
errorHandler receives ErrorType.FlushError with Flush failed: TypeError: Cannot read property 'status' of undefined (Hermes wording) on every flush.
Because the throw escapes sendEvents after Promise.all has uploaded the batches but before processUploadResults runs, nothing is ever dequeued. The queue therefore never drains, the same events are re-uploaded on every flush, and the device recovers only when pruneExpiredEvents discards the events at maxTotalBackoffDuration — 12 hours by default. We saw this on roughly 200 devices across both platforms in a single release, each reporting a FlushError every 30 seconds.
There is a second, independent problem in the same function: rollingKBSize is never reset when a new chunk starts. Once the cumulative size crosses maxKB, the size branch fires for every remaining item and each one becomes its own batch, so the count limit becomes unreachable. Measured with 1200 events of about 1KB: 700 batches, 699 of them single-event, against 3 batches once fixed. That is 700 HTTP requests per flush where 3 would do.
Suggested fix
Build the chunks by appending, and reset the accumulator per chunk. This removes the hole, restores the count limit, and keeps an oversized item isolated in its own batch:
export const chunk = <T>(array: T[], count: number, maxKB?: number): T[][] => {
if (!array.length || !count) {
return [];
}
let rollingKBSize = 0;
return array.reduce((chunks: T[][], item: T) => {
const itemKBSize = maxKB === undefined ? 0 : sizeOf(item);
const currentChunk = chunks[chunks.length - 1];
const isOverMaxKB = maxKB !== undefined && rollingKBSize + itemKBSize >= maxKB;
if (currentChunk === undefined || currentChunk.length >= count || isOverMaxKB) {
rollingKBSize = itemKBSize;
chunks.push([item]);
return chunks;
}
rollingKBSize += itemKBSize;
currentChunk.push(item);
return chunks;
}, []);
};
Happy to open a PR if that would help.
- Dominant language
- TypeScript
- Stars
- 383
- Forks
- 206
- Avg merge
- 14h 45m
- Merged PRs (30d)
- 11
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from segmentio/analytics-react-native
-
bug investigate
Difficulty 1/5 Under an hour Newbie friendliness 72/100
segmentio/analytics-react-native#1145 ·
-
enhancement
Difficulty 4/5 3-5 days Newbie friendliness 68/100
segmentio/analytics-react-native#1328 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 45/100
segmentio/analytics-react-native#1124 · 3 comments ·
-
enhancement
Difficulty 3/5 1-2 days Newbie friendliness 48/100
segmentio/analytics-react-native#1114 ·
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 55/100
segmentio/analytics-react-native#1109 ·
All issues in segmentio/analytics-react-native
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
area:tools bug good first issue help wanted priority:P2
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
TaewoooPark/Motifcode#14 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
newrelic-experimental/preflight#793 · 1 comment ·
-
bug 🐞
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
[Bounty proposal] fix(web): memory insights count an evening memory on the next day ($25 proposed) Open
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
BasedHardware/omi#15320 ·