elastic / elastic/integrations
[httpjson-pagination] Box Events cursor can regress to scientific notation and corrupt stream_position
- Dominant language
- Handlebars
- Stars
- 333
- Forks
- 647
- Avg merge
- 3d 4h
- Merged PRs (30d)
- 209
Description
## Findings
### 1. Box Events `next_stream_position` cursor can be emitted in scientific notation
**Severity:** High
**Location:**
- `packages/box_events/data_stream/events/agent/stream/httpjson.yml.hbs:20`
- `packages/box_events/data_stream/events/agent/stream/httpjson.yml.hbs:37`
- `packages/box_events/data_stream/events/agent/stream/httpjson.yml.hbs:42`
- Supporting sample payload: `packages/box_events/data_stream/events/_dev/test/pipeline/test-empty.log:1`
**Evidence:**
`httpjson.yml.hbs` directly interpolates cursor/page token values without integer/string normalization:
```yaml
# L20
value: '[[if index .cursor "next_stream_position"]][[.cursor.next_stream_position]][[else]][["0"]][[end]]'
# L37
value: '[[ if and (index .last_response.body "next_stream_position") (ge (len .last_response.body.entries) \{\{limit}}) ]][[.last_response.body.next_stream_position]][[end]]'
# L42
value: '[[.last_response.body.next_stream_position]]'
```
The package also contains a numeric `next_stream_position` sample:
```json
{"chunk_size":0,"entries":[],"next_stream_position":564670747717431}
```
Runtime reproduction of JSON round-trip behavior for this value:
- JSON unmarshal type: `float64`
- Rendered form: `5.64670747717431e+14`
So a numeric cursor value can be persisted and then sent as:
```text
...?stream_position=5.64670747717431e+14
```
instead of the expected decimal token.
**What is wrong:**
The stream assumes `next_stream_position` is always safe to interpolate as-is. If the API returns a numeric token (as seen in package test data), cursor JSON round-trip converts it to `float64`, and later template rendering emits scientific notation.
**Why it matters:**
`stream_position` is the pagination/cursor anchor for exactly-once collection. Scientific notation or precision drift can make the next request start from the wrong position, causing duplicate events, missed events, or stalled pagination.
**Suggested fix:**
Normalize `next_stream_position` before storing/using it in request params, e.g. enforce canonical decimal string/integer formatting in both request transform and cursor assignment (similar to the prior large-offset handling that existed before `bdc4247035`).
## Investigated and found correct
- `packages/m365_defender/data_stream/alert/agent/stream/httpjson.yml.hbs` and `.../incident/...`: cursor field is RFC3339 datetime string (`lastUpdateDateTime`), so no numeric scientific-notation drift path in cursor serialization.
- `packages/ti_eset/data_stream/ip/agent/stream/httpjson.yml.hbs`: cursor is string header/token based, not large numeric cursor state.
- `packages/panw_cortex_xdr/data_stream/incidents/agent/stream/httpjson.yml.hbs`: cursor/time filters are consistently handled as numeric values in request body without string interpolation of JSON-round-tripped cursor tokens.
> [!WARNING]
>
> ⚠️ Firewall blocked 1 domain
>
> The following domain was blocked by the firewall during workflow execution:
>
> - `developer.box.com`
>
> To allow these domains, add them to the `network.allowed` list in your workflow frontmatter:
>
> ```yaml
> network:
> allowed:
> - defaults
> - "developer.box.com"
> ```
>
> See [Network Configuration](https://github.github.com/gh-aw/reference/network/) for more information.
>
>
---
[What is this?](https://ela.st/github-ai-tools) | [From workflow: Sweeper: httpjson and CEL Pagination and Cursor Integrity](https://github.com/elastic/integrations/actions/runs/25209738540)
Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.
> - [x] expires on May 8, 2026, 9:47 AM UTC
Contributor guide
Assessment
This issue has not been assessed yet.