box / box/box-python-sdk

Implement generate_events_with_long_polling for enterprise event streams

Open
#734 7 comments 0 reactions 4 assignees Claimed by @mwwoda View on GitHub
enhancement
Dominant language
Python
Stars
459
Forks
223
Avg merge
8h 57m
Merged PRs (30d)
13

Description

### Is your feature request related to a problem? Please describe.
Yes. We're seeing far fewer events when streaming enterprise events via the Box Python SDK in comparison to the Node one, which we want to migrate off of due to better Kafka support in Python.

In order to switch from Node to Python, I believe that we need `generate_events_with_long_polling()` to be implemented for the enterprise event stream type.

You can see the warning that this is not implemented [here in source](https://github.com/box/box-python-sdk/blob/main/boxsdk/object/events.py#L238) and [here in docs](https://box-python-sdk.readthedocs.io/en/latest/boxsdk.object.html#boxsdk.object.events.Events.generate_events_with_long_polling).

The reason I believe we need `generate_events_with_long_polling()` is because it allows you to make one call and continuously receive events from that position in the stream forward (like we do in Node). Trying to do that manually has the following problem: you need to manually track your position in the stream, and positions in the stream are constantly moving as data falls off the end of the retention window. i.e. The positions are not static, they're moving as data comes in, so there's no reliable way to keep fetching data in order. We've been told by Box devs in the past that manually managing the stream position is unreliable for that reason, and will result in data loss and duplication.

### Describe the solution you'd like
Implement `generate_events_with_long_polling()` for the enterprise event stream type.

### Describe alternatives you've considered
Staying on Node for the time being. This is the only alternative I can think of, as we can't seem to reliably capture all events in our enterprise instance using the Python SDK.

### Additional context
I've reached out via a support ticket with Box through my company IBM, and they suggested also opening an issue here to communicate with the SDK team directly. Sakora said:

> It looks like our Python SDK team would need to implement this functionality into our SDK. I have reached out to our Python SDK team for further clarification when this will be implemented. You can also reach out directly to our Python SDK team by opening a GitHub Issues ticket in the GitHub repo to communicate directly with them. I will provide an update once I receive a response back from our Python SDK team.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.