esnet / esnet/pond

Time-Index in Milliseconds?

Open
#191 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
TypeScript
Stars
211
Forks
42
PR merge metrics
No merged PRs in 30d

Description

I am looking to do data aggregation of events that are streaming into my timeseries graph. Currently I am using Pond's `pipeline` and `ring` objects to "stream" data into a graph. I am receiving nearly 60 pieces of data per second (meaning I have timestamps on each datapoint that go out to the millisecond). Although realistically I am only able to process about 30 data-points per second into the graph. Since I am getting 60 data-points per second, I would like to us an windowBy function on the pipeline to group the data-points in a 34 millisecond window based on their timestamp. Which is where I run into a problem. I don't see any examples of a time index based on milliseconds. I see "5m" for 5 minutes, or "1s" for 1 second. Can I window by milliseconds? I think this would be very useful for high-volume data streaming.

I am relatively new to timeseries. So if this is off the mark because of my lack of understanding of a Time Index or how the streams can work in other ways, I'd be happy to hear about it.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the pipeline and ring usage described in the issue, then inspect windowBy's time-index handling and existing examples for units such as 5m and 1s. Determine whether millisecond windows are supported and what behavior should be verified; the issue names no files or tests, and its only comment is over five years old.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
stream-processing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.