influxdata / influxdata/giraffe

Allow parsing of partial CSV strings

Open
#332 0 comments 0 reactions 0 assignees View on GitHub
team/ui
Dominant language
TypeScript
Stars
189
Forks
33
Avg merge
3d 15h
Merged PRs (30d)
4

Description

There are some cool things going on with the `influxdb-client-js` library (namely utilizing the streaming interface it exposed from `fetch`), that we can't support, because we can't parse partial CSVs in the fromFlux parser.

There have been talks about appending / truncating the data based on a shifting time frame (ie: progressive updates for dashboards). This is not [that](https://github.com/influxdata/influxdb/issues/19793). There's also talks about adding a pagination interface to expose partial views of the dataset. This is not that.

Instead of waiting for the full CSV to come in from the API before parsing, we want to be able to generate an instance of a parser, and append partial CSV strings like:
```
#group,false,false,tr
```

or

```
true,true,false,false
#datat
```

or

```
,,2,2020-10-23T15:14:52.865415455Z,2020-10-23T16:14:52.865415455Z,usage_percent,docker_container_cpu,plexinc/pms-docker,plex,runnin
```

and have that append rows to the data set as they come in

Contributor guide

Open the contributing guide

Research direction

Start with the fromFlux parser entry point named in the issue and review how it currently consumes a complete CSV string. Define the incremental parser behavior using the partial CSV examples, then verify that appended fragments produce the corresponding rows as data arrives.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.