influxdata / influxdata/influxdb

[master] Influx shell: CSV format with > 10k results includes repeated header lines

Open
#6,415 7 comments 5 reactions 0 assignees View on GitHub
1.x area/cli kind/enhancement
Dominant language
Rust
Stars
31.7k
Forks
3.7k
Avg merge
13h 37m
Merged PRs (30d)
8

Description

### Bug report

**System info:**

OSX, InfluxDB built from source @ 7a1274790c943dadda7648960343510b1e4c5bd3

**Steps to reproduce:**

Use the Influx shell to print out more than 10k results from a single series, e.g.

```
influx -format csv -execute 'SELECT * FROM m WHERE ...'
```

**Expected behavior:**

Header line should only occur once, at top of file.

**Actual behavior:**

Header line repeats every 10k lines, presumably as a result of chunking in the JSON response, i.e.

``` sh
$ influx -database b3l182bzucoz_0 -format csv -execute "select n from ctr limit 10005" > /tmp/repro.csv

$ head /tmp/repro.csv
name,time,n
ctr,1460943429781749937,0
ctr,1460943429781792719,1
ctr,1460943429781794333,2
ctr,1460943429781795564,3
ctr,1460943429781796721,4
ctr,1460943429781797889,5
ctr,1460943429781799065,6
ctr,1460943429781800269,7
ctr,1460943429781801454,8

$ tail /tmp/repro.csv # header repeated after 10k records
ctr,1460943429794417363,9996
ctr,1460943429794418473,9997
ctr,1460943429794419598,9998
ctr,1460943429794420719,9999
name,time,n
ctr,1460943429794421828,10000
ctr,1460943429794422928,10001
ctr,1460943429794424085,10002
ctr,1460943429794425278,10003
ctr,1460943429794426393,10004
```

**Additional info:**

For anyone else running into this, here's an awk one-liner to strip the header from the rest of the file, as a workaround:

```
awk '{if(NR == 1){hdr=$0; print $0} else if($0 != hdr){print $0}}' YOURFILE.csv
```

Contributor guide

Open the contributing guide

Research direction

Run the provided influx shell command with CSV output and more than 10,000 results, then trace how response chunks are rendered. Done means the generated CSV has one header at the top and all result rows without repeated headers; add or update coverage if the project provides a relevant test path.

Written by the indexing model from the issue text.

Assessment

Domain
cli
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.