influxdata / influxdata/influxdb

influx-toos import panic: keys must be added in sorted order

Open
#21,575 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
31.7k
Forks
3.7k
Avg merge
13h 37m
Merged PRs (30d)
8

Description

__Steps to reproduce:__
List the minimal actions needed to reproduce the behavior.

1. Run a influxdb 1.8.6
2. Execute the following `write.sh`
```
#!/bin/bash

curl -X POST 'http://127.0.0.1:8086/query' --data-urlencode 'q=CREATE DATABASE test'

echo ""

curl -i -X POST 'http://127.0.0.1:8086/write?db=test' --data-binary \
'kpi.ebip.ret-call.cpu1,host=server02 value=0.67,id=2i,running=true,status="ok"
kpi.ebip.ret-call.\ \ .cpu1,host=server02,region=us-west value=0.55,id=2i,running=true,status="ok"
cpu1,host=server01,region=us-east,direction=in value=2.0,id=1i,running=false,status="fail"'

curl -i -X POST 'http://127.0.0.1:8086/write?db=test&precision=s' --data-binary \
'kpi.ebip.ret-call.cpu2,host=server02 float=1422568543,long=1422568544i,running=true,status="ok"
kpi.ebip.ret-call.\ \ .cpu2,host=server03,region=us-east float=1422568543702,long=1422568543703i 1422568543
cpu2,host=server04,region=us-west,direction=out float=1422568543702568543,long=1422568543702568544i,running=false,status="fail"'

curl -i -X POST 'http://127.0.0.1:8086/write?db=test&precision=h' --data-binary \
'kpi.ebip.ret-call.cpu3,host=server05,region=cn\ north,tag\ key=tag\ value idle=64,system=1i,user="Dwayne Johnson",admin=true
kpi.ebip.ret-call.\ \ .cpu3,host=server06,region=cn\ south,tag\ key=value\=with"equals" idle=16,system=16i,user="Jay Chou",admin=false
cpu3,host=server07,region=cn\ south,tag\ key=value\,with"commas" idle=74,system=23i,user="Stephen Chow"'

curl -i -X POST 'http://127.0.0.1:8086/write?db=test&precision=ms' --data-binary \
'kpi.ebip.ret-call.\ \ .cpu\ idle=14,system=31i,user="Dwayne Johnson",admin=true,character="\", ,\,\\,\\\,\\\\"
kpi.ebip.ret-call.\ \ .cpu\ idle=39,system=56i,user="Jay Chou",brief\ desc="the best \"singer\""
kpi.ebip.ret-call.\ \ .cpu\ idle=47,system=93i,user="Stephen Chow",admin=true,brief\ desc="the best \"novelist\""'

curl -i -X POST 'http://127.0.0.1:8086/write?db=test' --data-binary \
'measurement\ with\ spaces\,\ commas\ and\ "quotes",tag\ key\ with\ equals\ \==tag\ value\ with"spaces" field_k\ey\ with\ \=="string field value, multiple backslashes \,\\,\\\,\\\\"
"measurement\ with\ spaces\,\ commas\ and\ "quotes"",tag\ key\ with\ equals\ \==tag\,value\,with"commas" field_k\ey\ with\ \=="string field value, only \" need be escaped"'
```
3. Stop influxdb
4. Run influx-toos export and import: `influx_tools export -config influxdb.conf -no-conflict-path -database db -format binary | influx_tools import -config influxdb-import.conf -database test -rp autogen -build-tsi`
5. See error
![image](https://user-images.githubusercontent.com/4902714/119947648-8e803d00-bfca-11eb-82e7-b22b4e9b90fb.png)
![image](https://user-images.githubusercontent.com/4902714/119947749-a952b180-bfca-11eb-9bce-07835031dae3.png)

__Expected behavior:__
Bug fixed

__Actual behavior:__
Panic occured

__Environment info:__

1.8.6

__Config:__
default

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the panic using the supplied write.sh data and the `influx_tools export ... | influx_tools import ...` command, with the default configuration. Start by tracing the import path that reports “keys must be added in sorted order.” Done means the provided dataset imports without a panic.

Written by the indexing model from the issue text.

Assessment

Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.