influxdata / influxdata/influxdb-comparisons
Bench-Marker tool is not able to read queries from a file having 1M queries
- Dominant language
- Go
- Stars
- 313
- Forks
- 110
- PR merge metrics
- No merged PRs in 30d
Description
Use case : "Identifying a data base which supports 1.08 million events per hour"
Steps I am following :
1) I am populating data in InfluxDB by using Java API. ( Not using `bulk_data_gen` and bulk_data_load tools ) .
2) In my Data model, I have one measurement with 1 million rows. All tags and keys are String field.
3) now I am using "bulk_query_gen " tool to generate queries and write it into a file.
I have modified the tool to create queries like "select from measurement where time < X and Time < y"
**Bulk_gen_query tool**
` ./bulk_query_gen --debug=0 --seed=321 --format=influx-http --timestamp-end="2016-01-18T14:44:00Z" -queries=1000000 --query-type="1-host-1-hr" --db="benchmark" --debug 1 | gzip > EventQueries1M.gz`
** Query_Benchmarker_tool**
bash-3.2$
`cat EventQueries1M.gz | gunzip | ./query_benchmarker_influxdb --url=http://localhost:8086 --print-interval=0 –limit=1000 --workers=2 --print-responses=true -- db="benchmark" --debug 1`
`**2017/01/27 21:45:51 Error during request: dialing to the given TCP address timed out**`
( If the query size is > 10k, I am running into this issue )
Please note, I could reproduce this issue without making any changes to the benchmarking tool.
Contributor guide
No contributing guide indexed for this repository
Research direction
The report names the bulk_query_gen and query_benchmarker_influxdb entry points. First reproduce the pipeline with EventQueries1M.gz, gunzip, and more than 10k queries, then inspect where query_benchmarker_influxdb handles input and request timeouts. Done means the benchmarker can process the large query file without the reported TCP timeout.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, java
- Domain
- cli, databases, tooling
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100