An easy method to count the total number of time-series data
- Dominant language
- Java
- Stars
- 6.4k
- Forks
- 1.2k
- Avg merge
- 1d 23h
- Merged PRs (30d)
- 115
Description
I used workbench to write a large number of data into iotdb, e.g. 50000 devices and 100 sensors for each device. Then I wanted to verify whether the total number of data actually written into iotdb is equal to the number expected.
I used sql statements like "select count(*) from root.test.** group by level =3", but can't get the correct answer which should be 50000\*100. It seems like that result columns over 1000 has been truncated.
What I expect is, when I execute one single sql statement, it can return the correct number of all data in db. In industry, large data is ubiquitous, so a good support for large data might be a common scene.
Now, I used a sql and shell combined way to verify the number, such like "select count(*) from root.test.**.d_${i} group by level =3". Here i ranges from 1 to 50000. And finally add 50000 results together to get the correct number.
Contributor guide
Research direction
Start by reproducing the single query `select count(*) from root.test.** group by level =3` with more than 1000 result columns and compare it with the expected 50000×100 count. Trace the query result handling to identify where columns are truncated. Done means one SQL statement returns the complete count for the reported large dataset.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, sql
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100