apache / apache/iotdb

An easy method to count the total number of time-series data

Open
#6,762 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
6.4k
Forks
1.2k
Avg merge
1d 23h
Merged PRs (30d)
115

Description

I used workbench to write a large number of data into iotdb, e.g. 50000 devices and 100 sensors for each device. Then I wanted to verify whether the total number of data actually written into iotdb is equal to the number expected.

I used sql statements like "select count(*) from root.test.** group by level =3", but can't get the correct answer which should be 50000\*100. It seems like that result columns over 1000 has been truncated.

What I expect is, when I execute one single sql statement, it can return the correct number of all data in db. In industry, large data is ubiquitous, so a good support for large data might be a common scene.

Now, I used a sql and shell combined way to verify the number, such like "select count(*) from root.test.**.d_${i} group by level =3". Here i ranges from 1 to 50000. And finally add 50000 results together to get the correct number.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the single query `select count(*) from root.test.** group by level =3` with more than 1000 result columns and compare it with the expected 50000×100 count. Trace the query result handling to identify where columns are truncated. Done means one SQL statement returns the complete count for the reported large dataset.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, sql
Domain
databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.