influxdata / influxdata/influxdb
where clause with time does not work
- Dominant language
- Rust
- Stars
- 31.7k
- Forks
- 3.7k
- Avg merge
- 13h 37m
- Merged PRs (30d)
- 8
Description
List the minimal actions needed to reproduce the behavior.
1. We migrated influxdb into another server by physically moving whole data directory.
Influxdb service was ran with same influxdb.conf copied from old server.
Describe what you expected to happen.
We do have data keep being stored, but 'where time >=' clause does not collect ones from 2020-09-22.
It happens on only few specific databases ("monitoring"), time filter works well in another databases.
Describe What actually happened.
**1) using time filter >= 2020-09-22**
> select * from service_count where time >= '2020-09-22T00:00:00Z';
name: service_count
-------------------
time value
2020-09-22T00:00:00.14352484Z 57
2020-09-22T01:00:00.060768845Z 57
2020-09-22T02:00:00.058494673Z 57
2020-09-22T03:00:00.067200599Z 57
2020-09-22T04:00:00.039242917Z 57
2020-09-22T05:00:00.160716641Z 57
2020-09-22T06:00:00.856101018Z 57
2020-09-22T07:00:00.086886293Z 57
2020-09-22T08:00:00.085656305Z 57
2020-09-22T09:00:00.071923318Z 57
2020-09-22T10:00:00.053328845Z 57
**2) order by time desc to see if we actually have data stored after 2020-09-23**
> select * from service_count order by time desc limit 30;
name: service_count
-------------------
time value
2020-10-22T06:01:06.531931291Z 57
2020-10-22T05:01:06.117032605Z 57
2020-10-22T04:01:05.780487894Z 57
2020-10-22T03:01:05.406677534Z 57
2020-10-22T02:01:05.190119718Z 57
2020-10-22T01:01:05.036090857Z 57
2020-10-22T00:01:04.355914658Z 57
2020-10-21T23:01:04.564889203Z 57
2020-10-21T22:01:03.644918806Z 57
2020-10-21T21:01:03.299852097Z 57
2020-10-21T20:01:02.92402566Z 57
2020-10-21T19:01:02.585879266Z 57
2020-10-21T18:01:02.228306161Z 57
2020-10-21T17:01:02.374265115Z 57
2020-10-21T16:01:01.553891477Z 57
2020-10-21T15:01:01.922276162Z 57
2020-10-21T14:01:01.36392151Z 57
2020-10-21T13:01:00.45601343Z 57
2020-10-21T12:01:00.09353682Z 57
2020-10-21T11:00:59.757545374Z 57
2020-10-21T10:00:59.380965046Z 57
2020-10-21T09:00:59.067704466Z 57
2020-10-21T08:00:58.682240904Z 57
2020-10-21T07:00:58.765410316Z 57
2020-10-21T06:00:57.969622146Z 57
2020-10-21T05:00:58.326654539Z 57
2020-10-21T04:00:57.275257078Z 57
2020-10-21T03:00:57.103636784Z 57
2020-10-21T02:00:57.115618686Z 57
2020-10-21T01:00:56.233867699Z 57
**3) where clause with "value" works correctly.**
> select * from service_count where value=57 order by time desc limit 10;
name: service_count
-------------------
time value
2020-10-22T06:01:06.531931291Z 57
2020-10-22T05:01:06.117032605Z 57
2020-10-22T04:01:05.780487894Z 57
2020-10-22T03:01:05.406677534Z 57
2020-10-22T02:01:05.190119718Z 57
2020-10-22T01:01:05.036090857Z 57
2020-10-22T00:01:04.355914658Z 57
2020-10-21T23:01:04.564889203Z 57
2020-10-21T22:01:03.644918806Z 57
2020-10-21T21:01:03.299852097Z 57
* System info: Linux 5.4.0-1028-aws x86_64
* InfluxDB version: v1.0.0
* Other relevant environment details:
**tsm file structure for the databse**
root@ip-10-2-2-6:/iosp/influxdb/data/monitoring/autogen# ll
total 20
drwx------. 5 influxdb influxdb 4096 Aug 4 00:00 ./
drwx------. 4 influxdb influxdb 4096 Sep 28 2017 ../
drwxr-xr-x. 2 influxdb influxdb 4096 Oct 22 07:55 12572/
drwxr-xr-x. 2 influxdb influxdb 4096 Aug 4 03:55 12782/
drwxr-xr-x. 2 influxdb influxdb 4096 Sep 23 11:24 12938/
root@ip-10-2-2-6:/iosp/influxdb/data/monitoring/autogen# ll ./12938
total 1780
drwxr-xr-x. 2 influxdb influxdb 4096 Sep 23 11:24 ./
drwx------. 5 influxdb influxdb 4096 Aug 4 00:00 ../
-rw-r--r-- 1 influxdb influxdb 1812504 Sep 23 11:24 000017541-000000002.tsm
root@ip-10-2-2-6:/iosp/influxdb/data/monitoring/autogen# ll ./12572
total 4124
drwxr-xr-x. 2 influxdb influxdb 4096 Oct 22 07:55 ./
drwx------. 5 influxdb influxdb 4096 Aug 4 00:00 ../
-rw-r--r-- 1 influxdb influxdb 3104091 Sep 22 12:10 000029914-000000004.tsm
-rw-r--r-- 1 influxdb influxdb 1091983 Oct 22 07:01 000040508-000000005.tsm
-rw-r--r-- 1 influxdb influxdb 2038 Oct 22 07:10 000040512-000000003.tsm
-rw-r--r-- 1 influxdb influxdb 403 Oct 22 07:26 000040516-000000003.tsm
-rw-r--r-- 1 influxdb influxdb 308 Oct 22 07:46 000040520-000000003.tsm
-rw-r--r-- 1 influxdb influxdb 239 Oct 22 07:51 000040521-000000001.tsm
Contributor guide
Research direction
Start by reproducing the time predicate query against the migrated monitoring database and compare it with the value filter and descending time query. Inspect the TSM files under data/monitoring/autogen, especially shard directories 12572, 12782, and 12938. Done means time filters return the same post-migration records that ordering exposes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100