influxdata / influxdata/influxdb

InfluxDB 2.x: Query measurements with different data types = hangs/timeout

Open
#20,980 4 comments 0 reactions 0 assignees View on GitHub
area/2.x area/flux area/queries
Dominant language
Rust
Stars
31.7k
Forks
3.7k
Avg merge
13h 37m
Merged PRs (30d)
8

Description

Unfortunately I have a measurement that have fields with an number as the “_value” but the type is a **string instead of an integer.** _(This because I made a mistake in the script that writes the data to the InfluxDB.)_

__What I want:__
**I want to change the data type of this fields for the whole bucket.**
My Idea was to get all the data, convert the data types and writes it to another bucket like this:
```
datatoconvert = from(bucket: "telegraf")
|> range(start: v.timeRangeStart, stop: v.timeRangeStop)
|> filter(fn: (r) => r._measurement == "asset_vm_cmdb")
|> toInt()

datatoconvert
|> filter(fn: (r) => exists r._value)
|> to(bucket: "typeconversion", org: "")
```

__Steps to reproduce:__
1. Write data to an InfluxDB: Some fields _value as integer (double in Influx) and some fields _value a string.
2. Query this data in the InfluxDB Explore GUI and write this to a new bucket

__Expected behavior:__
I can query the data, convert the data types of the all "_values" and write this to a new bucket.

__Actual behavior:__
The **InfluxDB Explore GUI hangs after some seconds** and did not respond. Even when I use a short timerange this always hangs. More information's in the community post (https://community.influxdata.com/t/change-type-from-string-to-integer)

__Environment info:__
* System info: Linux 4.18.0-240.10.1.el8_3.x86_64 x86_64
* InfluxDB version: InfluxDB 2.0.4 (git: 4e7a59bb9a) build_date: 2021-02-08T17:47:02Z & InfluxDB 2.0.2 (git: 84496e507a) build_date: 2020-11-19T03:59:35Z
* VM sizing: (8 Cores & 64GB RAM)
* Data sizing: Per each interval my measurement has approx. 2800 rows with 40 tags.

__Logs:__
I found no error logs with `sudo journalctl -u influxdb.service`.

First, please read this community post: https://community.influxdata.com/t/change-type-from-string-to-integer

Contributor guide

Open the contributing guide

Research direction

Start by reading the linked community post and reproducing the mixed-type query in the InfluxDB Explore GUI using the supplied Flux and short time range. Compare the hanging behavior with the expected conversion and write to the new bucket; done means the query completes without the GUI timing out and the converted data is written.

Written by the indexing model from the issue text.

Assessment

Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.