influxdata / influxdata/influxdb

InfluxDB Cloud csv.form() does not accept dateTime:number format

Open
#21,012 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
31.7k
Forks
3.7k
Avg merge
13h 37m
Merged PRs (30d)
8

Description

__Steps to reproduce:__
List the minimal actions needed to reproduce the behavior.

1. create a .csv file with this content:
```
#group,false,false,false,false,false,false
#datatype,string,long,string,string,long,dateTime:number
#default,to1,0,analogvalue,testsubject1,,
,result,table,_measurement,_field,_value,_time
,,,,,0,1482669077000000000
,,,,,-1,1482669078000000000
,,,,,-1,1482669079000000000
,,,,,-1,1482669080000000000
```

2. Upload that in the cloud
3. Log into your InfluxDB Cloud account
4. Choose "Explore" in the menu on the left
5. Type this in the script editor:

```
import "experimental/csv"

csv.from(url: "https://your-s3-bucket.s3.eu-central-1.amazonaws.com/your-csv.csv")
|> to(bucket: "test-bucket")
```

__Expected behavior:__
I expect the data inside the .csv file to be saved into the InfluxDB bucket called test-bucket.

__Actual behavior:__
After clicking the "Submit" button i get:
error in csv.from(): parsing time "1482669077000000000" as "number": cannot parse "1482669077000000000" as "number"

However if i use dateTime:RFC3339 datatype and time data like 2018-05-08T20:50:00Z then the data gets imported into the InfluxDB bucket.

I am recording analog signal with a high sampling rate and add timestamp data later on in google sheets. Problem is google sheets can not auto-increment RFC3339 formatted timestamps at sub-second levels (have several thousand rows, so manual entry is not an option 😉 ). Google sheets would easily autoincrement nanosecond values...but for some reason InfluxDB Cloud does not consume dateTime:number formatted data. 😕

__Environment info:__

InfluxDB Cloud deployment in Frankfurt on AWS.
CSV file written with Notepad on a Win10 x64 machine
Using Chrome V89.0.4389.90

This bug combined with #21011 prevents me using InfluxDB Cloud for storing my time-series.

Hints (if any) are highly appreciated!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the experimental/csv csv.from entry point and reproduce the import with the supplied CSV using dateTime:number, then compare it with dateTime:RFC3339. Done means numeric nanosecond timestamps are accepted and written to the target bucket without the parsing error.

Written by the indexing model from the issue text.

Assessment

Domain
database
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.