influxdata / influxdata/influxdb-java

Incorrect long values returned due to json to java conversion

Open
#276 23 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

help wanted kind/bug
Dominant language
Java
Stars
1.2k
Forks
469
PR merge metrics
No merged PRs in 30d

Description

I am having a problem with bad long values being returned due to the conversion to double that happens when all JSON numeric values are stored as Doubles.

This came up first when trying to get back time values as nanoseconds instead of rfc3339 string values. But we need to store very large long values in other fields as well. The application here is scientific instruments, so numerical errors are pretty high priority problems when we are looking at data storage.

From looking around, it does not look like there is a good way to intercept the JSON and convert to Long instead of Double... but maybe I am missing something? I would like to find some way to get these values returned as Long instead of Double. Maybe a different converter? I would appreciate any suggestions or workarounds.

Here is a snippet that recreates the problem:

        long lval = 1485370052974000001L;

        BatchPoints batchPoints = BatchPoints
                .database(dbName)
                .tag("async", "true")
                .retentionPolicy("autogen")
                .consistency(ConsistencyLevel.ALL)
                .build();
        Point point = Point.measurement("testProblems")
                .time(innano, TimeUnit.NANOSECONDS)
                .addField("double", 3.14)
                .addField("long", lval)
                .build();

        batchPoints.point(point);
        influxDB.write(batchPoints);

        Series series0 = result.getResults().get(0).getSeries().get(0);
        List<String> cols = series0.getColumns();
        List<Object> val0 = series0.getValues().get(0);

        Object outlval_obj = val0.get(cols.indexOf("long"));
        if (outlval_obj instanceof Double)
        {
            long outlval = ((Double)outlval_obj).longValue();
            if (outlval != lval)
            {
                System.err.println("Got bad lval back as double [" + (outlval_obj) + "] -> " + outlval + " != " + lval);
            }
        }

Here is a gist with more code and outputs:
https://gist.github.com/ghmegan/4ed652c8c8dbaf18976c6f5f4c0c6b55

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the provided BatchPoints/Point reproduction and the linked gist, then trace the JSON result conversion that turns the long field into a Double. Determine how large integer values can round-trip without loss, and verify completion by confirming the reproduced value is returned exactly rather than as a corrupted Double.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
database
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.