apache / apache/druid

ParseException while parsing uintiger and IEEE-754 64-bit floating-point number using the InfluxParser

Open
#10,993 0 comments 0 reactions 0 assignees View on GitHub
Area - Ingestion Bug
Dominant language
Java
Stars
14.1k
Forks
3.8k
Avg merge
2d 58m
Merged PRs (30d)
233

Description

### Affected Version
ArtifactID: druid-influx-extensions
Version: 0.13.0-incubating

### Description
My team and I are currently working on a project, and we used the [InfluxParser](https://github.com/apache/druid/blob/master/extensions-contrib/influx-extensions/src/main/java/org/apache/druid/data/input/influx/InfluxParser.java) in our project to parse string line protocols to [data point objects](https://github.com/influxdata/influxdb-client-java/blob/master/client/src/main/java/com/influxdb/client/write/Point.java). After testing it with different [field value types](https://docs.influxdata.com/influxdb/cloud/reference/syntax/line-protocol/#data-types-and-format) we noticed that some of the field values are not supported. As for an example, consider the following test:

```java
@Test
void shouldNotParseLineWithFieldValueAsUnsignedIntegerToDataPoint() {
final String lineProtocol = "myMeasurement fieldKey=12485903u";

final Point actualDataPoint = InfluxParser.parseToDataPoint(lineProtocol);

assert actualDataPoint != null;
final Long expectedFieldValue = 12485903L;
assertEquals(expectedFieldValue, actualDataPoint.getField("fieldKey"));
}
```

Currently, this test throws a `ParseException` and fails. The reason behind it is that in the [InfluxLineProtogol.g4](https://github.com/apache/druid/blob/master/extensions-contrib/influx-extensions/src/main/antlr4/org/apache/druid/data/input/influx/InfluxLineProtocol.g4#L70) the value context NUMBER doesn't support `u` yet.

This issue also arises whenever we tested IEEE-754 64-bit floating-point numbers, which is [supported by InfluxDB](https://docs.influxdata.com/influxdb/cloud/reference/syntax/line-protocol/#float). Consider the following test case:

```java
@Test
void shouldParseLineWithFieldValueAsFloatToDataPoint() throws ParseException {
final String lineProtocol = "myMeasurement fieldKey=-1.234456e+78";
final Point actualDataPoint = InfluxParser.parseToDataPoint(lineProtocol);

assert actualDataPoint != null;
final double expectedFieldValue =-1.234456e+78;
assertEquals(expectedFieldValue, actualDataPoint.getField("fieldKey"));
}
```
Unfortunately, this test will fail since the parser is not caple of parsing the float filed value. The parser should parse these lines since they are all examples provided by InfluxDB [documentation](https://docs.influxdata.com/influxdb/cloud/reference/syntax/line-protoco).
If necessary I will create a PR and contribute :)

Contributor guide

Open the contributing guide

Research direction

Start with extensions-contrib/influx-extensions/src/main/antlr4/org/apache/druid/data/input/influx/InfluxLineProtocol.g4 and InfluxParser.parseToDataPoint. Reproduce the unsigned-integer and IEEE-754 floating-point examples from the issue. Done means both line protocols parse into data points with the expected field values.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.