The timestamp column format is ignored if the column value is numeric
- Dominant language
- Java
- Stars
- 14.1k
- Forks
- 3.8k
- Avg merge
- 2d 58m
- Merged PRs (30d)
- 233
Description
### Affected Version
Druid 0.16.0 and probably earlier also
### Description
If the value of the JSON is a `Number` then the format is ignored and god knows what happens instead (millis is used?).
The issue is in this line of code: https://github.com/apache/incubator-druid/blob/84598fba3b283cbfd6a5addd2602c7b12ba8c00c/core/src/main/java/org/apache/druid/java/util/common/parsers/TimestampParser.java#L129
# Repro
Try to ingest
```json
{"name":"V","time":2019102120}
{"name":"D","time":2019102121}
```
With format of `yyyyMMddHH`
Here is a ready made query:
```json
{
"type": "index",
"spec": {
"type": "index",
"ioConfig": {
"type": "index",
"firehose": {
"type": "inline",
"data": "{\"name\":\"V\",\"time\":2019102120}\n{\"name\":\"D\",\"time\":2019102121}"
}
},
"dataSchema": {
"dataSource": "sample",
"parser": {
"type": "string",
"parseSpec": {
"format": "json",
"timestampSpec": {
"column": "time",
"format": "yyyyMMddHH"
},
"dimensionsSpec": {}
}
}
}
},
"samplerConfig": {
"numRows": 500,
"timeoutMs": 15000,
"cacheKey": "8451100357df41f5ac5502381506e674"
}
}
```
Results in:
```json
{
"cacheKey": "8451100357df41f5ac5502381506e674",
"numRowsRead": 2,
"numRowsIndexed": 2,
"data": [
{
"raw": "{\"name\":\"V\",\"time\":2019102120}",
"parsed": {
"__time": 2019102120,
"name": "V"
}
},
{
"raw": "{\"name\":\"D\",\"time\":2019102121}",
"parsed": {
"__time": 2019102121,
"name": "D"
}
}
]
}
```
The format is totally ignored. If you send it as a string then all is good.
# Impact
This is strange and confusing behavior. One would intuitively expect that setting a JODA time format would cast the value to a string.
I was confused by it as was this ASF Slack user:

# Workaround
You could either change your data to represent these columns as strings or add the timestamp in the transform stage using `timestamp_parse("time_column_to_parse", 'yyyyMMddHH')`
Contributor guide
Research direction
Start with core/src/main/java/org/apache/druid/java/util/common/parsers/TimestampParser.java at line 129 and run the provided inline ingestion query with numeric time values and format yyyyMMddHH. Compare the result with string input; done when numeric values honor the configured format and regression coverage verifies the behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 48/100