apache / apache/pinot

Datetime format problem

Open
#7,195 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
1d 21h
Merged PRs (30d)
189

Description

I need to import the timestamp from a json file with the format of: YYYY-MM-DDThh:mm:ss.SSSZ for example: 2021-06-22T09:16:31.260Z. The characters T and Z makes problem while trying to visualize with superset. So I tried some transform functions such as:
FromDateTime('timestamp', 'yyyy-MM-dd''T''HH:mm:ss.SSS''Z''')
DATETIMECONVERT(timestamp, '1:MILLISECONDS:SIMPLE_DATE_FORMAT:yyyy-MM-dd''T''HH:mm:ss.SSS''Z', '1:SECONDS:SIMPLE_DATE_FORMAT:yyyy-MM-dd HH:mm:ss', '1:SECONDS')
And many more options, but no luck.
What can be the problem here?
The exception is as follows:

Error: {'errorCode': 150, 'message': 'PQLParsingError:\n' 'org.apache.pinot.sql.parsers.SqlCompilationException: Caught ' 'exception while parsing query: SELECT 'timestamp\n' 'FROM "mytable"\n' 'LIMIT 10000\n' '\tat ' 'org.apache.pinot.sql.parsers.CalciteSqlParser.compileCalciteSqlToPinotQuery(CalciteSqlParser.java:324)\n' '\tat ' 'org.apache.pinot.sql.parsers.CalciteSqlParser.compileToPinotQuery(CalciteSqlParser.java:108)\n' '\tat ' 'org.apache.pinot.sql.parsers.CalciteSqlCompiler.compileToBrokerRequest(CalciteSqlCompiler.java:35)\n' '\tat ' 'org.apache.pinot.core.requesthandler.PinotQueryParserFactory.parseSQLQuery(PinotQueryParserFactory.java:46)\n' '\tat ' 'org.apache.pinot.broker.requesthandler.BaseBrokerRequestHandler.handleSQLRequest(BaseBrokerRequestHandler.java:212)\n' '\tat ' 'org.apache.pinot.broker.requesthandler.BaseBrokerRequestHandler.handleRequest(BaseBrokerRequestHandler.java:194)'}

Contributor guide

Open the contributing guide

Research direction

Reproduce the Superset query with the provided timestamp and inspect the failure path at org.apache.pinot.sql.parsers.CalciteSqlParser.java around line 324. Compare the reported format expressions with the parser's accepted datetime syntax. Done means the timestamp handling is clearly identified as supported or requiring a documented fix, with regression coverage if code changes are needed.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.