Datetime format problem
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 189
Description
I need to import the timestamp from a json file with the format of: YYYY-MM-DDThh:mm:ss.SSSZ for example: 2021-06-22T09:16:31.260Z. The characters T and Z makes problem while trying to visualize with superset. So I tried some transform functions such as:
FromDateTime('timestamp', 'yyyy-MM-dd''T''HH:mm:ss.SSS''Z''')
DATETIMECONVERT(timestamp, '1:MILLISECONDS:SIMPLE_DATE_FORMAT:yyyy-MM-dd''T''HH:mm:ss.SSS''Z', '1:SECONDS:SIMPLE_DATE_FORMAT:yyyy-MM-dd HH:mm:ss', '1:SECONDS')
And many more options, but no luck.
What can be the problem here?
The exception is as follows:
Error: {'errorCode': 150, 'message': 'PQLParsingError:\n' 'org.apache.pinot.sql.parsers.SqlCompilationException: Caught ' 'exception while parsing query: SELECT 'timestamp\n' 'FROM "mytable"\n' 'LIMIT 10000\n' '\tat ' 'org.apache.pinot.sql.parsers.CalciteSqlParser.compileCalciteSqlToPinotQuery(CalciteSqlParser.java:324)\n' '\tat ' 'org.apache.pinot.sql.parsers.CalciteSqlParser.compileToPinotQuery(CalciteSqlParser.java:108)\n' '\tat ' 'org.apache.pinot.sql.parsers.CalciteSqlCompiler.compileToBrokerRequest(CalciteSqlCompiler.java:35)\n' '\tat ' 'org.apache.pinot.core.requesthandler.PinotQueryParserFactory.parseSQLQuery(PinotQueryParserFactory.java:46)\n' '\tat ' 'org.apache.pinot.broker.requesthandler.BaseBrokerRequestHandler.handleSQLRequest(BaseBrokerRequestHandler.java:212)\n' '\tat ' 'org.apache.pinot.broker.requesthandler.BaseBrokerRequestHandler.handleRequest(BaseBrokerRequestHandler.java:194)'}
Contributor guide
Research direction
Reproduce the Superset query with the provided timestamp and inspect the failure path at org.apache.pinot.sql.parsers.CalciteSqlParser.java around line 324. Compare the reported format expressions with the parser's accepted datetime syntax. Done means the timestamp handling is clearly identified as supported or requiring a documented fix, with regression coverage if code changes are needed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100