DateTimeFormatter is used in static context, but not thread safe
- Dominant language
- Java
- Stars
- 3.1k
- Forks
- 1.6k
- Avg merge
- 3d 12h
- Merged PRs (30d)
- 33
Description
DateTimeFormatter is used in static context but not thread safe, a formatter instance is created in PrimitiveStringifer.DateStringifier, and DateStringifier is created in static final DATE_STRINGIFIER, TIMESTAMP_MILLIS_STRINGIFIER, TIMESTAMP_MICROS_STRINGIFIER, TIMESTAMP_NANOS_STRINGIFIER, TIMESTAMP_MILLIS_UTC_STRINGIFIER, TIMESTAMP_MICROS_UTC_STRINGIFIER, and TIMESTAMP_NANOS_UTC_STRINGIFIER.
This causes exceptions like the following to be thrown from parquet-code:
java.lang.ArrayIndexOutOfBoundsException: Index 633 out of bounds for length 13
stacktrace:
at java.base/sun.util.calendar.BaseCalendar.getCalendarDateFromFixedDate(BaseCalendar.java:457)
at java.base/java.util.GregorianCalendar.computeFields(GregorianCalendar.java:2358)
at java.base/java.util.GregorianCalendar.computeFields(GregorianCalendar.java:2273)
at java.base/java.util.Calendar.setTimeInMillis(Calendar.java:1827)
at java.base/java.util.Calendar.setTime(Calendar.java:1793)
at java.base/java.text.SimpleDateFormat.format(SimpleDateFormat.java:978)
at java.base/java.text.SimpleDateFormat.format(SimpleDateFormat.java:971)
at java.base/java.text.DateFormat.format(DateFormat.java:339)
at java.base/java.text.Format.format(Format.java:159)
at org.apache.parquet.schema.PrimitiveStringifier$DateStringifier.toFormattedString(PrimitiveStringifier.java:265)
at org.apache.parquet.schema.PrimitiveStringifier$DateStringifier.stringify(PrimitiveStringifier.java:256)
at org.apache.parquet.column.statistics.IntStatistics.stringify(IntStatistics.java:92)
at org.apache.parquet.column.statistics.IntStatistics.stringify(IntStatistics.java:25)
at org.apache.parquet.column.statistics.Statistics.minAsString(Statistics.java:423)
(... unrelated code)
A simple solution would be to change those from static to non static values.
I can create a PR if the solution is ok by the maintainers of the library.
**Environment**: Linux, OpenJDK 17 (based on docker image openjdk:17-slim)
**Reporter**: [Shani Elharrar](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=shani.elha)
**Note**: *This issue was originally created as [PARQUET-2240](https://issues.apache.org/jira/browse/PARQUET-2240). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in PrimitiveStringifier.DateStringifier, especially the toFormattedString and stringify paths shown in the stack trace, and inspect the seven static stringifier instances named in the issue. Verify the failure under concurrent formatting and use the project’s relevant tests to confirm that shared use no longer throws the reported exception.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100