apache / apache/parquet-java

DateTimeFormatter is used in static context, but not thread safe

Open
#2,738 1 comment 0 reactions 0 assignees View on GitHub
Component: Java Component: Parquet Priority: Trivial Type: bug
Dominant language
Java
Stars
3.1k
Forks
1.6k
Avg merge
3d 12h
Merged PRs (30d)
33

Description

DateTimeFormatter is used in static context but not thread safe, a formatter instance is created in PrimitiveStringifer.DateStringifier, and DateStringifier is created in static final DATE_STRINGIFIER, TIMESTAMP_MILLIS_STRINGIFIER, TIMESTAMP_MICROS_STRINGIFIER, TIMESTAMP_NANOS_STRINGIFIER, TIMESTAMP_MILLIS_UTC_STRINGIFIER, TIMESTAMP_MICROS_UTC_STRINGIFIER, and TIMESTAMP_NANOS_UTC_STRINGIFIER.

This causes exceptions like the following to be thrown from parquet-code:

java.lang.ArrayIndexOutOfBoundsException: Index 633 out of bounds for length 13

stacktrace:

    at java.base/sun.util.calendar.BaseCalendar.getCalendarDateFromFixedDate(BaseCalendar.java:457)
    at java.base/java.util.GregorianCalendar.computeFields(GregorianCalendar.java:2358)
    at java.base/java.util.GregorianCalendar.computeFields(GregorianCalendar.java:2273)
    at java.base/java.util.Calendar.setTimeInMillis(Calendar.java:1827)
    at java.base/java.util.Calendar.setTime(Calendar.java:1793)
    at java.base/java.text.SimpleDateFormat.format(SimpleDateFormat.java:978)
    at java.base/java.text.SimpleDateFormat.format(SimpleDateFormat.java:971)
    at java.base/java.text.DateFormat.format(DateFormat.java:339)
    at java.base/java.text.Format.format(Format.java:159)
    at org.apache.parquet.schema.PrimitiveStringifier$DateStringifier.toFormattedString(PrimitiveStringifier.java:265)
    at org.apache.parquet.schema.PrimitiveStringifier$DateStringifier.stringify(PrimitiveStringifier.java:256)
    at org.apache.parquet.column.statistics.IntStatistics.stringify(IntStatistics.java:92)
    at org.apache.parquet.column.statistics.IntStatistics.stringify(IntStatistics.java:25)
    at org.apache.parquet.column.statistics.Statistics.minAsString(Statistics.java:423)

    (... unrelated code)

A simple solution would be to change those from static to non static values.

I can create a PR if the solution is ok by the maintainers of the library.

**Environment**: Linux, OpenJDK 17 (based on docker image openjdk:17-slim)

 
**Reporter**: [Shani Elharrar](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=shani.elha)

**Note**: *This issue was originally created as [PARQUET-2240](https://issues.apache.org/jira/browse/PARQUET-2240). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in PrimitiveStringifier.DateStringifier, especially the toFormattedString and stringify paths shown in the stack trace, and inspect the seven static stringifier instances named in the issue. Verify the failure under concurrent formatting and use the project’s relevant tests to confirm that shared use no longer throws the reported exception.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.