apache / apache/beam

Beam Schema RowCoder can't roundtrip DateTime

Open
#20,906 0 comments 0 reactions 0 assignees View on GitHub
bug core java P3
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

Beam 2.21.0**** won't roundtrip DateTime. The test was changed in the PR that introduced the breakage: https://github.com/apache/beam/commit/fc6cef9972780ca6b7525d4aadd65a8344221f1b#diff-6dc06eab59c695aaa6c73cf608b392a6a6ab0809e72f1fc3b11805752b8eb69eL68

This is what doesn't work:
```

Row.withSchema(...).addValues(inputRow.getDateTime()).build();

```

Workaround:
```

Row.withSchema(...).addValues(inputRow.getDateTime().toInstant()).build();

```

Stack trace:
```

Caused by: java.lang.ClassCastException: org.joda.time.DateTime cannot be cast to org.joda.time.Instant

at org.apache.beam.sdk.coders.InstantCoder.encode(InstantCoder.java:34)

at org.apache.beam.sdk.coders.RowCoderGenerator$EncodeInstruction.encodeDelegate(RowCoderGenerator.java:274)

at org.apache.beam.sdk.coders.Coder$ByteBuddy$qZItcXc5.encode(Unknown
Source)
at org.apache.beam.sdk.coders.Coder$ByteBuddy$qZItcXc5.encode(Unknown
Source)
at org.apache.beam.sdk.schemas.SchemaCoder.encode(SchemaCoder.java:118)

at org.apache.beam.sdk.coders.Coder.encode(Coder.java:136)

at org.apache.beam.sdk.util.CoderUtils.encodeToSafeStream(CoderUtils.java:85)

at org.apache.beam.sdk.util.CoderUtils.encodeToByteArray(CoderUtils.java:69)

at org.apache.beam.sdk.util.CoderUtils.encodeToByteArray(CoderUtils.java:54)

at org.apache.beam.sdk.util.CoderUtils.clone(CoderUtils.java:144)

at org.apache.beam.sdk.util.MutationDetectors$CodedValueMutationDetector.(MutationDetectors.java:118)

at org.apache.beam.sdk.util.MutationDetectors.forValueWithCoder(MutationDetectors.java:49)

at org.apache.beam.runners.direct.ImmutabilityCheckingBundleFactory$ImmutabilityEnforcingBundle.add(ImmutabilityCheckingBundleFacto
ry.java:115)


at org.apache.beam.runners.direct.ParDoEvaluator$BundleOutputManager.output(ParDoEvaluator.java:305)

at org.apache.beam.repackaged.direct_java.runners.core.SimpleDoFnRunner.outputWindowedValue(SimpleDoFnRunner.java:272)

at org.apache.beam.repackaged.direct_java.runners.core.SimpleDoFnRunner.access$900(SimpleDoFnRunner.java:84)

at org.apache.beam.repackaged.direct_java.runners.core.SimpleDoFnRunner$DoFnProcessContext.output(SimpleDoFnRunner.java:418)

at org.apache.beam.repackaged.direct_java.runners.core.SimpleDoFnRunner$DoFnProcessContext.output(SimpleDoFnRunner.java:406)

```

Imported from Jira [BEAM-12105](https://issues.apache.org/jira/browse/BEAM-12105). Original Jira may contain additional context.
Reported by: apilloud.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.