Unable to compile generated class when avro schema name is too long
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 87
- Forks
- 73
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 1
Description
When the name of an avro schema is too long, the Java class is unable to be created as the file name becomes too long. This occurs because the `SchemaAssistant` class returns the name of the schema when the type is a record, and uses that as a component of the generated class name in `FastDeserializerGeneratorBase` and `FastSerializerGenerator`.
An example stack trace for a failure in creating the Deserializer is
```
com.linkedin.avro.fastserde.FastDeserializerGeneratorException: java.io.FileNotFoundException: /tmp/generated18270964553371151670/com/linkedin/avro/fastserde/generated/deserialization/AVRO_1_11/StrategyKey_AND_FilteringData_JOINED_AND_PreviousDayPricingParametersDerivativePricer_v1_AND_PricingParametersDerivativePricer_v1_JOINED_AND_AutoPriceOffsetParameters_JOINED_AND_JarvisBidAskOffsetParameters_JOINED_AND_AutoPriceOffset_JOINED_AND_ManualPriceOffset_JOINED_AND_VolatilityOffset_JOINED_JOINED_AND_AutoBidAskOffset_AND_DimerParameters_JOINED_AND_QuoterParameters_JOINED_AND_HitterParameters_JOINED_AND_InstrumentTradingParameters_JOINED_AND_ListingPricingParameters_JOINED_JOINED_GenericDeserializer_385168959_385168959.java (File name too long)
at com.linkedin.avro.fastserde.FastDeserializerGenerator.generateDeserializer(FastDeserializerGenerator.java:181)
at com.linkedin.avro.fastserde.FastSerdeCache.buildFastGenericDeserializer(FastSerdeCache.java:503)
at com.linkedin.avro.fastserde.FastSerdeCache.buildGenericDeserializer(FastSerdeCache.java:531)
at com.linkedin.avro.fastserde.FastSerdeCache.lambda$getFastGenericDeserializer$4(FastSerdeCache.java:276)
at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1768)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.io.FileNotFoundException: /tmp/generated18270964553371151670/com/linkedin/avro/fastserde/generated/deserialization/AVRO_1_11/StrategyKey_AND_FilteringData_JOINED_AND_PreviousDayPricingParametersDerivativePricer_v1_AND_PricingParametersDerivativePricer_v1_JOINED_AND_AutoPriceOffsetParameters_JOINED_AND_JarvisBidAskOffsetParameters_JOINED_AND_AutoPriceOffset_JOINED_AND_ManualPriceOffset_JOINED_AND_VolatilityOffset_JOINED_JOINED_AND_AutoBidAskOffset_AND_DimerParameters_JOINED_AND_QuoterParameters_JOINED_AND_HitterParameters_JOINED_AND_InstrumentTradingParameters_JOINED_AND_ListingPricingParameters_JOINED_JOINED_GenericDeserializer_385168959_385168959.java (File name too long)
at java.base/java.io.FileOutputStream.open0(Native Method)
at java.base/java.io.FileOutputStream.open(FileOutputStream.java:293)
at java.base/java.io.FileOutputStream.(FileOutputStream.java:235)
at java.base/java.io.FileOutputStream.(FileOutputStream.java:184)
at com.sun.codemodel.writer.FileCodeWriter.openBinary(FileCodeWriter.java:91)
at com.sun.codemodel.CodeWriter.openSource(CodeWriter.java:100)
at com.sun.codemodel.writer.FilterCodeWriter.openSource(FilterCodeWriter.java:67)
at com.sun.codemodel.writer.ProgressCodeWriter.openSource(ProgressCodeWriter.java:76)
at com.sun.codemodel.JPackage.createJavaSourceFileWriter(JPackage.java:486)
at com.sun.codemodel.JPackage.build(JPackage.java:437)
at com.sun.codemodel.JCodeModel.build(JCodeModel.java:311)
at com.sun.codemodel.JCodeModel.build(JCodeModel.java:280)
at com.sun.codemodel.JCodeModel.build(JCodeModel.java:259)
at com.sun.codemodel.JCodeModel.build(JCodeModel.java:287)
at com.linkedin.avro.fastserde.FastSerdeBase.compileClass(FastSerdeBase.java:165)
at com.linkedin.avro.fastserde.FastDeserializerGenerator.generateDeserializer(FastDeserializerGenerator.java:173
```
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with SchemaAssistant and trace how record names flow into FastDeserializerGeneratorBase and FastSerializerGenerator, using the stack trace as the entry point. Inspect the generated class-name construction and compilation path in FastDeserializerGenerator and FastSerdeBase. Done means a schema with a very long name can generate and compile without a FileNotFoundException caused by the output filename.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100