Serialization/Deserialization fails for large objects with ArrayIndexOutOfBoundsException
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 5.1k
- Forks
- 877
- PR merge metrics
- No merged PRs in 30d
Description
I have a custom serializer:
class CustomSerializerDesc implements Serializer<DescribeSObjectResult>
{
private final XStream xstream = new XStream();
@Override
public void serialize(final DataOutput2 out, final DescribeSObjectResult sObject) throws IOException
{
xstream.toXML(sObject, out);
}
@Override
public DescribeSObjectResult deserialize(final DataInput2 input, final int available) throws IOException
{
final DataInputToStream input2 = new DataInput2.DataInputToStream(input);
return (DescribeSObjectResult) xstream.fromXML(input2);
}
}
where the usage drops across
final DB db = DBMaker
.fileDB(SalesforceOptions.cachePath.resolve("SalesforceObjects").toFile())
.fileMmapEnable()
.fileDeleteAfterClose()
.cleanerHackEnable()
.make();
final CustomSerializerDesc customSerializer = new CustomSerializerDesc();
final List<DescribeSObjectResult> finalDescs = db.indexTreeList("test", customSerializer).createOrOpen();
However, when iterating through the lists, it throws an exception
Exception in thread "main" with the root cause pointing here:
Caused by: java.lang.ArrayIndexOutOfBoundsException
at java.lang.System.arraycopy(Native Method)
at org.mapdb.DataInput2$ByteArray.readFully(DataInput2.java:28)
at org.mapdb.DataInput2$DataInputToStream.read(DataInput2.java:242)
at java.io.FilterInputStream.read(FilterInputStream.java:133)
at java.io.PushbackInputStream.read(PushbackInputStream.java:186)
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
at java.io.InputStreamReader.read(InputStreamReader.java:184)
This doesn't happen to small objects. I tried a custom serializer with a different class that works fine. However, this is a large class and somehow fails. Does serialization/deserialization have any limits on the buffer size?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with DataInput2.java, especially ByteArray.readFully and DataInputToStream.read, then reproduce the failure using the custom serializer, file-backed DB, and large DescribeSObjectResult shown here. Determine whether large-object deserialization exceeds a buffer or exposes another read-boundary problem; done means iteration completes without ArrayIndexOutOfBoundsException for the large object.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- database
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100