Azure / Azure/azure-documentdb-java

Lack of Thread Safety causing ArrayIndexOutOfBoundsException in ArrayList in StoreReader

Open
#135 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
51
Forks
52
PR merge metrics
No merged PRs in 30d

Description

0.002% of our calls to load operations result in the following error, which means we get around 5 errors per hour. This looks like it is caused by a lack of thread safety around the way an ArrayList is used such that the underlying array inside the ArrayList is size zero but an item is being added at index 1. If you assume the ArrayList does not have a bug (which is likely due to how widely it is used) then the way the ArrayList is being used inside the StoreReader is not thead safe.

```java
com.microsoft.azure.documentdb.DocumentClientException: java.util.concurrent.ExecutionException: java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 0,
RequestStartTime: "26 Oct 2019 00:46:55.566", RequestEndTime: "26 Oct 2019 00:46:55.587", Number of regions attempted: 1
StoreResponseStatistics{requestResponseTime="26 Oct 2019 00:46:55.587", storeReadResult=StoreReadResult{statusCode=200, storePhysicalAddress=https://cdb-ms-prod-westeurope1-fd14.documents.azure.com:14786/apps/413f848b-ce17-40fc-ad7f-14c0e21e9633/services/b6c727a9-51a9-434e-be69-84d1b99e26e2/partitions/bc2ffcc0-d4de-43d0-b854-453cabf7c920/replicas/132151774908527505s//, LSN=2019735, sessionToken=-1#2019735, partitionKeyRangeId='0', quorumAckedLSN=-1, globalCommittedLSN=2019735, itemLSN=-1, numberOfReadRegions=0, requestCharge=1.0, currentReplicaSetSize=-1, currentWriteQuorum=-1, isValid=true, isGoneException=false, isNotFoundException=false, exception='null'}, requestResourceType=Document, requestOperationType=Read}

at com.microsoft.azure.documentdb.internal.directconnectivity.StoreReader.readMultipleReplica(StoreReader.java:161)
at com.microsoft.azure.documentdb.internal.directconnectivity.QuorumReader.readQuorum(QuorumReader.java:164)
at com.microsoft.azure.documentdb.internal.directconnectivity.QuorumReader.readStrong(QuorumReader.java:84)
at com.microsoft.azure.documentdb.internal.directconnectivity.ConsistencyReader.read(ConsistencyReader.java:73)
at com.microsoft.azure.documentdb.internal.directconnectivity.ReplicatedResourceClient.invoke(ReplicatedResourceClient.java:71)
at com.microsoft.azure.documentdb.internal.directconnectivity.ServerStoreModel$1.apply(ServerStoreModel.java:101)
at com.microsoft.azure.documentdb.internal.RetryUtility.executeStoreClientRequest(RetryUtility.java:147)
at com.microsoft.azure.documentdb.internal.directconnectivity.ServerStoreModel.processMessage(ServerStoreModel.java:118)
at com.microsoft.azure.documentdb.DocumentClient$8.apply(DocumentClient.java:3102)
at com.microsoft.azure.documentdb.internal.RetryUtility.executeDocumentClientRequest(RetryUtility.java:73)
at com.microsoft.azure.documentdb.DocumentClient.doRead(DocumentClient.java:3108)
at com.microsoft.azure.documentdb.DocumentClient.readDocument(DocumentClient.java:1071)
at com.tesco.crypt.common.persistence.cosmosdb.CosmosDbClient.getDocument(CosmosDbClient.java:218)
at com.tesco.crypt.common.persistence.cosmosdb.CosmosDbClient.lambda$loadSynchronously$3(CosmosDbClient.java:155)
... 15 more
Caused by: java.util.concurrent.ExecutionException: java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 0
at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
at com.microsoft.azure.documentdb.internal.directconnectivity.StoreReader.readMultipleReplica(StoreReader.java:157)
... 28 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 0
at java.base/java.util.ArrayList.add(ArrayList.java:486)
at java.base/java.util.ArrayList.add(ArrayList.java:498)
at com.microsoft.azure.documentdb.internal.directconnectivity.StoreReader.recordReadResponse(StoreReader.java:406)
at com.microsoft.azure.documentdb.internal.directconnectivity.StoreReader.access$100(StoreReader.java:33)
at com.microsoft.azure.documentdb.internal.directconnectivity.StoreReader$1.call(StoreReader.java:221)
at com.microsoft.azure.documentdb.internal.directconnectivity.StoreReader$1.call(StoreReader.java:217)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
... 4 more
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.