MetadataStore updateObject API fails occasionally with a conditional check error.
- Lenguaje dominante
- Java
- Estrellas
- 8
- Forks
- 6
- Merge medio
- 13 h 17 min
- PR fusionados (30 d)
- 2
Descripción
The core of the updateObject API implementation is [here](https://github.com/adobe/S3-FileSystem/blob/main/src/main/java/com/adobe/s3fs/metastore/internal/dynamodb/storage/AmazonDynamoDBStorage.java#L83).
In some rare cases (mostly observed when running a very high number of metadata only operations against the file system: e.g. restoring an HBase snapshot into a directory) this method will fail with a ConditionalCheck.
What seems to be happening is that an I/O error occurs, on the server side in DynamoDB the operation completes successfully and the client retries the operation (since from its perspective it failed with an I/O error). At this point the operation fails because it is issued with a ConditionalCheck which is not idempotent.
The proposed solution is to handle this particular case: when we receive a ConditionalCheck failed error we should read the state of the item from DynamoDB and check it against the item state that was received as an update. If the two are equal that means the operation was in fact executed successfully in DynamoDB and we should not raise the error up the call stack.
Guía de contribución
Línea de trabajo
Comienza en src/main/java/com/adobe/s3fs/metastore/internal/dynamodb/storage/AmazonDynamoDBStorage.java, en la implementación de updateObject. Rastrea cómo se gestionan los fallos de ConditionalCheck y los reintentos del cliente durante operaciones de solo metadatos de gran volumen. Se considera terminado cuando un ConditionalCheck fallido se suprime únicamente si una lectura posterior de DynamoDB muestra que el elemento almacenado coincide con la actualización solicitada; de lo contrario, el error sigue propagándose.
Escrito por el modelo de indexación a partir del texto del issue.
Evaluación
- Stack tecnológico
- aws, java
- Área
- backend, database
- Tipo de issue
- Error
- Dificultad
- 4/5
- Tiempo estimado
- 3-5 días
- Estado de actividad
- Estancado
- Claridad
- Bien especificado
- Aptitud para principiantes
- 45/100