googleapis / googleapis/google-cloud-java

[java-storage] It's impossible to reliably upload a complete file

Aperta
#12,635 6 commenti 0 reazioni 0 assegnatari Vedi su GitHub
api: storage
Lingua principale
Java
Stelle
2.1k
Fork
1.2k
Merge medio
1g 23h
PR unite (30g)
154

Descrizione

We're trying to upload a file to GCS, and make sure the file is only published if the upload succeeds. Consider this code:

```java
BlobInfo blobInfo = ...;
ByteBuffer buf1, buf2;
WriteChannel writer = storage.writer(blobInfo, Storage.BlobWriteOption.crc32cMatch());
try {
writer.write(buf1);
writer.write(buf2); // network goes down before this call
} finally {
writer.close(); // network is back up before this call
}
```

The write of `buf1` is successful, but before writing `buf2`, the network goes down. As expected, an `IOException` is thrown (both my buffers are 32MB). The execution proceeds to the `finally` clause to call `close()`. At this point, the network is up again. The `close` call finishes the file upload, and the file is published with partial contents!

However, the problem is more generic. While writing the file, any error can occur (e.g. an IO error while reading the file to upload). In this case, the `close()` call will also publish a partial file.

I have tested moving the `close()` call to the `try` block, so that the `WriteChannel` is abandoned without closing it. This seems to have the desired effect, but since the `close()` method is inherited from `AutoCloseable`, it's very unintuitive, and some code quality tools will suggest or force calling the `close()` method.

One possible backward-compatible fix to this problem would be to add `abort()` method, that will clean up the resources without finishing the upload, and after which `close()` will be no-op. That method could be automatically called for internal exceptions.

I'm using version 2.50.0 on Java 21.

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.