Ensure that written data is correctly written

Output of rest-server --version

current dev version 07.09.2023

What should rest-server do differently?

There is an option "--no-verify-upload do not verify the integrity of uploaded data. DO NOT enable unless the rest-server runs on a very low-power device"

I think rest-server just calculates the checksum of the blob, if it matches the filename.

However restic should have an option to re-read the file from storage, to ensure, that it got written correctly.

What are you trying to do? What is your use case?

I am using rest-server for my backups and cloning these backups to another HDD with rsync. Doing a restic check --read-data gave a message "Pack ID does not match, want". Luckily on the system rest-server is running the file is fine. So there must be something gone wrong with rsync, which is also quite strange.
Anyway, rest-server should have an option to re-read the file from file-system and ensure that it got written correctly.
For implementing this, please keep the linux file system cache in mind. Since dropping the cache is only possible for all the cache (and requires root rights), which we don't want, I suggest reading the file with to ensure, that it is really written from disc syscall.O_DIRECT.

I also suggest using something like POSIX_FADV_DONTNEED when writing the files, like nocache does https://github.com/Feh/nocache
This improves the genrall perfomance, since it is very unlikely, that the just written blocks will be read again. And if rest-server reads them, we want to read it directly from disk.

One question is how this would degenerate performance, because maybe other blocks have to wait for the OK of the verification process. Maybe somebody with more experience can give an opinion. Maybe this option would be too slow. Nevertheless, we really should think about POSIX_FADV_DONTNEED

Did rest-server help you today? Did it make you happy in any way?

Yes. Rest-server makes my backups crypto trojan safe, which is a very calming feeling.

Start by tracing the upload and filesystem write path, including handling for --no-verify-upload; the issue names no files or tests. Evaluate the requested post-write verification alongside Linux O_DIRECT and POSIX_FADV_DONTNEED, then establish whether an option can detect write corruption with acceptable performance. Done should include agreed behavior and validation for the failure case.