aio reads broken?

Hi all,

We've just been trying out the aio option in IOR, and a trivial "write 4K, read 4K back" with aio fails with corruption on the read; this is to XFS on RHEL 8 and to Lustre (also on RHEL 8). If you keep the file and read it in POSIX mode, the read succeeds, so the write is OK. (AIO reads fail even if it's just a read by itself on an existing file)

So AIO read mode seems to be totally broken, at least on RHEL 8. (This is with tip of tree as of today; acd3a154be765083d2d2543885aee983ac3ae18f / ior: improve error for existing output file (#459) )

Here's the relevant output - for aio:
mpirun --allow-run-as-root -np 1 ior -C -Q 1 -g -G=-1385218502 -t 4096 -b 4096 -s 1 -k -e -w -r -R -a aio -o /tmp/iorfile
[...]
Results:
access bw(MiB/s) IOPS Latency(s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter

write 0.761126 196.54 0.005088 4.00 4.00 0.000037 0.005088 0.000004 0.005132 0
WARNING: Incorrect data on read (1 errors found).Used Time Stamp 2909748794 (0xad6f3e3a) for Data Signature
read 277.69 127100 0.000008 4.00 4.00 0.000002 0.000008 0.000004 0.000014 0

Reading the file back with POSIX:
mpirun --allow-run-as-root -np 1 ior -C -Q 1 -g -G=-1385218502 -t 4096 -b 4096 -s 1 -k -e -r -R -o /tmp/iorfile
[...]
Results:
access bw(MiB/s) IOPS Latency(s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter

read 34.57 37118 0.000027 4.00 4.00 0.000064 0.000027 0.000022 0.000113 0Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Max(OPs) Min(OPs) Mean(OPs) StdDev Mean(s) Stonewall(s)

Start by reproducing the reported -a aio commands with the 4 KiB write/read case and the read-only existing-file case on XFS or Lustre under RHEL 8. Compare the AIO results with POSIX mode; done means AIO reads return the written data without the reported corruption warning.