Performance on large data

We have to send many small files ( ~8MB) to a hdf5 server in a short time. Although there is a 1Gbit/s connection on both sides, only around 6MB/s are achieved.

Is there any way we can improve the speed or is it a general problem. Most of the time seems to be lost by writing to the hdf5 file.

The issue names no files, tests, or entry points. Start by reproducing the many-small-file upload against the h5serv HDF5 REST service and profile whether time is spent in transfer or HDF5 writes; done would require an identified bottleneck and a measured improvement or a documented limitation.

Written by the indexing model from the issue text.