Reduce memory consumption of data post-processing

We currently use up a lot of memory at post-processing because we load up everything. This can cause post-processing jobs to fail due to insufficient RAM.

There should be a way to be much more efficient, e.g., by processing/writing data chunk by chunk.
The main obstacle here is the train/test/infeasible split, which requires checking the status of every instance first, then moving data to either dataset. The raw results files are stored in chunk-size HDF5 files, which need to parsed and filtered.

Not the worst bottleneck as long as we have access to machines with large amounts of RAM, but definitely a direction for improvement.