oom triggered when run dsync

When running dsync for large amount of files, but small memory configuration, OOM triggered.
Normally we need to use add more memory or use more nodes to avoid OOM, but it's not always possible to some users, because of the limitation of hardware equipments or some other.
Though we can use cgroup (add launched threads into cgroup) to prevent other threads from OOM killer, but doesn't make much sense to dsync, because it really needs such lot of memory to move forward.

I was thinking about adding option to flush the generated metadata flist to storage/disk rather keeping them in the memory, and read them back at the real sync/copy phase, but not quite sure if it works or not.

@adammoody @dinatale2 do you have any better ideas for this issue?

Thanks

Start by tracing the dsync execution path, especially where the generated metadata flist is built and later consumed during synchronization or copying. Reproduce the OOM with many files and a small memory configuration, then determine whether a disk-backed flist approach is feasible; done should include successful synchronization without exhausting available memory.