out of memory(cpu) when finetuning with 50M text image pairs

thanks for your nice job, there is a "out of memory(cpu)" error when finetuning with 50M text image pairs, after loading the data(takes about 2 days), the process died in the beginning of the training. we found the reason is out of memory(cpu), for details please see the following screenshot. For now I am trying to split the dataset into 10 patches and loading one of these 10 patches every epoch. Is there any solution else to support huge dataset such as 100M text image pairs?

No source file, test, or entry point is named. Start by reproducing finetuning with the reported 50M text-image-pair dataset and trace the data-loading phase that exhausts CPU memory. Done means supporting very large datasets without the process dying from CPU out-of-memory.

Written by the indexing model from the issue text.