CUDA out of memory even with store unused models in CPU memory operation

Thanks for your nice work! I'm applying this model to a very long video trajectory, but I'm finding that CUDA out of memory occurs. According to my understanding of the code, there is an operation of putting all previously unused tensorf on the CPU, which greatly reduces the occupation of CUDA memory. In theory, does it support infinitely long video sequences in model training? However, during the training process, the gpu card memory is still exceeded. What is the reason? Looking forward to your reply.