Question on experiences with Efficiency

Nobody has claimed this yet.

Hi @lucidrains,
Thanks a lot for providing this implementation so quickly.
I have a question regarding your (or other's) experience on the efficiency of Lambdalayers.
I tried to implement a LambdaUNet where I changed the 3x3conv layers with lambdalayers and avg pooling.

The Conv-UNet has 17Mio parameters while the LambdaUNet only 3Mio. Still, inference and training take much longer in the LambdaUNet than in the ConvUNet (approx 1s ConvUnet vs 10s Lamndaunet). I also used a receptive field of r=23. I am not sure where this parameter originates from or what receptive field should be set. In the paper, the authors talk about "controlled experiments". I assume they chose the lambdalayer hyperparameter (in some way) similar to the conv parameters? It is not very clear from the paper (at least from my initial reading).

I was wondering if others share my experience on slower training and inference time when blindly changing conv layer with lambda layers. Maybe someone can share his expertise on how I can control my LambdaUnet to be comparable to a regular UNet to reproduce the performance and efficiency results from the paper.
Thanks again

No file or test is named. Start by locating the Lambda layer and the Conv-UNet/LambdaUNet implementations, then compare the paper's controlled setup with the reported r=23 experiment. Done means explaining the 10x timing difference and identifying how the receptive-field and related hyperparameters should be selected for a fair comparison.

Written by the indexing model from the issue text.