Struggling to learn using Opt. Flow

I'm training this model on my own dataset. I trained it successfully on the RGB stream of the data. Now, I"m trying got do that on the optical flow stream. However, the model is not learning anything at all.

My optical flow output is scaled from [-1,1] and is of `float32` type. I'm detecting it using `method=cv2.cuda.FarnebackOpticalFlow_create( numLevels = 10, pyrScale = 0.5, winSize = 1, numIters = 20 )` for speed. By visualizing it, I can see that it is detecting something coherent. However, the model is not learning absolutely anything. Given that I'm using already pre-trained weights, my thought is that maybe the input is not what the model is expecting and hence can't learn anything because it stats on "the wrong foot". Could there be anything I'm missing regarding preprocessing steps or others?

Or is the model just very sensitive the to the flow method and it needs to have TVL1 as a flow method?

Start by reviewing the optical-flow input described in the issue, including its [-1,1] float32 scaling and Farneback configuration, against the model's expected preprocessing and pretrained-input assumptions. Compare the observed behavior with the question about TVL1, and define done as identifying whether preprocessing or flow-method compatibility accounts for the lack of learning.

Written by the indexing model from the issue text.