Barely moving queries on the boundary of the frame

Hi, I'm using the online mode of CoTrackerV3 for tracking points in 2D on static scenes with a moving camera. For some videos, I experience the problem of almost static point tracks close to the image boundary:

https://github.com/user-attachments/assets/b645e827-5aa5-40da-84ac-ab942d6f15e6

For another video with a few similarly placed points, all tracks evolve meaningfully:

https://github.com/user-attachments/assets/99c22a6e-119b-4bbd-bb0d-ef35247fe2aa

These are the visuals for the first sixty frames with point tracks, interleaved with step=6 for visualization purposes:

Here are the points whose position at frames 0 and 20 has not changed by more than 10 pixels:

`kpts=pred_tracks[:,:,np.linalg.norm(pred_tracks.cpu()[0, 0] - pred_tracks.cpu()[0, 20], axis=1)<10]`

At the same time, when I use grid-based point queries, the tracks look plausible:

https://github.com/user-attachments/assets/87d27082-a466-4a77-a10e-e8ae49367c0f

I wonder what can cause such behavior for keypoints obtained from a detector? I tried changing the resolution of the images, but it had no effect.

Start by reproducing the CoTrackerV3 online-mode case with detector-derived keypoints and compare it with the grid-based query behavior shown in the report. Check the first 60 frames and the frame-0/frame-20 displacement calculation; done means identifying why boundary tracks remain nearly static and documenting or fixing the cause.

Written by the indexing model from the issue text.