About fewer vision tokens

This is an excellent paper. It discusses how to output fewer vision tokens in a high-resolution visual encoder. I have some ideas, as shown in the figure. In the decode module of object detection in DETR, k queries are randomly initialized and cross-attention is performed with the visual encoder. Generally, six cross-attention operations are sufficient to extract good features. This has several advantages: 1. The number of vision tokens can be fixed manually. 2. The number of vision tokens is not affected by image resolution. 3. Fewer queries perform cross-attention with a large visual encoder, keeping the computational cost controllable. 4. The model can support input images of different resolutions, and images of different resolutions can obtain multi-scale information through data augmentation. These may be some of my naive ideas.