Can we extract feature from image encode with respect to each bbox?

Hi I'm trying to understand the model, especicially the way how it works. I noticed that before going in deep down to hidden layers, every annotation (text, bbox, point) must go through a prompt encoder. My question is how can the encoder keep the distinct information for each kind of annotation (text, box and point have different ways to represent of course) and keep the information so the mask can be predicted correctly right on the spot we want.
Beyond that, can we store the vision feature of the each output or bbox so we can use in different tasks or compare with different outputs? Thanks, I appreciate every help :D