Different inference time between single face and crowded face

Hi, I found when I do the inference, the forward pass time (i.e. the time of result = net(input)) varies greatly among images with a single face and images with many tiny faces? May I ask is this normal or they should have the same time. (they are under the same image size)