Some tiles cannot be deleted when zooming in or out

Nobody has claimed this yet.

**mapbox-gl-js version**: 2.3.1

**browser**: Google Chrome(92.0.4515.107)

### Steps to Trigger Behavior
Hello, I found a problem when using Mapbox-GL-JS. I have been searching for a long time but I don't know what caused the problem. The problem is described as follows(This is always a problem when raster-fader-duration = 0, and occasionally when it is not set):
1.Use the Raster Source to add tiles to the map
2.Zoom out and then zoom in
3.At this point, you can see that the tiles of the previous level are not deleted

### Link to Demonstration

`
'label-tile-source': {
"type": "raster",
'tiles': [
'http://t0.tianditu.com/DataServer?T=cva_w&X={x}&Y={y}&L={z}&tk=b055e3a80de633c788f673dfcb164aa4'
],
'tileSize': 256
}
`

`
{
"id": "label-tile-layer",
"type": "raster",
"source": "label-tile-source",
"minzoom": 0,
"maxzoom": 12,
"layout": {},
"paint": {
'raster-fade-duration': 0
}
}
`


### Expected Behavior
It is expected that all tiles will be the same size when the map is scaled
### Actual Behavior
But now the labels that were not deleted are smaller because the previous layer of tiles was not deleted

No source files or tests are named. Reproduce the issue in mapbox-gl-js 2.3.1 with a raster source, zoom out and back in, and compare behavior with raster-fade-duration set to 0 and when it is omitted. Done means tiles from the previous zoom level are removed and the raster labels remain correctly sized.

Written by the indexing model from the issue text.