Enhanced silence skipping implementation

### Issue description
We are playing HLS tracks and facing skip silence issues. Attached project reproduces problem exactly. On some track, especially on film music, skip silence just rolls over few second with no apparent reason. This produces very strange audio glitches and it's confusing for the end user.

### Reproduction steps
[SkipSilenceRepro.zip](https://github.com/google/ExoPlayer/files/4676427/SkipSilenceRepro.zip)
Run attached project, then seek to 40th second. The issue is very apparent on span 40-46 seconds of this track. If we turn off skip silence - everything works perfectly.

### Link to test content
Test content is embedded in attached reproduction code

### A full bug report captured from the device
These is no bug on the device, no strange logs or errors in console.

### Version of ExoPlayer being used
2.11.4

### Device(s) and version(s) of Android being used
Any device we tested it on - Emulators, Samsung S6, OnePlus 7, Samsung Note 10 and so on