Can't play short (sub-second) audio files if auto-play is disabled and it is played a while (1 or 2 seconds) after initialization

# Steps to reproduce

1. `git clone https://github.com/google/ExoPlayer.git`
2. apply this patch:

```diff
--- a/demos/main/src/main/assets/media.exolist.json
+++ b/demos/main/src/main/assets/media.exolist.json
@@ -1,4 +1,13 @@
[
+ {
+ "name": "Bug Test",
+ "samples": [
+ {
+ "name": "Pop",
+ "uri": "https://soundbible.com/grab.php?id=2067&type=mp3"
+ }
+ ]
+ },
{
"name": "Clear DASH",
"samples": [
diff --git a/demos/main/src/main/java/com/google/android/exoplayer2/demo/PlayerActivity.java b/demos/main/src/main/java/com/google/android/exoplayer2/demo/PlayerActivity.java
index 5fb342be4..1c7940df3 100644
--- a/demos/main/src/main/java/com/google/android/exoplayer2/demo/PlayerActivity.java
+++ b/demos/main/src/main/java/com/google/android/exoplayer2/demo/PlayerActivity.java
@@ -280,7 +280,7 @@ public class PlayerActivity extends AppCompatActivity
player.addListener(new PlayerEventListener());
player.addAnalyticsListener(new EventLogger(trackSelector));
player.setAudioAttributes(AudioAttributes.DEFAULT, /* handleAudioFocus= */ true);
- player.setPlayWhenReady(startAutoPlay);
+ //player.setPlayWhenReady(startAutoPlay);
playerView.setPlayer(player);
debugViewHelper = new DebugTextViewHelper(player, debugTextView);
debugViewHelper.start();
```

3. Build and run
4. Click on `Bug Test` and then `Pop`
5. Observe the `rb:` value (`renderedOutputBufferCount`) on the second line at the top
6. Once it reaches 12 (for me is consistently around 1 or 2 seconds), click the **Play** button.

# Expected behavior

The sound can be heard.

# Actual behavior

The sound can't be heard.

**Note:** If it is clicked before the `rb` value reaches 12, then it plays correctly.

# Tested on

- ExoPlayer: release-v2 branch (commit b1000940eaec9e1202d9abf341a48a58b728053f)
- Android version: 9
- Android device: LG G6 H870

(also tested on Alldocube iPlay 20, Android 10)

If there's something you don't want to post publicly, please submit the issue,
then email the link/bug report to dev.exoplayer@gmail.com using a subject in the
format "Issue #1234", where #1234 is your issue number (we don't reply to
emails).

If you need the contents of `adb bugreport`, please let me know and I will send them to dev.exoplayer@gmail.com.

**Disclaimer:** I'm not a native Android developer, I'm just a Flutter user that bumped into this problem while using several Flutter plug-ins to play audio that use ExoPlayer as a backend for Android. I did my best to provide a reproducible test case.