Newly enabled renderers for next playlist item start playing while previous playback is still ongoing

I am using [jcodec](https://github.com/jcodec/jcodec) to create video files from images like the example attached at the bottom.

These videos are being displayed as a playlist in `SimpleExoPlayer`. These are created using `MediaItem.fromUri(uri)` and with the `DefaultDataSourceFactory`. I am currently using exoplayer version `2.12.3` but I have also tested `2.16.0` which has the same error.

A single jcodec video like this, or even multiple jcodec videos, will playback as expected without error.

However, if you add any regular camera video to the playlist in combination with these videos the playback fails. Specifically the jcodec videos are skipped over and only the camera video is played.

Summary of behavior:
- single or multiple camera videos ✅
- single or multiple jcodec videos ✅
- jcodec video + camera video ❌

Video of playback is attached. As you can see the video is 20 seconds long but flashes a frame before being skipped.

The example video created by jcodec:
https://user-images.githubusercontent.com/16091920/140962455-af644053-5e33-46e9-81aa-96c7ac4b45c9.mp4

Playback error:
https://user-images.githubusercontent.com/16091920/140965056-c02c0a66-a1f2-4e3b-9937-aa7e1f90ef62.mp4