Support renderer mode that only renders the first frame (for trick play)

Hello dear developers!

First, let me explain what I'm doing.
I'm trying to implement reverse trick mode by using I-Frames provided by #EXT-X-I-FRAME-STREAM-INF track.
The current algorithm is as follows:
1. Set playback to pause by player.setPlayWhenReady(false)
2. Select trick-play tracks by SelectionOverride.
3. Get current position by player.getCurrentPosition()
4. Set a new position reduced by N seconds relative to the current.
5. Wait for onRenderedFirstFrame() and perform step 3 again. So, steps 3,4,5 are in the loop.

This algorithm is working for me but has some disadvantages.
The main problem is the time between seek() operation and onRenderedFirstFrame event.
It takes about 500ms and looks pretty big.
I have debuged the code and found that first frame is rendered only after 4-5 I-Frames pushed to the decoder.
I found it by additional trace before https://github.com/google/ExoPlayer/blob/4b1e0fa9fc643f96ab5bbf4cec9aa7a79e4bc789/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/MediaCodecRenderer.java#L1349.
I've tried to control buffering by overriding DefaultLoadControl.shouldContinueLoading() and I see if I load only 2 or 3 I-Frame chunks after seek(), then the onRenderedFirstFrame event does not come at all.

In theory, sending one i-frame to the decoder is enough to receive onRenderedFirstFrame.
But maybe MediaCodec or Renderer has some internal buffering and requires mulptiple frames to display the first.

Could you tell me why this is happening and how to avoid it?

Thanks.