speed=1.0+ in encoding mode stutters, video playback never goes above 1.0 speed

Important Information

Provide following Information:

• mpv 0.32.0-783-g49d6a1e77d
• Windows 10 and Ubuntu 16.04

If you're not using git master or the latest release, update.
Releases are listed here: https://github.com/mpv-player/mpv/releases

Reproduction steps

mpv --audio-pitch-correction=no --speed=2  --no-config --of=matroska --ovc=libx264 --ovcopts=profile=main,preset=veryfast,level=4,b=5000k,maxrate=5000k,minrate=2000k,bufsize=5000k --oac=pcm_s16le --oacopts=ac=2 --profile=low-latency input.mkv --o=- --start=3600 | ffplay -

Playing a media file (w/ video+audio streams) at speed>1.0 in encoding mode (--o=) causes the video to stutter in an unusual way. The video speed will never go above 1.0, but the audio will.
So the audio will sound sped up, but the video will remain at 1.0 speed, jumping 0.25 seconds every half second or so to keep up with the audio.
However, speed lower than 1.0 however works exactly as expected when output directly to ffplay.

However, if outputted to an RTMP server, I noticed a slightly different behaviour - if speed was set to 2.0, instead of the video jumping forward to keep up with the audio, the audio (playing at 2.0x speed) would pause every moment to keep up with the video. Similarly, at speed 0.5, the audio would play (at 0.5x speed) and skip forward every moment to keep up with the 1.0x speed video.

Log file

mpv.log

Start with the reproduction command using --speed, encoding mode (--of), and piped output (--o=-), then compare its behavior with RTMP output. Read the relevant mpv playback and encoding paths, using the attached mpv.log to trace timing; done means video and audio remain synchronized at speeds above and below 1.0.