Pausing and unpausing slows down Fast-Forwad output rate

Description

When I normally run games with the Fast-Forward Toggle, they run at the correct speed/fps. For systems like the GB, GBA, NES, SNES etc. setting Fast-Forward Rate to any reasonable rate means the output will be at that speed, or close enough. This is easily evidenced by the game speed and the FPS counter.

The issue is that after I Pause and Unpause emulation (by the dedicated pause function, either using Pause hotkey, or the windows menu Command->Pause option, NOT by toggling the emulator menu), fast-forwarding (at Fast-Forward Rate values between 2.0x and 13.0x) results in emulation at much slower speed than expected.

---

Expected behavior

RetroArch behaves the same way after pause/unpause as before.

At 2.0x FF rate: Output = ~120fps
At 3.0x FF rate: Output = ~180fps
At 4.0x FF rate: Output = ~240fps
At 5.0x FF rate: Output = ~300fps
...
At 13.0x FF rate: Output = ~1000fps

---

Actual behavior

BEFORE Pause/Unpause:
Same as Expected behavior.

AFTER Pause/Unpause:

At 2.0x FF rate: Output = ~100fps
At 3.0x FF rate: Output = ~130fps
At 4.0x FF rate: Output = ~130fps
At 5.0x FF rate: Output = ~130fps
...
At 13.0x FF rate: Output = ~200fps

DOES NOT HAPPEN when FF rate is uncapped (set to 0.0x or 14.0x or more)

---

Steps to reproduce the bug

1. Start a game
2. Set Fast-Forward rate to e.g. 5.0x
3. See that Fast-Forwarding runs at the correct rate
4. Pause
5. Unpause
6. Try to Fast-Forward again
7. Game is faster than normal speed, but slower than the expected fast-forward speed.

https://github.com/user-attachments/assets/23ceb6d1-28fc-48de-9994-c83d17c52b5a

---

Bisect Results

Hard to say by experience as I only just started using the dedicated (non-menu) pause and immediately noticed this issue, but I did check some older clean stable builds - see the section below.

---

Version/Commit

• RetroArch: 1.19.1

• Originally noticed in a Nightly build from August (not sure exactly which one, it was simply the last I updated). Updated to latest nightly for the same result:

• • Git Version: 1e4b81c
• • Build Date: Oct 13 2024
• • Multiple cores (mGBA, Gamebatte, Snes 9x)

• Checked some Clean Stable builds to at least somewhat pinpoint when this started:

• REPRODUCED ON:

• • 1.19.1
• • 1.17.0

• NOT REPRODUCED ON:

• • 1.10.0
• • 1.14.0
• • 1.15.0
• • 1.16.0

---

Environment information

• OS: Windows 11
• Tried 2 different machines with different hardware for exact same result:
• • Desktop PC with i5 13600k
• • Handheld PC Aya Neo Flip DS with Ryzen 8840U

Reproduce on Windows using a supported core and a 5.0x Fast-Forward rate, comparing output before and after the dedicated Pause hotkey or Command->Pause unpause cycle. Inspect the pause and fast-forward timing paths; done means bounded rates from 2.0x to 13.0x remain consistent after unpausing, while uncapped rates continue to behave correctly.