Short (encoding) lag can destroy A/V sync for both monitoring audio and recordings in sub systems (Render Delay filter, NDI, Teleport)

Operating System Info

Windows 10

Other OS

No response

OBS Studio Version

27.2.4

OBS Studio Version (Other)

No response

OBS Studio Log URL

https://obsproject.com/logs/3WNHLPaQpUstffUB

OBS Studio Crash Log URL

No response

Expected Behavior

Whatever lag occurs, after it is over OBS should be able to go back to normal operation, i.e. synchronized audio/video for both monitoring audio and recordings (especially if buffers were not permanently exceeded), and any Sync Offset or Render Delay filters should still be correctly applied.

Current Behavior

When keeping on restarting OBS and repeating the lag test, any of these will happen randomly:

• A/V desync only for monitoring/preview but the recording is fine
• A/V desync only for the recording but monitoring/preview is fine
• A/V desync for both recording and monitoring/preview
• the synchronization immediately recovers after the lags end and both monitoring/preview and recording are fine
• sometimes a desync monitoring/preview will slowly recover within the next 20 seconds, but sometimes not even after 30+ seconds and it looks like it will never - and A/V sync in the recording will never recover, not even in cases where monitoring/preview recovers within 20 seconds

Steps to Reproduce

1. Run a video loop suitable for testing A/V sync such as this based on either Media source or VLC source
2. Configure a Render Delay filter of 500 ms for that video loop
3. Configure a Sync Offset of 500 ms for that video loop
4. Run a recording and do something that causes a lag in OBS, e.g. toggle visibility of an Image source a few times that has the "Unload image when not showing" option enabled and loads a rather big image like this one
5. Check the audio monitoring vs. preview sync and also the recording.
   As stated above repeat the test with OBS restarts between test runs to see the various results, as the behavior is inconsistent.

This is the capture of such a test session showing how monitoring audio vs. preview becomes desync and and this is the recording that was produced by the OBS instance that is seen there showing that also the recorded video file is desync.

Anything else we should know?

What initially brought me to this is investigating A/V desyncs that I had with NDI. I always thought the NDI plugin was at fault. Then Teleport was released, I switched and was confused to see that it suffers from exactly the same issues, so I started to investigate further. My test scenario initially was between two OBS instances on the same PC transferring video and audio to each other by both Teleport and NDI. And surely lags on the sender system (also produced by the Image source) would cause A/V desync for both the NDI and Teleport streams and recordings on the receiving OBS instance.
To make matters worse, if a chain of several OBS instances is used each of them having a small hiccup will cause the whole chain to be desync, because the desync will transfer over NDI (the same for the newer Teleport plugin) through the next instances.

It seems that when lag occurs OBS somehow compensates that lag for itself but doesn't enable any sub system like the Teleport/NDI plugin or filters like Render Delay to do the same (e.g. by not giving them necessary information like corrected timestamps?).

Since the scenario with the Render Delay filter is simpler I decided to open the issue for this. I would assume that any fix for this would also fix the Teleport/NDI issues, if it turns out to be wrong I could still open a follow-up for this.

Bust just in case, here's videos showing these scenarios too:

• Test session with Teleport
• Recording created in that Teleport test session
• Sender and Receiver log for the Teleport test session
• Test session with NDI
• Recording created in that NDI test session
• Sender and Receiver log for the NDI test session

Note that this really is not about the Image source causing lags, although maybe an issue of its own should be opened for this, why can't it load images asynchronously? But that's not the point here, lag can also be caused by a Game Capture source when switching between full-screen game and desktop (especially when they have different screen resolutions) or is randomly caused by the Media source or the system has a short load peak. My scenario was tested on a Ryzen 5800X that never went above 12% even during causing the lags. Saying that lag should just always be prevented is simply not realistic, it can occur on the best systems and as shown can be caused by OBS Studio's very own standard sources. Especially during an hour-long streaming or recording session it's almost impossible to never have any hiccup. Therefore OBS should should always be able to recover after lags including any plugins.

Start by reproducing the reported OBS 27.2.4 scenario with the linked sync-test video, Render Delay and Sync Offset filters, then compare monitoring, recording, NDI and Teleport behavior using the provided logs and captures. Done means short OBS lags no longer leave monitoring or recordings permanently out of sync, while the configured offsets remain effective.