crash report about using freed video_t

Operating System Info

Windows 11

Other OS

No response

OBS Studio Version

31.0.3

OBS Studio Version (Other)

debugging code of obs 31.0.3

OBS Studio Log URL

I have attched log file

OBS Studio Crash Log URL

No response

Expected Behavior

no crash

Current Behavior

crashed

During debugging code of obs-31.0.3, this crash can be reproduced easiy;
During running released packet of obs-31.0.3, this crash can be reproduced sometimes (because accessing deleted memory will not cause crash often in release mode).

Steps to Reproduce

1. settings dialog -> Advanced -> IP Family -> select "IPV6 Only" (in fact, I have no ipv6. This is only for causing connect failture)
2. settings dialog -> Stream -> select any platform and input streamkey
3. settings dialog -> Output-> advanced mode -> Streaming -> select x264 encoder
4. click "start streaming", alert appeared with "Failed to connect to server"

5. settings dialog -> Video -> only change "Output Resolution"

6. click "start streaming" again, crash happen!

Anything else we should know?

log file during running released obs-31.0.3:
2025-05-27 10-02-58.txt

crash callstack during debugging:

My Analysis Conclusion

1. In step-4, obs_output_signal_stop is called and obs_output_end_data_capture_internal will return directly without creating end_data_capture_thread.

compared with normal stop of streaming, in this situation, obs_encoder_shutdown will not be called.

2. in step-5, obs_reset_video is called and all old video_t will be freed.
   however, BasicOutputHandler is not re-created

3. in step-6, callstack in BasicOutputHandler is as below (as the crash callstack):
   AdvancedOutput::SetupStreaming -> AdvancedOutput::UpdateStreamSettings -> obs_encoder_update -> obs_x264_update -> obs_encoder_get_width -> get_const_root(video_t*)

however, now the video_t is freed in step-5. then crashed!

4. For normal stop of steam, obs_encoder_shutdown will be called and obs_encoder::context.data will be set with NULL.
   So in step-4, obs_encoder_update will return as below:

after that for nomal stop of stream, the new video_t will be set to encoder by obs_encoder_set_video before obs_output_start

Solution？

metho-1
modify OBSBasicSettings::SaveSettings().
BasicOutputHandler should always be re-created if obs_reset_video is called and it should be called after obs_reset_video.

AS-IS

TO-BE

metho-2
in obs_output_end_data_capture_internal, if it is not a normal stop of stream, should we clear any variable before returning ?
But as my understand, end_data_capture must be paired with hook_data_capture. In this crash, hook_data_capture has not been called because RTMP failed to connect server

Since the impact of my two methods is relatively large. I paste they here for talking about.
Maybe they are not right~