godotengine / godotengine/godot

Out of memory in SubViewport destructor

Open
#87,499 1 comment 0 reactions 0 assignees View on GitHub
bug crash topic:rendering
Dominant language
C++
Stars
117k
Forks
26.8k
PR merge metrics
PR metrics pending

Description

### Tested versions

4.3.dev.custom_build. 6fea273ed

### System information

Ubuntu 22.04 CI

### Issue description

When executing (this code was automatically minimized, so it is possible, that an even more "minimal" project can be created)
```
extends Node
func _process(delta):
var temp_variable14793 = SubViewport.new()
add_child(temp_variable14793)
var temp_argument14793_f7_0 = NinePatchRect.new()
temp_variable14793.set_editable_instance(temp_argument14793_f7_0, true)
temp_argument14793_f7_0.queue_free()
temp_variable14793.queue_free()
var temp_variable14796 = SubViewport.new()
add_child(temp_variable14796)
temp_variable14796.set_snap_controls_to_pixels(false)
temp_variable14796.set_positional_shadow_atlas_quadrant_subdiv(2, 6)
temp_variable14796.set_process_shortcut_input(true)
temp_variable14796.queue_free()
```
Godot crashes:
```
Godot Engine v4.3.dev.custom_build.6fea273ed - https://godotengine.org
ERROR: Condition "!is_ancestor_of(p_node)" is true.
at: set_editable_instance (scene/main/node.cpp:2356)
=================================================================
==25360==ERROR: AddressSanitizer: allocator is out of memory trying to allocate 0x2000000020 bytes
#0 0x7f1e5dab4887 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145
#1 0x55fb470664d3 in Memory::alloc_static(unsigned long, bool) core/os/memory.cpp:75
#2 0x55fb443fc775 in Error CowData::resize(long) core/templates/cowdata.h:318
#3 0x55fb443d808c in Vector::resize(long) core/templates/vector.h:95
#4 0x55fb443a8bcb in RendererRD::LightStorage::shadow_atlas_set_size(RID, int, bool) servers/rendering/renderer_rd/storage_rd/light_storage.cpp:2006
#5 0x55fb443a72b3 in RendererRD::LightStorage::shadow_atlas_free(RID) servers/rendering/renderer_rd/storage_rd/light_storage.cpp:1969
#6 0x55fb465b6f15 in RendererViewport::free(RID) servers/rendering/renderer_viewport.cpp:1426
#7 0x55fb4376218e in RenderingServerDefault::_free(RID) servers/rendering/rendering_server_default.cpp:57
#8 0x55fb43826f08 in RenderingServerDefault::free(RID) servers/rendering/rendering_server_default.h:992
#9 0x55fb3c6c52fe in Viewport::~Viewport() scene/main/viewport.cpp:4937
#10 0x55fb3c6d50cb in SubViewport::~SubViewport() scene/main/viewport.cpp:5134
#11 0x55fb2dcf3873 in void memdelete(Object*) core/os/memory.h:109
#12 0x55fb3c4d3ea2 in SceneTree::_flush_delete_queue() scene/main/scene_tree.cpp:1355
#13 0x55fb3c4b97d3 in SceneTree::process(double) scene/main/scene_tree.cpp:516
#14 0x55fb2dcb7c83 in Main::iteration() main/main.cpp:3814
#15 0x55fb2d9bb805 in OS_LinuxBSD::run() platform/linuxbsd/os_linuxbsd.cpp:933
#16 0x55fb2d999e8b in main platform/linuxbsd/godot_linuxbsd.cpp:74
#17 0x7f1e5ce29d8f (/lib/x86_64-linux-gnu/libc.so.6+0x29d8f)
==25360==HINT: if you don't care about these errors you may set allocator_may_return_null=1
SUMMARY: AddressSanitizer: out-of-memory ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145 in __interceptor_malloc
==25360==ABORTING
```

This example was found by Godot fuzzer - [Qarminer](https://github.com/qarmin/Qarminer), so it is quite unlikelly that this code could be used in real project, but still this should be handled gracefully.

Memory leaks or asan backtraces are visible when using Godot build with sanitizers support - https://github.com/qarmin/GodotBuilds/actions (linux -> linux-editor-sanitizers)

### Steps to reproduce

Above

### Minimal reproduction project (MRP)

Above

Contributor guide

Open the contributing guide

Research direction

Start with the reproducer in the issue and the sanitizer stack, then inspect scene/main/viewport.cpp around Viewport::~Viewport() and SubViewport::~SubViewport(). Trace the cleanup into renderer_viewport.cpp and servers/rendering/renderer_rd/storage_rd/light_storage.cpp, especially shadow_atlas_free() and shadow_atlas_set_size(). Done means the reproducer no longer crashes or attempts an excessive allocation, with sanitizer builds passing.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
computer-graphics, game-dev
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.