HaxeFoundation / HaxeFoundation/haxe
haxe.Serializer encounters Stack Overflow when processing 31 element TEnum on cpp
- Dominant language
- Haxe
- Stars
- 6.9k
- Forks
- 715
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 11
Description
Dev Environment: Windows
Build command: lime test windows -debug
haxe 4.3.6
lime 8.1.2
openfl 9.3.3
haxeflixel 5.8.0
hxcpp-debugger 1.1
I'm trying to use the hxcpp debugger and the application keeps crashing due to a stack overflow in haxe.Serialize.
Here is what's being passed to Serialize.run()
| Name | Value | Type
-- | -- | -- | --
◢ | __vfptr | 0x00007ff608685428 {FlxProject.exe!void(* debugger::Message_obj::`vftable'[34])()} {0x00007ff60600664e {FlxProject.exe!hx::EnumBase_obj::_hx_isInstanceOf(int)}, ...} | void * *
| [0] | 0x00007ff60600664e {FlxProject.exe!hx::EnumBase_obj::_hx_isInstanceOf(int)} | void *
| [1] | 0x00007ff6060077f1 {FlxProject.exe!hx::EnumBase_obj::__Mark(class hx::MarkContext *)} | void *
| [2] | 0x00007ff606009632 {FlxProject.exe!hx::EnumBase_obj::__Visit(class hx::VisitContext *)} | void *
| [3] | 0x00007ff605ff1a6e {FlxProject.exe!hx::EnumBase_obj::__GetType(void)const } | void *
| [4] | 0x00007ff6060145fa {FlxProject.exe!hx::Object::__GetHandle(void)const } | void *
| [5] | 0x00007ff605ffd238 {FlxProject.exe!hx::Object::__FieldRef(class String const &)} | void *
| [6] | 0x00007ff60602342e {FlxProject.exe!debugger::Message_obj::__ToString(void)const } | void *
| [7] | 0x00007ff605ff7b8f {FlxProject.exe!hx::Object::__ToInt(void)const } | void *
| [8] | 0x00007ff60600e777 {FlxProject.exe!hx::Object::__ToDouble(void)const } | void *
| [9] | 0x00007ff60602170f {FlxProject.exe!hx::Object::__ToInt64(void)const } | void *
| [10] | 0x00007ff606036a51 {FlxProject.exe!hx::Object::__CStr(void)const } | void *
| [11] | 0x00007ff605ff1320 {FlxProject.exe!hx::EnumBase_obj::toString(void)} | void *
| [12] | 0x00007ff60601a28e {FlxProject.exe!hx::Object::__HasField(class String const &)} | void *
| [13] | 0x00007ff606017a93 {FlxProject.exe!debugger::Message_obj::__Field(class String const &,enum hx::PropertyAccessMode)} | void *
| [14] | 0x00007ff6060262a5 {FlxProject.exe!hx::Object::_hx_getInterface(int)} | void *
| [15] | 0x00007ff606033f04 {FlxProject.exe!hx::Object::__SetField(class String const &,struct cpp::Variant const &,enum hx::PropertyAccessMode)} | void *
| [16] | 0x00007ff606038266 {FlxProject.exe!hx::Object::__SetThis(class Dynamic)} | void *
| [17] | 0x00007ff605ff18ac {FlxProject.exe!hx::Object::__Run(class Array const &)} | void *
| [18] | 0x00007ff60602a125 {FlxProject.exe!hx::Object::__GetFieldMap(void)} | void *
| [19] | 0x00007ff6060135c4 {FlxProject.exe!hx::Object::__GetFields(class Array &)} | void *
| [20] | 0x00007ff605ffc2ca {FlxProject.exe!debugger::Message_obj::__GetClass(void)const } | void *
| [21] | 0x00007ff6060213e5 {FlxProject.exe!hx::EnumBase_obj::__Compare(class hx::Object const *)const } | void *
| [22] | 0x00007ff606013dd0 {FlxProject.exe!hx::Object::__length(void)const } | void *
| [23] | 0x00007ff606015054 {FlxProject.exe!hx::EnumBase_obj::__GetItem(int)const } | void *
| [24] | 0x00007ff606033946 {FlxProject.exe!hx::Object::__SetItem(int,class Dynamic)} | void *
| [25] | 0x00007ff60600fe51 {FlxProject.exe!hx::Object::__run(class Dynamic const &,class Dynamic const &,class Dynamic const &,class Dynamic const &,class Dynamic const &)} | void *
| [26] | 0x00007ff6060269ee {FlxProject.exe!hx::Object::__run(class Dynamic const &,class Dynamic const &,class Dynamic const &,class Dynamic const &)} | void *
| [27] | 0x00007ff60602305f {FlxProject.exe!hx::Object::__run(class Dynamic const &,class Dynamic const &,class Dynamic const &)} | void *
| [28] | 0x00007ff605ff96e7 {FlxProject.exe!hx::Object::__run(class Dynamic const &,class Dynamic const &)} | void *
| [29] | 0x00007ff606027b64 {FlxProject.exe!hx::Object::__run(class Dynamic const &)} | void *
| [30] | 0x00007ff60602d433 {FlxProject.exe!hx::Object::__run(void)} | void *
| [31] | 0x00007ff606020c15 {FlxProject.exe!hx::Object::__ArgCount(void)const } | void *
Here's the stack trace.
Unhandled exception at 0x00007FF6080AA4D7 in FlxProject.exe: 0xC00000FD: Stack overflow (parameters: 0x0000000000000001, 0x00000052A5003000).
> FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 387 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::serialize(Dynamic v) Line 1398 C++
FlxProject.exe!haxe::Serializer_obj::run(Dynamic v) Line 1430 C++
FlxProject.exe!debugger::DebuggerThread_obj::debuggerThreadMain() Line 432 C++
debugger.DebuggerThread that contains the Serialize.run() call looks like this, I tried to do some tracing or serializing to generate a reproducible sample but I didn't have much luck.
```
case GetStructured(unsafe, expression):
trace(expression);
var message = this.getStructured(unsafe, expression);
var json = haxe.Json.stringify(message);
trace("JSON INCOMING");
trace("");
trace(json);
trace("");
trace("Stopped?");
trace("");
var serialized = haxe.Serializer.run(message);
trace(serialized);
emit(message);
```
```
debugger/DebuggerThread.hx:254: this
debugger/DebuggerThread.hx:257: JSON INCOMING
debugger/DebuggerThread.hx:258:
debugger/DebuggerThread.hx:259: 28
debugger/DebuggerThread.hx:260:
debugger/DebuggerThread.hx:261: Stopped?
debugger/DebuggerThread.hx:262:
```
Line 1398 in haxe.Serializer.cpp is `this->serialize(enumBase->_hx_getParamI(p));`
```
HXLINE( 482) {
HXLINE( 482) int _g5 = 0;
HXDLIN( 482) int _g6 = len;
HXDLIN( 482) while((_g5 < _g6)){
HXLINE( 482) _g5 = (_g5 + 1);
HXDLIN( 482) HX_VARI( int,p) = (_g5 - 1);
HXLINE( 483) this->serialize(enumBase->_hx_getParamI(p));
}
}
```
It looks like this line in the source code: https://github.com/HaxeFoundation/haxe/blob/6863504c3ad4a48e4fc2ef3e4a1ccb641658f0c6/std/haxe/Serializer.hx#L472
Contributor guide
Research direction
Start at haxe.Serializer.run() and haxe.Serializer_obj::serialize(), particularly the generated C++ locations shown in the stack trace. Reproduce with a 31-element TEnum on the cpp target using the reported Windows build command, then verify serialization completes without a stack overflow and add coverage for the failing case if the repository's tests provide a suitable entry point.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100