Memory correctness issues

Hi,

I just found Lua's HARDMEMTESTS define. When this is set, Lua forces a full GC cycle whenever possible. Thus, I build a Lua interpreter (Lua 5.3.4) with this setting and used that to run LGI's test suite. The result is already running for two hours and so far the following tests failed:

• gireg :103:obj_prop_boxed
• gireg :104:obj_prop_hash
• gireg :105:obj_prop_list
• gobject : 9:subclass_override1
• gobject : 10:subclass_override2

I looked at obj_prop_list and it can be "fixed" by making marshal_2c_list leak the returned list instead of creating a guard for it (lgi_guard_create). The reason for this is that this function is called by marshal_container_marshaller which is in turn called by Lua. Thus, the guard that is created here basically "fires" as soon as the marshaller is done. Ownership of the object is never transferred to Lua. I have no idea how this is supposed to work. Similar things for obj_prop_hash: Make marshal_2c_hash leak and the test starts to pass.

The failures for both subclass_override fail to reproduce. Perhaps some memory corruption due to the previous failures? Yay...

One possible patch with the idea "dunno, this makes stuff work" (No idea if this is correct, this black magic needs @pavouk and is far beyond my actual understanding):

diff --git a/lgi/marshal.c b/lgi/marshal.c
index 555beaf..012a482 100644
--- a/lgi/marshal.c
+++ b/lgi/marshal.c
@@ -720,7 +720,9 @@ marshal_2c_hash (lua_State *L, GITypeInfo *ti, GHashTable **table, int narg,
          hash_func = NULL;
          equal_func = NULL;
        }
-      *guarded_table = *table = g_hash_table_new (hash_func, equal_func);
+      *table = g_hash_table_new (hash_func, equal_func);
+      if (transfer == GI_TRANSFER_NOTHING)
+             *guarded_table = *table;
 
       /* Iterate through Lua table and fill hashtable. */
       lua_pushnil (L);

Start with lgi/marshal.c, especially marshal_2c_list, marshal_2c_hash, and marshal_container_marshaller; run the Lua interpreter built with HARDMEMTESTS against the LGI test suite. Compare the five reported failures, including gireg tests 103-105 and gobject tests 9-10, and establish the correct guard and ownership behavior. Done means these tests pass without memory corruption.