ERROR in finalizer: ?

Problem statement

On testing of a private module with a significant amount of code I got the following output at end of 5 minutes of tests[^1]:

ERROR in finalizer: ?

The error message has a question, but I have a question too: how to understand which finalizer fails?

My way to answer is to ask a LuaJIT wizard @Buristan, which performs some gdb magic and points the needed finalizer. The problem is that Sergey is a human and we can't scale him. We need other ways to debug the finalizers.

Investigation

Let's create a minimal reproducer:

#!/usr/bin/env tarantool

local function myerror(msg)
    local err = newproxy(true)
    getmetatable(err).__tostring = function()
        return msg
    end
    return err
end

local ud = newproxy(true)
getmetatable(ud).__gc = function()
    error(myerror('my error message'))
end

ud = nil
collectgarbage()

Run it using tarantool (3.3.0-entrypoint-160-g11145f6a82):

$ tarantool t.lua 
ERROR in finalizer: ?

Run it using LuaJIT (2.1.0-beta3):

$ luajit t.lua
luajit: my error message
stack traceback:
	[C]: in function 'error'
	t2.lua:13: in function <t2.lua:12>
	[C]: in function 'collectgarbage'
	t2.lua:17: in main chunk
	[C]: at 0x55fc2665bfe0

The code that prints the error is the following:

static int error_finalizer(lua_State *L)
{
  const char *s = lua_tostring(L, -1);
  fputs("ERROR in finalizer: ", stderr);
  fputs(s ? s : "?", stderr);
  fputc('\n', stderr);
  fflush(stderr);
  return 0;
}

Note that lua_tostring() doesn't call the __tostring metamethod. It looks like tarantool and LuaJIT both should print ?, but LuaJIT surprisingly shows the error message.

The reason (pointed to me by @Buristan) is that luajit executable (unlike the LuaJIT library) has the following top-level exception handler:

static int traceback(lua_State *L)
{
  if (!lua_isstring(L, 1)) { /* Non-string error object? Try metamethod. */
    if (lua_isnoneornil(L, 1) ||
	!luaL_callmeta(L, 1, "__tostring") ||
	!lua_isstring(L, -1))
      return 1;  /* Return non-string error object. */
    lua_remove(L, 1);  /* Replace object by result of __tostring metamethod. */
  }
  luaL_traceback(L, L, lua_tostring(L, 1), 1);
  return 1;
}

It calls the __tostring metamethod (and adds the traceback to the error message).

[^1]: The issue in the module: https://github.com/tarantool/etcd-client/issues/88.

Start with src/lib_aux.c#error_finalizer and src/luajit.c#traceback, then run the minimal t.lua reproducer with Tarantool and LuaJIT. Trace how finalizer errors reach the handler and compare the resulting diagnostics. Done means the failing finalizer can be identified from the emitted error output rather than showing only “?”.