lgi-devs / lgi-devs/lgi

This program creates garbage faster than the garbage collector can collect it

Open
#157 9 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Lua
Stars
491
Forks
81
PR merge metrics
No merged PRs in 30d

Description

I'm not exactly sure what is happening here, but the following program seems to have ever-increasing memory usage:

local lgi = require("lgi")
local glib = lgi.GLib
local gio = lgi.Gio

local count = 1000

local main = glib.MainLoop()

local function launch(i)
	print(i, collectgarbage("count"))

	local launcher = gio.SubprocessLauncher.new(gio.SubprocessFlags.STDOUT_PIPE)
	local proc = launcher:spawnv({"/bin/cat", "/proc/meminfo"})
	local pipe = proc:get_stdout_pipe()
	pipe = gio.DataInputStream.new(pipe)

	local start_read, finish_read
	start_read = function()
		pipe:read_line_async(glib.PRIORITY_DEFAULT, nil, finish_read)
	end
	finish_read = function(obj, res)
		local line, length = obj:read_line_finish(res)
		if line then
			start_read()
		else
			-- End of file
			assert(obj:close())
			assert(proc:wait())
			assert(proc:get_successful())
			if i < count then
				launch(i+1)
			else
				main:quit()
			end
		end
	end
	start_read()
end

launch(1)

main:run()

When setting more agressive GC settings, this no longer happens. For example:

collectgarbage("setstepmul", 1000)
collectgarbage("setpause", 110)

I looked a bit into "things" and I managed to figure out that the finish_read function is still reachable via, for example. debug.getregistry()[34709]. This shows that lots of things are still referenced via the registry, which means via luaL_ref. This function is only used for callables in lgi: lgi_closure_allocated() and lgi_closure_create().
Callables are "left dangling" after use (lgi_guard_create) and the GC can collect them. Apparently the above program causes some behaviour so that garbage accumulates quicker than it can be collected. Perhaps somehow the callables references each other...? (Although I do not really see how this would happen)

Original issue: https://github.com/awesomeWM/awesome/issues/1490

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the Lua program and its memory growth with the stated garbage-collector settings. Then trace the named lgi_closure_allocated(), lgi_closure_create(), and lgi_guard_create() paths, including registry references observed through debug.getregistry(). Done means identifying why callable references remain reachable and adding a regression check showing repeated subprocess reads no longer accumulate garbage.

Written by the indexing model from the issue text.

Assessment

Tech stack
lua
Domain
performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.