SciML / SciML/RuntimeGeneratedFunctions.jl
Can RuntimeGeneratedFunctions.jl cause memory leak issue?
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 112
- Forks
- 19
- Avg merge
- 6h 11m
- Merged PRs (30d)
- 9
Description
Hi,
I am writing a Genetic Programming-styled code to search for solutions to specific problems. The solutions are in the form of functions. Thus, I use RuntimeGeneratedFunctions.jl to generate functions in runtime in order to evaluate their fitnesses. As the code runs (on WSL) and time goes by, the amount of available RAM on my computer becomes less and less until the system forcibly closes the terminal. I suspect it is due to the generated functions. I wonder if it is a known problem and if there exists a solution. Thank you.
Here is the part of the code that involves RuntimeGeneratedFunctions.jl:
function eval_solution(expr, data, eval_genfunc)
f = expr
f1 = @RuntimeGeneratedFunction(f)
fitness = evaluate_genfunc(f1, data)
return fitness
end
Here, expr is the Expr containing the content of the function to be generated, data is the data necessary to calculate the fitness of the generated function, eval_genfunc is a custom function to calculate the fitness of a generated function. eval_genfunc looks like this:
function eval_genfunc(f1, data)
parameters = f1(data)
score = g(parameters) % g performs a simulation with given parameters and extracts some information from there as the score
return score
end
The function eval_solution( ) is used in multithreading mode in a main function:
function main(...)
...
while iterate > 0
Threads.@threads for i in n_threads
expr = ... % Calling the function to generate an expr
fitness = eval_solution(expr, data, eval_genfunc)
...
end
...
iterate -= 1
end
...
end
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the eval_solution and eval_genfunc entry points, then reproduce the threaded main loop that repeatedly calls @RuntimeGeneratedFunction. Profile memory around generated-function creation and the simulation in g, including whether retained Expr or function objects accumulate. Done means a minimal reproducible case identifies the leaking component or shows that no package issue can be confirmed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100