mcdc-project / mcdc-project/mcdc
Early Memory Deallocation/Re-Use in Local Arrays
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 61
- Forks
- 38
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 10
Description
While working on the AMD functionality, I came across an issue in Numba. I'm also filing an issue with the Numba folks, but until they get back, be advised that memory associated with arrays can (apparently) be freed and re-used before the last reference to its content is used.
Here's a minimum viable reproduction:
import numba
import numpy as np
kind = np.dtype([("member", np.int64)])
@numba.njit()
def func_a(param_record):
local_record = np.zeros(1,kind)[0]
local_record["member"] = 12345
print(param_record["member"])
@numba.njit()
def func_b():
local_record = np.zeros(1,kind)[0]
local_record["member"] = 67890
func_a(local_record)
func_b()
This program, which should print 67890 (the value most recently assigned to func_b's local_record), actually prints 12345 (the value most recently assigned to func_a's local_record).
It looks like Numba thinks we aren't using the local_record from func_b anymore because we are passing one of its elements, rather than the whole thing.
The memory associated with func_b's local_record gets re-used as the local_record for func_a, and now we have two separate variables seemingly occupying the same memory.
Here is a fix that works for the mvr:
import numba
import numpy as np
kind = np.dtype([("member", np.int64)])
@numba.njit()
def func_a(param_record):
local_record = np.zeros(1,kind)[0]
local_record["member"] = 12345
print(param_record[0]["member"])
@numba.njit()
def func_b():
local_record = np.zeros(1,kind)
local_record[0]["member"] = 67890
func_a(local_record)
func_b()
It's a bit clunky, but it may be our only option in the short term. I'm tweaking my AMD branch to match this fix. I'll let you folks know how it goes, in case you want to perform similar changes to dev ahead of the merge.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the minimum reproduction centered on the Numba entry points func_a and func_b, then compare it with the provided array-based workaround. Done means the original program prints 67890 and the referenced local array storage is not freed or reused before its content is consumed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- compilers
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100