mcdc-project / mcdc-project/mcdc

Early Memory Deallocation/Re-Use in Local Arrays

Open
#224 4 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

stale
Dominant language
Python
Stars
61
Forks
38
Avg merge
1d 13h
Merged PRs (30d)
10

Description

While working on the AMD functionality, I came across an issue in Numba. I'm also filing an issue with the Numba folks, but until they get back, be advised that memory associated with arrays can (apparently) be freed and re-used before the last reference to its content is used.

Here's a minimum viable reproduction:

import numba
import numpy as np

kind = np.dtype([("member", np.int64)])

@numba.njit()
def func_a(param_record):
    local_record = np.zeros(1,kind)[0]
    local_record["member"] = 12345
    print(param_record["member"])


@numba.njit()
def func_b():
    local_record = np.zeros(1,kind)[0]
    local_record["member"] = 67890
    func_a(local_record)

func_b()

This program, which should print 67890 (the value most recently assigned to func_b's local_record), actually prints 12345 (the value most recently assigned to func_a's local_record).
It looks like Numba thinks we aren't using the local_record from func_b anymore because we are passing one of its elements, rather than the whole thing.
The memory associated with func_b's local_record gets re-used as the local_record for func_a, and now we have two separate variables seemingly occupying the same memory.

Here is a fix that works for the mvr:

import numba
import numpy as np

kind = np.dtype([("member", np.int64)])

@numba.njit()
def func_a(param_record):
    local_record = np.zeros(1,kind)[0]
    local_record["member"] = 12345
    print(param_record[0]["member"])


@numba.njit()
def func_b():
    local_record = np.zeros(1,kind)
    local_record[0]["member"] = 67890
    func_a(local_record)

func_b()

It's a bit clunky, but it may be our only option in the short term. I'm tweaking my AMD branch to match this fix. I'll let you folks know how it goes, in case you want to perform similar changes to dev ahead of the merge.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the minimum reproduction centered on the Numba entry points func_a and func_b, then compare it with the provided array-based workaround. Done means the original program prints 67890 and the referenced local array storage is not freed or reused before its content is consumed.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.