numpy / numpy/numpy

BUG: free-threaded dynamic loop addition and ideally loop replacements in ufuncs

Open
#31,083 8 comments 4 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

00 - Bug 39 - free-threading
Dominant language
Python
Stars
32.8k
Forks
12.8k
Avg merge
1d 7h
Merged PRs (30d)
197

Description

I didn't think this would get back so quickly, but we should allow dynamic addition of loops in ufuncs.
Previously, this was OK at least when done during promotion (which is to me the interesting part -- other times could add their own lock, but this path cannot).

Now, we could just disable the duplicate addition check and things will probably work, but not sure what is best.
I am not sure there is much of a point to not just lock everything when we do promotion (promotion should be very rare!).

I also would like to allow loop replacement eventually, and this has come around a bit quicker now as well.
Now, that requires clearing the cache, which is already similar to what is happening when growing, but the trivial way to add it requires leaking the cache buckets (which growing also currently does?!).

We can just move back the locking one level up, but I do wonder a bit if all of this is actually worth it compared to an RWlock?

Right now, I added dynamic loops for imag/real, but it is internal, so there is the "ignore that it is already added" flag public already.

CC @kumaraditya303 and @ngoldbaum.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No file or test is named. Start by locating the ufunc promotion, dynamic-loop, cache-growth, and locking paths, then trace how free-threaded duplicate loop addition is handled. Done should mean dynamic loop addition is safe during promotion, with a clear approach for cache clearing and eventual loop replacement.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
performance
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.