numpy / numpy/numpy-quaddtype

Errors using with concurrent.futures

Open
#107 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
24
Forks
6
Avg merge
5d 41m
Merged PRs (30d)
3

Description

Hi. I use concurrent.futures to handle multi-threading/multi-processing. I can get a basic example to work fine with numpy_quaddtype. But when I try my full normal test (that works with standard numpy) I get:

  File "/Users/kaplan/miniforge3/envs/pintdev_arm/lib/python3.14/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/kaplan/miniforge3/envs/pintdev_arm/lib/python3.14/concurrent/futures/process.py", line 242, in _process_worker
    call_item = call_queue.get(block=True)
  File "/Users/kaplan/miniforge3/envs/pintdev_arm/lib/python3.14/multiprocessing/queues.py", line 120, in get
    return _ForkingPickler.loads(res)
           ~~~~~~~~~~~~~~~~~~~~~^^^^^
RuntimeError: Using legacy SETITEM with NULL array object is only supported for basic NumPy DTypes.

I don't know if I am not triggering this function with my basic example or what. I understand that it may be using pickling to pass between the different processes, but don't know if this can be fixed through a change on my end or if this has to be done here or in multiprocessing. Any ideas?

(sorry about the poorly formatted question)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by comparing the working basic example with the full test using concurrent.futures, then trace the failure through multiprocessing/process.py, concurrent/futures/process.py, and multiprocessing/queues.py. Confirm whether the numpy_quaddtype value fails during pickling or unpickling, and document a minimal reproducible case plus the component that needs the fix.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.