imagej / imagej/pyimagej

Support wrapping a Java-side block of memory allocated off-heap to Python

Open
#203 6 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
534
Forks
95
PR merge metrics
No merged PRs in 30d

Description

With `ByteBuffer.allocateDirect` you can allocate memory off-heap in Java, which can then be shared with other processes. We want to easy manufacturing of numpy arrays and xarrays that wrap this sort of off-heap memory, so that you can directly change data in Python that originated in Java (by some definition of "originated"—since it's off-heap).

In #73, @hanslovsky wrote:
> should be possible already, albeit I don't think there is a convenience method for that. Things may have changed since I was last involved with imglyb (it was still pyjnius back then), but there is/was a way to generate ImgLib2 ArrayImgs backed by native memory and you can then simply pass that pointer into a numpy array.

See also https://github.com/imglib/imglib2-cache-python

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing the ByteBuffer.allocateDirect use case in this issue, then read issue #73 and the linked imglib2-cache-python project for existing native-memory and pointer-handling approaches. The payload names no implementation files or tests; done would mean a documented, tested way to create NumPy or xarray views over Java off-heap memory.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, numpy, python
Domain
backend, data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.