PyArray not converting a PyObject with __array_interface__ property, pointer, and writable?
- Langage dominant
- Julia
- Étoiles
- 1.5k
- Forks
- 186
- Métriques de merge des PR
- Aucune PR mergée en 30 j
Description
I am trying to obtain a Julia ndarray from a Python interface to a C++ program ([Psi4](/http://www.psicode.org)). Psi4 exports its internal arrays to Python through NumPy Array interface, but one needs to use `np.asarray` for them to behave as normal NumPy arrays. I get the matrix with:
```julia
julia> matrix = psi4.core.Matrix(3,3)
julia> typeof(matrix) # I would expect some Julia Matrix type
PyObject
```
The returned object is not a Julia ndarray, as I expected. Since it is not a traditional NumPy array I did the transformation
with NumPy
```julia
julia> npmatrix = np.asarray(matrix) # delivers a Julia matrix, but ...
3×3 Array{Float64,2}:
0.0 0.0 0.0
0.0 0.0 0.0
0.0 0.0 0.0
julia> npmatrix[1,2] = 4
4
julia> matrix.np # ... npmatrix is a copy, not a view
3×3 Array{Float64,2}:
0.0 0.0 0.0
0.0 0.0 0.0
0.0 0.0 0.0
```
In Python you get a view, but in Julia a copy. Looking inside I found
```julia
julia> matrix.__array_interface__ # It certainly has an array_interface
Dict{Any,Any} with 6 entries:
"shape" => (3, 3) # dimensions (C ordering)
"strides" => nothing # C ordering
"typestr" => " (140701706682928, false) # ptr to first element and writable!
"descr" => [("", " 3
```
It certainly has a NumPy array interface, and it is writable. So I thought that this could be transformed with `PyArray`.
```julia
julia> PyArray(matrix)
ERROR: PyError ($(Expr(:escape, :(ccall(#= .../.julia/packages/PyCall/zqDXB/src/pybuffer.jl:124 =# @pysym(:PyObject_GetBuffer), Cint, (PyPtr, Ref{PyBuffer}, Cint), o, b, flags)))))
BufferError('pybind11_getbuffer(): Internal error')
Stacktrace:
[1] pyerr_check at ../.julia/packages/PyCall/zqDXB/src/exception.jl:60 [inlined]
[2] pyerr_check at .../.julia/packages/PyCall/zqDXB/src/exception.jl:64 [inlined]
[3] _handle_error(::String) at .../.julia/packages/PyCall/zqDXB/src/exception.jl:81
[4] macro expansion at .../.julia/packages/PyCall/zqDXB/src/exception.jl:95 [inlined]
[5] PyBuffer! at .../.julia/packages/PyCall/zqDXB/src/pybuffer.jl:124 [inlined]
[6] PyBuffer(::PyObject, ::Int32) at .../.julia/packages/PyCall/zqDXB/src/pybuffer.jl:119
[7] PyArray_Info(::PyObject) at .../.julia/packages/PyCall/zqDXB/src/pyarray.jl:16
[8] PyArray(::PyObject) at .../.julia/packages/PyCall/zqDXB/src/pyarray.jl:124
[9] top-level scope at REPL[10]:1
```
It seems that PyCall treats it as a buffer when it is not `PyCall.isbuftype(matrix) == false`.
However, I can define a function that accesses the metadata of the NumPy array to
wrap the memory block with a Julia array, although it is very naive, not general, and
probably something better is already in PyCall, maybe `PyArray` can with some guidance?
```julia
julia>function psi4view(psi4matrix) # My ad-hoc solution that assumes Float64, ...
array_interface = psi4matrix.__array_interface__
array_interface["data"][2] == false || @warn "Not writable"
array_interface["strides"] == nothing || @warn "Different ordering than C"
array_interface["typestr"] == " jlmatrix = psi4view(matrix) # It works
3×3 Array{Float64,2}:
0.0 0.0 0.0
0.0 0.0 0.0
0.0 0.0 0.0
```
Effectively, it works (only for "lvalues"), as I want:
```julia
julia> jlmatrix .= 1 # Changing the Julia matrix
3×3 Array{Float64,2}:
1.0 1.0 1.0
1.0 1.0 1.0
1.0 1.0 1.0
julia> matrix.np # changes the original matrix. That is what I want.
3×3 Array{Float64,2}:
1.0 1.0 1.0
1.0 1.0 1.0
1.0 1.0 1.0
```
I do not know what is impeding the transformation (maybe not being a normal NumPy array) but it is possible
and the data is there. I wonder if there is a way I can get the transformation using PyCall instead of relying on my solution.
For more context I have a Jupyter notebook [showing the scenario where it happened](https://github.com/zyth0s/psi4julia/blob/master/Tutorials/01_Psi4Julia-Basics/1c_psi4-numpy-datasharing.ipynb).
If you are interested in reproducing it, the [following instructions](https://github.com/zyth0s/psi4julia#getting-started) tell you the dependencies. I am using Julia 1.4.2 and PyCall v1.91.4.
Thank you in advance,
Daniel
Guide de contribution
Aucun guide de contribution indexé pour ce dépôt
Évaluation
Cette issue n'a pas encore été évaluée.