PyArray not converting a PyObject with __array_interface__ property, pointer, and writable?
- Lenguaje dominante
- Julia
- Estrellas
- 1.5k
- Forks
- 186
- Métricas de merge de PR
- Sin PR fusionados en 30 d
Descripción
I am trying to obtain a Julia ndarray from a Python interface to a C++ program ([Psi4](/http://www.psicode.org)). Psi4 exports its internal arrays to Python through NumPy Array interface, but one needs to use `np.asarray` for them to behave as normal NumPy arrays. I get the matrix with:
```julia
julia> matrix = psi4.core.Matrix(3,3)
julia> typeof(matrix) # I would expect some Julia Matrix type
PyObject
```
The returned object is not a Julia ndarray, as I expected. Since it is not a traditional NumPy array I did the transformation
with NumPy
```julia
julia> npmatrix = np.asarray(matrix) # delivers a Julia matrix, but ...
3×3 Array{Float64,2}:
0.0 0.0 0.0
0.0 0.0 0.0
0.0 0.0 0.0
julia> npmatrix[1,2] = 4
4
julia> matrix.np # ... npmatrix is a copy, not a view
3×3 Array{Float64,2}:
0.0 0.0 0.0
0.0 0.0 0.0
0.0 0.0 0.0
```
In Python you get a view, but in Julia a copy. Looking inside I found
```julia
julia> matrix.__array_interface__ # It certainly has an array_interface
Dict{Any,Any} with 6 entries:
"shape" => (3, 3) # dimensions (C ordering)
"strides" => nothing # C ordering
"typestr" => " (140701706682928, false) # ptr to first element and writable!
"descr" => [("", " 3
```
It certainly has a NumPy array interface, and it is writable. So I thought that this could be transformed with `PyArray`.
```julia
julia> PyArray(matrix)
ERROR: PyError ($(Expr(:escape, :(ccall(#= .../.julia/packages/PyCall/zqDXB/src/pybuffer.jl:124 =# @pysym(:PyObject_GetBuffer), Cint, (PyPtr, Ref{PyBuffer}, Cint), o, b, flags)))))
BufferError('pybind11_getbuffer(): Internal error')
Stacktrace:
[1] pyerr_check at ../.julia/packages/PyCall/zqDXB/src/exception.jl:60 [inlined]
[2] pyerr_check at .../.julia/packages/PyCall/zqDXB/src/exception.jl:64 [inlined]
[3] _handle_error(::String) at .../.julia/packages/PyCall/zqDXB/src/exception.jl:81
[4] macro expansion at .../.julia/packages/PyCall/zqDXB/src/exception.jl:95 [inlined]
[5] PyBuffer! at .../.julia/packages/PyCall/zqDXB/src/pybuffer.jl:124 [inlined]
[6] PyBuffer(::PyObject, ::Int32) at .../.julia/packages/PyCall/zqDXB/src/pybuffer.jl:119
[7] PyArray_Info(::PyObject) at .../.julia/packages/PyCall/zqDXB/src/pyarray.jl:16
[8] PyArray(::PyObject) at .../.julia/packages/PyCall/zqDXB/src/pyarray.jl:124
[9] top-level scope at REPL[10]:1
```
It seems that PyCall treats it as a buffer when it is not `PyCall.isbuftype(matrix) == false`.
However, I can define a function that accesses the metadata of the NumPy array to
wrap the memory block with a Julia array, although it is very naive, not general, and
probably something better is already in PyCall, maybe `PyArray` can with some guidance?
```julia
julia>function psi4view(psi4matrix) # My ad-hoc solution that assumes Float64, ...
array_interface = psi4matrix.__array_interface__
array_interface["data"][2] == false || @warn "Not writable"
array_interface["strides"] == nothing || @warn "Different ordering than C"
array_interface["typestr"] == " jlmatrix = psi4view(matrix) # It works
3×3 Array{Float64,2}:
0.0 0.0 0.0
0.0 0.0 0.0
0.0 0.0 0.0
```
Effectively, it works (only for "lvalues"), as I want:
```julia
julia> jlmatrix .= 1 # Changing the Julia matrix
3×3 Array{Float64,2}:
1.0 1.0 1.0
1.0 1.0 1.0
1.0 1.0 1.0
julia> matrix.np # changes the original matrix. That is what I want.
3×3 Array{Float64,2}:
1.0 1.0 1.0
1.0 1.0 1.0
1.0 1.0 1.0
```
I do not know what is impeding the transformation (maybe not being a normal NumPy array) but it is possible
and the data is there. I wonder if there is a way I can get the transformation using PyCall instead of relying on my solution.
For more context I have a Jupyter notebook [showing the scenario where it happened](https://github.com/zyth0s/psi4julia/blob/master/Tutorials/01_Psi4Julia-Basics/1c_psi4-numpy-datasharing.ipynb).
If you are interested in reproducing it, the [following instructions](https://github.com/zyth0s/psi4julia#getting-started) tell you the dependencies. I am using Julia 1.4.2 and PyCall v1.91.4.
Thank you in advance,
Daniel
Guía de contribución
No hay ninguna guía de contribución indexada para este repositorio
Evaluación
Este issue todavía no se ha evaluado.