JuliaPy / JuliaPy/PythonCall.jl

Make `to_numpy` conversions consistent with NumPy `dtype`

Aberta
#719 7 comentários 0 reações 0 responsáveis Ver no GitHub
Linguagem predominante
Julia
Estrelas
1.1k
Forks
86
Merge médio
1d 22h
PRs com merge (30d)
3

Descrição

Moved discussion in #462 into this separate issue.

Other PythonCall issues related to NumPy `dtype` and Numpy arrays that may be relevant:
- #319
- #439
- #441
- #486

My system details (click to expand)

### Julia

```julia
julia> versioninfo()
Julia Version 1.12.1
Commit ba1e628ee49 (2025-10-17 13:02 UTC)
Build Info:
Official https://julialang.org release
Platform Info:
OS: Linux (x86_64-linux-gnu)
CPU: 48 × AMD EPYC 7V13 64-Core Processor
WORD_SIZE: 64
LLVM: libLLVM-18.1.7 (ORCJIT, znver3)
GC: Built with stock GC
Threads: 1 default, 1 interactive, 1 GC (on 48 virtual cores)

julia> Pkg.status()
Status `~/temp/Project.toml`
[992eb4ea] CondaPkg v0.2.33
[6099a3de] PythonCall v0.9.28
```

### Python Environment

- python 3.13.9
- numpy 2.3.4

I noticed a difference in how the PythonCall [`to_numpy()`](https://github.com/JuliaPy/PythonCall.jl/blob/71666ca0a30eafee2456a191fa2490cdd0b0ed22/src/JlWrap/array.jl#L374) function and NumPy treat property values of `dtype` as follows:

```julia
julia> using PythonCall

julia> Py(rand(10)).to_numpy().dtype
Python: dtype('float64')

julia> Py([[1,2,3], [4,5,6]]).to_numpy().dtype
Python: dtype('O')
```

I would expect the latter to be `dtype('int64')` to match Python:

```python
>>> import numpy
>>> a = numpy.array([[1,2,3], [4,5,6]])
>>> a.dtype
dtype('int64')
```

While Julia does provide `Matrix{Int64}` and `to_numpy()` outputs `dtype('int64')` as I would expect
```julia
julia> [1 2 3; 4 5 6] |> typeof
Matrix{Int64} (alias for Array{Int64, 2})
```
```python
julia> Py([1 2 3; 4 5 6]).to_numpy().dtype
Python: dtype('int64')
```

I am working on a project where I need to use PythonCall to deal with `Vector{Vector{Int64}}` instances and changing into `Matrix{Int64}` is not an option.

In Julia, using PythonCall `to_numpy()`, it is clear that `dtype('int64')` is only output for `Vector{Int64}` not `Vector{Vector{Int64}}` nor `Vector{Vector{Vector{Int64}}}` regardless of how many levels of nesting:

```julia
julia> Py([1, 2, 3]).to_numpy().dtype
Python: dtype('int64')

julia> Py([[1,2,3], [4,5,6]]).to_numpy().dtype
Python: dtype('O')

julia> Py([[[1,2,3], [4,5,6]], [[7,8,9], [10,11,12]]]).to_numpy().dtype
Python: dtype('O')
```

Whereas in Python, `dtype('int64')` is output for all the above:

```python
>>> numpy.array([1, 2, 3]).dtype
dtype('int64')

>>> numpy.array([[1,2,3], [4,5,6]]).dtype
dtype('int64')

>>> numpy.array([[[1,2,3], [4,5,6]], [[7,8,9], [10,11,12]]]).dtype
dtype('int64')
```

Thus the value of the property `.dtype` in NumPy is defined based on the innermost elements in the array, whereas in Julia the value of `.dtype` upon using `to_numpy()` is not based on the innermost elements (i.e. 1, 2, 3, etc) but on the whole structure containing them (i.e `Vector{Int64}` for the 2nd array, and `Vector{Vector{Int64}}` for the 3rd array).

I was expecting the same behaviour from Python's NumPy and the conversions from `to_numpy()` given by PythonCall, but it turns out the conversion does not agree with NumPy on `dtype` property values.

Guia de contribuição

Nenhum guia de contribuição indexado para este repositório

Direção de pesquisa

Start with src/JlWrap/array.jl at the linked to_numpy() implementation, then compare its handling of nested vectors with the NumPy examples in the issue. Done means nested integer arrays produce dtype values consistent with NumPy, including the two- and three-dimensional cases described here.

Escrita pelo modelo de indexação a partir do texto da issue.

Avaliação

Stack de tecnologia
numpy, python
Domínio
data
Tipo de issue
Bug
Dificuldade
4/5
Tempo estimado
3-5 dias
Status de atividade
Estagnada
Clareza
Razoavelmente clara
Facilidade para iniciantes
35/100

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.