JuliaPy / JuliaPy/PythonCall.jl
More convenient conversion
- Lenguaje dominante
- Julia
- Estrellas
- 1.1k
- Forks
- 86
- Merge medio
- 1 d 22 h
- PR fusionados (30 d)
- 3
Descripción
First off, thanks for all the effort you've put into this package! It's been wonderfully useful, and the integration with CondaPkg is just :ok_hand:.
Unfortunately, at the moment, it feels like whenever I want to write functions that work with python, or (even worse) python *and* julia I have to engage in a somewhat annoying dance with `pyconvert`.
I can't help but feel that the situation could be much nicer.
### On `pyconvert` and `convert`
As an example consider if I want to write an MSE function that works with data from python and Julia. At the moment, I'd need to do something like this:
```julia
function mse(y, ŷ)
native_y = if y isa Py
pyconvert(Vector, y)
else y end
native_ŷ = if ŷ isa Py
pyconvert(Vector, ŷ)
else ŷ end
sum((native_y .- native_ŷ).^2)
end
```
I find myself regularly writing `convert` on instinct, getting method errors, and feeling tricked! — as that's the muscle memory I have in julia for "I want type X, I have y". From what I can tell, defining `Base.convert(T::Type, obj::Py)` wouldn't cause any issues, so I'm confused by why `pyconvert` actually exists?
Furthermore, since `convert(::Type{T}, t::T)` just returns `t`, it would allow for equivalent but much simpler code in quite a few places I think. Returning to the earlier example, the `mse` function could be rewritten as:
```julia
mse(y, ŷ) = sum((convert(Vector, y) .- convert(Vector, ŷ).^2)
```
I imagine this change could be made in a straightforward backwards compatible way like so:
```julia
Base.convert(T::Type, obj::Py) = pyconvert(T, obj)
```
### On type information
Currently, `Py` is a struct a single field and no parameters. I feel like it would be nice to expose python type information. Short of constructing a python type tree, perhaps it would be possible to cheaply encode some type information in a type parameter? Just as an example, say `Py` was changed to:
```julia
mutable struct Py{pytype}
ptr :: C.PyPtr
Py(::Val{:new}, ptr::C.PyPtr) = finalizer(py_finalizer, new(ptr))
end
```
From a quick benchmark, `pytype` seems very fast to run (10ns on my machine), perhaps it one could use this to populate `pytype` information? E.g. `Py{:list}`? There's likely a better way of doing this, it just occurs to me that something along these lines could be helpful. This definitely needs more thought.
### On automatic conversion
The fact that PythonCall does not eagerly make best-guess conversions from Python to Julia types has been quite good from a performance perspective, but I feel like there is still room for a convenience function to "just give me X in pure Julia form", say via a `pynative` function or similar.
It occurs to me that should `Py` have type annotations, as proposed earlier, then this might be able to be done quite cleanly with multiple dispatch.
```julia
pynative(juliatype::Any) = juliatype
pynative(::Py{T}) where {T} = throw(MethodError(pynative, Tuple{Py{T}}))
pynative(list::Py{:list}) = map(pynative, convert(Vector, list))
...
```
### On type promotion
I imagine guessing when it would make sense to convert to Julia would be a bit fraught, so while this would be nice theoretically, I have no particular ideas here.
Guía de contribución
No hay ninguna guía de contribución indexada para este repositorio
Línea de trabajo
Start with the issue's discussion of pyconvert, Base.convert, Py type parameters, and a possible pynative function; no source files or tests are identified. Compare these proposals and determine a focused, backward-compatible conversion design, with completion defined by an agreed scope and corresponding behavior.
Escrito por el modelo de indexación a partir del texto del issue.
Evaluación
- Stack tecnológico
- julia, python
- Área
- api, backend
- Tipo de issue
- Nueva funcionalidad
- Dificultad
- 5/5
- Tiempo estimado
- Más de una semana
- Estado de actividad
- Estancado
- Claridad
- Necesita aclaración
- Aptitud para principiantes
- 25/100