JuliaPy / JuliaPy/PythonCall.jl
More convenient conversion
- Dominant language
- Julia
- Stars
- 1.1k
- Forks
- 86
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 3
Description
First off, thanks for all the effort you've put into this package! It's been wonderfully useful, and the integration with CondaPkg is just :ok_hand:.
Unfortunately, at the moment, it feels like whenever I want to write functions that work with python, or (even worse) python *and* julia I have to engage in a somewhat annoying dance with `pyconvert`.
I can't help but feel that the situation could be much nicer.
### On `pyconvert` and `convert`
As an example consider if I want to write an MSE function that works with data from python and Julia. At the moment, I'd need to do something like this:
```julia
function mse(y, ŷ)
native_y = if y isa Py
pyconvert(Vector, y)
else y end
native_ŷ = if ŷ isa Py
pyconvert(Vector, ŷ)
else ŷ end
sum((native_y .- native_ŷ).^2)
end
```
I find myself regularly writing `convert` on instinct, getting method errors, and feeling tricked! — as that's the muscle memory I have in julia for "I want type X, I have y". From what I can tell, defining `Base.convert(T::Type, obj::Py)` wouldn't cause any issues, so I'm confused by why `pyconvert` actually exists?
Furthermore, since `convert(::Type{T}, t::T)` just returns `t`, it would allow for equivalent but much simpler code in quite a few places I think. Returning to the earlier example, the `mse` function could be rewritten as:
```julia
mse(y, ŷ) = sum((convert(Vector, y) .- convert(Vector, ŷ).^2)
```
I imagine this change could be made in a straightforward backwards compatible way like so:
```julia
Base.convert(T::Type, obj::Py) = pyconvert(T, obj)
```
### On type information
Currently, `Py` is a struct a single field and no parameters. I feel like it would be nice to expose python type information. Short of constructing a python type tree, perhaps it would be possible to cheaply encode some type information in a type parameter? Just as an example, say `Py` was changed to:
```julia
mutable struct Py{pytype}
ptr :: C.PyPtr
Py(::Val{:new}, ptr::C.PyPtr) = finalizer(py_finalizer, new(ptr))
end
```
From a quick benchmark, `pytype` seems very fast to run (10ns on my machine), perhaps it one could use this to populate `pytype` information? E.g. `Py{:list}`? There's likely a better way of doing this, it just occurs to me that something along these lines could be helpful. This definitely needs more thought.
### On automatic conversion
The fact that PythonCall does not eagerly make best-guess conversions from Python to Julia types has been quite good from a performance perspective, but I feel like there is still room for a convenience function to "just give me X in pure Julia form", say via a `pynative` function or similar.
It occurs to me that should `Py` have type annotations, as proposed earlier, then this might be able to be done quite cleanly with multiple dispatch.
```julia
pynative(juliatype::Any) = juliatype
pynative(::Py{T}) where {T} = throw(MethodError(pynative, Tuple{Py{T}}))
pynative(list::Py{:list}) = map(pynative, convert(Vector, list))
...
```
### On type promotion
I imagine guessing when it would make sense to convert to Julia would be a bit fraught, so while this would be nice theoretically, I have no particular ideas here.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.