JuliaPy / JuliaPy/PythonCall.jl

More convenient conversion

Open
#267 6 comments 1 reaction 0 assignees View on GitHub
Dominant language
Julia
Stars
1.1k
Forks
86
Avg merge
1d 22h
Merged PRs (30d)
3

Description

First off, thanks for all the effort you've put into this package! It's been wonderfully useful, and the integration with CondaPkg is just :ok_hand:.

Unfortunately, at the moment, it feels like whenever I want to write functions that work with python, or (even worse) python *and* julia I have to engage in a somewhat annoying dance with `pyconvert`.

I can't help but feel that the situation could be much nicer.

### On `pyconvert` and `convert`

As an example consider if I want to write an MSE function that works with data from python and Julia. At the moment, I'd need to do something like this:

```julia
function mse(y, ŷ)
native_y = if y isa Py
pyconvert(Vector, y)
else y end
native_ŷ = if ŷ isa Py
pyconvert(Vector, ŷ)
else ŷ end
sum((native_y .- native_ŷ).^2)
end
```

I find myself regularly writing `convert` on instinct, getting method errors, and feeling tricked! — as that's the muscle memory I have in julia for "I want type X, I have y". From what I can tell, defining `Base.convert(T::Type, obj::Py)` wouldn't cause any issues, so I'm confused by why `pyconvert` actually exists?

Furthermore, since `convert(::Type{T}, t::T)` just returns `t`, it would allow for equivalent but much simpler code in quite a few places I think. Returning to the earlier example, the `mse` function could be rewritten as:

```julia
mse(y, ŷ) = sum((convert(Vector, y) .- convert(Vector, ŷ).^2)
```

I imagine this change could be made in a straightforward backwards compatible way like so:

```julia
Base.convert(T::Type, obj::Py) = pyconvert(T, obj)
```

### On type information

Currently, `Py` is a struct a single field and no parameters. I feel like it would be nice to expose python type information. Short of constructing a python type tree, perhaps it would be possible to cheaply encode some type information in a type parameter? Just as an example, say `Py` was changed to:

```julia
mutable struct Py{pytype}
ptr :: C.PyPtr
Py(::Val{:new}, ptr::C.PyPtr) = finalizer(py_finalizer, new(ptr))
end
```

From a quick benchmark, `pytype` seems very fast to run (10ns on my machine), perhaps it one could use this to populate `pytype` information? E.g. `Py{:list}`? There's likely a better way of doing this, it just occurs to me that something along these lines could be helpful. This definitely needs more thought.

### On automatic conversion

The fact that PythonCall does not eagerly make best-guess conversions from Python to Julia types has been quite good from a performance perspective, but I feel like there is still room for a convenience function to "just give me X in pure Julia form", say via a `pynative` function or similar.

It occurs to me that should `Py` have type annotations, as proposed earlier, then this might be able to be done quite cleanly with multiple dispatch.

```julia
pynative(juliatype::Any) = juliatype
pynative(::Py{T}) where {T} = throw(MethodError(pynative, Tuple{Py{T}}))
pynative(list::Py{:list}) = map(pynative, convert(Vector, list))
...
```

### On type promotion

I imagine guessing when it would make sense to convert to Julia would be a bit fraught, so while this would be nice theoretically, I have no particular ideas here.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.