JuliaPy / JuliaPy/PythonCall.jl

PythonCall w/ Jax: Fast inference w/ numpy but only jax.numpy works for jax.grad

Đang mở
#273 8 bình luận 0 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
Julia
Star
1.1k
Fork
86
Merge trung bình
1 ngày 22 giờ
Pull request đã merge (30 ngày)
3

Mô tả

I am trying to use Jax in my Julia codebase for something that Zygote cannot do well (meta-learning). Someone recommended PythonCall as a solution to some issues I was having with PyCall.

So far, PythonCall has been great. Things work and it is generally quite quick.

There is one pain point: `jax.grad` should work with `numpy.array` (it does in Python) but it errors with PythonCall:

```
PyException( as an abstract array; it does not have a dtype attribute")>)
````

Instead, `jax.numpy.array` works with `jax.grad` but it is slower.

I assume this is due to the way PythonCall handles non-copying conversions under the hood.

Is there any way to reconcile these two and get `jax.grad` to work with `numpy.array` and the optimizations for non-copying arrays?

I've included an MWE:

```
using PythonCall
using Flux

jax = pyimport("jax")
jnp = pyimport("jax.numpy")
np = pyimport("numpy")
stax = pyimport("jax.example_libraries.stax")
optimizers = pyimport("jax.example_libraries.optimizers")
optax = pyimport("optax")
random = pyimport("jax.random")

rngkey = random.PRNGKey(123)

dense = stax.Dense
Relu = stax.Relu

in_shape = (-1, 1)

learner_init, learner_apply = stax.serial(dense(1))
out_shape, learner_params = learner_init(input_shape=in_shape, rng=rngkey)

x = transpose(rand(Float32, 1, 1))

opt_init, opt_update, get_params = optimizers.adam(step_size=1e-3)

opt_state = opt_init(learner_params)

p = get_params(opt_state)

function mse_loss(p,x)
y_hat = learner_apply(p,x)
return jnp.mean(optax.l2_loss(y_hat, x))
end

###
### For inference, both np.array and jnp.array work fine
###

learner_apply(p, np.array(x)); # Faster if you check with @btime
learner_apply(p, jnp.array(x)); # Slower

###
### For gradients, only jnp.array works:
###

try
jax.grad(mse_loss)(p, jnp.array(x))
println("jax.numpy.array works with jax.grad but is slow")
println()
catch e
println(e)
end

try
jax.grad(mse_loss)(p, np.array(x))
catch e
println(e)
println()
println("numpy.array does not work with jax.grad")
end
```

The CondaPkg.toml file:
```
[deps]
python = "3.10"

[pip.deps]
jax = ""
jaxlib = ""
optax = ""
```

Hướng dẫn đóng góp

Chưa lập chỉ mục được hướng dẫn đóng góp cho kho mã nguồn này

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.