JuliaPy / JuliaPy/PythonCall.jl

PythonCall w/ Jax: Fast inference w/ numpy but only jax.numpy works for jax.grad

Aberta
#273 8 comentários 0 reações 0 responsáveis Ver no GitHub
Linguagem predominante
Julia
Estrelas
1.1k
Forks
86
Merge médio
1d 22h
PRs com merge (30d)
3

Descrição

I am trying to use Jax in my Julia codebase for something that Zygote cannot do well (meta-learning). Someone recommended PythonCall as a solution to some issues I was having with PyCall.

So far, PythonCall has been great. Things work and it is generally quite quick.

There is one pain point: `jax.grad` should work with `numpy.array` (it does in Python) but it errors with PythonCall:

```
PyException( as an abstract array; it does not have a dtype attribute")>)
````

Instead, `jax.numpy.array` works with `jax.grad` but it is slower.

I assume this is due to the way PythonCall handles non-copying conversions under the hood.

Is there any way to reconcile these two and get `jax.grad` to work with `numpy.array` and the optimizations for non-copying arrays?

I've included an MWE:

```
using PythonCall
using Flux

jax = pyimport("jax")
jnp = pyimport("jax.numpy")
np = pyimport("numpy")
stax = pyimport("jax.example_libraries.stax")
optimizers = pyimport("jax.example_libraries.optimizers")
optax = pyimport("optax")
random = pyimport("jax.random")

rngkey = random.PRNGKey(123)

dense = stax.Dense
Relu = stax.Relu

in_shape = (-1, 1)

learner_init, learner_apply = stax.serial(dense(1))
out_shape, learner_params = learner_init(input_shape=in_shape, rng=rngkey)

x = transpose(rand(Float32, 1, 1))

opt_init, opt_update, get_params = optimizers.adam(step_size=1e-3)

opt_state = opt_init(learner_params)

p = get_params(opt_state)

function mse_loss(p,x)
y_hat = learner_apply(p,x)
return jnp.mean(optax.l2_loss(y_hat, x))
end

###
### For inference, both np.array and jnp.array work fine
###

learner_apply(p, np.array(x)); # Faster if you check with @btime
learner_apply(p, jnp.array(x)); # Slower

###
### For gradients, only jnp.array works:
###

try
jax.grad(mse_loss)(p, jnp.array(x))
println("jax.numpy.array works with jax.grad but is slow")
println()
catch e
println(e)
end

try
jax.grad(mse_loss)(p, np.array(x))
catch e
println(e)
println()
println("numpy.array does not work with jax.grad")
end
```

The CondaPkg.toml file:
```
[deps]
python = "3.10"

[pip.deps]
jax = ""
jaxlib = ""
optax = ""
```

Guia de contribuição

Nenhum guia de contribuição indexado para este repositório

Avaliação

Esta issue ainda não foi avaliada.

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.