EnzymeAD / EnzymeAD/Reactant.jl
Bad sharding (all* operations) for non-divisible outputs
Open
- Dominant language
- Julia
- Stars
- 370
- Forks
- 74
- Avg merge
- 18h 47m
- Merged PRs (30d)
- 30
Description
Example:
```julia
y_ra = Reactant.to_rarray(
rand(Float32, 8, 8, 4); sharding=Sharding.NamedSharding(mesh, (:x, nothing, :y))
)
function wrap(x)
y = x[:, :, 1:1]
return cat(x, y, dims=3)
end
@code_xla wrap(y_ra)
```
Ideally we detect this and pad the outputs to prevent an all-gather here
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.