EnzymeAD / EnzymeAD/Reactant.jl

Bad sharding (all* operations) for non-divisible outputs

Open
#1,176 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Julia
Stars
370
Forks
74
Avg merge
18h 47m
Merged PRs (30d)
30

Description

Example:

```julia
y_ra = Reactant.to_rarray(
rand(Float32, 8, 8, 4); sharding=Sharding.NamedSharding(mesh, (:x, nothing, :y))
)

function wrap(x)
y = x[:, :, 1:1]
return cat(x, y, dims=3)
end

@code_xla wrap(y_ra)
```

Ideally we detect this and pad the outputs to prevent an all-gather here

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.