EnzymeAD / EnzymeAD/Enzyme-JAX

reduce of broadcast

Open
#1,788 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
MLIR
Stars
131
Forks
53
Avg merge
1d 10h
Merged PRs (30d)
193

Description

from neural gcm

```
%2003 = stablehlo.broadcast_in_dim %2002, dims = [1, 2, 3] : (tensor<128x2x1xf32>) -> tensor<128x128x2x1xf32>
%2004 = stablehlo.reshape %2003 : (tensor<128x128x2x1xf32>) -> tensor<128x256x1xf32>
%2005 = stablehlo.reduce(%2004 init: %cst_304) applies stablehlo.add across dimensions = [1] : (tensor<128x256x1xf32>, tensor) -> tensor<128x1xf32>
```

we should do the reduce first and then broadcast. this should be done in 2 parts:

1. fuse reshape reduce

```
%2004 = stablehlo.reshape %2003 : (tensor<128x128x2x1xf32>) -> tensor<128x256x1xf32>
%2005 = stablehlo.reduce(%2004 init: %cst_304) applies stablehlo.add across dimensions = [1] : (tensor<128x256x1xf32>, tensor) -> tensor<128x1xf32>
```

should become `reduce(%2003) dims = [1, 2]`

2. reduce(bcast) where all the reduction dims are part of the original tensor, we should reduce and then bcast

Contributor guide

No contributing guide indexed for this repository

Research direction

Start from the StableHLO broadcast_in_dim, reshape, and reduce sequence in the issue and locate the rewrite entry point that handles these operations. Implement the two requested transformations: fuse reshape and reduce dimensions, then reduce before broadcasting when reduction dimensions come from the original tensor. Done means the resulting form performs the reduction before the broadcast.

Written by the indexing model from the issue text.

Assessment

Domain
compilers
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.