pymc-devs / pymc-devs/pytensor

Add CAReduce(DimShuffle(x)) -> CAReduce(x) rewrite when DimShuffle is a no-op for the reduction

Open
#2,132 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

graph rewriting
Dominant language
Python
Stars
644
Forks
208
Avg merge
2d 14h
Merged PRs (30d)
16

Description

Description

When a DimShuffle/ExpandDims operation adds dimensions that are immediately reduced away by a CAReduce, the DimShuffle is unnecessary and should be removed.

Motivating example

After the local_careduce_join rewrite (#2130), Sum{axes=None}(Join(0, ExpandDims(a), ExpandDims(b))) becomes:

Add
 ├─ Sum{axes=None}(ExpandDims{axis=0}(a))
 ├─ Sum{axes=None}(ExpandDims{axis=0}(b))
 └─ Sum{axes=None}(ExpandDims{axis=0}(c))

The ExpandDims{axis=0} adds a dimension at position 0, but Sum{axes=None} reduces all dimensions including that one. The same applies when the reduction axis is a subset: e.g., Sum{axis=(1,)}(ExpandDims{axis=0}(x)) could drop the ExpandDims since axis=0 is not in the reduction — the ExpandDims is still needed for broadcasting but could be kept. The easy case is axis=None.

Proposed rewrite

@node_rewriter([CAReduce])
def local_careduce_dimshuffle(fgraph, node):
    """CAReduce(DimShuffle(x), axis=ax) -> CAReduce(x)
    
    When DimShuffle only adds dimensions (no transpose/reshape),
    and those dimensions are all reduced away, it can be removed.
    """
    [inp] = node.inputs
    if not isinstance(inp.owner_op, (DimShuffle, ExpandDims)):
        return None
    # Check if the DimShuffle is "expanding" only (no transpose/reshape)
    # and the expansion axes are all in the reduction set
    ...

Related

  • PR #2130 (local_careduce_join) exposes this pattern
  • Issue #2131 (Numba backend bug with Sum(ExpandDims(...)))

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the proposed local_careduce_dimshuffle node rewriter and inspect the CAReduce, DimShuffle, and ExpandDims behavior described in the issue. Verify the axis=None motivating example and the reduction-axis conditions; done means the redundant expansion is removed only when its added dimensions are reduced away, with the existing examples still producing the intended graph.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, performance
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.