numpy / numpy/numpy

ENH: Add a "broadcast" option to numpy.concatenate

Open
#28,549 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
32.8k
Forks
12.8k
Avg merge
1d 7h
Merged PRs (30d)
197

Description

Proposed new feature or change:

Mailing list post

Often, I've wanted to concatenate arrays with different ndims along a particular axis, broadcasting the other axes as needed. Others have sought this functionality as well:

However, numpy.concatenate raises an error if the ndims don't match. For example:

import numpy as np

a = np.full([2], 0.1)
b = np.full([3, 2], 0.2)
c = np.full([5, 3, 2], 0.3)

arrays = [a, b, c]
axis = -1

try:
    np.concatenate(arrays, axis)
except ValueError as e:
    print(repr(e))
ValueError('all the input arrays must have same number of dimensions, but the array at index 0 has 1 dimension(s) and the array at index 1 has 2 dimension(s)')

It would be convenient to add to numpy.concatenate an optional boolean argument called broadcast that broadcasts the input arrays along the axes that are not the concatenation axis, before concatenating them. Its default value can be False, which is the current behavior.

Below is an example implementation:

def tuple_replace(tupl, index, item):
    return tupl[:index] + (item,) + tupl[index:][1:]

def broadcast_concat(arrays, axis):
    shape = np.broadcast_shapes(*(tuple_replace(a.shape, axis, 0) for a in arrays))
    bcast_arrays = [
        np.broadcast_to(a, tuple_replace(shape, axis, a.shape[axis])) for a in arrays
    ]
    return np.concatenate(bcast_arrays, axis)

output = broadcast_concat(arrays, axis)
assert output.shape[axis] == sum(a.shape[axis] for a in arrays)

If desired, I can submit a PR for this.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the numpy.concatenate API reference and the linked mailing-list discussion, then compare the requested behavior with the current default behavior. Define the broadcast option's axis and shape semantics, including error cases, before implementation. Done means the optional behavior is specified while the default remains unchanged, with corresponding tests and documentation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.