Encode broadcasting explicitly, not implicitly from "volatile" static shape of inputs

Latest proposal

Make the broadcasting behavior, not a function of the static type of the Variables but of the Ops that do the implicit broadcasting. So Elemwise may have a broadcast_pattern with the indexes of the inputs that will be broadcasted along each dimension. That way if a graph replacement or a rewrite provides something with more defined static shape (1,) instead of the original (None,) the Op will not change it's behavior, and we no longer need this distinction at the type level.

The first time an Op is created, we would read it from the input static types, so everything works with backwards compatibility. This means however, that there would no longer be a single instance of non-unary Elemwises like pt.add, they would need to be created by the helper functions (or like we did for RVs with the dtype argument, with the call method).

This makes sense to me as the same core operation with and without broadcasting are semantically quite different

Old proposal

Less in favor because broacasting is not really a property of the input, but the Op.

Allow TensorType(shape=(1,), broadcastable=(False,))

This requires re-introducing the broadcastable flags as independent from shape. It seems needed to:

1. Not force static shape to be unknown
2. Not change the meaning of the graph accidentally due to shape inference / rewrites

Affected Ops (anything that performs broadcasting of existing dims):

• Elemwise
• Alloc
• GEMM Ops
• Unbroadcast

Will require re-introducing Rebroadcast which could toggle broadcastable flags directly independently from static shape gains from SpecifyShape. Probably better named SpecifyBroadcastable.

Probably Elemwise outputs will have to be unbroadcastable as long as at least on input is also unbroadcastable along the same dimension.

x = pt.vector(shape=(1,), broadcastable=(False,))
y = x + x
assert y.type.broadcastable == (False,)

Start by reading the TensorType and Elemwise behavior described in the proposal, then trace how Alloc, GEMM Ops, and Unbroadcast handle broadcasting. The work is done when broadcasting behavior is explicit at the operation level, remains stable through graph rewrites, and preserves the stated backward-compatibility expectations.