Detect and remove dead channels

In QNNs we sometimes get "dead neurons" (for FC layers) or "dead channels" (for conv layers). Sometimes the output from the dot product is always guaranteed to fall within a certain range that corresponds to the same threshold output.

As an example, consider a (MatMul, MultiThreshold) sequence, where the matmul does a dot product between 32 binary (0/1) weights and 32 binary (0/1) activations. The output of the dot product can range between 0 (if everything is 0) to 32 (if everything is 1). If the threshold value in the following MultiThreshold op is 33, the post-activation output for that channel/neuron is always guaranteed to be 0.

To reduce the amount of compute/memory resources, we can take advantage of this by pruning away that neuron/channel completely. This is easy if the constant output from thresholding is 0 since removing the dead neuron/channel is enough and no further action is required. If the constant output is 1 or greater, this needs to be compensated by either adding a bias after the thresholding that reflects that constant output (which may be streamlined away), or by generating a fixed output for that neuron/channel from thresholding without doing any computation. The latter option requires hardware specialization.

Trace the MatMul/MultiThreshold sequence in the compiler and inspect how channel ranges and thresholds are represented. Define handling for channels whose threshold output is constant, including nonzero outputs, then verify that pruning preserves graph behavior and reduces compute or memory.