Add `min` and `max` rev specializations?
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 839
- Forks
- 220
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 14
Description
From this stackoverflow post it looks like we can phrase max(a,b) into
$$
max(a,b) = \frac{a + b + |a-b|}{2}
$$
and similarly for min
$$
min(a,b) = \frac{a + b - |a-b|}{2}
$$
So except for the case when $a=b$ this function is differentiable. For max we have
$$
\begin{align}
\frac{\partial max}{a} = \frac{1}{2} + \frac{sign(a-b)}{2}
\frac{\partial max}{a} = \frac{1}{2} + \frac{sign(a-b)}{2}
\end{align}
$$
so if a>b the gradient is 1 and if a<b the gradient is 1/2. For min we have
$$
\begin{align}
\frac{\partial min}{a} = \frac{1}{2} - \frac{sign(a-b)}{2}
\frac{\partial min}{a} = \frac{1}{2} - \frac{sign(a-b)}{2}
\end{align}
$$
where if $a>b$ the gradient is 0 and if $a<b$ the gradient is 1/2
From this it seems like pytorch does something similar to the above. I think this would be nice to implement. We can also write min(vector) for these as well.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read the linked Stack Exchange and PyTorch discussions first, then inspect the existing min/max APIs and reverse-mode specialization patterns in the library. Clarify the intended scalar and vector behavior, including equal arguments and the proposed gradients; done means the agreed specializations are implemented with tests covering those cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100