Do we need src/cudnn/activations.jl?

I noticed there was a thread on Discourse recently where this came up as the culprit: https://github.com/FluxML/NNlibCUDA.jl/blob/master/src/cudnn/activations.jl. Are these overloads still useful now (for performance) or can they be nixed?

Start with src/cudnn/activations.jl in NNlibCUDA.jl and review the linked Discourse thread for the reported concern. Check whether these overloads are still used and whether they provide a performance benefit. Done means documenting a supported conclusion about retaining or removing them.

Written by the indexing model from the issue text.