explosion / explosion/thinc

Implementing Leaky Relu, parametric and other forms of Relu

Open
#322 4 comments 0 reactions 0 assignees View on GitHub
enhancement feat / layers
Dominant language
Python
Stars
2.9k
Forks
295
PR merge metrics
No merged PRs in 30d

Description

I am working on an implementation of LeakyRelu. I would like some input on how to go about with the implementation of the same. There are 2 options.
1. A separate layer named LeakyRelu, ParamRelu etc for each of the relu variations.
2. One single Relu layer that instead takes optional params and implements them. (This would greatly reduce duplicated code, but also does reduce the visibility of the layer to end users unless they spend some time on the documentation).

Keras and Pytorch seem to have separate layers for each of the Relu variations, but I am inclined more towards a single relu with the right parameters. What would you guys suggest?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.