CodingTrain / CodingTrain/Toy-Neural-Network-JS

Neural Network Feature (Wish) List

Open
#77 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
437
Forks
242
PR merge metrics
No merged PRs in 30d

Description

# Neural Network Feature (Wish) List
## These are the basics:
- [x] Basic 2-layer network with bias
- [x] Activation functions which reuse the activation on the backwards pass (sigmoid, tanh, ReLU)
- [x] MSE cost function
- [x] Adjustable learning rate
- [ ] Additional examples and tests
See #41 and #66 and #76.

## These would be interesting to add:
- [ ] Multiple hidden layers
Per #61.
- [ ] Semi-arbitrary activation functions
Per #70 and #75.
- [ ] Arbitrary cost functions
See [here](https://stats.stackexchange.com/questions/154879/a-list-of-cost-functions-used-in-neural-networks-alongside-applications).
- [ ] Automatically adapting learning rate (Momentum)
Per #65. Also see [here](https://distill.pub/2017/momentum/) via [here](https://ml4a.github.io/ml4a/how_neural_networks_are_trained/).
- [ ] Multiple initial weighting strategies
See [here](https://prateekvjoshi.com/2016/03/29/understanding-xavier-initialization-in-deep-neural-networks/).
- [ ] Convolution layers
See [here](https://towardsdatascience.com/building-a-deep-neural-net-in-google-sheets-49cdaf466da0). Also see [here](http://cs231n.github.io/convolutional-networks/) via [here](http://cs231n.github.io/).
- [ ] Simple RNN
See [here](https://gist.github.com/karpathy/d4dee566867f8291f086) via [here](http://karpathy.github.io/2015/05/21/rnn-effectiveness/).
- [ ] Advanced optimization
See [here](https://towardsdatascience.com/neural-networks-with-optimal-accuracy-and-speed-in-their-predictions-d2cdc3b21b50).

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.