alpaca-lib / alpaca-lib/alpaca

Deep Neural Network initial implementation

未关闭
#50 1 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
Backlog
主要语言
没有语言数据
星标
0
派生
0
PR 合并指标
30 天内没有已合并 PR

描述

As mentioned in #7, we would like to have a basic implementation of a Deep Neural Network. The design should be consistent with other major libraries and should have enough flexibility to add other types of layers (for the future when we add things like RNN or CNN).

Here is an example with Keras
```
model = Sequential()
model.add(Dense(512, input_dim=1000))
model.add(Dense(2, activation='softmax'))
model.compile(loss = 'mean_squared_error', optimizer='adam', metrics=['accuracy'])
model.fit(x_train, y_train, epochs=200, batch_size=100, verbose=0)
```

And a complex one in [TFLearn](https://github.com/tflearn/tflearn/blob/master/examples/images/dnn.py).

We need to define our interface as well.

So, what do we need?
- A model structure (TBD). Keras creates the model from the beginning and adds new layers to it. TFLearn connects the layers one by one, creating a net, and then runs tflearn.regression to the last layer. What do you think works better? How would you like you like to see users to create neural networks?
- Layer structure - this should be composed of nodes.
- Implement a Node class. Nodes should have a value and lists for inbound and outbound nodes.
- Implement an Input class (type of Node). This shouldn't have any inbound nodes.
- Implement a Linear class (type of Node as well).
- This one should take as input 3 things: values (matrix of values - to explain later), weights (list of weights), and a bias (just a number). Calculating the value of the linear node is done with [this formula](https://d17h27t6h515a5.cloudfront.net/topher/2017/February/5892a66c_neuron-output/neuron-output.png), where wi and xi correspond to the weights and values, and b is the bias. Numpy has some functions that should make this easy (np.dot).
- Nodes should have a forward and backward method (I'll explain this later).
- Finally, we need to define forward and backward passes. In the forward pass, a value goes through the neural network. This is easy, as each node will have a forward method, so we will need to go through each node (To do this, we need to do a topological sort, which is the complex part. This will allow us to go through the graph in an ordered way). The backpropagation step is more complicated and requires some calculus, but, for now, just focus in implementing the forward pass.

@EdgarGar and @Biller17 what do you think?

贡献指南

这个仓库没有索引到贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。