scikit-learn / scikit-learn/scikit-learn
Bug - wrong activations and delta length in BaseMultilayerPerceptron ?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 67.3k
- Forks
- 27.4k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 58
Description
In multilayer_perceptron.py at line 623 we have
activations = [X]
activations.extend(np.empty((batch_size, n_fan_out)) for n_fan_out in layer_units[1:])
deltas = [np.empty_like(a_layer) for a_layer in activations]
coef_grads = [ np.empty((n_fan_in_, n_fan_out_))
for n_fan_in_, n_fan_out_ in zip(layer_units[:-1], layer_units[1:]) ]
and at line 268 we have defined
self.n_layers_ = len(layer_units)
as such activations and deltas should has the same the layer_units.
However, in the doc strings from other methods of this class, it is claimed at both activations and deltas has the length of self.n_layers - 1, and it seems to me the logic within those methods also follows this assumption. For exampe at line 236, we have
last = self.n_layers_ - 2
deltas[last] = activations[-1] - y
This does not look right to me. If the network has only three layer (input, hidden, and output) the last would be refer to the the hidden layer. But if my understanding is right the deltas formula is meant for the output layer.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in sklearn/neural_network/multilayer_perceptron.py around the cited activation, delta, and n_layers_ definitions. Trace the training methods and their docstrings to determine whether the output-layer delta indexing and the lengths of activations and deltas are consistent. Done means the implementation and documentation agree and the output-layer calculation is correctly addressed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100