Another way of gradient backward backpropagation
- Dominant language
- Jupyter Notebook
- Stars
- 17.6k
- Forks
- 2.8k
- PR merge metrics
- No merged PRs in 30d
Description
Hi @karpathy, I followed your code and created my own implementation. I also made some modifications that I believe enhance the code's clarity and understanding.
When forward(add operation as example):
```
out.backwards.extend([(self, 1), (other, 1)])
```
When backward:
```
def backward(self):
for node, partial_derivative in self.backwards:
node.grad += self.grad * partial_derivative
node.backward()
if not isinstance(node, Parameter):
node.zero_grad()
```
I tested it, and it appears to be working well. On the Iris dataset, I achieved an accuracy of 93%. I believe it could reach 100% if I use a more effective loss function.
My code: https://github.com/ickma/picograd
Contributor guide
No contributing guide indexed for this repository
Research direction
Read the existing autograd implementation and compare its backward traversal with the linked picograd example. Run the repository's current tests or examples first, then determine whether this alternative is an intended change; completion would require an agreed scope and regression coverage for the proposed behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100