karpathy / karpathy/micrograd

Another way of gradient backward backpropagation

Open
#81 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
17.6k
Forks
2.8k
PR merge metrics
No merged PRs in 30d

Description

Hi @karpathy, I followed your code and created my own implementation. I also made some modifications that I believe enhance the code's clarity and understanding.

When forward(add operation as example):
```
out.backwards.extend([(self, 1), (other, 1)])
```
When backward:
```
def backward(self):

for node, partial_derivative in self.backwards:
node.grad += self.grad * partial_derivative
node.backward()
if not isinstance(node, Parameter):
node.zero_grad()
```
I tested it, and it appears to be working well. On the Iris dataset, I achieved an accuracy of 93%. I believe it could reach 100% if I use a more effective loss function.
My code: https://github.com/ickma/picograd

Contributor guide

No contributing guide indexed for this repository

Research direction

Read the existing autograd implementation and compare its backward traversal with the linked picograd example. Run the repository's current tests or examples first, then determine whether this alternative is an intended change; completion would require an agreed scope and regression coverage for the proposed behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.