An Alternative Approach: Recursive Backpropagation Without Topological Sorting
- Dominant language
- Jupyter Notebook
- Stars
- 17.6k
- Forks
- 2.8k
- PR merge metrics
- No merged PRs in 30d
Description
Very intuitive and simple, @karpathy! I was coding in parallel while watching your video and took a slightly different approach.
- For computing gradients of `+` and `*`, I used the fundamental derivative formula:
$$ L = \lim_{h \to 0} \frac{f(a+h) - f(a)}{h} $$
- Instead of using **topological sorting** for backpropagation, I implemented a **recursive approach**, where each parent node checks its child nodes and calculates gradients accordingly. While this method is probably less efficient—as it can recompute gradients for child nodes multiple times when gradients flow from multiple paths—it still serves as a valid alternative that produces the same results.
Link to my repo:
https://github.com/wahabaftab/micrograd/
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.