karpathy / karpathy/micrograd

microjax: like micrograd but following JAX's functional style

Open
#101 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
17.6k
Forks
2.8k
PR merge metrics
No merged PRs in 30d

Description

[microjax](https://github.com/chuan97/microjax)

Like micrograd, it implements backpropagation (reverse-mode autodiff) over a dynamically built DAG of scalar values and a small neural networks library on top of it.

Unlike micrograd, which is implemented following PyTorch's OOP API, microjax replicates JAX's functional API. In particular, it exposes the transformation `microjax.engine.grad`. If you have a Python function `f` that evaluates the mathematical function $f$, then `grad(f)` is a Python function that evaluates the mathematical function $\nabla f$. That means that `grad(f)(x1, ..., xn)` represents the value $\nabla f(x_1, \ldots, x_n)$.

In combination with micrograd, microjax could be useful to illustrate the differences between the OOP and functional paradigms. The functional paradigm is characterized by the use of pure functions acting on immutable state, and higher order functions (transformations) that act on pure functions to return new pure functions. These are all apparent in the implementation of microjax, e.g. `f` -> `grad(f)`.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.