karpathy / karpathy/micrograd

Sequential MLP implementation

Open
#31 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
17.6k
Forks
2.8k
PR merge metrics
No merged PRs in 30d

Description

Maybe not PR worthy, but I guess one can abstract the MLP implementation even more, making use of the layers instead of number of inputs and outputs yet again, since each individual layer already knows them.

As such, I wrote it as:

```python
class MLP:

def __init__(self, layers):
self.layers = layers

def __call__(self, x):
for l in self.layers:
x = l(x)
return x

def parameters(self):
return [p for layer in self.layers for p in layer.parameters()]
```

by which you can define a network more intuitively, much like PyTorch's Sequential:

```python
n = MLP([Layer(3, 6), Layer(6, 3), Layer(3, 1)])
```

To be even more rigorous, a dimension assertion can be added in the `__init__`:

```python
class MLP:

def __init__(self, layers):
self.layers = layers
for i in range(1, len(layers)):
assert layers[i-1].nout == layers[i].nin
```

for which I would have to store the `nin` & `nout` for the layers in the as well:

```python
class Layer:

def __init__(self, nin, nout):
self.nin = nin
self.nout = nout
self.neurons = [Neuron(nin) for _ in range(nout)]
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the existing Layer, Neuron, and MLP implementations to understand how dimensions, forward calls, and parameters are currently handled. Compare that behavior with the proposed sequential MLP and Layer dimension checks; done means the abstraction works for the shown layer sequence without breaking parameter collection or evaluation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.