karpathy / karpathy/minGPT

The definitions of `B, T, C `

Open
#39 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
24.9k
Forks
3.3k
PR merge metrics
No merged PRs in 30d

Description

https://github.com/karpathy/minGPT/blob/4050db60409b5bbaaa3302cee1e49847fc145c65/mingpt/model.py#L62

and referred from http://jalammar.github.io/illustrated-gpt2/

I am remain confused about the definitions of `B, T, C = x.size()`.

vocabulary length, batch_size, and tokenizer size, etc?

Thanks.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with mingpt/model.py at line 62 and compare the surrounding code with the linked Illustrated GPT-2 article. Determine what clarification the project should provide for the three dimensions, then document the definitions in the appropriate location and verify that the wording matches how x.size() is used.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.