PAIR-code / PAIR-code/tiny-transformers

Provide a visualization for the norms of different parts of a transformer model

Open
#25 0 comments 0 reactions 1 assignee View on GitHub

@AAlberich8 is already working on this.

Since Aug 29, 2024.

enhancement project:TTP
Dominant language
Jupyter Notebook
Stars
24
Forks
6
PR merge metrics
No merged PRs in 30d

Description

A common debugging method is to monitor the norm of some part of a ML model's weight and activation space.

Goal:

  • Create a visualization component for norms of a set of vectors and/or weights. Allow the norms to be tracked over time, like we do with loss/accuracy.
  • The code and implementation should be reusable and clean.

Code pointers:

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.