EleutherAI / EleutherAI/delphi

Refactor latents and prompt creation

Open
#124 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
275
Forks
61
PR merge metrics
No merged PRs in 30d

Description

The `latents` code and the prompt creation explainer code could be significantly simplified.

Simplifying the code would facilitate prototyping and adding new features.

E.g.: reshaping the input tokens to be of sequence length `window_size` before caching the latents would allow us to remove most or all of the window handling logic in `constructors`.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.