huggingface / huggingface/course

Chapter 1 "how transformers work"

Open
#953 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
MDX
Stars
4.2k
Forks
1.4k
Avg merge
13m
Merged PRs (30d)
1

Description

I think there is a typo in this section: https://huggingface.co/learn/llm-course/chapter1/4?fw=pt#the-original-architecture

"Note that the first attention layer in a decoder block pays attention to all (past) **inputs** to the decoder, but the second attention layer uses the output of the encoder. It can thus access the whole input sentence to best predict the current word. This is very useful as different languages can have grammatical rules that put the words in different orders, or some context provided later in the sentence may be helpful to determine the best translation of a given word."

The word INPUT should actually be OUTPUT for the first attention layer correct?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.