HackerAttention in transformers, step-by-step | Deep Learning Chapter 6
3Blue1BrownYouTube ↗
Further reading 1
Caption notes
An attention head compares queries with keys to decide which positions should interact. What changes the target representation is a weighted combination of value vectors. The adjective example illustrates the distinction.
Creator’s caption track ↗Original passages · 2
To measure how well each key matches each query, you compute a dot product between each possible key-query pair.For each column in this diagram, you multiply each of the value vectors by the corresponding weight in that column.