Livoa LogoLivoa

Tokens: Input words or tokens (the text so far)

[The]
[cat]
[sat]
[on]
Embedding + Positional Encoding: Convert words into numbers and add order info
Transformer Backbone: Looks at all tokens together to understand context

Multi-head Attention

Feed-forward Network

Linear + Softmax Head: Directly guesses next word probabilities from big list

Output Word: Next token chosen based on probabilities
[the]
[the] [cat] [sat] [on] ... (bar chart illustration)

sam

by sam

0
0 uses