⚙️ Under the hood
Attention
The mechanism by which a model weighs, for each token, which other tokens in the context are relevant right now.
💭 Imagine …
In “The cat chased the mouse because it was hungry”, the model asks: who is “it”? Attention puts weight on “cat”.
🔎 Example
When translating, the model looks at the matching parts of the source text for each word.
🎯 Why it matters
Explains why models grasp long-range connections and why compute grows sharply with context length.
🎓 Prefer step by step?
The AI Literacy course: 6 short modules with quizzes, a solid foundation in about 60 minutes.