Concepts & capability filters
architecture
Attention mechanism
The attention mechanism allows large language models to weigh the importance of different tokens in a sequence relative to each other when processing input, enabling focus on relevant context regardless of position.
- Category
- architecture
- Difficulty
- Not classified
- Aliases
- None tracked
- Last reviewed
- 2026-07-02
Key facts
- It is core to transformer architectures, powering parallel computation and long-range dependencies.