Attention Mechanism

A component of neural networks that allows a model to selectively focus on the most relevant parts of its input when producing each part of its output, rather than treating all input equally. Attention was the key innovation behind the transformer architecture, introduced in the 2017 paper "Attention Is All You Need", and is now central to virtually all state-of-the-art models in language, vision, and beyond.
See also: transformer, self-attention.