{"version":"1.0","type":"rich","provider_name":"gaks.ai AI Glossary","provider_url":"https://gaks.ai/glossary","title":"Self-Attention — AI Glossary","author_name":"Glenn Katrud Solheim","author_url":"https://gaks.ai","width":600,"height":200,"html":"

AI Glossary — gaks.ai

Self-Attention

A mechanism in which each element of a sequence attends to every other element in the same sequence, allowing the model to capture relationships and dependencies regardless of distance. Self-attention is the defining operation of the transformer architecture. It's what enables a language model to understand how any word in a sentence relates to any other word across the full context window simultaneously, rather than processing text in a fixed local window. See also: attention mechanism, transformer.

Source: gaks.ai/glossary/self-attention →
"}