Cross-Attention

A form of attention mechanism where one sequence attends to a different sequence, rather than to itself. In encoder-decoder models, cross-attention allows the decoder to focus on relevant parts of the encoder's output when generating each output token. It is the mechanism that connects the two halves of a transformer-based translation or summarization model.
See also: attention mechanism, transformer, self-attention.