multi-head attention

Last updated: March 2026 A transformer is a neural network architecture introduced in the 2017 paper “Attention Is All You Need” that processes entire sequences in parallel using a mechanism called self-attention. Instead of reading tokens one by one like earlier recurrent models, transformers compute relationships between all tokens simultaneously — enabling faster training and … Read more