EpochLex entry

FlashAttention

FLASH uh-TEN-shun

An IO-aware exact attention algorithm that reorganizes attention computation into blocks to reduce memory reads and writes between GPU memory levels, improving speed and memory efficiency without approximating the attention result.