EpochLex entry

KV Cache

K-V kash

A key-value cache that stores attention-layer key and value states for previously processed tokens so an autoregressive model can reuse them during later generation steps instead of recomputing them.