EpochLex entry

Model Offloading

MOD-ul OFF-loh-ding

A memory-management technique that moves some model parameters, buffers, or other state from a faster device such as a GPU to slower memory such as CPU RAM or disk so models larger than the fastest device memory can still run.