EpochLex entry
Speculative Decoding
SPEK-yuh-luh-tiv dee-KOH-ding
An inference technique that speeds generation by having a faster helper produce candidate tokens that a larger model verifies in fewer expensive forward passes.
EpochLex entry
SPEK-yuh-luh-tiv dee-KOH-ding
An inference technique that speeds generation by having a faster helper produce candidate tokens that a larger model verifies in fewer expensive forward passes.