EpochLex entry
Context Caching
KON-tekst KASH-ing
A technique for reusing computation associated with previously processed context so repeated or shared prompt prefixes do not have to be fully recomputed for every generation.
EpochLex entry
KON-tekst KASH-ing
A technique for reusing computation associated with previously processed context so repeated or shared prompt prefixes do not have to be fully recomputed for every generation.