EpochLex entry

Prompt Caching

prompt KASH-ing

Reusing previously processed prompt or context tokens so repeated model requests can be handled with lower latency, lower cost, or both.