EpochLex entry

Throughput

THROO-put

The amount of inference work an AI serving system completes over a unit of time, such as requests, samples, or tokens processed per second.