vllm.benchmarks

Modules:

Name	Description
datasets	This module defines a framework for sampling benchmark requests from various
endpoint_request_func	The request function for API endpoints.
latency	Benchmark the latency of processing a single batch of requests.
serve	Benchmark online serving throughput.
throughput	Benchmark offline inference throughput.
utils