vLLM v0.33.0rc0 released
Key point
vLLM v0.33.0rc0 has been released, which is a high-throughput and memory-efficient inference and serving engine for LLMs.
vLLM v0.33.0rc0 has been released, which is a high-throughput and memory-efficient inference and serving engine for LLMs.
This release includes [Bugfix][MRV2] to skip adaptive verification cost profiling in the CUDA.
This item is a summary. The full, checkable version is at the original source:
Read the original at vLLM ↗