vLLM v0.34.0rc2 released, changes to gpu_memory_utilization settings
Key point
The new version of vLLM, v0.34.0rc2, has been released. In most extensible features, setting gpu_memory_utilization is no longer being done.
The new version of vLLM, v0.34.0rc2, has been released.
In this update, setting gpu_memory_utilization is being stopped in most extensible features.
This item is a summary. The full, checkable version is at the original source:
Read the original at vLLM ↗