vLLM v0.34.0rc2 released, changes to gpu_memory_utilization settings

Local & Open SourcePublished: Source: vLLM

#Ollama

Key point

The new version of vLLM, v0.34.0rc2, has been released. In most extensible features, setting gpu_memory_utilization is no longer being done.

The new version of vLLM, v0.34.0rc2, has been released.

In this update, setting gpu_memory_utilization is being stopped in most extensible features.

This item is a summary. The full, checkable version is at the original source:

Read the original at vLLM ↗