748B DeepSeek becomes even faster with 1 GPU! Generation is up to approximately 2x
Key point
The hurdle for running a massive MoE model on a handheld GPU has dropped one more step. The LLM inference software "Project Maya" being developed on GitHub released its latest version "v1.1.28" on October 26.
The LLM inference software "Project Maya," being developed on GitHub, released its latest version "v1.1.28" on October 26.
With this update, it is reported that generation speed increases by up to approximately 2x when running the 748B DeepSeek with a single GPU. This further lowers the hurdle for running massive MoE models on a handheld GPU.
This item is a summary. The full, checkable version is at the original source:
Read the original at PC Watch ↗