I followed the steps as is for using vllm with Dolphin. I installed the plugins as described in the documentation here: . I'll try downgrading vllm and hopefully it will work - but just wanted to call this out. https://github.com/bytedance/Dolphin/tree/master/deployment/vllm.