[Bug]: LoRA not working in v0.11.0rc0

[chenfengt]

Your current environment

The output of `python collect_env.py`

Your output of above commands here

🐛 Describe the bug

While using the LoRA feature in vllm-ascend v0.11.0rc0, I noticed that the responses generated by the LoRA model and the base model are identical. This does not happen in the standard vllm. Could there be an issue here? Below is my test startup script:
vllm serve /llm/model/Qwen1.5-4B-Chat --max-lora-rank 64 --max-loras 1 --max-cpu-loras 100 --enable-lora --lora-modules test-lora=/llm/lora_models/Qwen1.5-4B-Chat_lora_huanhuan

[AlphaINF]

same issue!
we still found that we set the lora adapter and set the lora name, it doesn't work, still output the same output as the base model.

[wangxiyuan]

this is known issue with v0.11.0rc0.We have point it in release note. And we have fixed it on main branch now.

[AlphaINF]

this is known issue with v0.11.0rc0.We have point it in release note. And we have fixed it on main branch now.

I using the following command to update a new docker in 2025/10/24T22.49 UTC+8:

docker pull quay.io/ascend/vllm-ascend:main
sha256:aa97775c2452c6edd6029240a70f5d4dee1edaa7db37dcf6e9c75aaa3e05b576
and start the docker image.

However it still don't work, I using the following commands to start the docker:

docker run --rm   --name lbot-14b-1   --shm-size 16g   --device /dev/davinci1   --device /dev/davinci_manager   --device /dev/devmm_svm   --device /dev/hisi_hdc   -v /usr/local/dcmi:/usr/local/dcmi   -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi   -v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/   -v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info   -v /etc/ascend_install.info:/etc/ascend_install.info   -v /root/.cache:/root/.cache   -v /data/disk2/alphainf/models/Qwen/Qwen3-14B:/mnt/data/models/Qwen3-14B   -v /data/disk2/alphainf/lora:/mnt/data/auto-train/volumes/save   -e VLLM_ALLOW_RUNTIME_LORA_UPDATING=True   -e VLLM_LORA_RESOLVER_CACHE_DIR=/mnt/data/auto-train/volumes/save   -p 6215:6215   -e OMP_NUM_THREADS=4   -it quay.io/ascend/vllm-ascend:main bash

after i start the docker, the command I used to start vllm as follow:

python3 -m vllm.entrypoints.openai.api_server \
        --model=/mnt/data/models/Qwen3-14B --served-model-name lbot-14b --host 0.0.0.0 --port 6215 --gpu-memory-utilization 0.8  --max-num-seqs 200 --max-model-len 32768 --enable-prefix-caching --reasoning-parser qwen3 --enable-lora  --max-lora-rank 64 --lora-modules '{"name": "wxey", "path": "/mnt/data/auto-train/volumes/save/qwen14b_wxey", "base_model_name": "lbot-14b"}'

and the interact command like this:

curl http://localhost:6215/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
        "model": "wxey",
        "messages": [
            {"role": "user", "content": "你是谁"}
        ]
    }'

If vllm adapt lora correctly, the output must output the model's name which have been modified.

[wangxiyuan]

@paulyu12 can you take a look? Thanks

[paulyu12]

@chenfengt @AlphaINF Please add "--dtype half" to the launch options and retry. We recently found the issue about supporting default dtype bfloat16, and we are working on it now.

[AlphaINF]

--dtype half

I test on the main image, it works!
Thank you !

[AlphaINF]

Your current environment

The output of python collect_env.py

🐛 Describe the bug

While using the LoRA feature in vllm-ascend v0.11.0rc0, I noticed that the responses generated by the LoRA model and the base model are identical. This does not happen in the standard vllm. Could there be an issue here? Below is my test startup script: vllm serve /llm/model/Qwen1.5-4B-Chat --max-lora-rank 64 --max-loras 1 --max-cpu-loras 100 --enable-lora --lora-modules test-lora=/llm/lora_models/Qwen1.5-4B-Chat_lora_huanhuan

I think the issue can be closed.

[chenfengt]

@paulyu12 @AlphaINF @wangxiyuan Thanks for your reply. I tested with the --dtype , and LoRA is working now. The issue is resolved.

[paulyu12]

@chenfengt @AlphaINF After PR #4046 merged, the bfloat16 dtype will be supported, too. FYI.