-
Notifications
You must be signed in to change notification settings - Fork 595
[CI] Integrate mooncake to vllm-ascend base image #4062
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Signed-off-by: wangli <wangli858794774@gmail.com>
Signed-off-by: wangli <wangli858794774@gmail.com>
|
👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:
If CI fails, you can run linting and testing checks locally according Contributing and Testing. |
Signed-off-by: wangli <wangli858794774@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request integrates Mooncake into the vllm-ascend base images by updating several Dockerfiles and adding a new installer script. While the goal is clear, the implementation introduces several inefficiencies in the Docker build process and a few critical bugs in the new installer script. My review focuses on optimizing the Dockerfiles for smaller image size and faster builds by consolidating package installation steps, and on fixing critical issues in the installer script related to incorrect directory handling, flawed error checking, and ineffective environment variable management.
Signed-off-by: wangli <wangli858794774@gmail.com>
oc) Signed-off-by: wangli <wangli858794774@gmail.com>
86fb954 to
4725969
Compare
734110c to
eed6bf4
Compare
### What this PR does / why we need it? This patch aims to integrate the mooncake [v0.3.7.2.post2](https://github.com/kvcache-ai/Mooncake/releases/tag/v0.3.7.post2) to vllm-ascend images - vLLM version: v0.11.0 - vLLM main: vllm-project/vllm@83f478b --------- Signed-off-by: wangli <wangli858794774@gmail.com> Signed-off-by: luolun <luolun1995@cmbchina.com>
### What this PR does / why we need it? This patch aims to integrate the mooncake [v0.3.7.2.post2](https://github.com/kvcache-ai/Mooncake/releases/tag/v0.3.7.post2) to vllm-ascend images - vLLM version: v0.11.0 - vLLM main: vllm-project/vllm@83f478b --------- Signed-off-by: wangli <wangli858794774@gmail.com> Signed-off-by: hwhaokun <haokun0405@163.com>
What this PR does / why we need it?
This patch aims to integrate the mooncake v0.3.7.2.post2 to vllm-ascend images
Does this PR introduce any user-facing change?
How was this patch tested?
Test is locally:
docker build -t vllm-ascend-with-mooncake -f Dockerfile.a3 --network host .Start the container after build is complete:
IMAGE=vllm-ascend-with-mooncake docker run --rm \ --name test_nightly \ --device /dev/davinci2 \ --device /dev/davinci_manager \ --device /dev/devmm_svm \ --device /dev/hisi_hdc \ -v /usr/local/dcmi:/usr/local/dcmi \ -v /usr/local/Ascend/driver/tools/hccn_tool:/usr/local/Ascend/driver/tools/hccn_tool \ -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \ -v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \ -v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \ -v /etc/ascend_install.info:/etc/ascend_install.info \ -v /root/.cache:/root/.cache \ -it $IMAGE bashtest mooncake: