-
Notifications
You must be signed in to change notification settings - Fork 660
[PD Disaggregation] [Refine] Refine splitwise deployment #5151
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Thanks for your contribution! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR refines the splitwise deployment feature as part of PD (Paddle) disaggregation work. The changes split a larger PR (#5027) and optimize interfaces for PD-related modules.
主要改动:
- 重构
splitwise_connector.py中的缓存信息发送逻辑,将send_cache_infos拆分为send_cache_info_to_messager和send_cache_info_to_prefill两个职责更清晰的方法 - 优化
resource_manager_v1.py中的资源管理接口,重命名prerelease_resource为pre_recycle_resource,新增has_resource_for_prefilled_req和add_prefilled_request方法 - 标准化 router 模块的参数处理,引入
RouterArgs数据类和FlexibleArgumentParser
Reviewed Changes
Copilot reviewed 15 out of 15 changed files in this pull request and generated 12 comments.
Show a summary per file
| File | Description |
|---|---|
| fastdeploy/splitwise/splitwise_connector.py | 重构缓存信息发送逻辑,将单一方法拆分为两个职责明确的方法,分别处理发送到 messager 和 prefill 的场景 |
| fastdeploy/engine/sched/resource_manager_v1.py | 重命名资源回收方法,新增预填充请求的资源检查和添加方法,优化 P/D 分离部署的资源管理 |
| fastdeploy/router/router.py | 引入 RouterArgs 数据类标准化路由参数,重命名 start_router 为 launch_router |
| fastdeploy/router/launch.py | 简化启动脚本,使用 RouterArgs 和 FlexibleArgumentParser 替代手动参数解析 |
| fastdeploy/router/init.py | 导出 RouterArgs 和 launch_router 供外部使用 |
| fastdeploy/engine/engine.py | 添加属性存在性检查,避免在退出时访问不存在的属性 |
| fastdeploy/engine/common_engine.py | 更新方法调用以适配 splitwise_connector 的重构,优化预填充请求的处理逻辑 |
| examples/splitwise/utils.sh | 新增通用工具函数:端口检查和健康检查 |
| examples/splitwise/stop.sh | 简化停止脚本,统一使用 pkill 命令 |
| examples/splitwise/start_v1_tp1.sh | 移除重复的 wait_for_health 函数,改用 utils.sh,添加端口冲突检查 |
| examples/splitwise/start_v0_tp1.sh | 同上,提升脚本复用性和健壮性 |
| examples/splitwise/start_mixed.sh | 同上,保持示例脚本的一致性 |
| examples/splitwise/start_v1_tp2.sh | 删除文件,可能合并到其他示例中 |
| examples/splitwise/start_v0_tp2.sh | 删除文件,可能合并到其他示例中 |
| examples/splitwise/README.md | 新增示例文档,说明环境准备、构建安装和运行方式 |
Motivation
将优化PD分离模块的部分代码,从PR(https://github.com/PaddlePaddle/FastDeploy/pull/5027/files)进行拆分。
Modifications
Usage or Command
没有改变
Accuracy Tests
参考单侧和示例
Checklist
[FDConfig],[APIServer],[Engine],[Scheduler],[PD Disaggregation],[Executor],[Graph Optimization],[Speculative Decoding],[RL],[Models],[Quantization],[Loader],[OP],[KVCache],[DataProcessor],[BugFix],[Docs],[CI],[Optimization],[Feature],[Benchmark],[Others],[XPU],[HPU],[GCU],[DCU],[Iluvatar],[Metax]]pre-commitbefore commit.releasebranch, make sure the PR has been submitted to thedevelopbranch, then cherry-pick it to thereleasebranch with the[Cherry-Pick]PR tag.