[Feature] KTransformers Fine-tuning Feature Compatibility & Key Enhancements Support Tracking

[JimmyPeilinLi]

Checklist

• 1. If the issue you raised is not a feature but a question, please raise a discussion at https://github.com/kvcache-ai/ktransformers/discussions. Otherwise, it will be closed.
• 2. To help the community, I will use Chinese/English or attach an Chinese/English translation if using another language. Non-English/Chinese content without translation may be closed.

Motivation

This issue tracks the progress of addressing compatibility issues in the recently released KTransformers fine-tuning feature and supporting key new enhancements.

1. Python version compatibility: It has been found that the fine-tuning feature does not work well with Python 3.10. Users are advised to use Python ≥3.11 for better compatibility. Additionally, for the user of LLaMA-Factory+KTransformers integration，Python 3.13 (latest version) is currently unavailable, as some dependency packages in Llamafactory do not yet support it. fix: remove py310 as guide #1572
2. Llamafile integration issues: KTransformers cannot fine-tune models with llamafile, with errors occurring during KT operator operation. We will debug it soon, but we still recommend AMX for the best solution.
3. AMD CPU adaptation: There is user demand for fine-tuning support on AMD CPUs to expand hardware coverage. Current efforts focus on adapting the heterogeneous scheduling logic to AMD's instruction sets.
4. Qwen-MoE model adaptation: Need to optimize fine-tuning compatibility for Qwen-MoE models, including handling their specific MoE layer structures and inference logic.

Additionally, community members have suggested related enhancements (e.g., VL model support, fine-tuning with reinforcement learning), which will be evaluated after resolving core compatibility issues. Contributions or PRs targeting the above issues are welcome!

CC: @JimmyPeilinLi @Azure-Tang @KMSorSMS @Atream @yangqianrui

Related resources

No response