[Feature]: Improve the chat method for offline interface to make it closer to the functionality of online service #29969
Description
🚀 The feature, motivation and pitch
Improve the LLM.chat method for offline interface to make it closer to the functionality of online service, enchance LLM.chat and LLM.generate interfaces to support streaming output, and support message filtering with reasoning_paperer and tool_caperer. This way, libraries like Outline can better support streaming output for VLLM offline interface as well
Alternatives
Inherit LLM classes to achieve simple functionality
Additional context
No response
Before submitting a new issue...
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.