Streaming response for MS Teams Botframework SDK bot

[gykung]

Use this query to search for the most popular feature requests.

Is your feature request related to a problem? Please describe.

Describe the solution you'd like

We run an internal MS Teams bot using the bot framework SDK, that connects to internal resources, RAG, APIs, to produce responses. To improve user experience, especially for long responses, we would like to have (or simulate) streaming responses (word by word or chunk by chunk).

Describe alternatives you've considered

We've considered breaking the responses into multiple messages, but it is not elegant and not always easy to decide when to split a message.

Additional context
Nil

[stevkan]

@axelsrz - Can you take a look at this?

[octopus-eran]

I was under the impression that such text streaming functionality has already been implemented using the "typing" and "message" activity types and "final", "streaming" and "informative" stream types. It does work with the node.js SDK, however, when I tried to implement it with python I keep getting the same error? Is there a working example for such streaming capabilities?

botbuilder.schema._models_py3.ErrorResponseException: (BadSyntax) Only start streaming and continue streaming types are allowed as a typing activity

@gykung did you find a way to simulate this streaming?

[gykung]

@octopus-eran I did not manage to find a solution, unfortunately.

[wgrcunha]

Same issue here, i've tried:

Activity(
            type=ActivityTypes.typing,
            text="generating...",
            entities=[
                {
                    "type": "streaminfo",
                    "streamType": "informative",
                    "streamSequence": 1,
                }
            ],
        )

Error:

botbuilder.schema._models_py3.ErrorResponseException: (BadSyntax) Only start streaming and continue streaming types are allowed as a typing activity

Anyone have any idea?

[octopus-eran]

@wgrcunha switch to https://github.com/microsoft/teams.py

[wgrcunha]

Thanks @octopus-eran btw i've found the problem, is related to the Entity model that is only accepting the type field, i did this:

class Entity(Model):
    """Metadata object pertaining to an activity.

    :param type: Type of this entity (RFC 3987 IRI)
    :type type: str
    :param stream_type: Stream type
    :type stream_type: str
    :param stream_sequence: Stream sequence
    :type stream_sequence: int
    :param stream_id: Stream ID
    :type stream_id: str
    """

    _attribute_map = {
        "type": {"key": "type", "type": "str"},
        "stream_type": {"key": "streamType", "type": "str"},
        "stream_sequence": {"key": "streamSequence", "type": "int"},
        "stream_id": {"key": "streamId", "type": "str"},
    }

    def __init__(self, *, type: str = None, stream_type: str = None, stream_sequence: int = None, stream_id: str = None, **kwargs) -> None:
        super(Entity, self).__init__(**kwargs)
        self.type = type
        self.stream_type = stream_type
        self.stream_sequence = stream_sequence
        self.stream_id = stream_id

Changed to:

Activity(
            type=ActivityTypes.typing,
            text="generating...",
            entities=[
                Entity(
                    type="streaminfo",
                    stream_type="Informative",
                    stream_sequence=sequence,
                )
            ],
        )

And now it works fine