Handle Anthropic pause_turn finish reason by resubmitting request with response so far
#3661
Conversation
DouweM
changed the title
pause_turn finish reason by resubmitting request with response so far
| 'stop_sequence': 'stop', | ||
| 'tool_use': 'tool_call', | ||
| 'pause_turn': 'stop', | ||
| 'pause_turn': 'stop', # TODO: should this be a different finish reason? |
There was a problem hiding this comment.
I think we can remove it here because we never end up returning a ModelResponse from pause_turn, right? As we always resubmit the request
| anthropic_messages.append( | ||
| { | ||
| 'role': 'assistant', | ||
| 'content': response.content, |
There was a problem hiding this comment.
Similar to #3637, should we also send back response.container.id so that it can be reused?
| ) | ||
| continue | ||
|
|
||
| return response |
There was a problem hiding this comment.
We're currently losing the initial response.usage. We have to make sure that this method (or really the request method) has the combined usage of both responses on it
| ) | ||
|
|
||
| # Handle pause_turn for non-streaming | ||
| assert isinstance(response, BetaMessage) |
There was a problem hiding this comment.
This currently breaks streaming entirely. We have to make this work for that scenario as well.
| # Assuming `BetaCodeExecutionToolResultBlock` covers it or we need to add a check. | ||
| # Let's assume for now `BetaCodeExecutionToolResultBlock` is sufficient or we'll see. | ||
| # But wait, `bash_code_execution_tool_result` implies a specific type. | ||
| # Let's check if we can import it. |
There was a problem hiding this comment.
This comment doesn't seem necessary
| args=cast(dict[str, Any], item.input) or None, | ||
| tool_call_id=item.id, | ||
| ) | ||
| elif item.name == 'bash_code_execution': |
There was a problem hiding this comment.
Why do we need this now if we didn't need it before?
| pytest.mark.anyio, | ||
| ] | ||
|
|
||
| async def test_pause_turn_retry_loop(allow_model_requests: None): |
There was a problem hiding this comment.
Please add this to the existing test_anthropic.py
| [BetaTextBlock(text='paused', type='text')], | ||
| usage=BetaUsage(input_tokens=10, output_tokens=5), | ||
| ) | ||
| c1.stop_reason = 'pause_turn' # type: ignore |
There was a problem hiding this comment.
Instead of faking it, can we record a real interaction that has a pause_turn response? Note that in #2600 (comment) I mentioned that I reproduced the issue. We should reproduce it in a test, to make sure that bug is now fixed.
2 participants
stop_reasonpause_turnis not handled correctly, resulting in errors with long-running built-in tools #2600