-
Notifications
You must be signed in to change notification settings - Fork 2.5k
Fix ContextWindowExceededError after 3 retries in react loop #9110
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
When _call_with_potential_trajectory_truncation exhausts all retry attempts, it now raises a clear ValueError instead of returning None. This prevents the AttributeError: 'NoneType' object has no attribute 'next_thought' that occurred when accessing properties on the None return value. The ValueError is caught in forward/aforward and causes the loop to break gracefully, allowing the extract phase to proceed with whatever trajectory was collected. Added tests for both sync and async versions of this scenario. Co-authored-by: TomeHirata <33407409+TomeHirata@users.noreply.github.com>
Co-authored-by: TomeHirata <33407409+TomeHirata@users.noreply.github.com>
dspy/predict/react.py
Outdated
| except ContextWindowExceededError: | ||
| logger.warning("Trajectory exceeded the context window, truncating the oldest tool call information.") | ||
| trajectory = self.truncate_trajectory(trajectory) | ||
| try: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@copilot We don't need this try-catch
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removed the try-catch blocks in both sync and async methods. The ValueError from truncate_trajectory now naturally propagates up and is caught by forward(). Fixed in 2dd54fc.
tests/predict/test_react.py
Outdated
|
|
||
|
|
||
| def test_context_window_exceeded_after_retries(): | ||
| """Test that context window errors are handled gracefully after exhausting retry attempts. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@copilot we don't need docstring in tests
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removed the docstrings from both test functions. Fixed in fa8a09b.
Co-authored-by: TomeHirata <33407409+TomeHirata@users.noreply.github.com>
Co-authored-by: TomeHirata <33407409+TomeHirata@users.noreply.github.com>
tests/predict/test_react.py
Outdated
| assert result.output_text == "Final output" | ||
|
|
||
|
|
||
| def test_context_window_exceeded_after_retries(): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@copilot can we combine two tests by parameterizing?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Combined the sync and async tests into a single test function. Fixed in e95c2dd.
Co-authored-by: TomeHirata <33407409+TomeHirata@users.noreply.github.com>
|
Oh wow this is kinda cool |
📝 Changes Description
_call_with_potential_trajectory_truncationwas implicitly returningNoneafter exhausting 3 retry attempts, causingAttributeError: 'NoneType' object has no attribute 'next_thought'when accessing the prediction.Fix:
ValueErrorwhen retries are exhaustedValueErrorfromtruncate_trajectory(when trajectory cannot be truncated further) naturally propagates upforward()/aforward()which log and break gracefully, allowing the extract phase to proceed with the collected trajectoryChanges:
dspy/predict/react.py: Added explicitraise ValueErrorat end of retry loops in both sync and async versionstests/predict/test_react.py: Added combined test for sync and async context window exhaustion scenarios✅ Contributor Checklist
None.
Original prompt
💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.