Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Dec 4, 2025

📝 Changes Description

_call_with_potential_trajectory_truncation was implicitly returning None after exhausting 3 retry attempts, causing AttributeError: 'NoneType' object has no attribute 'next_thought' when accessing the prediction.

Fix:

  • Raise explicit ValueError when retries are exhausted
  • The ValueError from truncate_trajectory (when trajectory cannot be truncated further) naturally propagates up
  • Both errors are caught by forward()/aforward() which log and break gracefully, allowing the extract phase to proceed with the collected trajectory
# Before: would crash with AttributeError on None.next_thought
react = dspy.ReAct("question -> answer", tools=[...])
result = react(question="...")  # with very large context

# After: gracefully ends trajectory and proceeds to extraction

Changes:

  • dspy/predict/react.py: Added explicit raise ValueError at end of retry loops in both sync and async versions
  • tests/predict/test_react.py: Added combined test for sync and async context window exhaustion scenarios

✅ Contributor Checklist

  • Pre-Commit checks are passing (locally and remotely)
  • Title of your PR / MR corresponds to the required format
  • Commit message follows required format {label}(dspy): {message}

⚠️ Warnings

None.

Original prompt

This section details on the original issue you should resolve

<issue_title>[Bug] ContextWindowExceededError after 3 retries in react loop</issue_title>
<issue_description>### What happened?

Context

We get this error:NoneType' object has no attribute 'next_thought' when we work with observations/trajectories that continue to exceed the context window limit

  • in the forward pass, we run _call_with_potential_trajectory_truncation() (code)

  • this has a hardcoded limit of 3 retries.

  • once that is hit, we return None (code) which causes attribute error over NoneType (code)

Question

  1. What is the expected behavior? Doesn't a while loop until success make more sense?
  2. On our end, we can monkey patch the truncation logic or even add our own logic for long trajectories but I think this is a bug

Steps to reproduce

explained above

DSPy version

3.0.3</issue_description>

<agent_instructions>Let's handle failure case (None) gracefully</agent_instructions>

Comments on the Issue (you are @copilot in this section)


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI and others added 2 commits December 4, 2025 07:35
When _call_with_potential_trajectory_truncation exhausts all retry
attempts, it now raises a clear ValueError instead of returning None.
This prevents the AttributeError: 'NoneType' object has no attribute
'next_thought' that occurred when accessing properties on the None
return value.

The ValueError is caught in forward/aforward and causes the loop to
break gracefully, allowing the extract phase to proceed with whatever
trajectory was collected.

Added tests for both sync and async versions of this scenario.

Co-authored-by: TomeHirata <33407409+TomeHirata@users.noreply.github.com>
Co-authored-by: TomeHirata <33407409+TomeHirata@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix ContextWindowExceededError after retries in react loop Fix ContextWindowExceededError after 3 retries in react loop Dec 4, 2025
Copilot finished work on behalf of TomeHirata December 4, 2025 07:39
Copilot AI requested a review from TomeHirata December 4, 2025 07:39
except ContextWindowExceededError:
logger.warning("Trajectory exceeded the context window, truncating the oldest tool call information.")
trajectory = self.truncate_trajectory(trajectory)
try:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot We don't need this try-catch

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the try-catch blocks in both sync and async methods. The ValueError from truncate_trajectory now naturally propagates up and is caught by forward(). Fixed in 2dd54fc.



def test_context_window_exceeded_after_retries():
"""Test that context window errors are handled gracefully after exhausting retry attempts.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot we don't need docstring in tests

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the docstrings from both test functions. Fixed in fa8a09b.

Co-authored-by: TomeHirata <33407409+TomeHirata@users.noreply.github.com>
Copilot AI requested a review from TomeHirata December 4, 2025 08:02
Copilot finished work on behalf of TomeHirata December 4, 2025 08:02
Co-authored-by: TomeHirata <33407409+TomeHirata@users.noreply.github.com>
Copilot finished work on behalf of TomeHirata December 4, 2025 08:07
assert result.output_text == "Final output"


def test_context_window_exceeded_after_retries():
Copy link
Collaborator

@TomeHirata TomeHirata Dec 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot can we combine two tests by parameterizing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Combined the sync and async tests into a single test function. Fixed in e95c2dd.

Co-authored-by: TomeHirata <33407409+TomeHirata@users.noreply.github.com>
Copilot AI requested a review from TomeHirata December 4, 2025 09:24
Copilot finished work on behalf of TomeHirata December 4, 2025 09:24
@ianyu93
Copy link

ianyu93 commented Dec 6, 2025

Oh wow this is kinda cool

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] ContextWindowExceededError after 3 retries in react loop

3 participants