stanfordnlp
diff --git a/‎docs/docs/api/optimizers/GEPA/GEPA_Advanced.md‎
Lines changed: 143 additions & 0 deletions b/‎docs/docs/api/optimizers/GEPA/GEPA_Advanced.md‎
Lines changed: 143 additions & 0 deletions
diff --git a/‎docs/docs/api/optimizers/GEPA/overview.md‎
Lines changed: 6 additions & 0 deletions b/‎docs/docs/api/optimizers/GEPA/overview.md‎
Lines changed: 6 additions & 0 deletions
@@ -443,3 +443,146 @@ gepa = dspy.GEPA(
     auto="medium"
 )
 ```
+
+## Tool Optimization
+
+### What is enable_tool_optimization?
+
+When `enable_tool_optimization=True`, GEPA jointly optimizes `dspy.ReAct` modules: predictor instructions and tool descriptions and argument descriptions are updated together, instead of being tuned in isolation. This lets the model learn better patterns for when to call a tool and how to use it from the same execution traces and feedback that drive core GEPA.
+
+### Usage and constraints
+
+- **Expose tools as `dspy.Tool` in signatures and examples.** GEPA only optimizes tools that are represented as `dspy.Tool` and actually passed as `dspy.Tool` objects into your modules.
+- **Treat `Tool.name` as a stable identifier.** `Tool.name` is the tool's name, and GEPA uses it to attach improved descriptions and argument descriptions. If you reuse the same `Tool.name` for different tools, they will share the same text updates.
+- **Avoid custom tools named `"finish"`.** The built-in ReAct `"finish"` tool is reserved and excluded from optimization. Custom tools with the name `"finish"` are also not optimized.
+- **Custom instruction proposers handle all modules and tool updates.** When you provide an `instruction_proposer`, GEPA routes every optimized module through your proposer instead of the built-in instruction proposer. If `enable_tool_optimization=True`, modules that call tools are still included, and your proposer is also responsible for updating their tool descriptions and argument descriptions.
+
+### Tool Module Optimization Prompt
+
+GEPA uses `ToolProposer` to optimize ReAct modules when `enable_tool_optimization=True`. For each module, the proposer builds a dynamic signature from the base `GenerateImprovedToolModuleDescriptionsFromFeedback` signature shown below, then appends output fields for each tool description and each tool argument description in that module. For ReAct modules, the proposer also appends input and output fields for the extract instruction.
+
+```python
+class GenerateImprovedToolModuleDescriptionsFromFeedback(dspy.Signature):
+    """I provided an assistant with predictor instructions and tool descriptions,
+    but its performance needs improvement based on the examples_with_feedback below.
+
+    Your task is to propose better predictor instructions, tool descriptions, and
+    tool argument descriptions that address the issues shown in these examples.
+    Focus on reinforcing patterns that clearly improve the assistant's performance
+    on similar tasks, rather than rewriting everything from scratch unless necessary.
+    These components are progressively optimized - refine only what needs to change.
+
+    Analyze the examples_with_feedback to identify success and failure patterns,
+    and write improved instructions and descriptions at their appropriate level
+    of abstraction and/or specificity, so that each layer plays a clear,
+    complementary role without unnecessary repetition or verbosity unless
+    redundancy clearly helps the assistant's performance.
+    """
+
+    current_predictor_instruction = dspy.InputField(
+        desc="Current instruction guiding the predictor"
+    )
+    current_tools = dspy.InputField(
+        annotation=list[dspy.Tool],
+        desc="Available tools with their complete schemas"
+    )
+    examples_with_feedback = dspy.InputField(
+        desc="Execution examples with feedback showing successes and failures"
+    )
+
+    improved_predictor_instruction: str | None = dspy.OutputField(
+        desc="Improved instruction for the predictor",
+        default=None
+    )
+
+    # GEPA appends output fields dynamically for each tool and argument:
+    # - improved_tool_{name}_desc with desc="Improved description of tool '{name}'"
+    # - improved_tool_{name}_arg_{param}_desc with desc="Improved description of the argument '{param}' of tool '{name}'"
+    # For ReAct modules, GEPA also appends:
+    # - current_extract_instruction (input) with desc="Current instruction for extraction predictor"
+    # - improved_extract_instruction (output) with desc="Improved instruction for extraction"
+```
+
+The reflection LM uses this dynamically-built signature to jointly propose updates across predictor instructions, tool descriptions, and argument descriptions based on execution feedback. Updates are coordinated rather than made in isolation: the LM sees all current components together and can selectively update any subset by returning new text, or return `None` to keep a component unchanged.
+
+### How Tool Optimization Works
+
+When `enable_tool_optimization=True`, GEPA:
+
+1. **Discovers ReAct modules** - Identifies `dspy.ReAct` modules and their associated tools
+2. **Treats them as joint optimization units** - Instead of only optimizing predictor instructions, GEPA optimizes predictor instructions and tool descriptions together as a coordinated set; for ReAct this includes both the react and extract instructions
+3. **Routes to specialized proposer** - Separates components by type and routes them appropriately:
+   - **With custom `instruction_proposer`**: Your custom proposer receives both ReAct modules and plain predictors, and is responsible for updating all components
+   - **With default proposer**: Plain predictors use the default instruction proposer; ReAct modules use `ToolProposer`, which employs the dynamic signature mechanism described above
+4. **Optimizes jointly** - `ToolProposer` improves predictor instructions and tool descriptions together based on execution feedback, coordinating updates across all components rather than tuning them in isolation
+5. **Applies updates** - Improved instructions update predictor signatures; improved tool descriptions and argument descriptions update all `dspy.Tool` objects with matching tool names throughout the program
+
+Modules without tools (like `dspy.Predict` or `dspy.ChainOfThought`) continue using standard GEPA instruction-only optimization.
+
+### When to Use Tool Optimization
+
+Enable `enable_tool_optimization=True` when tools are central to your program's behavior and you want GEPA to jointly optimize predictor instructions and tool descriptions together. Common scenarios:
+
+1. **Wrong tool selection** - Predictor with `search` and `weather` tools keeps searching when it should check weather, or vice versa. GEPA refines predictor instructions and tool descriptions to clarify when to use each tool.
+
+2. **Underused tools** - Predictor responds "I don't know" without using available tools that could answer the question. GEPA improves predictor instructions to be more proactive about tool usage.
+
+3. **Tool call loops** - Agent keeps calling `web_search` multiple times with similar queries instead of synthesizing information. GEPA improves instructions to encourage synthesis and tool descriptions to clarify when searches are sufficient.
+
+4. **Extraction failures (ReAct)** - Agent executes tools correctly but fails to extract the final answer from the trajectory. GEPA improves extract instruction to better identify and format answers from tool outputs.
+
+5. **Multi-agent delegation** - Parent agent has delegation tools to specialized sub-agents but doesn't understand when to use each. GEPA optimizes instructions and tool descriptions across both parent and sub-agent modules for coherent delegation.
+
+See the usage example below for tool-using programs.
+
+### Usage Example
+
+```python
+import dspy
+
+def search_web(query: str) -> str:
+    return f"Search results for: {query}"
+
+def get_weather(city: str) -> str:
+    """Get the current weather for a city."""
+    return f"The weather in {city} is sunny and 75°F"
+
+# Create tools with basic descriptions
+search_tool = dspy.Tool(search_web, name="search_web", desc="Search tool")
+weather_tool = dspy.Tool(get_weather, name="get_weather", desc="Weather tool")
+
+program = dspy.ReAct("question -> answer", tools=[search_tool, weather_tool])
+
+# Enable tool optimization
+gepa = dspy.GEPA(
+    metric=my_metric,
+    reflection_lm=dspy.LM(model="gpt-5-mini"),
+    enable_tool_optimization=True,
+    auto="medium"
+)
+
+optimized_program = gepa.compile(program, trainset=train_examples, valset=val_examples)
+```
+
+### Inspecting Optimized Programs
+
+View optimization results and metadata (requires `track_stats=True`):
+
+```python
+# High-level optimization metadata
+optimized_program.detailed_results
+```
+
+Access optimized instructions and tool descriptions directly:
+
+```python
+# Predictor instructions
+for name, predictor in optimized_program.named_predictors():
+    print(f"{name}: {predictor.signature.instructions}")
+
+# Tool descriptions and argument descriptions
+for tool_name, tool in optimized_program.tools.items():
+    print(f"{tool_name}: {tool.desc}")
+    for arg_name, arg_schema in tool.args.items():
+        print(f"  {arg_name}: {arg_schema.get('description', 'N/A')}")
+```
@@ -117,6 +117,12 @@ Practical Recipe for GEPA-Friendly Feedback:
 - **Multi-Objective Tasks** (e.g., PUPA): Decompose aggregate scores to reveal contributions from each objective, highlighting tradeoffs (e.g., quality vs. privacy).
 - **Stacked Pipelines** (e.g., code generation: parse → compile → run → profile → evaluate): Expose stage-specific failures; natural-language traces often suffice for LLM self-correction.
 
+## Tool Optimization with GEPA
+
+When `enable_tool_optimization=True`, GEPA jointly optimizes `dspy.ReAct` modules with the tools - GEPA updates predictor instructions and tool descriptions/argument descriptions together, based on execution traces and feedback, instead of keeping tool behavior fixed.
+
+For details, examples, and the underlying design (tool discovery, naming requirements, and interaction with custom instruction proposers), see [Tool Optimization](GEPA_Advanced.md#tool-optimization).
+
 ## Custom Instruction Proposal
 
 For advanced customization of GEPA's instruction proposal mechanism, including custom instruction proposers and component selectors, see [Advanced Features](GEPA_Advanced.md).