Skip to content

Conversation

@GangGreenTemperTatum
Copy link
Contributor

@GangGreenTemperTatum GangGreenTemperTatum commented Dec 11, 2025

enable native fallback_model parameter to llm_judge() sco

Key Changes:

  • suggestion PR: dded native fallback_model parameter to llm_judge() scorer that silently falls back to a secondary
    model if the primary judge fails.

Added:

  • aded fallback_model parameter to llm_judge() function signature
  • created _get_generator() helper function to create generators from model strings or return existing Generator instances
  • implement silent fallback behavior
  • added fallback_model to inner evaluate() function

Generated Summary:

  • Added a fallback_model parameter to the llm_judge function to specify an alternative model in case the primary model fails.
  • Introduced a private helper function _get_generator to streamline the creation of model generators from either string identifiers or Generator instances.
  • Updated the evaluate method to handle exceptions by attempting to use the fallback_model if the primary model encounters an error.
  • Improved error handling for model types by consolidating generator retrieval into a single line of logic, enhancing code clarity and maintainability.
  • These changes allow for more robust judgement scoring by providing alternatives when the primary model fails to respond correctly, potentially improving the system's overall reliability in scoring tasks.

This summary was generated with ❤️ by rigging

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants