Skip to content

Conversation

@xcpky
Copy link
Contributor

@xcpky xcpky commented Nov 30, 2025

This PR closes #3490

This PR handles supports_json_schema_output and supports_json_object_output together:

  1. Check both supports_json_schema_output and supports_json_object_output for prompted output
  2. Add use_structured_output_for_prompted_output field to ModelProfile to toggle the checking.

Todos:

  • support other models than openai
  • doc
  • test

@DouweM
Copy link
Collaborator

DouweM commented Dec 1, 2025

@xcpky Please see #3490 (comment).

I think it is more consistent with current behavior and expectations of what the different modes mean to have a model profile flag that makes NativeOutput include the schema in the prompt, than a flag that makes PromptedOutput use the provider API native output feature.

Essentially the question is whether this behavior (schema in prompt + native output feature) is a flavor of "native output" or a flavor of "prompted output", and I feel strongly it's the latter: you're still using the provider's native JSON schema enforcement feature, you're just including the schema in the prompt manually instead of relying on the provider to do so automatically (as OpenAI etc do). The "prompted output" mode is for the case where you don't want any provider-side schema enforcement.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Some models ran through OpenAI-compatible APIs need JSON schema in instructions even when native structured output is used

2 participants