release mode produces invalid json for structured output

[bradyguyc]

Thanks much for the article and code very helpful.

The optimizer produces different json with this code between release mode and debug mode. Debug mode works just fine with structured output however in release mode the json is incompatible with structured_output. It products json that utilizes $ref referencing nodes not at the top level. The following json is what is produced in release mode for one of my classes.

{
  "type": "object",
  "properties": {
    "Name": {
      "type": "string"
    },
    "seriesId": {
      "type": "string"
    },
    "booksInSeries": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "BookTitle": {
            "type": "string"
          },
          "Author": {
            "type": "string"
          },
          "ISBN_10": {
            "type": "string"
          },
          "ISBN_13": {
            "type": "string"
          },
          "sysMynbIDAsString": {
            "type": "string"
          }
        },
        "additionalProperties": false,
        "required": [
          "BookTitle",
          "Author",
          "ISBN_10",
          "ISBN_13",
          "sysMynbIDAsString"
        ]
      }
    },
    "booksMissingFromSeries": {
      "type": "array",
      "items": {
        "$ref": "#/properties/booksInSeries/items"
      }
    }
  },
  "additionalProperties": false,
  "required": [
    "Name",
    "seriesId",
    "booksInSeries",
    "booksMissingFromSeries"
  ]
}

To rectify the problem I utilized newton soft and updated ProcessJsonObject I should say that AI did the updates, it was either meta, phind, or github copilot I ended up using all three before I found a solution that worked.

I forked the project and made my updates there in case you would like to reference.

https://github.com/bradyguyc/openai-structured-outputs-with-dotnet

Commit Message:
Update dependencies and refactor JSON schema handling

Updated StructuredOutputs.csproj to include new package references:

• Microsoft.Extensions.Logging version 9.0.0
• Newtonsoft.Json version 13.0.3
• Newtonsoft.Json.Schema version 4.0.2-beta1

In StructuredOutputsExtensions.cs:

• Added new using directives.
• Modified StructuredOutputsTransform to accept a nullable JsonSchemaExporterContext.
• Refactored ProcessJsonObject to simplify additionalProperties addition, conditionally add required field, and handle nested objects in arrays.
• Updated CreateJsonSchemaFormat to accept an optional ILogger, use JSchemaGenerator for schema generation, transform the schema, and serialize it to a formatted JSON string.

No changes were made to spelling.dic.

`
using System.Diagnostics;
using System.Linq.Expressions;
using System.Text.Json;
using System.Text.Json.Nodes;
using System.Text.Json.Schema;

using Microsoft.Extensions.Logging;

using Newtonsoft.Json.Schema;
using Newtonsoft.Json.Schema.Generation;

using OpenAI.Chat;

namespace MyNextBookBackEnd;

internal static class StructuredOutputsExtensions
{

public static Func<JsonSchemaExporterContext?, JsonNode, JsonNode> StructuredOutputsTransform = new((context, node) =>
{
    static void ProcessJsonObject(JsonObject jsonObject)
    {
        if (jsonObject.ContainsKey("properties"))
        {
            jsonObject["additionalProperties"] = false;

            var properties = jsonObject["properties"] as JsonObject;
            if (properties != null)
            {
                foreach (var property in properties)
                {
                    if (property.Value is JsonObject nestedObject)
                    {
                        ProcessJsonObject(nestedObject);
                    }
                }
            }
        }

        // Check for nested objects in arrays
        if (jsonObject.ContainsKey("items"))
        {
            var items = jsonObject["items"] as JsonObject;
            if (items != null)
            {
                ProcessJsonObject(items);
            }
        }
    }

    if (node is JsonObject rootObject)
    {
        ProcessJsonObject(rootObject);
    }

    return node;
});

public static T? CompleteChat<T>(this ChatClient chatClient, List<ChatMessage> messages, ChatCompletionOptions options)
{
    ChatCompletion completion = chatClient.CompleteChat(messages, options);
    return JsonSerializer.Deserialize<T>(completion.Content[0].Text);
}

public static ChatResponseFormat CreateJsonSchemaFormat<T>(
       string jsonSchemaFormatName,
       string? jsonSchemaFormatDescription = null,
       bool? jsonSchemaIsStrict = null,
       ILogger? logger = null)
{
    JSchemaGenerator generator = new JSchemaGenerator
    {
        SchemaReferenceHandling = SchemaReferenceHandling.None,
    };
    JSchema schema = generator.Generate(typeof(T));
    JsonNode? schemaNode = JsonNode.Parse(schema.ToString());
    JsonSchemaExporterContext context = new JsonSchemaExporterContext(); // Create an empty context
    schemaNode = StructuredOutputsTransform(context, schemaNode);

    // Serialize the modified schema node to a formatted JSON string
    string jsonString = schemaNode.ToJsonString(new JsonSerializerOptions { WriteIndented = true });

    return ChatResponseFormat.CreateJsonSchemaFormat(
        jsonSchemaFormatName,
        jsonSchema: BinaryData.FromString(jsonString),
        jsonSchemaFormatDescription: jsonSchemaFormatDescription,
        jsonSchemaIsStrict: jsonSchemaIsStrict
    );
}

}

`

[jamesmcroft]

Thanks for letting me know and finding a work around. I originally created this with dotnet 9 in preview, but let me take another look if there's a way to resolve this without the additional JSON lib.

[jamesmcroft]

@bradyguyc thanks for your patience while I investigated this. It turns out in general, what I was trying to achieve by this is not completely compatible with the new .NET System.Text.Json libs.

Instead, I have updated this sample to not rely on these JSON libraries, instead opting to construct the required JSON schema through reflection. You can find the OpenAIJsonSchema class in the repo which performs the necessary construction.

[bradyguyc]

Oh wow, thanks for the persistence and fix, not to mention the concept/class it'self. I have recently moved on to .NET 10 preview and will test it there shortly (couple of weeks or so), several personal projects in the hopper right now.

Thanks!