Skip to content

Commit

Permalink
.Net: Fixed JSON schema name in Structured Outputs for generic types (m…
Browse files Browse the repository at this point in the history
…icrosoft#9490)

### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

Resolves: microsoft#9416

When C# type is passed as a response format to OpenAI chat completion
service, we use the type name as [JSON schema
name](https://platform.openai.com/docs/api-reference/chat/create#chat-create-response_format).
When the type is generic, its name will contain characters that are not
allowed in JSON schema name according to OpenAI API. This PR fixes the
issue by replacing invalid characters with empty string.

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
  • Loading branch information
dmytrostruk authored Nov 1, 2024
1 parent 936366e commit f80528f
Show file tree
Hide file tree
Showing 2 changed files with 41 additions and 9 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -1179,9 +1179,13 @@ public async Task GetChatMessageContentsSendsValidJsonSchemaForStructuredOutputs
}

[Theory]
[InlineData(typeof(TestStruct))]
[InlineData(typeof(TestStruct?))]
public async Task GetChatMessageContentsSendsValidJsonSchemaWithStruct(Type responseFormatType)
[InlineData(typeof(TestStruct), "TestStruct")]
[InlineData(typeof(TestStruct?), "TestStruct")]
[InlineData(typeof(TestStruct<string>), "TestStructString")]
[InlineData(typeof(TestStruct<string>?), "TestStructString")]
[InlineData(typeof(TestStruct<List<float>>), "TestStructListSingle")]
[InlineData(typeof(TestStruct<List<float>>?), "TestStructListSingle")]
public async Task GetChatMessageContentsSendsValidJsonSchemaWithStruct(Type responseFormatType, string expectedSchemaName)
{
// Arrange
var executionSettings = new OpenAIPromptExecutionSettings { ResponseFormat = responseFormatType };
Expand All @@ -1204,7 +1208,7 @@ public async Task GetChatMessageContentsSendsValidJsonSchemaWithStruct(Type resp
var requestResponseFormat = requestJsonElement.GetProperty("response_format");

Assert.Equal("json_schema", requestResponseFormat.GetProperty("type").GetString());
Assert.Equal("TestStruct", requestResponseFormat.GetProperty("json_schema").GetProperty("name").GetString());
Assert.Equal(expectedSchemaName, requestResponseFormat.GetProperty("json_schema").GetProperty("name").GetString());
Assert.True(requestResponseFormat.GetProperty("json_schema").GetProperty("strict").GetBoolean());

var schema = requestResponseFormat.GetProperty("json_schema").GetProperty("schema");
Expand All @@ -1219,8 +1223,8 @@ public async Task GetChatMessageContentsSendsValidJsonSchemaWithStruct(Type resp
schema.GetProperty("required")[1].GetString(),
};

Assert.Contains("TextProperty", requiredParentProperties);
Assert.Contains("NumericProperty", requiredParentProperties);
Assert.Contains("Property1", requiredParentProperties);
Assert.Contains("Property2", requiredParentProperties);
}

[Fact]
Expand Down Expand Up @@ -1571,9 +1575,16 @@ private sealed class MathReasoningStep

private struct TestStruct
{
public string TextProperty { get; set; }
public string Property1 { get; set; }

public int? NumericProperty { get; set; }
public int? Property2 { get; set; }
}

private struct TestStruct<TProperty>
{
public TProperty Property1 { get; set; }

public int? Property2 { get; set; }
}
#pragma warning restore CS8618, CA1812
}
Original file line number Diff line number Diff line change
Expand Up @@ -573,7 +573,28 @@ private static ChatResponseFormat GetJsonSchemaResponseFormat(Type formatObjectT
var schema = KernelJsonSchemaBuilder.Build(type, configuration: s_jsonSchemaMapperConfiguration);
var schemaBinaryData = BinaryData.FromString(schema.ToString());

return ChatResponseFormat.CreateJsonSchemaFormat(type.Name, schemaBinaryData, jsonSchemaIsStrict: true);
var typeName = GetTypeName(type);

return ChatResponseFormat.CreateJsonSchemaFormat(typeName, schemaBinaryData, jsonSchemaIsStrict: true);
}

/// <summary>
/// Returns a type name concatenated with generic argument type names if they exist.
/// </summary>
private static string GetTypeName(Type type)
{
if (!type.IsGenericType)
{
return type.Name;
}

// If type is generic, base name is followed by ` character.
string baseName = type.Name.Substring(0, type.Name.IndexOf('`'));

Type[] typeArguments = type.GetGenericArguments();
string argumentNames = string.Concat(Array.ConvertAll(typeArguments, GetTypeName));

return $"{baseName}{argumentNames}";
}

/// <summary>Checks if a tool call is for a function that was defined.</summary>
Expand Down

0 comments on commit f80528f

Please sign in to comment.