Skip to content

Commit

Permalink
.Net Agents - Protect against null-reference for Assistant file-searc…
Browse files Browse the repository at this point in the history
…h result when streaming (#9742)

### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

Fixes: #9723

Must distinguish between file-search result and function-call result for
assistant streaming without triggering `NullReferenceException`

### Description
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

A file-search result provide any streaming result; rather, it results in
an annotation on the assistant message. Still it must be distinguished
from a function-call result.

Since `RunStepDetailsUpdate.FunctionName` property throws a
`NullReferenceException` when being evaluated, using
`RunStepDetailsUpdate.FunctionResult` property.

Merged fix to Open AI SDK:
openai/openai-dotnet#293


### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
  • Loading branch information
crickman authored Nov 19, 2024
1 parent c436624 commit f8d7a7e
Show file tree
Hide file tree
Showing 4 changed files with 56 additions and 7 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -451,7 +451,7 @@ public static async IAsyncEnumerable<StreamingChatMessageContent> InvokeStreamin
{
yield return toolContent;
}
else
else if (detailsUpdate.FunctionOutput != null)
{
yield return
new StreamingChatMessageContent(AuthorRole.Assistant, null)
Expand Down
53 changes: 53 additions & 0 deletions dotnet/src/IntegrationTests/Agents/OpenAIAssistantAgentTests.cs
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
using System;
using System.ClientModel;
using System.ComponentModel;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
Expand All @@ -11,6 +12,8 @@
using Microsoft.SemanticKernel.Agents;
using Microsoft.SemanticKernel.Agents.OpenAI;
using Microsoft.SemanticKernel.ChatCompletion;
using OpenAI.Files;
using OpenAI.VectorStores;
using SemanticKernel.IntegrationTests.TestSettings;
using xRetry;
using Xunit;
Expand Down Expand Up @@ -216,6 +219,56 @@ await OpenAIAssistantAgent.CreateAsync(
finally
{
await agent.DeleteThreadAsync(threadId);
await agent.DeleteAsync();
}
}

/// <summary>
/// Integration test for <see cref="OpenAIAssistantAgent"/> using function calling
/// and targeting Open AI services.
/// </summary>
[Fact]
public async Task AzureOpenAIAssistantAgentStreamingFileSearchAsync()
{
var azureOpenAIConfiguration = this._configuration.GetSection("AzureOpenAI").Get<AzureOpenAIConfiguration>();
Assert.NotNull(azureOpenAIConfiguration);

OpenAIClientProvider provider = OpenAIClientProvider.ForAzureOpenAI(new AzureCliCredential(), new Uri(azureOpenAIConfiguration.Endpoint));
OpenAIAssistantAgent agent =
await OpenAIAssistantAgent.CreateAsync(
provider,
new(azureOpenAIConfiguration.ChatDeploymentName!),
new Kernel());

// Upload file - Using a table of fictional employees.
OpenAIFileClient fileClient = provider.Client.GetOpenAIFileClient();
await using Stream stream = File.OpenRead("TestData/employees.pdf")!;
OpenAIFile fileInfo = await fileClient.UploadFileAsync(stream, "employees.pdf", FileUploadPurpose.Assistants);

// Create a vector-store
VectorStoreClient vectorStoreClient = provider.Client.GetVectorStoreClient();
CreateVectorStoreOperation result =
await vectorStoreClient.CreateVectorStoreAsync(waitUntilCompleted: false,
new VectorStoreCreationOptions()
{
FileIds = { fileInfo.Id }
});

string threadId = await agent.CreateThreadAsync();
try
{
await agent.AddChatMessageAsync(threadId, new(AuthorRole.User, "Who works in sales?"));
ChatHistory messages = [];
var chunks = await agent.InvokeStreamingAsync(threadId, messages: messages).ToArrayAsync();
Assert.NotEmpty(chunks);
Assert.Single(messages);
}
finally
{
await agent.DeleteThreadAsync(threadId);
await agent.DeleteAsync();
await vectorStoreClient.DeleteVectorStoreAsync(result.VectorStoreId);
await fileClient.DeleteFileAsync(fileInfo.Id);
}
}

Expand Down
8 changes: 2 additions & 6 deletions dotnet/src/IntegrationTests/IntegrationTests.csproj
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
<Project Sdk="Microsoft.NET.Sdk">
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<AssemblyName>IntegrationTests</AssemblyName>
<RootNamespace>SemanticKernel.IntegrationTests</RootNamespace>
<TargetFramework>net8.0</TargetFramework>
<IsTestProject>true</IsTestProject>
<IsPackable>false</IsPackable>
<NoWarn>$(NoWarn);CA2007,CA1861,VSTHRD111,SKEXP0001,SKEXP0010,SKEXP0020,SKEXP0040,SKEXP0050,SKEXP0060,SKEXP0070,SKEXP0080,SKEXP0110</NoWarn>
<NoWarn>$(NoWarn);CA2007,CA1861,VSTHRD111,SKEXP0001,SKEXP0010,SKEXP0020,SKEXP0040,SKEXP0050,SKEXP0060,SKEXP0070,SKEXP0080,SKEXP0110,OPENAI001</NoWarn>
<UserSecretsId>b7762d10-e29b-4bb1-8b74-b6d69a667dd4</UserSecretsId>
</PropertyGroup>
<ItemGroup>
Expand Down Expand Up @@ -39,7 +39,6 @@
<None Remove="prompts\GenerateStoryHandlebars.yaml" />
<None Remove="skills\FunSkill\Joke\config.json" />
<None Remove="skills\FunSkill\Joke\skprompt.txt" />
<None Remove="TestData\semantic-kernel-info.txt" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Extensions.Configuration" />
Expand Down Expand Up @@ -179,9 +178,6 @@
<EmbeddedResource Include="skills\FunSkill\Joke\skprompt.txt">
<CopyToOutputDirectory>Always</CopyToOutputDirectory>
</EmbeddedResource>
<EmbeddedResource Include="TestData\semantic-kernel-info.txt">
<CopyToOutputDirectory>Always</CopyToOutputDirectory>
</EmbeddedResource>
</ItemGroup>

<ItemGroup>
Expand Down
Binary file not shown.

0 comments on commit f8d7a7e

Please sign in to comment.