[BUG] GetRealtimeConversationClient fails with ApiKeyCredential

[tom-b-iodigital]

Library name and version

Azure.AI.OpenAI 2.1.0

Describe the bug

Starting from version 2.1.0 the following code will fail:

var openAiClient = new AzureOpenAIClient(new Uri("https://xxxxx.openai.azure.com/"), 
    new System.ClientModel.ApiKeyCredential("xxxxx"));
var realtimeClient = openAiClient.GetRealtimeConversationClient("gpt-4o-realtime-preview");

By throwing an exception with message: Value cannot be null. (Parameter 'credential')

In version 2.1.0-beta2 this still worked.

Expected behavior

A realtime conversaton client should be available after the function call without throwing an exception.

Actual behavior

Exception with message 'Value cannot be null. (Parameter 'credential')' is thrown

Reproduction Steps

Add following nuget packages to your project:

<PackageReference Include="Azure.AI.OpenAI" Version="2.1.0" />
<PackageReference Include="OpenAI" Version="2.1.0" />

Use the code below to reproduce (replace the x's with actual values):

var openAiClient = new AzureOpenAIClient(new Uri("https://xxxxx.openai.azure.com/"), 
    new System.ClientModel.ApiKeyCredential("xxxxx"));
var realtimeClient = openAiClient.GetRealtimeConversationClient("gpt-4o-realtime-preview");

Replace the nuget packages to observe the correct behavior

<PackageReference Include="Azure.AI.OpenAI" Version="2.1.0-beta.2" />
<PackageReference Include="OpenAI" Version="2.1.0" />

Environment

No response

[github-actions]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @jpalvarezl @ralph-msft @trrwilson.

[code-Miracle49]

Hello everyone. Any update on this? Facing same issue right now

[code-Miracle49]

@tom-b-iodigital Any workaround for this?

[tom-b-iodigital]

@tom-b-iodigital Any workaround for this?

I just use the beta 2 version of the Azure.AI.OpenAI and the 2.1.0 version of the openai package. That seems to work fine.

<PackageReference Include="Azure.AI.OpenAI" Version="2.1.0-beta.2" /> <PackageReference Include="OpenAI" Version="2.1.0" />

[code-Miracle49]

Oh okay, thank you @tom-b-iodigital.

@tom-b-iodigital Any workaround for this?

I just use the beta 2 version of the Azure.AI.OpenAI and the 2.1.0 version of the openai package. That seems to work fine.

<PackageReference Include="Azure.AI.OpenAI" Version="2.1.0-beta.2" /> <PackageReference Include="OpenAI" Version="2.1.0" />

[octupul]

They have removed the azure OpenAI credentialing overrides for the 2.1.0 release version when using realtime which forces it to try to fallback to the regular OpenAI endpoints which aren't setup when you initialize in this manner. Is this being left out by intention for the team because it is a final release or was this a mistake?

[onionhammer]

So this is still completely borked now? _tokenCredential is completely ignored and keyCredential is used exclusively now? Where are the regression tests?

[IEvangelist]

Yes, this is certainly still a bug that needs to be addressed. When you try targeting NuGet package 2.1.0 of the Azure.AI.OpenAI, you cannot successfully call GetRealtimeConversationClient if you're attempting to use the AzureOpenAIClient constructor that accepts the Uri and ApiKeyCredential. In my case, using the:

var client = new AzureOpenAIClient(endpoint, new AzureKeyCredential(key));

// This throws
var conversationClient = client.GetRealtimeConversationClient("gpt-4o-realtime-preview");

The exception is:

Exception with message 'Value cannot be null. (Parameter 'credential')' is thrown

I experienced this too, the workaround as suggested by @tom-b-iodigital does work:

<PackageReference Include="Azure.AI.OpenAI" Version="2.1.0-beta.2" />
<PackageReference Include="OpenAI" Version="2.1.0" />

Hopefully this is fixed in the next release. /cc @trrwilson FYI.

[adiazcan]

Any plans for fixing this bug?