OpenAiService.ChatCompletion.CreateCompletion Exception: The input does not contain any JSON tokens (Root cause: HttpClient.Timeout)
#226
-
|
Starting Words Again, thank you very much for this handy library and sorry to have to bother you again🥲 Describe the bug Your code piece var system = "In this setup, you are going to help the user to generate text-based contents according to their specifications.";
var user = """
In this setup, you are going to help me populate a database that represents personel information of a company. The database is going to be described using Prolog, specifically, the SWI implementation of Prolog.
See below snippet for schema and some information:
```prolog
human('Charles Wang', 55, male, 'Professional Engineer').
human('Alice Yu', 27, female, 'Software Developer').
human('Bob Jones', 30, male, 'Doctor').
human('Cynthia Lee', 25, female, 'Nurse').
human('David Smith', 5, male, 'Student').
father('Charles Wang', 'Alice Yu').
father('Charles Wang', 'Bob Jones').
mother('Cynthia Lee', 'David Smith').
father(X, Y) :- human(X, _, male, _), father(X, Y).
mother(X, Y) :- human(X, _, female, _), father(X, Y).
parent(X, Y) :- father(X, Y).
parent(X, Y) :- mother(X, Y).
son(X, Y) :- human(X, _, male, _), parent(Y, X).
daughter(X, Y) :- human(X, _, female, _), parent(Y, X).
grandparent(X, Y) :- parent(X, Z), parent(Z, Y).
grandfather(X, Y) :- grandparent(X, Y), human(X, _, male, _).
grandmother(X, Y) :- grandparent(X, Y), human(X, _, female, _).
company('Top Tech', 'Technology Consultation', 'Toronto', 150).
employee('Charles Wang', 'Top Tech', 2007, 125000).
employee('Thomas Ho', 'Top Tech', 2002, 140000).
employee('Sarah Adams', 'Top Tech', 2010, 100000).
employee('Jenny Smith', 'Top Tech', 2015, 90000).
teacher('William Brown', 'Top Tech', 2006, 110000).
manager('Thomas Ho', 'Charles Wang').
manager('Thomas Ho', 'Sarah Adams').
supervisor('Charles Wang', 'Jenny Smith').
supervisor('Sarah Adams', 'William Brown').
directly_reports_to(Subordinate, Manager) :- manager(Manager, Subordinate).
directly_reports_to(Subordinate, Supervisor) :- supervisor(Supervisor, Subordinate).
indirectly_reports_to(Subordinate, Manager) :- directly_reports_to(Subordinate, Supervisor), directly_reports_to(Supervisor, Manager).
team_member_of(EmployeeA, EmployeeB) :-
directly_reports_to(EmployeeA, Manager),
directly_reports_to(EmployeeB, Manager),
EmployeeA \== EmployeeB.
```
Where:
1. `human` describes character name, age, gender and current profession/title.
2. `company` describes company name, company type, location, and number of employees.
3. `employee` describes employee name, employer name, start year, and current salary.
You may fix any syntax/grammar errors in existing code if necessary.
Now, help me populate a database with 25 employee information for a fictional VR hardware (technology) company named Dreamscape, where a majority of employee have an Asian background but most have adopted western first names; Other employees are local Canadians. The database should also populate all relative or related person information, i.e. people with direct relations to those 25 people, like parents, and children.
""";Result Run with: OpenAIService OpenAiService = new OpenAIService(new OpenAiOptions()
{
ApiKey = "Key......................................"
});
var CurrentModel = Models.Gpt_4;
var CurrentMaxTokenLimit = 8192;
var messages = new ChatMessage[]
{
ChatMessage.FromSystem(system),
ChatMessage.FromUser(user),
}
var completionResult = await OpenAiService.ChatCompletion.CreateCompletion(new ChatCompletionCreateRequest()
{
Messages = messages,
Model = CurrentModel,
MaxTokens = CurrentMaxTokenLimit - messages.Sum(m => m.Content.Length) - 21, /*MAX is 4096 including inputs and outputs; The additional 21 (sometimes 20) seems like the default input token size.*/
Temperature = 1
});And you will encounter a Expected behavior Even if the service cannot handle it, it should not throw an exception - I thought that's what the Screenshots
Desktop (please complete the following information):
Additional context Will upload shorter inputs that may trigger the same issue when available. Complete Code Copy and paste below code for a fresh Console Application created with Visual Studio targeting Net 7: using OpenAI.GPT3.ObjectModels.RequestModels;
using OpenAI.GPT3.ObjectModels;
using OpenAI.GPT3.Managers;
using OpenAI.GPT3;
namespace ConsoleApp
{
internal class Program
{
static async Task Main(string[] args)
{
var system = "In this setup, you are going to help the user to generate text-based contents according to their specifications.";
var user = """
In this setup, you are going to help me populate a database that represents personel information of a company. The database is going to be described using Prolog, specifically, the SWI implementation of Prolog.
See below snippet for schema and some information:
```prolog
human('Charles Wang', 55, male, 'Professional Engineer').
human('Alice Yu', 27, female, 'Software Developer').
human('Bob Jones', 30, male, 'Doctor').
human('Cynthia Lee', 25, female, 'Nurse').
human('David Smith', 5, male, 'Student').
father('Charles Wang', 'Alice Yu').
father('Charles Wang', 'Bob Jones').
mother('Cynthia Lee', 'David Smith').
father(X, Y) :- human(X, _, male, _), father(X, Y).
mother(X, Y) :- human(X, _, female, _), father(X, Y).
parent(X, Y) :- father(X, Y).
parent(X, Y) :- mother(X, Y).
son(X, Y) :- human(X, _, male, _), parent(Y, X).
daughter(X, Y) :- human(X, _, female, _), parent(Y, X).
grandparent(X, Y) :- parent(X, Z), parent(Z, Y).
grandfather(X, Y) :- grandparent(X, Y), human(X, _, male, _).
grandmother(X, Y) :- grandparent(X, Y), human(X, _, female, _).
company('Top Tech', 'Technology Consultation', 'Toronto', 150).
employee('Charles Wang', 'Top Tech', 2007, 125000).
employee('Thomas Ho', 'Top Tech', 2002, 140000).
employee('Sarah Adams', 'Top Tech', 2010, 100000).
employee('Jenny Smith', 'Top Tech', 2015, 90000).
teacher('William Brown', 'Top Tech', 2006, 110000).
manager('Thomas Ho', 'Charles Wang').
manager('Thomas Ho', 'Sarah Adams').
supervisor('Charles Wang', 'Jenny Smith').
supervisor('Sarah Adams', 'William Brown').
directly_reports_to(Subordinate, Manager) :- manager(Manager, Subordinate).
directly_reports_to(Subordinate, Supervisor) :- supervisor(Supervisor, Subordinate).
indirectly_reports_to(Subordinate, Manager) :- directly_reports_to(Subordinate, Supervisor), directly_reports_to(Supervisor, Manager).
team_member_of(EmployeeA, EmployeeB) :-
directly_reports_to(EmployeeA, Manager),
directly_reports_to(EmployeeB, Manager),
EmployeeA \== EmployeeB.
```
Where:
1. `human` describes character name, age, gender and current profession/title.
2. `company` describes company name, company type, location, and number of employees.
3. `employee` describes employee name, employer name, start year, and current salary.
You may fix any syntax/grammar errors in existing code if necessary.
Now, help me populate a database with 25 employee information for a fictional VR hardware (technology) company named Dreamscape, where a majority of employee have an Asian background but most have adopted western first names; Other employees are local Canadians. The database should also populate all relative or related person information, i.e. people with direct relations to those 25 people, like parents, and children.
""";
var CurrentModel = Models.Gpt_4;
var CurrentMaxTokenLimit = 8192;
var messages = new ChatMessage[]
{
ChatMessage.FromSystem(system),
ChatMessage.FromUser(user),
};
OpenAIService OpenAiService = new OpenAIService(new OpenAiOptions()
{
ApiKey = "Key......................................"
});
var completionResult = await OpenAiService.ChatCompletion.CreateCompletion(new ChatCompletionCreateRequest()
{
Messages = messages,
Model = CurrentModel,
MaxTokens = CurrentMaxTokenLimit - messages.Sum(m => m.Content.Length) - 21, /*MAX is 4096 including inputs and outputs; The additional 21 (sometimes 20) seems like the default input token size.*/
Temperature = 1
});
}
}
} |
Beta Was this translation helpful? Give feedback.
Replies: 8 comments
-
|
🙏🏻 You can do this easily with Laser Cat Eyes. It is a library (written by me and it is free to use)that shows you incoming and outgoing requests and responses. If you need help, let me know. Also, A sample usage is available in Playground Project. I think the error is not related to the library. We can see more detail after you catch the response. |
Beta Was this translation helpful? Give feedback.
-
|
By the way, with current execution speed of GPT-4, above code takes around 1:39 minutes to execute. |
Beta Was this translation helpful? Give feedback.
-
Anyway, I tried to make the same request in raw JSON REST request and here is the request and the response. Request{
"model": "gpt-4",
"messages": [
{
"role": "system",
"content": "In this setup, you are going to help the user to generate text-based contents according to their specifications."
},
{
"role": "user",
"content": "In this setup, you are going to help me populate a database that represents personel information of a company. The database is going to be described using Prolog, specifically, the SWI implementation of Prolog.\r\n\r\nSee below snippet for schema and some information:\r\n\r\n```prolog\r\nhuman('Charles Wang', 55, male, 'Professional Engineer').\r\nhuman('Alice Yu', 27, female, 'Software Developer').\r\nhuman('Bob Jones', 30, male, 'Doctor').\r\nhuman('Cynthia Lee', 25, female, 'Nurse').\r\nhuman('David Smith', 5, male, 'Student').\r\nfather('Charles Wang', 'Alice Yu').\r\nfather('Charles Wang', 'Bob Jones').\r\nmother('Cynthia Lee', 'David Smith').\r\n\r\nfather(X, Y) :- human(X, _, male, _), father(X, Y).\r\nmother(X, Y) :- human(X, _, female, _), father(X, Y).\r\nparent(X, Y) :- father(X, Y).\r\nparent(X, Y) :- mother(X, Y).\r\nson(X, Y) :- human(X, _, male, _), parent(Y, X).\r\ndaughter(X, Y) :- human(X, _, female, _), parent(Y, X).\r\ngrandparent(X, Y) :- parent(X, Z), parent(Z, Y).\r\ngrandfather(X, Y) :- grandparent(X, Y), human(X, _, male, _).\r\ngrandmother(X, Y) :- grandparent(X, Y), human(X, _, female, _).\r\n\r\ncompany('Top Tech', 'Technology Consultation', 'Toronto', 150).\r\nemployee('Charles Wang', 'Top Tech', 2007, 125000).\r\nemployee('Thomas Ho', 'Top Tech', 2002, 140000).\r\nemployee('Sarah Adams', 'Top Tech', 2010, 100000).\r\nemployee('Jenny Smith', 'Top Tech', 2015, 90000).\r\nteacher('William Brown', 'Top Tech', 2006, 110000).\r\nmanager('Thomas Ho', 'Charles Wang').\r\nmanager('Thomas Ho', 'Sarah Adams').\r\nsupervisor('Charles Wang', 'Jenny Smith').\r\nsupervisor('Sarah Adams', 'William Brown').\r\ndirectly_reports_to(Subordinate, Manager) :- manager(Manager, Subordinate).\r\ndirectly_reports_to(Subordinate, Supervisor) :- supervisor(Supervisor, Subordinate).\r\nindirectly_reports_to(Subordinate, Manager) :- directly_reports_to(Subordinate, Supervisor), directly_reports_to(Supervisor, Manager).\r\nteam_member_of(EmployeeA, EmployeeB) :-\r\n directly_reports_to(EmployeeA, Manager),\r\n directly_reports_to(EmployeeB, Manager),\r\n EmployeeA \\== EmployeeB.\r\n```\r\n\r\nWhere:\r\n\r\n1. `human` describes character name, age, gender and current profession\/title.\r\n2. `company` describes company name, company type, location, and number of employees.\r\n3. `employee` describes employee name, employer name, start year, and current salary.\r\n\r\nYou may fix any syntax\/grammar errors in existing code if necessary.\r\nNow, help me populate a database with 25 employee information for a fictional VR hardware (technology) company named Dreamscape, where a majority of employee have an Asian background but most have adopted western first names; Other employees are local Canadians. The database should also populate all relative or related person information, i.e. people with direct relations to those 25 people, like parents, and children."
}
]
}Response
ConclusionOk, it turns out this is indeed NOT a JSON reply, my bad🥲 According to OpenAI: A Timeout error indicates that your request took too long to complete and our server closed the connection. This could be due to a network issue, a heavy load on our services, or a complex request that requires more processing time. Again, I think this should be handled by the library, not by the user code. Also, on above link it showed that:
However I don't know how I can change that parameter (and honestly, I shouldn't unless it's exposed by the library). (EDIT: In Insomnia, one can disable timeout globally) |
Beta Was this translation helpful? Give feedback.
-
|
I see this old issue, not sure about the strategy by the library author here: #186 |
Beta Was this translation helpful? Give feedback.
-
|
Some observations:
Also, just to prove that even set infinite time out this issue still happens:
|
Beta Was this translation helpful? Give feedback.
-
|
Hi, I can confirm that with one successful request that takes 109 seconds to complete using raw request in Insomnia, it cannot be done with current API because The call to I saw this commit but the available options do not seem to allow configuring time out. |
Beta Was this translation helpful? Give feedback.
-
|
For anyone else having the same problem, currently I am dealing with it with the following customized service provider (largely inspired by existing design of Betalgo OpenAI service): using System;
using System.Collections.Generic;
using System.Net.Http.Headers;
using System.Net.Http;
using System.Net;
using System.Text;
using System.Threading.Tasks;
using System.Linq;
using Newtonsoft.Json;
namespace ChatService.Types
{
public class ChatMessage
{
[JsonProperty("role")]
public string Role { get; set; }
[JsonProperty("content")]
public string Content { get; set; }
public ChatMessage(string role, string content)
{
Role = role;
Content = content;
}
public static ChatMessage FromAssistant(string content)
{
return new ChatMessage("assistant", content);
}
public static ChatMessage FromUser(string content)
{
return new ChatMessage("user", content);
}
public static ChatMessage FromSystem(string content)
{
return new ChatMessage("system", content);
}
}
public class ChatService
{
#region Construction
public string APIKey { get; }
public string Model { get; }
public ChatService(string key, string model)
{
APIKey = key;
Model = model;
}
#endregion
#region Interface Method
public async Task<string> Request(IEnumerable<ChatMessage> messages, int tokens, float temperature)
{
var inputJson = new InputJson
{
Model = Model,
Messages = messages.ToArray()
};
var inputJsonString = JsonConvert.SerializeObject(inputJson);
var content = new StringContent(inputJsonString, Encoding.UTF8, "application/json");
using var client = new HttpClient();
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", APIKey);
client.Timeout = TimeSpan.FromMinutes(5); // Deal with longer execution time that's usually needed for GPT4
var response = await client.PostAsync(EndpointUrl, content);
if (response.StatusCode != HttpStatusCode.OK)
{
throw new Exception($"HTTP request failed with status code {response.StatusCode} and message {await response.Content.ReadAsStringAsync()}");
}
var responseJsonString = await response.Content.ReadAsStringAsync();
var outputJson = JsonConvert.DeserializeObject<OutputJson>(responseJsonString);
return outputJson.Choices.FirstOrDefault()?.Message.Content ?? null;
}
#endregion
#region Implementation Details
private readonly string EndpointUrl = "https://api.openai.com/v1/chat/completions";
#endregion
#region Types
private class InputJson
{
[JsonProperty("model")]
public string Model { get; set; }
[JsonProperty("messages")]
public ChatMessage[] Messages { get; set; }
}
private class OutputJson
{
[JsonProperty("id")]
public string ID { get; set; }
[JsonProperty("object")]
public string Object { get; set; }
[JsonProperty("model")]
public string Model { get; set; }
[JsonProperty("usage")]
public Usage Usage { get; set; }
[JsonProperty("choices")]
public Choice[] Choices { get; set; }
}
private class Usage
{
[JsonProperty("prompt_tokens")]
public string PromptTokens { get; set; }
[JsonProperty("completion_tokens")]
public string CompletionTokens { get; set; }
[JsonProperty("total_tokens")]
public string TotalTokens { get; set; }
}
private class Choice
{
[JsonProperty("message")]
public ChatMessage Message { get; set; }
[JsonProperty("finish_reason")]
public string FinishReason { get; set; }
}
#endregion
}
}This works well for both GPT3.5-turbo and GPT4. For JSON serialization, it uses Newtonsoft.Json - you may replace it with Net 7 native JSON serializer. |
Beta Was this translation helpful? Give feedback.
-
|
Hi, sorry. I was too busy last week and couldn't reply to any messages. I know the library should handle HTML responses too but I don't have time for that yet. I appreciate it if you would like to send a PR fix that. Here are 2 samples if you would like to customize httpclient. Since we both agree the main issue is not related to the library, I am going to move this thread to discussions. If you don't agree or something else please let me know 🙏🏻 |
Beta Was this translation helpful? Give feedback.
Hi, sorry. I was too busy last week and couldn't reply to any messages.
I know the library should handle HTML responses too but I don't have time for that yet. I appreciate it if you would like to send a PR fix that. Here are 2 samples if you would like to customize httpclient.
Since we both agree the main issue is not related to the library, I am going to move this thread to discussions.
If you don't agree or something else please let me know 🙏🏻