OpenAI Batch API support

[zigzag312]

Hi!

OpenAI offers special batch API that is 50% cheaper than the normal (synchronous) API.

With batch API all requests are combined into a single jsonl (where json requests are newline-delimited). The reason batch API is cheaper is that the response is not immediate. It can take up to 24h to process the batch.

https://platform.openai.com/docs/guides/batch
https://platform.openai.com/docs/api-reference/batch

Book translations seems to be a perfect fit for the batch API. Very high token counts and results usually do not need to be immediate.

Single batch can contain up to 50'000 requests or up to 100MB.

[bookfere]

Thank you for the suggestion! It's a fantastic choice for our translation needs, and I'm excited to implement it. You can create an issue from this discussion to keep track of this requirement.

[zigzag312]

Thank you for a kind and welcoming response! As you suggested I created an issue for this #319.

[bookfere]

I initially implemented the ChatGPT batch translation feature based on OpenAI's API documentation (commit d90fcbf). Since I do not have access to the actual API, there may be many faults within the functionality. To improve the feature's usability, ongoing feedback is needed.

[bookfere]

When you are in Advanced mode and choose "ChatGPT (OpenAI)," a button called "Batch Translate (Beta)" will appear next to the "Translate All" button.

[zigzag312]

I found it, thank you!

I did a quick test.

1. I click "Batch Translate (Beta)" and "ChatGPT Batch Translation" popup opens.
2. I click on "Create Batch Translation" button. Progress bar shows for about a second, then an error occurs:

Traceback (most recent call last):
  File "calibre_plugins.ebook_translator.components.chatgpt", line 27, in wrapper
  File "calibre_plugins.ebook_translator.components.chatgpt", line 83, in create_batch
  File "calibre_plugins.ebook_translator.engines.openai", line 210, in upload
  File "json\__init__.py", line 339, in loads
TypeError: the JSON object must be str, bytes or bytearray, not response_seek_wrapper

[DomBedos]

Thanks @bookfere for implementing this feature! I'd be really happy if it worked, but I'm getting exactly the same error message as @zigzag312

[bookfere]

Thank you for your interest in the feature. At the moment, I need to prioritize taking care of a family member who is in the hospital. Once I have some free time available, I will resume the debugging process. I truly appreciate your patience during this time.

[zigzag312]

@bookfere Thank you for the update. Wishing your family member a smooth recovery!