Best way to use an OpenAI-compatible embedding API

[BeautyyuYanli]

Hello everyone!

I'm using my own OpenAI-compatible embedding API, the runnable code:

from llama_index.embeddings.openai import OpenAIEmbedding

emb_model = OpenAIEmbedding(
    api_key="DUMMY_API_KEY",
    api_base="http://192.168.0.1:8000",
    model_name="intfloat/multilingual-e5-large",
)
emb = emb_model.get_text_embedding("hello world")
print(emb)

But if I use the arg model instead of model_name, It will raise:

ValueError: 'intfloat/multilingual-e5-large' is not a valid OpenAIEmbeddingModelType

I checked the code in llama_index/embeddings/openai/base.py,

class OpenAIEmbedding(BaseEmbedding):
    ...
    def __init__(
        ...
        self._query_engine = get_engine(mode, model, _QUERY_MODE_MODEL_DICT)
        self._text_engine = get_engine(mode, model, _TEXT_MODE_MODEL_DICT)

        if "model_name" in kwargs:
            model_name = kwargs.pop("model_name")
            self._query_engine = self._text_engine = model_name
        else:
            model_name = model

So I'm not sure if it is a good idea to set the model_name arg to use my custom OpenAI-compatible API? If not, is there a better way? If yes, maybe we need to add some documents?

[dosubot[bot]]

Hey there @BeautyyuYanli! Good to see you back in the world of embeddings 🌐🦙

Based on the code and context you've provided, it seems like you're on the right track. Using the model_name argument to specify your custom OpenAI-compatible API is a valid approach. This is because the model_name parameter allows for directly specifying the embedding engine without going through the mapping logic that uses the model parameter.

In the OpenAIEmbedding class, if model_name is provided in the keyword arguments (kwargs) during the initialization, it overrides the engine selection logic based on the model parameter. Both the _query_engine and _text_engine attributes are set directly to the value of model_name. This provides flexibility for users who know exactly which engine they want to use or need to use an engine that doesn't fit the predefined model types.

Here's the relevant part of the code from llama_index/embeddings/openai/base.py:

class OpenAIEmbedding(BaseEmbedding):
    ...
    def __init__(
        ...
        self._query_engine = get_engine(mode, model, _QUERY_MODE_MODEL_DICT)
        self._text_engine = get_engine(mode, model, _TEXT_MODE_MODEL_DICT)

        if "model_name" in kwargs:
            model_name = kwargs.pop("model_name")
            self._query_engine = self._text_engine = model_name
        else:
            model_name = model

As for the documentation, I agree with you. It would be beneficial to have more detailed documentation about the usage of model and model_name parameters in the OpenAIEmbedding class. This would help users to better understand how to use these parameters and how they affect the behavior of the class. I recommend opening an issue in the LlamaIndex repository to suggest this improvement.

I hope this helps! If you have any more questions, feel free to ask.

Sources

• llama-index-legacy/llama_index/legacy/llms/openai.py
• llama-index-legacy/llama_index/legacy/embeddings/openai.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[BeautyyuYanli]

Do you generate the answer by your illusion?

[Searcherr]

Hello everyone!

I'm using my own OpenAI-compatible embedding API, the runnable code:

from llama_index.embeddings.openai import OpenAIEmbedding

emb_model = OpenAIEmbedding(
    api_key="DUMMY_API_KEY",
    api_base="http://192.168.0.1:8000",
    model_name="intfloat/multilingual-e5-large",
)
emb = emb_model.get_text_embedding("hello world")
print(emb)

But if I use the arg model instead of model_name, It will raise:

ValueError: 'intfloat/multilingual-e5-large' is not a valid OpenAIEmbeddingModelType

I checked the code in llama_index/embeddings/openai/base.py,

class OpenAIEmbedding(BaseEmbedding):
    ...
    def __init__(
        ...
        self._query_engine = get_engine(mode, model, _QUERY_MODE_MODEL_DICT)
        self._text_engine = get_engine(mode, model, _TEXT_MODE_MODEL_DICT)

        if "model_name" in kwargs:
            model_name = kwargs.pop("model_name")
            self._query_engine = self._text_engine = model_name
        else:
            model_name = model

So I'm not sure if it is a good idea to set the model_name arg to use my custom OpenAI-compatible API? If not, is there a better way? If yes, maybe we need to add some documents?

Greetings!

Did you find the way to solve this issue?

Thank you in advance!

[juanluisrosaramos]

But this model is in hugginface no? if yes you need to load it with
from llama_index.embeddings.huggingface import HuggingFaceEmbedding

[Searcherr]

But this model is in hugginface no? if yes you need to load it with from llama_index.embeddings.huggingface import HuggingFaceEmbedding
Thank you so much!

What if I created OpenAI-like web application with huggingface models?
So I want to call this API with, for example OpenAIEmbedding class (or similar). But change base_url to my endpoint and model_name to one of the huggingface models?

Thank you in advance!

[david1542]

Did anyone find a solution to this problem?

[Searcherr]

Some kind of yes. Using OllamaEmbedding object instead:
https://docs.llamaindex.ai/en/stable/examples/embeddings/ollama_embedding/

[vikrantdeshpande09876]

You might wanna check out TextEmbed.
It seems to support OpenAI-compatible Embedding API format.
However, I noticed the source code needed a base-URL change from /embedding to /embeddings.
Ended up recreating TextEmbed as my own custom Embedding class.