Error using rerank twice

[knaga35]

Self Checks

• This is only for bug report, if you would like to ask a question, please head to Discussions.
• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• Please do not modify this template :) and fill in all the required fields.

Dify version

0.11.0

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

1. Create knowledge

　Retrieval Setting : Vector Search
　Rerank Model : BAAI/bge-reranker-v2-m3

2. Integrate knowledge base within application

　Create a chatflow as below.
　KNOWLEDGE RETRIEVAL:
　　Retrieval Setting : Rerank Model
　　Rerank Model : BAAI/bge-reranker-v2-m3

3. Enter content in the box to start debugging the Chatbot

Depending on the content of the question, there is no problem, but the following error may occur.
Dify API Log:

2025-03-06T15:27:05+09:00	stderr	2025-03-06 06:27:05,961.961 ERROR [Thread-327 (_generate_worker)] [knowledge_retrieval_node.py:65] - Error when running knowledge retrieval node
2025-03-06T15:27:05+09:00	stderr	Traceback (most recent call last):
2025-03-06T15:27:05+09:00	stderr	  File "/app/api/core/model_runtime/model_providers/__base/rerank_model.py", line 42, in invoke
2025-03-06T15:27:05+09:00	stderr	    return self._invoke(model, credentials, query, docs, score_threshold, top_n, user)
2025-03-06T15:27:05+09:00	stderr	  File "/app/api/core/model_runtime/model_providers/localai/rerank/rerank.py", line 68, in _invoke
2025-03-06T15:27:05+09:00	stderr	    response.raise_for_status()
2025-03-06T15:27:05+09:00	stderr	  File "/app/api/.venv/lib/python3.10/site-packages/requests/models.py", line 1021, in raise_for_status
2025-03-06T15:27:05+09:00	stderr	    raise HTTPError(http_error_msg, response=self)
2025-03-06T15:27:05+09:00	stderr	requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http://localai:8080/v1/rerank
2025-03-06T15:27:05+09:00	stderr	
2025-03-06T15:27:05+09:00	stderr	During handling of the above exception, another exception occurred:
2025-03-06T15:27:05+09:00	stderr	
2025-03-06T15:27:05+09:00	stderr	Traceback (most recent call last):
2025-03-06T15:27:05+09:00	stderr	  File "/app/api/core/workflow/nodes/knowledge_retrieval/knowledge_retrieval_node.py", line 58, in _run
2025-03-06T15:27:05+09:00	stderr	    results = self._fetch_dataset_retriever(node_data=self.node_data, query=query)
2025-03-06T15:27:05+09:00	stderr	  File "/app/api/core/workflow/nodes/knowledge_retrieval/knowledge_retrieval_node.py", line 155, in _fetch_dataset_retriever
2025-03-06T15:27:05+09:00	stderr	    all_documents = dataset_retrieval.multiple_retrieve(
2025-03-06T15:27:05+09:00	stderr	  File "/app/api/core/rag/retrieval/dataset_retrieval.py", line 420, in multiple_retrieve
2025-03-06T15:27:05+09:00	stderr	    all_documents = data_post_processor.invoke(
2025-03-06T15:27:05+09:00	stderr	  File "/app/api/core/rag/data_post_processor/data_post_processor.py", line 37, in invoke
2025-03-06T15:27:05+09:00	stderr	    documents = self.rerank_runner.run(query, documents, score_threshold, top_n, user)
2025-03-06T15:27:05+09:00	stderr	  File "/app/api/core/rag/rerank/rerank_model.py", line 44, in run
2025-03-06T15:27:05+09:00	stderr	    rerank_result = self.rerank_model_instance.invoke_rerank(
2025-03-06T15:27:05+09:00	stderr	  File "/app/api/core/model_manager.py", line 226, in invoke_rerank
2025-03-06T15:27:05+09:00	stderr	    return self._round_robin_invoke(
2025-03-06T15:27:05+09:00	stderr	  File "/app/api/core/model_manager.py", line 310, in _round_robin_invoke
2025-03-06T15:27:05+09:00	stderr	    return function(*args, **kwargs)
2025-03-06T15:27:05+09:00	stderr	  File "/app/api/core/model_runtime/model_providers/__base/rerank_model.py", line 44, in invoke
2025-03-06T15:27:05+09:00	stderr	    raise self._transform_invoke_error(e)
2025-03-06T15:27:05+09:00	stderr	core.model_runtime.errors.invoke.InvokeError: [localai] Error: 500 Server Error: Internal Server Error for url: http://localai:8080/v1/rerank

✔️ Expected Behavior

No response

❌ Actual Behavior

No response

[dosubot]

Hello @knaga35! 👋 I'm here to help you with bug fixes, answer questions, and support you as you wait for a human maintainer.

To address the "500 Server Error: Internal Server Error" when using the rerank model BAAI/bge-reranker-v2-m3 in a self-hosted Dify setup, you should check the logs of the 'api' container. Since the setup is using Docker, you can view these logs using the following command:

docker-compose logs api

This will help you diagnose the issue by providing more detailed error messages or stack traces that can indicate the root cause of the server error ^[1].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}