langchain-python example not working

[akshatdev0]

After going through all the steps metioned here

I ran command python test.py

And got the below output:

➜ python test.py
/home/aksh/.local/lib/python3.10/site-packages/langchain/llms/openai.py:171: UserWarning: You are trying to use a chat model. This way of initializing it is no longer supported. Instead, please use: `from langchain.chat_models import ChatOpenAI`
  warnings.warn(
/home/aksh/.local/lib/python3.10/site-packages/langchain/llms/openai.py:740: UserWarning: You are trying to use a chat model. This way of initializing it is no longer supported. Instead, please use: `from langchain.chat_models import ChatOpenAI`
  warnings.warn(
Retrying langchain.llms.openai.completion_with_retry.<locals>._completion_with_retry in 4.0 seconds as it raised APIError: could not load model - all backends returned error: 11 errors occurred:
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model

 {"error":{"code":500,"message":"could not load model - all backends returned error: 11 errors occurred:\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\n","type":""}} 500 {'error': {'code': 500, 'message': 'could not load model - all backends returned error: 11 errors occurred:\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\n', 'type': ''}} {'Date': 'Wed, 07 Jun 2023 09:53:09 GMT', 'Content-Type': 'application/json', 'Content-Length': '406'}.
Retrying langchain.llms.openai.completion_with_retry.<locals>._completion_with_retry in 4.0 seconds as it raised APIError: could not load model - all backends returned error: 11 errors occurred:
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model

 {"error":{"code":500,"message":"could not load model - all backends returned error: 11 errors occurred:\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\n","type":""}} 500 {'error': {'code': 500, 'message': 'could not load model - all backends returned error: 11 errors occurred:\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\n', 'type': ''}} {'Date': 'Wed, 07 Jun 2023 09:53:13 GMT', 'Content-Type': 'application/json', 'Content-Length': '406'}.
Retrying langchain.llms.openai.completion_with_retry.<locals>._completion_with_retry in 4.0 seconds as it raised APIError: could not load model - all backends returned error: 11 errors occurred:
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model

 {"error":{"code":500,"message":"could not load model - all backends returned error: 11 errors occurred:\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\n","type":""}} 500 {'error': {'code': 500, 'message': 'could not load model - all backends returned error: 11 errors occurred:\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\n', 'type': ''}} {'Date': 'Wed, 07 Jun 2023 09:53:17 GMT', 'Content-Type': 'application/json', 'Content-Length': '406'}.
Retrying langchain.llms.openai.completion_with_retry.<locals>._completion_with_retry in 8.0 seconds as it raised APIError: could not load model - all backends returned error: 11 errors occurred:
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model

 {"error":{"code":500,"message":"could not load model - all backends returned error: 11 errors occurred:\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\n","type":""}} 500 {'error': {'code': 500, 'message': 'could not load model - all backends returned error: 11 errors occurred:\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\n', 'type': ''}} {'Date': 'Wed, 07 Jun 2023 09:53:21 GMT', 'Content-Type': 'application/json', 'Content-Length': '406'}.
Retrying langchain.llms.openai.completion_with_retry.<locals>._completion_with_retry in 10.0 seconds as it raised APIError: could not load model - all backends returned error: 11 errors occurred:
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model

 {"error":{"code":500,"message":"could not load model - all backends returned error: 11 errors occurred:\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\n","type":""}} 500 {'error': {'code': 500, 'message': 'could not load model - all backends returned error: 11 errors occurred:\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\n', 'type': ''}} {'Date': 'Wed, 07 Jun 2023 09:53:29 GMT', 'Content-Type': 'application/json', 'Content-Length': '406'}.
Traceback (most recent call last):
  File "/home/aksh/development/LocalAI/examples/langchain-python/test.py", line 6, in <module>
    print(llm(text))
  File "/home/aksh/.local/lib/python3.10/site-packages/langchain/llms/base.py", line 310, in __call__
    self.generate([prompt], stop=stop, callbacks=callbacks)
  File "/home/aksh/.local/lib/python3.10/site-packages/langchain/llms/base.py", line 192, in generate
    raise e
  File "/home/aksh/.local/lib/python3.10/site-packages/langchain/llms/base.py", line 186, in generate
    self._generate(prompts, stop=stop, run_manager=run_manager)
  File "/home/aksh/.local/lib/python3.10/site-packages/langchain/llms/openai.py", line 791, in _generate
    full_response = completion_with_retry(self, messages=messages, **params)
  File "/home/aksh/.local/lib/python3.10/site-packages/langchain/llms/openai.py", line 106, in completion_with_retry
    return _completion_with_retry(**kwargs)
  File "/home/aksh/.local/lib/python3.10/site-packages/tenacity/__init__.py", line 289, in wrapped_f
    return self(f, *args, **kw)
  File "/home/aksh/.local/lib/python3.10/site-packages/tenacity/__init__.py", line 379, in __call__
    do = self.iter(retry_state=retry_state)
  File "/home/aksh/.local/lib/python3.10/site-packages/tenacity/__init__.py", line 325, in iter
    raise retry_exc.reraise()
  File "/home/aksh/.local/lib/python3.10/site-packages/tenacity/__init__.py", line 158, in reraise
    raise self.last_attempt.result()
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 451, in result
    return self.__get_result()
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/home/aksh/.local/lib/python3.10/site-packages/tenacity/__init__.py", line 382, in __call__
    result = fn(*args, **kwargs)
  File "/home/aksh/.local/lib/python3.10/site-packages/langchain/llms/openai.py", line 104, in _completion_with_retry
    return llm.client.create(**kwargs)
  File "/home/aksh/.local/lib/python3.10/site-packages/openai/api_resources/chat_completion.py", line 25, in create
    return super().create(*args, **kwargs)
  File "/home/aksh/.local/lib/python3.10/site-packages/openai/api_resources/abstract/engine_api_resource.py", line 153, in create
    response, _, api_key = requestor.request(
  File "/home/aksh/.local/lib/python3.10/site-packages/openai/api_requestor.py", line 298, in request
    resp, got_stream = self._interpret_response(result, stream)
  File "/home/aksh/.local/lib/python3.10/site-packages/openai/api_requestor.py", line 700, in _interpret_response
    self._interpret_response_line(
  File "/home/aksh/.local/lib/python3.10/site-packages/openai/api_requestor.py", line 763, in _interpret_response_line
    raise self.handle_error_response(
openai.error.APIError: could not load model - all backends returned error: 11 errors occurred:
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model
        * failed loading model

 {"error":{"code":500,"message":"could not load model - all backends returned error: 11 errors occurred:\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\n","type":""}} 500 {'error': {'code': 500, 'message': 'could not load model - all backends returned error: 11 errors occurred:\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\n', 'type': ''}} {'Date': 'Wed, 07 Jun 2023 09:53:39 GMT', 'Content-Type': 'application/json', 'Content-Length': '406'}

[mudler]

hey @akshatdev0

good catch, the example was incomplete. I've updated it now and simplified it - can you pull and try again? I've updated the steps in the README as well

[akshat-io]

Yes, it is working now.