Quick start documentation on example is not working

[eugeniosegala]

Hello,

I'm creating an application following the quick start section: https://www.trulens.org/trulens_eval/getting_started/quickstarts/quickstart/

However, there seems to be a problem with some depedencies:

Traceback (most recent call last):
  File "trulens/app.py", line 40, in <module>
    from trulens_eval import Tru
  File "trulens/.venv/lib/python3.12/site-packages/trulens_eval/__init__.py", line 96, in <module>
    from trulens_eval.tru_llama import TruLlama
  File "trulens/.venv/lib/python3.12/site-packages/trulens_eval/tru_llama.py", line 19, in <module>
    from trulens_eval.utils.llama import WithFeedbackFilterNodes
  File "trulens/.venv/lib/python3.12/site-packages/trulens_eval/utils/llama.py", line 117, in <module>
    class WithFeedbackFilterNodes(VectorIndexRetriever):
  File "trulens/.venv/lib/python3.12/site-packages/trulens_eval/utils/imports.py", line 46, in __getattr__
    raise ModuleNotFoundError(self.message)
ModuleNotFoundError: llama_index 0.8.29post1 or above is required for instrumenting llama_index apps. Please install it before use: `pip install llama_index>=0.8.29post1`.

I do have a version of llama_index greater than 0.8.29. Even when I install exactly v0.8.29, then LangChain will fail.

I don't seem to be able to comprehend the nature of these errors as the "quick start" tutorial is not based either on llama_index or LangChain.

Can anyone help me? Perhaps the documentation is out of date?

[dosubot[bot]]

Based on the error message you provided, it seems like the llama_index module is not being found, despite you having a version greater than 0.8.29. This could be due to a few reasons:

1. Environment Mismatch: The Python environment in which you're running the script might not have llama_index installed. This can happen if you have multiple Python environments and the script is executed in an environment different from where llama_index was installed.

2. Incorrect Installation: The package might not have been installed correctly. This could be due to network issues during installation, permissions problems, or conflicts with other installed packages.

3. Path Issues: Python might not be able to locate the llama_index module if it's installed in a location that's not included in Python's path. This can happen if the package is installed in a non-standard directory or if the Python path is not set up correctly.

To resolve the issue, you can try the following steps:

• Verify Environment: Ensure that you're working in the correct Python environment. You can check the current environment and installed packages using commands like which python or pip list.

• Reinstall Package: Try reinstalling llama_index using pip install --upgrade --force-reinstall llama_index. This can help fix any issues with a previous installation.

• Check Python Path: Ensure that Python's path includes the directory where llama_index is installed. You can check the Python path by printing sys.path in your script.

• Dependencies: Ensure all dependencies required by llama_index are installed and compatible. Sometimes, module not found errors can be indirectly caused by missing or incompatible dependencies.

If these steps do not resolve the issue, it might be helpful to create a fresh Python environment and reinstall the required packages to ensure a clean setup.

Regarding the version of llama_index that is compatible with TruLens Eval v0.25.1, any version newer than or equal to 0.10.0 should be compatible. This is inferred from the condition that sets the legacy flag to True if the llama_index version is below 0.10.0, indicating that versions below 0.10.0 are considered legacy and potentially incompatible with the newer features expected by TruLens Eval v0.25.1. You can find this in the tru_llama.py file.

I hope this helps! If you have any other questions or if the issue persists, please provide more details so I can assist you better.

Sources

• import error
• trulens_eval/examples/quickstart/py_script_quickstarts/llama_index_quickstart.py
• trulens_eval/examples/expositional/frameworks/llama_index/llama_index_agents.ipynb
• trulens_eval/trulens_eval/tru_llama.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[joshreini1]

Thanks for reaching out @eugeniosegala - these sorts of dependency issues are often caused by older installs existing in your python environment. Can you try installing the latest trulens_eval in a fresh python environment and let me know if this issue still persists?

[eugeniosegala]

Hello @joshreini1, thanks for the help!

Unfortunately I have tried with a fresh new Python virtual env (v3.12) and the error still occur!

[joshreini1]

Happy to help @eugeniosegala - we don't support python 3.12 yet, can you try on 3.11 or earlier?

[eugeniosegala]

I tried with Python 3.9.6 and it worked! 🥳

Thank you for the help.

Something I noticed is that I had to change:

# Question/statement relevance between question and each context chunk.
f_context_relevance = (
-    Feedback(provider.context_relevance_with_cot_reasons, name = "Context Relevance")
+    Feedback(provider.relevance_with_cot_reasons, name="Context Relevance")
    .on(Select.RecordCalls.retrieve.args.query)
    .on(Select.RecordCalls.retrieve.rets.collect())
    .aggregate(np.mean)
)

As I was getting a different error but I'm assuming that context_relevance_with_cot_reasons was removed/changed from the latest version of trulens_eval.

[joshreini1]

Glad it worked!

Curious how you were using context_relevance_with_cot_reasons before, we're newly renaming qs_relevance_with_cot_reasons to this name in the upcoming 0.26.0 release.

[eugeniosegala]

Regarding your question, I copied the code from the doc: https://www.trulens.org/trulens_eval/getting_started/quickstarts/quickstart/#set-up-feedback-functions