ZeroDivisionError Problem with score = re_0_10_rating(line) / normalize in f_context_relevance ? #1096
Unanswered
AmineMekki01
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hello,
I am using Trulens for evaluating a Rag app.
`from trulens_eval.feedback.provider.openai import AzureOpenAI
from trulens_eval import Feedback, Select
provider = AzureOpenAI(deployment_name="gpt-4-32k")
f_context_relevance = (
Feedback(
provider.context_relevance_with_cot_reasons, name="Context Relevance"
)
.on(Select.RecordCalls.retrieve.args.query)
.on(Select.RecordCalls.retrieve.rets.collect())
.aggregate(np.mean)
)`
i am facing the following error which is weird :
Traceback (most recent call last): File "/opt/conda/envs/genai-model-eval/lib/python3.9/runpy.py", line 197, in _run_module_as_main return _run_code(code, main_globals, None, File "/opt/conda/envs/genai-model-eval/lib/python3.9/runpy.py", line 87, in _run_code exec(code, run_globals) File "/home/oneai/oneai-mns-genai-model_lib/misc/eval_run.py", line 17, in <module> evaluation_results = run_evaluation_pipeline( File "/home/oneai/oneai-mns-genai-model_lib/genair/genai/msat/evaluation/evaluation_pipeline.py", line 108, in run_evaluation_pipeline section_results = eval_obj.run_eval(use_ragas, use_trulens, True) File "/home/oneai/oneai-mns-genai-model_lib/genair/genai/msat/evaluation/evaluation_metrics.py", line 231, in run_eval trulens_results = self.trulens_metrics() File "/home/oneai/oneai-mns-genai-model_lib/genair/genai/msat/evaluation/evaluation_metrics.py", line 158, in trulens_metrics context_relevance = f_context_relevance( File "/opt/conda/envs/genai-model-eval/lib/python3.9/site-packages/trulens_eval/feedback/feedback.py", line 444, in __call__ return self.imp(*args, **kwargs) File "/opt/conda/envs/genai-model-eval/lib/python3.9/site-packages/trulens_eval/feedback/provider/base.py", line 392, in context_relevance_with_cot_reasons return self.generate_score_and_reasons(system_prompt, user_prompt, temperature) File "/opt/conda/envs/genai-model-eval/lib/python3.9/site-packages/trulens_eval/feedback/provider/base.py", line 253, in generate_score_and_reasons score = re_0_10_rating(line) / normalize ZeroDivisionError: float division by zero
looked into the repo and normalize is set to 10.0 by default. Why this error then ?
Beta Was this translation helpful? Give feedback.
All reactions