Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1_basic_agent.py produces inconsistent results #2

Open
analytic-garden opened this issue Nov 7, 2024 · 1 comment
Open

1_basic_agent.py produces inconsistent results #2

analytic-garden opened this issue Nov 7, 2024 · 1 comment

Comments

@analytic-garden
Copy link

1_basic_agent.py from the tutorial produces inconsistent results when run multiple times. Should the path through the graph be consistent?

The query was: "What is 20+(2*4)? Use a tool to calculate every step."

$ python 1_basic_agent.py

Running step b19cd878-82e8-43d3-9e42-ccb3fa0f29a0. Step input: What is 20+(2*4)? Use a tool to calculate every step.
Thought: The current language of the user is English. I need to use the add and multiply tools to help me calculate the expression step by step.
Action: add
Action Input: {'a': 2, 'b': 4}
Observation: 6
Running step ee5f48f2-f76a-42d1-9fb7-58ee173d7c9e. Step input: None
Thought: (Implicit) I can answer without any more tools!
Answer: Action: multiply
Action Input: {"a": 2, "b": 4}
Action: multiply
Action Input: {"a": 2, "b": 4}

$ python 1_basic_agent.py

Running step 1249e3f7-485f-4818-9578-09bde5a90dfb. Step input: What is 20+(2*4)? Use a tool to calculate every step.
Thought: The current language of the user is English. I need to use tools to help me calculate the expression step by step.
Action: multiply
Action Input: {'a': 2, 'b': 4}
Observation: 8
Running step e50dc969-faf8-4082-9afe-4607ea98b971. Step input: None
Thought: (Implicit) I can answer without any more tools!
Answer: Action: add
Action Input: {"a": 20, "b": 8}
Action: add
Action Input: {"a": 20, "b": 8}

The first example is incorrect. The first action that example should have been multiply.

@analytic-garden
Copy link
Author

The example also doesn't print the full response:

...
Observation: 28
Thought: I can answer without using any more tools. I'll use the user's language to answer
Answer: The result of 20 + (2 * 4) is 28.
The result of 20 + (2 * 4) is 28.

as shown here: https://docs.llamaindex.ai/en/stable/understanding/agent/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant