How do I load back the tool call and its outputs if I am locally managing the history ?

[HailTheLotusJewel]

Please read this first

• Have you read the docs?Agents SDK docs: yes
• Have you searched for related issues? Others may have had similar requests: yes

Question

Example case:

from agents import Agent, Runner, ToolOutputImage, function_tool
import asyncio
import json
import dotenv
dotenv.load_dotenv()

def save_to_file(filename, json_list):
    with open(filename, "w") as json_file:
        json.dump(json_list, json_file, indent=2)

@function_tool
def fetch_multiple_images() -> list[ToolOutputImage]:
    """Returns two different images for the AI to compare."""
    
    # Image 1: Golden Gate Bridge
    img1 = ToolOutputImage(
        image_url="https://foyrai-pixi.s3.amazonaws.com/outputs/720856df-9457-42b6-85cf-d7fcbb255fa24e42ed4e-5aac-4771-9808-c925fef7d661.webp",
        detail="auto"
    )
    
    # Image 2: Eiffel Tower
    img2 = ToolOutputImage(
        image_url="https://foyrai-pixi.s3.amazonaws.com/outputs/55fcc325-7e26-4fb8-989e-b596a7fd9fb73c1d54cf-2c7b-4b99-a72c-174326ef1f6a.webp",
        detail="auto"
    )

    # Return them as a list
    return [img1, img2]

async def main():
    agent = Agent(
        name="Assistant",
        instructions="You are a helpful assistant.",
        tools=[fetch_multiple_images],
    )

    result = await Runner.run(
        agent,
        input="hi, what can you do ? and also fetch me some images.",
    )
    
    print(result.final_output)
    print(result.raw_responses)
    history = result.to_input_list()
    save_to_file("history.json", history)

if __name__ == "__main__":
    asyncio.run(main())

Suppose I get this tool call history from the above file:

[...{
    "arguments": "{}",
    "call_id": "call_f2DRrPCketzMFeqbMouSc4Ha",
    "name": "fetch_multiple_images",
    "type": "function_call",
    "id": "fc_026f0126d3a925fc006957b38e93248193bffc6e2f10c6ea93",
    "status": "completed"
  },
  {
    "call_id": "call_f2DRrPCketzMFeqbMouSc4Ha",
    "output": [
      {
        "type": "input_image",
        "image_url": "https://foyrai-pixi.s3.amazonaws.com/outputs/720856df-9457-42b6-85cf-d7fcbb255fa24e42ed4e-5aac-4771-9808-c925fef7d661.webp",
        "detail": "auto"
      },
      {
        "type": "input_image",
        "image_url": "https://foyrai-pixi.s3.amazonaws.com/outputs/55fcc325-7e26-4fb8-989e-b596a7fd9fb73c1d54cf-2c7b-4b99-a72c-174326ef1f6a.webp",
        "detail": "auto"
      }
    ],
    "type": "function_call_output"
  }...]

Actual case:
Error:
Error code: 400 - {'error': {'message': "Invalid value: 'tool'. Supported values are: 'assistant', 'system', 'developer', and 'user'.", 'type': 'invalid_request_error', 'param': 'input[2]', 'code': 'invalid_value'}}

History:

[...{
      "role": "assistant",
      "content": "",
      "tool_calls": [
        {
          "id": "call_FsM8hSMoCoI7ekijMm9Yikyk",
          "type": "function",
          "function": {
            "name": "fetch_services",
            "arguments": "{args}"
          },
          "status": "completed"
        }
      ]
    },
    {
      "role": "tool",
      "tool_call_id": "call_FsM8hSMoCoI7ekijMm9Yikyk",
      "content": "stringified json output",
      "name": "fetch_services"
    },...]

My question is how do I properly store the tool output and load it back, so that I can seamlessly run this in a multi turn setting.

[ihower]

@HailTheLotusJewel Using to_input_list() is the correct and intended approach.

You should persist the history using result.to_input_list(), then load it back unchanged and simply append the next user message. For example, your second turn can be written like this:

    with open("history.json", "r") as json_file:
        chat_items = json.load(json_file)
   
    chat_items.append({"role": "user", "content": "What is the difference between the two images?"})

    result = await Runner.run(
        agent,
        input=chat_items,
    )

    print(result.final_output)

[HailTheLotusJewel]

@ihower I was trying to manipulate the history and remove some of the redundant fields that may not be mandatory. Anyways, thanks a ton for the response!

[HailTheLotusJewel]

@ihower Thanks for the previous information But I ran into an issue. Consider a two agent scenario where handoff happens in between the run:

async def main():
    agent_two = Agent(
        name="Agent Two",
        handoff_description="Pass control to Agent Two to finish the run.",
        instructions="Call your only tool to finish the request, then end the run cleanly.",
        tools=[finisher_tool],
    )

    agent_one = Agent(
        name="Agent One",
        # handoff_description=,
        instructions=(
            "Use your single tool to gather the starter response, then hand off to Agent Two "
            "so they can finish."
        ),
        tools=[starter_tool],
        handoffs=[handoff(agent=agent_two)],
    )

    input_items = [
        {"role": "user", "content": "Kick off the two-agent workflow."},
    ]

    result = await Runner.run(agent_one, input_items)
    print("Final output:", result.final_output)

    history = result.to_input_list()
    save_history("two_agent_history.json", history)

This is the raw history of this run:

[
  {
    "role": "assistant",
    "content": "For context, here is the conversation so far between the user and the previous agent:\n<CONVERSATION HISTORY>\n1. user: Kick off the two-agent workflow.\n2. function_call: {\"arguments\": \"{}\", \"call_id\": \"call_4aRTBpoGzHfMwFzQXT8W2jFC\", \"name\": \"starter_tool\", \"id\": \"fc_0cb86b419925643100695acd645bbc8194b69e50c876313b97\", \"status\": \"completed\"}\n3. function_call_output: {\"call_id\": \"call_4aRTBpoGzHfMwFzQXT8W2jFC\", \"output\": \"Agent One's tool ran and prepared the handoff note.\"}\n4. function_call: {\"arguments\": \"{}\", \"call_id\": \"call_u533nEOSzFSsjJ8nnEqTr09p\", \"name\": \"transfer_to_agent_two\", \"id\": \"fc_0cb86b419925643100695acd6529508194b706a3283855e29b\", \"status\": \"completed\"}\n5. function_call_output: {\"call_id\": \"call_u533nEOSzFSsjJ8nnEqTr09p\", \"output\": \"{\\\"assistant\\\": \\\"Agent Two\\\"}\"}\n</CONVERSATION HISTORY>"
  },
  {
    "arguments": "{}",
    "call_id": "call_4aRTBpoGzHfMwFzQXT8W2jFC",
    "name": "starter_tool",
    "type": "function_call",
    "id": "fc_0cb86b419925643100695acd645bbc8194b69e50c876313b97",
    "status": "completed"
  },
  {
    "call_id": "call_4aRTBpoGzHfMwFzQXT8W2jFC",
    "output": "Agent One's tool ran and prepared the handoff note.",
    "type": "function_call_output"
  },
  {
    "arguments": "{}",
    "call_id": "call_u533nEOSzFSsjJ8nnEqTr09p",
    "name": "transfer_to_agent_two",
    "type": "function_call",
    "id": "fc_0cb86b419925643100695acd6529508194b706a3283855e29b",
    "status": "completed"
  },
  {
    "call_id": "call_u533nEOSzFSsjJ8nnEqTr09p",
    "output": "{\"assistant\": \"Agent Two\"}",
    "type": "function_call_output"
  },
  {
    "arguments": "{}",
    "call_id": "call_LQp6Vo0xMjnNQTNArZkrzGrv",
    "name": "finisher_tool",
    "type": "function_call",
    "id": "fc_0cb86b419925643100695acd660b8c819482c9ccefc5051ba3",
    "status": "completed"
  },
  {
    "call_id": "call_LQp6Vo0xMjnNQTNArZkrzGrv",
    "output": "Agent Two's tool ran and wrapped up the workflow.",
    "type": "function_call_output"
  },
  {
    "id": "msg_0cb86b419925643100695acd66c5108194ba18a7865584484c",
    "content": [
      {
        "annotations": [],
        "text": "The two-agent workflow has successfully completed. Agent One initiated the process and prepared the handoff note, and Agent Two received the handoff and wrapped up the workflow. If you need to run another workflow or have any questions, let me know!",
        "type": "output_text",
        "logprobs": []
      }
    ],
    "role": "assistant",
    "status": "completed",
    "type": "message"
  }
]

If you see over here, the first assistant message is something which I guess is SDK/Handoff specific behaviour. What if I want the normal input list which just lists all the actions taken by the agent(s) ? Is there a separate inbuilt method which supports normal agent run history ?