How to inject history using Live API?

[pelguetat]

Hi, I'm working on building a voice agent, and one of the main issues currently is the low function calling performance with the new native audio models.
Once solution I have considered is to run a second LLM agent in the background that ingests the chat history with a certain frequency and makes the function calls and then injects them into the voice agent events history.
I have the following code snippet that seems to work, in the sense that if asked, the agent will remember the injected information, but it doesn't acknowledge it unless asked, which fails my purpose. Ideally when a tool response is injected, it would act as if it had received the actual response from the backend:

async def inject_tool_response_from_client(
    live_request_queue: LiveRequestQueue,
    function_name: str,
    function_args: dict,
    function_result: dict,
):
    """
    Inject a tool response from client side to make server think
    it made the tool call. This uses BidiGenerateContentToolResponse.
    """
    try:
        logger.info(f"Injecting tool response for {function_name}: {function_result}")

        # Send as content with function role to simulate tool execution
        tool_id = str(uuid.uuid4())

        # Create function call content (what the model "thinks" it called)
        tool_content = types.Content(
            role="model",
            parts=[
                types.Part(
                    function_call=types.FunctionCall(
                        id=tool_id,
                        name=function_name,
                        args=function_args,
                    )
                )
            ],
        )

        # Create function response content (the "result" from the tool)
        tool_response = types.Content(
            role="model",
            parts=[
                types.Part(
                    function_response=types.FunctionResponse(
                        id=tool_id,
                        name=function_name,
                        response=function_result,
                    )
                )
            ],
        )
        system_message_update = types.Content(
            role="system",
            parts=[
                types.Part(
                    text="Se ha creado una cita con los siguientes parámetros: "
                    + json.dumps(function_args)
                )
            ],
        )
        # Send both to make the conversation think the model called the function
        live_request_queue.send_content(content=tool_content)
        live_request_queue.send_content(content=tool_response)
        live_request_queue.send_content(content=system_message_update)

        logger.info(f"Successfully injected tool call and response for {function_name}")

    except Exception as e:
        logger.error(f"Error injecting tool response: {e}")
        logger.error(f"Full traceback:\n{traceback.format_exc()}")

Do you have any suggestions on how to achieve this? Maybe modifying the agent memory in ADK directly instead of the live api events?

[hangfei]

@adk-bot