questions about evaluation

[ZhijieChung]

Thanks for your great work.
During the process of running the evaluation, I used your open-source 4B model.
The commands I used are python main.py --agent_config /memalpha-qwen3-4b_agent_0.05-0.1.yaml --dataset memoryagentbench After the run is completed, only the agents state and result.json and other information are obtained. How can the indicators such as the precise retrieval (accuracy rate), learning during testing (classification accuracy rate), and long-term understanding (F1 score) of memoryagentbench in the paper be obtained

[wangyu-ustc]

Oh you can use https://github.com/wangyu-ustc/Mem-alpha/blob/main/evaluate_agent_results.py
to run the evaluation. You need to pass in --base_dir to get the evaluation metrics for the results in the dir. base_dir should have the files result.json in it.