Clarification about memory baseline configurations (Mem0, MemP, SimpleMem)

[claw-opus]

Hello, thank you for releasing this interesting work and the accompanying code.

I am trying to better understand the exact configurations used for the memory-based baselines in the paper, especially for reproducibility. I would greatly appreciate some clarification regarding the following details.

Mem0

1. Which embedding model was used for memory retrieval?
2. Which model was used for fact extraction / memory writing?

MemP

1. Which LLM was used for the procedural memory construction (e.g., trajectory distillation / reflection)?
2. Was the same model used for build / update / reflection, or were different models used for different stages?
3. Which embedding model was used for retrieval?

SimpleMem

1. Which LLM model was used for memory construction / compression?
2. Which embedding model was used for retrieval?

Trajectories used for memory construction

For the trajectories used to generate memories in these baselines:

• Were they generated by the teacher model (o3) or by the policy model (Qwen2.5-7B-Instruct)?
• Approximately how many trajectories were used to construct the memory entries?

Understanding these details would help reproduce the experimental setup more faithfully.

Thank you very much for your help!

[yanan1116]

the paper says that OpenAI o3 (OpenAI, 2025a) as the teacher model for skill distillation and SFT data generation
also see here in skill_generation/webshop.py class OpenAIClient : o3 model is used to distill skills from memories json files:

memory_data/webshop/generated_memories_webshop_100.json 
memory_data/webshop/generated_memories_webshop_101-200.json 

---

however,

• it seems that there are no trajectories provided in this repo which is used for memory construction or sft, under the task of webshop.
• and there are no scripts with regard to the implementations of baselines, such as Prompt-based Agentic or Memory-based Methods or Memory-Augmented RL-based Methods

@richard-peng-xia could you please provide some clarifications on these frustrations, thanks

[richard-peng-xia]

Hi @claw-opus,

Thank you for your interest in our work! We appreciate you taking the time to reproduce the baselines.

Here are the exact configurations we used for the memory-based baselines to help clarify the experimental setup:

1. Mem0

• Embedding model for memory retrieval: text-embedding-3-small

• Model: Qwen2.5-7B-Instruct

2. MemP

• LLM for procedural memory construction: Qwen2.5-7B-Instruct

• Model consistency across stages: Yes, we used the exact same model for all stages, including build, update, and reflection.

• Embedding model for retrieval: text-embedding-3-small

3. SimpleMem

• LLM for memory construction / compression: Qwen2.5-7B-Instruct

• Embedding model for retrieval: sentence-transformers/all-MiniLM-L6-v2

4. Trajectories used for memory construction

• Source of trajectories: For the cold start, we used distilled trajectories generated by the teacher model.

• Quantity: We used exactly 100 trajectories to construct the initial memory entries.

I hope this helps you reproduce the setup more faithfully. Please feel free to let me know if you run into any other questions or need further clarification on the code!