QAgent: A modular Search Agent with Interactive Query Understanding

🌞 Introduction

QAgent, a unified agentic RAG framework that employs a search agent for adaptive retrieval. This agent optimizes its understanding of the query through interactive reasoning and retrieval. To facilitate real-world application, we focus on modular search agent for query understanding that are plug-and-play in complex systems. Secifically, the agent follows a multi-step decision process trained with RL to maximize retrieval quality and support accurate downstream answers. We further analyze the strengths and weaknesses of end-to-end RL and propose a strategy that focuses on effective information retrieval, thereby enhancing generalization in LLM applications.

Framework

Key Features: (1) System-friendly; (2) Query Understanding

💡 Preparation

Download Corpus & Index & retrievers

Details will be completed soon

📑 Code Architecture

.
├── README.md
├── assets        
├── config.py
├── grpo_loss.py
├── main_grpo_v0.py         # Training start, trl
├── main_grpo_v1.py         # Training started, using LigerKernel optimization
├── refer_llm               # Reference Model service
│   ├── __init__.py
│   ├── refer_client.py
│   ├── refer_server.py
│   └── tensor_utils.py
├── requirements.txt
├── retrieval               
│   ├── retrieval_bm25.py  
│   └── retrieval_e5.py    
├── rewards                
│   ├── __init__.py
│   └── reward_QAgent.py
├── rollout                
│   ├── __init__.py
│   ├── base_rollout.py
│   └── rollout_QAgent.py
├── run.py
├── run.sh
└── tools.py                # Encapsulate tool functions, such as search requests

🎯 Run Training

Training Configuration

Details will be completed soon

Run Training

Details will be completed soon

📈 Run Evaluation

Prepare Evaluation Data

Details will be completed soon

Run Evaluation

Details will be completed soon

📊 Performance

Main results of end-to-end performance

Main results when used as a submodule.

🔬 Analysis

Key Features: (1)Information utilization; (2) Different retriever; (3)Combined Gain

Limitation

• Training on larger models and practical scenarios.
• Failure to control passage diversity.

Acknowledge

We sincerely appreciate the efforts of these teams for their contributions to open-source research and development: Search-R1, LigerKernel, TRL, vLLM, Simple-GRPO.

Citation

@article{jiang2025qagent,
  title={QAgent: A modular Search Agent with Interactive Query Understanding},
  author={Jiang, Yi and Shen, Lei and Niu, Lujie and Zhao, Sendong and Su, Wenbo and Zheng, Bo},
  journal={arXiv preprint arXiv:2510.08383},
  year={2025}
}

Thanks for your interest in our work!