feat(tuner): Enhance Agent Tune Interface

[pan-x-c]

AgentScope Version

1.0.11

Description

Enhance the current tune related modules.

flowchart TD
    Model[Model] --> WorkflowFunction[Workflow Function]
    WorkflowFunction --> JudgeFunction[Judge Function]
    Task[Task] --> WorkflowFunction
    Task[Task] --> JudgeFunction
    JudgeFunction --> Reward[Reward]

    classDef wfcolor fill:#e67e22,stroke:#333,color:#111;
    classDef judgecolor fill:#1abc9c,stroke:#333,color:#111,stroke-dasharray: 5 5;
    classDef taskcolor fill:#3498db,stroke:#333,color:#111;
    class WorkflowFunction wfcolor;
    class JudgeFunction judgecolor;
    class Task taskcolor;

Loading

Enhancements include:

1. Pass auxiliary_models (Dict[str, BaseChatModel]) to workflow_function to support the use of different models in multi-agent applications.
2. workflow_function can return raw response with any type.
3. Add judge_function to calculate rewards based on the raw response returned by workflow_function, and the judge_function can use auxiliary_models (Dict[BaseChatModel]) to implement LLM-as-a-judge.
4. Simplify configuration file, use Dataset, TunerChatModel and Algorithm to construct the tuning configuration.

Checklist

Please check the following items before code is ready to be reviewed.

• Code has been formatted with pre-commit run --all-files command
• All tests are passing
• Docstrings are in Google style
• Related documentation has been updated (e.g. links, examples, etc.)
• Code is ready for review

[cla-assistant]

All committers have signed the CLA.

[cla-assistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[yanxi-chen]

Some minor comments

[Copilot]

Pull request overview

Copilot reviewed 29 out of 30 changed files in this pull request and generated 19 comments.

[DavdGao]

Please see my inline comments. I have some concerns about whether abstractions like Algorithm actually simplify RL training configuration. In my view, consolidating all tuning parameters in a single JSON file would be more straightforward and easier to manage.

[lingzhq]

To support the data-augmentation example in AgentScope-Tuner, we need to pass the Task Selector parameters to Trinity and fix an issue with loading eval tasksets via YAML. The corresponding implementation can be checked in the samples PR. Thanks for looking into this!

[DavdGao]

plz see inline comments

[DavdGao]

lgtm