😷 Refactor GRPO/RLOO to isolate _generate

[qgallouedec]

This PR belongs to a sequence of PR that aims to refactor the generation part of GRPO/RLOO to allow for easier customization and ultimately tool calling

Previous:

• 🟩 Drop image_split_sizes in favour of image_grid_thw #4111
• 📽 Multi image support for GRPO/RLOO #4113

Next:

• 🧺 [1/N] Refactor _generate in GRPO/RLOO: list of ints instead of tensors #4146
• 🧺 [2/N] Refactor _generate in GRPO/RLOO: Use prompt_ids from generation #4152
• 🧺 [3/N] Refactor _generate in GRPO/RLOO: Rely on generator for prompt truncation #4153
• 🧺 [4/N] Refactor _generate in GRPO/RLOO: Move forward_kwargs outside generation method #4154
• 🧺 [5/N] Refactor _generate in GRPO/RLOO: Insert images in the prompt #4155

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[qgallouedec]

cc @Peter-Chou, customization is made easier with this one

[lewtun]

LGTM once the tests pass and a question about whether we should split _generate from scoring entirely

[Peter-Chou]

@qgallouedec Yes. The original _generate_and_score_completions method was way too lengthy.
Breaking it down into finer-grained sub-methods and chaining them together like this is an excellent approach!