How to prepare dataset to train a chatbot llm.

[WeiXiaoSummer]

Hi everyone!

Newbie here, perhaps the question sounds very stupid, so let's assume I have a dataset prepared for a chatbot that contains function calling, how should I feed this dataset to the LLM? What I mean is, as the conversation has multiple rounds, should we generate an individual training sample from the beginning to each round or should we have only one training sample for each conversation, where the input is the full conversation except the last LLM response, and the output is the last LLM response.

Thank you in advance!

[NanoCode012]

You should do the former! Each individual's message being one. You can checkout the sharegpt or vicuna datasets to see examples for this!