Using Reinforcement Learning to Guide Chains of Thought

• Uses Hugging Face TRL for PPO
• Uses Hugging Face Peft for LoRA.
• Uses Bitsandbytes internally for 4bits and 8bits reference model modes.
• Uses our QLora standalone lib for QLora.

With TRL:

Where the reinforcement learning is located.

Approach SFT:

There, one finds the supervised baselines:

• Generate, then learn, masked.

Launch all SFT jobs with

cd approach_sft
./queue_all_jobs.sh