Add 4-bit quantization and QLoRA

[jeff52415]

#486

• Add 4-bit Quantization in Training/Inference Script.
• Add QLoRA Optimizer.
• Dependency Update: I've updated our dependencies to their latest stable versions to ensure optimal functionality and security. Please see the updated requirements.txt file for the specific changes.
• Readme Update: The Readme file has been updated to reflect these changes, and to provide clearer instructions for users and contributors.

Please review the changes at your earliest convenience. Any feedback or suggestions are welcome.

Best Regards,
Jeff Yang

[tloen]

Noting down here: while I'm not actively maintaining this repo, this is one of the PRs I'd like to get tested and merged in the next day or two.

[jeff52415]

@tloen , if you require any assistance or support, please do not hesitate to inform me.

[shaonianyr]

@jeff52415 i wonder 4-bit train speed is the same as 8-bit or slower?

[jeff52415]

@jeff52415 i wonder 4-bit train speed is the same as 8-bit or slower?

While I haven't conducted formal benchmarking to confirm this, my empirical evidence suggests a higher speed for 4-bit training.

[chand1012]

Is this going to get merged? I found it quite helpful!