llama factory

redhat/ocp4/4.16/2024.12.llama.factory.md

@@ -4,16 +4,24 @@
 
 Customer wants to try running [llama factory](https://github.com/hiyouga/LLaMA-Factory) on OpenShift AI. We know that we can run llama factory on openshift using deployment, but this requires additional configuration like image building and deployment configuration, like setting start up configuration parameters. The most difficult part is to get the master/head's ip address, and set to worker task's parameter to ensure that the workers can communicate with the master node.
 
+> [!NOTE]
+> In recently [commits](https://github.com/hiyouga/LLaMA-Factory/issues/6543), llama factory support ray, but with some limitation.
+
 In this artical, we try to set up the openshift ai env, and start a llama factory distribution training task involving deepspeed.
 
 The logic diagram illustrating the process flow for setting up and running LLaMA Factory on OpenShift AI:
 
 ![](./dia/ocp.ai.llama.factory.01.drawio.png)
 
+The logic diagram illustrating the process flow for setting up and running LLaMA Factory on OpenShift AI with Ray:
+
+![](./dia/ocp.ai.llama.factory.ray.drawio.png)
+
 We will use such source code, listed here:
 - https://github.com/wangzheng422/LLaMA-Factory/tree/wzh-stable/wzh
 - [python notebook to run llama factory in python venv mode](./files/ray.notebook/2.5_run.llama.factory.ipynb)
 - [python notebook to run llama factory in python direct mode (with deepspeed)](./files/ray.notebook/2.5_run.llama.factory.deepspeed.ipynb)
+- [python notebook to run llama factory with build-in ray support](./files/ray.notebook/2.5_run.llama.factory.ray.ipynb)
 
 # run using llama factory offical way