Deploy LLM on sagemaker

使用 LMI 容器在 Sagemker 部署模型做推理

相关文档

DLC上的LMI容器列表，请点击这里。

对于可用的BYOC容器列表, 请点击这里。

有关SageMaker上的LMI文档的更多信息,请点击这里。

对于您可以在DJLServing上设置的所有serving.properties选项,请点击这里。

不同部署方式推理时对应的 input/output schema,请点击这里。

Name		Name	Last commit message	Last commit date
Latest commit History 17 Commits
model_tokenizer		model_tokenizer
utils		utils
.gitignore		.gitignore
README.md		README.md
tensorrt_0.7.1_aot_baichuan2_13B.ipynb		tensorrt_0.7.1_aot_baichuan2_13B.ipynb
tensorrt_0.7.1_aot_chatglm3_6B_32K.ipynb		tensorrt_0.7.1_aot_chatglm3_6B_32K.ipynb
tensorrt_0.7.1_aot_qwen7B.ipynb		tensorrt_0.7.1_aot_qwen7B.ipynb
tensorrt_0.7.1_aot_qwen7B_chat.ipynb		tensorrt_0.7.1_aot_qwen7B_chat.ipynb
vllm_deploy_qwen1.5_72B_chat_AWQ.ipynb		vllm_deploy_qwen1.5_72B_chat_AWQ.ipynb
vllm_deploy_qwen1.5_7B_chat.ipynb		vllm_deploy_qwen1.5_7B_chat.ipynb
vllm_rollingbatch_deploy_Mixtral-8x7B-DPO-AWQ.ipynb		vllm_rollingbatch_deploy_Mixtral-8x7B-DPO-AWQ.ipynb
vllm_rollingbatch_deploy_mixtral_8x7b_instruct.ipynb		vllm_rollingbatch_deploy_mixtral_8x7b_instruct.ipynb
vllm_rollingbatch_deploy_yi_34B_chat.ipynb		vllm_rollingbatch_deploy_yi_34B_chat.ipynb
vllm_rollingbatch_deploy_yi_34B_chat_4bits_AWQ.ipynb		vllm_rollingbatch_deploy_yi_34B_chat_4bits_AWQ.ipynb
zepher_lmi_awq_vllm.ipynb		zepher_lmi_awq_vllm.ipynb