Demos/05. 尝试部署你的第一个语言模型.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ce62e879-c21f-4a27-8e08-547fb5e88e18",
   "metadata": {},
   "source": [
    "# 部署你的第一个语言模型：本地或云端\n",
    "\n",
    "> 指导文章：[06. 开始实践：部署你的第一个语言模型](https://github.com/Hoper-J/LLM-Guide-and-Demos-zh_CN/blob/master/Guide/06.%20开始实践：部署你的第一个语言模型.md)\n",
    "\n",
    "在之前的章节中，我们已经了解了 Hugging Face 中 `AutoModel` 系列的不同类。现在，我们将使用一个参数量较小的模型，为你进行演示，并向你展示如何使用 FastAPI 和 Flask 将模型部署为 API 服务。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "18464b86-548c-4d85-904d-4023e1ad60dc",
   "metadata": {},
   "source": [
    "## 安装库"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2acccf64-20c1-4864-bb30-daca602dc459",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!pip install transformers\n",
    "!pip install sentencepiece"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b6c3128-d166-4287-b0e5-99d364c22661",
   "metadata": {},
   "source": [
    "## 选择并加载模型\n",
    "\n",
    "我们选择一个参数量较小的模型，如 `distilgpt2`，这是 GPT-2 的精简版本（或者说蒸馏），只有约 8820 万参数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5d765488-51f4-4579-9ed4-44cb68b3ba27",
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "from transformers import AutoTokenizer, AutoModelForCausalLM\n",
    "\n",
    "# 指定模型名称\n",
    "model_name = \"distilgpt2\"\n",
    "\n",
    "# 加载 Tokenizer\n",
    "tokenizer = AutoTokenizer.from_pretrained(model_name)\n",
    "\n",
    "# 加载预训练模型\n",
    "model = AutoModelForCausalLM.from_pretrained(model_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fbf8bf8c-139c-437b-a383-6ba260a059db",
   "metadata": {},
   "source": [
    "## 移动模型到合适的设备\n",
    "\n",
    "如果你的计算机有 GPU，可将模型移动到 GPU，加快推理速度。如果你使用的是 Apple 芯片的 Mac，可以移动到 `mps` 上。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e12e9eab-6af4-4952-a591-d300b726f0cd",
   "metadata": {},
   "outputs": [],
   "source": [
    "device = torch.device(\"cuda\" if torch.cuda.is_available() \n",
    "                      else \"mps\" if torch.backends.mps.is_available() \n",
    "                      else \"cpu\")\n",
    "model.to(device)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7471945b-5703-4f1f-b006-3de1c82b2e1f",
   "metadata": {},
   "source": [
    "## 进行推理\n",
    "\n",
    "现在，我们可以使用模型进行文本生成。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "776b0ea0-c50a-4a0c-a579-0e9c5dc6c15a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 设置模型为评估模式\n",
    "model.eval()\n",
    "\n",
    "# 输入文本\n",
    "input_text = \"Hello GPT\"\n",
    "\n",
    "# 编码输入文本\n",
    "inputs = tokenizer(input_text, return_tensors=\"pt\")\n",
    "inputs = {key: value.to(device) for key, value in inputs.items()}\n",
    "\n",
    "# 生成文本\n",
    "with torch.no_grad():\n",
    "    outputs = model.generate(\n",
    "        **inputs,\n",
    "        max_length=200,\n",
    "        num_beams=5,\n",
    "        no_repeat_ngram_size=2,\n",
    "        early_stopping=False\n",
    "    )\n",
    "\n",
    "# 解码生成的文本\n",
    "generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)\n",
    "print(\"模型生成的文本：\")\n",
    "print(generated_text)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "38d85917-79de-449b-a939-35dd4533d079",
   "metadata": {},
   "source": [
    "## 部署模型为 API 服务（可选）\n",
    "\n",
    "如果你希望将模型部署为一个 API 服务，供其他应用调用，可以使用 `fastapi` 框架。\n",
    "\n",
    "### 使用 FastAPI 部署模型\n",
    "#### 安装 `fastapi` 和 `uvicorn`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "63857f59-416b-4aab-8a90-ddbab8704b58",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install fastapi uvicorn"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "6b96fe34-56dc-44e8-955e-d1192e964d66",
   "metadata": {},
   "source": [
    "#### 创建 API 服务\n",
    "\n",
    "把下面这段代码保存到 `app_fastapi.py` 文件中（你也可以命名为其他的）\n",
    "\n",
    "```python\n",
    "from fastapi import FastAPI, HTTPException\n",
    "from pydantic import BaseModel\n",
    "from transformers import AutoTokenizer, AutoModelForCausalLM\n",
    "import torch\n",
    "\n",
    "# 定义请求体的数据模型\n",
    "class PromptRequest(BaseModel):\n",
    "    prompt: str\n",
    "\n",
    "app = FastAPI()\n",
    "\n",
    "# 加载模型和分词器\n",
    "model_name = \"distilgpt2\"\n",
    "tokenizer = AutoTokenizer.from_pretrained(model_name)\n",
    "model = AutoModelForCausalLM.from_pretrained(model_name)\n",
    "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
    "model.to(device)\n",
    "\n",
    "@app.post(\"/generate\")\n",
    "def generate_text(request: PromptRequest):\n",
    "    prompt = request.prompt\n",
    "    if not prompt:\n",
    "        raise HTTPException(status_code=400, detail=\"No prompt provided\")\n",
    "\n",
    "    inputs = tokenizer(prompt, return_tensors=\"pt\").to(device)\n",
    "    with torch.no_grad():\n",
    "        outputs = model.generate(\n",
    "            **inputs,\n",
    "            max_length=200,\n",
    "            num_beams=5,\n",
    "            no_repeat_ngram_size=2,\n",
    "            early_stopping=True\n",
    "        )\n",
    "    generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)\n",
    "    return {\"generated_text\": generated_text}\n",
    "\n",
    "```\n",
    "\n",
    "然后在命令行执行以下命令：\n",
    "\n",
    "```bash\n",
    "uvicorn app_fastapi:app --host 0.0.0.0 --port 8000\n",
    "```\n",
    "\n",
    "你应该可以看到：\n",
    "![image-20240914112627874](../Guide/assets/image-20240914131113829.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4676f1cf-3edf-481a-a57b-df47b24b6bbe",
   "metadata": {},
   "source": [
    "#### 通过 API 调用模型\n",
    "\n",
    "现在，你可以访问 [http://localhost:8000/docs](http://localhost:8000/docs) 交互使用，或者通过发送 HTTP 请求来调用模型。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "17726267-d159-40d3-a768-4d663b8a0d52",
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "\n",
    "response = requests.post(\n",
    "    \"http://localhost:8000/generate\",\n",
    "    json={\"prompt\": \"Hello GPT\"}\n",
    ")\n",
    "print(response.json())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c0e8ab4-c604-4f10-a2f3-6554d1d6db63",
   "metadata": {},
   "source": [
    "### 使用 Flask 部署模型\n",
    "\n",
    "#### 安装 Flask\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "939cd948-806f-4185-a45d-d0b21a55d7d9",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install flask"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bcbfe1db-7990-4b5a-a7e9-f057054796fc",
   "metadata": {},
   "source": [
    "#### 创建 API 服务\n",
    "\n",
    "把下面这段代码保存到 `app_flask.py` 文件中（你也可以命名为其他的）\n",
    "\n",
    "```python\n",
    "from flask import Flask, request, jsonify\n",
    "from transformers import AutoTokenizer, AutoModelForCausalLM\n",
    "import torch\n",
    "\n",
    "app = Flask(__name__)\n",
    "\n",
    "# 加载模型和分词器\n",
    "model_name = \"distilgpt2\"\n",
    "tokenizer = AutoTokenizer.from_pretrained(model_name)\n",
    "model = AutoModelForCausalLM.from_pretrained(model_name)\n",
    "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
    "model.to(device)\n",
    "\n",
    "@app.route('/generate', methods=['POST'])\n",
    "def generate():\n",
    "    prompt = request.json.get('prompt')\n",
    "    if not prompt:\n",
    "        return jsonify({'error': 'No prompt provided'}), 400\n",
    "\n",
    "    inputs = tokenizer(prompt, return_tensors=\"pt\").to(device)\n",
    "    with torch.no_grad():\n",
    "        outputs = model.generate(\n",
    "            **inputs,\n",
    "            max_length=200,\n",
    "            num_beams=5,\n",
    "            no_repeat_ngram_size=2,\n",
    "            early_stopping=True\n",
    "        )\n",
    "    generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)\n",
    "    return jsonify({'generated_text': generated_text})\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    app.run(host='0.0.0.0', port=8000)\n",
    "\n",
    "\n",
    "```\n",
    "\n",
    "然后在命令行执行以下命令：\n",
    "\n",
    "```bash\n",
    "python app_flask.py\n",
    "```\n",
    "\n",
    "你应该可以看到：\n",
    "![image-20240914112627874](../Guide/assets/20240914143358.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7672d01a-b824-4278-b986-a8d804d650fd",
   "metadata": {},
   "source": [
    "#### 通过 API 调用模型\n",
    "\n",
    "现在，你可以通过发送 HTTP 请求来调用模型。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1c50d324-9b16-4741-ab25-fcb995613815",
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "\n",
    "response = requests.post(\n",
    "    \"http://localhost:8000/generate\",\n",
    "    json={\"prompt\": \"Hello GPT\"}\n",
    ")\n",
    "print(response.json())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7f211f84-0284-4545-90b9-578fe8bdea1b",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}