Demos/17. 使用 LangChain 实现 RAG.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ab3d27f2-84c5-4bae-9d40-ffcf63ba4d44",
   "metadata": {},
   "source": [
    "# 使用 LangChain 实现 RAG\n",
    "\n",
    "> 引导文章：[20. RAG 入门实践：从文档拆分到向量数据库与问答构建](https://github.com/Hoper-J/AI-Guide-and-Demos-zh_CN/blob/master/Guide/20.%20RAG%20入门实践：从文档拆分到向量数据库与问答构建.md)\n",
    "\n",
    "一份值得关注的基准测试榜单：[MTEB (Massive Text Embedding Benchmark) Leaderboard](https://huggingface.co/spaces/mteb/leaderboard)。\n",
    "\n",
    "在线链接：[Kaggle](https://www.kaggle.com/code/aidemos/17-langchain-rag) | [Colab](https://colab.research.google.com/drive/1260befv1nLiEzV7SvzPPb0n-u3IXlp6E?usp=sharing)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c4fe40c9-e1c5-4a9d-b06e-0876b1176e41",
   "metadata": {},
   "source": [
    "## 安装库\n",
    "\n",
    "```bash\n",
    "# 处理图片，tesseract 进行 OCR（以下为可选下载，sudo相关命令请跳转命令行执行）\n",
    "sudo apt-get update\n",
    "sudo apt-get install python3-pil tesseract-ocr libtesseract-dev tesseract-ocr-eng tesseract-ocr-script-latn\n",
    "pip install \"unstructured[image]\" tesseract tesseract-ocr\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "78ae14e2-26bb-4ec7-b85e-91e02b5e596a",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!pip install langchain langchain-community langchain-huggingface unstructured \n",
    "!pip install pandas\n",
    "!pip install transformers sentence-transformers accelerate\n",
    "!pip install faiss-gpu\n",
    "!pip install optimum\n",
    "!pip install \"numpy<2.0\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "031fe481-99e4-4542-a185-e411bc5d468e",
   "metadata": {},
   "source": [
    "loader.load() 报错的时候执行以下代码\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ec84e705-620e-4e46-84bc-31c0b208b50c",
   "metadata": {},
   "outputs": [],
   "source": [
    "import nltk\n",
    "\n",
    "nltk.download('punkt')\n",
    "nltk.download('punkt_tab')\n",
    "nltk.download('averaged_perceptron_tagger')\n",
    "nltk.download('averaged_perceptron_tagger_eng')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "72add581-aad1-490b-a63a-98063f90544e",
   "metadata": {},
   "source": [
    "## RA"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "229e819b-c471-4010-aace-80d8fa0138ac",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "from langchain.document_loaders import DirectoryLoader\n",
    "\n",
    "# 定义文件所在的路径\n",
    "DOC_PATH = \"../Guide\"\n",
    "\n",
    "# 使用 DirectoryLoader 从指定路径加载文件。\"*.md\" 表示加载所有 .md 格式的文件，这里仅导入文章 10（避免文章 20 的演示内容对结果的影响）\n",
    "loader = DirectoryLoader(DOC_PATH, glob=\"10*.md\")\n",
    "\n",
    "# 加载目录中的指定的 .md 文件并将其转换为文档对象列表\n",
    "documents = loader.load()\n",
    "\n",
    "# 打印查看加载的文档内容（可以注意到是去除了 markdown 标记的）\n",
    "print(documents[0].page_content[:200])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b2024a56-bd30-4999-9819-79872a9722f1",
   "metadata": {},
   "source": [
    "### 文本处理\n",
    "\n",
    "> 或许你对 chunk 会有一点印象，在 [15. 用 API 实现 AI 视频摘要：动手制作属于你的 AI 视频助手](https://github.com/Hoper-J/AI-Guide-and-Demos-zh_CN/blob/master/Guide/15.%20用%20API%20实现%20AI%20视频摘要：动手制作属于你的%20AI%20视频助手.md#拆分文本)中，我们使用了非常简单的分块方法（直接截断）。\n",
    ">\n",
    "> LangChain 提供了多种文本分块方式，例如 [RecursiveCharacterTextSplitter](https://python.langchain.com/api_reference/text_splitters/character/langchain_text_splitters.character.RecursiveCharacterTextSplitter.html)、[HTMLSectionSplitter](https://python.langchain.com/api_reference/text_splitters/html/langchain_text_splitters.html.HTMLSectionSplitter.html)、[MarkdownTextSplitter](https://python.langchain.com/api_reference/text_splitters/markdown/langchain_text_splitters.markdown.MarkdownTextSplitter.html) 等，可以根据需求选择。本文将演示 `RecursiveCharacterTextSplitter`。\n",
    "\n",
    "不过，在使用 `split_documents()` 处理文档之前，我们先使用 `split_text()` 来看看它究竟是怎么进行分块的。摘取一段[长隆万圣节](https://www.chimelong.com/gz/chimelongparadise/news/1718.html)的文本介绍："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1ba0eb02-a932-4883-81e6-fc0ae97d677a",
   "metadata": {},
   "outputs": [],
   "source": [
    "text = \"\"\"长隆广州世界嘉年华系列活动的长隆欢乐世界潮牌玩圣节隆重登场，在揭幕的第一天就吸引了大批年轻人前往打卡。据悉，这是长隆欢乐世界重金引进来自欧洲的12种巨型花车重磅出巡，让人宛若进入五彩缤纷的巨人国；全新的超级演艺广场每晚开启狂热的电音趴，将整个狂欢氛围推向高点。\n",
    "\n",
    "记者在现场看到，明日之城、异次元界、南瓜欢乐小镇、暗黑城、魔域，五大风格迥异的“鬼”域在夜晚正式开启，全新重磅升级的十大“鬼”屋恭候着各位的到来，各式各样的“鬼”开始神出“鬼”没：明日之城中丧尸成群出行，寻找新鲜的“血肉”。异次元界异形生物游走，美丽冷艳之下暗藏危机。暗黑城亡灵出没，诅咒降临。魔域异“鬼”横行，上演“血腥恐怖”。南瓜欢乐小镇小丑当家，滑稽温馨带来欢笑。五大“鬼”域以灯光音效科技情景+氛围营造360°沉浸式异域次元界探险模式为前来狂欢的“鬼”友们献上“惊奇、恐怖、搞怪、欢乐”的玩圣体验。持续23天的长隆欢乐玩圣节将挑战游客的认知极限，让你大开眼界！\n",
    "据介绍，今年长隆玩圣节与以往相比更为隆重，沉浸式场景营造惊悚氛围，两大新“鬼”王隆重登场，盛大的“鬼”王出巡仪式、数十种集声光乐和高科技于一体的街头表演、死亡巴士酷跑、南瓜欢乐小镇欢乐电音、暗黑城黑暗朋克、魔术舞台双煞魔舞、异形魔幻等一系列精彩节目无不让人拍手称奇、惊叹不止的“玩圣”盛宴让 “鬼”友们身临其境，过足“戏”瘾！\n",
    "\"\"\"\n",
    "print(len(text))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1c29c603-13a9-4477-bdbd-e3bb6fc566e1",
   "metadata": {},
   "source": [
    "这段文本长度为 581。接下来看看结果如何："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "efae196a-42ae-4509-83ce-20120fa8aae6",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
    "\n",
    "# 创建一个文本分割器。\n",
    "text_splitter = RecursiveCharacterTextSplitter(\n",
    "    chunk_size=100,   # 每个文本块的最大长度\n",
    "    chunk_overlap=20  # 文本块之间的字符重叠数量\n",
    ")\n",
    "\n",
    "# 将文本分割成多个块\n",
    "texts = text_splitter.split_text(text)\n",
    "\n",
    "# 打印分割后的文本块数量\n",
    "print(len(texts))\n",
    "\n",
    "# 打印第一个文本块的长度\n",
    "print(len(texts[0]))\n",
    "\n",
    "# 打印第一个文本块的最后 20 个字符\n",
    "print(texts[0][80:])\n",
    "\n",
    "# 打印第二个文本块的前 20 个字符\n",
    "print(texts[1][:20])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12fa7183-9c02-42a8-9118-8ae856d99835",
   "metadata": {},
   "source": [
    "> 你可以通过下图来理解 overlap：\n",
    ">\n",
    "> ![文本处理](../Guide/assets/%E6%96%87%E6%9C%AC%E5%A4%84%E7%90%86.png)\n",
    "\n",
    "到目前为止，`RecursiveCharacterTextSplitter` 的表现就像是一个简单的文本截断，没有什么特别之处。但是，让我们观察 `len(text)` 和 `len(texts)`：原文本长度为 581，分割后的段落数为 9，问题出现了。按照直接截断的假设，前 8 段应为 100 个字符，即便去除 overlap，总长度仍应超过 600，这与原始文本的长度不符。说明文本分割过程中一定执行了其他操作，而不仅仅是直接截断。\n",
    "\n",
    "实际上，`RecursiveCharacterTextSplitter()` 的关键在于 **RecursiveCharacter**，即**递归地按照指定的分隔符**（默认为 `[\"\\n\\n\", \"\\n\", \" \", \"\"]`）进行文本拆分。也就是说，在文本拆分的时候，它会尝试使用较大的分隔符来拆分文本，如果长度仍超过 `chunk_size`，则逐步使用更小的分隔符，直到长度满足或最终进行截断，也就是出现第一次分块当中的结果。所以说，第一次的分块实际上是一个“妥协”。\n",
    "\n",
    "为了更好的进行理解，现在将 `chunk_overlap` 设置为 0，并打印输出："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5eefda1e-089b-42b4-9ea3-8b69ad46dec6",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
    "\n",
    "text_splitter = RecursiveCharacterTextSplitter(\n",
    "    chunk_size=100,\n",
    "    chunk_overlap=0  # 不重叠\n",
    ")\n",
    "texts = text_splitter.split_text(text)\n",
    "\n",
    "# 输出每个片段的长度和内容\n",
    "for i, t in enumerate(texts):\n",
    "    print(f\"Chunk {i+1} length: {len(t)}\")\n",
    "    print(t)\n",
    "    print(\"-\" * 50)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "37d583a2-4d19-48ed-a41a-6ff42c14a724",
   "metadata": {},
   "source": [
    "可以看到，文本在 `全新` 和 `出巡` 之间被截断，因为达到了 100 个字符的限制，这符合直觉。然而，接下来的 `chunk 2` 只有 30 个字符，这是因为 `RecursiveCharacterTextSplitter` 并不是逐「段」分割，而是逐「分隔符」分割。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "762c5210-7529-459e-937d-ee386064be92",
   "metadata": {},
   "source": [
    "❌ `length_function` 错误示范：\n",
    "\n",
    "此时无论多长，`text_splitter._length_function()` 都返回为1，所以对任一分割的文本来说，都是一个 `_good_splits`。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "30977432-4013-488e-9e89-65da6ab32660",
   "metadata": {},
   "outputs": [],
   "source": [
    "text_splitter = RecursiveCharacterTextSplitter(\n",
    "    chunk_size=100,\n",
    "    chunk_overlap=0,\n",
    "    length_function=lambda x: 1,\n",
    ")\n",
    "texts = text_splitter.split_text(text)\n",
    "\n",
    "# 输出每个片段的长度和内容\n",
    "for i, t in enumerate(texts):\n",
    "    print(f\"Chunk {i+1} length: {len(t)}\")\n",
    "\n",
    "print(text_splitter._length_function(\"Hello\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d38beada-7b0b-41dc-b743-c458d9cb589f",
   "metadata": {},
   "source": [
    "回归正题，处理文档：\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ba1e1960-8b17-4cce-be50-4ff84641e39e",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
    "\n",
    "text_splitter = RecursiveCharacterTextSplitter(\n",
    "    chunk_size=500,  # 尝试调整它\n",
    "    chunk_overlap=100,  # 尝试调整它\n",
    "    #length_function=len,  # 可以省略\n",
    "    #separators=[\"\\n\\n\", \"\\n\", \" \", \"。\", \"\"]  # 可以省略\n",
    ")\n",
    "docs = text_splitter.split_documents(documents)\n",
    "\n",
    "print(len(docs))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6bc8f595-327c-4e8e-9308-a920ee09f1ea",
   "metadata": {},
   "source": [
    "### 加载编码模型\n",
    "\n",
    "接下来，使用 `HuggingFaceEmbeddings` 加载 Hugging Face 上的预训练模型：\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f2e2a7b0-4f46-4756-983a-af9a60e274bf",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "from langchain_huggingface import HuggingFaceEmbeddings\n",
    "\n",
    "# 指定要加载的预训练模型的名称，参考排行榜：https://huggingface.co/spaces/mteb/leaderboard\n",
    "model_name = \"chuxin-llm/Chuxin-Embedding\"\n",
    "\n",
    "# 创建 Hugging Face 的嵌入模型实例，这个模型将用于将文本转换为向量表示（embedding）\n",
    "embedding_model = HuggingFaceEmbeddings(model_name=model_name)\n",
    "\n",
    "# 打印嵌入模型的配置信息，显示模型结构和其他相关参数\n",
    "print(embedding_model)\n",
    "\n",
    "# embed_query() 方法会将文本转换为嵌入的向量\n",
    "query_embedding = embedding_model.embed_query(\"Hello\")\n",
    "\n",
    "# 打印生成的嵌入向量的长度，向量长度应与模型的输出维度一致（这里是 1024），你也可以选择打印向量看看\n",
    "print(f\"嵌入向量的维度为: {len(query_embedding)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fe18e044-4b70-4fbf-8f8a-2abbdd43bb68",
   "metadata": {},
   "source": [
    "### 建立向量数据库\n",
    "\n",
    "现在，使用预训练嵌入模型对文本片段生成实际的向量表示，然后建立向量数据库来存储和检索这些向量。这里使用 FAISS（Facebook AI Similarity Search）："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "248b1332-d453-4f8d-85d7-3792cd8e662d",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.vectorstores import FAISS\n",
    "\n",
    "# 使用嵌入模型生成向量并创建向量数据库\n",
    "vectorstore = FAISS.from_documents(docs, embedding_model)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "87b0c808-1984-44f5-bfee-7c426f3c5618",
   "metadata": {},
   "source": [
    "`FAISS.from_documents()` 方法会调用 `embedding_model` 对 `docs` 中的每个文本片段生成相应的向量表示。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1ec7a47-ddc9-4fe8-8281-d1714dc7a367",
   "metadata": {},
   "source": [
    "### 保存和加载向量数据库（可选）\n",
    "\n",
    "为了避免每次运行程序都重新计算向量表示，可以将向量数据库保存到本地，以便下次直接加载："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c926e775-257b-4bc4-bfdf-1791e088d33e",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.vectorstores import FAISS\n",
    "\n",
    "# 保存向量数据库\n",
    "vectorstore.save_local(\"faiss_index\")\n",
    "\n",
    "# 加载向量数据库\n",
    "# 注意参数 allow_dangerous_deserialization，确保你完全信任需要加载的数据库（当然，自己生成的不需要考虑这一点）\n",
    "vectorstore = FAISS.load_local(\"faiss_index\", embedding_model, allow_dangerous_deserialization=True)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f10bbd18-b010-4a91-be91-de992df94fea",
   "metadata": {},
   "source": [
    "### 创建检索器\n",
    "\n",
    "现在，我们需要创建一个检索器，用于在用户提出问题时，从向量数据库中检索相关的文本片段。\n",
    "\n",
    "`k=3` 表示每次检索返回最相似的 3 个文档片段，`k` 的大小可以根据需要调整，较大的 `k` 值会返回更多的文档片段，但可能会包含较多无关信息，也可以通过 `score` 的大小进行初筛。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2681ccce-9002-41ab-9e50-33ccc5e4d70f",
   "metadata": {},
   "outputs": [],
   "source": [
    "retriever = vectorstore.as_retriever(search_kwargs={\"k\": 3})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bde4ddda-5f3f-4267-a6cf-4763c3e974ee",
   "metadata": {},
   "source": [
    "试着检索一下：\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2781d19c-a46c-4cd6-9c30-722b02d2781a",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "query = \"Top-K 和 Top-P 的区别是什么？\"\n",
    "\n",
    "# 检索与 query 相关的文档片段\n",
    "retrieved_docs = retriever.invoke(query)\n",
    " \n",
    "# 打印检索到的文档片段\n",
    "for i, doc in enumerate(retrieved_docs):\n",
    "    print(f\"Document {i+1}:\")\n",
    "    print(f\"Content: {doc.page_content}\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ed6a1deb-a39c-424a-adb4-33bfab99d085",
   "metadata": {},
   "source": [
    "你需要注意到的是，即便是这么简单的一个文档检索，也会出现一个问题：观察 Document 1，因为文章在引言和目录部分一般会精炼总体的信息，所以 retriever 非常有可能捕捉到它，通过下面的代码查看分数（注意，这里的 score 越低越好，参见[源码](https://python.langchain.com/api_reference/_modules/langchain_community/vectorstores/faiss.html#FAISS.similarity_search_with_score_by_vector)）："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "95f4c0c0-66b6-465d-ab4d-aa6d8f278838",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# 使用 FAISS 数据库进行相似性搜索，返回最相关的文档片段\n",
    "retrieved_docs = vectorstore.similarity_search_with_score(query, k=3)\n",
    "\n",
    "# 现在的 retrieved_docs 包含 (Document, score)\n",
    "for doc, score in retrieved_docs:\n",
    "    print(f\"Score: {score}\")\n",
    "    print(f\"Content: {doc.page_content}\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "47dd5429-6f81-4cf9-9894-9b8e0ef9e08c",
   "metadata": {},
   "source": [
    "## G\n",
    "\n",
    "通过 Transformers 以及 LangChain 的 `HuggingFacePipeline`，完成文本生成任务。\n",
    "\n",
    "> 常用于 G （生成）的模型通常是 **Decoder-only** 架构，典型代表是 GPT 系列。\n",
    "\n",
    "### 加载文本生成模型\n",
    "\n",
    "这里我们选择 [19a](https://github.com/Hoper-J/AI-Guide-and-Demos-zh_CN/blob/master/Guide/19a.%20从加载到对话：使用%20Transformers%20本地运行量化%20LLM%20大模型（GPTQ%20%26%20AWQ）.md#前言) 所使用的量化模型，当然，你可以替换它："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "91f516ed-d4c2-48a0-95d1-3d049940941f",
   "metadata": {},
   "outputs": [],
   "source": [
    "from transformers import AutoTokenizer, AutoModelForCausalLM\n",
    "\n",
    "# 以下二选一，也可以进行替换\n",
    "# 本地\n",
    "model_path = './Mistral-7B-Instruct-v0.3-GPTQ-4bit'\n",
    "# 远程\n",
    "model_path = 'neuralmagic/Mistral-7B-Instruct-v0.3-GPTQ-4bit'\n",
    "\n",
    "# 加载\n",
    "tokenizer = AutoTokenizer.from_pretrained(model_path)\n",
    "model = AutoModelForCausalLM.from_pretrained(\n",
    "    model_path,\n",
    "    torch_dtype=\"auto\",  # 自动选择模型的权重数据类型\n",
    "    device_map=\"auto\",   # 自动选择可用的设备（CPU/GPU）\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9fd29b7d-aafb-4f8c-9046-cdae8633a28b",
   "metadata": {},
   "source": [
    "### 创建管道\n",
    "\n",
    "使用 Transformers 的 `pipeline` 创建一个文本生成器："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dd88441e-4e75-4d72-91ae-a2f8d469f233",
   "metadata": {},
   "outputs": [],
   "source": [
    "from transformers import pipeline\n",
    "\n",
    "generator = pipeline(\n",
    "    \"text-generation\",  # 指定任务类型为文本生成\n",
    "    model=model,\n",
    "    tokenizer=tokenizer,\n",
    "    max_length=4096,    # 指定生成文本的最大长度\n",
    "    pad_token_id=tokenizer.eos_token_id\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7e2a86c3-f9dc-44cd-baa7-0566122213fe",
   "metadata": {},
   "source": [
    "`pipeline()` 的第一个参数 [task](https://huggingface.co/docs/transformers/main_classes/pipelines#transformers.pipeline.task) 并不是可以随意自定义的名称，而是特定任务的标识。例如，\"text-generation\" 对应于构造一个 [TextGenerationPipeline](https://huggingface.co/docs/transformers/v4.45.2/en/main_classes/pipelines#transformers.TextGenerationPipeline)，用于生成文本。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9a444cda-b454-437f-9f29-9cbeb795d5fa",
   "metadata": {},
   "source": [
    "### 集成到 LangChain\n",
    "\n",
    "使用 LangChain 的 `HuggingFacePipeline` 将生成器包装为 LLM 接口："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5b1fde7d-8eb7-474c-bdfc-481a61c5a816",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_huggingface import HuggingFacePipeline\n",
    "\n",
    "llm = HuggingFacePipeline(pipeline=generator)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bdc0246c-ff99-483d-8df3-6b3038ab6791",
   "metadata": {},
   "source": [
    "### 定义提示词模版\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b094ece9-f1ff-48b7-a6a1-eaafaf123343",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.prompts import PromptTemplate\n",
    "\n",
    "custom_prompt = PromptTemplate(\n",
    "    template=\"\"\"Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer.\n",
    "\n",
    "{context}\n",
    "\n",
    "Question: {question}\n",
    "Answer:\"\"\",\n",
    "    input_variables=[\"context\", \"question\"]\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "45de0416-083c-4d57-8367-e5ab53c5701e",
   "metadata": {},
   "source": [
    "## 构建问答链\n",
    "\n",
    "使用检索器和 LLM 创建问答链："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9f1f7587-71cd-4d49-a241-1c570c48be62",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.chains import RetrievalQA\n",
    "\n",
    "qa_chain = RetrievalQA.from_chain_type(\n",
    "    llm=llm,\n",
    "    chain_type=\"stuff\",   # 直接堆叠所有检索到的文档\n",
    "    retriever=retriever,  # 使用先前定义的检索器来获取相关文档\n",
    "    # chain_type_kwargs={\"prompt\": custom_prompt}  # 可以选择传入自定义提示模板（传入的话记得取消注释），如果不需要可以删除这个参数\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2c9b7be2-4ae8-4329-a919-7ea4f4124ad0",
   "metadata": {},
   "source": [
    "`chain_type` 参数说明：\n",
    "\n",
    "- **stuff**\n",
    "  将所有检索到的文档片段直接与问题“堆叠”在一起，传递给 LLM。这种方式简单直接，但当文档数量较多时，可能会超过模型的上下文长度限制。\n",
    "\n",
    "- **map_reduce**\n",
    "\n",
    "  对每个文档片段分别生成回答（map 阶段），然后将所有回答汇总为最终答案（reduce 阶段）。\n",
    "\n",
    "- **refine**\n",
    "\n",
    "  先对第一个文档片段生成初始回答，然后依次读取后续文档，对答案进行逐步细化和完善。\n",
    "\n",
    "- **map_rerank**\n",
    "\n",
    "  对每个文档片段分别生成回答，并为每个回答打分，最终选择得分最高的回答作为答案。\n",
    "\n",
    "> `map_reduce` 和 `refine` 在[用 API 实现 AI 视频摘要](https://github.com/Hoper-J/AI-Guide-and-Demos-zh_CN/blob/master/Guide/15.%20用%20API%20实现%20AI%20视频摘要：动手制作属于你的%20AI%20视频助手.md#这里演示两种摘要方式)一文中有简单的概念解释。\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ff945771-dea1-4252-acae-e4d2eaaf0f0c",
   "metadata": {},
   "source": [
    "## 进行 QA\n",
    "\n",
    "生成可能需要等待几分钟。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bd4fc917-9ccf-4651-a4d1-50b842460776",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 提出问题\n",
    "query = \"Top-K 和 Top-P 的区别是什么？\"\n",
    "\n",
    "# 获取答案\n",
    "answer = qa_chain.invoke(query)\n",
    "# print(answer)  # 可以对比 qa_chain.run() 和 qa_chain.invoke() 在返回上的差异\n",
    "print(answer['result'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62f57144-58bb-4f55-86e5-373d66a27879",
   "metadata": {},
   "source": [
    "实际上，LangChain 并非必需。你可以观察到，代码对于模型的处理完全可以基于 Transformers，文档的递归分割实际上也可以自己构造函数来实现，使用 LangChain 只是为了将其引入我们的视野。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eaaf6b33-6ad2-4178-9f84-65634127d3d6",
   "metadata": {},
   "source": [
    "## 完整代码\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "56805985-689a-4a41-b939-d3a5e2f29b8b",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.document_loaders import DirectoryLoader\n",
    "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
    "from langchain.prompts import PromptTemplate\n",
    "from langchain.vectorstores import FAISS\n",
    "from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline\n",
    "from langchain.chains import RetrievalQA\n",
    "from langchain_huggingface import HuggingFaceEmbeddings, HuggingFacePipeline\n",
    "\n",
    "# 定义文件所在的路径\n",
    "DOC_PATH = \"../Guide\"\n",
    "\n",
    "# 使用 DirectoryLoader 从指定路径加载文件。\"*.md\" 表示加载所有 .md 格式的文件，这里仅导入文章 10（避免文章 20 的演示内容对结果的影响）\n",
    "loader = DirectoryLoader(DOC_PATH, glob=\"10*.md\")\n",
    "\n",
    "# 加载目录中的指定的 .md 文件并将其转换为文档对象列表\n",
    "documents = loader.load()\n",
    "\n",
    "# 文本处理\n",
    "text_splitter = RecursiveCharacterTextSplitter(\n",
    "    chunk_size=500,  # 尝试调整它\n",
    "    chunk_overlap=100,  # 尝试调整它\n",
    "    #length_function=len,  # 可以省略\n",
    "    #separators=[\"\\n\\n\", \"\\n\", \" \", \"。\", \"\"]  # 可以省略\n",
    ")\n",
    "docs = text_splitter.split_documents(documents)\n",
    "\n",
    "# 生成嵌入（使用 Hugging Face 模型）\n",
    "# 指定要加载的预训练模型的名称，参考排行榜：https://huggingface.co/spaces/mteb/leaderboard\n",
    "model_name = \"chuxin-llm/Chuxin-Embedding\"\n",
    "\n",
    "# 创建 Hugging Face 的嵌入模型实例，这个模型将用于将文本转换为向量表示（embedding）\n",
    "embedding_model = HuggingFaceEmbeddings(model_name=model_name)\n",
    "\n",
    "# 建立向量数据库\n",
    "vectorstore = FAISS.from_documents(docs, embedding_model)\n",
    "\n",
    "# 保存向量数据库（可选）\n",
    "#vectorstore.save_local(\"faiss_index\")\n",
    "\n",
    "# 加载向量数据库（可选）\n",
    "# 注意参数 allow_dangerous_deserialization，确保你完全信任需要加载的数据库（当然，自己生成的不需要考虑这一点）\n",
    "#vectorstore = FAISS.load_local(\"faiss_index\", embedding_model, allow_dangerous_deserialization=True)\n",
    "\n",
    "# 创建检索器\n",
    "retriever = vectorstore.as_retriever(search_kwargs={\"k\": 3})\n",
    "\n",
    "# 加载文本生成模型\n",
    "# 本地\n",
    "model_path = './Mistral-7B-Instruct-v0.3-GPTQ-4bit'\n",
    "# 远程\n",
    "#model_path = 'neuralmagic/Mistral-7B-Instruct-v0.3-GPTQ-4bit'\n",
    "\n",
    "# 加载\n",
    "tokenizer = AutoTokenizer.from_pretrained(model_path)\n",
    "model = AutoModelForCausalLM.from_pretrained(\n",
    "    model_path,\n",
    "    torch_dtype=\"auto\",  # 自动选择模型的权重数据类型\n",
    "    device_map=\"auto\",   # 自动选择可用的设备（CPU/GPU）\n",
    ")\n",
    "\n",
    "# 创建文本生成管道\n",
    "generator = pipeline(\n",
    "    \"text-generation\",  # 指定任务类型为文本生成\n",
    "    model=model,\n",
    "    tokenizer=tokenizer,\n",
    "    max_length=4096,    # 指定生成文本的最大长度\n",
    "    pad_token_id=tokenizer.eos_token_id\n",
    ")\n",
    "\n",
    "# 包装为 LangChain 的 LLM 接口\n",
    "llm = HuggingFacePipeline(pipeline=generator)\n",
    "\n",
    "custom_prompt = PromptTemplate(\n",
    "    template=\"\"\"Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer.\n",
    "\n",
    "{context}\n",
    "\n",
    "Question: {question}\n",
    "Answer:\"\"\",\n",
    "    input_variables=[\"context\", \"question\"]\n",
    ")\n",
    "\n",
    "# 构建问答链\n",
    "qa_chain = RetrievalQA.from_chain_type(\n",
    "    llm=llm,\n",
    "    chain_type=\"stuff\",   # 直接堆叠所有检索到的文档\n",
    "    retriever=retriever,  # 使用先前定义的检索器来获取相关文档\n",
    "    # chain_type_kwargs={\"prompt\": custom_prompt}  # 可以选择传入自定义提示模板（传入的话记得取消注释），如果不需要可以删除这个参数\n",
    ")\n",
    "\n",
    "# 提出问题\n",
    "query = \"Top-K 和 Top-P 的区别是什么？\"\n",
    "\n",
    "# 获取答案\n",
    "answer = qa_chain.invoke(query)\n",
    "print(answer['result'])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3745b38f-1bcb-4e54-aad5-56d74e93bfe6",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}