Prompt_Engineering_with_Qwen.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# Qwen 提示工程指南\n",
        "\n",
        "本教程翻译并改编自 [Prompt Engineering with Llama 2](https://github.com/facebookresearch/llama-recipes/blob/main/examples/Prompt_Engineering_with_Llama_2.ipynb)，使用 Qwen（通义千问） 作为 LLM，并采用全中文提示词。\n",
        "\n",
        "旨在通过交互式 Notebook 展示使用 Qwen 进行提示词工程的最佳实践。Qwen 模型推理基于 Xinference，可以[在 Colab 中直接运行](https://colab.research.google.com/github/onesuper/Prompt_Engineering_with_Qwen/blob/main/Prompt_Engineering_with_Qwen.ipynb)。\n"
      ],
      "metadata": {
        "id": "FAhwDgtUGIEo"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 1 介绍"
      ],
      "metadata": {
        "id": "Veb5lDspPRkY"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 1.1 提示词\n",
        "\n",
        "大型语言模型的交互主要依靠文本：我们向模型提供一段文本，模型随后返回一段新的文本。 模型的响应质量和准确性与我们的提问方式紧密相关。通过提供清晰、具体且针对性的指令，我们可以帮助 AI 更准确地理解我们的问题。这种设计和优化提示词的技巧被称为“提示词工程”（Prompt Engineering）。"
      ],
      "metadata": {
        "id": "0OQoK44TQtAO"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 1.2 Qwen（通义千问）模型\n",
        "\n",
        "[Qwen](https://github.com/QwenLM/Qwen)（通义千问）系列工作，当前开源模型的参数规模为18亿（1.8B）、70亿（7B）、140亿（14B）和720亿（72B）。\n",
        "\n",
        "其中包括基础模型 Qwen，即：\n",
        "\n",
        "- Qwen-1.8B\n",
        "- Qwen-7B\n",
        "- Qwen-14B\n",
        "- Qwen-72B\n",
        "\n",
        "\n",
        "对话模型 Qwen-Chat，即：\n",
        "\n",
        "- Qwen-1.8B-Chat\n",
        "- Qwen-7B-Chat\n",
        "- Qwen-14B-Chat\n",
        "- Qwen-72B-Chat"
      ],
      "metadata": {
        "id": "4Zf9AuboPT9f"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "### 1.3 使用 Xinference 本地本地模型推理\n",
        "\n",
        "这份指南中使用的 Qwen，由 [Xinference](github.com/xorbitsai/inference) 在本地运行并提供 API 访问。Xinference 支持的 Qwen 系列[模型列表可以参考这个链接](https://inference.readthedocs.io/en/latest/models/builtin/llm/qwen-chat.html)。\n",
        "\n",
        "\n",
        "Xinference 是一个开源模型平台，旨在简化各种 AI 模型的操作和集成。借助 Xinference，你可以轻松地在云端或本地环境中运行任何开源、嵌入式模型和多模态模型，并创建基于人工智能的应用程序。"
      ],
      "metadata": {
        "id": "WzJegrOpGH4N"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 2. Notebook 准备\n",
        "\n",
        "\n",
        "推荐在 GPU 上运行这个 Notebook。如果你在 Colab 中运行此示例，可以通过以下方法修改实例类型：\n",
        "\n",
        "在工具栏菜单中，点击 **Runtime** > **Change runtime type** > **Select the GPU (T4)**"
      ],
      "metadata": {
        "id": "LovckG0kGr9j"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 2.1 检查内存和 GPU 资源"
      ],
      "metadata": {
        "id": "bPDeDltCGABt"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import psutil\n",
        "import torch\n",
        "\n",
        "ram = psutil.virtual_memory()\n",
        "ram_total = ram.total / (1024**3)\n",
        "print('RAM: %.2f GB' % ram_total)\n",
        "\n",
        "if torch.cuda.is_available():\n",
        "  print('GPU is abailable')\n",
        "else:\n",
        "  print('GPU NOT available')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "qhoItBBhF7uY",
        "outputId": "f59db9c5-5cc7-4e2d-cb0e-66be289a494f"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "RAM: 12.67 GB\n",
            "GPU is abailable\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "\n",
        "\n",
        "### 2.2 安装 Xinference 和 OpenAI 客户端\n",
        "\n"
      ],
      "metadata": {
        "id": "eFzlnU4gG_JL"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "1eReutJA_jS_"
      },
      "outputs": [],
      "source": [
        "!pip install -U -q xinference[transformers] openai"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import pkg_resources\n",
        "\n",
        "def get_version(package):\n",
        "    try:\n",
        "        return pkg_resources.get_distribution(package).version\n",
        "    except pkg_resources.DistributionNotFound:\n",
        "        return None\n",
        "\n",
        "# 防止 typing_extensions 版本太老，openai 报错。重启后版本将更新，可以继续往下执行\n",
        "print(get_version(\"typing_extensions\"))\n",
        "if get_version(\"typing_extensions\") < \"4.8.0\":\n",
        "    import os\n",
        "    os.kill(os.getpid(), 9)"
      ],
      "metadata": {
        "id": "P8OUwyOzoSah",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "f351f9ad-de8a-44b1-84f5-2702c28e32f1"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "4.9.0\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "\n",
        "### 2.3 使用 Xinference 运行 Qwen-Chat 模型\n",
        "\n",
        "\n",
        "要启动Xinference的本地实例，请使用以下命令在后台运行 `xinference`:"
      ],
      "metadata": {
        "id": "2EACA0GYHm2o"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "%%bash\n",
        "if ps ax | grep -v grep | grep \"xinference-local\" > /dev/null\n",
        "then\n",
        "    echo \"Service is already running, exiting.\"\n",
        "else\n",
        "    echo \"Service is not running, starting service.\"\n",
        "    nohup xinference-local  > xinference.log 2>&1 &\n",
        "fi"
      ],
      "metadata": {
        "id": "5EM01Gq7IQ2y",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "be7fac3a-754c-4de5-eddb-9f0ac1772cf2"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Service is not running, starting service.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "恭喜！您现在已经成功在 Colab 机器上安装并启动了 Xinference。默认的主机和 IP 地址分别为127.0.0.1 和 9997。"
      ],
      "metadata": {
        "id": "WXtUJSC3I3kh"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "使用如下命令运行模型：`Qwen-14B-Chat`。您可以在[这里](https://inference.readthedocs.io/en/latest/models/builtin/llm/qwen-chat.html)查看其他模型的版本。由于 T4 GPU 只有 15 GB的 CUDA 内存，所以选择 Int4 版本的 GPTQ 量化格式。"
      ],
      "metadata": {
        "id": "wvhIEjcHKXc5"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!xinference launch \\\n",
        "  --model-uid my-llm \\\n",
        "  --model-name qwen-chat \\\n",
        "  --size-in-billions 14 \\\n",
        "  --model-format gptq \\\n",
        "  --quantization Int4"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "B_hQFqxOKiww",
        "outputId": "422e133d-500e-4f60-a91e-781c11c56c5b"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model uid: my-llm\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "第一次启动一个模型时，Xinference 将从 HuggingFace 下载该模型的参数，这可能需要几分钟时间取决于该模型权重的大小。\n",
        "\n"
      ],
      "metadata": {
        "id": "Q--ic56eNDyo"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 3 使用 OpenAI API 访问模型\n",
        "\n",
        "当模型启动成功后，我们就可以使用 OpenAI 的 Python 客户端进行连接。下面我们封装了一些 util 方法，核心调用有两个，一个是 Completion API，另一个是 Chat Completion API。"
      ],
      "metadata": {
        "id": "6FwIeBXQK1mM"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from typing import Dict, List\n",
        "\n",
        "import openai\n",
        "\n",
        "client = openai.Client(\n",
        "    api_key=\"cannot be empty\",\n",
        "    base_url=\"http://127.0.0.1:9997/v1\"\n",
        ")\n",
        "\n",
        "def completion(\n",
        "    client,\n",
        "    model,\n",
        "    prompt: str,\n",
        "    temperature: float = 0.7,\n",
        "    top_p: float = 0.9\n",
        "):\n",
        "    return client.completions.create(\n",
        "        model=model,\n",
        "        prompt=prompt,\n",
        "        top_p=top_p,\n",
        "        temperature=temperature\n",
        "    ).choices[0].text\n",
        "\n",
        "def chat_completion(\n",
        "    client,\n",
        "    model,\n",
        "    messages,\n",
        "    temperature: float = 0.6,\n",
        "    top_p: float = 0.9\n",
        "):\n",
        "    return client.chat.completions.create(\n",
        "        model=model,\n",
        "        messages=messages,\n",
        "        top_p=top_p,\n",
        "        temperature=temperature\n",
        "    ).choices[0].message.content\n",
        "\n",
        "def assistant(content: str):\n",
        "    return { \"role\": \"assistant\", \"content\": content }\n",
        "\n",
        "def user(content: str):\n",
        "    return { \"role\": \"user\", \"content\": content }\n",
        "\n",
        "def complete_and_print(prompt: str, client=client, model=\"my-llm\"):\n",
        "    print(f'==============\\n{prompt}\\n==============')\n",
        "    response = completion(client, model, prompt)\n",
        "    print(response, end='\\n\\n')\n",
        "\n",
        "\n",
        "def chat_complete_and_print(messages, client=client, model=\"my-llm\"):\n",
        "    response = chat_completion(client, model, messages)\n",
        "    print(f'==============\\nassistant: {response}\\n\\n')\n"
      ],
      "metadata": {
        "id": "n9R7x6Cf8U39"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 3.1 Completion API\n",
        "\n",
        "补全 API 用来通过一段文本（也叫 prompt）进行文本生成。直接来看两个例子"
      ],
      "metadata": {
        "id": "7-zgfgSc0pcm"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "complete_and_print(\"天空通常的颜色是:  \")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "HqcgmDACBM8I",
        "outputId": "53e39854-59ce-4bfc-919a-995ba42707f2"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "==============\n",
            "天空通常的颜色是:  \n",
            "==============\n",
            " A. 白色   B. 灰色   C. 蓝色\n",
            "\n",
            "C. 蓝色\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "complete_and_print(\"你是什么模型？\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "RiqLCShBC1s-",
        "outputId": "43de9ddb-519c-46e7-a55c-a65576e54fd3"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "==============\n",
            "你是什么模型？\n",
            "==============\n",
            "我的模型是大模型，我叫通义千问。\n",
            "\n",
            "非常抱歉，我是来自阿里云的超大规模语言模型，我叫通义千问。请问有什么可以帮助您的吗？\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 3.2 Chat Completion API\n",
        "\n",
        "聊天补全可以为与 LLM 进行交互时提供更加结构化的方式。相比于单纯传递一段文字，我们将一个包含多个结构化信息对象的数组发送给 LLM 作为输入。这样做可以让语言模型在继续回复时有一些\"上下文\"或者\"历史\"可参考。\n",
        "\n",
        "通常情况下，每条信息都会有一个角色（role）和内容（content）：\n",
        "\n",
        "- 系统角色（system）用来向语言模型传达开发者定义好的核心指令。\n",
        "\n",
        "- 用户角色（user）则代表着用户自己输入或者产生出来的信息。\n",
        "\n",
        "- 助手角色（assistant）则是由语言模型自动生成并回复出来。"
      ],
      "metadata": {
        "id": "Q858sbHf0ryU"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "chat_complete_and_print(\n",
        "    messages=[\n",
        "        user(\"我最喜欢的颜色是蓝色\"),\n",
        "        assistant(\"听到这个消息真是令人欣喜！\"),\n",
        "        user(\"我最喜欢的颜色是什么？\"),\n",
        "    ]\n",
        ")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wIkaVaC6ESb3",
        "outputId": "51f1f466-cc1a-4120-e7ee-c61eb098dcac"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "==============\n",
            "user: 我最喜欢的颜色是蓝色\n",
            "==============\n",
            "assistant: 听到这个消息真是令人欣喜！\n",
            "==============\n",
            "user: 我最喜欢的颜色是什么？\n",
            "==============\n",
            "assistant: 您最喜欢的顏色是蓝色。\n",
            "\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 3.3 LLM 超参配置\n",
        "\n",
        "API 提供了一些参数可以影响输出结果的创造力和确定性。\n",
        "\n",
        "在每个步骤中，LLM 会生成一个最有可能出现的标记列表以及它们对应的概率。根据 `top_p` 值，概率较低的标记将被排除在列表之外，并且从剩余候选项中随机选择一个标记（使用 `temperature` 来调整）。\n",
        "\n",
        "简单来说：`top_p` 参数控制着生成文本时所使用词汇范围大小，而 `temperature` 参数则决定了在这个范围内文本生成时是否具有随机性。当温度接近 0 时，则会得到几乎是确定性结果。\n"
      ],
      "metadata": {
        "id": "olBYmVXp1IPM"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def tuned_complete_and_print(\n",
        "    top_p, temperature, client=client, model=\"my-llm\"):\n",
        "    prompt = \"写一首关于通义千问的三行俳句诗\"\n",
        "    response = completion(client, model, prompt=prompt,\n",
        "                          top_p=top_p,\n",
        "                          temperature=temperature)\n",
        "    print(f'[temperature: {temperature} | top_p: {top_p}]\\n{response.strip()}\\n')\n",
        "\n",
        "# 前两次生成会非常雷同，\n",
        "tuned_complete_and_print(0.01, 0.01)\n",
        "tuned_complete_and_print(0.01, 0.01)\n",
        "\n",
        "#后两次生成会不太一样\n",
        "tuned_complete_and_print(1.0, 1.0)\n",
        "tuned_complete_and_print(1.0, 1.0)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "754Rj8GJ1jLA",
        "outputId": "7f8a43e4-9eaf-46fb-ab50-27c3f432dda7"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[temperature: 0.01 | top_p: 0.01]\n",
            "通义千问，智慧如泉涌。问题解答，知识无边界。\n",
            "\n",
            "[temperature: 0.01 | top_p: 0.01]\n",
            "通义千问，智慧如泉涌。问题解答，知识无边界。\n",
            "\n",
            "[temperature: 1.0 | top_p: 1.0]\n",
            "通义千问者，知识之海中航行，智慧之光闪烁。\n",
            "\n",
            "[temperature: 1.0 | top_p: 1.0]\n",
            "通义千问来，知识海洋汇。人工智能新纪元。\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 4 提示词技术\n",
        "\n"
      ],
      "metadata": {
        "id": "uFCHRj21GgsW"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 4.1 显示指令（Explicit Instructions）\n",
        "\n",
        "详细、明确的指导比模糊不清的提示能够产生更好的效果："
      ],
      "metadata": {
        "id": "cBG576r1QTDd"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "complete_and_print(\"用不超过30个字的一句话描述量子物理学的基本原理。\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "25EmEeSZF9bf",
        "outputId": "8fcf5035-7706-4991-9760-ed906d81cf98"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "==============\n",
            "用不超过30个字的一句话描述量子物理学的基本原理。\n",
            "==============\n",
            " 量子物理学是研究微观粒子行为的科学，其中存在着波粒二象性、不确定性原理等奇特现象。\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "显示指令其实就是使用**规则**和**限制**来规定 Qwen 如何响应你的提示词。\n",
        "\n",
        "* 格式化\n",
        "  * 就像在儿童教育网节目中教小学生一样给我解释一下。\n",
        "\n",
        "  * 我是一名软件工程师，使用大型语言模型进行总结。用2 50 个单词概括以下文本:\n",
        "\n",
        "  * 给出你的答案，就像一个古老的私家侦探一步一步地追查一个案件。\n",
        "\n",
        "* 格式化\n",
        "\n",
        "  * 使用项目符号。\n",
        "\n",
        "  * 作为 JSON 对象返回。\n",
        "\n",
        "  * 少用一些专业术语，帮助我在沟通工作中运用。\n",
        "\n",
        "* 限制\n",
        "\n",
        "  * 只使用学术论文。\n",
        "\n",
        "  * 不要提供 2020 年以后的信息来源。\n",
        "\n",
        "  * 如果你不知道答案，就说你不知道。\n",
        "\n",
        "\n",
        "  下面是一个给出明确指示的例子，通过限制回答范围为最近创建的来源来获得更具体的结果。\n"
      ],
      "metadata": {
        "id": "g_D3wQxJOGpx"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "complete_and_print(\"向我解释一下大型语言模型的最新进展。\")\n",
        "\n",
        "complete_and_print(\"请向我解释一下大型语言模型的最新进展，确保提供所引用的来源，且这些来源不能早于2020年。\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BOIzbH7-RGDD",
        "outputId": "40f4dd2a-95ae-4168-e77d-9d886d18ff56"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "==============\n",
            "向我解释一下大型语言模型的最新进展。\n",
            "==============\n",
            " 大型语言模型是一种深度学习技术，用于处理和生成自然语言文本。最新的进展包括更大、更复杂的模型架构（如GPT-3和T5），更多的训练数据，以及更先进的训练方法（如自回归预训练和迁移学习）。这些进展使得大型语言模型在各种自然语言处理任务上取得了前所未有的性能，并且正在推动人工智能领域的发展。\n",
            "\n",
            "==============\n",
            "请向我解释一下大型语言模型的最新进展，确保提供所引用的来源，且这些来源不能早于2020年。\n",
            "==============\n",
            " 大型语言模型是人工智能领域的一种技术，其目的是训练出能够理解和生成自然语言的算法模型。近年来，这一领域的研究取得了一些重要进展。\n",
            "\n",
            "首先，Google在2019年推出了BERT（Bidirectional Encoder Representations from Transformers），这是一种基于Transformer架构的预训练语言模型。BERT通过无监督学习的方式，在大量文本数据上进行训练，从而获得对自然语言的理解能力。BERT的出现标志着预训练语言模型进入了新的阶段，并在多项自然语言处理任务中取得了优秀的表现（Devlin et al., 2019）。\n",
            "\n",
            "其次，OpenAI在2020年发布了GPT-3（Generative Pre-trained Transformer 3），这是目前最大的预训练语言模型之一。GPT-3可以生成高质量的文章、代码、诗歌等，并且在多项自然语言处理任务中表现出色。GPT-3的发布引起了广泛的关注和讨论，人们开始重新思考如何利用这种强大的工具来推动自然语言处理的发展（Brown et al., 2020）。\n",
            "\n",
            "此外，Facebook也在2020年推出了一款名为RoBERTa的预训练语言模型，该模型在多项自然语言处理任务中表现优秀。RoBERTa采用了更多的训练数据和更长的训练时间，使得模型更好地理解了语言的复杂性和多样性（Liu et al., 2019）。\n",
            "\n",
            "总的来说，大型语言模型的研究正在不断推进，研究人员们希望通过改进模型的结构、优化训练策略等方式，使模型更加准确、高效地理解和生成自然语言。这将为自然语言处理领域带来更多的可能性和机遇（Raffel et al., 2019）。 \n",
            "\n",
            "参考文献：\n",
            "- Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2019). BERT: A pre-training approach for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) (pp. 4171–4183).\n",
            "- Brown, P., Mann, B., Ryder, N., Subbiah, M., Vaswani, A., Adiwardana, D., ... & Zettlemoyer, L. (2020). Language models are few-shot learners. Advances in Neural Information Processing Systems, 33.\n",
            "- Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., ... & Le, Q. V. (2019). RoBERTa: A robustly optimized transformer for language understanding. arXiv preprint arXiv:1907.11692.\n",
            "- Raffel, C. L., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Kaiser, L. (2019). Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv:1910.10658.\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 4.2 零样本（Zero-shot）和少样本（Few-Shot）学习\n",
        "\n",
        "\"Shot\" 是指对大型语言模型所期望得到的提示和回答类型进行演示或举例。这个术语最初来源于在计算机视觉领域中使用照片训练模型时，其中一张照片被称为一个 \"shot\"，表示该模型用它来分类图像。\n"
      ],
      "metadata": {
        "id": "TPwAMfK7T3rU"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "#### 4.2.1 零样本提示（Zero-Shot Prompting）\n",
        "\n",
        "大型语言模型之所以独特，是因为它们能够根据指令产生回应，而无需事先见过任务示例。\n",
        "\n",
        "不需要示例就能进行提示被称为“Zero-shot提示”。让我们尝试将Qwen 用作情感检测器。通过改进提示方式，我们可以提升其表现。"
      ],
      "metadata": {
        "id": "2w66vcwPQMax"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "complete_and_print(\"文本: 这部电影是我看过的最棒的！\\n文本的情感是: \")\n",
        "\n",
        "complete_and_print(\"文本: 导演过于刻意。\\n文本的情感是: \")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Z-c7xy4ZThHR",
        "outputId": "4de94d48-1b9a-40c8-e0ca-8334a8bcd56e"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "==============\n",
            "文本: 这部电影是我看过的最棒的！\n",
            "文本的情感是: \n",
            "==============\n",
            "。\n",
            "\n",
            "根据文本内容，可以判断出情感是正面的。这是因为文本中使用了形容词\"最棒的\"来形容这部电影，这是一个积极的评价词汇。因此，可以得出结论，该文本表达了正面的情感。\n",
            "\n",
            "==============\n",
            "文本: 导演过于刻意。\n",
            "文本的情感是: \n",
            "==============\n",
            "。\n",
            "\n",
            "负面情感。\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### 4.2.2 少样本提示（Few-Shot Prompting）\n",
        "\n",
        "通过添加具体示例来指导模型生成结果通常能够得到更准确、一致的输出。这种技术被称为\"少样本提示\"（Few-Shot Prompting）。\n",
        "\n",
        "在这个例子中，生成的回复符合我们期望的格式要求，并提供了一个更精细化的情感分类器，可以给出积极、中性和消极回应对应的置信度百分比。"
      ],
      "metadata": {
        "id": "KuJwCYH4VW5_"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def print_sentiment(text):\n",
        "    print(f'INPUT: {text}')\n",
        "    chat_complete_and_print(\n",
        "        messages=[\n",
        "            user(\"你是一个情感分类器。对于每条信息，给出积极/中性/消极的百分比。\"),\n",
        "            user(\"我喜欢它\"),\n",
        "            assistant(\"70% 积极 30% 中性 0% 消极\"),\n",
        "            user(\"可以更好一些\"),\n",
        "            assistant(\"0% 积极 50% 中性 50% 消极\"),\n",
        "            user(\"还行吧\"),\n",
        "            assistant(\"25% 积极 50% 中性 25% 消极\"),\n",
        "            user(text),\n",
        "        ]\n",
        "    )\n",
        "print_sentiment(\"我觉得还可以！\")\n",
        "\n",
        "print_sentiment(\"我喜欢！\")\n",
        "\n",
        "print_sentiment(\"糟糕的服务 0/10\")\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Uo9GnOh8VWt3",
        "outputId": "3fefd7db-01cd-4049-b5c1-f843cf5a6cd1"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "INPUT: 我觉得还可以！\n",
            "==============\n",
            "assistant: 40% 积极 60% 中性 0% 消极\n",
            "\n",
            "\n",
            "INPUT: 我喜欢！\n",
            "==============\n",
            "assistant: 90% 积极 10% 中性 0% 消极\n",
            "\n",
            "\n",
            "INPUT: 糟糕的服务 0/10\n",
            "==============\n",
            "assistant: 0% 积极 0% 中性 100% 消极\n",
            "\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 4.3 角色提示词\n",
        "\n",
        "当给定角色时，大语言模型通常会给出更一致的回应。角色为LLM提供了所需答案类型的背景。\n",
        "\n",
        "让我们使用 Qwen 来针对使用 PyTorch **的利弊问题创建一个技术回答。**"
      ],
      "metadata": {
        "id": "Pcz1jW5usSMA"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "complete_and_print(\"解释使用PyTorch的利弊。\")\n",
        "\n",
        "complete_and_print(\"你是一名机器学习专家，为处理复杂数据集的高级工程师提供高度技术性的建议。解释使用PyTorch的利弊。\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PDsy4EursiTi",
        "outputId": "0ca1a74c-22c2-43ac-e4d2-d4e644fa6f28"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "==============\n",
            "解释使用PyTorch的利弊。\n",
            "==============\n",
            " PyTorch是一种开源机器学习库，用于构建和训练深度学习模型。\n",
            "\n",
            "优势：\n",
            "1. 易于理解和使用：PyTorch提供了一个直观的接口，使用户能够更容易地理解深度学习的概念。\n",
            "2. 动态计算图：PyTorch允许用户在运行时创建和修改神经网络架构，这对于实验和原型设计非常有用。\n",
            "3. 强大的GPU支持：PyTorch可以利用GPU进行大规模并行计算，从而加速训练过程。\n",
            "4. 社区活跃：PyTorch有一个庞大的社区，提供了大量的教程、示例代码和工具包，帮助用户解决问题。\n",
            "\n",
            "劣势：\n",
            "1. 性能问题：虽然PyTorch可以利用GPU加速训练，但在某些情况下，它的性能可能不如TensorFlow等静态计算图框架。\n",
            "2. 内存管理：与静态计算图框架相比，动态计算图可能会导致更多的内存消耗。\n",
            "3. 系统要求高：PyTorch需要一个强大的硬件环境才能有效地运行大型深度学习模型。\n",
            "\n",
            "==============\n",
            "你是一名机器学习专家，为处理复杂数据集的高级工程师提供高度技术性的建议。解释使用PyTorch的利弊。\n",
            "==============\n",
            "请确保您的答案具有挑战性，并考虑到需要解决的问题。\n",
            "\n",
            "PyTorch是一个强大的开源深度学习框架，它提供了许多功能来处理和分析复杂的多维数据集。然而，任何工具都有其优点和缺点，下面我将详细解释一下在使用PyTorch时的一些优势和劣势：\n",
            "\n",
            "**优势：**\n",
            "\n",
            "1. **灵活性和可扩展性**：PyTorch的设计允许用户自由地定义和修改模型架构。这使得它非常适合处理非标准或非常规的数据集。此外，PyTorch可以轻松地与其他Python库（如NumPy、Pandas等）集成，从而增加了其应用范围和灵活性。\n",
            "\n",
            "2. **动态图计算**：PyTorch采用动态图计算方式，这意味着你可以随时查看并修改你的计算流程，这对于调试和优化是非常有用的。相比之下，静态图计算方式（如TensorFlow）则需要预先确定计算流程。\n",
            "\n",
            "3. **易于使用和学习**：PyTorch的API设计简洁明了，对于初学者来说很容易上手。同时，PyTorch社区也非常活跃，有许多优秀的教程和示例可供参考。\n",
            "\n",
            "4. **高效的GPU加速**：PyTorch充分利用了现代GPU的性能，可以在大规模数据集上进行快速训练。而且，PyTorch支持自动混合精度（APuS）训练，能够进一步提高训练速度和效率。\n",
            "\n",
            "**劣势：**\n",
            "\n",
            "1. **内存管理**：PyTorch的内存管理机制可能导致内存泄漏或者占用过多的内存，特别是在处理大型数据集时。因此，你需要小心地管理和释放内存，以避免这些问题的发生。\n",
            "\n",
            "2. **不稳定性**：虽然PyTorch的灵活性是一种优势，但也可能导致代码不稳定。由于PyTorch允许你在运行时改变计算流程，所以可能会出现一些难以预测的行为。\n",
            "\n",
            "3. **模型转换**：如果你的目标是部署到生产环境，那么你需要将PyTorch模型转换成其他格式（如ONNX）。这个过程可能需要额外的时间和精力。\n",
            "\n",
            "总的来说，PyTorch是一个强大且灵活的深度学习框架，适合处理各种复杂的多维数据集。但是，在使用它时需要注意内存管理和代码稳定性等问题，以及模型转换的需求。\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 4.4 思考链（Chain-of-Thought）\n",
        "\n",
        "通过简单地添加一句鼓励逐步思考的短语，“CoT”或“Chain-of-Thought”提示技术可以显著提升大型语言模型进行复杂推理的能力（Wei等人，2022）。"
      ],
      "metadata": {
        "id": "ojZ9Gst84v97"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "complete_and_print(\"莫扎特和猫王谁活得更久一些？\")\n",
        "\n",
        "complete_and_print(\"莫扎特和猫王谁活得更久一些？请一步一步地思考。\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "qjVru1NdsjgY",
        "outputId": "e268bfa7-0d7a-444c-b66e-a1af4f25af02"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "==============\n",
            "莫扎特和猫王谁活得更久一些？\n",
            "==============\n",
            " 莫扎特于1756年出生，1791年去世，享年35岁。\n",
            "\n",
            "猫王即埃尔维斯·普雷斯利，于1935年出生，1977年去世，享年42岁。\n",
            "\n",
            "因此，莫扎特比猫王早去世了3年。\n",
            "\n",
            "==============\n",
            "莫扎特和猫王谁活得更久一些？请一步一步地思考。\n",
            "==============\n",
            " 首先，我们需要知道莫扎特和猫王的出生年份和死亡年份。 莫扎特出生于1756年，死于1791年，享年35岁。 猫王出生于1935年，死于1977年，享年42岁。\n",
            "然后，我们可以比较他们的年龄。 莫扎特去世时只有35岁，而猫王去世时是42岁。 所以，猫王比莫扎特多活了7年。\n",
            "因此，猫王活得更久一些。\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 4.5 自恰性（Self-Consistency）\n",
        "\n",
        "LLM 是具有概率性质的模型，所以即使使用了 Chain-of-Thought 方法，单次生成可能会产生错误结果。为了提高准确性，Self-Consistency（Wang等人，2022年）引入了一种方法：从多次生成中选择出现频率最高的答案（但需要更多计算资源）。"
      ],
      "metadata": {
        "id": "NpJpaHqx7bIQ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import re\n",
        "from statistics import mode\n",
        "\n",
        "def gen_answer():\n",
        "    response = completion(\n",
        "        client,\n",
        "        \"my-llm\",\n",
        "        prompt=\"15个数的平均值是40。\"\n",
        "        \"如果每个数字都加上10，那么这些数字的平均值是多少？\"\n",
        "        \"返回用三个反引号号括起来的答案，例如: ```123```\",\n",
        "\n",
        "    )\n",
        "    match = re.search(r'```(\\d+)```', response)\n",
        "    if match is None:\n",
        "        return None\n",
        "    return match.group(1)\n",
        "\n",
        "answers = [gen_answer() for i in range(10)]\n",
        "\n",
        "print(\n",
        "    f\"Answers: {answers}\\n\",\n",
        "    f\"Final answer: {mode(answers)}\",\n",
        ")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ZJujX05d73lL",
        "outputId": "ecea1205-0202-4ae2-cc86-1cb8c6128284"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Answers: ['50', '40', '70', '150', '50', None, '50', '50', '50', '40']\n",
            " Final answer: 50\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 4.6 RAG（检索增强生成）\n",
        "\n",
        "在你的应用程序中，您可能需要使用一些事实性知识。您可以直接从现有的大型模型中提取常见事实（即仅使用模型权重）。"
      ],
      "metadata": {
        "id": "tB8M1ZLJEISk"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "complete_and_print(\"阿根廷的首都是哪里？\")\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "el9sId5o8SPi",
        "outputId": "56700000-bc3d-4e27-cd8c-def70d695cb3"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "==============\n",
            "阿根廷的首都是哪里？\n",
            "==============\n",
            " 阿根的首都是布宜诺斯艾利斯。\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "然而，更加具体的事实或私人信息无法可靠地获取。模型可能会表示不清楚或者提供错误的答案。"
      ],
      "metadata": {
        "id": "q7IfGy47EdLQ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "complete_and_print(\"请问2023年12月12日在上海的气温是多少？\")\n",
        "\n",
        "complete_and_print(\"请问星期六晚上的晚餐预订是几点？还有，我应该穿什么呢？\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "k3qQ40neEfxI",
        "outputId": "aa1a809f-dfd9-495b-913a-cd90e516ba14"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "==============\n",
            "请问2023年12月12日在上海的气温是多少？\n",
            "==============\n",
            " 很抱歉，作为一个AI语言模型，我无法预测未来的天气情况。建议您关注当地的气象预报或者使用相关的天气查询工具来获取更准确的信息。\n",
            "\n",
            "==============\n",
            "请问星期六晚上的晚餐预订是几点？还有，我应该穿什么呢？\n",
            "==============\n",
            "谢谢。 星期六晚上的晚餐预订时间可能会因餐厅而异。建议您提前联系餐厅以确认预订时间和要求的着装规定。通常，正式的餐厅会要求客人穿着得体，例如男士穿西装或衬衫和领带，女士则可以穿着裙装或套装。但具体情况最好还是直接询问餐厅。\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "检索增强生成（Retrieval-Augmented Generation），简称 RAG，在您从外部数据库中获取提示时将信息包括其中。这是一种有效地将事实融入 LLM 应用程序的方式，相较于费用高昂且可能对基础模型能力产生负面影响的精细调整来说更加经济实惠。\n",
        "\n",
        "它可以简单地使用 lookup 表，也可以复杂到使用包含公司全部知识的向量数据库。"
      ],
      "metadata": {
        "id": "_jNU4wKRFSCn"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "MENLO_PARK_TEMPS = {\n",
        "    \"2023-12-11\": \"30摄氏度\",\n",
        "    \"2023-12-12\": \"29摄氏度\",\n",
        "    \"2023-12-13\": \"28摄氏度\",\n",
        "}\n",
        "\n",
        "\n",
        "def prompt_with_rag(retrived_info, question):\n",
        "    complete_and_print(\n",
        "        f\"已知信息: '{retrived_info}', 回答这个问题: '{question}'\"\n",
        "    )\n",
        "\n",
        "\n",
        "def ask_for_temperature(day):\n",
        "    temp_on_day = MENLO_PARK_TEMPS.get(day) or \"未知温度\"\n",
        "    prompt_with_rag(\n",
        "        f\"在{day}，上海的温度是{temp_on_day}\",  # Retrieved fact\n",
        "        f\"{day}上海的温度是？\",  # User question\n",
        "    )\n",
        "\n",
        "ask_for_temperature(\"2023-12-12\")\n",
        "\n",
        "ask_for_temperature(\"2023-07-18\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "MHCVap0MFtYK",
        "outputId": "26ce8977-53ac-456d-ebf2-61a18b08ab5d"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "==============\n",
            "已知信息: '在2023-12-12，上海的温度是29摄氏度', 回答这个问题: '2023-12-12上海的温度是？'\n",
            "==============\n",
            " \n",
            "\n",
            "根据提供的信息，可以知道在2023年12月12日，上海的温度是29摄氏度。因此，答案为29摄氏度。\n",
            "\n",
            "==============\n",
            "已知信息: '在2023-07-18，上海的温度是未知温度', 回答这个问题: '2023-07-18上海的温度是？'\n",
            "==============\n",
            " 这个问题无法回答，因为缺乏关于上海2023年7月18日的具体天气预报数据。建议查阅可靠的天气预报来源以获取更准确的信息。\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 4.7 基于程序辅助的语言模型（Program-Aided Language Model）\n",
        "\n",
        "\n",
        "从本质上讲，LLMs并不擅长进行计算，试一试（正确答案应该是 91383）：\n"
      ],
      "metadata": {
        "id": "PQtfme7xHc94"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "complete_and_print(\"\"\"\n",
        "Calculate the answer to the following math problem:\n",
        "\n",
        "((-5 + 93 * 4 - 0) * (4^4 + -7 + 0 * 5))\n",
        "\"\"\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "A3ixFe51Hipi",
        "outputId": "6a12954c-9642-4181-a4b8-2f5649f1ba20"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "==============\n",
            "\n",
            "Calculate the answer to the following math problem:\n",
            "\n",
            "((-5 + 93 * 4 - 0) * (4^4 + -7 + 0 * 5))\n",
            "\n",
            "==============\n",
            "6. Answer: \n",
            "\n",
            "((-5 + 93 * 4 - 0) * (4^4 + -7 + 0 * 5)) = 36821\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Gao等人（2022）提出了“程序辅助语言模型”（PAL）的概念。尽管 LLM 在进行算术运算时效果不好，但它非常适合生成代码。PAL 利用这个特点，指导 LLM 编写代码来解决计算任务。"
      ],
      "metadata": {
        "id": "uziOlOsrHqyt"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "complete_and_print(\"\"\"\n",
        "# Python code to calculate: ((-5 + 93 * 4 - 0) * (4^4 + -7 + 0 * 5))\n",
        "\"\"\"\n",
        ")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ZFLEApiuH13r",
        "outputId": "10176bf8-2dde-449d-8446-64d16540e71c"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "==============\n",
            "\n",
            "# Python code to calculate: ((-5 + 93 * 4 - 0) * (4^4 + -7 + 0 * 5))\n",
            "\n",
            "==============\n",
            "# We need to first calculate the value of the expression inside the parenthesis\n",
            "expressionInsideParenthesis = (-5 + 93 * 4 - 0) * (4**4 + -7 + 0 * 5)\n",
            "print(expressionInsideParenthesis)\n",
            "```\n",
            "\n",
            "The output of this code will be:\n",
            "\n",
            "```\n",
            "126552.0\n",
            "```\n",
            "\n",
            "Therefore, the result of evaluating the given expression using the order of operations is `126552.0`.\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# The following code was generated by qwen-chat 14B\n",
        "\n",
        "expressionInsideParenthesis = (-5 + 93 * 4 - 0) * (4**4 + -7 + 0 * 5)\n",
        "print(expressionInsideParenthesis)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "tc85nhNwIBvV",
        "outputId": "4de76792-c87a-4fd6-8834-2438d26da66e"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "91383\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 4.8 限制多余令牌\n",
        "\n",
        "解决过程中常遇到的问题之一就是生成带有多余标记（例如“当然！这里还有更多相关信息...”）的输出。\n",
        "\n",
        "以下是一种改进方法，通过结合角色、规则和限制、明确指示以及一个例子来解决该问题："
      ],
      "metadata": {
        "id": "57_FYJE-Hqe3"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "complete_and_print(\n",
        "    \"请提供Menlo Park的邮政编码，使用JSON格式，并且在结果中包含字段'zip_code'。\",\n",
        ")\n",
        "\n",
        "complete_and_print(\n",
        "    \"\"\"你是一个只输出JSON的机器人。以JSON格式回复，并包含字段'zip_code'。\n",
        "\n",
        "样例问题: 帝国大厦的邮政编码是多少?\n",
        "样例回答: {'zip_code': 10118}\n",
        "\n",
        "问题: Menlo Park的邮政编码是多少?\n",
        "    \"\"\"\n",
        ")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "34liaBUqIRLb",
        "outputId": "405190c5-b3dc-4d0c-d909-a86ead7f151e"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "==============\n",
            "请提供Menlo Park的邮政编码，使用JSON格式，并且在结果中包含字段'zip_code'。\n",
            "==============\n",
            " Menlo Park的邮政编码是94025。\n",
            "\n",
            "以下是使用JSON格式提供的Menlo Park邮政编码：\n",
            "```json\n",
            "{\n",
            "  \"zip_code\": \"94025\"\n",
            "}\n",
            "```\n",
            "请注意，这个答案只提供了Menlo Park的邮政编码。如果你需要其他的信息，比如Menlo Park的位置、人口等等，你需要查询其他的资料或者API。\n",
            "\n",
            "==============\n",
            "你是一个只输出JSON的机器人。以JSON格式回复，并包含字段'zip_code'。\n",
            "\n",
            "样例问题: 帝国大厦的邮政编码是多少?\n",
            "样例回答: {'zip_code': 10118}\n",
            "\n",
            "问题: Menlo Park的邮政编码是多少?\n",
            "    \n",
            "==============\n",
            " 回答:\n",
            "\n",
            "{'zip_code': 94025}\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 5 参考资料\n",
        "\n",
        "\n",
        "- [PromptingGuide.ai](https://www.promptingguide.ai/)\n",
        "- [LearnPrompting.org](https://learnprompting.org/)\n",
        "- [Lil'Log Prompt Engineering Guide](https://lilianweng.github.io/posts/2023-03-15-prompt-engineering/)"
      ],
      "metadata": {
        "id": "9orenI5yKLdX"
      }
    }
  ],
  "metadata": {
    "colab": {
      "provenance": [],
      "gpuType": "T4"
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "nbformat": 4,
  "nbformat_minor": 0
}