Notebook.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](img.png)\n",
    "# LLMs in Real Products\n",
    "<a target=\"_blank\" href=\"https://colab.research.google.com/github/life-efficient/OpenAI-Powered-Products/blob/main/Notebook.ipynb\">\n",
    "  <img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/>\n",
    "</a>\n",
    "\n",
    "Aim: Create an AI powered product to illustrate the general approach to building OpenAI \"wrapper startups\".\n",
    "\n",
    "> ### To demonstrate this, we will build: __a product that tells a back and forth story, and creates visualisations of the story as it evolves.__\n",
    "\n",
    "## Outline\n",
    "\n",
    "### Part 1: Build an application that runs from a Python notebook\n",
    "1. Allow the user to type out what happens next in the story\n",
    "1. Using prompt engineering to pre-process the user's text\n",
    "1. Make a request to the OpenAI API to continue the story by completing our text\n",
    "1. Use the moderation API to ensure our responses aren't violating any guidelines\n",
    "1. Generate images to visualise the story as we progress\n",
    "1. Run the application in Python as an example\n",
    "\n",
    "![](./images/Local%20Application.png)\n",
    "\n",
    "### Part 2: Build a user interface and an API to deploy the application online\n",
    "1. Wrap all of our code in an API\n",
    "1. Use ChatGPT to code up a working user interface that could be deployed online\n",
    "1. Make the user interface interact with our API\n",
    "\n",
    "![](./images/API%20Application.png)\n",
    "\n",
    "Let's do this...\n",
    "\n",
    "## Before we start, it's important to understand what an API is\n",
    "\n",
    "API stands for Application Programmable Interface\n",
    "- Simply put, it's a piece of software that can accept \"requests\" made my code and knows how to respond to different types of requests\n",
    "  - E.g. the OpenAI API knows how to return a text completion as a response to a request to complete some text\n",
    "- The OpenAI API is running on a computer in the cloud\n",
    "- The API we will build in part 2 will be running on your own computer\n",
    "    - Eventually, you could deploy this to a computer running in the cloud and provide access to everyone on the internet\n",
    "\n",
    "![](./images/API%20Request%20Response.png)\n",
    "\n",
    "# Part 1: Build an application that runs from a Python notebook\n",
    "\n",
    "![](./images/Local%20Application.png)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Installing dependencies\n",
    "\n",
    "The cell below runs several commands in the terminal that each install a Python library. If you're on Google Colab, they will install the library to the machine running in the cloud. If you're running this notebook locally, they will install the libraries to your computer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# !conda install - y - c conda-forge uvicorn\n",
    "# !conda install - c conda-forge openai\n",
    "# !conda install - c conda-forge fastapi\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setting Up\n",
    "\n",
    "To set up, you'll need to install the `openai` Python library, and then set your API key. You create/find an API key on your [OpenAI console](https://platform.openai.com/account/api-keys).\n",
    "\n",
    "I've seen some people have trouble with this, so if you run into issues:\n",
    "- Ensure you've [created an account on OpenAI](https://platform.openai.com/overview#:~:text=Examples-,Log,-in)\n",
    "- Ensure you've created an API key [on your account](https://platform.openai.com/account/api-keys)\n",
    "- Ensure you've replaced my placeholder API key in the next code cell with your own API key\n",
    "- Ensure you've set up a paid account on OpenAI\n",
    "    - You will need to add a credit card, but don't worry, the amount of requests we will make will only cost a few pence\n",
    "- Try [creating a different API key](https://platform.openai.com/account/api-keys) and changing it in the code things still don't work"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import openai\n",
    "\n",
    "openai.api_key = \"YOUR_API_KEY\""
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## List the available models\n",
    "\n",
    "The OpenAI API provides many different services including:\n",
    "- Providing chat responses\n",
    "- Completing a starting prompt\n",
    "- Moderating inappropriate text\n",
    "- Generating images\n",
    "\n",
    "Under the hood, each of those services is an AI model (a function which takes in some input and makes a prediction about the result, e.g. text -> completion).\n",
    "\n",
    "Check the comparison in the documentation, [here](https://platform.openai.com/docs/models/gpt-3-5), or uncomment and run the cell below (but prepare for hundreds of lines of output to scroll through).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# openai.Model.list() # shows hundreds of lines"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Extra: Fine tune a model\n",
    "\n",
    "If you acquire your own dataset, you can fine-tune a model to your specific domain.\n",
    "\n",
    "Check out the details [here](https://platform.openai.com/docs/guides/fine-tuning)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Requesting Text Completions\n",
    "\n",
    "Firstly, let's get a hang of how to use one of the chat-optimised models, `gpt-3.5-turbo`.\n",
    "\n",
    "As I always emphasise, the real skill is being able to read the documentation. Check it out [here](https://platform.openai.com/docs/api-reference/completions).\n",
    "\n",
    "Make sure to take a look into the different parameters and what they do."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "completion = \n",
    "\n",
    "print(completion)\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Unpacking the response\n",
    "\n",
    "The response that you get back contains more than just the text completion. As a next step, write a line of code to get just the text out."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "prediction = \n",
    "print(prediction)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Wrap all of this into a function\n",
    "\n",
    "In our case, both making the request and unpacking the response are things that are always going to need to happen together, so we should encapsulate them into the same function.\n",
    "\n",
    "You'll need to:\n",
    "- Pass in a list representing the sequence of messages which have been said in the chat so far, rather than just a single prompt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_completion(messages):\n",
    "\n",
    "    completion = ## make a request to the API to get the completion that should follow the messages\n",
    "\n",
    "    prediction = ## extract the predicted text from the completion\n",
    "    return prediction\n",
    "\n",
    "\n",
    "print(\"EXAMPLE 1\")\n",
    "messages = [\n",
    "    {\"role\": \"user\", \"content\": \"What is the meaning of life?\"}\n",
    "]\n",
    "print(get_completion(messages))\n",
    "\n",
    "\n",
    "print(\"EXAMPLE 2\")\n",
    "messages = [\n",
    "    {\"role\": \"user\", \"content\": \"Hi, how are you?\"},\n",
    "    {\"role\": \"assistant\", \"content\": \"Hi, how are you?\"},\n",
    "    {\"role\": \"user\", \"content\": \"I'm good. What's for lunch?\"}\n",
    "]\n",
    "print(get_completion(messages))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Moderations\n",
    "\n",
    "As a precaution against innapropriate messaging getting out to your users, you can run responses from the models through the moderations API.\n",
    "\n",
    "It's unlikely that the OpenAI models will respond with anything inappropriate, because they're aligned not to. It's more likely that this should be used to make sure your user requests are not harmful."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "response = ## make a request to the API to get the completion that should follow the messages\n",
    "\n",
    "print(response)\n",
    "\n",
    "result = ## make a request to determine whether the response is toxic or not\n",
    "print(result)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Now let's combine the code into a prototype application\n",
    "\n",
    "You will need to:\n",
    "- Prompt the system to provide a start to the story\n",
    "- Then in a loop:\n",
    "    - Ask the user to input what happens next\n",
    "    - Check that the user input does not violate the content policy\n",
    "    - Ask OpenAI to continue the story\n",
    "\n",
    "Feel free to add extra functionality!\n",
    "\n",
    "> Note: You'll need to \"interrupt\" the code cell by hitting the stop icon to run the code again after you make changes, so that it doesn't get stuck in the infinite loop."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_next_line(messages):\n",
    "    \"\"\"Return the list of messages appended with the next line of the story\"\"\"\n",
    "    response = ## get completion\n",
    "    print(\"AI:\", response) ## print the next line of the story to the user\n",
    "    ## append the next line of the story to the messages\n",
    "\n",
    "    message = ## get the next line of the story from the user\n",
    "    print(\"You:\", message)\n",
    "    ## append the next line of the story to the messages\n",
    "    ## add a final prompt from the user askign the assistant for the next line of the story\n",
    "    return messages\n",
    "\n",
    "\n",
    "def main():\n",
    "    messages = ## create an initial message from the user asking the assistant for the first line of the story\n",
    "    ## loop forever\n",
    "        messages = ## get the next line of the story\n",
    "\n",
    "\n",
    "main()\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Adding AI Generated Images\n",
    "\n",
    "To immerse our users in the story, let's use the OpenAI [image generation API endpoint](https://platform.openai.com/docs/api-reference/images) to visualise what the most recent scene in the story looks like.\n",
    "\n",
    "This can be accomplished using OpenAI's image generation endpoint, which creates an image from a text prompt.\n",
    "\n",
    "To do this, we'll:\n",
    "1. Take part of the story so far and transform it into an image prompt\n",
    "1. Make a request to generate an image from that prompt\n",
    "1. Unpack the image response\n",
    "1. Show the image to the user (or just give them the link to open for now - it's hard to visualise images in a notebook)\n",
    "\n",
    "## Generating the image prompt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_image_prompt_from_text(text):\n",
    "    print(\"TEXT TO IMAGE PROMPT\", text)\n",
    "\n",
    "    text = ## combine the text from the story with some text asking the AI to create an image (engineer the prompt)\n",
    "\n",
    "    response = ## make a request to the API to get the image prompt\n",
    "    response = ## extract the image prompt from the response\n",
    "    return response\n",
    "\n",
    "story = get_completion([{\"role\": \"user\", \"content\": \"Give me the first line of a story!\"}])\n",
    "print(\"STORY\")\n",
    "print(story)\n",
    "print(\"\\n\\n\\n\")\n",
    "scene = create_image_prompt_from_text(story)\n",
    "print(\"SCENE\")\n",
    "print(scene)\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Generate an Image from the Prompt\n",
    "\n",
    "Now you've got the description of an image that visualises this part of the story, you can generate an image from it using DALL-E, OpenAI's image generation API."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def generate_image_from_prompt(prompt):\n",
    "    response = ## make a request to the API to get the image\n",
    "    return response\n",
    "\n",
    "\n",
    "image = generate_image_from_prompt(scene)\n",
    "print(image)\n",
    "\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Unpacking the image from the response\n",
    "\n",
    "Again, the response contains more than just the image, so we'll need to extract the image URL."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "image_url = ## unpack the image URL from the response\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The following cell displays an image in a notebook"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.display import Image\n",
    "\n",
    "Image(url=image_url) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_and_display_image_from_prompt(prompt):\n",
    "    print(prompt)\n",
    "    image = ## generate image response from API\n",
    "    image_url = ## unpack the image URL from the response\n",
    "    print(image_url)\n",
    "    ## display the image in the notebook\n",
    "\n",
    "create_and_display_image_from_prompt(scene)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Tying that all together into our application\n",
    "\n",
    "Let's take everything we've built so far and run that as a single application"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_story_with_images():\n",
    "    ## create the loop that generates the story line by line and shows an image at each step\n",
    "    \n",
    "\n",
    "create_story_with_images()\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Part 2: Build a user interface and an API to deploy the application online\n",
    "\n",
    "To serve our application as a product, we need to put all of this in a Python file, then turn it into an API\n",
    "\n",
    "![](./images/API%20Application.png)\n",
    "\n",
    "## Move to VSCode\n",
    "\n",
    "VSCode is the industry standard code development environment - a place for writing code.\n",
    "\n",
    "I know one of you is about to ask whether they can use a different editor like Spyder or PyCharm, but please just get with the program and use VSCode - it's professional and almost certainly what you'd be required to use in industry. \n",
    "We will use a VSCode extension to run the user interface that shows the application.\n",
    "If you refuse to use VSCode you'll need to find another way of running the user interface, and I'm not sure how to do that.\n",
    "\n",
    "> ### At this point, you should move away from Google Colab, and instead to VSCode then start working in Python files (`.py`)\n",
    "\n",
    "In VSCode, put the utilities you've developed in this file into a Python file called `utils.py`.\n",
    "\n",
    "## Building the back-end\n",
    "\n",
    "The \"back-end\" of our application will be an API (application programmable interface). Here's the essentials of what you need to know about an API:\n",
    "- Simply a program which runs on a computer somewhere, typically in a datacentre (on the cloud)\n",
    "- Can accept \"requests\" and return responses\n",
    "- Can implement different, custom responses to different requests\n",
    "\n",
    "Our API will be able to respond to requests for things like:\n",
    "- Text completion\n",
    "- Moderation\n",
    "- Converting our story so far into an image prompt\n",
    "- Getting recommendations for follow-up prompts\n",
    "\n",
    "_FastAPI_ is a Python library that can be used to create APIs. It defines simple ways to define the different things that your API can respond to. Read more about FastAPI [here](https://fastapi.tiangolo.com/).\n",
    "\n",
    "Here's the basic structure:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# highlight the basic strcuture for using fastapi\n",
    "\n",
    "import uvicorn\n",
    "from fastapi import FastAPI\n",
    "from pydantic import BaseModel\n",
    "\n",
    "app = FastAPI()\n",
    "\n",
    "class EndpointData(BaseModel): # define the schema for the incoming data\n",
    "    text: str # this schema expects an attribute of type string called \"text\"\n",
    "\n",
    "@app.get(\"/\")\n",
    "def read_root():\n",
    "    return {\"Hello\": \"World\"}\n",
    "\n",
    "@app.post(\"/demo-endpoint\") # define what happens to POST requests made to the /chat-response endpoint \n",
    "def predict(endpoint_data: EndpointData): # define a method that expects the incoming data to follow the Chat schema\n",
    "    print(\"incoming payload text attribute:\", endpoint_data.text)\n",
    "    return {\"response\": \"This is a response\"} # return a response\n",
    "\n",
    "if __name__ == '__main__': # if this file is being run directly\n",
    "    uvicorn.run(app, host='localhost', port=8000) # run the API\n",
    "\n",
    "# run the following from terminal to run this API: `uvicorn get_prediction:app --reload`"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before this works, you'll need to use `conda` to install some code that it depends on. You may want to do this in a new and separate conda environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!conda install -y -c conda-forge uvicorn\n",
    "!conda install -c conda-forge openai\n",
    "!conda install -c conda-forge fastapi"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Note again, this API code should be in a different `.py` file. To run it, run `uvicorn get_prediction: app --reload` from the terminal.\n",
    "\n",
    "The API will continue to run until you kill the terminal process or until it runs into an error.\n",
    "\n",
    "Remember that the API needs to be running still.\n",
    "\n",
    "## Testing the back-end\n",
    "\n",
    "We can test the back end by running it locally, and then making requests to it via Python.\n",
    "\n",
    "There are other ways to make these request too, including:\n",
    "- Using `curl` from the command line\n",
    "- Opening the [URL](http://localhost:8000/demo-endpoint) that's serving the webpage\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "\n",
    "response = ## make a request to your API\n",
    "print(response)\n",
    "print(response.json())"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that the back end is working, you can begin to connect the front end to it. I've included a file called `index.html` that contains HTML which defines the structure of the webpage.\n",
    "\n",
    "To run it locally with minimal hassle, click on the extensions tab on the far left of VSCode and download the extension shown in the following image.\n",
    "\n",
    "![](./images/extension.png)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As per the instructions shown, you can hit `CTRL + SHIFT + L` when you're inside an HTML file, and it will run a preview of that webpage on a tab in your browser. It also updates as soon as you hit save.\n",
    "\n",
    "When you run that you should see something like this:\n",
    "\n",
    "![](./images/app.png)\n",
    "\n",
    "If you take a look into the HTML file you can begin to decipher how the code works. You'll see it contains not just HTML, but also CSS and JavaScript.\n",
    "\n",
    "But there's one secret weapon you can use here even if you don't know HTML, CSS, JavaScript... You can ask ChatGPT to write the code for you."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Challenge: Use what you've learnt so far to build your own application\n",
    "\n",
    "You should:\n",
    "- Ask ChatGPT to help you write the UI\n",
    "    - I don't expect you to know HTML, JS, or CSS, and you won't be responsible for this in industry either\n",
    "    - If you're in lecture with me, then I can help you write this\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Key Takeaways\n",
    "- The OpenAI API is extremely easy to use\n",
    "- Many products can be built using prompt engineering to adjust the input to OpenAI API endpoints to do things like create images and respond to a chat\n",
    "- You can easily build a simple prototype of a product with just a few tools, in just a few hours\n",
    "- You can use ChatGPT to write a mock up of the front-end with you"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "base",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.3"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}