LLM-Custom-RAG-Kendra-Llama2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "b213e484-b98b-494f-a9df-d5251ca68161",
   "metadata": {
    "tags": []
   },
   "source": [
    "# Large Language Model Customization using Retrieval-Augmented Generation (RAG) Pattern, Amazon Kendra Enterprise Search Service and Llama-2-70b-chat Language Model\n",
    "\n",
    "---\n",
    "This Amazon SageMaker Studio notebook demonstrates how to use [SageMaker](https://sagemaker.readthedocs.io/en/stable/) and [boto3](https://boto3.amazonaws.com/v1/documentation/api/latest/index.html) SDKs to generate text using the Retrieval-Augmented Generation (RAG) pattern. The notebook implements semantic search using [Amazon Kendra](https://aws.amazon.com/kendra/) enterprise search service. The language model used for text generation is [Llama-2-70b-chat](https://huggingface.co/meta-llama/Llama-2-70b-chat).\n",
    "\n",
    "This notebook has the following prerequisites:\n",
    "- Select an AWS region where [Amazon SageMaker JumpStart](https://aws.amazon.com/sagemaker/jumpstart) is available. \n",
    "- [Setup Amazon SageMaker Domain](https://docs.aws.amazon.com/sagemaker/latest/dg/onboard-quick-start.html).\n",
    "- [Add an additional permission to the SageMaker Execution Role](https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-roles.html) to call the Amazon Kendra Retrieve API.\n",
    "- [Available service queta](https://docs.aws.amazon.com/general/latest/gr/sagemaker.html) for \"ml.g5.48xlarge for endpoint usage\".\n",
    "- Select the [Amazon SageMaker Kernel](https://docs.aws.amazon.com/sagemaker/latest/dg/notebooks-available-kernels.html), \"Python 3 (Data Science 2.0) with Python 3.8\" or higher.\n",
    "- Accept the [end-user license agreement](https://ai.meta.com/resources/models-and-libraries/llama-downloads/) of the LLM and pass custom_attributes='accept_eula=true' as part of predict call header.\n",
    "- Familiarity with [Retrieval Augmented Generation (RAG)](https://docs.aws.amazon.com/sagemaker/latest/dg/jumpstart-foundation-models-customize-rag.html) pattern.\n",
    "- About $20 per hour to spend on Amazon SageMaker JumpStart model deployments and usage of other AWS services.\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4e64c3e2-30ed-49dd-9ac6-b2060abdf44d",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# install pythn libraries\n",
    "!pip install --upgrade pip --quiet\n",
    "!pip install --upgrade boto3 --quiet\n",
    "!pip install --upgrade sagemaker --quiet\n",
    "!pip install ipywidgets --quiet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cebcd571-7bf5-4f0f-a9c6-c86e497e096e",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# important required libraries\n",
    "import boto3\n",
    "from sagemaker.jumpstart.model import JumpStartModel"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4beeca04-9cdd-4c22-91d2-129b5f1f75c0",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Step 1: Create Amazon Kendra Index (this step can take between 30 to 60 minutes)\n",
    "\n",
    "Create Amazon Kendra index and configure it to index your dataset. Alternatively, you can use the provided [AWS CloudFormation template](/kendra-docs-index.yaml) to create a new Amazon Kendra index containing AWS online documentation for Amazon Kendra, Amazon Lex, and Amazon SageMaker.\n",
    "This notebook must be granted AWS IAM permission to call Amazon Kendra APIs and deployed in the same AWS region where Amazon Kendra will be created."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "287f0bb6-a8fa-4a30-99f3-bd3f878a8f42",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# set your kendra_index_id \n",
    "kendra_index_id = \"xxxxxxx-xxxx-xxxxx-xxxx-xxxxxxxxxxxx\""
   ]
  },
  {
   "cell_type": "raw",
   "id": "9edc9509-0b25-4f38-83ac-02f951992b41",
   "metadata": {},
   "source": [
    "### Step 2: Deploy a SageMaker endpoint for Llama-2-70b-chat LLM on G5 instance (NVIDIA A10G GPU)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "33cccb3d-c57a-4466-a95b-23b77c6c719e",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%%time\n",
    "\n",
    "# Define SageMaker JumpStart Model using model id, instance type, and endpoint timeout\n",
    "my_model = JumpStartModel(model_id=\"meta-textgeneration-llama-2-70b-f\",\n",
    "                          instance_type=\"ml.g5.48xlarge\")\n",
    "\n",
    "# Host the model on the instance and deploy an inference endpoint\n",
    "# Because the model size is >80GB, expecy deploy() to take 15 min!\n",
    "predictor = my_model.deploy()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8309fdca-4e8f-40e5-a24c-23a54917f4bc",
   "metadata": {},
   "source": [
    "#### Step 3: Define context search function "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2161f36e-aae8-4120-99c1-08a465676aca",
   "metadata": {},
   "outputs": [],
   "source": [
    "def retrieve_context(question, top_k):\n",
    "    context = \"\"\n",
    "    documentURI = \"\" \n",
    "    client = boto3.client('kendra')\n",
    "    response = client.retrieve(IndexId=kendra_index_id, QueryText=question, PageSize=top_k)\n",
    "    \n",
    "    for query_result in response[\"ResultItems\"]:\n",
    "        context = context + query_result[\"Content\"] + \"\\n\\n\"\n",
    "        documentURI = documentURI + query_result[\"DocumentURI\"] + \"\\n\"\n",
    " \n",
    "    return context.strip(), documentURI.strip()\n",
    "\n",
    "# test retrieve_context() function\n",
    "input_question = \"Is Amazon SageMaker a machine-learning service?\"\n",
    "context, documentURI = retrieve_context(question=input_question, top_k=1) \n",
    "print(\"Question:\", input_question)\n",
    "print(\"Context:\", context)\n",
    "print(\"URI:\", documentURI)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b12a1b04-5d57-43b5-b722-6273ddb7775a",
   "metadata": {},
   "source": [
    "### Step 4: Define LLM prompt function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0426d791-95fa-43b0-8a86-85a953623152",
   "metadata": {},
   "outputs": [],
   "source": [
    "def prompt_model(question, context):\n",
    "    prompt = \"\"\n",
    "\n",
    "    if context:\n",
    "        prompt = [[\n",
    "            {\"role\": \"system\", \"content\": f\"\"\"{context}\\n\\nUse the above paragraph(s) to answer the user question.\"\"\"},\n",
    "            {\"role\": \"user\", \"content\": question},\n",
    "        ]]\n",
    "    else:\n",
    "        prompt = [[\n",
    "            {\"role\": \"user\", \"content\": question},\n",
    "        ]]\n",
    "        \n",
    "    payload = {\n",
    "        \"inputs\": prompt,\n",
    "        \"parameters\": {\n",
    "            \"max_new_tokens\": 512,\n",
    "            \"temperature\": 0.6,\n",
    "            \"top_p\": 0.9\n",
    "        }\n",
    "    }\n",
    "    \n",
    "    response = predictor.predict(payload, custom_attributes='accept_eula=false')    \n",
    "    return response[0][\"generation\"]['content'].strip()\n",
    "\n",
    "# test prompt_model() function\n",
    "question = \"What is Amazon SageMaker?\"\n",
    "context = \"Amazon SageMaker is a machine-learning service.\"\n",
    "response = prompt_model(question=question, context=context)\n",
    "print(\"Question:\", question)\n",
    "print(\"Context:\", context)\n",
    "print(\"Response:\", response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ec254a7d-ff8a-4746-be82-3c9167ab84a7",
   "metadata": {},
   "source": [
    "### Step 4: Test LLM prompting (with and without RAG) using an interactive widget "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c5c338fe-89a2-4bc3-89af-fb168cb5b8fe",
   "metadata": {},
   "outputs": [],
   "source": [
    "import ipywidgets as widgets\n",
    "from IPython.display import display, clear_output\n",
    "\n",
    "def on_send_button_click(button):\n",
    "    input_question = input_field.value\n",
    "    documentURI = \"\"\n",
    "    context = \"\"\n",
    "        \n",
    "    if rag_check.value:\n",
    "        context, documentURI = retrieve_context(question=input_question, top_k=3) \n",
    "        response = prompt_model(question=input_question, context=context)\n",
    "    else:\n",
    "        response = prompt_model(question=input_question, context=None)\n",
    "    \n",
    "    with output:\n",
    "        print(\"Q:\", input_question)\n",
    "        print(\"A:\", response)\n",
    "        print(\"URIs:\", documentURI if documentURI else \"RAG not active!\")\n",
    "        print(\"-\"*40)\n",
    "\n",
    "    input_field.value = \"\"\n",
    "\n",
    "def on_input_field_submit(text):\n",
    "    on_send_button_click(None)\n",
    "\n",
    "# Create the input field and send button\n",
    "input_field = widgets.Text(placeholder='Type your question here...')\n",
    "rag_check = widgets.Checkbox(value=True, description='Enable RAG', indent=False)\n",
    "send_button = widgets.Button(description='Send')\n",
    "top_box = widgets.HBox([input_field, rag_check])\n",
    "bottom_box = widgets.HBox([send_button])\n",
    "v_box = widgets.VBox([top_box, bottom_box])\n",
    "output = widgets.Output()\n",
    "\n",
    "# Assign the function to the button click event and the input field submit event\n",
    "send_button.on_click(on_send_button_click)\n",
    "input_field.on_submit(on_input_field_submit)\n",
    "\n",
    "# Display the chat interface\n",
    "display(output, v_box)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3918c93-72c9-4414-92d1-975121da9906",
   "metadata": {
    "tags": []
   },
   "source": [
    "Here are some sample questions to get you started:\n",
    "- What are the instance types recommended for training in SageMaker?\n",
    "- Can Amazon Kendra extract content of images from Power Point slides?\n",
    "- Does Amazon SageMaker support any GPUs made by Microsoft?\n",
    "- Write a summary about Amazon Kendra Experience Builder.\n",
    "- What is P4d?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "09596a6c-bb5e-4cee-90f6-d0cdaf70a665",
   "metadata": {},
   "source": [
    "### SageMaker Clean up"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b14c9af3-aad3-4b40-80f5-1ee4c4410f14",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Delete the SageMaker endpoint\n",
    "predictor.delete_model()\n",
    "predictor.delete_endpoint()"
   ]
  }
 ],
 "metadata": {
  "availableInstances": [
   {
    "_defaultOrder": 0,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.t3.medium",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 1,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.t3.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 2,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.t3.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 3,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.t3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 4,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 5,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 6,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 7,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 8,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 9,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 10,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 11,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 12,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5d.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 13,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5d.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 14,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5d.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 15,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5d.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 16,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5d.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 17,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5d.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 18,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5d.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 19,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 20,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": true,
    "memoryGiB": 0,
    "name": "ml.geospatial.interactive",
    "supportedImageNames": [
     "sagemaker-geospatial-v1-0"
    ],
    "vcpuNum": 0
   },
   {
    "_defaultOrder": 21,
    "_isFastLaunch": true,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.c5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 22,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.c5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 23,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.c5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 24,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.c5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 25,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 72,
    "name": "ml.c5.9xlarge",
    "vcpuNum": 36
   },
   {
    "_defaultOrder": 26,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 96,
    "name": "ml.c5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 27,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 144,
    "name": "ml.c5.18xlarge",
    "vcpuNum": 72
   },
   {
    "_defaultOrder": 28,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.c5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 29,
    "_isFastLaunch": true,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g4dn.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 30,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g4dn.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 31,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g4dn.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 32,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g4dn.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 33,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g4dn.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 34,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g4dn.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 35,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 61,
    "name": "ml.p3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 36,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 244,
    "name": "ml.p3.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 37,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 488,
    "name": "ml.p3.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 38,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.p3dn.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 39,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.r5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 40,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.r5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 41,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.r5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 42,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.r5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 43,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.r5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 44,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.r5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 45,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.r5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 46,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.r5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 47,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 48,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 49,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 50,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 51,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 52,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 53,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.g5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 54,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.g5.48xlarge",
    "vcpuNum": 192
   },
   {
    "_defaultOrder": 55,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 56,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4de.24xlarge",
    "vcpuNum": 96
   }
  ],
  "instance_type": "ml.t3.medium",
  "kernelspec": {
   "display_name": "Python 3 (Data Science 2.0)",
   "language": "python",
   "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-east-1:081325390199:image/sagemaker-data-science-38"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}