Skip to content

Commit

Permalink
Prompt Tuning With PEF - Modified Image Directory to use it from coob…
Browse files Browse the repository at this point in the history 
…ook-images dataset
  • Loading branch information
@peremartra
peremartra committed Feb 18, 2024
1 parent 562e6d7 commit 025bd73
Showing 1 changed file with 28 additions and 55 deletions.
83 changes: 28 additions & 55 deletions notebooks/en/Prompt_Tuning_PEFT.ipynb
View file
Original file line number Diff line number Diff line change
@@ -1,32 +1,5 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"jp-MarkdownHeadingCollapsed": true,
"id": "oCA0sm55VaS-"
},
"source": [
"<div align=\"center\">\n",
"<h1><a href=\"https://github.com/peremartra/Large-Language-Model-Notebooks-Course\">LLM Hands On Course</a></h1>\n",
" <h3>Understand And Apply Large Language Models</h3>\n",
" <h2>Introduction to Prompt Tuning using PEFT from Hugging Face.</h2>\n",
" <h3>Fine-tune a Foundational Model effortless</h3>\n",
" <p>by <b>Pere Martra</b></p>\n",
"</div>\n",
"\n",
"<br>\n",
"\n",
"<div align=\"center\">\n",
" &nbsp;\n",
" <a target=\"_blank\" href=\"https://www.linkedin.com/in/pere-martra/\"><img src=\"https://img.shields.io/badge/style--5eba00.svg?label=LinkedIn&logo=linkedin&style=social\"></a>\n",
" \n",
"</div>\n",
"\n",
"<br>\n",
"<hr>"
]
},
{
"cell_type": "markdown",
"metadata": {
Expand All @@ -41,7 +14,7 @@
},
"source": [
"# Prompt Tuning\n",
"In this notebook I'm introducing how to apply prompt tuning with the PEFT library to a pre-trained model.\n",
"In this notebook we are introducing how to apply prompt tuning with the PEFT library to a pre-trained model.\n",
"\n",
"For a complete list of models compatible with PEFT refer to their [documentation](https://huggingface.co/docs/peft/main/en/index#supported-methods).\n",
"\n",
Expand All @@ -52,7 +25,7 @@
"\n",
"Considering it’s an Additive technique and its name is Prompt-Tuning, it seems clear that the layers we’re going to add and train are related to the prompt.\n",
"\n",
"![My Image](https://github.com/peremartra/Large-Language-Model-Notebooks-Course/blob/main/img/Martra_Figure_5_Prompt_Tuning.jpg?raw=true)\n",
"![Prompt_Tuning_Diagram](https://huggingface.co/datasets/huggingface/cookbook-images/resolve/main/Martra_Figure_5_Prompt_Tuning.jpg)\n",
"\n",
"We are creating a type of superprompt by enabling a model to enhance a portion of the prompt with its acquired knowledge. However, that particular section of the prompt cannot be translated into natural language. **It's as if we've mastered expressing ourselves in embeddings and generating highly effective prompts.**\n",
"\n",
Expand Down Expand Up @@ -80,7 +53,7 @@
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": null,
"metadata": {
"application/vnd.databricks.v1+cell": {
"cellMetadata": {},
Expand Down Expand Up @@ -122,7 +95,7 @@
"base_uri": "https://localhost:8080/"
}
},
"execution_count": 2,
"execution_count": null,
"outputs": [
{
"output_type": "stream",
Expand All @@ -148,7 +121,7 @@
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": null,
"metadata": {
"application/vnd.databricks.v1+cell": {
"cellMetadata": {},
Expand Down Expand Up @@ -179,7 +152,7 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": null,
"metadata": {
"id": "MnqIhv2UVaTC"
},
Expand All @@ -193,7 +166,7 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": null,
"metadata": {
"id": "fSMu3qRsVaTC",
"outputId": "520da7df-a7fd-4be1-b69b-d5a315418c26",
Expand Down Expand Up @@ -362,7 +335,7 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": null,
"metadata": {
"id": "47j2D3WWVaTC"
},
Expand Down Expand Up @@ -405,7 +378,7 @@
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": null,
"metadata": {
"application/vnd.databricks.v1+cell": {
"cellMetadata": {},
Expand Down Expand Up @@ -446,7 +419,7 @@
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": null,
"metadata": {
"id": "1Xhm3jZMVaTD",
"outputId": "7ea55d99-bdb5-42c6-dd18-33397b7a1800",
Expand Down Expand Up @@ -493,7 +466,7 @@
},
{
"cell_type": "code",
"execution_count": 9,
"execution_count": null,
"metadata": {
"id": "RD8H_LLaVaTD"
},
Expand All @@ -505,7 +478,7 @@
},
{
"cell_type": "code",
"execution_count": 10,
"execution_count": null,
"metadata": {
"application/vnd.databricks.v1+cell": {
"cellMetadata": {},
Expand Down Expand Up @@ -714,7 +687,7 @@
"base_uri": "https://localhost:8080/"
}
},
"execution_count": 11,
"execution_count": null,
"outputs": [
{
"output_type": "stream",
Expand All @@ -727,7 +700,7 @@
},
{
"cell_type": "code",
"execution_count": 12,
"execution_count": null,
"metadata": {
"id": "WeM66LmEVaTD",
"outputId": "f8bc65f6-5a95-4432-eedd-f3dfee2fa0f5",
Expand Down Expand Up @@ -937,7 +910,7 @@
},
{
"cell_type": "code",
"execution_count": 13,
"execution_count": null,
"metadata": {
"application/vnd.databricks.v1+cell": {
"cellMetadata": {},
Expand Down Expand Up @@ -972,7 +945,7 @@
},
{
"cell_type": "code",
"execution_count": 14,
"execution_count": null,
"metadata": {
"id": "c_D8oDQZVaTD",
"outputId": "a4bf353d-db50-4d62-d167-c22835bcd3be",
Expand All @@ -997,7 +970,7 @@
},
{
"cell_type": "code",
"execution_count": 15,
"execution_count": null,
"metadata": {
"id": "IktYfj68VaTE",
"outputId": "e8b93e58-6c89-4f5f-d0cb-6d1ecaa3f9e8",
Expand Down Expand Up @@ -1040,7 +1013,7 @@
},
{
"cell_type": "code",
"execution_count": 16,
"execution_count": null,
"metadata": {
"id": "SJoznfzjVaTE"
},
Expand All @@ -1060,7 +1033,7 @@
},
{
"cell_type": "code",
"execution_count": 17,
"execution_count": null,
"metadata": {
"application/vnd.databricks.v1+cell": {
"cellMetadata": {},
Expand Down Expand Up @@ -1103,7 +1076,7 @@
},
{
"cell_type": "code",
"execution_count": 18,
"execution_count": null,
"metadata": {
"id": "D4v4RSSeVaTE"
},
Expand Down Expand Up @@ -1133,7 +1106,7 @@
},
{
"cell_type": "code",
"execution_count": 19,
"execution_count": null,
"metadata": {
"id": "uVAfNdEIVaTE"
},
Expand All @@ -1152,7 +1125,7 @@
},
{
"cell_type": "code",
"execution_count": 20,
"execution_count": null,
"metadata": {
"application/vnd.databricks.v1+cell": {
"cellMetadata": {},
Expand Down Expand Up @@ -1222,7 +1195,7 @@
},
{
"cell_type": "code",
"execution_count": 21,
"execution_count": null,
"metadata": {
"id": "afTotMckVaTF",
"outputId": "e405d99d-abac-4fca-83e5-84443cbfb9ae",
Expand Down Expand Up @@ -1304,7 +1277,7 @@
},
{
"cell_type": "code",
"execution_count": 22,
"execution_count": null,
"metadata": {
"application/vnd.databricks.v1+cell": {
"cellMetadata": {},
Expand Down Expand Up @@ -1341,7 +1314,7 @@
},
{
"cell_type": "code",
"execution_count": 23,
"execution_count": null,
"metadata": {
"application/vnd.databricks.v1+cell": {
"cellMetadata": {},
Expand All @@ -1364,7 +1337,7 @@
},
{
"cell_type": "code",
"execution_count": 24,
"execution_count": null,
"metadata": {
"application/vnd.databricks.v1+cell": {
"cellMetadata": {},
Expand Down Expand Up @@ -1416,7 +1389,7 @@
},
{
"cell_type": "code",
"execution_count": 25,
"execution_count": null,
"metadata": {
"id": "MuwAsq3uVaTG"
},
Expand All @@ -1430,7 +1403,7 @@
},
{
"cell_type": "code",
"execution_count": 26,
"execution_count": null,
"metadata": {
"id": "IQm--PWSVaTH",
"outputId": "2d86bd1c-2832-4ed3-9f1a-83c4e8b17dd6",
Expand Down

0 comments on commit 025bd73

Please sign in to comment.