context_based_sa.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import openai\n",
    "import os\n",
    "#from langchain.llms import CTransformers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'\\ndef load_model():\\n    # Load the locally downloaded model here\\n    llm = CTransformers(\\n        model = \"/home/txjia/work/freelance_demo/models/codellama-7b-instruct.ggmlv3.Q4_0.bin\",\\n        model_type=\"llama\",\\n        max_new_tokens = 1096,\\n        temperature = 0.2,\\n        repetition_penalty = 1.13\\n        #stream=True\\n        #gpu_layers = 10\\n    )\\n\\n    return llm\\n'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Loading the model\n",
    "'''\n",
    "def load_model():\n",
    "    # Load the locally downloaded model here\n",
    "    llm = CTransformers(\n",
    "        model = \"/home/txjia/work/freelance_demo/models/codellama-7b-instruct.ggmlv3.Q4_0.bin\",\n",
    "        model_type=\"llama\",\n",
    "        max_new_tokens = 1096,\n",
    "        temperature = 0.2,\n",
    "        repetition_penalty = 1.13\n",
    "        #stream=True\n",
    "        #gpu_layers = 10\n",
    "    )\n",
    "\n",
    "    return llm\n",
    "'''"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_completion(prompt, model=\"gpt-3.5-turbo\"):\n",
    "#def get_completion(prompt, model=load_model()):\n",
    "    messages = [{\"role\": \"user\", \"content\": prompt}]\n",
    "    response = openai.ChatCompletion.create(\n",
    "        model=model,\n",
    "        messages=messages,\n",
    "        temperature=0, # this is the degree of randomness of the model's output\n",
    "    )\n",
    "    return response.choices[0].message[\"content\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This is the task\n",
    "text = f\"\"\"\n",
    "       <Context:Any,\n",
    "    Phrase: It would be good to have free shipping when I spend more than $50.>\n",
    "\"\"\"\n",
    "\n",
    "# The tutorial for this task is included in the prompt\n",
    "prompt = f\"\"\"\n",
    "This is a context based sentiment analysis task. Here is the tutorial for it:\n",
    "<tutorial>\n",
    "Read the phrase from a customer's feedback and the context associated with the phrase to decide the sentiment category for the phrase based on the context. \n",
    "The meaning of \"Phrase\" ande \"Context\" sentiment judgments are below: \n",
    "-Phrase: A free-form customer comment in the survey. \n",
    "-Context: The question asked in the survey which is the context associated with the answered comments from the customer. they are classified into four: “positive”, “neutral”, “suggestion”, and “negative”. \n",
    "\n",
    "\n",
    "Below we will show you some questions (used as context) and possible answers(used as phrase) from different customers. \n",
    "\n",
    "Question example 1: What do you like about your experience? \n",
    "Answer 1: Service is good\n",
    "Answer 2: Speed of service\n",
    "Answer 3: Nothing, the overall service is not good.\n",
    "Answer 4: The hotel is far from the street.\n",
    "\n",
    "The sentiment for each phrase:\n",
    "Answer 1: “Service is good”  is positive\n",
    "Answer 2: “Service” is positive because it infers that the customer likes the service.\n",
    "Answer 3: “Nothing, the overall service is not good” is negative. \\\n",
    "    The customer might not agree with the question being prompted for the positive feedback and they can write anything. \\\n",
    "    In this case, they want to express their opinions that service is not good. \n",
    "Answer 4: “The hotel is far from the street” is a statement in the neutral context. \\\n",
    "    In a positive context, it implies that this statement is a positive.\n",
    "\n",
    "Tip:\n",
    "    -If the statement in a positive context sounds like the reason (“the hotel is far from the street), then it is positive as it implies the customer likes the reason as mentioned.\n",
    "    -If the statement in a positive context does not sound like the reason, then it is no clear opinion.\n",
    "\n",
    "Question example 2: What do you not like about your experience? \n",
    "Answer 1: Service is not good\n",
    "Answer 2: Speed of service\n",
    "Answer 3: Overall service is good and I have nothing to complain about.\n",
    "Answer 4: The hotel is far from the street\n",
    "\n",
    "The sentiment for each phrase:\n",
    "Answer 1: “Service is not good” is negative\n",
    "Answer 2: “Service” is negative because it infers that the customer did not like the service.\n",
    "Answer 3: “Overall service is good and I have nothing to complain about” is positive. The customer might not agree with the question being prompted for the negative feedback and they can definitely write anything. In this case, they want to express their opinions that the service is good. \n",
    "Answer 4: “The hotel is far from the street” is a statement. In a negative context, it implies that this statement is a negative sentiment. (Explanation: some guests might not like the hotel being far away from the street as it’s inconvenient to go to places)\n",
    "\n",
    "Question example 3: Please tell us why you gave the score of 10. \n",
    "<Explanation: The customer gave a score of 10 in “likelihood to recommend” question.  The follow-up question asks the reason for the rating.>\n",
    "\n",
    "Likelihood to recommend question: “How likely are you to recommend the product or service?”. Also the context can be said like please tell us why you gave the score of 10(highest high).\n",
    "Answer 1: I gave a 10 because of the service. \n",
    "Answer 2: Speed of service\n",
    "\n",
    "The answer 1 and answer 2 are all positive\n",
    "\n",
    "Question example 4: Please tell us why you gave the score of 1(lowest score). \n",
    "Answer 1: I gave a 1 because of the service.\n",
    "Answer 2: Speed of service\n",
    "\n",
    "The answer 1 and answer 2 are all negative\n",
    "\n",
    "“The service” can be positive or negative depending on the score (high or low).\n",
    "\n",
    "Question example 5: What can we do to improve your experience? \n",
    "Answer 1: I am not happy with the service.\n",
    "Answer 2: Better service/Quicker service\n",
    "Answer 3: Speed of service\n",
    "Answer 4: Overall service is good and I have nothing to complain about.\n",
    "\n",
    "The sentiment for each phrase:\n",
    "Answer 1: “I am not happy with the service.” is negative\n",
    "Answer 2: “Better Service” is a suggestion. The customer suggests the company provide better service.\n",
    "Answer 3: “Service” is a suggestion based on the context. The customer suggests the company improve the service.\n",
    "Answer 4: “Overall service is good and I have nothing to complain about” is positive. The customer might not agree with the question being prompted for the suggestion feedback. In this case, they want to express their opinions that the service is good. \n",
    "\n",
    "\n",
    "Here are four variants of context: positive, neutral, negative, and suggestion. Here are some examples of them:\n",
    "1. Positive context\n",
    " - What do you like about your experience?\n",
    " - What is the reason for the score of x? (x can be any high score)\n",
    "\n",
    "2. Neutral context\n",
    " - What is your overall experience?\n",
    " - Is there anything else you would like to ask or tell us about? \n",
    "\n",
    "3. Negative context\n",
    " - What do you NOT like about your experience?\n",
    " - What is the reason for the score of x? (x can be any low score)\n",
    "\n",
    "4. Suggestion context\n",
    " - What can we do to improve your experience?\n",
    " - What would you suggest to make your experience better?\n",
    "\n",
    "Here are the sentiment labels of the phrase on the condition of context: \n",
    "    - No Clear Opinion (Statement of Fact/Description or Neutral Sentiment)\n",
    "    - Positive (The phrase contains positive sentiment)\n",
    "    - Strongly Positive (The phrase contains strongly positive sentiment)\n",
    "    - Negative (The phrase contains negative sentiment)\n",
    "    - Strongly Negative (The phrase contains strongly negative sentiment)\n",
    "    - Suggestion  (The phrase makes suggestions to the company to take action)\n",
    "    - Mixed Opinion (The phrase contains both positive and negative sentiment)\n",
    "    - Incorrect Language or Gibberish (The phrase is not in the right language or it’s completely gibberish)\n",
    "    - Unsure: the text is clear. If the text is not clear because it’s not a complete sentence or it might require more information, choose “No Clear Opinion”\n",
    "\n",
    "Here are the examples of labelling the phrase on the condition of context.\n",
    "1. No Clear Opinion\n",
    "    In all the sub-class sutiation, all phrase are labelled as No Clear Opinion\n",
    "    1) Phrase is purely an observation, description, or a statement of fact;  \n",
    "        <Context: any, Phrase: This was our anniversary weekend get-away.>        \n",
    "        Sentiment: No clear opinion\n",
    "\n",
    "    2) the sentiment is neutral about the product or company.\n",
    "        <Context: Any, Phrase: The product is okay/fine.>        \n",
    "        Sentiment: No clear opinion       \n",
    "\n",
    "\n",
    "    3) If the phrase is not clear but mostly descriptive, or you might need more context (surrounding texts in the comment) to make the judgment, label “No clear opinion”. This usually results from poor segmentation, too many typos and the text itself does not imply any sentiment.\n",
    "        <Context: Any, Phrase: I say all that to say this.>        \n",
    "        Sentiment: No clear opinion\n",
    "    \n",
    "    4) a question with a neutral tone that don not imply any positive or negative sentiment.\n",
    "        <Context: Any, Phrase: When will the product be released?>        \n",
    "        Sentiment: No clear opinion\n",
    "\n",
    "    \n",
    "    5) The phrase describes the product is not lacking in quality or customers are not experiencing any issues or their expectations are met, \\\n",
    "        sometimes phrases with clear sentiments or in positive context and note the sentiment changes\n",
    "        <Context: Neutral,  Phrase: The cloth is not see-through!>      \n",
    "        Sentiment: No clear opinion    \n",
    "    \n",
    "    Tips:\n",
    "\n",
    "        - Adjectives like soft/hard, thin/thick, or cold/hot do not exhibit inherent sentiment and should be labeled neutral or no clear opinion as it can be either positive or negative depending on personal preferences. \n",
    "        - Adjectives like good, nice, or awesome, which are inherently positive should be labeled positive sentiment.\n",
    "        - Adjectives like bad, terrible, or lousy, which are inherently negative should be labeled negative sentiment.\n",
    "\n",
    "\n",
    "    6) The phrase only contains nouns in neutral context. The noun does not contain any positive/negative sentiment.  They become positive or negative only in positive or negative contexts but remain no clear opinion in the neutral context.\n",
    "        <Context: Neutral,   Phrase: Professionalism.>      \n",
    "        Sentiment: No clear opinion\n",
    "    \n",
    "\n",
    "    7) The sign-off language such as “thank you”, “thanks”\n",
    "        <Context: Neutral,  Phrase: Thank you REDACT_NAME>        \n",
    "        Sentiment: No clear opinion\n",
    "\n",
    "    \n",
    "2. Positive:\n",
    "Phrase contains any positive opinion from the customer regarding customer experience. Adjectives that are always associated with good customer experiences (“comfortable bed”, “fast/quick service”, “friendly/knowledgeable/helpful staff”, “inexpensive product/low cost”) in any context should be labeled positive sentiment.\n",
    "    <Context: Any, Phrase: You delivered on time.>\n",
    "    Sentiment: Positive  \n",
    "\n",
    "\n",
    "3. Strongly Positive\n",
    "    Phrase generally contains a strong tone for positive sentiment. The customer might use the modifier like \"very\", \"extremely\", \"really\" or any strong adjective like great, awesome, or best. \n",
    "    <Context:Any,  Phrase: Jimmy at the Craps table was AWESOME and a great teacher for this first-timer!>\n",
    "    Sentiment: Strongly positive\n",
    "    \n",
    "4. Negative\n",
    "    Phrase contains \n",
    "        1) any negative opinion from the customer regarding customer experience (anger, frustration, dissatisfied) or \n",
    "        2) any issues the customer is experiencing\n",
    "    Adjectives that are always associated with poor customer experiences (“uncomfortable bed”, “slow/poor service”, “rude staff”, “expensive product/high cost”, “unacceptable responses”) should be labeled negative sentiment.\n",
    "\n",
    "        <Context:Any,  Phrase: She did not seem to understand the issue or want to.>\n",
    "        Sentiment: Negative\n",
    "        \n",
    "5. Strongly Negative\n",
    "    Phrase generally contains a strong tone for negative sentiment. The customer might use the modifier like \"very\", \"extremely\", \"really\", \"worst\" or any strong adjective like worst.\n",
    "    <Context:Any, Phrase: I was very disappointed as I had been anticipating enjoying this buffet for a long time.>\n",
    "    Sentiment: Strongly negative\n",
    "\n",
    "\n",
    "6. Mixed Opinion\n",
    "    Phrase contains both positive and negative sentiments. \n",
    "    <Context:Any, Phrase: Even though the restaurant choices were limited, we enjoyed Charlie’s food so much we ate there twice in one day.>\n",
    "    Sentiment: Mixed opinion   \n",
    "\n",
    "7 Suggestion\n",
    "    Phrase contains some specific actions or suggestions that lead to fulfilling the customer's wish/intent. \n",
    "\n",
    "     - In any context, some common suggestions have patterns \"It would be nice/great/good ...\", \"you need to\", \"You have to\", \"you should\", \"I wish\", \"..would be helpful\", \"...would help..\"\n",
    "     - In a negative or suggestion context, phrases starting with comparative words (better, lower, fewer, easier, etc) imply the customers want the company to take action to make certain things better, lower, fewer, or easier, and hence it’s a suggestion.\n",
    "     - In a suggestion context, phrases with “Being able to do something” or “ability to do something” imply the customers want the company to fulfill their needs, and hence it’s a suggestion.\n",
    "     - The phrase contains “I would like to” but it’s not a suggestion to the company to take action to fulfill the customers’ needs.\n",
    "        ◦ I would like to comment that the service is slow\n",
    "        ◦ I wish I can give you a 10 but the service this time is slow\n",
    "     - The phrases are descriptions of the problem or complaints. It does not contain any suggested “action” to take.\n",
    "        ◦ The service is slow\n",
    "        ◦ The price is too high\n",
    "\n",
    "    <Context:Any,   Phrase: Provide the Spanish mobile app>\n",
    "    Sentiment: Suggestion  \n",
    "\n",
    "    \n",
    "8. Incorrect language or gibberish\n",
    "If the text is not in the correct language or the phrase does not make any sense to you (gibberish), then choose \"incorrect language or gibberish\". If it’s just the part of the sentence is gibberish but you still can interpret the meaning, choose your best sentiment judgment. \n",
    "Note that “%d” is the symbol to replace sensitive numeric data (see Step above) as we remove personal sensitive information from the data. You should interpret them as some numbers instead of gibberish.\n",
    "    \n",
    "    <Context:Any,  Phrase: jjo>\n",
    "    Sentiment: gibberish    \n",
    "\n",
    "9. Unsure\n",
    "    The text is clear to you but you don’t know how to annotate it, that is the probability of being divided into any category is almost equal\n",
    "\n",
    "Above is all about the tutorial,\n",
    "</tutorial>\n",
    "\n",
    "Please do the context based sentiment analysis:\n",
    "    - Read the phrase and context. Due to data privacy and security reasons, we replace the company name with REDACT_COMPANY, emails with REDACT_EMAIL and digit with '#', so you might see repeating '###' in the comment. \n",
    "    - The sentiment category should one of No Clear Opinion,positive, strongly Positive, negative, strongly Negative, suggestion, mixed Opinion, incorrect Language or Gibberish,and Unsure.\n",
    "\n",
    "The context and phrase: ```{text}```\n",
    "\"\"\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The sentiment category for the given phrase and context is \"Suggestion\".\n"
     ]
    }
   ],
   "source": [
    "response = get_completion(prompt)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "llm311",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}