21-AdvancedPython.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "**Course Announcements**\n",
    "\n",
    "Due Dates:\n",
    "- Final Project/Exam (Fri 6/11 at 11:59 PM)\n",
    "    - Practice Final exam now available on datahub  (answer key will be posted this weekend)\n",
    "\n",
    "\n",
    "Notes:\n",
    "- Please fill out your CAPEs (+1% if >=85% response rate) - current: ~30%\n",
    "- Coding Labs - nothing to turn in; ~OH; get your questions answered\n",
    "- Additional OH will be Wed of Finals Week \n",
    "    - 10 min slot by appt\n",
    "    - 9AM-noon; 2-3 PM\n",
    "    - details are pinned on Piazza\n",
    "    - will use a different zoom link than normal"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "source": [
    "**Q&A**\n",
    "\n",
    "Q: Must we use the terminal in tandem with the notebook? Or can we simply use the notebook? What is the best way to go about this project?  \n",
    "A: Yup - you can. You're encouraged to approach the project however you're most comfortable. Many students are most comfortable in the notebook, so we encourage you to start there (like we are doing in lecture) and move to a .py file from there. However, if you're more comfortable/have experience in a different environment, feel free to use that!\n",
    "\n",
    "Q: Will you be creating new files for the exam or will they already be in the folder and we will have to be adding code inside of them?  \n",
    "A: I will create them and have them in the folder for you. You will then add/edit code in them.\n",
    "\n",
    "Q: From the project we were working on during lecture, would it have been possible to keep punctuations and have them be printed out in the output? Would we have just changed the part of the loop where we said it would be == None ?  \n",
    "A: Yup! A conditional could be used to control this. At the end of the projects set of notes, we're going to discuss improvements. This is one of them! \n",
    "\n",
    "Q: I am curious on how extensive the final exam will be in terms of how many functions or classes will need to be written, since the example from this lecture only had 3 functions.  \n",
    "A: It will be similar to example in class. You can expect to write 2 functions and debug/fix code style on a third. You will also have to write code tests. The code written on the final exam is *not* super extensive.\n",
    "\n",
    "Q: I was confused about refactoring and if it only involves a chatbot or is that most commonly used?  \n",
    "A: Refactoring is *not* specific to chatbots. Rather, it's specific to code - we'll discuss this more in today's lecture.\n",
    "\n",
    "Q: When do we need to use numpy method, and when we can just use the normal python method?  \n",
    "A: When you want to operate on matrices/arrays, you'll need `numpy`.\n",
    "\n",
    "Q: Why do we need to use the terminal if we can use the notebook for most things? The purpose of the terminal is somewhat confusing.  \n",
    "A: Notebooks are big and bulky. The terminal is faster and allows for streamlined organization. You won't *need* to use the terminal, but it can save you *a lot* of time.\n",
    "\n",
    "Q: Do we have to use a separate file for the tests we write? Can we just include them in the Jupyter notebook?  \n",
    "A: They must be in a separate file. We will be using `pytest` on that file.\n",
    "\n",
    "Q: How do you create a new module and import the code on Jupyter?  \n",
    "A: You create a .py file in the same directory as your notebook. You store code in that .py file. Then, you can import it into the notebook.\n",
    "\n",
    "Q: For the atbash example, if we write those three functions in a module, would it be more efficient to write `from atbash import *` than `from atbash import atbash_encrypt, atbash_decrypt, atbash_wrapper`? \n",
    "\n",
    "A: While that would be more efficient, it would also be unclear in your code where those functions you're using came from. Instead, you could use `import atbash as ab` and then use `ab.` to refer to your functions in your code, if you prefer.\n",
    "\n",
    "Q: How does running pytest in the terminal rather than the notebook differ?  \n",
    "A: It doesn't. Anytime you start with `!` in the notebook, it's the same as running from the terminal.\n",
    "\n",
    "Q: Do we get a feedback if we submit the project early and can we fix it again and resubmit? Thank you so much  \n",
    "A: No sorry - we just won't have the capacity to do this for everyone; however, we could answer questions/look it over quickly in office hours?\n",
    "\n",
    "Q: I'm confused how to figure out what tests to use.  \n",
    "A: Think about what your function/class/method does or should do, and be sure you test that given an input, the expected output is returned. Those are the most important tests.\n",
    "\n",
    "Q: How extensive should our tests be?  \n",
    "A: They should test the functionality of your class/method/function. They do not have to consider all edge cases (if this were an intermediate python course, that answer would be different and tests would need to be more involved).\n",
    "\n",
    "Q: I want to ask why my Sublime page looks different with our professor's Sublime. Also, I want to ask where should we save the sublime file?  \n",
    "A: I have different settings specified to color my code the way I like it. You can Google how to change code highliting in Sublime to change your's too!\n",
    "\n",
    "Q: Is it better to poorly refactor a code or to just leave it as is when it is totally functioning.?   \n",
    "A: First priority: functioning code; Second priority: refactored, clean code.\n",
    "\n",
    "Q: I am interested in finding more \"assignments\" to do to get more familiar with python after the conclusion of this class. Are there any public assignments from previous Cogs 18 courses that are viewable?  \n",
    "A: Not really, *but* I'm going to be editing [this](https://shanellis.github.io/pythonbook/content/intro.html) over the summer to include lots of old COGS 18 examples in the exercises and to add an answer key, so there should be come Fall!\n",
    "\n",
    "Q: For a first time coder how do you suggest I go about the coding project to not stress myself out entirely and question everything I am doing?   \n",
    "A: Go in with a plan and *think* before doing. New programmers too often just \"try everything\" before stepping back and thinking about what they want to do and what they need to accomplish that.\n",
    "\n",
    "Q: If we decide to do the final exam, will we have to do our own assertion tests?  \n",
    "A: Yes.\n",
    "\n",
    "Q: Are we allowed to use other imported functions in our code?  \n",
    "A: Yup! On the final project, it's up to you!\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Advanced Python\n",
    "\n",
    "- string formatting (`format`)\n",
    "- sets\n",
    "- `collections`\n",
    "- args & kwargs\n",
    "- list comprehensions\n",
    "- map\n",
    "- APIs\n",
    "- overloading operators"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "...or: a day of shortcuts\n",
    "\n",
    "Note: Shortcuts make *your* life easier, but your code *less* understandable to beginners."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "It's hard to search for things you don't know exist.\n",
    "\n",
    "Today's goal : let you know these things exist.\n",
    "\n",
    "NOT today's goal : teach you all the details."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "You do *not* need to incorporate these into your project, but if they'll help you, go for it!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "import antigravity"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## String Formatting"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Use the `format` method to manipulate and format strings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Hello! My name is Shannon. My job is Assistant Teaching Professor, and I work on Python'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "name = \"Shannon\"\n",
    "job = \"Assistant Teaching Professor\"\n",
    "topic = \"Python\"\n",
    "\n",
    "\"Hello! My name is {}. My job is {}, and I work on {}\".format(name, job, topic)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Sets "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Sets are a variable type that store **only unique entries**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{1, 2, 3, 4}"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "my_set = set([1, 1, 2, 3, 4])\n",
    "my_set"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Like lists, they are iterable & mutable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{1, 2, 3, 4, 6}"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "my_set.add(6)\n",
    "my_set"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#### Clicker Question #1\n",
    "\n",
    "How many values would be in the following set?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{1, 2, 3, 4, 6}"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clicker_set = set([1, 1, 2, 2, 3, 4])\n",
    "clicker_set.add(6)\n",
    "clicker_set.add(1)\n",
    "clicker_set"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- A) 0\n",
    "- B) 4\n",
    "- C) 5\n",
    "- D) 6\n",
    "- E) ¯\\\\\\_(ツ)\\_/¯"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "source": [
    "Reminder that if you add a value that is already in the set...the set will not change"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Collections"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "The `collections` module has a number of useful variable types, with special properties. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "\"Special editions\" of collections we've discussed before (lists, tuples, and dictionaries)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "from collections import Counter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Counter({1: 3, 0: 1, 2: 1})"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# count how many elements there are in collection\n",
    "# return values in a dictionary\n",
    "Counter([1, 0, 1, 2, 1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "import collections"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Counter({1: 3, 0: 1, 2: 1})"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "collections.Counter([1, 0, 1, 2, 1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Counter({'I': 2,\n",
       "         ' ': 9,\n",
       "         'w': 2,\n",
       "         'o': 2,\n",
       "         'n': 2,\n",
       "         'd': 1,\n",
       "         'e': 7,\n",
       "         'r': 2,\n",
       "         'h': 2,\n",
       "         'm': 2,\n",
       "         'a': 1,\n",
       "         'y': 1,\n",
       "         't': 4,\n",
       "         'i': 1,\n",
       "         's': 2,\n",
       "         'u': 1,\n",
       "         'l': 1,\n",
       "         \"'\": 2})"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# can count how many of each letter are in there\n",
    "Counter(\"I wonder how many times I use the letter 'e'\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Think back to our encryption scheme! \n",
    "\n",
    "The most common letter in the English language is 'e'.\n",
    "\n",
    "If you just move each letter over by 1 position, you can just use a Counter to crack the code.\n",
    "\n",
    "Why we moved to a variable encoder!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## `any` & `all`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "`any` and `all` evaluate collections for whether elements evaluate as True. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "my_bool_list = [True, False, True]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "any(my_bool_list)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all(my_bool_list)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#### Clicker Question #2\n",
    "\n",
    "What would the following return?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "my_list = [True, True, False, True]\n",
    "any(my_list)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- A) `True`\n",
    "- B) `False`\n",
    "- C) `[True, True, True]`\n",
    "- D) `[False]`\n",
    "- E) ¯\\\\\\_(ツ)\\_/¯"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## `getattr` & `setattr`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "`getattr` and `setattr` can be used to get and set attributes, respectively. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Flexibly return and define instance attributes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "class MyClass():\n",
    "    \n",
    "    def __init__(self):        \n",
    "        self.var_a = 'string'\n",
    "        self.var_b = 12\n",
    "\n",
    "# create instance \n",
    "instance = MyClass()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'string'"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "instance.var_a"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'string'"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Get a specified attribute\n",
    "# define var_a as an input\n",
    "getattr(instance, 'var_a')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "13\n"
     ]
    }
   ],
   "source": [
    "# Set a specified attribute\n",
    "setattr(instance, 'var_b', 13)\n",
    "print(instance.var_b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## `args` & `kwargs`\n",
    "\n",
    "- allow you to pass a variable number of arguments to a function\n",
    "    - `*args` - allow any number of extra arguments (including zero)\n",
    "    - `**kwargs` - allow any number of *keyword* arguments to be passed (as a dictionary)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "### `*args`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "# use *arguments to demonstrate args\n",
    "def tell_audience(*arguments):\n",
    "    for arg in arguments:\n",
    "        print(arg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "asdf\n"
     ]
    }
   ],
   "source": [
    "tell_audience('asdf')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Hello!\n",
      "My name is Shannon.\n",
      "My job is Assistant Teaching Professor.\n"
     ]
    }
   ],
   "source": [
    "tell_audience(\"Hello!\", \n",
    "              \"My name is Shannon.\", \n",
    "              \"My job is Assistant Teaching Professor.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "### `**kwargs`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "def tell_audience_kwargs(**info):\n",
    "    print(\"type: \", type(info), '\\n')\n",
    "    \n",
    "    for key, value in info.items():\n",
    "        print(\"key: \", key)\n",
    "        print(\"value: \", value, \"\\n\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "type:  <class 'dict'> \n",
      "\n",
      "key:  first\n",
      "value:  Shannon \n",
      "\n",
      "key:  last\n",
      "value:  Ellis \n",
      "\n",
      "key:  email\n",
      "value:  sellis@ucsd.edu \n",
      "\n"
     ]
    }
   ],
   "source": [
    "tell_audience_kwargs(first='Shannon', \n",
    "                     last='Ellis', \n",
    "                     email='sellis@ucsd.edu')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## List Comprehensions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "List comprehensions allow you to run loops and conditionals, *inside lists*."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The general format is :\n",
    "\n",
    "`[ expression for item in list if conditional ]`\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1, 2, 3]"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# NOT a list comprehension\n",
    "# how we've been doing it\n",
    "input_list = [0, 1, 2]\n",
    "output_list = []\n",
    "\n",
    "for ind in input_list:\n",
    "    output_list.append(ind + 1)\n",
    "    \n",
    "# look at output\n",
    "output_list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1, 2, 3]"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# This list comprehension is equivalent to the cell above\n",
    "# note the square brackets on the outside\n",
    "[ind + 1 for ind in [0, 1, 2]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# You can also include conditionals inside a list comprehension\n",
    "my_list = [1, 2, 3, 4, 5]\n",
    "my_list2 = [val for val in my_list if val % 2 == 0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1, 2, 3, 4, 5]\n",
      "[2, 4]\n"
     ]
    }
   ],
   "source": [
    "print(my_list)\n",
    "print(my_list2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#### Clicker Question #3\n",
    "\n",
    "What would the following return?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[9, 12, 15]"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list1 = [3, 4, 5]\n",
    "multiplied = [item * 3 for item in list1] \n",
    "multiplied"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- A) `True`\n",
    "- B) `False`\n",
    "- C) `[9, 12, 15]`\n",
    "- D) `15`\n",
    "- E) ¯\\\\\\_(ツ)\\_/¯"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Note: there are also tuple & dictionary comprehensions. They're just used less frequently."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Regular Expressions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Regular expressions allow you to work with **specified patterns in strings**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "import re"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "my_string = \"If 12 Python programmers try to complete 14 tasks in 16 minutes, why?\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['o', 'o', 'o', 'o']"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# can just search for a letter\n",
    "re.findall('o', my_string)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['12', '14', '16']"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# but the patterns are where these shine\n",
    "re.findall('\\d\\d', my_string)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## filter"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "`filter` is a function that takes in another function, and a collection object, and **filters the collection with the function**. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "def is_bool(val):\n",
    "    if isinstance(val, bool):\n",
    "        return True\n",
    "    else:\n",
    "        return False"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#filter?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[False, True, False, True]"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "out = []\n",
    "\n",
    "for val in [1, False, 's', True]:\n",
    "    out.append(is_bool(val))\n",
    "\n",
    "out"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[False, True]"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# apply function to every element of list\n",
    "# function to apply is first input to filter\n",
    "list(filter(is_bool, [1, False, 's', True]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Lambda Functions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Lambda functions are small, anonymous functions. \n",
    "\n",
    "Can define them in line."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "increment = lambda a: a + 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "increment(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# not a lambda function\n",
    "def lambda_equivalent(a):\n",
    "    \n",
    "    output = a + 1\n",
    "    \n",
    "    return output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lambda_equivalent(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#### Clicker Question #4\n",
    "\n",
    "What would the following return?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[4, 6, 8, 12]"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "my_list = [1, 5, 4, 6, 8, 11, 3, 12]\n",
    "new_list = list(filter(lambda x: (x%2 == 0), my_list))\n",
    "new_list"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- A) `[1, 5, 11, 3]`\n",
    "- B) `[1, 3, 5, 11]`\n",
    "- C) `[4, 6, 8, 12]`\n",
    "- D) `[1, 5, 4, 6, 8, 11, 3, 12]`\n",
    "- E) ¯\\\\\\_(ツ)\\_/¯"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## map"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "`map` applies a given function to each element in a collection. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# create a function\n",
    "def double(val):\n",
    "    return val * 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[2, 4, 6, 8]"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# map function to each element of list\n",
    "my_list = [1, 2, 3, 4]\n",
    "list(map(double, my_list))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[2, 4, 6, 8]"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Note - we can use lambda functions with map\n",
    "list(map(lambda x: x *2, my_list))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Conditional Assignment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "You can use a conditional within an assignment, to manipulate what gets assigned given some condition. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# variable assignment\n",
    "condition = False\n",
    "my_var = 1 if condition else 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2\n"
     ]
    }
   ],
   "source": [
    "print(my_var)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Unpacking"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "You can unpack collection objects with `*`, and key, value pairs with `**`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "def my_func(xx, yy, zz):\n",
    "    print(xx, yy, zz)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# create list\n",
    "my_list = [1, 2, 3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "my_func() missing 2 required positional arguments: 'yy' and 'zz'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-54-e14a904b0af2>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;31m# this will error\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mmy_func\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmy_list\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m: my_func() missing 2 required positional arguments: 'yy' and 'zz'"
     ]
    }
   ],
   "source": [
    "# this will error\n",
    "my_func(my_list)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1 2 3\n"
     ]
    }
   ],
   "source": [
    "# unpack the list\n",
    "my_func(*my_list)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# create dictionary\n",
    "my_dict = {'xx' : 1, 'yy' : 2, 'zz' : 3}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1 2 3\n"
     ]
    }
   ],
   "source": [
    "# unpack the dictionary\n",
    "my_func(**my_dict)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Application Programming Interface"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "<div class=\"alert alert-success\">\n",
    "An <b>API</b> is a programmatic interface to an application - a software to software interface, or way for programs to talk to other programs. \n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "If you are pointing and clicking on computer applications, that is _not_ an API.\n",
    "\n",
    "If you are writing code to interact with software and get information, that _IS_ an API."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Module APIs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Modules have an API. Every time you write or use a set of functions and/or classes, you are writing or using an API. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Web APIs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "APIs are an interface to interact with an application, _designed for programmatic use_ :\n",
    "- They allow systematic, controlled access to (for example) an applications database and procedures\n",
    "- They can be used to request data and/or to request that the the application perform some procedure"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### RESTful API\n",
    "\n",
    "(Representational State Transfer API) \n",
    "\n",
    "An approach to interact with web addresses"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Twitter API"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "outputs": [],
   "source": [
    "# to use this on your computer\n",
    "# uncomment and run following line\n",
    "# !pip install tweepy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "source": [
    "Then, follow the instructions [here](https://www.digitalocean.com/community/tutorials/how-to-authenticate-a-python-application-with-twitter-using-tweepy-on-ubuntu-14-04) for authetication of tweepy with Python."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "scrolled": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Alberto Cairo\n",
      "RT @FeliciaKnaul: Check our +recent publications on failure of public policies for the containment of #COVID19 in #Brasil and #México Subna… \n",
      "\n",
      "Dr. Karen Kelsky\n",
      "RT @Reshmi777: &lt;Here is my latest: “An Extraordinary Firing.” &gt;Terminations like Universities that fire tenured faculty without due process… \n",
      "\n",
      "Sherry Pagoto\n",
      "Love this course assignment to help students learn to debunk misinformation! ⬇️ https://t.co/NgADg5SbRk \n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Accessing Twitter API from Python\n",
    "#  Note: to run this, you will have to fill in stw.py with your OAuth credentials.\n",
    "#    You can do that here: https://apps.twitter.com/\n",
    "\n",
    "# Import tweepy to access API\n",
    "import tweepy\n",
    "from tweepy import OAuthHandler\n",
    "\n",
    "# Import my API credentials\n",
    "from stw import *\n",
    "\n",
    "# Twitter API requires Authentification with OAuth\n",
    "auth = OAuthHandler(CONSUMER_KEY, CONSUMER_SECRET)\n",
    "auth.set_access_token(ACCESS_TOKEN, ACCESS_TOKEN_SECRET)\n",
    "\n",
    "# # Create an API object to access Twitter\n",
    "api = tweepy.API(auth)\n",
    "\n",
    "for status in tweepy.Cursor(api.home_timeline).items(3):\n",
    "    # Process a single status\n",
    "    print(status.user.name)\n",
    "    print(status.text, '\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Overloading Operators"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "In other words: Nothing in Python is sacred."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Operators and special operations for objects are defined by double underscore methods, which we can overload to change how the object acts. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "class Language():\n",
    "    \n",
    "    def __init__(self, name):\n",
    "        self.name = name.lower()\n",
    "    \n",
    "    # Overload what the printed representation of the object as a string\n",
    "    def __repr__(self):\n",
    "        return \"The programming language \" + self.name\n",
    "    \n",
    "    # Overload the greater than symbol\n",
    "    def __gt__(self, other):\n",
    "        return True if self.name == 'python' else False"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "Note: \"dunder\" is how we refer to those double underscore methods"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "python = Language('python')\n",
    "perl = Language('perl')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The programming language python\n"
     ]
    }
   ],
   "source": [
    "print(python)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "python > perl"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  },
  "rise": {
   "scroll": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}