about_py.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "<a id='about-py'></a>\n",
    "<div id=\"qe-notebook-header\" align=\"right\" style=\"text-align:right;\">\n",
    "        <a href=\"https://quantecon.org/\" title=\"quantecon.org\">\n",
    "                <img style=\"width:250px;display:inline;\" width=\"250px\" src=\"https://assets.quantecon.org/img/qe-menubar-logo.svg\" alt=\"QuantEcon\">\n",
    "        </a>\n",
    "</div>\n",
    "\n",
    "\n",
    "<a id='index-0'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# About Python"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Contents\n",
    "\n",
    "- [About Python](#About-Python)  \n",
    "  - [Overview](#Overview)  \n",
    "  - [What’s Python?](#What’s-Python?)  \n",
    "  - [Scientific Programming](#Scientific-Programming)  \n",
    "  - [Learn More](#Learn-More)  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overview\n",
    "\n",
    "In this lecture we will\n",
    "\n",
    "- Outline what Python is  \n",
    "- Showcase some of its abilities  \n",
    "- Compare it to some other languages  \n",
    "\n",
    "\n",
    "At this stage, it’s **not** our intention that you try to replicate all you see.\n",
    "\n",
    "We will work through what follows at a slow pace later in the lecture series.\n",
    "\n",
    "Our only objective for this lecture is to give you some feel of what Python is, and what it can do."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What’s Python?\n",
    "\n",
    "[Python](https://www.python.org) is a general-purpose programming language conceived in 1989 by Dutch programmer [Guido van Rossum](https://en.wikipedia.org/wiki/Guido_van_Rossum).\n",
    "\n",
    "Python is free and open source, with development coordinated through the [Python Software Foundation](https://www.python.org/psf/).\n",
    "\n",
    "Python has experienced rapid adoption in the last decade and is now one of the most popular programming languages."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Common Uses\n",
    "\n",
    "Python is a general-purpose language used in almost all application domains\n",
    "\n",
    "- communications  \n",
    "- web development  \n",
    "- CGI and graphical user interfaces  \n",
    "- games  \n",
    "- multimedia, data processing, security, etc., etc., etc.  \n",
    "\n",
    "\n",
    "Used extensively by Internet service and high tech companies such as\n",
    "\n",
    "- [Google](https://www.google.com/)  \n",
    "- [Dropbox](https://www.dropbox.com/)  \n",
    "- [Reddit](https://www.reddit.com/)  \n",
    "- [YouTube](https://www.youtube.com/)  \n",
    "- [Walt Disney Animation](https://pydanny-event-notes.readthedocs.org/en/latest/socalpiggies/20110526-wda.html), etc., etc.  \n",
    "\n",
    "\n",
    "Often used to [teach computer science and programming](http://cacm.acm.org/blogs/blog-cacm/176450-python-is-now-the-most-popular-introductory-teaching-language-at-top-us-universities/fulltext).\n",
    "\n",
    "For reasons we will discuss, Python is particularly popular within the scientific community\n",
    "\n",
    "- academia, NASA, CERN, Wall St., etc., etc.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Relative Popularity\n",
    "\n",
    "The following chart, produced using Stack Overflow Trends, shows one measure of the relative popularity of Python\n",
    "\n",
    "<img src=\"https://s3-ap-southeast-2.amazonaws.com/lectures.quantecon.org/py/_static/lecture_specific/about_py/python_vs_rest.png\" style=\"\">\n",
    "\n",
    "  \n",
    "The figure indicates not only that Python is widely used but also that adoption of Python has accelerated significantly since 2012.\n",
    "\n",
    "We suspect this is driven at least in part by uptake in the scientific\n",
    "domain, particularly in rapidly growing fields like data science.\n",
    "\n",
    "For example, the popularity of [pandas](http://pandas.pydata.org/), a library for data analysis with Python has exploded, as seen here.\n",
    "\n",
    "(The corresponding time path for MATLAB is shown for comparison)\n",
    "\n",
    "<img src=\"https://s3-ap-southeast-2.amazonaws.com/lectures.quantecon.org/py/_static/lecture_specific/about_py/pandas_vs_matlab.png\" style=\"\">\n",
    "\n",
    "  \n",
    "Note that pandas takes off in 2012, which is the same year that we seek\n",
    "Python’s popularity begin to spike in the first figure.\n",
    "\n",
    "Overall, it’s clear that\n",
    "\n",
    "- Python is [one of the most popular programming languages worldwide](http://spectrum.ieee.org/computing/software/the-2017-top-programming-languages).  \n",
    "- Python is a major tool for scientific computing, accounting for a rapidly rising share of scientific work around the globe.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Features\n",
    "\n",
    "Python is a [high-level language](https://en.wikipedia.org/wiki/High-level_programming_language) suitable for rapid development.\n",
    "\n",
    "It has a relatively small core language supported by many libraries.\n",
    "\n",
    "Other features\n",
    "\n",
    "- A multiparadigm language, in that multiple programming styles are supported (procedural, object-oriented, functional, etc.).  \n",
    "- Interpreted rather than compiled.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Syntax and Design\n",
    "\n",
    "\n",
    "<a id='index-2'></a>\n",
    "One nice feature of Python is its elegant syntax — we’ll see many examples later on.\n",
    "\n",
    "Elegant code might sound superfluous but in fact it’s highly beneficial because it makes the syntax easy to read and easy to remember.\n",
    "\n",
    "Remembering how to read from files, sort dictionaries and other such routine tasks means that you don’t need to break your flow in order to hunt down correct syntax.\n",
    "\n",
    "Closely related to elegant syntax is an elegant design.\n",
    "\n",
    "Features like iterators, generators, decorators, list comprehensions, etc. make Python highly expressive, allowing you to get more done with less code.\n",
    "\n",
    "[Namespaces](https://en.wikipedia.org/wiki/Namespace) improve productivity by cutting down on bugs and syntax errors."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Scientific Programming\n",
    "\n",
    "\n",
    "<a id='index-3'></a>\n",
    "Python has become one of the core languages of scientific computing.\n",
    "\n",
    "It’s either the dominant player or a major player in\n",
    "\n",
    "- [Machine learning and data science](http://scikit-learn.org/stable/)  \n",
    "- [Astronomy](http://www.astropy.org/)  \n",
    "- [Artificial intelligence](https://wiki.python.org/moin/PythonForArtificialIntelligence)  \n",
    "- [Chemistry](http://chemlab.github.io/chemlab/)  \n",
    "- [Computational biology](http://biopython.org/wiki/Main_Page)  \n",
    "- [Meteorology](https://pypi.org/project/meteorology/)  \n",
    "- etc., etc.  \n",
    "\n",
    "\n",
    "Its popularity in economics is also beginning to rise.\n",
    "\n",
    "This section briefly showcases some examples of Python for scientific programming.\n",
    "\n",
    "- All of these topics will be covered in detail later on.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Numerical Programming\n",
    "\n",
    "\n",
    "<a id='index-4'></a>\n",
    "Fundamental matrix and array processing capabilities are provided by the excellent [NumPy](http://www.numpy.org/) library.\n",
    "\n",
    "NumPy provides the basic array data type plus some simple processing operations.\n",
    "\n",
    "For example, let’s build some arrays"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "import numpy as np                     # Load the library\n",
    "\n",
    "a = np.linspace(-np.pi, np.pi, 100)    # Create even grid from -π to π\n",
    "b = np.cos(a)                          # Apply cosine to each element of a\n",
    "c = np.sin(a)                          # Apply sin to each element of a"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let’s take the inner product"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "b @ c"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The number you see here might vary slightly but it’s essentially zero.\n",
    "\n",
    "(For older versions of Python and NumPy you need to use the [np.dot](http://docs.scipy.org/doc/numpy/reference/generated/numpy.dot.html) function)\n",
    "\n",
    "The [SciPy](http://www.scipy.org) library is built on top of NumPy and provides additional functionality.\n",
    "\n",
    "\n",
    "<a id='tuple-unpacking-example'></a>\n",
    "For example, let’s calculate $ \\int_{-2}^2 \\phi(z) dz $ where $ \\phi $ is the standard normal density."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "from scipy.stats import norm\n",
    "from scipy.integrate import quad\n",
    "\n",
    "ϕ = norm()\n",
    "value, error = quad(ϕ.pdf, -2, 2)  # Integrate using Gaussian quadrature\n",
    "value"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "SciPy includes many of the standard routines used in\n",
    "\n",
    "- [linear algebra](http://docs.scipy.org/doc/scipy/reference/linalg.html)  \n",
    "- [integration](http://docs.scipy.org/doc/scipy/reference/integrate.html)  \n",
    "- [interpolation](http://docs.scipy.org/doc/scipy/reference/interpolate.html)  \n",
    "- [optimization](http://docs.scipy.org/doc/scipy/reference/optimize.html)  \n",
    "- [distributions and random number generation](http://docs.scipy.org/doc/scipy/reference/stats.html)  \n",
    "- [signal processing](http://docs.scipy.org/doc/scipy/reference/signal.html)  \n",
    "- [etc., etc.](http://docs.scipy.org/doc/scipy/reference/index.html)  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Graphics\n",
    "\n",
    "\n",
    "<a id='index-5'></a>\n",
    "The most popular and comprehensive Python library for creating figures and graphs is [Matplotlib](http://matplotlib.org/).\n",
    "\n",
    "- Plots, histograms, contour images, 3D, bar charts, etc., etc.  \n",
    "- Output in many formats (PDF, PNG, EPS, etc.)  \n",
    "- LaTeX integration  \n",
    "\n",
    "\n",
    "Example 2D plot with embedded LaTeX annotations\n",
    "\n",
    "<img src=\"https://s3-ap-southeast-2.amazonaws.com/lectures.quantecon.org/py/_static/lecture_specific/about_py/qs.png\" style=\"\">\n",
    "\n",
    "  \n",
    "Example contour plot\n",
    "\n",
    "<img src=\"https://s3-ap-southeast-2.amazonaws.com/lectures.quantecon.org/py/_static/lecture_specific/about_py/bn_density1.png\" style=\"\">\n",
    "\n",
    "  \n",
    "Example 3D plot\n",
    "\n",
    "<img src=\"https://s3-ap-southeast-2.amazonaws.com/lectures.quantecon.org/py/_static/lecture_specific/about_py/career_vf.png\" style=\"\">\n",
    "\n",
    "  \n",
    "More examples can be found in the [Matplotlib thumbnail gallery](http://matplotlib.org/gallery.html).\n",
    "\n",
    "Other graphics libraries include\n",
    "\n",
    "- [Plotly](https://plot.ly/python/)  \n",
    "- [Bokeh](http://bokeh.pydata.org/en/latest/)  \n",
    "- [VPython](http://www.vpython.org/) — 3D graphics and animations  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Symbolic Algebra\n",
    "\n",
    "It’s useful to be able to manipulate symbolic expressions, as in Mathematica or Maple.\n",
    "\n",
    "\n",
    "<a id='index-6'></a>\n",
    "The [SymPy](http://www.sympy.org/) library provides this functionality from within the Python shell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "from sympy import Symbol\n",
    "\n",
    "x, y = Symbol('x'), Symbol('y')  # Treat 'x' and 'y' as algebraic symbols\n",
    "x + x + x + y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can manipulate expressions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "expression = (x + y)**2\n",
    "expression.expand()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "solve polynomials"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "from sympy import solve\n",
    "\n",
    "solve(x**2 + x + 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and calculate limits, derivatives and integrals"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "from sympy import limit, sin, diff\n",
    "\n",
    "limit(1 / x, x, 0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "limit(sin(x) / x, x, 0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "diff(sin(x), x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The beauty of importing this functionality into Python is that we are working within a fully fledged programming language.\n",
    "\n",
    "Can easily create tables of derivatives, generate LaTeX output, add it to figures, etc., etc."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Statistics\n",
    "\n",
    "Python’s data manipulation and statistics libraries have improved rapidly over\n",
    "the last few years."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Pandas\n",
    "\n",
    "\n",
    "<a id='index-7'></a>\n",
    "One of the most popular libraries for working with data is [pandas](http://pandas.pydata.org/).\n",
    "\n",
    "Pandas is fast, efficient, flexible and well designed.\n",
    "\n",
    "Here’s a simple example, using some fake data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "np.random.seed(1234)\n",
    "\n",
    "data = np.random.randn(5, 2)  # 5x2 matrix of N(0, 1) random draws\n",
    "dates = pd.date_range('28/12/2010', periods=5)\n",
    "\n",
    "df = pd.DataFrame(data, columns=('price', 'weight'), index=dates)\n",
    "print(df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "df.mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Other Useful Statistics Libraries\n",
    "\n",
    "\n",
    "<a id='index-8'></a>\n",
    "- [statsmodels](http://statsmodels.sourceforge.net/) — various statistical routines  \n",
    "\n",
    "\n",
    "\n",
    "<a id='index-9'></a>\n",
    "- [scikit-learn](http://scikit-learn.org/) — machine learning in Python (sponsored by Google, among others)  \n",
    "\n",
    "\n",
    "\n",
    "<a id='index-10'></a>\n",
    "- [pyMC](http://pymc-devs.github.io/pymc/) — for Bayesian data analysis  \n",
    "\n",
    "\n",
    "\n",
    "<a id='index-11'></a>\n",
    "- [pystan](https://pystan.readthedocs.org/en/latest/) Bayesian analysis based on [stan](http://mc-stan.org/)  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Networks and Graphs\n",
    "\n",
    "Python has many libraries for studying graphs.\n",
    "\n",
    "\n",
    "<a id='index-12'></a>\n",
    "One well-known example is [NetworkX](http://networkx.github.io/)\n",
    "\n",
    "- Standard graph algorithms for analyzing network structure, etc.  \n",
    "- Plotting routines  \n",
    "- etc., etc.  \n",
    "\n",
    "\n",
    "Here’s some example code that generates and plots a random graph, with node color determined by shortest path length from a central node."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "import networkx as nx\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "np.random.seed(1234)\n",
    "\n",
    "# Generate a random graph\n",
    "p = dict((i, (np.random.uniform(0, 1), np.random.uniform(0, 1)))\n",
    "         for i in range(200))\n",
    "g = nx.random_geometric_graph(200, 0.12, pos=p)\n",
    "pos = nx.get_node_attributes(g, 'pos')\n",
    "\n",
    "# Find node nearest the center point (0.5, 0.5)\n",
    "dists = [(x - 0.5)**2 + (y - 0.5)**2 for x, y in list(pos.values())]\n",
    "ncenter = np.argmin(dists)\n",
    "\n",
    "# Plot graph, coloring by path length from central node\n",
    "p = nx.single_source_shortest_path_length(g, ncenter)\n",
    "plt.figure()\n",
    "nx.draw_networkx_edges(g, pos, alpha=0.4)\n",
    "nx.draw_networkx_nodes(g,\n",
    "                       pos,\n",
    "                       nodelist=list(p.keys()),\n",
    "                       node_size=120, alpha=0.5,\n",
    "                       node_color=list(p.values()),\n",
    "                       cmap=plt.cm.jet_r)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Cloud Computing\n",
    "\n",
    "\n",
    "<a id='index-13'></a>\n",
    "Running your Python code on massive servers in the cloud is becoming easier and easier.\n",
    "\n",
    "\n",
    "<a id='index-14'></a>\n",
    "A nice example is [Anaconda Enterprise](https://www.anaconda.com/enterprise/).\n",
    "\n",
    "See also\n",
    "\n",
    "\n",
    "<a id='index-15'></a>\n",
    "- [Amazon Elastic Compute Cloud](http://aws.amazon.com/ec2/)  \n",
    "\n",
    "\n",
    "\n",
    "<a id='index-16'></a>\n",
    "- The [Google App Engine](https://cloud.google.com/appengine/) (Python, Java, PHP or Go)  \n",
    "\n",
    "\n",
    "\n",
    "<a id='index-17'></a>\n",
    "- [Pythonanywhere](https://www.pythonanywhere.com/)  \n",
    "\n",
    "\n",
    "\n",
    "<a id='index-18'></a>\n",
    "- [Sagemath Cloud](https://cloud.sagemath.com/)  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Parallel Processing\n",
    "\n",
    "\n",
    "<a id='index-19'></a>\n",
    "Apart from the cloud computing options listed above, you might like to consider\n",
    "\n",
    "\n",
    "<a id='index-20'></a>\n",
    "- [Parallel computing through IPython clusters](http://ipython.org/ipython-doc/stable/parallel/parallel_demos.html).  \n",
    "\n",
    "\n",
    "\n",
    "<a id='index-21'></a>\n",
    "- The [Starcluster](http://star.mit.edu/cluster/) interface to Amazon’s EC2.  \n",
    "\n",
    "\n",
    "\n",
    "<a id='index-23'></a>\n",
    "- GPU programming through [PyCuda](https://wiki.tiker.net/PyCuda), [PyOpenCL](https://mathema.tician.de/software/pyopencl/), [Theano](http://deeplearning.net/software/theano/) or similar.  \n",
    "\n",
    "\n",
    "\n",
    "<a id='intfc'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Other Developments\n",
    "\n",
    "There are many other interesting developments with scientific programming in Python.\n",
    "\n",
    "Some representative examples include\n",
    "\n",
    "\n",
    "<a id='index-24'></a>\n",
    "- [Jupyter](http://jupyter.org/) — Python in your browser with code cells, embedded images, etc.  \n",
    "\n",
    "\n",
    "\n",
    "<a id='index-25'></a>\n",
    "- [Numba](http://numba.pydata.org/) — Make Python run at the same speed as native machine code!  \n",
    "\n",
    "\n",
    "\n",
    "<a id='index-26'></a>\n",
    "- [Blaze](http://blaze.pydata.org/) — a generalization of NumPy.  \n",
    "\n",
    "\n",
    "\n",
    "<a id='index-27'></a>\n",
    "- [PyTables](http://www.pytables.org) — manage large data sets.  \n",
    "\n",
    "\n",
    "\n",
    "<a id='index-28'></a>\n",
    "- [CVXPY](https://github.com/cvxgrp/cvxpy) — convex optimization in Python.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Learn More\n",
    "\n",
    "- Browse some Python projects on [GitHub](https://github.com/trending?l=python).  \n",
    "- Have a look at [some of the Jupyter notebooks](http://nbviewer.jupyter.org/) people have shared on various scientific topics.  \n",
    "\n",
    "\n",
    "\n",
    "<a id='index-29'></a>\n",
    "- Visit the [Python Package Index](https://pypi.org/).  \n",
    "- View some of the questions people are asking about Python on [Stackoverflow](http://stackoverflow.com/questions/tagged/python).  \n",
    "- Keep up to date on what’s happening in the Python community with the [Python subreddit](https://www.reddit.com:443/r/Python/).  "
   ]
  }
 ],
 "metadata": {
  "filename": "about_py.rst",
  "kernelspec": {
   "display_name": "Python",
   "language": "python3",
   "name": "python3"
  },
  "title": "About Python"
 },
 "nbformat": 4,
 "nbformat_minor": 2
}