xdress-kitware2014.ipynb

{
 "metadata": {
  "celltoolbar": "Slideshow",
  "name": "",
  "signature": "sha256:dd60c9ea00b74b4d9c4f0621a53f97abcaa1e20c83cc183b1f5ec0bb26cf728e"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "<div style=\"text-align:center\"><h1>XDress - Idiomatic, Automatic Wrapper Generation</h1>\n",
      "<div style=\"text-align:center\"><img style=\"margin:auto\" src=\"thief.jpg\" /></div>\n",
      "<h2>Kitware</h2>\n",
      "</div>\n",
      "<div style=\"text-align:center\">Anthony Michael Scopatz</div>\n",
      "<div style=\"text-align:center\">The University of Wisconsin-Madison, Numfocus</div>\n",
      "<div style=\"text-align:center\">Clifton Park, August 20th, 2014</div>"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import os\n",
      "import sys\n",
      "import shutil\n",
      "import numpy as np\n",
      "import matplotlib\n",
      "import collections\n",
      "matplotlib.rc('font', family='serif', size=16)\n",
      "import matplotlib.pyplot as plt\n",
      "from IPython.display import *\n",
      "from utils import htmltable\n",
      "from pprint import pprint\n",
      "from glob import glob"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "skip"
      }
     },
     "outputs": [],
     "prompt_number": 1
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "skip"
      }
     },
     "source": [
      "## Cheat Sheet\n",
      "\n",
      "**Center:**  ``<div style=\"text-align:center;\"></div>``\n",
      "    \n",
      "**Right:**   ``<div style=\"text-align:right;\"></div>``\n",
      "    \n",
      "**Center H3:**  ``<div style=\"text-align:center;\"><h3></h3></div>``\n",
      "\n",
      "**Right H3:**  ``<div style=\"text-align:right;\"><h3></h3></div>``\n",
      "    \n",
      "**Center Image:**  ``<div style=\"text-align:center;\"><img src=\"\" width=\"80%\" /></div>``"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## The Big Idea\n",
      "\n",
      "<br/><div style=\"text-align:center;\">At its core, xdress is a suite of code generation routines built on top of a dynamic type system.</div>"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "<div style=\"text-align:center;\">The interface that is created should be beautiful, or at least not ugly.</div>"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "## Motivation\n",
      "\n",
      "* First class C++ support\n",
      "* Deep access to the type system\n",
      "* A multitude of target languages"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## What XDress Is"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "skip"
      }
     },
     "source": [
      "Created via [this link](http://interactive.blockdiag.com/?compression=deflate&src=eJyr5lJQUHIM8FRwy8xLSS1SUoguLqnMSVWwVUhJLM5ITYm1BioISCwqTi0qVtC1Q1UL5geHKIRlFmeW5EMFnPNTUhXcU_NSixKBYsVK1ly1AOqXHj4)."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "<div style=\"text-align:center;\"><img src=\"workflow.svg\" /></div>"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "## What XDress Is Not\n",
      "\n",
      "* **Parser**\n",
      "* **Compiler**\n",
      "* **Build System**"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Who Uses XDress\n",
      "\n",
      "<div style=\"text-align:center;\">\n",
      "<img src=\"pyne_icon_small.png\" style=\"margin:auto\" width=\"20%\" />\n",
      "<img src=\"uni_logo.png\" style=\"margin:auto\" width=\"20%\" />\n",
      "</div>"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "<div style=\"text-align:center\"><h1>Let's Talk About Parsers!</h1></div>\n",
      "<br/>\n",
      "<div style=\"text-align:center\"><img src=\"c_hierophant.jpg\" style=\"margin:auto\" width=\"30%\" /></div>"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Parsers\n",
      "\n",
      "Parsing is worse than hard, it is annoying!"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "XDress completely outsources the task of parsing source code to other projects. It mostly outsources the syntax tree too."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "Parses that XDress supports currently are:\n",
      "\n",
      "* pycparser, for pure C-code"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "* GCC-XML, for C++ code"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "* Clang 3.2+, for C and C++ code"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "This tactic of relying on external parsers enables flexibility w.r.t. the source language."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Clang\n",
      "\n",
      "Clang is the *preferred* parser."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "Gives us full C++11 support."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "It is about 10x faster than GCC-XML."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "It is more robust than pycparser, no need to call to an external cpp."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "It was also a huge pain to interface with!"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Ideal Clang Stack\n",
      "\n",
      "<div style=\"text-align:center;\"><img src=\"clang-stack-ideal.svg\" width=\"80%\" /></div>"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "skip"
      }
     },
     "source": [
      "Created via [this link](http://interactive.blockdiag.com/?compression=deflate&src=eJyr5lJQUHLOScxLV9Bw1tbWVFLQtVNQyslMSoaKQUUKKhFiAZUlGfl5UAnH4BAla65aALXWE3g)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Actual Clang Stack\n",
      "\n",
      "<div style=\"text-align:center;\"><img src=\"clang-stack-actual.svg\" width=\"80%\" /></div>"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "skip"
      }
     },
     "source": [
      "Created via [this link](http://interactive.blockdiag.com/?compression=deflate&src=eJyr5lJQUHLOScxLV9Bw1tbWVFLQtVNQyslMSoaKQUUKKhFiAZUlGfl5UAnH4BAlhejk_Jz8IgVbhaLUlFhrrEZWpBSlFhcrwExBFtRDEgEZZ81VCwBU-ipV)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Clang\n",
      "\n",
      "This really works!"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "As much as we were forced to fork libclang, we are very interested in either \n",
      "\n",
      "* breaking this fork into a separate project or repo, or "
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "* getting our changes in upstream clang."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "All kudos go to [Geoffery Irving](http://naml.us/~irving/) here!"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Parsers\n",
      "\n",
      "Once we have loaded an AST, there are two things that xdress can do:"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "* Search and categorize symbols accoring to whether they are variables, functions, or classes."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "* Create a canonical, language-independent descriptions of the API."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "In this sense, xdress is yet-another-API-API. "
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Parsers - Major Point!\n",
      "\n",
      "<br/><div style=\"text-align:center\">**XDress only cares about APIs.**</div>"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "The implementation does not matter."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "In this context, Clang and pycparser do too much."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## API Descriptions\n",
      "\n",
      "*So how do the API descriptions work?*"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "To keep them as a portable as possible, they are nothing fancy."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "In memory descriptions (*desc*) are just Python strings, ints, dicts, tuples, lists, and other built-in types.  They can easily be translated into a JSON format."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## API Description Example\n",
      "\n",
      "Code is generated from descriptions of API elements to produce idomatic Python wrappers.\n",
      "\n",
      "```python\n",
      "brienne_desc = {\n",
      "    'name': 'Brienne',\n",
      "    'parents': ['LordSelwyn'],\n",
      "    'namespace': 'westeros',\n",
      "    'attrs': {'moniker': 'str'},\n",
      "    'methods': {\n",
      "        ('Brienne',): None,\n",
      "        ('Brienne', ('monkier', 'str', '\"Maid of Tarth\"')): None,\n",
      "        ('~Brienne',): None, \n",
      "        ('attack', ('person', ('char' '*'), ('weapon_id', 'i2', 1))): 'float32',\n",
      "        },\n",
      "    'docstrings': {\n",
      "        'class': \"I am so tall!\",\n",
      "        'attrs': {\n",
      "            'monkier': 'extra name for character',\n",
      "            },\n",
      "        'methods': {\n",
      "            'attack': \"Attack someone with a weapon and returns damage.\"\n",
      "            },\n",
      "        },\n",
      "    'extra': {},\n",
      "    }\n",
      "```"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "They heavy lifting of the API decsriptions is done by the type system."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "<div style=\"text-align:center\"><h1>Stay Awake For Type Theory!</h1></div>\n",
      "<br/>\n",
      "<div style=\"text-align:center\"><img src=\"baroque-moon.jpg\" style=\"margin:auto\" width=\"30%\" /></div>"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Type System\n",
      "\n",
      "* Since we are not doing any parsing or compiling, the type system doesn't need to know the concrete size of any types."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "* Rather the type system is a convient, consistent, and complete way to *spell* type names!"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "* Type names are composed of immutable Python types:\n",
      "    \n",
      "    * ``str``\n",
      "    * ``tuple``\n",
      "    * ``int``\n",
      "    * ``TypeMatcher``"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Kinds of Types\n",
      "\n",
      "As a brief diversion into type theory, xdress supports:\n",
      "    \n",
      "<div style=\"text-align:center;\"><h3>Base Types</h3></div><br/>\n",
      "\n",
      "<div style=\"text-align:center;\"><h3>Dependent Types</h3></div><br/>\n",
      "\n",
      "<div style=\"text-align:center;\"><h3>Refinement Types</h3></div><br/>"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Base Types\n",
      "\n",
      "Base types are the normal primitive types that we all know and lovehate.  \n",
      "\n",
      "These are represented by ``str``.  \n",
      "\n",
      "Classes and structs also become base types."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from xdress.types.system import TypeSystem\n",
      "ts = TypeSystem()\n",
      "print(sorted(ts.base_types))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "['bool', 'char', 'complex128', 'exception', 'file', 'float128', 'float32', 'float64', 'int128', 'int16', 'int32', 'int64', 'str', 'type', 'uchar', 'uint128', 'uint16', 'uint32', 'uint64', 'void']\n"
       ]
      }
     ],
     "prompt_number": 2
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Dependent Types\n",
      "\n",
      "Dependent types are parameterized by static values *or* types!"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "These are used to represent templated types in C++."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "They are spelled with ``tuple``s."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "set_of_strings = ('set', 'str')\n",
      "map_int_double = ('map', 'int32', 'float64')"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "outputs": [],
     "prompt_number": 3
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Refined Types\n",
      "\n",
      "Refined types are defined such that all of their elements are subset of another type.  "
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "They are specified by the refinement type name, the super type name, and a predicate."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "__Examples:__\n",
      "    \n",
      "* Even integers are a refinement of ``int32``"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "* Strings which begin with \"A\" are a refinement of ``str``"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "* ``set([\"Awesome\", \"As\", \"Anthony\"])`` is a refinement of \"A\" strings"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Refined Types\n",
      "\n",
      "These may be spelled with ``str`` or the (super type, type name) ``tuple``s."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "type_name = 'evenint'\n",
      "final_form = ('int32', 'evenint')"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "outputs": [],
     "prompt_number": 4
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "## Refined Dependent Types\n",
      "\n",
      "Recall that dependent types are fully composable with other types."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "int_range = ('range', 'int32', 1, 42)\n",
      "even_int_range = ('range', ('int32', 'evenint'), 1, 42)\n",
      "too_much_fun = ('map', 'evenint', ('range', ('int32', 'evenint'), 1, 42))"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "outputs": [],
     "prompt_number": 5
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## And More!\n",
      "\n",
      "* What you have seen so far is the shorthand form.  "
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "* There is also a canonical form: ``ts.canon(t)``"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "* There are aliases: ``ts.canon('f8') == 'float64'``"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "* Standard refined & dependent types built-in:\n",
      "    \n",
      "    * **templates:** ``pair``, ``set``, ``map``, ``vector``\n",
      "    * **refinements:** ``enum``, ``fucntion``, ``function_pointer``"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "* There is also a ``TypeMatcher`` class - inspired by ``re`` - which may replace any type or portion of a type."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "<div style=\"text-align:center\"><h1>Talkig 'Bout My Code Generation!</h1></div>\n",
      "<br/>\n",
      "<div style=\"text-align:center\"><img src=\"bc06lovers.jpg\" style=\"margin:auto;\" width=\"30%\" /></div>"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Code Generation \n",
      "\n",
      "After the AST has been turned into an API description, xdress can pass this to as many code generators as are supported."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "Currently there are three code generators."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "The primary one is called `cythongen` and targets Python via Cython."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "*How many people have used Cython?*"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Cython\n",
      "\n",
      "Cython is great!"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "It is a superset language of Python that adds C-like features:"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "* Type declarations"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "* Pointers & references "
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "* Removal of GIL, bounds checking, etc."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "* and more!"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "Cython is compiled into non-idiomatic C or C++ code."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "This is then compiled normally into a Python extension module."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Cython Example\n",
      "\n",
      "<br/>\n",
      "\n",
      "```cython\n",
      "cdef class Rectangle:\n",
      "    cdef int x0, y0\n",
      "    cdef int x1, y1\n",
      "    \n",
      "    def __init__(self, int x0, int y0, int x1, int y1):\n",
      "        self.x0 = x0; self.y0 = y0; self.x1 = x1; self.y1 = y1\n",
      "        \n",
      "    cpdef int area(self):\n",
      "        cdef int area\n",
      "        area = (self.x1 - self.x0) * (self.y1 - self.y0)\n",
      "        if area < 0:\n",
      "            area = -area\n",
      "        return area\n",
      "\n",
      "def rect_area(x0, y0, x1, y1):\n",
      "    cdef Rectangle rect\n",
      "    rect = Rectangle(x0, y0, x1, y1)\n",
      "    return rect.area()\n",
      "```"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Cython is Fast\n",
      "\n",
      "Cython can be thought of as a bonus optimization level."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "For pure Python, speedups of 3x-4x are common."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "The more type information you add, the faster it goes. 1000x+ is easy to achieve."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "Lest you think that this is because \"Python is Slow\", Cython can be faster than idiomatic C in certain cases."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## (Some) Cython Optimizations\n",
      "\n",
      "* Loops are in C, not Python"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "* if-else statements become switch-case"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "* Avoide function calls when a jump will do"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "* OpenMP integration"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "* Python & NumPy C-API aware"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Cython\n",
      "\n",
      "Cython eats the lunches of ctypes and Boost.Python, "
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "And faster than SWIG can."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "But we are really here to talk about xdress..."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "<div style=\"text-align:center\"><h1>Example Time!</h1></div>\n",
      "<br/>\n",
      "<div style=\"text-align:center;\"><img src=\"hanged-cat.jpg\" style=\"margin:auto;\" width=\"30%\" /></div>"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## ``cythongen`` via Hoover Example\n",
      "\n",
      "### jehsrc/hoover.h\n",
      "\n",
      "```CPP\n",
      "#if !defined(HOOVER)\n",
      "#define HOOVER\n",
      "namespace hoover {\n",
      "class A {\n",
      " public:\n",
      "   float a;\n",
      "   double power(int n=1);\n",
      "};};\n",
      "#endif\n",
      "```\n",
      "    \n",
      "### jehsrc/hoover.cpp\n",
      "\n",
      "```cpp\n",
      "#include \"hoover.h\"\n",
      "double hoover::A::power(int n){\n",
      "  int i = 1;\n",
      "  double val = a;\n",
      "  while (i < n){val *= a;};\n",
      "    return val;\n",
      "};\n",
      "```"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "!rm -rf jedgar jehbuild build"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "skip"
      }
     },
     "outputs": [],
     "prompt_number": 6
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Hoover Example\n",
      "\n",
      "### jeh_xdressrc.py\n",
      "\n",
      "```python\n",
      "package = 'jedgar'      # top-level python package name\n",
      "packagedir = 'jedgar'   # location of the python package\n",
      "builddir = 'jehbuild'\n",
      "\n",
      "classes = [('A', 'jehsrc/hoover.*')]\n",
      "```"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "### Run & Build"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%%bash\n",
      "xdress --rc jeh_xdressrc.py\n",
      "python3 jeh_setup.py build > /dev/null 2>&1"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "autoall: discovering API names\n",
        "autoall: no API names to discover!\n",
        "autodescribe: registering A\n",
        "dtypes: registering numpy dtypes\n",
        "stlwrap: registering C++ standard library types\n",
        "autodescribe: scraping C/C++ APIs from source\n",
        "autodescribe: describing A\n",
        "cythongen: creating C/C++ API wrappers\n",
        "extratypes: generating extra type header & source files for xdress\n",
        "dtypes: generating numpy dtype wrappers\n"
       ]
      }
     ],
     "prompt_number": 7
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Hoover Example\n",
      "\n",
      "### jedgar/hoover.pyx\n",
      "\n",
      "<br/>\n",
      "\n",
      "```cython\n",
      "from libc.stdlib cimport free\n",
      "\n",
      "cdef class A:\n",
      "    \"\"\"no docstring for A, please file a bug report!\"\"\"\n",
      "\n",
      "    # constuctors\n",
      "    def __cinit__(self, *args, **kwargs):\n",
      "        self._inst = NULL\n",
      "        self._free_inst = True\n",
      "\n",
      "    def __init__(self, ):\n",
      "        \"\"\"A(self, )\n",
      "        \"\"\"\n",
      "        self._inst = new cpp_hoover.A()\n",
      "    \n",
      "    def __dealloc__(self):\n",
      "        if self._free_inst and self._inst is not NULL:\n",
      "            free(self._inst)\n",
      "\n",
      "    # attributes\n",
      "    property a:\n",
      "        \"\"\"no docstring for a, please file a bug report!\"\"\"\n",
      "        def __get__(self):\n",
      "            return float((<cpp_hoover.A *> self._inst).a)\n",
      "    \n",
      "        def __set__(self, value):\n",
      "            (<cpp_hoover.A *> self._inst).a = <float> value\n",
      "    \n",
      "    # methods\n",
      "    def power(self, n=1):\n",
      "        \"\"\"power(self, n=1)\n",
      "        no docstring for power, please file a bug report!\"\"\"\n",
      "        cdef double rtnval\n",
      "        rtnval = (<cpp_hoover.A *> self._inst).power(<int> n)\n",
      "        return float(rtnval)\n",
      "```"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "builddir = [os.path.join('build', d) for d in os.listdir('build') if d.startswith('lib.') and d.endswith(sys.version[:3])]\n",
      "builddir = builddir[0]\n",
      "builddir = os.path.abspath(builddir)\n",
      "if builddir not in sys.path:\n",
      "    sys.path.insert(0, builddir)"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "skip"
      }
     },
     "outputs": [],
     "prompt_number": 8
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Hoover Example"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from jedgar.hoover import A\n",
      "\n",
      "a = A()\n",
      "a.a = 10\n",
      "a.power(42)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 9,
       "text": [
        "9.999999999999999e+41"
       ]
      }
     ],
     "prompt_number": 9
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "<div style=\"text-align:center;\"><h1>Idiom Proof</h1></div>\n",
      "<br/>\n",
      "<div style=\"text-align:center;\"><img src=\"the-fool.jpg\" style=\"margin:auto;\" width=\"30%\" /></div>"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Function Pointer Callbacks\n",
      "\n",
      "### ThreeNums Struct in C\n",
      "\n",
      "```c\n",
      "typedef struct ThreeNums\n",
      "{\n",
      "  double a;\n",
      "  double b;\n",
      "  double c;\n",
      "  double (*op)(double, double, double);\n",
      "} ThreeNums;\n",
      "```\n",
      "    \n",
      "### Op Caller in C\n",
      "\n",
      "```C\n",
      "double call_threenums_op_from_c(ThreeNums x) {\n",
      "  return x.op(x.a, x.b, x.c);\n",
      "}\n",
      "```"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "d = glob(os.path.expanduser('~/xdress/tests/cproj/install-*/lib*/*/site-packages'))\n",
      "d = os.path.abspath(d[0])\n",
      "if d not in sys.path:\n",
      "    sys.path.insert(0, d)\n",
      "import cproj.pybasics as mod"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "skip"
      }
     },
     "outputs": [],
     "prompt_number": 10
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Function Pointer Callbacks"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "x = mod.ThreeNums(5, 10, 15)\n",
      "print(\"a and b and c = \", x.a, x.b, x.c)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "a and b and c =  5.0 10.0 15.0\n"
       ]
      }
     ],
     "prompt_number": 11
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "f = lambda i, j, k: i + j + k\n",
      "\n",
      "print(\"f's refcount before assignment: \", sys.getrefcount(f))\n",
      "x.op = f\n",
      "print(\"f's refcount after assignment: \", sys.getrefcount(f))\n",
      "print(\"x.op = \", x.op)"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "f's refcount before assignment:  2\n",
        "f's refcount after assignment:  4\n",
        "x.op =  <function <lambda> at 0x7f8a0e7ccd08>\n"
       ]
      }
     ],
     "prompt_number": 12
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# Test that function pointer in C is getting set properly\n",
      "x._op = None\n",
      "print(\"x._op is None:\", x._op is None)\n",
      "print(\"f's refcount after None assignment: \", sys.getrefcount(f))\n",
      "print(\"x.op = \", x.op)"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "x._op is None: True\n",
        "f's refcount after None assignment:  3\n",
        "x.op =  <cyfunction ThreeNums.__get__.<locals>.op_proxy at 0x7f8a0e7dd1b8>\n"
       ]
      }
     ],
     "prompt_number": 13
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Function Pointer Callbacks - From C"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "x.op(14, 16, 18)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 14,
       "text": [
        "48.0"
       ]
      }
     ],
     "prompt_number": 14
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "mod.call_threenums_op_from_c(x)"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 15,
       "text": [
        "30.0"
       ]
      }
     ],
     "prompt_number": 15
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Function Pointer Callbacks"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "x.op = lambda i, j, k: 2*i + j**2 + k/2\n",
      "print(\"x.op = \", x.op)\n",
      "print(\"f's refcount after op re-assignment: \", sys.getrefcount(f))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "x.op =  <function <lambda> at 0x7f8a0e7ccd90>\n",
        "f's refcount after op re-assignment:  3\n"
       ]
      }
     ],
     "prompt_number": 16
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "x.op(14, 16, 18)"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 17,
       "text": [
        "293.0"
       ]
      }
     ],
     "prompt_number": 17
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "mod.call_threenums_op_from_c(x)"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 18,
       "text": [
        "117.5"
       ]
      }
     ],
     "prompt_number": 18
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Function Pointer Callbacks - Two Instances"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "y = mod.ThreeNums(5, 10, 15)\n",
      "x.op = lambda i, j, k: i + j + k\n",
      "y.op = lambda i, j, k: 2*i + j**2 + k/2"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "-"
      }
     },
     "outputs": [],
     "prompt_number": 19
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "x.op(14, 16, 18)"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 20,
       "text": [
        "48"
       ]
      }
     ],
     "prompt_number": 20
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "y.op(14, 16, 18)"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 21,
       "text": [
        "293.0"
       ]
      }
     ],
     "prompt_number": 21
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Function and Method Overloading\n",
      "\n",
      "**Example member function declaration:**\n",
      "\n",
      "```cpp\n",
      "int func();\n",
      "int func(double x, int y=10);\n",
      "int func(std::string x, double y=10.0);\n",
      "```"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Function and Method Overloading\n",
      "\n",
      "**Concrete wrappers:**\n",
      "\n",
      "    def _func_0():\n",
      "        \"\"\"func()\n",
      "        I am a weird function.\"\"\"\n",
      "        cdef int rtnval\n",
      "        rtnval = cpp_reprocess.func()\n",
      "        return int(rtnval)\n",
      "\n",
      "    def _func_1(x, y=10):\n",
      "        \"\"\"func(x, y=10)\n",
      "        I am a weird function.\"\"\"\n",
      "        cdef int rtnval\n",
      "        rtnval = cpp_reprocess.func(<double> x, <int> y)\n",
      "        return int(rtnval)\n",
      "\n",
      "    def _func_2(x, y=1.0e+1):\n",
      "        \"\"\"func(x, y=1.0e+1)\n",
      "        I am a weird function.\"\"\"\n",
      "        cdef char * x_proxy\n",
      "        cdef int rtnval\n",
      "        x_bytes = x.encode()\n",
      "        rtnval = cpp_reprocess.func(std_string(<char *> x_bytes), <double> y)\n",
      "        return int(rtnval)\n"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Function and Method Overloading\n",
      "\n",
      "**Virtual table wrapper:**\n",
      "\n",
      "    _func_0_argtypes = frozenset()\n",
      "    _func_1_argtypes = frozenset(((0, float), (1, int), (\"x\", float), (\"y\", int)))\n",
      "    _func_2_argtypes = frozenset(((0, str), (1, float), (\"x\", str), (\"y\", float)))\n",
      "\n",
      "    def func(self, *args, **kwargs):\n",
      "        \"\"\"func(x, y=1.0e+1)\n",
      "        I am a weird function.\"\"\"\n",
      "        types = set([(i, type(a)) for i, a in enumerate(args)])\n",
      "        types.update([(k, type(v)) for k, v in kwargs.items()])\n",
      "        # vtable-like dispatch for exactly matching types\n",
      "        if types <= self._func_0_argtypes:\n",
      "            return self._func_0(*args, **kwargs)\n",
      "        if types <= self._func_1_argtypes:\n",
      "            return self._func_1(*args, **kwargs)\n",
      "        if types <= self._func_2_argtypes:\n",
      "            return self._func_2(*args, **kwargs)\n",
      "        # duck-typed dispatch based on whatever works!\n",
      "        try:\n",
      "            return self._func_0(*args, **kwargs)\n",
      "        except (RuntimeError, TypeError, NameError):\n",
      "            pass\n",
      "        try:\n",
      "            return self._func_1(*args, **kwargs)\n",
      "        except (RuntimeError, TypeError, NameError):\n",
      "            pass\n",
      "        try:\n",
      "            return self._func_2(*args, **kwargs)\n",
      "        except (RuntimeError, TypeError, NameError):\n",
      "            pass\n",
      "        raise RuntimeError('method func() could not be dispatched')"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## C++ Standard Library Container Wrappers\n",
      "\n",
      "Another code generator on top of the type system are idomatic wrappers which provide **views** into C++ containers."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "Currently this includes ``pair``, ``map``, ``set``, and ``vector``. Others are ~~trivial~~ easy enough to add."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "### xdressrc.py (what the user writes)\n",
      "\n",
      "```python\n",
      "stlcontainers_module = 'conts'\n",
      "stlcontainers = [\n",
      "    ('vector', 'float64'),\n",
      "    ('vector', 'str'),\n",
      "    ('vector', 'int32'),\n",
      "    ('vector', ('vector', 'float64')),\n",
      "    ('set', 'int'),\n",
      "    ('map', 'str', 'str'),\n",
      "    ('map', 'str', 'int'),\n",
      "    ('map', 'str', 'float'),\n",
      "    ('map', 'int', 'int'),\n",
      "    ('map', 'int', 'float'),\n",
      "    ('map', 'int', ('vector', 'str')),\n",
      "    ('map', 'int', ('vector', 'float')),\n",
      "    ('map', 'int', ('map', 'int', ('vector', 'float'))),\n",
      "    ]\n",
      "```"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## ``std::map<T, U>`` Wrappers"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "d = glob(os.path.expanduser('~/xdress/tests/cppproj/install-*/lib*/*/site-packages'))\n",
      "d = os.path.abspath(d[0])\n",
      "if d not in sys.path:\n",
      "    sys.path.insert(0, d)\n",
      "import cppproj.stlc as conts"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "skip"
      }
     },
     "outputs": [],
     "prompt_number": 22
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print(sorted([m for m in dir(conts) if m.startswith('Map')]))"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "-"
      }
     },
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "['MapIntBool', 'MapIntChar', 'MapIntComplex', 'MapIntDouble', 'MapIntInt', 'MapIntMapIntBool', 'MapIntMapIntChar', 'MapIntMapIntDouble', 'MapIntMapIntVectorBool', 'MapIntMapIntVectorChar', 'MapIntMapIntVectorDouble', 'MapIntSetChar', 'MapIntSetInt', 'MapIntSetStr', 'MapIntSetUInt', 'MapIntStr', 'MapIntVectorBool', 'MapIntVectorChar', 'MapIntVectorDouble', 'MapIntVectorInt', 'MapIntVectorSetInt', 'MapIntVectorStr', 'MapIntVectorUInt', 'MapIntVectorVectorDouble', 'MapPairIntIntDouble', 'MapStrDouble', 'MapStrInt', 'MapStrStr', 'MapStrUInt', 'MapThreeNumsDouble', 'MapUIntDouble', 'MapUIntStr', 'MapUIntUInt']\n"
       ]
      }
     ],
     "prompt_number": 23
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "scores = conts.MapStrInt({'Joan': 42, 'Leslie': 6})\n",
      "print(scores)\n",
      "print(\"scores is mapping:\", isinstance(scores, collections.MutableMapping))"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "{'Joan': 42, 'Leslie': 6}\n",
        "scores is mapping: True\n"
       ]
      }
     ],
     "prompt_number": 24
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "scores['Leslie'] = 14\n",
      "print(scores['Joan'] / scores['Leslie'])"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "3.0\n"
       ]
      }
     ],
     "prompt_number": 25
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Bad Types Fail\n",
      "\n",
      "<br /><br />"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "scores[\"Hoover\"] = [1.92e3, \"hell0\"]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "ename": "TypeError",
       "evalue": "an integer is required",
       "output_type": "pyerr",
       "traceback": [
        "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m\n\u001b[1;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
        "\u001b[1;32m<ipython-input-26-117d2e157c41>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mscores\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;34m\"Hoover\"\u001b[0m\u001b[1;33m]\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;33m[\u001b[0m\u001b[1;36m1.92e3\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34m\"hell0\"\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
        "\u001b[1;32m/home/scopatz/xdress/tests/cppproj/install-clang/lib/python3.4/site-packages/cppproj/stlc.cpython-34m.so\u001b[0m in \u001b[0;36mcppproj.stlc._MapStrInt.__setitem__ (/home/scopatz/xdress/tests/cppproj/build/cppproj/stlc.cxx:23108)\u001b[1;34m()\u001b[0m\n",
        "\u001b[1;31mTypeError\u001b[0m: an integer is required"
       ]
      }
     ],
     "prompt_number": 26
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## ``std::vector<T>`` Wrappers *are* NumPy Arrays!"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Vectors are wrapped by providing a 1D numpy view into the vector's memory with a dtype that matches the template type.\n",
      "\n",
      "```cpp\n",
      "std::vector<double> func0(double x, std::vector<double> vec) {\n",
      "  std::vector<double> d (vec.size(), 0.0);\n",
      "  return d;\n",
      "};\n",
      "```"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from cppproj.basics import a_better_name as func0"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "skip"
      }
     },
     "outputs": [],
     "prompt_number": 27
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "arr = np.array([6, 28, 496, 8128], dtype='f8')\n",
      "func0(42.0, arr)"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 28,
       "text": [
        "array([ 0.,  0.,  0.,  0.])"
       ]
      }
     ],
     "prompt_number": 28
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Dtypes\n",
      "\n",
      "This works by generating custom dtypes for all desired types - including user created templated classes!"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "```python\n",
      "dtypes = [\n",
      "    ('map', 'str', 'int'),\n",
      "    ('set', 'int'),\n",
      "    'float32',\n",
      "    ('vector', 'int32'),\n",
      "    'ThreeNums',\n",
      "    ]\n",
      "```"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "(Ufuncs have not been implemented yet.)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Dtypes"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from cppproj import dt\n",
      "print(dt.xd_double.name)\n",
      "print(\"Is dtype?\", isinstance(dt.xd_double, np.dtype))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "xd_double\n",
        "Is dtype? True\n"
       ]
      }
     ],
     "prompt_number": 29
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "**Numpy array of ``std::string``s**"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.array([\"Joan\", 'Hoover', 'Mulan', 'Leslie'], dtype=dt.xd_str)"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 30,
       "text": [
        "array(['Joan', 'Hoover', 'Mulan', 'Leslie'], dtype='xd_str')"
       ]
      }
     ],
     "prompt_number": 30
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "**And now for something truly evil...**"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.array([[42, 65], [6, 28, 496, 8128]], dtype=dt.xd_vector_double)"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 31,
       "text": [
        "array([array([ 42.,  65.]),\n",
        "       array([  6.00000000e+00,   2.80000000e+01,   4.96000000e+02,\n",
        "         8.12800000e+03])], dtype='xd_vector_double')"
       ]
      }
     ],
     "prompt_number": 31
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## C++ Standard Library Container Wrappers\n",
      "\n",
      "All of these wrappers are composable with other types, including classes and structs wrapped from ``cythongen``."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "yolo = conts.MapIntMapIntVectorBool({\n",
      "    42: {65: [1, 0, False, True]}, \n",
      "    18: {}, \n",
      "    2040: {3: [\"yes\", \"no\"]}})\n",
      "print(yolo)\n",
      "print(\"Value is MapIntVectorBool?\", isinstance(yolo[42], conts.MapIntVectorBool))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "{18: {}, 42: {65: array([ True, False, False,  True], dtype=bool)}, 2040: {3: array([ True,  True], dtype=bool)}}\n",
        "Value is MapIntVectorBool? True\n"
       ]
      }
     ],
     "prompt_number": 32
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Templates\n",
      "\n",
      "Templates must be specialized..."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "...but that doesn't mean that they can't look nice."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "```cpp\n",
      "template <class T, int U> bool lessthan(T x) {return (x < U ? true : false);};\n",
      "lessthan<int, 3>;\n",
      "```"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from cppproj.basics import lessthan, lessthan_int_3"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "skip"
      }
     },
     "outputs": [],
     "prompt_number": 33
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "lessthan[int, 3](2)"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 34,
       "text": [
        "True"
       ]
      }
     ],
     "prompt_number": 34
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "lessthan_int_3(42)"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 35,
       "text": [
        "False"
       ]
      }
     ],
     "prompt_number": 35
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Not Just for Code\n",
      "\n",
      "Doxygen comments are also translated into numpydoc rST and inserted as docstrings."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## `__future__`\n",
      "\n",
      "<div style=\"text-align:center;\"><h3>Operator Overloading</h3></div><br/>"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "<div style=\"text-align:center;\"><h3>Ctypes, Boost, Julia, and Other Frontends</h3></div><br/>"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "<div style=\"text-align:center;\"><h3>Fortran Support</h3></div><br/>"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "<div style=\"text-align:center;\"><h3>Monitoring API Changes</h3></div><br/>"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Questions\n",
      "\n",
      "<div style=\"text-align:center;\"><b>http://xdress.org/</b><br /><img src=\"magician.jpg\" style=\"margin:auto;\" width=\"30%\" /><br /><b>xdress@googlegroups.com</b></div>"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "!make"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "skip"
      }
     },
     "outputs": []
    }
   ],
   "metadata": {}
  }
 ]
}