DataImport.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Importing data from other sources"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "Julia offers a number of packages with which to import data from other languages and formats"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Importing from Matlab"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There is a `MAT` package that imports `.mat` files as a `Julia` dictionary. For this example, we will use the file located at <https://github.com/jmxpearson/duke-julia-ssri/carbig.mat>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "using MAT"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Dict{String,Any} with 13 entries:\n",
       "  \"Weight\"       => [3504.0; 3693.0; … ; 2625.0; 2720.0]\n",
       "  \"Acceleration\" => [12.0; 11.5; … ; 18.6; 19.4]\n",
       "  \"Mfg\"          => String[\"chevrolet\",\"buick\",\"plymouth\",\"amc\",\"ford\",\"ford\",\"…\n",
       "  \"cyl4\"         => String[\"Other\",\"Other\",\"Other\",\"Other\",\"Other\",\"Other\",\"Oth…\n",
       "  \"Origin\"       => String[\"USA\",\"USA\",\"USA\",\"USA\",\"USA\",\"USA\",\"USA\",\"USA\",\"USA…\n",
       "  \"when\"         => String[\"Early\",\"Early\",\"Early\",\"Early\",\"Early\",\"Early\",\"Ear…\n",
       "  \"Displacement\" => [307.0; 350.0; … ; 120.0; 119.0]\n",
       "  \"MPG\"          => [18.0; 15.0; … ; 28.0; 31.0]\n",
       "  \"Model\"        => String[\"chevrolet chevelle malibu\",\"buick skylark 320\",\"ply…\n",
       "  \"Cylinders\"    => [8.0; 8.0; … ; 4.0; 4.0]\n",
       "  \"org\"          => String[\"USA\",\"USA\",\"USA\",\"USA\",\"USA\",\"USA\",\"USA\",\"USA\",\"USA…\n",
       "  \"Model_Year\"   => [70.0; 70.0; … ; 82.0; 82.0]\n",
       "  \"Horsepower\"   => [130.0; 165.0; … ; 79.0; 82.0]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vars = matread(\"carbig.mat\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "Weight       = vars[\"Weight\"];\n",
    "Acceleration = vars[\"Acceleration\"];\n",
    "Mfg          = vars[\"Mfg\"];\n",
    "cyl4         = vars[\"cyl4\"];\n",
    "Origin       = vars[\"Origin\"];\n",
    "when         = vars[\"when\"];\n",
    "Displacement = vars[\"Displacement\"];\n",
    "MPG          = vars[\"MPG\"];\n",
    "Model        = vars[\"Model\"];\n",
    "Cylinders    = vars[\"Cylinders\"];\n",
    "org          = vars[\"org\"];\n",
    "Model_Year   = vars[\"Model_Year\"];\n",
    "Horsepower   = vars[\"Horsepower\"];"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                  Acceleration   3248 bytes  406×1 Array{Float64,2}\n",
      "                          Base  35290 KB     Module\n",
      "                         Blosc     45 KB     Module\n",
      "               BufferedStreams   5252 KB     Module\n",
      "                        Compat   5324 KB     Module\n",
      "                          Core  13366 KB     Module\n",
      "                     Cylinders   3248 bytes  406×1 Array{Float64,2}\n",
      "                  Displacement   3248 bytes  406×1 Array{Float64,2}\n",
      "                          HDF5   5835 KB     Module\n",
      "                    Horsepower   3248 bytes  406×1 Array{Float64,2}\n",
      "                        IJulia   6435 KB     Module\n",
      "                          JSON   5404 KB     Module\n",
      "                          Libz   5317 KB     Module\n",
      "                           MAT   5686 KB     Module\n",
      "                           MPG   3248 bytes  406×1 Array{Float64,2}\n",
      "                          Main  44110 KB     Module\n",
      "                           Mfg   9022 bytes  406-element Array{String,1}\n",
      "                         Model     12 KB     406-element Array{String,1}\n",
      "                    Model_Year   3248 bytes  406×1 Array{Float64,2}\n",
      "                        Nettle   5327 KB     Module\n",
      "                        Origin   8123 bytes  406-element Array{String,1}\n",
      "                        Weight   3248 bytes  406×1 Array{Float64,2}\n",
      "                           ZMQ   5335 KB     Module\n",
      "                          cyl4   8319 bytes  406-element Array{String,1}\n",
      "                           org   8091 bytes  406-element Array{String,1}\n",
      "                          vars     77 KB     Dict{String,Any} with 13 entries\n",
      "                          when   8097 bytes  406-element Array{String,1}\n"
     ]
    }
   ],
   "source": [
    "whos()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Exporting to Matlab"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Similarly, we can write to Matlab format by creating a dictionary the other direction:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X=rand(50000,5);\n",
    "matwrite(\"tester.mat\", Dict(\n",
    "    \"X\" => X,\n",
    "    \"Acceleration\" => Acceleration,\n",
    "    \"Horsepower\" => Horsepower\n",
    "));"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Importing delimited text files as Julia arrays"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As with other scientific software, it is straightforward to import any delimited  text file into Julia. Let's consider the file located at https://github.com/jmxpearson/duke-julia-ssri-2016/auto.csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "search: \u001b[1mr\u001b[22m\u001b[1me\u001b[22m\u001b[1ma\u001b[22m\u001b[1md\u001b[22m\u001b[1md\u001b[22m\u001b[1ml\u001b[22m\u001b[1mm\u001b[22m \u001b[1mr\u001b[22m\u001b[1me\u001b[22m\u001b[1ma\u001b[22m\u001b[1md\u001b[22m\u001b[1md\u001b[22mir \u001b[1mr\u001b[22m\u001b[1me\u001b[22m\u001b[1ma\u001b[22m\u001b[1md\u001b[22man\u001b[1md\u001b[22mwrite\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "```\n",
       "readdlm(source, T::Type; options...)\n",
       "```\n",
       "\n",
       "The columns are assumed to be separated by one or more whitespaces. The end of line delimiter is taken as `\\n`.\n",
       "\n",
       "```\n",
       "readdlm(source, delim::Char, T::Type; options...)\n",
       "```\n",
       "\n",
       "The end of line delimiter is taken as `\\n`.\n",
       "\n",
       "```\n",
       "readdlm(source; options...)\n",
       "```\n",
       "\n",
       "The columns are assumed to be separated by one or more whitespaces. The end of line delimiter is taken as `\\n`. If all data is numeric, the result will be a numeric array. If some elements cannot be parsed as numbers, a heterogeneous array of numbers and strings is returned.\n",
       "\n",
       "```\n",
       "readdlm(source, delim::Char; options...)\n",
       "```\n",
       "\n",
       "The end of line delimiter is taken as `\\n`. If all data is numeric, the result will be a numeric array. If some elements cannot be parsed as numbers, a heterogeneous array of numbers and strings is returned.\n",
       "\n",
       "```\n",
       "readdlm(source, delim::Char, eol::Char; options...)\n",
       "```\n",
       "\n",
       "If all data is numeric, the result will be a numeric array. If some elements cannot be parsed as numbers, a heterogeneous array of numbers and strings is returned.\n",
       "\n",
       "```\n",
       "readdlm(source, delim::Char, T::Type, eol::Char; header=false, skipstart=0, skipblanks=true, use_mmap, quotes=true, dims, comments=true, comment_char='#')\n",
       "```\n",
       "\n",
       "Read a matrix from the source where each line (separated by `eol`) gives one row, with elements separated by the given delimiter. The source can be a text file, stream or byte array. Memory mapped files can be used by passing the byte array representation of the mapped segment as source.\n",
       "\n",
       "If `T` is a numeric type, the result is an array of that type, with any non-numeric elements as `NaN` for floating-point types, or zero. Other useful values of `T` include `String`, `AbstractString`, and `Any`.\n",
       "\n",
       "If `header` is `true`, the first row of data will be read as header and the tuple `(data_cells, header_cells)` is returned instead of only `data_cells`.\n",
       "\n",
       "Specifying `skipstart` will ignore the corresponding number of initial lines from the input.\n",
       "\n",
       "If `skipblanks` is `true`, blank lines in the input will be ignored.\n",
       "\n",
       "If `use_mmap` is `true`, the file specified by `source` is memory mapped for potential speedups. Default is `true` except on Windows. On Windows, you may want to specify `true` if the file is large, and is only read once and not written to.\n",
       "\n",
       "If `quotes` is `true`, columns enclosed within double-quote (\") characters are allowed to contain new lines and column delimiters. Double-quote characters within a quoted field must be escaped with another double-quote.  Specifying `dims` as a tuple of the expected rows and columns (including header, if any) may speed up reading of large files.  If `comments` is `true`, lines beginning with `comment_char` and text following `comment_char` in any line are ignored.\n"
      ],
      "text/plain": [
       "```\n",
       "readdlm(source, T::Type; options...)\n",
       "```\n",
       "\n",
       "The columns are assumed to be separated by one or more whitespaces. The end of line delimiter is taken as `\\n`.\n",
       "\n",
       "```\n",
       "readdlm(source, delim::Char, T::Type; options...)\n",
       "```\n",
       "\n",
       "The end of line delimiter is taken as `\\n`.\n",
       "\n",
       "```\n",
       "readdlm(source; options...)\n",
       "```\n",
       "\n",
       "The columns are assumed to be separated by one or more whitespaces. The end of line delimiter is taken as `\\n`. If all data is numeric, the result will be a numeric array. If some elements cannot be parsed as numbers, a heterogeneous array of numbers and strings is returned.\n",
       "\n",
       "```\n",
       "readdlm(source, delim::Char; options...)\n",
       "```\n",
       "\n",
       "The end of line delimiter is taken as `\\n`. If all data is numeric, the result will be a numeric array. If some elements cannot be parsed as numbers, a heterogeneous array of numbers and strings is returned.\n",
       "\n",
       "```\n",
       "readdlm(source, delim::Char, eol::Char; options...)\n",
       "```\n",
       "\n",
       "If all data is numeric, the result will be a numeric array. If some elements cannot be parsed as numbers, a heterogeneous array of numbers and strings is returned.\n",
       "\n",
       "```\n",
       "readdlm(source, delim::Char, T::Type, eol::Char; header=false, skipstart=0, skipblanks=true, use_mmap, quotes=true, dims, comments=true, comment_char='#')\n",
       "```\n",
       "\n",
       "Read a matrix from the source where each line (separated by `eol`) gives one row, with elements separated by the given delimiter. The source can be a text file, stream or byte array. Memory mapped files can be used by passing the byte array representation of the mapped segment as source.\n",
       "\n",
       "If `T` is a numeric type, the result is an array of that type, with any non-numeric elements as `NaN` for floating-point types, or zero. Other useful values of `T` include `String`, `AbstractString`, and `Any`.\n",
       "\n",
       "If `header` is `true`, the first row of data will be read as header and the tuple `(data_cells, header_cells)` is returned instead of only `data_cells`.\n",
       "\n",
       "Specifying `skipstart` will ignore the corresponding number of initial lines from the input.\n",
       "\n",
       "If `skipblanks` is `true`, blank lines in the input will be ignored.\n",
       "\n",
       "If `use_mmap` is `true`, the file specified by `source` is memory mapped for potential speedups. Default is `true` except on Windows. On Windows, you may want to specify `true` if the file is large, and is only read once and not written to.\n",
       "\n",
       "If `quotes` is `true`, columns enclosed within double-quote (\") characters are allowed to contain new lines and column delimiters. Double-quote characters within a quoted field must be escaped with another double-quote.  Specifying `dims` as a tuple of the expected rows and columns (including header, if any) may speed up reading of large files.  If `comments` is `true`, lines beginning with `comment_char` and text following `comment_char` in any line are ignored.\n"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "?readdlm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "auto = readdlm(\"auto.csv\", ',',header=true);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(\n",
       "Any[\"AMC Concord\" 4099 … 3.58 0; \"AMC Pacer\" 4749 … 2.53 0; … ; \"VW Scirocco\" 6850 … 3.78 1; \"Volvo 260\" 11995 … 2.98 1],\n",
       "\n",
       "AbstractString[\"make\" \"price\" … \"gear_ratio\" \"foreign\"])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "auto"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Tuple{Array{Any,2},Array{AbstractString,2}}"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "typeof(auto)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "74×12 Array{Any,2}:\n",
       " \"AMC Concord\"      4099  22  3    2.5  11  2930  186  40  121  3.58  0\n",
       " \"AMC Pacer\"        4749  17  3    3.0  11  3350  173  40  258  2.53  0\n",
       " \"AMC Spirit\"       3799  22   \"\"  3.0  12  2640  168  35  121  3.08  0\n",
       " \"Buick Century\"    4816  20  3    4.5  16  3250  196  40  196  2.93  0\n",
       " \"Buick Electra\"    7827  15  4    4.0  20  4080  222  43  350  2.41  0\n",
       " \"Buick LeSabre\"    5788  18  3    4.0  21  3670  218  43  231  2.73  0\n",
       " \"Buick Opel\"       4453  26   \"\"  3.0  10  2230  170  34  304  2.87  0\n",
       " \"Buick Regal\"      5189  20  3    2.0  16  3280  200  42  196  2.93  0\n",
       " \"Buick Riviera\"   10372  16  3    3.5  17  3880  207  43  231  2.93  0\n",
       " \"Buick Skylark\"    4082  19  3    3.5  13  3400  200  42  231  3.08  0\n",
       " \"Cad. Deville\"    11385  14  3    4.0  20  4330  221  44  425  2.28  0\n",
       " \"Cad. Eldorado\"   14500  14  2    3.5  16  3900  204  43  350  2.19  0\n",
       " \"Cad. Seville\"    15906  21  3    3.0  13  4290  204  45  350  2.24  0\n",
       " ⋮                                       ⋮                      ⋮      \n",
       " \"Mazda GLC\"        3995  30  4    3.5  11  1980  154  33   86  3.73  1\n",
       " \"Peugeot 604\"     12990  14   \"\"  3.5  14  3420  192  38  163  3.58  1\n",
       " \"Renault Le Car\"   3895  26  3    3.0  10  1830  142  34   79  3.72  1\n",
       " \"Subaru\"           3798  35  5    2.5  11  2050  164  36   97  3.81  1\n",
       " \"Toyota Celica\"    5899  18  5    2.5  14  2410  174  36  134  3.06  1\n",
       " \"Toyota Corolla\"   3748  31  5    3.0   9  2200  165  35   97  3.21  1\n",
       " \"Toyota Corona\"    5719  18  5    2.0  11  2670  175  36  134  3.05  1\n",
       " \"VW Dasher\"        7140  23  4    2.5  12  2160  172  36   97  3.74  1\n",
       " \"VW Diesel\"        5397  41  5    3.0  15  2040  155  35   90  3.78  1\n",
       " \"VW Rabbit\"        4697  25  4    3.0  15  1930  155  35   89  3.78  1\n",
       " \"VW Scirocco\"      6850  25  4    2.0  16  1990  156  36   97  3.78  1\n",
       " \"Volvo 260\"       11995  17  5    2.5  14  3170  193  37  163  2.98  1"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "csvdata=auto[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "74-element Array{Any,1}:\n",
       "  4099\n",
       "  4749\n",
       "  3799\n",
       "  4816\n",
       "  7827\n",
       "  5788\n",
       "  4453\n",
       "  5189\n",
       " 10372\n",
       "  4082\n",
       " 11385\n",
       " 14500\n",
       " 15906\n",
       "     ⋮\n",
       "  3995\n",
       " 12990\n",
       "  3895\n",
       "  3798\n",
       "  5899\n",
       "  3748\n",
       "  5719\n",
       "  7140\n",
       "  5397\n",
       "  4697\n",
       "  6850\n",
       " 11995"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "csvdata[:,2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Notice that we can specify any type of delimter; we chose comma here because it is one of the most common."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Exporting Julia arrays as delimited files"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "To export, the process is simply reversed and we instead use the `writedlm()` function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "search: \u001b[1mw\u001b[22m\u001b[1mr\u001b[22m\u001b[1mi\u001b[22m\u001b[1mt\u001b[22m\u001b[1me\u001b[22m\u001b[1md\u001b[22m\u001b[1ml\u001b[22m\u001b[1mm\u001b[22m\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "```\n",
       "writedlm(f, A, delim='\\t'; opts)\n",
       "```\n",
       "\n",
       "Write `A` (a vector, matrix, or an iterable collection of iterable rows) as text to `f` (either a filename string or an [`IO`](:class:`IO`) stream) using the given delimiter `delim` (which defaults to tab, but can be any printable Julia object, typically a `Char` or `AbstractString`).\n",
       "\n",
       "For example, two vectors `x` and `y` of the same length can be written as two columns of tab-delimited text to `f` by either `writedlm(f, [x y])` or by `writedlm(f, zip(x, y))`.\n"
      ],
      "text/plain": [
       "```\n",
       "writedlm(f, A, delim='\\t'; opts)\n",
       "```\n",
       "\n",
       "Write `A` (a vector, matrix, or an iterable collection of iterable rows) as text to `f` (either a filename string or an [`IO`](:class:`IO`) stream) using the given delimiter `delim` (which defaults to tab, but can be any printable Julia object, typically a `Char` or `AbstractString`).\n",
       "\n",
       "For example, two vectors `x` and `y` of the same length can be written as two columns of tab-delimited text to `f` by either `writedlm(f, [x y])` or by `writedlm(f, zip(x, y))`.\n"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "?writedlm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "writedlm(\"autoout.csv\",csvdata,',');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Importing delimited files as Julia data frames"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you may have noticed, `readdlm()` was clunky when the file has mixed data types (e.g. strings and numbers). We can resolve this by directly importing to a data frame using the `readtable()` function in the `DataFrames` package:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "using DataFrames"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>make</th><th>price</th><th>mpg</th><th>rep78</th><th>headroom</th><th>trunk</th><th>weight</th><th>length</th><th>turn</th><th>displacement</th><th>gear_ratio</th><th>foreign</th></tr></thead><tbody><tr><th>1</th><td>AMC Concord</td><td>4099</td><td>22</td><td>3</td><td>2.5</td><td>11</td><td>2930</td><td>186</td><td>40</td><td>121</td><td>3.58</td><td>0</td></tr><tr><th>2</th><td>AMC Pacer</td><td>4749</td><td>17</td><td>3</td><td>3.0</td><td>11</td><td>3350</td><td>173</td><td>40</td><td>258</td><td>2.53</td><td>0</td></tr><tr><th>3</th><td>AMC Spirit</td><td>3799</td><td>22</td><td>NA</td><td>3.0</td><td>12</td><td>2640</td><td>168</td><td>35</td><td>121</td><td>3.08</td><td>0</td></tr><tr><th>4</th><td>Buick Century</td><td>4816</td><td>20</td><td>3</td><td>4.5</td><td>16</td><td>3250</td><td>196</td><td>40</td><td>196</td><td>2.93</td><td>0</td></tr><tr><th>5</th><td>Buick Electra</td><td>7827</td><td>15</td><td>4</td><td>4.0</td><td>20</td><td>4080</td><td>222</td><td>43</td><td>350</td><td>2.41</td><td>0</td></tr><tr><th>6</th><td>Buick LeSabre</td><td>5788</td><td>18</td><td>3</td><td>4.0</td><td>21</td><td>3670</td><td>218</td><td>43</td><td>231</td><td>2.73</td><td>0</td></tr><tr><th>7</th><td>Buick Opel</td><td>4453</td><td>26</td><td>NA</td><td>3.0</td><td>10</td><td>2230</td><td>170</td><td>34</td><td>304</td><td>2.87</td><td>0</td></tr><tr><th>8</th><td>Buick Regal</td><td>5189</td><td>20</td><td>3</td><td>2.0</td><td>16</td><td>3280</td><td>200</td><td>42</td><td>196</td><td>2.93</td><td>0</td></tr><tr><th>9</th><td>Buick Riviera</td><td>10372</td><td>16</td><td>3</td><td>3.5</td><td>17</td><td>3880</td><td>207</td><td>43</td><td>231</td><td>2.93</td><td>0</td></tr><tr><th>10</th><td>Buick Skylark</td><td>4082</td><td>19</td><td>3</td><td>3.5</td><td>13</td><td>3400</td><td>200</td><td>42</td><td>231</td><td>3.08</td><td>0</td></tr><tr><th>11</th><td>Cad. Deville</td><td>11385</td><td>14</td><td>3</td><td>4.0</td><td>20</td><td>4330</td><td>221</td><td>44</td><td>425</td><td>2.28</td><td>0</td></tr><tr><th>12</th><td>Cad. Eldorado</td><td>14500</td><td>14</td><td>2</td><td>3.5</td><td>16</td><td>3900</td><td>204</td><td>43</td><td>350</td><td>2.19</td><td>0</td></tr><tr><th>13</th><td>Cad. Seville</td><td>15906</td><td>21</td><td>3</td><td>3.0</td><td>13</td><td>4290</td><td>204</td><td>45</td><td>350</td><td>2.24</td><td>0</td></tr><tr><th>14</th><td>Chev. Chevette</td><td>3299</td><td>29</td><td>3</td><td>2.5</td><td>9</td><td>2110</td><td>163</td><td>34</td><td>231</td><td>2.93</td><td>0</td></tr><tr><th>15</th><td>Chev. Impala</td><td>5705</td><td>16</td><td>4</td><td>4.0</td><td>20</td><td>3690</td><td>212</td><td>43</td><td>250</td><td>2.56</td><td>0</td></tr><tr><th>16</th><td>Chev. Malibu</td><td>4504</td><td>22</td><td>3</td><td>3.5</td><td>17</td><td>3180</td><td>193</td><td>31</td><td>200</td><td>2.73</td><td>0</td></tr><tr><th>17</th><td>Chev. Monte Carlo</td><td>5104</td><td>22</td><td>2</td><td>2.0</td><td>16</td><td>3220</td><td>200</td><td>41</td><td>200</td><td>2.73</td><td>0</td></tr><tr><th>18</th><td>Chev. Monza</td><td>3667</td><td>24</td><td>2</td><td>2.0</td><td>7</td><td>2750</td><td>179</td><td>40</td><td>151</td><td>2.73</td><td>0</td></tr><tr><th>19</th><td>Chev. Nova</td><td>3955</td><td>19</td><td>3</td><td>3.5</td><td>13</td><td>3430</td><td>197</td><td>43</td><td>250</td><td>2.56</td><td>0</td></tr><tr><th>20</th><td>Dodge Colt</td><td>3984</td><td>30</td><td>5</td><td>2.0</td><td>8</td><td>2120</td><td>163</td><td>35</td><td>98</td><td>3.54</td><td>0</td></tr><tr><th>21</th><td>Dodge Diplomat</td><td>4010</td><td>18</td><td>2</td><td>4.0</td><td>17</td><td>3600</td><td>206</td><td>46</td><td>318</td><td>2.47</td><td>0</td></tr><tr><th>22</th><td>Dodge Magnum</td><td>5886</td><td>16</td><td>2</td><td>4.0</td><td>17</td><td>3600</td><td>206</td><td>46</td><td>318</td><td>2.47</td><td>0</td></tr><tr><th>23</th><td>Dodge St. Regis</td><td>6342</td><td>17</td><td>2</td><td>4.5</td><td>21</td><td>3740</td><td>220</td><td>46</td><td>225</td><td>2.94</td><td>0</td></tr><tr><th>24</th><td>Ford Fiesta</td><td>4389</td><td>28</td><td>4</td><td>1.5</td><td>9</td><td>1800</td><td>147</td><td>33</td><td>98</td><td>3.15</td><td>0</td></tr><tr><th>25</th><td>Ford Mustang</td><td>4187</td><td>21</td><td>3</td><td>2.0</td><td>10</td><td>2650</td><td>179</td><td>43</td><td>140</td><td>3.08</td><td>0</td></tr><tr><th>26</th><td>Linc. Continental</td><td>11497</td><td>12</td><td>3</td><td>3.5</td><td>22</td><td>4840</td><td>233</td><td>51</td><td>400</td><td>2.47</td><td>0</td></tr><tr><th>27</th><td>Linc. Mark V</td><td>13594</td><td>12</td><td>3</td><td>2.5</td><td>18</td><td>4720</td><td>230</td><td>48</td><td>400</td><td>2.47</td><td>0</td></tr><tr><th>28</th><td>Linc. Versailles</td><td>13466</td><td>14</td><td>3</td><td>3.5</td><td>15</td><td>3830</td><td>201</td><td>41</td><td>302</td><td>2.47</td><td>0</td></tr><tr><th>29</th><td>Merc. Bobcat</td><td>3829</td><td>22</td><td>4</td><td>3.0</td><td>9</td><td>2580</td><td>169</td><td>39</td><td>140</td><td>2.73</td><td>0</td></tr><tr><th>30</th><td>Merc. Cougar</td><td>5379</td><td>14</td><td>4</td><td>3.5</td><td>16</td><td>4060</td><td>221</td><td>48</td><td>302</td><td>2.75</td><td>0</td></tr><tr><th>&vellip;</th><td>&vellip;</td><td>&vellip;</td><td>&vellip;</td><td>&vellip;</td><td>&vellip;</td><td>&vellip;</td><td>&vellip;</td><td>&vellip;</td><td>&vellip;</td><td>&vellip;</td><td>&vellip;</td><td>&vellip;</td></tr></tbody></table>"
      ],
      "text/plain": [
       "74×12 DataFrames.DataFrame\n",
       "│ Row │ make             │ price │ mpg │ rep78 │ headroom │ trunk │ weight │\n",
       "├─────┼──────────────────┼───────┼─────┼───────┼──────────┼───────┼────────┤\n",
       "│ 1   │ \"AMC Concord\"    │ 4099  │ 22  │ 3     │ 2.5      │ 11    │ 2930   │\n",
       "│ 2   │ \"AMC Pacer\"      │ 4749  │ 17  │ 3     │ 3.0      │ 11    │ 3350   │\n",
       "│ 3   │ \"AMC Spirit\"     │ 3799  │ 22  │ NA    │ 3.0      │ 12    │ 2640   │\n",
       "│ 4   │ \"Buick Century\"  │ 4816  │ 20  │ 3     │ 4.5      │ 16    │ 3250   │\n",
       "│ 5   │ \"Buick Electra\"  │ 7827  │ 15  │ 4     │ 4.0      │ 20    │ 4080   │\n",
       "│ 6   │ \"Buick LeSabre\"  │ 5788  │ 18  │ 3     │ 4.0      │ 21    │ 3670   │\n",
       "│ 7   │ \"Buick Opel\"     │ 4453  │ 26  │ NA    │ 3.0      │ 10    │ 2230   │\n",
       "│ 8   │ \"Buick Regal\"    │ 5189  │ 20  │ 3     │ 2.0      │ 16    │ 3280   │\n",
       "│ 9   │ \"Buick Riviera\"  │ 10372 │ 16  │ 3     │ 3.5      │ 17    │ 3880   │\n",
       "│ 10  │ \"Buick Skylark\"  │ 4082  │ 19  │ 3     │ 3.5      │ 13    │ 3400   │\n",
       "│ 11  │ \"Cad. Deville\"   │ 11385 │ 14  │ 3     │ 4.0      │ 20    │ 4330   │\n",
       "⋮\n",
       "│ 63  │ \"Mazda GLC\"      │ 3995  │ 30  │ 4     │ 3.5      │ 11    │ 1980   │\n",
       "│ 64  │ \"Peugeot 604\"    │ 12990 │ 14  │ NA    │ 3.5      │ 14    │ 3420   │\n",
       "│ 65  │ \"Renault Le Car\" │ 3895  │ 26  │ 3     │ 3.0      │ 10    │ 1830   │\n",
       "│ 66  │ \"Subaru\"         │ 3798  │ 35  │ 5     │ 2.5      │ 11    │ 2050   │\n",
       "│ 67  │ \"Toyota Celica\"  │ 5899  │ 18  │ 5     │ 2.5      │ 14    │ 2410   │\n",
       "│ 68  │ \"Toyota Corolla\" │ 3748  │ 31  │ 5     │ 3.0      │ 9     │ 2200   │\n",
       "│ 69  │ \"Toyota Corona\"  │ 5719  │ 18  │ 5     │ 2.0      │ 11    │ 2670   │\n",
       "│ 70  │ \"VW Dasher\"      │ 7140  │ 23  │ 4     │ 2.5      │ 12    │ 2160   │\n",
       "│ 71  │ \"VW Diesel\"      │ 5397  │ 41  │ 5     │ 3.0      │ 15    │ 2040   │\n",
       "│ 72  │ \"VW Rabbit\"      │ 4697  │ 25  │ 4     │ 3.0      │ 15    │ 1930   │\n",
       "│ 73  │ \"VW Scirocco\"    │ 6850  │ 25  │ 4     │ 2.0      │ 16    │ 1990   │\n",
       "│ 74  │ \"Volvo 260\"      │ 11995 │ 17  │ 5     │ 2.5      │ 14    │ 3170   │\n",
       "\n",
       "│ Row │ length │ turn │ displacement │ gear_ratio │ foreign │\n",
       "├─────┼────────┼──────┼──────────────┼────────────┼─────────┤\n",
       "│ 1   │ 186    │ 40   │ 121          │ 3.58       │ 0       │\n",
       "│ 2   │ 173    │ 40   │ 258          │ 2.53       │ 0       │\n",
       "│ 3   │ 168    │ 35   │ 121          │ 3.08       │ 0       │\n",
       "│ 4   │ 196    │ 40   │ 196          │ 2.93       │ 0       │\n",
       "│ 5   │ 222    │ 43   │ 350          │ 2.41       │ 0       │\n",
       "│ 6   │ 218    │ 43   │ 231          │ 2.73       │ 0       │\n",
       "│ 7   │ 170    │ 34   │ 304          │ 2.87       │ 0       │\n",
       "│ 8   │ 200    │ 42   │ 196          │ 2.93       │ 0       │\n",
       "│ 9   │ 207    │ 43   │ 231          │ 2.93       │ 0       │\n",
       "│ 10  │ 200    │ 42   │ 231          │ 3.08       │ 0       │\n",
       "│ 11  │ 221    │ 44   │ 425          │ 2.28       │ 0       │\n",
       "⋮\n",
       "│ 63  │ 154    │ 33   │ 86           │ 3.73       │ 1       │\n",
       "│ 64  │ 192    │ 38   │ 163          │ 3.58       │ 1       │\n",
       "│ 65  │ 142    │ 34   │ 79           │ 3.72       │ 1       │\n",
       "│ 66  │ 164    │ 36   │ 97           │ 3.81       │ 1       │\n",
       "│ 67  │ 174    │ 36   │ 134          │ 3.06       │ 1       │\n",
       "│ 68  │ 165    │ 35   │ 97           │ 3.21       │ 1       │\n",
       "│ 69  │ 175    │ 36   │ 134          │ 3.05       │ 1       │\n",
       "│ 70  │ 172    │ 36   │ 97           │ 3.74       │ 1       │\n",
       "│ 71  │ 155    │ 35   │ 90           │ 3.78       │ 1       │\n",
       "│ 72  │ 155    │ 35   │ 89           │ 3.78       │ 1       │\n",
       "│ 73  │ 156    │ 36   │ 97           │ 3.78       │ 1       │\n",
       "│ 74  │ 193    │ 37   │ 163          │ 2.98       │ 1       │"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "autoDF = readtable(\"auto.csv\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Notice that the result allows for both strings and numbers to be stored, as well as missing values to be represented by `NA`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Other features of `readtable()`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`readtable()` offers the following other options:\n",
    "- Specify which strings convert to `NA` [default is `\"\"`]\n",
    "- Specify how text file is delimited [default is `','`]\n",
    "- Specify whether or not a header of column names exists"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Exporting Julia data frames as delimited text files"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `writetable()` function operates as the inverse of `readtable()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "writetable(\"auto1.csv\", autoDF, separator = ',', header = false);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Interfacing with other data types"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There exist packages for directly reading data from SAS, Stata, R, SPSS, etc. However, these packages are not well supported. (example: `DataRead` package)"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Julia 0.5.0",
   "language": "julia",
   "name": "julia-0.5"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "0.5.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}