04_venv_pip.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Week 4 - Managing depdendencies using venv and pip"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Why do we care?\n",
    "\n",
    "You project will probably import and call functions in projects written by others. We call these project depdendencies. \n",
    "\n",
    "When you work on multiple projects, you want to make sure that each project has its own versions of depdendencies. You don't want the dependencies from one project interfering with the dependencies of another. \n",
    "\n",
    "Also, if you work with others, you want to ensure that everyone has the same dependency versions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## Installing dependencies into a project's dependency directory\n",
    "\n",
    "You install project depdendencies using `npm` for node or `pip` for python. \n",
    "\n",
    "In node, your dependencies are automatically placed in a `node_modules` subdirectory of your project. Node is smart enough to know that when you are in a project directory, you want to get the dependencies from the `node_modules` subdirectory.\n",
    "\n",
    "Python doesn't do this automatically for you. You have to create the depdendency directory and tell python to use it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "jp-MarkdownHeadingCollapsed": true,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## Creating the dependency directory in python\n",
    "\n",
    "In python, project dependencies by convention are stored in a `.venv` subdirectory of your project. `venv` stands for *virtual environment*.  \n",
    "\n",
    "To create the `.venv` subdirectory for your project's virtual environment, run this command in your project directory: \n",
    "\n",
    "    python -m venv .venv\n",
    "\n",
    "This creates a `.venv` subdirectory and configures it to hold your project dependencies."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "jp-MarkdownHeadingCollapsed": true,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## Activating virtual environments in python\n",
    "\n",
    "Unlike, node, in python you have to manually **activate** the virtual environment each time you work on the project. When the virtual environment is activated, `pip` will put dependencies in the `.venv` directory and python will read the dependencies from the `.venv` directory.\n",
    "\n",
    "To activate your virtual environment, type the first line for bash or zsh, or the second line for fish:\n",
    "\n",
    "    source .venv/bin/activate\n",
    "    source .venv/bin/activate.fish\n",
    "\n",
    "To deactivate your python environment type `deactivate`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "jp-MarkdownHeadingCollapsed": true,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## The global python environment\n",
    "\n",
    "If you forget to activate a virtual environment before installing dependencies with pip, then those dependencies will go into the global python environment. In general, you should avoid this. You don't want to use the global python environment."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "jp-MarkdownHeadingCollapsed": true,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## Installing python dependencies using pip\n",
    "\n",
    "Pip is a package manager for python, similar to how npm is a package manager for node.\n",
    "\n",
    "Once your virtual environment is active, you can install project dependencies using pip. For example, to install the python requests library you would type:\n",
    "\n",
    "    pip install requests"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "jp-MarkdownHeadingCollapsed": true,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## Using your virtual environment in VS Code\n",
    "\n",
    "Open VS Code on the current directory\n",
    "\n",
    "    code .\n",
    "    \n",
    "Create a new python file called **test.py** that uses the requests dependency:\n",
    "\n",
    "    import requests\n",
    "    r = requests.get(\"https://jsonplaceholder.typicode.com/posts/1\")\n",
    "    print(r.json())\n",
    "    \n",
    "Before you can execute this file, you need to tell VS Code which python environment you want to run it in. Type **control-shift-p** to open the command palette, followed by **python: select interpreter**, and click on **('.venv': venv)**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "jp-MarkdownHeadingCollapsed": true,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## Sharing your python virtual environment with others\n",
    "\n",
    "In node, dependency versions go in `package.json` and `package-lock.json` so others can install the same dependency versions that you have. \n",
    "\n",
    "Python doesn't do this automatically. You have to create the dependency versions file yourself. By convention this file is called `requirements.txt`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "jp-MarkdownHeadingCollapsed": true,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## Creating requirements.txt\n",
    "\n",
    "An easy way to create requirements.txt is by running:\n",
    "\n",
    "    pip freeze > requirements.txt\n",
    "\n",
    "This writes every dependency in your current python virtual environment along with their versions to requirements.txt, very similar to package-lock.json. \n",
    "\n",
    "I don't like this approach, because it includes all of your dependencies' dependencies. It results in a huge requirements.txt file and it's impossible to see what your true top-level project dependencies are. It's as if you just had package-lock.json and not package.json."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "jp-MarkdownHeadingCollapsed": true,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## A better way to create requirements.txt\n",
    "\n",
    "Another way to create requirements.txt is each time you use pip to install a dependency:\n",
    "\n",
    "    pip install requests\n",
    "\n",
    "which installs requests plus all of its dependencies as before. But now use pip freeze followed by grep to find the version of requests that got installed and append just that to your requirements.txt file:\n",
    "\n",
    "    pip freeze | grep \"requests==\" >> requirements.txt\n",
    "\n",
    "This ensures that your requirements.txt file includes only your top-level dependencies."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "jp-MarkdownHeadingCollapsed": true,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## Installing dependencies from requirements.txt\n",
    "\n",
    "Once you have created the requirements.txt file, check it into git.\n",
    "\n",
    "When someone else clones your project, they first create the virtual environment directory:\n",
    "\n",
    "    python -m venv .venv\n",
    "\n",
    "And activate the virtual environment by typing one of the following as before:\n",
    "\n",
    "    source .venv/bin/activate\n",
    "    source .venv/bin/activate.fish\n",
    "\n",
    "Then install the dependencies listed in requirements.txt\n",
    "\n",
    "    pip install -r requirements.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "jp-MarkdownHeadingCollapsed": true,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## MISE to the rescue\n",
    "\n",
    "It can be easy to forget to activate the virtual environment. Fortunately, mise can do that for you automatically:\n",
    "\n",
    "Edit `.mise.toml` and change `python=\"3.11\"` to \n",
    "\n",
    "    python = {version=\"3.11\", virtualenv=\".venv\"}\n",
    "\n",
    "Then run (this only needs to be done once):\n",
    "\n",
    "    mise settings set experimental true\n",
    "    mise settings set python_venv_auto_create true\n",
    "\n",
    "MISE will now create and activate the python virtual environment automatically whenever you cd to your project directory."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## Conclusion\n",
    "\n",
    "- There is an even better way to manage virtual environments and dependencies than venv and pip, and it's called `poetry`. It's a bit more complicated than venv and pip. We will learn about it next week. I use venv and pip for simple projects and `poetry` for larger projects."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}