technical-notes/1-6-markov-channels.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Technical note: Markov channel simulation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This document lays out some of the basics of Markov models, and provides a formal mathematical notation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Important note about terminology\n",
    "\n",
    "The term \"Markov model\" is used in this document to describe what is known in physics and chemistry as a [kinetic scheme](https://en.wikipedia.org/wiki/Kinetic_scheme) or [master equation](https://en.wikipedia.org/wiki/Master_equation).\n",
    "It is subtly different from the way \"Markov model\" is used in probabilistics, which is the more common form in single-channel analysis (following the work of e.q. Colquhoun et al.). In particular, we use the form\n",
    "\n",
    "$$\\frac{dp}{dt} = A p$$\n",
    "\n",
    "as used in the [master equation](https://en.wikipedia.org/wiki/Master_equation) rather than the form\n",
    "\n",
    "$$\\frac{dp}{dt} = p Q$$\n",
    "\n",
    "as used in [Q-matrix notation](https://en.wikipedia.org/wiki/Continuous-time_Markov_chain#Transient_behaviour).\n",
    "To convert, note that $A = Q^{T}$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Single-channel models\n",
    "\n",
    "The fundamental assumption is that a channel is always in one of $n$ states $S_i$.\n",
    "Writing $x(t)$ for the channel's state at time $t$, we can express this as $x(t) \\in \\{S_1, S_2, \\ldots, S_n\\}$.\n",
    "The channel can transition instantaneoulsy between states, with rates characterised by $r_{ij}$ with $i,j \\in \\{1,2,\\ldots,n\\}$.\n",
    "This is represented schematically using notation similar to that for  chemical reactions, for example:\n",
    "\n",
    "\\begin{equation*}\n",
    "S_1 \\underset{r_{21}}{\\stackrel{r_{12}}{\\rightleftharpoons}} S_2 \\underset{r_{32}}{\\stackrel{r_{23}}{\\rightleftharpoons}} S_3\n",
    "\\end{equation*}\n",
    "\n",
    "At any time $t$, the chance of finding the channel in state $S_i$ is $s_i(t)$,\n",
    " such that $s_i \\in [0,1]$ and $\\sum s_i = 1$.\n",
    "\n",
    "\\begin{equation}\n",
    "s_i(t) = P[x(t) = S_i]\n",
    "\\end{equation}\n",
    "\n",
    "Transition rates $r_{ij}$ are strictly non-negative, and depend on the membrane  potential $V$ and a set of model parameters $\\underline{p}$.\n",
    "While $V$ may change over time, the rates themselves are not explicitly time-dependent.\n",
    "We assume that all transitions are reversible, so that the existence of any transition $S_i \\rightarrow S_j$ implies the existence of a transition  $S_j \\rightarrow S_i$.\n",
    "The transition rate from a state to itself is defined as zero.\n",
    "All other rates are either zero (if the states aren't connected) or given by some model-specific function $f_{ij}\\big(V(t), \\underline{p}\\big)$.\n",
    "\n",
    "\\begin{equation}\n",
    "r_{ij}(t) = \\begin{cases}\n",
    "    f_{ij}\\left(V(t), \\underline{p}\\right) & \\text{if $i \\neq $j} \\\\\n",
    "    0 & \\text{if $i = j$}\n",
    "\\end{cases}\n",
    "\\end{equation}\n",
    "\n",
    "The choice of functions $f_{ij}$ changes per model and is sometimes \\emph{ad-hoc} and sometimes based on physical assumptions.\n",
    "Similarly, the number of states and connections is chosen freely by the modeler."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One or multiple of the model's states allow it to conduct ionic currents.\n",
    "Here, we assume that all conducting states have the same fixed conductance $g_{max}$, and that the channel only conducts a single species of ion.\n",
    "Using an Ohmic driving force $(V - E)$ we find:\n",
    "\n",
    "\\begin{eqnarray}\n",
    "I(t) & = & g_\\text{max} \\cdot g(x(t)) \\cdot (V(t) - E) \\\\\n",
    "g(x(t)) & = & \\begin{cases}\n",
    "    1 & \\text{if } x(t) \\in S_\\text{conducting} \\\\\n",
    "    0 & \\text{otherwise}\n",
    "\\end{cases}\n",
    "\\end{eqnarray}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### State transitions\n",
    "\n",
    "The meaning of the transition rates is defined by postulating that, for a channel in state $S_i$ at time $t$, the probability of a transition $S_i \\rightarrow S_j$ occurring in the next infinitesimal interval $dt$ is approximately equal to $r_{ij}(t) dt$.\n",
    "\n",
    "More accurately:\n",
    "\n",
    "\\begin{equation}\n",
    "P\\Big[x(t + dt) = S_j \\,|\\, x(t) = S_i\\Big]\n",
    " = r_{ij}(t) dt + o(dt)\n",
    " ,\\quad\\quad i \\neq j\n",
    "\\end{equation}\n",
    "\n",
    "Where $o(dt)$ is an error term that vanishes with decreasing $dt$ (but is otherwise unbounded).\n",
    "\n",
    "The chance of a change $S_i \\rightarrow S_j$ occurring during the interval $(t, t + dt]$ is equal to the chance of the transition multiplied by the probability of being in state $S_i$ at time t:\n",
    "\n",
    "\\begin{equation}\n",
    "P\\Big[x(t) = S_i \\cap x(t + dt) = S_j \\Big]  = s_i(t)r_{ij}(t) dt + o(dt)\n",
    " ,\\quad\\quad i \\neq j\n",
    "\\end{equation}\n",
    "\n",
    "We can now write an equation for the change in a probability $s_i(t)$ during the interval $dt$, by summing the probability of changing to $S_i$ from any connected state $S_j$ and subtracting the probability of changing from $S_i$ to any state $S_j$.\n",
    "\n",
    "\\begin{equation}\n",
    "s_i(t + dt) - s_i(t) =\n",
    "    \\sum_{j=1}^n{s_j(t)r_{ji}(t)dt}\n",
    "     - \\sum_{j=1}^n{s_i(t)r_{ij}(t)dt}\n",
    "      + o(dt)\n",
    "\\end{equation}\n",
    "\n",
    "From which we can derive the ordinary differential equation (ODE) form:\n",
    "\n",
    "\\begin{equation}\n",
    "\\label{eq:derived-ode-form}\n",
    "\\frac{d}{dt}s_i(t) = \\sum_{j=1}^n({s_jr_{ji} - s_ir_{ij}})\n",
    "\\end{equation}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Formulation as an ODE\n",
    "\n",
    "Using the equation above we can write a system of ordinary differential equations (ODEs) that describe the probability of the system being in each state over time.\n",
    "\n",
    "For the three-state example\n",
    "\n",
    "\\begin{equation*}\n",
    "S_1 \\underset{r_{21}}{\\stackrel{r_{12}}{\\rightleftharpoons}} S_2 \\underset{r_{32}}{\\stackrel{r_{23}}{\\rightleftharpoons}} S_3\n",
    "\\end{equation*}\n",
    "\n",
    "we find\n",
    "\n",
    "\\begin{equation}\n",
    "\\begin{array}{ccccc}\n",
    "\\dot{s}_{1} & = & \\,\\,\\,\\,\\,\\left(s_{2}r_{21}-s_{1}r_{12}\\right)\\\\\n",
    "\\dot{s}_{2} & = & -\\left(s_{2}r_{21}-s_{1}r_{12}\\right) & + & \\,\\,\\,\\,\\,\\left(s_{3}r_{32}-s_{2}r_{23}\\right)\\\\\n",
    "\\dot{s}_{3} & = &  &  & -\\left(s_{3}r_{32}-s_{2}r_{23}\\right)\n",
    "\\end{array}\n",
    "\\end{equation}\n",
    "\n",
    "The transition rates are typically calculated using non-linear functions of $V$.\n",
    "\n",
    "However, for situations where $V$ is (piecewise) constant, we can evaluate the functions $f_{ij}$ to obtain a linearised system with time-independent rates $r_{ij}$.\n",
    "We can then define a matrix\n",
    "\n",
    "\\begin{equation}\n",
    "R = [r_{ij}]\n",
    "\\end{equation}\n",
    "\n",
    "and write the system of ODEs as\n",
    "\n",
    "\\begin{equation}\n",
    "\\dot{\\underline{s}} = A \\underline{s}\n",
    "    = \\left( R^{T} - \\text{diag}\\left(R\\underline{1}\\right) \\right) \\underline{s}\n",
    "\\end{equation}\n",
    "\n",
    "Here $\\underline{1}$ is a column-vector of $n$ ones and $\\text{diag}$ is the operator described below."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For the three-state example:\n",
    "\n",
    "\\begin{equation}\n",
    "\\frac{d}{dt}\n",
    "\\left[\\begin{array}{c}\n",
    "s_1 \\\\\n",
    "s_2 \\\\\n",
    "s_3 \\\\ \n",
    "\\end{array}\\right] = \\left[\\begin{array}{ccc}\n",
    "-r_{12} & r_{21}           & 0       \\\\\n",
    "r_{12}  & -r_{21} - r_{23} & r_{32}  \\\\\n",
    "0       & r_{23}           & -r_{32} \\\\\n",
    "\\end{array} \\right] \\left[\n",
    "\\begin{array}{c}\n",
    "s_1 \\\\\n",
    "s_2 \\\\\n",
    "s_3 \\\\\n",
    "\\end{array} \\right]\n",
    "= A\\underline{s}\n",
    "\\end{equation}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and\n",
    "\n",
    "\\begin{align*}\n",
    "A &=\n",
    "\\left[\\begin{array}{ccc}\n",
    "0       & r_{21} & 0      \\\\\n",
    "r_{12}  & 0      & r_{32} \\\\\n",
    "0       & r_{23} & 0      \\\\\n",
    "\\end{array}\\right]\n",
    "+ \\left[\\begin{array}{ccc}\n",
    "r_{12} & 0               & 0      \\\\\n",
    "0      & r_{21} + r_{23} & 0      \\\\\n",
    "0      & 0               & r_{32} \\\\\n",
    "\\end{array}\\right] \\\\\n",
    "&=\n",
    "R^{T} - \\text{diag}\\left(\\left[\\begin{array}{ccc}\n",
    "r_{12} \\\\\n",
    "r_{21} + r_{23} \\\\\n",
    "r_{32} \\\\\n",
    "\\end{array}\\right]\\right)\n",
    "\\\\\n",
    "&=\n",
    "R^{T} - \\text{diag}\\left(R\\underline{1}\\right)\n",
    "\\\\\n",
    "\\end{align*}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### The `diag()` operator\n",
    "\n",
    "The $\\text{diag}()$ operator, when applied to an $(n \\times 1)$ vector $\\underline{u}$ creates an $(n \\times n)$ diagonal matrix $D$ such that $D_{ii} = u_i$.\n",
    "\n",
    "\\begin{equation}\n",
    "\\text{diag}\\left[\n",
    "\\begin{array}{c}\n",
    "3\\\\5\\\\1\\\\4\n",
    "\\end{array}\n",
    "\\right] = \\left(\n",
    "\\begin{array}{cccc}\n",
    "3 & 0 & 0 & 0 \\\\\n",
    "0 & 5 & 0 & 0 \\\\\n",
    "0 & 0 & 1 & 0 \\\\\n",
    "0 & 0 & 0 & 4 \\\\\n",
    "\\end{array}\n",
    "\\right)\n",
    "\\end{equation}\n",
    "\n",
    "When applied to an $(n \\times n)$ matrix $A$, the $\\text{diag}()$ operator returns a vector containing only the elements on the diagonal.\n",
    "\n",
    "\\begin{equation}\n",
    "\\text{diag}\\left(\n",
    "\\begin{array}{cccc}\n",
    "7 & 0 & 0 & 2 \\\\\n",
    "2 & 2 & 1 & 8 \\\\\n",
    "6 & 3 & 1 & 9 \\\\\n",
    "1 & 7 & 6 & 5 \\\\\n",
    "\\end{array} \n",
    "\\right) = \\left[\n",
    "\\begin{array}{c}\n",
    "7\\\\2\\\\1\\\\5\n",
    "\\end{array}\n",
    "\\right]\n",
    "\\end{equation}\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### $A$ is an indefinite matrix\n",
    "\n",
    "Note that the sum of the values in each column of $A$ is always zero.\n",
    "This means that the system is, by definition, indefinite.\n",
    "However, since $\\sum{s_i}=1$, we can remove an arbitrary state from the system,  solve it, and then calculate the removed state as\n",
    "\n",
    "\\begin{equation}\n",
    "s_i = 1 - \\sum_{j \\neq i}{s_j}\n",
    "\\end{equation}."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Obtaining $R$ from $A$\n",
    "\n",
    "Given a matrix $A$ such that $\\dot{\\underline{s}} = A \\underline{s}$, the rate constant matrix $R$ can be obtained using\n",
    "\n",
    "\\begin{equation}\n",
    "R^{T} = A - \\text{diag}\\left(\\text{diag}\\left(A\\right)\\right)\n",
    "\\end{equation}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Loops: Microscopic reversibility\n",
    "\n",
    "Biophysical constraint: If loops then sum both ways is equal, otherwise energy consumption / generation.\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Solving the ODE\n",
    "\n",
    "For proof, see \"Kattrin Arning (2009) Mathematical Modelling and Simulation of Ion Channels\".\n",
    "For pairwise different eigenvalues, the solution ``converges to the equilibrium  solution as a sum of n-1 exponentials''.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Modeling whole-cell current\n",
    "\n",
    "The same system of ODEs used to describe single ion channels can be used to describe the aggregated \"whole-cell\" current created when multiple identical ion channels are measured at the same time.\n",
    "To do this, we assume that the number of channels in the cell is large enough for the fraction of channels in each state $S_i$ to equal $s_i$ exactly.\n",
    "\n",
    "The conductance can now be defined as the fraction of channels in a conducting state:\n",
    "\n",
    "\\begin{equation}\n",
    "G\\big(V(t), t\\big) = \\sum_\\text{conducting} s_i\n",
    "\\end{equation}\n",
    "\n",
    "For the combined current, we find\n",
    "\n",
    "\\begin{equation}\n",
    "I = G_\\text{max} \\cdot G\\big(V(t), t\\big) \\cdot \\big(V(t) - E\\big)\n",
    "\\end{equation}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Assuming there are $m$ identical (but not synchronised!) channels, the maximum conductance can, in principle, be broken down as\n",
    "\n",
    "\\begin{equation*}\n",
    "G_\\text{max} = m \\cdot g_\\text{max}\n",
    "\\end{equation*}\n",
    "\n",
    "where $g_\\text{max}$ is the single channel conductance.\n",
    "\n",
    "However, since both these quantities are typically unknown when a model is created, $G_\\text{max}$ is commonly set to scale the current to an appropriate size.\n",
    "This may lead to the inclusion of an unknown scaling constant in its definition, that should really have been applied to the transition rates:\n",
    "\n",
    "\\begin{equation*}\n",
    "G_\\text{max} = \\alpha m \\cdot g_\\text{max}\n",
    "\\end{equation*}\n",
    "\n",
    "As a result, it is very dangerous to draw conclusions about the number of channels or single channel conductance from a current model's \"maximum conductance\" constant."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Discrete simulation\n",
    "\n",
    "[Gillespie's algorithm](https://en.wikipedia.org/wiki/Gillespie_algorithm) can be used to simulate a group of $m$ channels with stochastic behavior, under the assumption of a constant membrane potential $V$.\n",
    "\n",
    "First, we define a state vector $\\underline{z}(t)$ such that $z_i(t)$ is the number of channels in state $S_i$ at time $t$.\n",
    "\n",
    "The initial state of the system is written as \n",
    "$$\\underline{z}(t=t_0) = \\underline{z}_0$$\n",
    "\n",
    "We number every transition $S_i \\rightarrow S_j$ so that we can refer to a transition $S_i \\rightarrow S_j$ as $R_k$ using the appropriate index $k$.\n",
    "\n",
    "Rates are defined as:\n",
    "$$\\lambda_{k}(t, z) = r_{ij} z_i(t)$$\n",
    "\n",
    "The sum of all $\\lambda_{k}$ is written as \n",
    "$$\\lambda(t, z) = \\sum{\\lambda_{ij}(t, z)}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The algorithm proceeds in the following steps:\n",
    "\n",
    "1. Initialise the system to $t = t_0$ and $\\underline{z} = \\underline{z}_0$\n",
    "2. Calculate the rates $\\lambda_{k}$ and the sum $\\lambda$.\n",
    "3. Determine the time $\\tau$ until the next transition (of any kind) by sampling from an exponential distribution with mean $1/\\lambda$.\n",
    "4. Determine the type of transition by drawing a random number $r_2$ from a uniform distribution on the interval $[0, \\lambda)$.\n",
    "   Choose transition $T_1$ if $0 \\leq r_2 < \\lambda_1$, choose $T_2$ if $\\lambda_1 \\leq r_2 < \\lambda_2$ and so on.\n",
    "5. Update the state by moving one channel through the selected transition.\n",
    "   Increase the time variable by $\\tau$.\n",
    "6. Repeat steps 2 to 5 until $t > t_\\text{end}$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Isolating Markov models from cell ephys models\n",
    "\n",
    "TODO"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Analytical solution\n",
    "\n",
    "In some situations, the membrane potential $V(t)$ is fixed to some (piecewise) constant value and the transition rates $r_{ij}$ become constant.\n",
    "In such cases, for example when simulating a voltage-clamp experiment, the  whole-cell currents can be calculated by solving the system of ODEs using eigenvalue decomposition."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Whole-cell simulation\n",
    "\n",
    "If $V(t)$ cannot be fixed, for example when simulating an action potential or an imperfect (i.e. realistic) voltage-clamp experiment, the system of ODEs to solve is non-linear.\n",
    "In these cases a numerical quadrature method is used, for example the forward Euler method, an adaptive Runge-Kutta method or advanced methods such as CVODE."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}