tenth-tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Intermine-Python: Tutorial 10: Enrichment Calculations on Lists"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This tutorial will talk about how you can perform enrichment calculations on lists that you have access to. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from intermine.webservice import Service"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "service = Service(\"www.flymine.org/query/service\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The intermine service has various widgets that can perform different functions. These widgets are stored in a dicitionary in the Service class. To view all the widgets use service.widgets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'bdgp': {'chartType': 'ColumnChart',\n",
       "  'description': 'Expression patterns of Drosophila mRNAs during embryogenesis - data from  <a href=\"http://www.fruitfly.org/cgi-bin/ex/insitu.pl\" target=\"_new\">BGDP</a>.  Note that not all genes have been assayed by <a href=\"http://www.fruitfly.org/cgi-bin/ex/insitu.pl\" target=\"_new\">BGDP</a>. ',\n",
       "  'filters': None,\n",
       "  'labels': {'x': 'Stage', 'y': 'Gene count'},\n",
       "  'name': 'bdgp',\n",
       "  'targets': ['Gene'],\n",
       "  'title': 'BDGP expression patterns',\n",
       "  'widgetType': 'chart'},\n",
       " 'bdgp_enrichment': {'description': 'ImaGO terms enriched for genes in this list - data from <a href=\"http://www.fruitfly.org/cgi-bin/ex/insitu.pl\" target=\"_new\">BDGP</a>.  Note that not all genes have been assayed by BDGP.',\n",
       "  'filters': None,\n",
       "  'name': 'bdgp_enrichment',\n",
       "  'targets': ['Gene'],\n",
       "  'title': 'BDGP Enrichment',\n",
       "  'widgetType': 'enrichment'},\n",
       " 'chromosome_distribution_for_gene': {'chartType': 'ColumnChart',\n",
       "  'description': 'Actual: number of items in this list found on each chromosome.  Expected: given the total number of items on the chromosome and the number of items in this list, the number of items expected to be found on each chromosome.',\n",
       "  'filters': 'organism.name=[list]',\n",
       "  'labels': {'x': 'Chromosome', 'y': 'Count'},\n",
       "  'name': 'chromosome_distribution_for_gene',\n",
       "  'targets': ['SequenceFeature'],\n",
       "  'title': 'Chromosome Distribution',\n",
       "  'widgetType': 'chart'},\n",
       " 'flyatlas_for_gene': {'chartType': 'BarChart',\n",
       "  'description': 'For each tissue in the fly, the number of genes from this list for which the levels of expression are significantly high (Up) or low (Down) according to <a href=\"http://www.flyatlas.org/\" target=\"_new\">FlyAtlas</a> AffyCall.',\n",
       "  'filters': None,\n",
       "  'labels': {'x': 'Tissue', 'y': 'Up (+) or Down (-) gene count'},\n",
       "  'name': 'flyatlas_for_gene',\n",
       "  'targets': ['Gene'],\n",
       "  'title': 'Gene Expression in the Fly (FlyAtlas)',\n",
       "  'widgetType': 'chart'},\n",
       " 'flyatlas_for_probeset': {'chartType': 'BarChart',\n",
       "  'description': 'For each tissue, the number of genes from this list for which the levels of expression are significantly high (Up) or low (Down) according to <a href=\"http://www.flyatlas.org/\" target=\"_new\">FlyAtlas</a> AffyCall.',\n",
       "  'filters': None,\n",
       "  'labels': {'x': 'Tissue', 'y': 'Up (+) or Down (-) gene count'},\n",
       "  'name': 'flyatlas_for_probeset',\n",
       "  'targets': ['ProbeSet'],\n",
       "  'title': 'Gene Expression in Fly Tissues (FlyAtlas Data)',\n",
       "  'widgetType': 'chart'},\n",
       " 'flyfish': {'chartType': 'ColumnChart',\n",
       "  'description': 'Expression patterns of Drosophila mRNAs at the subcellular level during early embryogenesis - data from  <a href=\"http://fly-fish.ccbr.utoronto.ca/\" target=\"_new\">Fly-FISH</a>.  Note that not all genes have been assayed by <a href=\"http://fly-fish.ccbr.utoronto.ca/\" target=\"_new\">Fly-FISH</a>.',\n",
       "  'filters': None,\n",
       "  'labels': {'x': 'Stage', 'y': 'Gene count'},\n",
       "  'name': 'flyfish',\n",
       "  'targets': ['Gene'],\n",
       "  'title': 'mRNA subcellular localisation (fly-FISH)',\n",
       "  'widgetType': 'chart'},\n",
       " 'go_enrichment_for_gene': {'description': 'GO terms enriched for items in this list.',\n",
       "  'filters': 'biological_process,cellular_component,molecular_function',\n",
       "  'name': 'go_enrichment_for_gene',\n",
       "  'targets': ['Gene'],\n",
       "  'title': 'Gene Ontology Enrichment',\n",
       "  'widgetType': 'enrichment'},\n",
       " 'miranda_enrichment': {'description': 'MiRNAs enriched for items in this list.',\n",
       "  'filters': None,\n",
       "  'name': 'miranda_enrichment',\n",
       "  'targets': ['Gene'],\n",
       "  'title': 'MiRNA Enrichment',\n",
       "  'widgetType': 'enrichment'},\n",
       " 'orthologues': {'description': 'Counts of genes in list with orthologues.',\n",
       "  'filters': None,\n",
       "  'name': 'orthologues',\n",
       "  'targets': ['Gene'],\n",
       "  'title': 'Orthologues',\n",
       "  'widgetType': 'table'},\n",
       " 'pathway_enrichment': {'description': 'Pathways enriched for genes in this list - data from KEGG and Reactome',\n",
       "  'filters': 'All,KEGG pathways data set,Reactome data set',\n",
       "  'name': 'pathway_enrichment',\n",
       "  'targets': ['Gene'],\n",
       "  'title': 'Pathway Enrichment',\n",
       "  'widgetType': 'enrichment'},\n",
       " 'prot_dom_enrichment_for_gene': {'description': 'Protein Domains enriched for items in this list.',\n",
       "  'filters': None,\n",
       "  'name': 'prot_dom_enrichment_for_gene',\n",
       "  'targets': ['Gene'],\n",
       "  'title': 'Protein Domain Enrichment',\n",
       "  'widgetType': 'enrichment'},\n",
       " 'prot_dom_enrichment_for_protein': {'description': 'Protein Domains enriched for items in this list.',\n",
       "  'filters': None,\n",
       "  'name': 'prot_dom_enrichment_for_protein',\n",
       "  'targets': ['Protein'],\n",
       "  'title': 'Protein Domain Enrichment',\n",
       "  'widgetType': 'enrichment'},\n",
       " 'protein_features': {'description': 'UniProt features enriched for proteins in this list.',\n",
       "  'filters': None,\n",
       "  'name': 'protein_features',\n",
       "  'targets': ['Protein'],\n",
       "  'title': 'UniProt Features',\n",
       "  'widgetType': 'enrichment'},\n",
       " 'publication_enrichment': {'description': 'Publications enriched for genes in this list.',\n",
       "  'filters': None,\n",
       "  'name': 'publication_enrichment',\n",
       "  'targets': ['Gene'],\n",
       "  'title': 'Publication Enrichment',\n",
       "  'widgetType': 'enrichment'},\n",
       " 'uniprot_keywords': {'description': 'UniProt keywords enriched for proteins in this list.',\n",
       "  'filters': None,\n",
       "  'name': 'uniprot_keywords',\n",
       "  'targets': ['Protein'],\n",
       "  'title': 'UniProt Keywords',\n",
       "  'widgetType': 'enrichment'}}"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "service.widgets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you want to view only those widgets that are related to enrichment, you can use filter based on the widget type. I've printed all the enrichment related widgets below. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'filters': None, 'description': 'Protein Domains enriched for items in this list.', 'title': 'Protein Domain Enrichment', 'name': 'prot_dom_enrichment_for_protein', 'widgetType': 'enrichment', 'targets': ['Protein']}\n",
      "{'filters': 'All,KEGG pathways data set,Reactome data set', 'description': 'Pathways enriched for genes in this list - data from KEGG and Reactome', 'title': 'Pathway Enrichment', 'name': 'pathway_enrichment', 'widgetType': 'enrichment', 'targets': ['Gene']}\n",
      "{'filters': None, 'description': 'Protein Domains enriched for items in this list.', 'title': 'Protein Domain Enrichment', 'name': 'prot_dom_enrichment_for_gene', 'widgetType': 'enrichment', 'targets': ['Gene']}\n",
      "{'filters': None, 'description': 'UniProt features enriched for proteins in this list.', 'title': 'UniProt Features', 'name': 'protein_features', 'widgetType': 'enrichment', 'targets': ['Protein']}\n",
      "{'filters': 'biological_process,cellular_component,molecular_function', 'description': 'GO terms enriched for items in this list.', 'title': 'Gene Ontology Enrichment', 'name': 'go_enrichment_for_gene', 'widgetType': 'enrichment', 'targets': ['Gene']}\n",
      "{'filters': None, 'description': 'Publications enriched for genes in this list.', 'title': 'Publication Enrichment', 'name': 'publication_enrichment', 'widgetType': 'enrichment', 'targets': ['Gene']}\n",
      "{'filters': None, 'description': 'MiRNAs enriched for items in this list.', 'title': 'MiRNA Enrichment', 'name': 'miranda_enrichment', 'widgetType': 'enrichment', 'targets': ['Gene']}\n",
      "{'filters': None, 'description': 'UniProt keywords enriched for proteins in this list.', 'title': 'UniProt Keywords', 'name': 'uniprot_keywords', 'widgetType': 'enrichment', 'targets': ['Protein']}\n",
      "{'filters': None, 'description': 'ImaGO terms enriched for genes in this list - data from <a href=\"http://www.fruitfly.org/cgi-bin/ex/insitu.pl\" target=\"_new\">BDGP</a>.  Note that not all genes have been assayed by BDGP.', 'title': 'BDGP Enrichment', 'name': 'bdgp_enrichment', 'widgetType': 'enrichment', 'targets': ['Gene']}\n"
     ]
    }
   ],
   "source": [
    "for key in service.widgets.keys():\n",
    "    if service.widgets[key][\"widgetType\"]==\"enrichment\":\n",
    "        print(service.widgets[key])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will now get a list on which we want to perform the analysis using the list manager. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "lm=service.list_manager()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "l=lm.get_list(name=\"PL FlyAtlas_brain_top\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To perform any enrichment analysis on the list use the calculate_enrichment method. I've stored the results in \"r\". We will now iterate through R and view the results. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "r=l.calculate_enrichment(widget=\"publication_enrichment\",maxp=0.01)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "22683328 Mutation of Drosophila dopamine receptor DopR leads to male-male courtship behavior. 0.0004688789939719088\n",
      "23895060 Temporally dimorphic recruitment of dopamine neurons into stress response circuitry in Drosophila. 0.0004688789939719088\n",
      "24128361 Sexually dimorphic recruitment of dopamine neurons into the stress response circuitry. 0.0004688789939719088\n",
      "15987944 Rapid, nongenomic responses to ecdysteroids and catecholamines mediated by a novel Drosophila G-protein-coupled receptor. 0.0018724024048451117\n",
      "17986026 Suppression of excitatory cholinergic synaptic transmission by Drosophila dopamine D1-like receptors. 0.0018724024048451117\n",
      "24303109 Pharmacological analysis of dopamine modulation in the Drosophila melanogaster larval heart. 0.0018724024048451117\n",
      "21286249 Sleep deprivation during early-adult development results in long-lasting learning deficits in adult Drosophila. 0.009331628487951982\n",
      "21455291 Dopamine signalling in mushroom bodies regulates temperature-preference behaviour in Drosophila. 0.009331628487951982\n",
      "25632118 Behavioral and circuit basis of sucrose rejection by Drosophila females in a simple decision-making task. 0.009331628487951982\n"
     ]
    }
   ],
   "source": [
    "for i in r:\n",
    "    print(i.identifier,i.description,i.p_value)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "This is how you can perform enrichment calculations on lists of your choice. "
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [conda root]",
   "language": "python",
   "name": "conda-root-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}