Workshop_Workflow1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Creating an InterMIne workflow using the API."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are going to re-create the workflow we did using the web interface using the python API."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We start by importing the Service class from Intermine's webservice module.  You will need to access your account on humanMine and you do this through an API token. You can get your token by logging into humanmine and going to the account details tab within MyMine.  Cut and paste your token into the code below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from intermine.webservice import Service\n",
    "service = Service(\"http://www.humanmine.org/humanmine/service\", token = \"Your Token\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our first query looked at genes that are upregulated in adipose tissue.  Using the API we can either generate a query object or a template object to do this.  The code below shows how to generate a query object.  The \"AtlasExpression\" passed to the query object defines the query class.  To run the template through the API is very similar except we generate a template object rather than a query object (template = service.get_template('TissueAtlas_Expression'). TissueAtlas_Expression is the name of the template)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "query = service.new_query(\"AtlasExpression\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First we will define the output columns that we want in our result - i.e the view.  Note that we have started our query from the Atlas Expression class.  \"Condition\", \"expression\", \"pValue\" and \"tStatistic\" are attributes of this class.  The gene class is referenced from the AtlasExpression class, so to return the gene information we give the path to that information from the Atlas Expression class - i.e gene.primaryIdentifier etc."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "query.add_view(\n",
    "    \"condition\", \"gene.primaryIdentifier\", \"gene.symbol\", \"gene.name\",\n",
    "    \"expression\", \"pValue\", \"tStatistic\", \"dataSets.name\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, add the constraints to your query.  We are only interested in genes expressed in Adipose tissue with a pValue <= 0.01."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "query.add_constraint(\"condition\", \"=\", \"Adipose tissue\")\n",
    "query.add_constraint(\"pValue\", \"<=\", \"0.01\")\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's check what the query returns by looping through the rows and printing the results:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for row in query.rows():\n",
    "    print (row[\"condition\"], row[\"gene.primaryIdentifier\"], row[\"gene.symbol\"], row[\"gene.name\"], \n",
    "        row[\"expression\"], row[\"pValue\"], row[\"tStatistic\"], row[\"dataSets.name\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that this gives a lot of rows.  If we just want to check we are getting the right results we could print just the first 10 rows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for row in query.rows(start=0,size=10):\n",
    "    print (row[\"condition\"], row[\"gene.primaryIdentifier\"], row[\"gene.symbol\"], row[\"gene.name\"], \n",
    "        row[\"expression\"], row[\"pValue\"], row[\"tStatistic\"], row[\"dataSets.name\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, remember that when we looked at the results table we used the filter options to show just the genes that are \"UP\" expressed in Adipose tissue.  We can do this by adding another constraint to our query.  (We could have added this straight away in our first set of constraints)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "query.add_constraint(\"expression\", \"=\", \"UP\", code = \"A\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's check our results again."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "for row in query.rows(start=0, size=10):\n",
    "    print (row[\"condition\"], row[\"gene.primaryIdentifier\"], row[\"gene.symbol\"], row[\"gene.name\"], \n",
    "        row[\"expression\"], row[\"pValue\"], row[\"tStatistic\"], row[\"dataSets.name\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We want to save this set of genes that are UP expressed in adipose for further analysis.  To do this we define our python list and loop through our results again - this time, instead of printing the results, we append just the primary identifiers returned to our list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "UpinAdipose = list()\n",
    "for row in query.rows():\n",
    "    UpinAdipose.append(row[\"gene.primaryIdentifier\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and check that the list we have created looks correct:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(UpinAdipose)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now need to save the list to our intermine account so we can use it again in a later query.  The ListManager class provides methods to manage list contents and operations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "lm=service.list_manager()\n",
    "lm.create_list(content=UpinAdipose, list_type=\"Gene\", name=\"UpinAdipose\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Log in to HumanMine and check your list has been created."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our second query looked at whether any of the genes that were UP expressed in adipose tissue interact with the pparg gene. First, we define our new query object.  This time we start our query from the Gene class:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "query2 = service.new_query(\"Gene\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Add the views and constraints:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "query2.add_view(\n",
    "    \"primaryIdentifier\", \"symbol\",\n",
    "    \"interactions.participant2.primaryIdentifier\",\n",
    "    \"interactions.participant2.symbol\", \"interactions.details.type\",\n",
    "    \"interactions.details.role1\", \"interactions.details.role2\",\n",
    "    \"interactions.details.experiment.interactionDetectionMethods.name\",\n",
    "    \"interactions.details.experiment.publication.pubMedId\",\n",
    "    \"interactions.details.dataSets.name\"\n",
    ")\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "query2.add_constraint(\"Gene\", \"LOOKUP\", \"pparg\", \"H. sapiens\", code = \"A\")\n",
    "query2.add_constraint(\"interactions.participant2\", \"IN\", \"UpinAdipose\", code = \"B\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In an interaction we have two participants.  Our first participant is from the Gene class and we have constrained this to be the gene PPARG.  Note that the pparg constraint is a LOOKUP. The lookup operator searches through all the fields in a particular class for the value specified. In the example given below, it will search through the entire gene class to find if any of the fields has an occurence of \"pparg\". The advantage of this is that you do not need to remember if pparg is a symbol or a name or a primaryIdentifier. Our second participant is from the interactions class and called participant2.  This is a bioentity like Gene and so shares some of the attributes, like primary identifier and symbol."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check the results:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for row in query2.rows():\n",
    "    print (row[\"primaryIdentifier\"], row[\"symbol\"], \n",
    "        row[\"interactions.participant2.primaryIdentifier\"], row[\"interactions.participant2.symbol\"], \n",
    "        row[\"interactions.details.type\"], row[\"interactions.details.role1\"], \n",
    "        row[\"interactions.details.role2\"], \n",
    "        row[\"interactions.details.experiment.interactionDetectionMethods.name\"], \n",
    "        row[\"interactions.details.experiment.publication.pubMedId\"], \n",
    "        row[\"interactions.details.dataSets.name\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Save the genes that interact with pparg to a list and save this list to your intermine account.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "UpinAdiposeInteractPparg = list()\n",
    "for row in query2.rows():\n",
    "    UpinAdiposeInteractPparg.append(row[\"interactions.participant2.primaryIdentifier\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "lm=service.list_manager()\n",
    "lm.create_list(content=UpinAdiposeInteractPparg, list_type=\"Gene\", name=\"UpinAdiposeInteractPparg\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, run the third query (genes that are associated with the diesease diabetes, that we originally created using the query builder) and again, save the set of genes that are returned to your intermine account."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "query3 = service.new_query(\"Gene\")\n",
    "query3.add_view(\"primaryIdentifier\", \"symbol\")\n",
    "query3.add_constraint(\"organism.name\", \"=\", \"Homo sapiens\", code = \"A\")\n",
    "query3.add_constraint(\"diseases.name\", \"CONTAINS\", \"diabetes\", code = \"B\")\n",
    "\n",
    "for row in query3.rows():\n",
    "    print (row[\"primaryIdentifier\"], row[\"symbol\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "diabetesGenes = list()\n",
    "for row in query3.rows():\n",
    "    diabetesGenes.append(row[\"primaryIdentifier\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "lm=service.list_manager()\n",
    "lm.create_list(content=diabetesGenes, list_type=\"Gene\", name=\"diabetesGenes\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we used a list intersect to find those genes that are upregulated in adipose tissue that also interact with pparg, that are also associated with the diease diabetes.  We need to intersect the second (UpinAdiposeInteractPparg) and third (diabetesGenes) lists that we created.  We can do this using the intersect method from the ListManager class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "lm.intersect([\"UpinAdiposeInteractPparg\", \"diabetesGenes\"], \"intersectedList\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "final = lm.get_list(\"intersectedList\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(final)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for gene in final:\n",
    "        print (gene.primaryIdentifier, gene.symbol)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}