notebooks/general_accessing_10x_snRNASeq_tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Accessing 10x RNA-seq gene expression data\n",
    "\n",
    "This notebook provides examples and functions for accessing the 10X expression matrix data stored in the ABC Atlas. These files require a large amount of memory to be available to load and analyize them if care is not taken. In this notebook, we present an example of how to access specific gene expressions from the data. The functions used below could be simplily parallized when processing data at scale, however, we leave them simple here.\n",
    "\n",
    "Care should still be taken not to attempt too load to many genes from the expression matrices."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from pathlib import Path\n",
    "import numpy as np\n",
    "import anndata\n",
    "\n",
    "from abc_atlas_access.abc_atlas_cache.abc_project_cache import AbcProjectCache\n",
    "from abc_atlas_access.abc_atlas_cache.anndata_utils import get_gene_data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will interact with the data using the **AbcProjectCache**. This cache object tracks which data has been downloaded and serves the path to the requsted data on disk. For metadata, the cache can also directly serve a up a Pandas Dataframe. See the ``getting_started`` notebook for more details on using the cache including installing it if it has not already been.\n",
    "\n",
    "**Change the download_base variable to where you have downloaded the data in your system.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'releases/20250131/manifest.json'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "download_base = Path('../../data/abc_atlas')\n",
    "abc_cache = AbcProjectCache.from_cache_dir(download_base)\n",
    "\n",
    "abc_cache.current_manifest"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Gene expression matrices\n",
    "\n",
    "The Whole Mouse Brain (WMB) and Whole Human Brain (WHB) datasets are formatted similarly. Each package is formatted as annadata h5ad files with minimal metadata. For each dataset, there are two h5ad files: one storing the raw counts and the other log2 normalization of the counts.\n",
    "\n",
    "To load the data by gene for either mouse or human dataset, we need to load two pieces of metadata, the ``cell``s table and the ``gene``s table in addition to our instantiated AbcProjectCache object. These metadata can be found in the directories WMB-10X and WHB-10Xv3 for mouse and human respectively. Below we use the human brain data in our example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "cell_metadata.csv: 100%|██████████| 830M/830M [00:24<00:00, 34.2MMB/s]    \n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cell_barcode</th>\n",
       "      <th>barcoded_cell_sample_label</th>\n",
       "      <th>library_label</th>\n",
       "      <th>feature_matrix_label</th>\n",
       "      <th>entity</th>\n",
       "      <th>brain_section_label</th>\n",
       "      <th>library_method</th>\n",
       "      <th>donor_label</th>\n",
       "      <th>donor_sex</th>\n",
       "      <th>dataset_label</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>cluster_alias</th>\n",
       "      <th>region_of_interest_label</th>\n",
       "      <th>anatomical_division_label</th>\n",
       "      <th>abc_sample_id</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cell_label</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>10X386_2:CATGGATTCTCGACGG</th>\n",
       "      <td>CATGGATTCTCGACGG</td>\n",
       "      <td>10X386_2</td>\n",
       "      <td>LKTX_210825_01_B01</td>\n",
       "      <td>WHB-10Xv3-Neurons</td>\n",
       "      <td>nuclei</td>\n",
       "      <td>H19.30.001.CX.51</td>\n",
       "      <td>10Xv3</td>\n",
       "      <td>H19.30.001</td>\n",
       "      <td>M</td>\n",
       "      <td>WHB-10Xv3</td>\n",
       "      <td>7.533584</td>\n",
       "      <td>-15.230048</td>\n",
       "      <td>20</td>\n",
       "      <td>Human MoAN</td>\n",
       "      <td>Myelencephalon</td>\n",
       "      <td>b3d3a6c0-3dc5-4738-af4c-96cc5ff6033f</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10X383_5:TCTTGCGGTGAATTGA</th>\n",
       "      <td>TCTTGCGGTGAATTGA</td>\n",
       "      <td>10X383_5</td>\n",
       "      <td>LKTX_210818_02_E01</td>\n",
       "      <td>WHB-10Xv3-Neurons</td>\n",
       "      <td>nuclei</td>\n",
       "      <td>H19.30.002.BS.94</td>\n",
       "      <td>10Xv3</td>\n",
       "      <td>H19.30.002</td>\n",
       "      <td>M</td>\n",
       "      <td>WHB-10Xv3</td>\n",
       "      <td>2.307856</td>\n",
       "      <td>-15.542040</td>\n",
       "      <td>20</td>\n",
       "      <td>Human MoSR</td>\n",
       "      <td>Myelencephalon</td>\n",
       "      <td>0a7ed99b-d5eb-4c50-b95e-eb31b448d2c7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10X386_2:CTCATCGGTCGAGCAA</th>\n",
       "      <td>CTCATCGGTCGAGCAA</td>\n",
       "      <td>10X386_2</td>\n",
       "      <td>LKTX_210825_01_B01</td>\n",
       "      <td>WHB-10Xv3-Neurons</td>\n",
       "      <td>nuclei</td>\n",
       "      <td>H19.30.001.CX.51</td>\n",
       "      <td>10Xv3</td>\n",
       "      <td>H19.30.001</td>\n",
       "      <td>M</td>\n",
       "      <td>WHB-10Xv3</td>\n",
       "      <td>6.740066</td>\n",
       "      <td>-16.186017</td>\n",
       "      <td>17</td>\n",
       "      <td>Human MoAN</td>\n",
       "      <td>Myelencephalon</td>\n",
       "      <td>458f6409-af8d-4c7c-ac9b-0b0d9b73b36d</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10X378_8:TTGGATGAGACAAGCC</th>\n",
       "      <td>TTGGATGAGACAAGCC</td>\n",
       "      <td>10X378_8</td>\n",
       "      <td>LKTX_210809_01_H01</td>\n",
       "      <td>WHB-10Xv3-Neurons</td>\n",
       "      <td>nuclei</td>\n",
       "      <td>H19.30.002.BS.93</td>\n",
       "      <td>10Xv3</td>\n",
       "      <td>H19.30.002</td>\n",
       "      <td>M</td>\n",
       "      <td>WHB-10Xv3</td>\n",
       "      <td>5.926133</td>\n",
       "      <td>-20.015151</td>\n",
       "      <td>18</td>\n",
       "      <td>Human PnAN</td>\n",
       "      <td>Pons</td>\n",
       "      <td>ff6b2d95-3752-49bf-9421-f17107538868</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10X387_7:TGAACGTAGTATTCCG</th>\n",
       "      <td>TGAACGTAGTATTCCG</td>\n",
       "      <td>10X387_7</td>\n",
       "      <td>LKTX_210825_02_G01</td>\n",
       "      <td>WHB-10Xv3-Neurons</td>\n",
       "      <td>nuclei</td>\n",
       "      <td>H19.30.001.CX.51</td>\n",
       "      <td>10Xv3</td>\n",
       "      <td>H19.30.001</td>\n",
       "      <td>M</td>\n",
       "      <td>WHB-10Xv3</td>\n",
       "      <td>5.622083</td>\n",
       "      <td>-13.561958</td>\n",
       "      <td>16</td>\n",
       "      <td>Human MoAN</td>\n",
       "      <td>Myelencephalon</td>\n",
       "      <td>589a1c5c-839b-480d-babc-4098c467211b</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                               cell_barcode barcoded_cell_sample_label  \\\n",
       "cell_label                                                               \n",
       "10X386_2:CATGGATTCTCGACGG  CATGGATTCTCGACGG                   10X386_2   \n",
       "10X383_5:TCTTGCGGTGAATTGA  TCTTGCGGTGAATTGA                   10X383_5   \n",
       "10X386_2:CTCATCGGTCGAGCAA  CTCATCGGTCGAGCAA                   10X386_2   \n",
       "10X378_8:TTGGATGAGACAAGCC  TTGGATGAGACAAGCC                   10X378_8   \n",
       "10X387_7:TGAACGTAGTATTCCG  TGAACGTAGTATTCCG                   10X387_7   \n",
       "\n",
       "                                library_label feature_matrix_label  entity  \\\n",
       "cell_label                                                                   \n",
       "10X386_2:CATGGATTCTCGACGG  LKTX_210825_01_B01    WHB-10Xv3-Neurons  nuclei   \n",
       "10X383_5:TCTTGCGGTGAATTGA  LKTX_210818_02_E01    WHB-10Xv3-Neurons  nuclei   \n",
       "10X386_2:CTCATCGGTCGAGCAA  LKTX_210825_01_B01    WHB-10Xv3-Neurons  nuclei   \n",
       "10X378_8:TTGGATGAGACAAGCC  LKTX_210809_01_H01    WHB-10Xv3-Neurons  nuclei   \n",
       "10X387_7:TGAACGTAGTATTCCG  LKTX_210825_02_G01    WHB-10Xv3-Neurons  nuclei   \n",
       "\n",
       "                          brain_section_label library_method donor_label  \\\n",
       "cell_label                                                                 \n",
       "10X386_2:CATGGATTCTCGACGG    H19.30.001.CX.51          10Xv3  H19.30.001   \n",
       "10X383_5:TCTTGCGGTGAATTGA    H19.30.002.BS.94          10Xv3  H19.30.002   \n",
       "10X386_2:CTCATCGGTCGAGCAA    H19.30.001.CX.51          10Xv3  H19.30.001   \n",
       "10X378_8:TTGGATGAGACAAGCC    H19.30.002.BS.93          10Xv3  H19.30.002   \n",
       "10X387_7:TGAACGTAGTATTCCG    H19.30.001.CX.51          10Xv3  H19.30.001   \n",
       "\n",
       "                          donor_sex dataset_label         x          y  \\\n",
       "cell_label                                                               \n",
       "10X386_2:CATGGATTCTCGACGG         M     WHB-10Xv3  7.533584 -15.230048   \n",
       "10X383_5:TCTTGCGGTGAATTGA         M     WHB-10Xv3  2.307856 -15.542040   \n",
       "10X386_2:CTCATCGGTCGAGCAA         M     WHB-10Xv3  6.740066 -16.186017   \n",
       "10X378_8:TTGGATGAGACAAGCC         M     WHB-10Xv3  5.926133 -20.015151   \n",
       "10X387_7:TGAACGTAGTATTCCG         M     WHB-10Xv3  5.622083 -13.561958   \n",
       "\n",
       "                           cluster_alias region_of_interest_label  \\\n",
       "cell_label                                                          \n",
       "10X386_2:CATGGATTCTCGACGG             20               Human MoAN   \n",
       "10X383_5:TCTTGCGGTGAATTGA             20               Human MoSR   \n",
       "10X386_2:CTCATCGGTCGAGCAA             17               Human MoAN   \n",
       "10X378_8:TTGGATGAGACAAGCC             18               Human PnAN   \n",
       "10X387_7:TGAACGTAGTATTCCG             16               Human MoAN   \n",
       "\n",
       "                          anatomical_division_label  \\\n",
       "cell_label                                            \n",
       "10X386_2:CATGGATTCTCGACGG            Myelencephalon   \n",
       "10X383_5:TCTTGCGGTGAATTGA            Myelencephalon   \n",
       "10X386_2:CTCATCGGTCGAGCAA            Myelencephalon   \n",
       "10X378_8:TTGGATGAGACAAGCC                      Pons   \n",
       "10X387_7:TGAACGTAGTATTCCG            Myelencephalon   \n",
       "\n",
       "                                                  abc_sample_id  \n",
       "cell_label                                                       \n",
       "10X386_2:CATGGATTCTCGACGG  b3d3a6c0-3dc5-4738-af4c-96cc5ff6033f  \n",
       "10X383_5:TCTTGCGGTGAATTGA  0a7ed99b-d5eb-4c50-b95e-eb31b448d2c7  \n",
       "10X386_2:CTCATCGGTCGAGCAA  458f6409-af8d-4c7c-ac9b-0b0d9b73b36d  \n",
       "10X378_8:TTGGATGAGACAAGCC  ff6b2d95-3752-49bf-9421-f17107538868  \n",
       "10X387_7:TGAACGTAGTATTCCG  589a1c5c-839b-480d-babc-4098c467211b  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cell = abc_cache.get_metadata_dataframe(\n",
    "    directory='WHB-10Xv3',\n",
    "    file_name='cell_metadata'\n",
    ").set_index('cell_label')\n",
    "cell.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "gene.csv: 100%|██████████| 4.23M/4.23M [00:00<00:00, 21.1MMB/s]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>gene_symbol</th>\n",
       "      <th>biotype</th>\n",
       "      <th>name</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>gene_identifier</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>ENSG00000000003</th>\n",
       "      <td>TSPAN6</td>\n",
       "      <td>protein_coding</td>\n",
       "      <td>tetraspanin 6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ENSG00000000005</th>\n",
       "      <td>TNMD</td>\n",
       "      <td>protein_coding</td>\n",
       "      <td>tenomodulin</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ENSG00000000419</th>\n",
       "      <td>DPM1</td>\n",
       "      <td>protein_coding</td>\n",
       "      <td>dolichyl-phosphate mannosyltransferase subunit...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ENSG00000000457</th>\n",
       "      <td>SCYL3</td>\n",
       "      <td>protein_coding</td>\n",
       "      <td>SCY1 like pseudokinase 3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ENSG00000000460</th>\n",
       "      <td>C1orf112</td>\n",
       "      <td>protein_coding</td>\n",
       "      <td>chromosome 1 open reading frame 112</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                gene_symbol         biotype  \\\n",
       "gene_identifier                               \n",
       "ENSG00000000003      TSPAN6  protein_coding   \n",
       "ENSG00000000005        TNMD  protein_coding   \n",
       "ENSG00000000419        DPM1  protein_coding   \n",
       "ENSG00000000457       SCYL3  protein_coding   \n",
       "ENSG00000000460    C1orf112  protein_coding   \n",
       "\n",
       "                                                              name  \n",
       "gene_identifier                                                     \n",
       "ENSG00000000003                                      tetraspanin 6  \n",
       "ENSG00000000005                                        tenomodulin  \n",
       "ENSG00000000419  dolichyl-phosphate mannosyltransferase subunit...  \n",
       "ENSG00000000457                           SCY1 like pseudokinase 3  \n",
       "ENSG00000000460                chromosome 1 open reading frame 112  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gene = abc_cache.get_metadata_dataframe(directory='WHB-10Xv3', file_name='gene').set_index('gene_identifier')\n",
    "gene.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Loading specific genes from the data\n",
    "\n",
    "The Whole Human Brain dataset [(Siletti et al. 2023)](https://www.science.org/doi/10.1126/science.add7046) consists of two sub-datasets: Neuron cells and Non-neuron cells. The neuron files are 30 GB in size and if we were to attempt to slice the dataset by gene we would have to load >30 GB once the data is uncompressed into memory. To avoid this, we load the data in chunks and recombine them into a pandas dataframe with all cells and the requested genes loaded.\n",
    "\n",
    "Below, we specify a select set of genes from the data that we'll load into our output data frame. These are the same genes that are available in the example file ``example_genes_all_cells_expression`` in the WHB-10Xv3 metadata directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "gene_names = ['SLC17A6', 'SLC17A7', 'SLC32A1', 'PTPRC', 'PLP1', 'AQP4', 'TTR']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below is a function that can be used to load specific genes from the from the full set of data in the WHB dataset. Adjust the chunk size down if you find your system still running out of memory. Additionally, reduce the number of genes if you still have issues. Note that the funciton can load either the raw expression data or the log2 data. The full code of this function can be found [here](https://github.com/AllenInstitute/abc_atlas_access/blob/7a53a08cff0f07e9b67911c0db04fa6932fa6e9d/src/abc_atlas_access/abc_atlas_cache/anndata_utils.py#L9)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\u001b[0;31mSignature:\u001b[0m\n",
       "\u001b[0mget_gene_data\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mabc_atlas_cache\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mabc_atlas_access\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mabc_atlas_cache\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mabc_project_cache\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mAbcProjectCache\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mall_cells\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mpandas\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcore\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mframe\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mDataFrame\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mall_genes\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mpandas\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcore\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mframe\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mDataFrame\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mselected_genes\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mList\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mstr\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mdata_type\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mstr\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m'log2'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mchunk_size\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mint\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m8192\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
       "\u001b[0;31mDocstring:\u001b[0m\n",
       "Load expression matrix data from the ABC Atlas and extract data for\n",
       "specific genes.\n",
       "\n",
       "Method will load all expression data required to process across multiple\n",
       "files to extract the full set of genes. This may result in downloading\n",
       "potentially ~100 GB of data.\n",
       "\n",
       "Parameters\n",
       "----------\n",
       "abc_atlas_cache: AbcProjectCache\n",
       "    An AbcProjectCache instance object to handle downloading and serving\n",
       "    the path to the expression matrix data.\n",
       "all_cells: pandas.DataFrame\n",
       "    cells metadata loaded as a pandas Dataframe from the AbcProjectCache\n",
       "    indexed on cell_label.\n",
       "all_genes: pandas.DataFrame\n",
       "    genes metadata loaded as a pandas Dataframe from the AbcProjectCache\n",
       "    indexed on gene_identifier.\n",
       "selected_genes: list of strings\n",
       "    List of gene_symbols that are a subset of those in the full genes\n",
       "    DataFrame.\n",
       "data_type: str (Default: \"log2\")\n",
       "    Kind of expression matrix to load either \"log2\" or \"raw\". Defaults to\n",
       "    \"log2\".\n",
       "chunk_size: int (Default: 8192)\n",
       "    Size of the chunk to load from the anndata files. Adjust this size if\n",
       "    needed based on memory/file io. Default: 8192.\n",
       "\n",
       "Returns\n",
       "-------\n",
       "output_gene_data: pandas.DataFrame\n",
       "    Subset of gene data indexed by cell.\n",
       "\u001b[0;31mFile:\u001b[0m      /allen/scratch/aibstemp/chris.morrison/src/abc_atlas_access/src/abc_atlas_access/abc_atlas_cache/anndata_utils.py\n",
       "\u001b[0;31mType:\u001b[0m      function"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "?get_gene_data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The code commented out below will create a gene expression DataFrame over the full WHB dataset. Running this full process takes around 10 minutes to processes, however downloading the full data can take up to an hour depending on your download speed. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'\\ngene_data = get_gene_data(\\n    abc_atlas_cache=abc_cache,\\n    all_cells=cell,\\n    all_genes=gene,\\n    selected_genes=gene_names\\n)'"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\"\"\"\n",
    "gene_data = get_gene_data(\n",
    "    abc_atlas_cache=abc_cache,\n",
    "    all_cells=cell,\n",
    "    all_genes=gene,\n",
    "    selected_genes=gene_names\n",
    ")\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For a quicker example, we'll load the Non-neurons from the WHB dataset only. This should take roughly a handful of minutes to complete and around 10 minutes download depending on your connection speed. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loading file: WHB-10Xv3-Nonneurons\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WHB-10Xv3-Nonneurons-log2.h5ad: 100%|██████████| 4.78G/4.78G [02:25<00:00, 33.0MMB/s]    \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " - time taken:  251.78768398399998\n",
      "total time taken: 297.553464587\n",
      "\ttotal cells: 888263 processed cells: 888263\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>gene_symbol</th>\n",
       "      <th>PTPRC</th>\n",
       "      <th>SLC17A6</th>\n",
       "      <th>SLC32A1</th>\n",
       "      <th>SLC17A7</th>\n",
       "      <th>TTR</th>\n",
       "      <th>PLP1</th>\n",
       "      <th>AQP4</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cell_label</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>10X362_3:TCAGTGAGTATTGACC</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>10.177927</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10X362_5:TCCGTGTGTGAAAGTT</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9.262379</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10X362_5:CACGGGTAGAGCAGAA</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>11.240114</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10X362_5:GATTCTTGTATGTCAC</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>8.314513</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10X362_6:AGGACTTGTATCCTTT</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9.736156</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10X194_8:GAAATGAGTTCGGCTG</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>10.210833</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10X350_4:TTTACCATCGCACGAC</th>\n",
       "      <td>9.587301</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>8.006084</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10X225_1:AGAAGCGTCCATATGG</th>\n",
       "      <td>8.567961</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10X221_5:TTGAACGCAGCCTTCT</th>\n",
       "      <td>10.119619</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10X385_3:CTACCCAGTGGCGCTT</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>888263 rows × 7 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "gene_symbol                    PTPRC SLC17A6 SLC32A1 SLC17A7  TTR       PLP1  \\\n",
       "cell_label                                                                     \n",
       "10X362_3:TCAGTGAGTATTGACC        0.0     0.0     0.0     0.0  0.0  10.177927   \n",
       "10X362_5:TCCGTGTGTGAAAGTT        0.0     0.0     0.0     0.0  0.0   9.262379   \n",
       "10X362_5:CACGGGTAGAGCAGAA        0.0     0.0     0.0     0.0  0.0  11.240114   \n",
       "10X362_5:GATTCTTGTATGTCAC        0.0     0.0     0.0     0.0  0.0   8.314513   \n",
       "10X362_6:AGGACTTGTATCCTTT        0.0     0.0     0.0     0.0  0.0   9.736156   \n",
       "...                              ...     ...     ...     ...  ...        ...   \n",
       "10X194_8:GAAATGAGTTCGGCTG        0.0     0.0     0.0     0.0  0.0  10.210833   \n",
       "10X350_4:TTTACCATCGCACGAC   9.587301     0.0     0.0     0.0  0.0   8.006084   \n",
       "10X225_1:AGAAGCGTCCATATGG   8.567961     0.0     0.0     0.0  0.0        0.0   \n",
       "10X221_5:TTGAACGCAGCCTTCT  10.119619     0.0     0.0     0.0  0.0        0.0   \n",
       "10X385_3:CTACCCAGTGGCGCTT        0.0     0.0     0.0     0.0  0.0        0.0   \n",
       "\n",
       "gene_symbol               AQP4  \n",
       "cell_label                      \n",
       "10X362_3:TCAGTGAGTATTGACC  0.0  \n",
       "10X362_5:TCCGTGTGTGAAAGTT  0.0  \n",
       "10X362_5:CACGGGTAGAGCAGAA  0.0  \n",
       "10X362_5:GATTCTTGTATGTCAC  0.0  \n",
       "10X362_6:AGGACTTGTATCCTTT  0.0  \n",
       "...                        ...  \n",
       "10X194_8:GAAATGAGTTCGGCTG  0.0  \n",
       "10X350_4:TTTACCATCGCACGAC  0.0  \n",
       "10X225_1:AGAAGCGTCCATATGG  0.0  \n",
       "10X221_5:TTGAACGCAGCCTTCT  0.0  \n",
       "10X385_3:CTACCCAGTGGCGCTT  0.0  \n",
       "\n",
       "[888263 rows x 7 columns]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nonneuron_cells = cell[cell['feature_matrix_label'] == 'WHB-10Xv3-Nonneurons']\n",
    "gene_data = get_gene_data(\n",
    "    abc_atlas_cache=abc_cache,\n",
    "    all_cells=nonneuron_cells,\n",
    "    all_genes=gene,\n",
    "    selected_genes=gene_names\n",
    ")\n",
    "gene_data[pd.notna(gene_data[gene_data.columns[0]])]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The returned DataFrame is indexed by ``cell_label`` and can thus be joined with the ``cell`` DataFrame for further analysis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cell_barcode</th>\n",
       "      <th>barcoded_cell_sample_label</th>\n",
       "      <th>library_label</th>\n",
       "      <th>feature_matrix_label</th>\n",
       "      <th>entity</th>\n",
       "      <th>brain_section_label</th>\n",
       "      <th>library_method</th>\n",
       "      <th>donor_label</th>\n",
       "      <th>donor_sex</th>\n",
       "      <th>dataset_label</th>\n",
       "      <th>...</th>\n",
       "      <th>region_of_interest_label</th>\n",
       "      <th>anatomical_division_label</th>\n",
       "      <th>abc_sample_id</th>\n",
       "      <th>PTPRC</th>\n",
       "      <th>SLC17A6</th>\n",
       "      <th>SLC32A1</th>\n",
       "      <th>SLC17A7</th>\n",
       "      <th>TTR</th>\n",
       "      <th>PLP1</th>\n",
       "      <th>AQP4</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cell_label</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>10X386_2:CATGGATTCTCGACGG</th>\n",
       "      <td>CATGGATTCTCGACGG</td>\n",
       "      <td>10X386_2</td>\n",
       "      <td>LKTX_210825_01_B01</td>\n",
       "      <td>WHB-10Xv3-Neurons</td>\n",
       "      <td>nuclei</td>\n",
       "      <td>H19.30.001.CX.51</td>\n",
       "      <td>10Xv3</td>\n",
       "      <td>H19.30.001</td>\n",
       "      <td>M</td>\n",
       "      <td>WHB-10Xv3</td>\n",
       "      <td>...</td>\n",
       "      <td>Human MoAN</td>\n",
       "      <td>Myelencephalon</td>\n",
       "      <td>b3d3a6c0-3dc5-4738-af4c-96cc5ff6033f</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10X383_5:TCTTGCGGTGAATTGA</th>\n",
       "      <td>TCTTGCGGTGAATTGA</td>\n",
       "      <td>10X383_5</td>\n",
       "      <td>LKTX_210818_02_E01</td>\n",
       "      <td>WHB-10Xv3-Neurons</td>\n",
       "      <td>nuclei</td>\n",
       "      <td>H19.30.002.BS.94</td>\n",
       "      <td>10Xv3</td>\n",
       "      <td>H19.30.002</td>\n",
       "      <td>M</td>\n",
       "      <td>WHB-10Xv3</td>\n",
       "      <td>...</td>\n",
       "      <td>Human MoSR</td>\n",
       "      <td>Myelencephalon</td>\n",
       "      <td>0a7ed99b-d5eb-4c50-b95e-eb31b448d2c7</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10X386_2:CTCATCGGTCGAGCAA</th>\n",
       "      <td>CTCATCGGTCGAGCAA</td>\n",
       "      <td>10X386_2</td>\n",
       "      <td>LKTX_210825_01_B01</td>\n",
       "      <td>WHB-10Xv3-Neurons</td>\n",
       "      <td>nuclei</td>\n",
       "      <td>H19.30.001.CX.51</td>\n",
       "      <td>10Xv3</td>\n",
       "      <td>H19.30.001</td>\n",
       "      <td>M</td>\n",
       "      <td>WHB-10Xv3</td>\n",
       "      <td>...</td>\n",
       "      <td>Human MoAN</td>\n",
       "      <td>Myelencephalon</td>\n",
       "      <td>458f6409-af8d-4c7c-ac9b-0b0d9b73b36d</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10X378_8:TTGGATGAGACAAGCC</th>\n",
       "      <td>TTGGATGAGACAAGCC</td>\n",
       "      <td>10X378_8</td>\n",
       "      <td>LKTX_210809_01_H01</td>\n",
       "      <td>WHB-10Xv3-Neurons</td>\n",
       "      <td>nuclei</td>\n",
       "      <td>H19.30.002.BS.93</td>\n",
       "      <td>10Xv3</td>\n",
       "      <td>H19.30.002</td>\n",
       "      <td>M</td>\n",
       "      <td>WHB-10Xv3</td>\n",
       "      <td>...</td>\n",
       "      <td>Human PnAN</td>\n",
       "      <td>Pons</td>\n",
       "      <td>ff6b2d95-3752-49bf-9421-f17107538868</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10X387_7:TGAACGTAGTATTCCG</th>\n",
       "      <td>TGAACGTAGTATTCCG</td>\n",
       "      <td>10X387_7</td>\n",
       "      <td>LKTX_210825_02_G01</td>\n",
       "      <td>WHB-10Xv3-Neurons</td>\n",
       "      <td>nuclei</td>\n",
       "      <td>H19.30.001.CX.51</td>\n",
       "      <td>10Xv3</td>\n",
       "      <td>H19.30.001</td>\n",
       "      <td>M</td>\n",
       "      <td>WHB-10Xv3</td>\n",
       "      <td>...</td>\n",
       "      <td>Human MoAN</td>\n",
       "      <td>Myelencephalon</td>\n",
       "      <td>589a1c5c-839b-480d-babc-4098c467211b</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10X194_8:GAAATGAGTTCGGCTG</th>\n",
       "      <td>GAAATGAGTTCGGCTG</td>\n",
       "      <td>10X194_8</td>\n",
       "      <td>LKTX_190529_02_H01</td>\n",
       "      <td>WHB-10Xv3-Nonneurons</td>\n",
       "      <td>nuclei</td>\n",
       "      <td>H18.30.002.CX.50</td>\n",
       "      <td>10Xv3</td>\n",
       "      <td>H18.30.002</td>\n",
       "      <td>M</td>\n",
       "      <td>WHB-10Xv3</td>\n",
       "      <td>...</td>\n",
       "      <td>Human SN</td>\n",
       "      <td>Midbrain</td>\n",
       "      <td>721e951b-b734-46ed-83e3-89f803f74f61</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>10.210833</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10X350_4:TTTACCATCGCACGAC</th>\n",
       "      <td>TTTACCATCGCACGAC</td>\n",
       "      <td>10X350_4</td>\n",
       "      <td>LKTX_210421_03_D01</td>\n",
       "      <td>WHB-10Xv3-Nonneurons</td>\n",
       "      <td>nuclei</td>\n",
       "      <td>H19.30.002.CB.62</td>\n",
       "      <td>10Xv3</td>\n",
       "      <td>H19.30.002</td>\n",
       "      <td>M</td>\n",
       "      <td>WHB-10Xv3</td>\n",
       "      <td>...</td>\n",
       "      <td>Human CbDN</td>\n",
       "      <td>Cerebellum</td>\n",
       "      <td>8752a645-853b-49a5-8c3a-2cf9f8557b94</td>\n",
       "      <td>9.587301</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>8.006084</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10X225_1:AGAAGCGTCCATATGG</th>\n",
       "      <td>AGAAGCGTCCATATGG</td>\n",
       "      <td>10X225_1</td>\n",
       "      <td>LKTX_190913_02_A01</td>\n",
       "      <td>WHB-10Xv3-Nonneurons</td>\n",
       "      <td>nuclei</td>\n",
       "      <td>H18.30.002.CX.51</td>\n",
       "      <td>10Xv3</td>\n",
       "      <td>H18.30.002</td>\n",
       "      <td>M</td>\n",
       "      <td>WHB-10Xv3</td>\n",
       "      <td>...</td>\n",
       "      <td>Human PAG</td>\n",
       "      <td>Midbrain</td>\n",
       "      <td>1a1738c0-1efe-4124-ae62-809cd88cb168</td>\n",
       "      <td>8.567961</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10X221_5:TTGAACGCAGCCTTCT</th>\n",
       "      <td>TTGAACGCAGCCTTCT</td>\n",
       "      <td>10X221_5</td>\n",
       "      <td>LKTX_190830_01_E01</td>\n",
       "      <td>WHB-10Xv3-Nonneurons</td>\n",
       "      <td>nuclei</td>\n",
       "      <td>H18.30.002.CX.49</td>\n",
       "      <td>10Xv3</td>\n",
       "      <td>H18.30.002</td>\n",
       "      <td>M</td>\n",
       "      <td>WHB-10Xv3</td>\n",
       "      <td>...</td>\n",
       "      <td>Human STG</td>\n",
       "      <td>Cerebral cortex</td>\n",
       "      <td>9606ce80-ab0a-4c31-a7bb-65054c0d6e01</td>\n",
       "      <td>10.119619</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10X385_3:CTACCCAGTGGCGCTT</th>\n",
       "      <td>CTACCCAGTGGCGCTT</td>\n",
       "      <td>10X385_3</td>\n",
       "      <td>LKTX_210818_04_C01</td>\n",
       "      <td>WHB-10Xv3-Nonneurons</td>\n",
       "      <td>nuclei</td>\n",
       "      <td>H19.30.002.CX.49</td>\n",
       "      <td>10Xv3</td>\n",
       "      <td>H19.30.002</td>\n",
       "      <td>M</td>\n",
       "      <td>WHB-10Xv3</td>\n",
       "      <td>...</td>\n",
       "      <td>Human GPi</td>\n",
       "      <td>Basal nuclei</td>\n",
       "      <td>a69fef25-9149-41e6-8056-030e283434ea</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3369219 rows × 23 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                               cell_barcode barcoded_cell_sample_label  \\\n",
       "cell_label                                                               \n",
       "10X386_2:CATGGATTCTCGACGG  CATGGATTCTCGACGG                   10X386_2   \n",
       "10X383_5:TCTTGCGGTGAATTGA  TCTTGCGGTGAATTGA                   10X383_5   \n",
       "10X386_2:CTCATCGGTCGAGCAA  CTCATCGGTCGAGCAA                   10X386_2   \n",
       "10X378_8:TTGGATGAGACAAGCC  TTGGATGAGACAAGCC                   10X378_8   \n",
       "10X387_7:TGAACGTAGTATTCCG  TGAACGTAGTATTCCG                   10X387_7   \n",
       "...                                     ...                        ...   \n",
       "10X194_8:GAAATGAGTTCGGCTG  GAAATGAGTTCGGCTG                   10X194_8   \n",
       "10X350_4:TTTACCATCGCACGAC  TTTACCATCGCACGAC                   10X350_4   \n",
       "10X225_1:AGAAGCGTCCATATGG  AGAAGCGTCCATATGG                   10X225_1   \n",
       "10X221_5:TTGAACGCAGCCTTCT  TTGAACGCAGCCTTCT                   10X221_5   \n",
       "10X385_3:CTACCCAGTGGCGCTT  CTACCCAGTGGCGCTT                   10X385_3   \n",
       "\n",
       "                                library_label  feature_matrix_label  entity  \\\n",
       "cell_label                                                                    \n",
       "10X386_2:CATGGATTCTCGACGG  LKTX_210825_01_B01     WHB-10Xv3-Neurons  nuclei   \n",
       "10X383_5:TCTTGCGGTGAATTGA  LKTX_210818_02_E01     WHB-10Xv3-Neurons  nuclei   \n",
       "10X386_2:CTCATCGGTCGAGCAA  LKTX_210825_01_B01     WHB-10Xv3-Neurons  nuclei   \n",
       "10X378_8:TTGGATGAGACAAGCC  LKTX_210809_01_H01     WHB-10Xv3-Neurons  nuclei   \n",
       "10X387_7:TGAACGTAGTATTCCG  LKTX_210825_02_G01     WHB-10Xv3-Neurons  nuclei   \n",
       "...                                       ...                   ...     ...   \n",
       "10X194_8:GAAATGAGTTCGGCTG  LKTX_190529_02_H01  WHB-10Xv3-Nonneurons  nuclei   \n",
       "10X350_4:TTTACCATCGCACGAC  LKTX_210421_03_D01  WHB-10Xv3-Nonneurons  nuclei   \n",
       "10X225_1:AGAAGCGTCCATATGG  LKTX_190913_02_A01  WHB-10Xv3-Nonneurons  nuclei   \n",
       "10X221_5:TTGAACGCAGCCTTCT  LKTX_190830_01_E01  WHB-10Xv3-Nonneurons  nuclei   \n",
       "10X385_3:CTACCCAGTGGCGCTT  LKTX_210818_04_C01  WHB-10Xv3-Nonneurons  nuclei   \n",
       "\n",
       "                          brain_section_label library_method donor_label  \\\n",
       "cell_label                                                                 \n",
       "10X386_2:CATGGATTCTCGACGG    H19.30.001.CX.51          10Xv3  H19.30.001   \n",
       "10X383_5:TCTTGCGGTGAATTGA    H19.30.002.BS.94          10Xv3  H19.30.002   \n",
       "10X386_2:CTCATCGGTCGAGCAA    H19.30.001.CX.51          10Xv3  H19.30.001   \n",
       "10X378_8:TTGGATGAGACAAGCC    H19.30.002.BS.93          10Xv3  H19.30.002   \n",
       "10X387_7:TGAACGTAGTATTCCG    H19.30.001.CX.51          10Xv3  H19.30.001   \n",
       "...                                       ...            ...         ...   \n",
       "10X194_8:GAAATGAGTTCGGCTG    H18.30.002.CX.50          10Xv3  H18.30.002   \n",
       "10X350_4:TTTACCATCGCACGAC    H19.30.002.CB.62          10Xv3  H19.30.002   \n",
       "10X225_1:AGAAGCGTCCATATGG    H18.30.002.CX.51          10Xv3  H18.30.002   \n",
       "10X221_5:TTGAACGCAGCCTTCT    H18.30.002.CX.49          10Xv3  H18.30.002   \n",
       "10X385_3:CTACCCAGTGGCGCTT    H19.30.002.CX.49          10Xv3  H19.30.002   \n",
       "\n",
       "                          donor_sex dataset_label  ...  \\\n",
       "cell_label                                         ...   \n",
       "10X386_2:CATGGATTCTCGACGG         M     WHB-10Xv3  ...   \n",
       "10X383_5:TCTTGCGGTGAATTGA         M     WHB-10Xv3  ...   \n",
       "10X386_2:CTCATCGGTCGAGCAA         M     WHB-10Xv3  ...   \n",
       "10X378_8:TTGGATGAGACAAGCC         M     WHB-10Xv3  ...   \n",
       "10X387_7:TGAACGTAGTATTCCG         M     WHB-10Xv3  ...   \n",
       "...                             ...           ...  ...   \n",
       "10X194_8:GAAATGAGTTCGGCTG         M     WHB-10Xv3  ...   \n",
       "10X350_4:TTTACCATCGCACGAC         M     WHB-10Xv3  ...   \n",
       "10X225_1:AGAAGCGTCCATATGG         M     WHB-10Xv3  ...   \n",
       "10X221_5:TTGAACGCAGCCTTCT         M     WHB-10Xv3  ...   \n",
       "10X385_3:CTACCCAGTGGCGCTT         M     WHB-10Xv3  ...   \n",
       "\n",
       "                           region_of_interest_label  \\\n",
       "cell_label                                            \n",
       "10X386_2:CATGGATTCTCGACGG                Human MoAN   \n",
       "10X383_5:TCTTGCGGTGAATTGA                Human MoSR   \n",
       "10X386_2:CTCATCGGTCGAGCAA                Human MoAN   \n",
       "10X378_8:TTGGATGAGACAAGCC                Human PnAN   \n",
       "10X387_7:TGAACGTAGTATTCCG                Human MoAN   \n",
       "...                                             ...   \n",
       "10X194_8:GAAATGAGTTCGGCTG                  Human SN   \n",
       "10X350_4:TTTACCATCGCACGAC                Human CbDN   \n",
       "10X225_1:AGAAGCGTCCATATGG                 Human PAG   \n",
       "10X221_5:TTGAACGCAGCCTTCT                 Human STG   \n",
       "10X385_3:CTACCCAGTGGCGCTT                 Human GPi   \n",
       "\n",
       "                           anatomical_division_label  \\\n",
       "cell_label                                             \n",
       "10X386_2:CATGGATTCTCGACGG             Myelencephalon   \n",
       "10X383_5:TCTTGCGGTGAATTGA             Myelencephalon   \n",
       "10X386_2:CTCATCGGTCGAGCAA             Myelencephalon   \n",
       "10X378_8:TTGGATGAGACAAGCC                       Pons   \n",
       "10X387_7:TGAACGTAGTATTCCG             Myelencephalon   \n",
       "...                                              ...   \n",
       "10X194_8:GAAATGAGTTCGGCTG                   Midbrain   \n",
       "10X350_4:TTTACCATCGCACGAC                 Cerebellum   \n",
       "10X225_1:AGAAGCGTCCATATGG                   Midbrain   \n",
       "10X221_5:TTGAACGCAGCCTTCT            Cerebral cortex   \n",
       "10X385_3:CTACCCAGTGGCGCTT               Basal nuclei   \n",
       "\n",
       "                                                  abc_sample_id      PTPRC  \\\n",
       "cell_label                                                                   \n",
       "10X386_2:CATGGATTCTCGACGG  b3d3a6c0-3dc5-4738-af4c-96cc5ff6033f        NaN   \n",
       "10X383_5:TCTTGCGGTGAATTGA  0a7ed99b-d5eb-4c50-b95e-eb31b448d2c7        NaN   \n",
       "10X386_2:CTCATCGGTCGAGCAA  458f6409-af8d-4c7c-ac9b-0b0d9b73b36d        NaN   \n",
       "10X378_8:TTGGATGAGACAAGCC  ff6b2d95-3752-49bf-9421-f17107538868        NaN   \n",
       "10X387_7:TGAACGTAGTATTCCG  589a1c5c-839b-480d-babc-4098c467211b        NaN   \n",
       "...                                                         ...        ...   \n",
       "10X194_8:GAAATGAGTTCGGCTG  721e951b-b734-46ed-83e3-89f803f74f61        0.0   \n",
       "10X350_4:TTTACCATCGCACGAC  8752a645-853b-49a5-8c3a-2cf9f8557b94   9.587301   \n",
       "10X225_1:AGAAGCGTCCATATGG  1a1738c0-1efe-4124-ae62-809cd88cb168   8.567961   \n",
       "10X221_5:TTGAACGCAGCCTTCT  9606ce80-ab0a-4c31-a7bb-65054c0d6e01  10.119619   \n",
       "10X385_3:CTACCCAGTGGCGCTT  a69fef25-9149-41e6-8056-030e283434ea        0.0   \n",
       "\n",
       "                          SLC17A6 SLC32A1 SLC17A7  TTR       PLP1 AQP4  \n",
       "cell_label                                                              \n",
       "10X386_2:CATGGATTCTCGACGG     NaN     NaN     NaN  NaN        NaN  NaN  \n",
       "10X383_5:TCTTGCGGTGAATTGA     NaN     NaN     NaN  NaN        NaN  NaN  \n",
       "10X386_2:CTCATCGGTCGAGCAA     NaN     NaN     NaN  NaN        NaN  NaN  \n",
       "10X378_8:TTGGATGAGACAAGCC     NaN     NaN     NaN  NaN        NaN  NaN  \n",
       "10X387_7:TGAACGTAGTATTCCG     NaN     NaN     NaN  NaN        NaN  NaN  \n",
       "...                           ...     ...     ...  ...        ...  ...  \n",
       "10X194_8:GAAATGAGTTCGGCTG     0.0     0.0     0.0  0.0  10.210833  0.0  \n",
       "10X350_4:TTTACCATCGCACGAC     0.0     0.0     0.0  0.0   8.006084  0.0  \n",
       "10X225_1:AGAAGCGTCCATATGG     0.0     0.0     0.0  0.0        0.0  0.0  \n",
       "10X221_5:TTGAACGCAGCCTTCT     0.0     0.0     0.0  0.0        0.0  0.0  \n",
       "10X385_3:CTACCCAGTGGCGCTT     0.0     0.0     0.0  0.0        0.0  0.0  \n",
       "\n",
       "[3369219 rows x 23 columns]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cells_with_genes = cell.join(gene_data)\n",
    "cells_with_genes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}