reordering_injects_randomness.Rmd

---
title: "Tabula Muris: single organ analysis"
output: html_notebook
---

Load the requisite packages and some additional helper functions.

```{r}
library(Seurat)
library(dplyr)
library(Matrix)
library(stringr)
library(tidyverse)
library(here)
```

Load the count data for one organ and add it to the Seurat object.

```{r}
organ = "Bladder"
```

```{r}
raw.data <- read.csv(here("00_data_ingest","facs_raw_data", "FACS",paste0(organ,"-counts.csv")),row.names = 1)
meta.data <- read.csv(here("00_data_ingest","facs_raw_data", "metadata_FACS.csv"))
```



```{r}
plates <- str_split(colnames(raw.data),"[.]", simplify = TRUE)[,2]

rownames(meta.data) <- meta.data$plate.barcode
cell.meta.data <- meta.data[plates,]
rownames(cell.meta.data) <- colnames(raw.data)
```

```{r}
# Find ERCC's, compute the percent ERCC, and drop them from the raw data.
erccs <- grep(pattern = "^ERCC-", x = rownames(x = raw.data), value = TRUE)
percent.ercc <- Matrix::colSums(raw.data[erccs, ])/Matrix::colSums(raw.data)
ercc.index <- grep(pattern = "^ERCC-", x = rownames(x = raw.data), value = FALSE)
raw.data <- raw.data[-ercc.index,]
```

```{r}
# Create the Seurat object with all the data
tiss <- CreateSeuratObject(raw.data = raw.data, min.cells = 5, min.genes = 5)

tiss <- AddMetaData(object = tiss, cell.meta.data)
tiss <- AddMetaData(object = tiss, percent.ercc, col.name = "percent.ercc")
# Change default name for sums of counts from nUMI to nReads
colnames(tiss@meta.data)[colnames(tiss@meta.data) == 'nUMI'] <- 'nReads'
```

```{r}
ribo.genes <- grep(pattern = "^Rp[sl][[:digit:]]", x = rownames(x = tiss@data), value = TRUE)
percent.ribo <- Matrix::colSums(tiss@raw.data[ribo.genes, ])/Matrix::colSums(tiss@raw.data)
tiss <- AddMetaData(object = tiss, metadata = percent.ribo, col.name = "percent.ribo")
```

```{r}
percent.Rn45s <- Matrix::colSums(tiss@raw.data[c('Rn45s'), ])/Matrix::colSums(tiss@raw.data)
tiss <- AddMetaData(object = tiss, metadata = percent.Rn45s, col.name = "percent.Rn45s")
```

A sanity check: genes per cell vs reads per cell.

```{r}
GenePlot(object = tiss, gene1 = "nReads", gene2 = "nGene", use.raw=T)
```

Filter out cells with few reads and few genes.

```{r}
tiss <- FilterCells(object = tiss, subset.names = c("nGene", "nReads"), 
    low.thresholds = c(500, 50000), high.thresholds = c(25000, 2000000))
```


Normalize the data, then regress out correlation with total reads
```{r}
tiss <- NormalizeData(object = tiss)
tiss <- ScaleData(object = tiss, vars.to.regress = c("nReads", "percent.ribo","Rn45s"))
tiss <- FindVariableGenes(object = tiss, do.plot = TRUE, x.high.cutoff = Inf, y.cutoff = 0.5)
```


Run Principal Component Analysis.
```{r}
tiss <- RunPCA(object = tiss, do.print = FALSE)
tiss <- ProjectPCA(object = tiss, do.print = FALSE)
```

```{r, echo=FALSE, fig.height=4, fig.width=8}
PCHeatmap(object = tiss, pc.use = 1:3, cells.use = 500, do.balanced = TRUE, label.columns = FALSE, num.genes = 8)
```

Later on (in FindClusters and TSNE) you will pick a number of principal components to use. This has the effect of keeping the major directions of variation in the data and, ideally, supressing noise. There is no correct answer to the number to use, but a decent rule of thumb is to go until the plot plateaus.

```{r}
PCElbowPlot(object = tiss)
```

Choose the number of principal components to use.
```{r}
# Set number of principal components. 
n.pcs = 10
```


The clustering is performed based on a nearest neighbors graph. Cells that have similar expression will be joined together. The Louvain algorithm looks for groups of cells with high modularity--more connections within the group than between groups. The resolution parameter determines the scale...higher resolution will give more clusters, lower resolution will give fewer.

For the top-level clustering, aim to under-cluster instead of over-cluster. It will be easy to subset groups and further analyze them below.

```{r}
# Set resolution 
res.used <- 0.5

tiss <- FindClusters(object = tiss, reduction.type = "pca", dims.use = 1:n.pcs, 
    resolution = res.used, print.output = 0, save.SNN = TRUE)
```

To visualize 
```{r}
# If cells are too spread out, you can raise the perplexity. If you have few cells, try a lower perplexity (but never less than 10).
tiss <- RunTSNE(object = tiss, dims.use = 1:n.pcs, seed.use = 10, perplexity=30)
```

```{r}
# note that you can set do.label=T to help label individual clusters
TSNEPlot(object = tiss, do.label = T)
```

Check expression of genes of interset.

```{r, echo=FALSE, fig.height=12, fig.width=8}
genes_to_check = c('Ins1', 'Gcg', 'Ppy', 'Sst', 'Chga', 'Krt19', 'Pecam1', 'Pdgfra', 'Ptprc', 'Ghrl')
#genes_to_check = c('Alb', 'Cyp2f2', 'Cyp2e1', 'Hamp')

FeaturePlot(tiss, genes_to_check, pt.size = 1, nCol = 3)
```





# Shuffle and do it again

```{r}
shuffle = colnames(raw.data)[shuffled_cells]
View(raw.data[,shuffle])
```

```{r}
dense = Matrix(as.matrix(raw.data), sparse = TRUE)
```

```{r}
dense[,shuffle]
```


```{r}
set.seed(1)
shuffled_cells = sample(ncol(raw.data))
raw.data <- raw.data[,shuffled_cells]
```

```{r}
plates <- str_split(colnames(raw.data),"[.]", simplify = TRUE)[,2]

rownames(meta.data) <- meta.data$plate.barcode
cell.meta.data <- meta.data[plates,]
rownames(cell.meta.data) <- colnames(raw.data)
```

```{r}
# Find ERCC's, compute the percent ERCC, and drop them from the raw data.
erccs <- grep(pattern = "^ERCC-", x = rownames(x = raw.data), value = TRUE)
percent.ercc <- Matrix::colSums(raw.data[erccs, ])/Matrix::colSums(raw.data)
ercc.index <- grep(pattern = "^ERCC-", x = rownames(x = raw.data), value = FALSE)
raw.data <- raw.data[-ercc.index,]
```

```{r}
# Create the Seurat object with all the data
siss <- CreateSeuratObject(raw.data = raw.data, min.cells = 5, min.genes = 5)

siss <- AddMetaData(object = siss, cell.meta.data)
siss <- AddMetaData(object = siss, percent.ercc, col.name = "percent.ercc")
# Change default name for sums of counts from nUMI to nReads
colnames(siss@meta.data)[colnames(siss@meta.data) == 'nUMI'] <- 'nReads'
```

```{r}
ribo.genes <- grep(pattern = "^Rp[sl][[:digit:]]", x = rownames(x = siss@data), value = TRUE)
percent.ribo <- Matrix::colSums(siss@raw.data[ribo.genes, ])/Matrix::colSums(siss@raw.data)
siss <- AddMetaData(object = siss, metadata = percent.ribo, col.name = "percent.ribo")
```

```{r}
percent.Rn45s <- Matrix::colSums(siss@raw.data[c('Rn45s'), ])/Matrix::colSums(siss@raw.data)
siss <- AddMetaData(object = siss, metadata = percent.Rn45s, col.name = "percent.Rn45s")
```

A sanity check: genes per cell vs reads per cell.

```{r}
GenePlot(object = siss, gene1 = "nReads", gene2 = "nGene", use.raw=T)
```

Filter out cells with few reads and few genes.

```{r}
siss <- FilterCells(object = siss, subset.names = c("nGene", "nReads"), 
    low.thresholds = c(500, 50000), high.thresholds = c(25000, 2000000))
```


Normalize the data, then regress out correlation with total reads
```{r}
siss <- NormalizeData(object = siss)
siss <- ScaleData(object = siss, vars.to.regress = c("nReads", "percent.ribo","Rn45s"))
siss <- FindVariableGenes(object = siss, do.plot = TRUE, x.high.cutoff = Inf, y.cutoff = 0.5)
```


Run Principal Component Analysis.
```{r}
siss <- RunPCA(object = siss, do.print = FALSE)
siss <- ProjectPCA(object = siss, do.print = FALSE)
```

```{r, echo=FALSE, fig.height=4, fig.width=8}
PCHeatmap(object = siss, pc.use = 1:3, cells.use = 500, do.balanced = TRUE, label.columns = FALSE, num.genes = 8)
```

Later on (in FindClusters and TSNE) you will pick a number of principal components to use. This has the effect of keeping the major directions of variation in the data and, ideally, supressing noise. There is no correct answer to the number to use, but a decent rule of thumb is to go until the plot plateaus.

```{r}
PCElbowPlot(object = siss)
```

Choose the number of principal components to use.
```{r}
# Set number of principal components. 
n.pcs = 10
```


The clustering is performed based on a nearest neighbors graph. Cells that have similar expression will be joined together. The Louvain algorithm looks for groups of cells with high modularity--more connections within the group than between groups. The resolution parameter determines the scale...higher resolution will give more clusters, lower resolution will give fewer.

For the top-level clustering, aim to under-cluster instead of over-cluster. It will be easy to subset groups and further analyze them below.

```{r}
# Set resolution 
res.used <- 0.5

siss <- FindClusters(object = siss, reduction.type = "pca", dims.use = 1:n.pcs, 
    resolution = res.used, print.output = 0, save.SNN = TRUE)
```

To visualize 
```{r}
# If cells are too spread out, you can raise the perplexity. If you have few cells, try a lower perplexity (but never less than 10).
siss <- RunTSNE(object = siss, dims.use = 1:n.pcs, seed.use = 10, perplexity=30)
```

```{r}
# note that you can set do.label=T to help label individual clusters
TSNEPlot(object = siss, do.label = T)
```

# Compare the shuffled and unshuffled versions

```{r}
all(colnames(tiss@raw.data)[shuffled_cells] == colnames(siss@raw.data))
```

```{r}

df <- as_tibble(tiss@ident)
df['cell'] <- names(tiss@ident)
df <- rename(df, cluster = value)

sf <- as_tibble(siss@ident)
sf <- rename(sf, scluster = value)
sf['cell'] <- names(siss@ident)

af <- left_join(df, sf, on='cell')

af %>% group_by(cluster, scluster) %>% summarize(count = n()) %>% spread(key = scluster, value = count)
```