processmining2.Rmd

---
title: "Process Mining using the R bupaR package"
author: "Vic Mertens"
date: "5-5-2020"
output: html_document
---

```{r setup, include=FALSE, eval=TRUE}


version_date <- lubridate::ymd("2020-03-11")

knitr::opts_chunk$set(echo = TRUE, cache=FALSE, eval=TRUE,
                      tidy.opts=list(width.cutoff=60),
                      tidy=FALSE)
```

## Process Mining

Process mining is the art of discovering, monitoring, and improving processes by extracting knowledge from event logs available in information systems.
Process mining results in knowledge about real processes followed in an organization that might be helpfull when resolving process bottlenecks or when checking if processes are compliant.

bupaR is an R package that allows to perform process mining activities in R.
At the end of this page you can find more details about bupaR.

An example of process mining is shown below.   
We start loading the necesssary libraries into R and by reading the log file (which is in an excel format for this example):

```{r, echo=TRUE, eval=TRUE, message=FALSE, warning=FALSE}
library(tidyverse)
library(bupaR)
library(openxlsx)
library(processanimateR)
library(heuristicsmineR)
library(petrinetR)
library(lubridate)
library(tictoc)
library(knitr)
library(pheatmap)
library(maditr)
library(viridis)
library(forcats)

# start timer
tic("Time to complete")

# setup section:
filename<-"Production_Data.xlsx"

# Read in a fresh file?
read_file<-TRUE

# filter setup


# Read file if required
if (read_file) {
eventlog<-read.xlsx(filename,sheet=1,detectDates=TRUE,colNames=TRUE)

eventlog$Start<-as.POSIXct(eventlog$Start, tz = "", format="%Y/%m/%d %H:%M:%OS" ,optional = FALSE)
eventlog$Complete<-as.POSIXct(eventlog$Complete, tz = "", format="%Y/%m/%d %H:%M:%OS" ,optional = FALSE)
}

# duplicate the records, one for start, one for completion, as bupaR needs that format
# copy the whole log
eventlog2<-eventlog

# the original records get status "start" and the corresponding timestamp
eventlog$Status<-"start"
eventlog$TimeStamp<-eventlog$Start
# the copied records get status "complete" and the corresponding timestamp
eventlog2$Status<-"complete"
eventlog2$TimeStamp<-eventlog2$Complete
# both get their row number as instance IDs
eventlog<-eventlog %>%  mutate(activity_instance = 1:nrow(.))
eventlog2<-eventlog2 %>%  mutate(activity_instance = 1:nrow(.))
# both data frames are now joined 
eventlog<-rbind(eventlog,eventlog2)


eventlog$dow<-weekdays(as.Date(eventlog$Complete))
eventlog<-eventlog[!is.na(eventlog$CaseID),]

eventlog$dow <- ordered(eventlog$dow, levels=c("maandag", "dinsdag", "woensdag", "donderdag",
 "vrijdag", "zaterdag", "zondag"))


eventlog <- eventlog %>%
  mutate(dow = fct_recode(dow,
    "Monday" = "maandag",
    "Tuesday" = "dinsdag",
    "Wednesday" = "woensdag",
    "Thursday" = "donderdag",
    "Friday" = "vrijdag",
    "Saturday" = "zaterdag",
    "Sunday" = "zondag"
  ))

```

### The log file read in looks as follows (first 10 lines shown):  

```{r, echo=TRUE, eval=TRUE, message=FALSE, warning=FALSE}

kable(eventlog[1:10,])

print(paste("Number of records read in from the log file: ",nrow(eventlog),sep=""))

```

### The log file is converted to an eventlog object:  

```{r, echo=TRUE, eval=TRUE, message=FALSE, warning=FALSE}


# convert data frame into an eventlog object
# activity_instance is set equal to the row number

evLog<-eventlog %>%
  eventlog(
    case_id = "CaseID",
    activity_id = "Activity",
    lifecycle_id = "Status",
    activity_instance_id = "activity_instance",
    timestamp = "TimeStamp",
    resource_id = "Resource"
  )
evLog<-evLog[order(evLog$TimeStamp),]

```

### For this example we select only records with a date in 2014 and we keep only complete cycles (from start to end):

```{r, echo=TRUE, eval=TRUE, message=FALSE, warning=FALSE}

# # filter on begin and end activities
# evLog <- evLog %>%
#   filter_endpoints(start_activities = c("Create REQ","Create REQ Line"), end_activities = c("Create PO","Create PO Line"))
# 
# print(paste("Number of records remaining after filtering: ",nrow(evLog),sep=""))

```


### Some statistics about our event log extract:

- mapping used from log file to event log:

```{r, echo=TRUE, eval=TRUE, message=FALSE, warning=FALSE}
# overview of the mapping used
mapp<-evLog %>% mapping()
kable(as.data.frame(mapp))

```

- activities found in the event log:

```{r, echo=TRUE, eval=TRUE, message=FALSE, warning=FALSE}
# overview of the different activities
mapp<-evLog %>% activity_labels()
kable(as.data.frame(mapp))

```

- frequency of the activities as found in the event log:

```{r, echo=TRUE, eval=TRUE, message=FALSE, warning=FALSE}
# overview of the mapping used
mapp<-evLog %>% activities()
kable(as.data.frame(mapp))

```


- Resource idle time

```{r, echo=TRUE, eval=TRUE, message=FALSE, warning=FALSE}
mapp<-evLog %>%
    idle_time("resource", units = "days")

kable(as.data.frame(mapp))
mapp %>% plot()


```

- Processing time

```{r, echo=TRUE, eval=TRUE, message=FALSE, warning=FALSE}
mapp<-evLog %>%
 processing_time("activity",units="days")
 
kable(as.data.frame(mapp))


mappi<-mapp %>% plot()
png(file="processing_time.png",width=1500, height=2000)
mappi
dev.off()
include_graphics("processing_time.png")

```

- Throughput time

```{r, echo=TRUE, eval=TRUE, message=FALSE, warning=FALSE}
mapp<-evLog %>%
 throughput_time("log")

kable(as.data.frame(mapp))
mapp %>% plot()

```

    

- activity duration:

Note: In below table the durations are all 0 because the log file used does only contain timestamps for the completion of the activities. If both start and end of activities are logged the durations can be calculated.
That information can also be used to calculate resource utilisation and resource idle times.


```{r, echo=TRUE, eval=TRUE, message=FALSE, warning=FALSE}
# overview of the mapping used
mapp<-evLog %>% processing_time("activity",units="mins")
kable(as.data.frame(mapp))

```


- Resource activity

The below table is indicating how many of the activities are carried out by each resource.

```{r, echo=TRUE, eval=TRUE, message=FALSE, warning=FALSE}
mapp<-evLog %>% resource_frequency("resource") 
kable(as.data.frame(mapp))
```



- Heatmap of activities:  

using a heatmap we can check if certain activities are executed more on a specific day in the week:

```{r, echo=TRUE, eval=TRUE, message=FALSE, warning=FALSE}

week_activity <- eventlog %>% 
  group_by(Activity, dow) %>%
  summarise(Freq = n())
week_activity<-week_activity[complete.cases(week_activity),]

p <- ggplot(week_activity, aes(dow, Activity)) + geom_tile(aes(fill = log(Freq)), colour = "white") +
  labs(x="Weekday",y="Activity") +
  scale_fill_viridis()

png(file="heatmap.png",width=1500,height=750)
p
dev.off()
include_graphics("heatmap.png")


```


### Dependency Matrix:

In this example we are using the heuristics miner. Other miners available for bupaR are the alpha miner and the inductive miner.
(threshold is set to 0.5, covering half of the activities in the log)

```{r, echo=TRUE, eval=TRUE, message=FALSE, warning=FALSE}
# dependency matrix with threshold
dependency_matrix(evLog, threshold = .5) %>% render_dependency_matrix()
```

### Causal net:

```{r, echo=TRUE, eval=TRUE, message=FALSE, warning=FALSE}
# causal net with threshold
causal_net(evLog, threshold = .5) %>% render_causal_net()
```

### Precedence Matrix:  

Preceeding activities are found in the rows. The numbers are the number of times an activity in the top row is preceeded by an activity in the first column.


```{r, echo=TRUE, eval=TRUE, message=FALSE, warning=FALSE}
# Efficient precedence matrix
m <- precedence_matrix_absolute(evLog)
kable(as.matrix(m))
```

### Process maps:  

(in below maps 90% of the activities are covered, this percentage can be set to avoid showing lesss frequent cases)  

- Relative

```{r, echo=TRUE, eval=TRUE, message=FALSE, warning=FALSE}
evLog <- evLog %>% filter_activity_frequency(percentage=0.5)

evLog %>%
  process_map(type = frequency("relative"), threshold=0.5)

```

- Absolute

```{r, echo=TRUE, eval=TRUE, message=FALSE, warning=FALSE}

evLog %>%
  process_map(type = frequency("absolute"), threshold=0.5)

```

- Relative - case

```{r, echo=TRUE, eval=TRUE, message=FALSE, warning=FALSE}
evLog %>%
  process_map(type = frequency("relative_case", color_scale = "Purples"), threshold=0.5)

```

- Performance (median, days)

```{r, echo=TRUE, eval=TRUE, message=FALSE, warning=FALSE}
evLog %>%
  process_map(performance(median, "days"), threshold=0.5)

```

- Performance (mean, hours)

```{r, echo=TRUE, eval=TRUE, message=FALSE, warning=FALSE}
evLog %>%
  process_map(performance(mean, "hours"), threshold=0.5)

```

- Performance (edges) and relative frequency (nodes)

```{r, echo=TRUE, eval=TRUE, message=FALSE, warning=FALSE}
evLog %>%
  process_map(type_nodes = frequency("relative_case"),
              type_edges = performance(mean), threshold=0.5)
```

- Trace Explorer  

In below diagram the coverage was set to 50% of all cases


```{r, echo=TRUE, eval=TRUE, message=FALSE, warning=FALSE}

evLog2<-evLog

evLog2 %>%
  trace_explorer(coverage=0.5) -> traces
png(file="traces.png",width=1000)
traces
dev.off()
include_graphics("traces.png")
```

- Filtering on trace frequency

```{r, echo=TRUE, eval=TRUE, message=FALSE, warning=FALSE}

evLog<-evLog %>%
  # filter_activity(c("LacticAcid", "CRP", "Leucocytes", "Return ER", "IV Liquid", "IV Antibiotics"), reverse = T) %>%
  filter_trace_frequency(percentage = 0.5)

```

- Dotted Chart  

The following chart only displays all cases fully contained in a 2 weeks period.
It is possible to show cases starting in that one month period, all cases completed in that time interval, or all cases showing some activity iin an4 time interval.


```{r, echo=TRUE, eval=TRUE, message=FALSE, warning=FALSE}

evLog %>%  
 # filter_time_period(interval = ymd(c("20140107", "20140121")),filter_method = "contained") %>%
  dotted_chart

```

- Animated Log  

Using an animated log the cases in the log can be re-played on the process map. The example illustrates this can be very useful when looking for bottlenecks in the process. 
The process looks simpler in below model as we are only using 90%% of the log (based on relative occurrence in the log).


```{r, echo=TRUE, eval=TRUE, message=FALSE, warning=FALSE}

# animate log on process map
animate_process(evLog %>% filter_trace_frequency(percentage=0.50),
               mode = "relative",
               legend = "CaseId",
               mapping = token_aes(color = token_scale("amount",
                                                       scale = "linear",
                                                       range = c("yellow","red"))))
```

Stopping the timer:

```{r, echo=TRUE, eval=TRUE, message=FALSE, warning=FALSE}

# stop timer
toc()

```


### Credits:

```{r, echo=TRUE, eval=TRUE, message=FALSE, warning=FALSE}
citation("processmapR")
```