efi_patientsafety.Rmd

---
title: "The deficit-accumulation electronic frailty index can predict patient 
        safety events in adult patients of all ages using only structured data in 
        the electronic health record."
author: 'Alex Bokov ^1,✉^, Sara Espinoza ^1^, Chandana Tripathy ^1^ , and Kathleen R. Stevens ^1^'
css: "production.css"
abstract: 'We have found that the electronic frailty index (EFI), a risk
  score developed using the Rockwood deficit-accumulation framework, is
  a strong predictor of patient safety events without relying on any predictors
  other than diagnoses, vital signs, and laboratory results from the electronic
  health record. Previous studies using electronic frailty indexes on claims 
  data or electronic health records focused on older patients, but we found that
  for most of the patient safety outcomes we examined EFI is also predictive 
  for patients in the 45-65 age range and even in the 18-45 age range.'
documentclass: article
description: 'Manuscript'
clean: false
bibliography: efi_paper.bib
csl: harvard-cite-them-right.csl
output:
  bookdown::word_document2:
    reference_docx: efi_template.docx
    fig_caption: true
    self_contained: true
    keep_md: true
    number_sections: false
  bookdown::html_document2:
    fig_caption: true
    self_contained: true
    number_sections: false
    keep_md: true
    clean: false
---

```{r load_deps, echo=FALSE, message=FALSE, warning=FALSE,results='hide'}
.projpackages <- c('GGally','pander','dplyr','ggplot2','data.table'
                   ,'survival','broom','forcats','table1','english','bookdown');
.deps <- c( 'analysis.R' );
.debug <- 0;
message(paste0('\n\n working dir: ',getwd(),'\n\n'))
cat('\n\nAvailable Files:',list.files(),sep='\n','\n')
.junk<-capture.output(source(file.path('./scripts/global.R'),chdir=TRUE,echo=!1
                             ,local=TRUE));
# Set some formatting options for this document
panderOptions('table.alignment.default','right');
panderOptions('table.alignment.rownames','right');
panderOptions('table.split.table',Inf);
panderOptions('p.wrap','');
panderOptions('p.copula',', and ');
panderOptions('graph.fontfamily','serif');
# theme_get() gives default theme
theme_set(theme_bw(base_family = 'serif',base_size=14) +
            theme(strip.background = element_rect(fill=NA,color=NA)
                  ,strip.text = element_text(size=15)));
knitr::opts_chunk$set(echo=.debug>0, warning=.debug>0, message=.debug>0);

# detect current output format
.outfmt <- knitr::opts_knit$get('rmarkdown.pandoc.to');
message('.outfmt = ',.outfmt);
if(is.null(.outfmt)){
  .outfmt <- if(knitr::is_html_output()) 'html' else {
    if(knitr::is_latex_output()) 'latex' else {
      if(interactive()) 'html' else {
        'unknown';
    }}}};
message('.outfmt = ',.outfmt);

.currentscript <- current_scriptname('efi_patientsafety.Rmd');
.outcomenames <- c('vi_c_psi','vi_c_hospinfect','vi_c_hosptrauma'
                   ,'vi_c_cardiac');
snip_outcomes <- rname(.outcomenames);

# limit the data to items of interest
tb2 <- subset(tb2,outcome %in% .outcomenames & predictor %in%
                c('Frailty','Frailty, age:18-45','Frailty, age:45-65'
                  ,'Frailty, age:65+'));
tb3 <- subset(tb3,Predictor %in% c('Frailty','Frailty, age:18-45'
                                   ,'Frailty, age:45-65','Frailty, age:65+') &
                Outcome %in% .outcomenames);
fits <- fits[.outcomenames];
# snippets ----
snip_resultsfold <- unique(resultsfold00(tb2));
snip_dat <- c(pat=length(unique(dat03$patient_num)),vis=nrow(dat03)) %>% 
  format(big.mark=',');
```


# Introduction/Background

Frailty is the lifelong erosion of stress resistance and accumulation of
impairments across multiple physiological systems. Among older community
dwelling adults 32% have been classified as pre-frail and 24% have been
classified as frail [@hoover13]. Frailty predicts disability, injurious falls,
and mortality [@pajewski19], emergency room visits and hospitalizations
[@fried01a], and long-term care admissions [@pajewski19; @rockwood06;
@rockwood05]. The Fried phenotype [@fried01a] and Rockwood deficit accumulation
index [@rockwood07; @mitnitski01a], are two commonly used measures of frailty. 
There is reasonable convergence between these two approaches 
[@li15; @malmstrom14] and Kulminsky @kulminski08 found evidence for the index 
outperforming the phenotype but without requiring patient questionnaires nor 
physical assessments. Recently Johnston et al. @johnston20 reported that frailty 
can improve the accuracy of predicted annualized Medicare costs over Centers for 
Medicare & Medicaid Services Hierarchical Condition Categories (CMS-HCC) alone. 
To be pro-active, health systems need to be able to replicate such analysis on
their own electronic medical record (EMR) data. Several groups have adapted the
Rockwood index to EMR-derived data [@clegg16a; @pajewski19; @stow18]. So far
`r note('mainly'
,'\\@bokov\\
\\
TODO: see if the literature supports a stronger statement than \'mainly\''
,date="2020-10-16T08:30:43Z")`
patients older than 65 have been studied yet value-based care is
broader than Medicare. As private payers start moving to the ACO model it is
important to understand whether EFI generalizes to outcomes such as preventable
safety events in other age groups.

Effective care of frail older persons is a public health priority for
individuals and for the healthcare delivery system.  Frailty identifies the
vulnerability of a person to endogenous and exogenous stressors, exposing the
individual to higher risk of negative health-related outcomes and marks a
transition between successful aging and disability.

Accurate assessment of Frailty can inform clinical management decisions and
assist with anticipating healthcare resource utilization.  Such assessments are
useful when validity of the Frailty Index scores are significantly associated
with clinically relevant variables.

In order to mitigate the risks posed by frailty through case management,
clinicians must be able to validly and efficiently assess the degree of frailty.
While multiple approaches to frailty assessment in the elderly have been
developed, these approaches remain inadequate in terms of efficiency (some are
time intensive), clinical meaningfulness (predictive of negative outcomes), or
both. Assessment approaches at the front line of care must be accurate and
efficient to integrate into the clinical workflow.

The capability to leverage clinical data and harness its potential to generate
knowledge rapidly to inform decisions can have transformative effects on complex
systems that provide services.  The Learning Health System (LHS) is one widely
held vision for realizing this transformation.  The LHS uses routine data from
service delivery and patient care to generate knowledge to continuously improve
healthcare [@friedman14].

`r (.nt<-note('','The following paragraph is the more detailed background of the
anchoring report on LHS from the IOM.  Provided FYI, not necessarily to include
in full narrative.',fmt='t', author='Stevens, Kathleen'))[1]`
In a 2013 report, the Institute of Medicine highlighted the use of routine data
from service delivery (e.g., electronic health records) to generate knowledge to
continuously improve healthcare.  “The committee believes that achieving a
learning health care system—one in which science and informatics,
patient-clinician partnerships, incentives, and culture are aligned to
promote and enable continuous and real-time improvement in both the
effectiveness and efficiency of care—is both necessary and possible for the
nation.” [@smith13a]
`r .nt[2]`


To address the need for assessment efficiency, we developed a Frailty Index that
is calculated from readily available electronic health record data, reflecting a
key improvement strategy from the Learning Healthcare System (LHS) paradigm.

# Methods

Following principles of implementation science [@meissner20a], clinical
stakeholders and experts (SE, CT) were directly engaged in the research design
to identify key outcomes that were meaningful, relevant, and useful in guiding
clinical care planning. We applied patient safety indicators used for routine
large-scale surveillance of hospital and health system performance [@southern17]
and extended the data set to include additional variables.

## Population

A random 1% sample (N=14,844) was drawn from the deidentified patient
records of a large academic health center and its teaching hospital
partner. Visits during which patient age was less than 18 years old were
excluded and then patients who had fewer than three visits in the
remaining data were excluded. To avoid bias, for each patient an index
visit date was chosen and only data recorded on or after that date was
used in analysis. To avoid distorted results in patients with sparse
visit histories, those who had fewer than two visit-dates after index
visit assignment were removed from the sample as were patients whose EFI
was never higher than 0. Most patients  had visits at the tails of their
histories during which EFI was 0 and no diagnoses, procedures, nor lab results
were recorded. These may have been missed appointments or accesses to patient
data that did not physically involve the patient, so they were excluded also.
Finally, the patients were randomly assigned to a development cohort 
(N=`r snip_dat['pat']` patients, `r snip_dat['vis']` visit-days) or a testing
cohort (N=3,220 patients, 56,320 visit-days). Sensitivity analysis was
done to see the effect of leaving in data from all visits by adult
patients and the overall direction of EFI's effect was the same but
`r fs('[the model performance was inflated, see supplement]')`.
All decisions about data processing and statistical analysis were made
using only the development cohort and blinded to the testing cohort. For
publication, the same analysis scripts were run on the testing cohort
and used to create all results reported here (the development version of
each table and figure is available in the supplemental materials). The
`r note('baseline characteristics of the testing cohort are shown in','
I wonder if there are any specific comparisons between frail and
non-frail in table 1 that should be discussed in the text. Race, sex,
age, number of visits?',author="Bokov, Alex F",date="2020-10-16T08:36:49Z")` 
Table \@ref(tab:table1).


***

```{r table1header}
    pander(rbind(c('.','','','','','','',''))
           ,caption='(\\#tab:table1) Cohort demographics\n\n');
```


```{r table1,results='asis'}
if(.outfmt == 'html'){
  .tb1formula <- c('sex_cd','race_cd','BLANK0',.outcomenames
                   ,'Patient age (years)','Frailty','Number of Visits') %>% 
    paste0('`',.,'`',collapse='+') %>% paste('~',.,'|`Frailty Stage`') %>% 
    formula;
  
  tb1 <- table1(.tb1formula,data=dat04,render.categorical=table1cat00) %>%
    rname(method='partial') %>% gsub('BLANK.','</br>',.) %>% 
    gsub('Patient age','<br/>Patient age',.);
  
  print(tb1);
};
```


## Electronic Frailty Index

@clegg16a developed an electronic frailty index (EFI) for UK health systems
following the methodology of Mitnitsky, Rockwood, et al. [@mitnitski01a;
@song10; @searle08a]. Recently, @pajewski19 adapted the EFI to use ICD10 and
ICD9. We further built on this work by also mapping laboratory tests to specific
LOINC codes, de-duplicating several items, and omitting data elements which are
not available in some EMR systems. We made the mapping tables and sample SQL 
code available for public use and discussion [@bokov20]. For each patient visit,
all distinct diagnoses and abnormal lab results over the preceding two-year
window for that patient were aggregated into a single EFI, a numeric value that
can range from 0 to 1 (but in practice seldom exceeded 0.6). As mentioned above,
we omitted all visits prior to a randomly select index visit for each patient.
This did not interfere with EFI calculation because those EFI values were
calculated separately for every distinct patient-date in our health system, and
then joined to the EHR data.

## Outcomes

The primary outcomes we predicted with EFI were hospital-acquired infections,
non-operative hospital-acquired trauma, cardiac complications, and the occurrence
of one or more of these or any other patient safety events defined in
@southern17. The advantage of using these PSI definitions is that they leverage
the greater specificity of ICD10 codes (rather than a straight mapping from AHCR
ICD9 codes) and the uniform denominator for all but maternal and fetal PSIs
greatly simplifies calculation. Of the remaining PSIs defined by @southern17, 2
were excluded because pregnancies were outside the scope of our protocol. Three
(drug adverse event, CNS complications, and VTIs) were rare enough that they are
difficult to interpret as survival curves but nevertheless are shown in 
`r fs('supplementary table S1')`. The remaining 10 patient safety indicators 
were too rare to reliably analyze in a sample this size taken from a _general_
patient population, but it is likely that more targeted samples will be more
enriched for these events.


## Statistical Analysis

For each outcome of interest, we used a Cox proportional hazard model to
estimate the risk of the first occurrences of the outcome after the patients'
respective index visits using EFI as the predictor. Unlike earlier studies, we
treated EFI as a time-varying numeric predictor with multiple follow-ups per
patient. Since only the first occurrence was being predicted, a recurring event
model was not necessary. Patients were followed up to the first occurrences of
the respective outcomes, so the standard form of the Cox model was sufficient.
In order to determine whether predictive accuracy of EFI was only limited to
some age subgroups, we repeated the analysis separately for the following age
ranges: 18-45 years old, 45-65 years old, and 65 or above. These split the
dataset into three roughly equal subgroups.


# Results

Table \@ref(tab:tb2) shows the results of Cox proportional hazard models for
each of the responses, with EFI as the predictor. 
`r (.nt <- note('','longnote00',date='2020-10-16T08:46',fmt='b'))[1]`
\@bokov
\
\
TODO: Fill in the appropriate values for these placeholders. 
\
\
Also say how many (number and %) of the frail and non-frail groups
experienced each event. Or, maybe put it into the table.
`r .nt[2]`
For each 0.1 increase in EFI, we found at least a doubling of risk: 
`r pander(snip_resultsfold)`. In other words, a patient with a frailty of 0 
would have a `r fs('X% ()')`chance of experiencing a patient safety compared to 
a patient with a frailty of 0.2, who would have a `r fs('Y% ()')` chance, while 
the overall risk is `r fs('Z%()')`. `r .nt[3]`
The p-values shown have been adjusted for multiple comparisons
(`r as.english(nrow(tb2)) %>% as.character` outcomes reported in
one study) using the @holm79 method and in all cases are 
`r note('highly significant','Report a comparison between the developmental and test data.'
,date='2020-10-16T08:42')`


```{r tb2}

tb2[,c('predictor','Outcome','β^ (95% CI)','fold-change (95% CI)','SE','Z','P, adjusted','# Events','# Visits')] %>%
  pander(digits=3, row.names=FALSE,caption='(\\#tab:tb2) Cox-proportional hazards with EFI as a predictor');
```

For each outcome, we re-fit the same Cox proportional on subsets of the data
separated by age at visit: 18-45 years old, 45-65 years old, and 65 or older.
We found that for most of the primary outcomes (as well as for most of the 
analyzable outcomes, see supplementary table S1) EFI was a good predictor in
all three age groups.

```{r tb3}
tb3 %>% mutate(Outcome = rnameshort(Outcome)) %>%
         pander(caption='(\\#tab:tb3) Performance of EFI models');
```


In Table \@ref(tab:tb3) we report the performance of EFI as the predictor for
each combination of age group and outcome, as well as for the cohort overall
(Frailty). 

In figures \@ref(fig:kmplots) a-`r letters[length(fits)]` we show Kaplan-Meier 
plots for each of the outcomes stratified by whether EFI is greater than 0.19 
(Frail=TRUE) or less (Frail=FALSE), the cut-point established by [@stow18].


::: {.MultiFigure custom-style="MultiFigure"}
```{r kmplots, fig.show="hold", fig.height=4, fig.width=12, fig.cap=" - "}
# . kmplots ----
.subfig <- 0;
for(jj in fits) {.subfig <- .subfig + 1;
with(jj,{
  message(dispname);
  print(ggmatrix(multiplot,nrow=1,ncol=3,legend=grab_legend(plot),title=dispname
           ,xAxisLabels = names(multidata)
           ,xlab=plot$label$x
           ,ylab=plot$label$y) + 
          labs(caption=paste0('Figure 1 ',letters[.subfig],'. \n')));
  cat('\n\n\n');
  # pander(modelsummary);
  # cat("\n******\n");
  })};

```
:::


# Discussion

We implemented an electronic Frailty Index using the Rockwood
deficit-accumulation framework and used it to predict patient safety events in
real-world data from an EHR system. Our results add to the evidence that
Rockwood indexes are a rapid, low-effort method of risk assessment that scales
to large patient populations. As yet there is no gold standard method to assess
Frailty in clinical practice. Currently available frailty assessment tools
geriatric practice have good validity (for example Fried et al. (2001)) are time
intensive and often difficult to implement in a busy general practice. Assessing
frailty helps clinicians identify high risk patients and tailor interventions to
preserve function and prevent health decline and poor outcomes and impact cost
of care. Because of the simplicity of the Rockwood Frailty Index, it is more
likely to be adopted by clinicians in a busy practice.

Unlike previous studies, we did not restrict our sample to older patients. We
found a highly significant relationship in all age groups between frailty and
hospital-acquired infections, hospital-acquired trauma, cardiac complications,
and overall occurrence of patient safety events even after a very conservative
multiple comparison correction.  [go through the rest, mention which ones were
not significant]. Moreover, except the 18-45 year old group for infections and
trauma, all the models had a concordance greater than 0.7.

Our sample population amounts to making predictions about randomly selected
patients assessed only on EHR data available up to a randomly selected encounter
for each of these patients. As a consequence some clinically relevant events
such as pressure ulcers, post-surgical complications, and ICU stays were less
frequent than they would be in a more narrowly focused sample (e.g. only
geriatric patients or only inpatients). Nevertheless frailty does predict
patient safety events that may be less anticipated in younger patients. As the
next step toward clinical deployment we intend to repeat this analysis with
patients seen at the internal medicine department of our health system, who tend
toward older ages and more comorbidities but are not solely geriatric patients.

Our data shares the fundamental limitation of the EHR system from which it was
obtained: like all EHR systems, it only has information that providers and
coders put into it. Events taking place outside the health system or at
un-connected health systems are not visible to our analysis. On the other hand,
providers who rely on EHR systems at point of care are also working under these
limitations on. The data we used is representative of this scenario, and despite
the limitations EFI provides accurate predictions of poor patient outcomes.
Because our implementation of the Rockwood index real EHR data, it is more
directly transferable to clinical use than implementations based on curated
registries and more immediate than claims data. EFI is most accurate for
patients who have accumulated a reasonable in-system visit-history but further
work is needed to find a more precise relationship between the length of a
patient's visit-history and the accuracy of EFI and to better distinguish
genuinely non-frail patients from those who have a lot of missing data because
they are often seen outside the researchers' health system.

There is no gold standard method to assess Frailty in clinical practice.
Currently available frailty assessment tools used in geriatric practice have
good validity (for example @fried01a) but these are time
intensive and often difficult to implement in a busy general practice.
Assessing frailty helps clinicians identify high risk patients and tailor
interventions to prevent health decline and poor outcomes. Because of the
simplicity of the Rockwood Frailty Index, it is more likely to be adopted by
clinicians in a busy practice.

# Conclusions

Results from our Cox proportional hazard models in Table 2 demonstrate a
statistically significant association of EFI with clinically meaningful outcomes
from EHR data. The c-statistics in Table 3 demonstrate that the Cox models are
well fitted to this data. This contributes to a growing body of evidence that
risk scores built using the Rockwood framework [@mitnitski01a] will be a
valuable tool for clinical decision support not restricted to any one illness or
specialty. The variables used to calculate EFI are ones that are in some form
available in every EMR system. If there are EMR systems where a few of these
variables are not available, Rockwood deficit accumulation indexes have been
shown to continue giving consistent results despite variations in what
individual codes are available as long as there is a sufficiently large and
representative collection of deficits into which these variables can be binned
`r fs('[ref]')`.

To facilitate adoption and refinement of EFI, we are publishing not only the
code mappings but also scripts that implement the mapping algorithm with
detailed notes on adapting them to local environments. We intend to evolve these
scripts into a self-contained app that interfaces with EMR systems (via FHIR) to
provide real-time frailty assessment at the clinical point of care and assist
clinicians in developing care plans to mitigate the risks of frailty.


`r note('...','Can we make a parallel statement in our CONCLUSIONS? 
AN INTERESTING QUOTE FROM Southern:  “The methodological work presented here
utilizes the unique potential of diagnosis timing information to produce a
clinically relevant listing of diagnosis codes that have potential as PSIs that
may overcome some of the notable shortcomings of existing PSI systems. The
resulting work has great potential to inform future approaches to monitoring
health system performance and quality/safety improvement internationally.”'
,author="Stevens, Kathleen",date="2020-10-16T08:30:43Z")`


# Acknowledgments

`r note("This work was supported by NIH/NCATS UL1TR001120 and UL1TR002645 (AFB,
KRS), the Long School of Medicine KL2 Award (AFB), the Claude D. Pepper Older
Americans Independence Center P30 AG044271 (SER),  Berneice Castella
Distinguished Professorship in Aging Research in Nursing  (KRS)"
,"Did I get everybody's correctly?",date='2020-12-04T14:20')`


# References



---
date: '2020-09-22'

---