07-loadtesting.Rmd

# Load Testing

Load testing helps developers and administrators estimate how many users their application can support. If an application requires tuning, load testing and load test result analysis can be used to identify performance bottlenecks and to guide changes to infrastructure, configuration, or code.

It’s a common misconception that _"Shiny doesn’t scale"_. In actuality, properly-architected Shiny applications can be scaled horizontally, a fact which Sean Lopp was recently able to demonstrate at rstudio::conf 2018. We used `shinycannon` to simulate 10,000 concurrent users interacting with an application deployed to AWS. You can see a recording of Sean’s talk and the load test demonstration here: [Scaling Shiny](https://www.rstudio.com/resources/videos/scaling-shiny/)

To perform a load test you’ll need two pieces of software: `shinyloadtest` and `shinycannon`.

- `shinyloadtest` is an R package used to generate recordings and analyze results. You should install it on your development machine. [GitHub](https://github.com/rstudio/shinyloadtest)
- `shinycannon` is the command-line replay tool. You can install it on your development machine for testing, but for best results we recommend installing it on a server, and preferably not the one the application under test is also on. [GitHub](https://github.com/rstudio/shinycannon)

See the Load Testing Quickstart [Here](https://rstudio.github.io/shinyloadtest/#quick-start).

## Optimization Loop Methodology

- **Benchmark:** Use `shinyloadtest::record_session()` to record interaction, `shinycannon` to simulate users
- **Analyze:** Visualize and interpret the metrics
- **Recommend:** Propose ways for the capacity of the application to be increased
- **Optimize:** Implement recommendations and benchmark again. Repeat until satisfied

_Workflow:_

![Load Test Workflow](imgs/loadtesting/loadtest-workflow.png)

- Use `shinyloadtest to record a session with a Shiny app
- Generate load with `shinycannon`
- Analyze metrics with `shinyloadtest`
- Make changes to the Shiny application
- Generate load and analyze again

![Load Test Steps](imgs/loadtesting/loadtest-steps.png)

[Load Testing Shiny Applications](https://rstudio.github.io/shinyloadtest/index.html)

## Activity: Load Testing

**First: Open runloadtest.R,do the pre-run checklist**

**Discussion:**

_Preparing for the load test_

- First, what is up with the pre-run checklist? Any idea why these steps are (currently) necessary?
- The `runloadtest.R` file is going to start with a baseline test of 1 and then a test of 25. Why is the baseline important?

**Deliverable: Run the load test**

- Follow the first set of commands in `runloadtest.R`
- How did the experience for 1 user compare to the experience for 25 users?

---

**References and Resources:**

**Webinar:**

- [Load testing Shiny by Alan Dipert](https://resources.rstudio.com/webinars/load-testing-shiny-alan-dipert)
- [Webinar Slides](https://github.com/rstudio/webinars/blob/master/63-shinyloadtest/slides.pdf)

**Vignettes:**

- [Analyzing Load Test Logs](https://rstudio.github.io/shinyloadtest/articles/analyzing-load-test-logs.html)
- [Case Study: Scaling an Application](https://rstudio.github.io/shinyloadtest/articles/case-study-scaling.html)
- [Limitations of `shinyloadtest`](https://rstudio.github.io/shinyloadtest/articles/limitations-of-shinyloadtest.html)
- [Load Testing Authenticated Apps](https://rstudio.github.io/shinyloadtest/articles/load-testing-authenticated-apps.html)