Shiny (Python) ML training example (#105)

* Shiny core example

readme

zip

removed unused function

shiny doc

removed parallel function

shiny core toc

renamed folder

rename

* reamde changes

* Delete examples/docker/shiny-core/Dockerfile

* removed core files

* env variables

* env variable

* dummy record

* readme

* zip

* polling function

* timestamp error

* timestamp error

* updates instructions

* minor updates

---------

Co-authored-by: Eduardo Blancas <[email protected]>

doc/_toc.yml

@@ -29,10 +29,11 @@ parts:
       - file: apps/panel
       - file: apps/chainlit
       - file: apps/gradio
+      - file: apps/shiny
       - file: apps/shiny-express
       - file: apps/dash
       - file: apps/docker
-      - file: apps/chalk-it      
+      - file: apps/chalk-it
 
   - caption: App examples
     chapters:
@@ -44,4 +45,4 @@ parts:
 
   - caption: FAQ
     chapters:
-      - file: faq/docker-error
+      - file: faq/docker-error

doc/apps/shiny.md

@@ -0,0 +1,43 @@
+# Shiny (Python)
+
+To deploy a [Shiny](https://shiny.posit.co/py/docs/overview.html) Python application to Ploomber Cloud you need:
+
+- A `Dockerfile`
+- A Shiny Python project
+
+## `Dockerfile`
+
+Use this [template](https://github.com/ploomber/doc/blob/main/examples/docker/shiny/Dockerfile) `Dockerfile`:
+
+```Dockerfile
+FROM python:3.11
+
+COPY app.py app.py
+COPY requirements.txt requirements.txt
+RUN pip install -r requirements.txt
+
+ENTRYPOINT ["shiny", "run", "app.py", "--host", "0.0.0.0", "--port", "80"]
+```
+
+## Testing locally
+
+To test your app, you can use `docker` locally:
+
+```sh
+# build the docker image
+docker build . -t shiny-app
+
+# run it
+docker run -p 5000:80 shiny-app
+```
+
+Now, open [http://0.0.0.0:5000/](http://0.0.0.0:5000/) to see your app.
+
+
+## Deploy
+
+Once you have all your files, create a zip file.
+
+To deploy a Shiny app from the deployment menu, select the Docker option and follow the instructions:
+
+![](../static/docker.png)

examples/docker/shiny/Dockerfile

@@ -0,0 +1,7 @@
+FROM python:3.11
+
+COPY app.py app.py
+COPY requirements.txt requirements.txt
+RUN pip install -r requirements.txt
+
+ENTRYPOINT ["shiny", "run", "app.py", "--host", "0.0.0.0", "--port", "80"]

examples/docker/shiny/README.md

@@ -0,0 +1,85 @@
+# Model monitoring application
+
+This app is built on Shiny and allows you to visualize model training accuracy scores in realtime in 2 steps:
+
+* Train multiple models with different parameters locally, and store the accuracy scores to a remote database at the end of every epoch.
+* The visualisation application dynamically detects changes in the remote database and generates a plot as the training progresses.
+
+## Connecting to MongoDB Cloud
+
+For the purpose of this tutorial we will use the `MongoDB Atlas` cloud database. It allows users to host a database for free.
+Let's see the steps for creating a cluster:
+
+First create an account on [MongoDB cloud](https://www.mongodb.com/atlas/database) and create a new project:
+
+![](./static/create-project.png)
+
+Then select `Create deployment` and select the free tier `M0` or your desired tier. 
+
+In the `Security Quickstart` page, select your authentication type. For this tutorial we have generated a username and password.
+
+Select the `Cloud Environment` and open the `Network Access Page` in a new tab.
+
+![](./static/network-access.png)
+
+Now select `ALLOW ACCESS FROM ANYWHERE` and confirm to allow all IP addresses to connect to the cluster.
+
+![](./static/ip-access.png)
+
+Once you have completed the access rules setup, select the `CONNECT` button on the `Overview` page to get the connection details. Specifically, copy the connection URI and set it in an environment variable `MONGODB_CONNECTION_URI`:
+
+```bash
+export MONGODB_CONNECTION_URI="connection_uri"
+```
+
+![](./static/connect.png)
+
+## Deploy application
+
+Create a zip file using `Dockerfile`, `app.py` and `requirements.txt` and deploy this file to Ploomber cloud.
+Refer to the [Shiny deployment documentation](https://docs.cloud.ploomber.io/en/latest/apps/shiny.html) for more details.
+
+Note that you also need to set the MongoDB connection URI as an environment variable:
+
+![](./static/env-variable.png)
+
+Once the `VIEW APPLICATION` button is enabled, you can start the training script and monitor the training.
+
+## Train models
+
+We will use the [MNIST dataset](https://www.tensorflow.org/datasets/catalog/mnist) and train two different models on the data. The models differ in the number of units in the `Dense` layer.
+The training scripts write the accuracy values at the end of every epoch to the MongoDB database in a collection called `accuracy_scores`.
+
+First create a virtual environment and install the required packages:
+
+```bash
+pip install tensorflow scikit-learn "pymongo[srv]"
+```
+
+To train two models parallely, open two terminals and run the below commands in each respectively:
+
+```bash
+python train.py --model model_1 --units 128
+```
+
+```bash
+python train.py --model model_2 --units 12
+```
+
+In case you wish to change the model names you need to update the same in `app.py`.
+
+Once the scripts are running you can monitor the accuracy scores as the training progresses on the Ploomber Cloud application deployed in above step:
+
+![](./static/monitor.png)
+
+Currently, the threshold for an accurate model (marked in green) is 0.9. You can modify it by setting `THRESHOLD_MID` in `app.py`.
+
+If you wish to restart the training process you may drop the collection to ensure a clean plot:
+
+![](./static/delete-collection.png)
+
+Or using Python:
+
+```python
+python clean.py
+```

examples/docker/shiny/app.py

@@ -0,0 +1,149 @@
+import os
+import pandas as pd
+from datetime import datetime
+
+import plotly.express as px
+from shinywidgets import output_widget, render_widget
+from faicons import icon_svg
+
+from pymongo.mongo_client import MongoClient
+from pymongo.server_api import ServerApi
+from shiny import App, Inputs, Outputs, Session, reactive, render, ui
+
+THRESHOLD_MID = 0.9
+THRESHOLD_MID_COLOR = "rgb(0, 137, 26)"
+THRESHOLD_LOW = 0.5
+THRESHOLD_LOW_COLOR = "rgb(193, 0, 0)"
+
+URI = os.environ.get("MONGODB_CONNECTION_URI", "")
+
+# Create a new client and connect to the server
+client = MongoClient(URI, server_api=ServerApi("1"))
+
+try:
+    client.admin.command("ping")
+    print("Successfully connected to MongoDB!")
+except Exception as e:
+    raise ConnectionError(str(e)) from e
+
+db = client.myDatabase
+my_collection = db["accuracy_scores"]
+
+
+def number_of_observations():
+    n = my_collection.count_documents({})
+    if n == 0:
+        # insert random record to prevent KeyError message when no records present in DB
+        my_collection.insert_one({"model": "", "score": 0, "timestamp": datetime.now()})
+    return n
+
+
+@reactive.poll(lambda: number_of_observations())
+def df():
+    """
+    @reactive.poll calls a cheap query (`last_modified()`) every 1 second to check if
+    the expensive query (`df()`) should be run and downstream calculations should be
+    updated.
+    """
+    results = my_collection.find().sort({"timestamp": -1}).limit(50)
+    tbl = pd.DataFrame(list(results))
+    tbl["time"] = tbl["timestamp"]
+    # Reverse order of rows
+    tbl = tbl.iloc[::-1]
+    return tbl
+
+
+model_names = ["model_1", "model_2"]
+model_colors = {
+    name: color
+    for name, color in zip(model_names, px.colors.qualitative.D3[0 : len(model_names)])
+}
+
+
+def make_value_box(model, score):
+    theme = "text-success"
+    icon = icon_svg("check", width="50px")
+    if score < THRESHOLD_MID:
+        theme = "text-warning"
+        icon = icon_svg("triangle-exclamation", width="50px")
+    if score < THRESHOLD_LOW:
+        theme = "bg-danger"
+        icon = icon_svg("circle-exclamation", width="50px")
+
+    return ui.value_box(model, ui.h2(score), theme=theme, showcase=icon)
+
+
+app_ui = ui.page_sidebar(
+    ui.sidebar(
+        ui.input_checkbox_group("models", "Models", model_names, selected=model_names),
+    ),
+    ui.row(
+        ui.h1("Model monitoring dashboard"),
+        ui.h4("Deploy your own at ploomber.io"),
+        ui.output_ui("value_boxes"),
+    ),
+    ui.row(
+        ui.card(output_widget("plot_timeseries")),
+    ),
+)
+
+
+def server(input: Inputs, output: Outputs, session: Session):
+    @reactive.Calc
+    def filtered_df():
+        """
+        Return the data frame that should be displayed in the app, based on the user's
+        input. This will be either the latest rows, or a specific time range. Also
+        filter out rows for models that the user has deselected.
+        """
+        data = df()
+
+        # Filter the rows so we only include the desired models
+        return data[data["model"].isin(input.models())]
+
+    @reactive.Calc
+    def filtered_model_names():
+        return filtered_df()["model"].unique()
+
+    @output
+    @render.ui
+    def value_boxes():
+        data = filtered_df()
+        models = data["model"].unique().tolist()
+        scores_by_model = {
+            x: data[data["model"] == x].iloc[-1]["score"] for x in models
+        }
+        # Round scores to 2 decimal places
+        scores_by_model = {x: round(y, 2) for x, y in scores_by_model.items()}
+
+        return ui.layout_columns(
+            *[
+                # For each model, return a value_box with the score, colored based on
+                # how high the score is.
+                make_value_box(model, score)
+                for model, score in scores_by_model.items()
+            ],
+            width="135px",
+        )
+
+    @render_widget
+    def plot_timeseries():
+        """
+        Returns a Plotly Figure visualization. Streams new data to the Plotly widget in
+        the browser whenever filtered_df() updates, and completely recreates the figure
+        when filtered_model_names() changes (see recreate_key=... above).
+        """
+        fig = px.line(
+            filtered_df(),
+            x="time",
+            y="score",
+            labels=dict(score="accuracy"),
+            color="model",
+            color_discrete_map=model_colors,
+            template="simple_white",
+        )
+
+        return fig
+
+
+app = App(app_ui, server)

examples/docker/shiny/clean.py

@@ -0,0 +1,23 @@
+import os
+
+
+from pymongo.mongo_client import MongoClient
+from pymongo.server_api import ServerApi
+
+
+if __name__ == "__main__":
+    URI = os.environ.get("MONGODB_CONNECTION_URI", "")
+
+    client = MongoClient(URI, server_api=ServerApi("1"))
+
+    try:
+        client.admin.command("ping")
+        print("Successfully connected to MongoDB!")
+    except Exception as e:
+        raise ConnectionError(str(e)) from e
+    else:
+        db = client.myDatabase
+        my_collection = db["accuracy_scores"]
+
+        my_collection.delete_many({})
+        print("Collection cleared!")

examples/docker/shiny/requirements.txt

@@ -0,0 +1,6 @@
+pandas
+plotly
+shiny
+shinywidgets
+faicons
+pymongo[srv]