#5 Bonus Model Deployment using Azure Machine Learning Model Management Service.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note: It is recommended to run this notebook from an [Azure DSVM](https://docs.microsoft.com/en-us/azure/machine-learning/data-science-virtual-machine/overview) instance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Useful for being able to dump images into the Notebook\n",
    "import IPython.display as D"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Big Picture\n",
    "\n",
    "In the previous notebooks, we tried together [Custom Vision service](https://github.com/CatalystCode/CVWorkshop/blob/master/%232%20Policy%20Classfication%20With%20Custom%20Vision%20Service.ipynb) in addition to [Transfer Learning](https://github.com/CatalystCode/CVWorkshop/blob/master/%233%20Policy%20Recognition%20with%20Resnet%20and%20Transfer%20Learning.ipynb) which is one of the popular approaches in deep learning where pre-trained models are used as the starting point on computer vision.\n",
    "\n",
    "So if we look on the big picture, we will realize that the previous notebooks are focusing on preparing/loading training data set, building models, training models then evaluating the output. \n",
    "\n",
    "In this tutorial, we will move the focus to operationalizing models by deploying trained models as web services so that you can consume it later from any client application via REST API call. For that purporse, we are using Azure Machine Learning Model Management Service.\n",
    "\n",
    "![](https://modelmanagementimages.blob.core.windows.net/notebookimages/big%20picture.png)\n",
    "\n",
    "\n",
    "# Azure Model Management Service\n",
    "\n",
    "Azure Machine Learning Model Management enables you to manage and deploy machine-learning models. It provides different services like creating Docker containers with models for local testing, deploying models to production through Azure ML Compute Environment with [Azure Container Service](https://azure.microsoft.com/en-us/services/container-service/) and versioning & tracking models. Learn more here: [Conceptual Overview of Azure Model Management Service](https://docs.microsoft.com/en-us/azure/machine-learning/preview/model-management-overview)\n",
    "\n",
    "### What's needed to deploy my model?\n",
    "* Your Model File or Directory of Model Files\n",
    "* You need to create a score.py that loads your model and returns the prediction result(s) using the model and also used to generates a schema JSON file\n",
    "* Schema JSON file for API parameters (validates API input and output)\n",
    "* Runtime Environment Choice e.g. python or spark-py \n",
    "* Conda dependency file listing runtime dependencies\n",
    "\n",
    "### How it works: \n",
    "![](https://docs.microsoft.com/en-us/azure/machine-learning/preview/media/model-management-overview/modelmanagement.png)\n",
    "\n",
    "Learn more here: [Conceptual Overview of Azure Model Management Service](https://docs.microsoft.com/en-us/azure/machine-learning/preview/model-management-overview)\n",
    "\n",
    "### Deployment Steps:\n",
    "* Use your saved, trained, Machine Learning model\n",
    "* Create a schema for your web service's input and output data\n",
    "* Create a Docker-based container image\n",
    "* Create and deploy the web service\n",
    "\n",
    "### Deployment Target Environments:\n",
    "\n",
    "1. Local Environment: You can set up a local environment to deploy and test your web service on your local machine or DSVM. (Requires you to install Docker on the machine)\n",
    "\n",
    "2. Production Environment: You can use Cluster deployment for high-scale production scenarios. It sets up an ACS cluster with Kubernetes as the orchestrator. The ACS cluster can be scaled out to handle larger throughput for your web service calls. (Kubernetes deployment on an Azure Container Service (ACS) cluster)\n",
    "\n",
    "![](https://modelmanagementimages.blob.core.windows.net/notebookimages/deployment%20targets.png)\n",
    "\n",
    "# Challenge \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run the following train.py from the notebook to generate a classifier model \n",
    "from sklearn.svm import SVC\n",
    "from cvworkshop_utils import ensure_exists\n",
    "import pickle\n",
    "\n",
    "# indicator1, NF1, cellprofiling\n",
    "X = [[362, 160, 88], [354, 140, 86], [320, 120, 76], [308, 108, 47], [332, 130, 80], [380, 180, 94], [350, 128, 78],\n",
    "     [354, 140, 80], [318, 110, 74], [342, 150, 84], [362, 170, 86]]\n",
    "\n",
    "Y = ['positive', 'positive', 'negative', 'negative', 'positive', 'positive', 'negative', 'negative', 'negative', 'positive', 'positive']\n",
    "\n",
    "clf = SVC()\n",
    "clf = clf.fit(X, Y)\n",
    "\n",
    "print('Predicted value:', clf.predict([[380, 140, 86]]))\n",
    "print('Accuracy', clf.score(X,Y))\n",
    "\n",
    "print('Export the model to output/trainedModel.pkl')\n",
    "ensure_exists('output')\n",
    "f = open('output/trainedModel.pkl', 'wb')\n",
    "pickle.dump(clf, f)\n",
    "f.close()\n",
    "\n",
    "print('Import the model from output/trainedModel.pkl')\n",
    "f2 = open('output/trainedModel.pkl', 'rb')\n",
    "clf2 = pickle.load(f2)\n",
    "\n",
    "X_new = [[308, 108, 70]]\n",
    "print('New Sample:', X_new)\n",
    "print('Predicted class:', clf2.predict(X_new))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now navigate to the repository root directory then **open \"output\" folder** and you should be able to see the **created trained model file \"trainedModel.pkl\"**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run the following score.py from the notebook to generate the web serivce schema JSON file\n",
    "# Learn more about creating score file from here: https://docs.microsoft.com/en-us/azure/machine-learning/preview/model-management-service-deploy\n",
    "\n",
    "def init():\n",
    "    from sklearn.externals import joblib\n",
    "\n",
    "    global model\n",
    "    model = joblib.load('output/trainedModel.pkl')\n",
    "\n",
    "def run(input_df):\n",
    "    import json\n",
    "    pred = model.predict(input_df)\n",
    "    return json.dumps(str(pred[0]))\n",
    "\n",
    "def main():\n",
    "  from azureml.api.schema.dataTypes import DataTypes\n",
    "  from azureml.api.schema.sampleDefinition import SampleDefinition\n",
    "  from azureml.api.realtime.services import generate_schema\n",
    "  import pandas\n",
    "\n",
    "  df = pandas.DataFrame(data=[[380, 120, 76]], columns=['indicator1', 'NF1', 'cellprofiling'])\n",
    "\n",
    "  # Check the output of the function\n",
    "  init()\n",
    "  input1 = pandas.DataFrame([[380, 120, 76]])\n",
    "  print(\"Result: \" + run(input1))\n",
    "  \n",
    "  inputs = {\"input_df\": SampleDefinition(DataTypes.PANDAS, df)}\n",
    "\n",
    "  # Generate the service_schema.json\n",
    "  generate_schema(run_func=run, inputs=inputs, filepath='output/service_schema.json')\n",
    "  print(\"Schema generated\")\n",
    "\n",
    "if __name__ == \"__main__\":\n",
    "    main()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Navigate again to the repository root directory then **open \"output\" folder** and you should be able to see the **created JSON schema file \"service_schema.json\"**\n",
    "\n",
    "By reaching this point, we now have what's needed (Score.py file, trained model and JSON schema file) to start deploying our trained model using Azure Model Management Service. Now it's the time to think which deployment environoment are you going to consider as deployment target (Local Deployment or Cluster Deploymment). In this tutorial, we will walk through both scenarios so feel free to either walk through **scenario A** or **scenario B** or even **both**.\n",
    "\n",
    "Before deploying, first login to you Azure subscription using your command prompt and register few environment providers.\n",
    "\n",
    "Once you execute this command, the command prompt will show you a message asking you to open your web browser then navigate to https://aka.ms/devicelogin to enter a specific code given in the terminal to login to your Azure subscription."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Return to your command prompt and execute the following commands\n",
    "!az login "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Once you are logged in, now let's execute the following commands to register our environment providers\n",
    "!az provider register -n Microsoft.MachineLearningCompute\n",
    "!az provider register -n Microsoft.ContainerRegistry\n",
    "!az provider register -n Microsoft.ContainerService"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Registering the environments takes some time so you can monitor the status using the following command: \n",
    "```\n",
    "az provider show -n {Envrionment Provider Name}\n",
    "```\n",
    "Before you complete this tutorial, make sure that all the registration status for all the providers are **\"Registered\"**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!az provider show -n Microsoft.MachineLearningCompute"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!az provider show -n Microsoft.ContainerRegistry"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!az provider show -n Microsoft.ContainerService"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "While waiting the enviroment providers to be registered, you can create a resource group to include all the resources that we are going to provision through this tutorial."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# command format az group create --name {group name} --location {azure region}\n",
    "!az group create --name capetownrg --location westus"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Also create a Model Management account to be used for our deployment whether the local deployment or the custer deployment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# command format az ml account modelmanagement create -l {resource targeted region} -n {model management name} -g {name of created resource group}\n",
    "!az ml account modelmanagement create -l eastus2 -n capetownmodelmgmt -g capetownrg"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once your model management account is create, set the model management you created to be used in our deployment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# command format az ml account modelmanagement set -n {your model management account name} -g {name of created resource group}\n",
    "!az ml account modelmanagement set -n capetownmodelmgmt -g capetownrg"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Cluster Deployment - Enviroment Setup:\n",
    "\n",
    "If you want to deploy from a cluster you need to setup a cluster deployment environment using the following command first to be able to deploy our trained model as a web service"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***Creating the environment may take 10-20 minutes. ***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# command format az ml env setup -c --name {your environment name} --location {azure region} -g {name of created resource group}\n",
    "!az ml env setup -c --name capetownenv --location eastus2 -g capetownrg -y --debug"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can use the following command to monitor the status:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# command format az ml env show -g {name of created resource group} -n {your environment name}\n",
    "!az ml env show -g capetownrg -n capetownenv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once your provisioning status is \"Succeeded\", open your web browser and login to your Azure subscription through the portal and you should be able to see the following resources created in your resource group:\n",
    "\n",
    "* A storage account\n",
    "* An Azure Container Registry (ACR)\n",
    "* A Kubernetes deployment on an Azure Container Service (ACS) cluster\n",
    "* An Application insights account\n",
    "\n",
    "Now set set your environment as your deployment enviroment using the following command:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# command format az ml env set -n {your environment name} -g {name of created resource group}\n",
    "!az ml env set -n capetownenv -g capetownrg --debug"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now feel free to choose one of the following deployment environments as your targeted environment.\n",
    "\n",
    "### Local Deployment - Enviroment Setup:\n",
    "\n",
    "You need to set up a local environment using the following command first to be able to deploy our trained model as a web service"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# command format az ml env setup -l {azure region} -n {your environment name} -g {name of created resource group}\n",
    "# !az ml env setup -l eastus2 -n capetownlocalenv -g capetownrg -y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Creating the enviroment may take some time so you can use the following command to monitor the status:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# command format az ml env show -g {name of created resource group} -n {your environment name}\n",
    "# !az ml env show -g capetownrg -n capetownlocalenv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once your provisioning status is \"Succeeded\", open your web browser and login to your Azure subscription through the portal and you should be able to see the following resources created in your resource group:\n",
    "\n",
    "* A storage account\n",
    "* An Azure Container Registry (ACR)\n",
    "* An Application insights account\n",
    "\n",
    "Now set set your environment as your deployment enviroment using the following command:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# command format az ml env set -n {your environment name} -g {name of created resource group}\n",
    "!az ml env set -n capetownenv -g capetownrg  --debug"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Whether you finish your enviroment setup by following Scenario A or Scenario B. Now you are ready to deploy our trained model as a web service to cosnume later from any application.**\n",
    "\n",
    "### Create your Web Service:\n",
    "\n",
    "As a reminder, here's what's needed to create your webservice:\n",
    "* Your trained model file -> in our case it's \"output/trainedModel.pkl\"\n",
    "* Your score.py file which loads your model and returns the prediction result(s) using the model -> in our case it's \"modelmanagement/score.py\"\n",
    "* Your JSON schema file that automatically validate the input and output of your web service -> in our case it's \"output/service_schema.json\"\n",
    "* You runtime environment for the Docker container -> in our case it's \"python\"\n",
    "* conda dependencies file for additional python packages. (We don't have it in our case)\n",
    "\n",
    "Use the following command to create your web service:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# command format az ml service create realtime --model-file {model file/folder path} -f {scoring file} -n {your web service name} -s {json schema file} -r {runtime choice} -c {conda dependencies file}\n",
    "!az ml service create realtime -m output/trainedModel.pkl -f score.py -n classifierservice -s output/service_schema.json -r python --debug"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Test your Web Service:\n",
    "\n",
    "Once the web service is successfully created, open your web browser and login to your Azure subscription through the portal then jump into your resource group and open your model management account.\n",
    " \n",
    "**Open** your model management account\n",
    "\n",
    "![](https://modelmanagementimages.blob.core.windows.net/notebookimages/test%2001.png)\n",
    "\n",
    "**Click** on \"Model Management\" under Application Settings\n",
    "\n",
    "![](https://modelmanagementimages.blob.core.windows.net/notebookimages/test%2002.png)\n",
    "\n",
    "**Click** on \"Services\" and you select your created \"classifier\" service from the righ hand side panel\n",
    "\n",
    "![](https://modelmanagementimages.blob.core.windows.net/notebookimages/test%2003.png)\n",
    "\n",
    "**Copy** your \"Service id\", \"URL\" and \"Primary key\"\n",
    "\n",
    "![](https://modelmanagementimages.blob.core.windows.net/notebookimages/test%2004.png)\n",
    "\n",
    "\n",
    "**Call your web service from your terminal:**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# command format az ml service run realtime -i {your service id} -d {json input for your web service}\n",
    "\n",
    "# usage example\n",
    "!az ml service run realtime -i YOUR_SERVICE_ID -d \"{\\\"input_df\\\": [{\\\"NF1\\\": 120, \\\"cellprofiling\\\": 76, \\\"indicator1\\\": 380}]}\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Call your web service from [Postman](https://www.getpostman.com/):**\n",
    "\n",
    "![](https://modelmanagementimages.blob.core.windows.net/notebookimages/headers.png)\n",
    "![](https://modelmanagementimages.blob.core.windows.net/notebookimages/body.png)\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}