4_grpc_requests_multi_model.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "f73046ff",
   "metadata": {},
   "source": [
    "# GRPC Inference"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "443e7e73-24cb-4f03-9491-a6edcc24f0cc",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Setup\n",
    "\n",
    "Verify that following variable settings match your deployed model's resource name and grpc URL. The following code assumes that the kube service is in the same namespace, but you could refer to it in full with the namespace, for example: `http://modelmesh-serving.project-name.svc.cluster.local:8008/v2/models/fraud/infer`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "db9df000-a171-4652-8160-272f81e49612",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install grpcio grpcio-tools"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1d17b252-7827-4cae-adb0-f98c9d80bcd7",
   "metadata": {},
   "outputs": [],
   "source": [
    "grpc_host = 'modelmesh-serving'\n",
    "grpc_port = 8033\n",
    "model_name = 'fraud'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4269da9e-5683-4531-9a3f-a1cdad42e3af",
   "metadata": {},
   "source": [
    "## Inspect the gRPC Endpoint\n",
    "\n",
    "Check the gRPC endpoint's model metadata:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "545aa5f4-356f-4e70-b7e6-cd352a68927a",
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "sys.path.append('./utils')\n",
    "\n",
    "# grpc_predict_v2_pb2 and grpc_predict_v2_pb2_grpc were created from grpc_predict_v2.proto using protoc\n",
    "import grpc\n",
    "import utils.grpc_predict_v2_pb2 as grpc_predict_v2_pb2\n",
    "import utils.grpc_predict_v2_pb2_grpc as grpc_predict_v2_pb2_grpc\n",
    "\n",
    "\n",
    "channel = grpc.insecure_channel(f\"{grpc_host}:{grpc_port}\")\n",
    "stub = grpc_predict_v2_pb2_grpc.GRPCInferenceServiceStub(channel)\n",
    "\n",
    "request = grpc_predict_v2_pb2.ModelMetadataRequest(name=model_name)\n",
    "response = stub.ModelMetadata(request)\n",
    "response"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fd5affbf-36c3-4e17-9788-5fc0904de143",
   "metadata": {},
   "source": [
    "### Request Function\n",
    "\n",
    "Build and submit the gRPC request. \n",
    "\n",
    "Note: You submit the data in the same format that you used for an ONNX inference.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "67c1d001-ff99-414a-95d4-5729d5849298",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "def grpc_request(data):\n",
    "    # request content building\n",
    "    inputs = []\n",
    "    inputs.append(grpc_predict_v2_pb2.ModelInferRequest().InferInputTensor())\n",
    "    inputs[0].name = \"dense_input\"\n",
    "    inputs[0].datatype = \"FP32\"\n",
    "    inputs[0].shape.extend([1, 5])\n",
    "    inputs[0].contents.fp32_contents.extend(data)\n",
    "\n",
    "    # request building\n",
    "    request = grpc_predict_v2_pb2.ModelInferRequest()\n",
    "    request.model_name = model_name\n",
    "    request.inputs.extend(inputs)\n",
    "\n",
    "    response = stub.ModelInfer(request)\n",
    "    result_arr = np.frombuffer(response.raw_output_contents[0], dtype=np.float32)\n",
    "    return result_arr"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "911b1015-28b0-4d60-bc17-7b30326b97bc",
   "metadata": {},
   "source": [
    "### Run the Request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4fc549f6",
   "metadata": {},
   "outputs": [],
   "source": [
    "#Load the scaler\n",
    "import pickle\n",
    "with open('artifact/scaler.pkl', 'rb') as handle:\n",
    "    scaler = pickle.load(handle)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "12947866-e0f5-4c72-ba9a-04229b1af990",
   "metadata": {},
   "outputs": [],
   "source": [
    "data = [0.3111400080477545, 1.9459399775518593, 1.0, 0.0, 0.0]\n",
    "prediction = grpc_request(scaler.transform([data]).tolist()[0])\n",
    "prediction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "946f9f1d-b24a-4aa6-b839-f0e8013ef84d",
   "metadata": {},
   "outputs": [],
   "source": [
    "threshhold = 0.95\n",
    "\n",
    "if (prediction[0] > threshhold):\n",
    "    print('fraud')\n",
    "else:\n",
    "    print('not fraud')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d7f6b51",
   "metadata": {},
   "source": [
    "## Example 1: user buys a coffee\n",
    "\n",
    "In this example, the user is buying a coffee. The parameters given to the model are:\n",
    "* same location as the last transaction (distance=0)\n",
    "* same median price as the last transaction (ratio_to_median=1)\n",
    "* using a pin number (pin=1)\n",
    "* using the credit card chip (chip=1)\n",
    "* not an online transaction (online=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f0a68b67-b109-4a2f-b097-092f4a4d25ce",
   "metadata": {},
   "outputs": [],
   "source": [
    "data = [0.0, 1.0, 1.0, 1.0, 0.0]\n",
    "prediction = grpc_request(scaler.transform([data]).tolist()[0])\n",
    "threshhold = 0.95\n",
    "\n",
    "if (prediction[0] > threshhold):\n",
    "    print('The model predicts that this is fraud')\n",
    "else:\n",
    "    print('The model predicts that this is not fraud')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1dd27d88",
   "metadata": {},
   "source": [
    "## Example 2: fraudulent transaction\n",
    "\n",
    "In this example, someone stole the user's credit card and is buying something online. The parameters given to the model are:\n",
    "* very far away from the last transaction (distance=100)\n",
    "* median price similar to the last transaction (ratio_to_median=1.2)\n",
    "* not using a pin number (pin=0)\n",
    "* not using the credit card chip (chip=0)\n",
    "* is an online transaction (online=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7a736a21",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "data = [100, 1.2, 0.0, 0.0, 1.0]\n",
    "prediction = grpc_request(scaler.transform([data]).tolist()[0])\n",
    "threshhold = 0.95\n",
    "\n",
    "if (prediction[0] > threshhold):\n",
    "    print('The model predicts that this is fraud')\n",
    "else:\n",
    "    print('The model predicts that this is not fraud')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.9",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}