Merge pull request #45 from sjwaight/master

.gitignore

@@ -328,3 +328,5 @@ ASALocalRun/
 
 # MFractors (Xamarin productivity tool) working folder 
 .mfractor/
+
+.vscode/

CHANGELOG.md

@@ -1,13 +1 @@
-## [project-title] Changelog
 
-<a name="x.y.z"></a>
-# x.y.z (yyyy-mm-dd)
-
-*Features*
-* ...
-
-*Bug Fixes*
-* ...
-
-*Breaking Changes*
-* ...

README.md

@@ -1,64 +1,86 @@
----
-page_type: sample
-languages:
-- csharp
-- java
-- python
-- typescript
-- go
-- ruby
-products:
-- azure
-- azure-cognitive-services
-- azure-anomaly-detector
-description: "This repository contains samples for Anomaly Detector API. The Anomaly Detector API enables you to monitor and find abnormalities in your time series data by automatically identifying and applying the correct statistical models, regardless of industry, scenario, or data volume."
-urlFragment: AnomalyDetector
----
-
-# Anomaly Detector API Samples
-
-This repository contains samples for [Anomaly Detector API](https://aka.ms/anomalydetector). The Anomaly Detector API enables you to monitor and find abnormalities in your time series data by automatically identifying and applying the correct statistical models, regardless of industry, scenario, or data volume. Using your time series data, the API can find anomalies as a batch throughout your data, or determine if your latest data point is an anomaly.
-
-## Contents
-
-| File/folder | Description |
-|-------------|-------------|
-| `example-data`       | Example data to be sent to the API, along with example API responses.  |
-| `ipython-notebook` | Jupyter notebooks. |
-| `quickstarts`       | Sample code for the Anomaly Detector API quickstarts.  |
+# Anomaly Detector Samples
+
+This repository contains API samples and SDK samples for [Anomaly Detector service](https://aka.ms/anomalydetector). Anomaly Detector enables you to monitor and find abnormalities in your time series data by automatically identifying and applying the correct statistical models, regardless of industry, scenario, or data volume.
+
+## What's new?
+
+### March 2024: Anomaly Detector Now Available on PYPI 🎉
+
+In March 2024, we proudly announce the release of the Anomaly Detector package on PYPI! 
+
+While the existing Anomaly Detector as a service will be deprecated by 2026, you can now seamlessly utilize the new package directly on your local machine. No need to create an Azure Anomaly Detector resource—simply install the package and start detecting anomalies right away.
+
+For the latest details and usage instructions, refer to our Python notebook available here: [anomaly-detector-pypi-demo.ipynb](anomaly-detector-pypi-demo.ipynb)
+
+## 👋About Anomaly Detector
+[Anomaly Detector](https://learn.microsoft.com/en-us/azure/cognitive-services/anomaly-detector/overview) is an AI service with a set of APIs, which enables you to monitor and detect anomalies in your time series data with little ML knowledge, either batch validation or real-time inference.
+
+[Univariate Anomaly Detection API](https://learn.microsoft.com/en-us/azure/cognitive-services/anomaly-detector/how-to/identify-anomalies) enables you to monitor and detect abnormalities in your single variable without having to know machine learning. The Anomaly Detector API's algorithms adapt by automatically identifying and applying the best-fitting models to your data, regardless of industry, scenario, or data volume. Using your time series data, the API determines boundaries for anomaly detection, expected values, and which data points are anomalies.
+
+[Multivariate anomaly detection API](https://learn.microsoft.com/en-us/azure/cognitive-services/anomaly-detector/how-to/create-resource) further enable developers by easily integrating advanced AI for detecting anomalies from groups of metrics, without the need for machine learning knowledge or labeled data. Dependencies and inter-correlations between up to 300 different signals are now automatically counted as key factors. This new capability helps you to proactively protect your complex systems such as software applications, servers, factory machines, spacecraft, or even your business, from failures.
 
 ## Prerequisites
 
-You must have an [Anomaly Detector API resource](https://aka.ms/adnew). Before continuing, you will need the API key and the endpoint from your Azure dashboard.
+You must have an [Anomaly Detector API resource](https://aka.ms/adnew). Before continuing, you will need the API **key** and the **endpoint** from your Azure dashboard.
    ![Get Anomaly Detector access keys](./media/cognitive-services-get-access-keys.png "Get Anomaly Detector access keys")
 
 Or you could create a 7-day free resource of Anomaly Detector from [here](https://azure.microsoft.com/en-us/try/cognitive-services/my-apis/).
 
-If you want to run the notebook with an on-premise version of [Anomaly Detector as container](https://aka.ms/adcontainerdocs), there're four prerequisites that must be met:
+## Content
 
-1. You have access to the Azure Container Registry which hosts the Anomaly Detector container images. Please complete and submit the [Anomaly Detector Container Request form](https://aka.ms/adcontainer) to request access to the container.
-1. You have created an Anomaly Detector resource on Azure.
-1. You have the proper container environment ready to host the Anomaly Detector container. Please read [Prerequisites](https://docs.microsoft.com/en-us/azure/cognitive-services/anomaly-detector/anomaly-detector-container-howto#prerequisites) and [The host computer](https://docs.microsoft.com/en-us/azure/cognitive-services/anomaly-detector/anomaly-detector-container-howto#the-host-computer) for details.
-1. You have [Jupyter Notebook](https://jupyter.org/install.html) installed on your computer. We recommend installing Python and Jupyter using the [Anaconda Distribution](https://www.anaconda.com/downloads).
+This repository is organized in the following structure, we recommend you go to `demo-notebook` first to try the simple samples if you are a fan of Python. 🤗
 
-After you pull the container image and spin it up, ensure there's an HTTP endpoint accessible to the APIs and this will be your **endpoint** for the demo.
+| Folder | Description |
+|-------------|-------------|
+| 🆕[ipython-notebook](/ipython-notebook/)      | [API](/ipython-notebook/API%20Sample/) and [SDK](/ipython-notebook/SDK%20Sample/) sample codes written in python notebook for UVAD adn MVAD. The latest update will start from here first. 😉 |
+| [sampledata](/sampledata/) | All the sample datasets that are used in this repository. |
+| [sample-multivariate](/samples-multivariate/)       | Sample SDK codes for MVAD(preview version) using 4 languages.  |
+| [sample-univariate](/samples-univariate/)       | Sample API and SDK codes for UVAD using 4 languages. |
+| [univariate-live-demo](/univariate-live-demo/)| This includes a live demo that you could clone directly and ran on your data or make any modifications.  |
+| [postman-demo](/postman-demo/)  |  This includes the tutorial of using postman to trigger Anomaly Detector, which could help better understand from API perspective.  |
 
-## Online demo
+## 🔗Important links
 
-To quickly start using the Anomaly Detector API, try an [online demo](https://notebooks.azure.com/AzureAnomalyDetection/projects/anomalydetector). This demo runs in a web-hosted Jupyter notebook and shows you how to send an API request, and visualize the result.
+### 1️⃣Microsoft Learn - Anomaly Detector
 
-To run the demo, complete the following steps:
-
-1. Sign in, and click **Clone**, in the upper right corner.
-1. Uncheck the "public" option in the dialog box before completing the clone operation, otherwise your notebook, including any subscription keys, will be public.
-1. Click **Run on free compute**
-1. Open one of the notebooks, for example, Anomaly Finder API Example Private Preview (Batch Method).ipynb
-1. Fill in the API key and the endpoint from your Anomaly Detector resource on Azure
-1. In the Notebook main menu, click Cell->run all
+- Learning module: [Identify abnormal time-series data with Anomaly Detector](https://learn.microsoft.com/en-us/training/modules/identify-abnormal-time-series-data-anomaly-detector/?WT.mc_id=data-12171-ruyakubu)
+
+### 2️⃣API/SDK Sample
 
+- [Anomaly Detector Sample](https://github.com/Azure-Samples/AnomalyDetector)
+- [Anomaly Detector Sample in python notebook](https://github.com/Azure-Samples/AnomalyDetector/tree/master/ipython-notebook)
+
+### 3️⃣Anomaly Detector in Synapse
+
+- [Tutorial: Use Multivariate Anomaly Detector in Azure Synapse Analytics](https://learn.microsoft.com/en-us/azure/cognitive-services/anomaly-detector/tutorials/multivariate-anomaly-detection-synapse)
+- [Sample notebook for MVAD in Synapse](https://github.com/jr-MS/MVAD-in-Synapse)
+
+### 4️⃣Anomaly Detector in Azure Databricks
+
+- [Blog: Detect Anomalies in Equipment with Anomaly Detector in Azure Databricks](https://techcommunity.microsoft.com/t5/ai-cognitive-services-blog/detect-anomalies-in-equipment-with-anomaly-detector-in-azure/ba-p/3390688)
+
+### 5️⃣Anomaly Detector in Azure Data Explorer
+
+- [Blog: Announcing Univariate Anomaly Detector in Azure Data Explorer](https://techcommunity.microsoft.com/t5/ai-applied-ai-blog/announcing-univariate-anomaly-detector-in-azure-data-explorer/ba-p/3285400)
+- [Documentation about anomaly detection function in ADX](https://learn.microsoft.com/en-us/azure/data-explorer/kusto/functions-library/series-uv-anomalies-fl?tabs=adhoc)
+
+### 6️⃣Anomaly Detector PowerBI
+
+- [Anomaly Detection in PowerBI - UI](https://learn.microsoft.com/en-us/power-bi/visuals/power-bi-visualization-anomaly-detection)
+- [Anomaly Detection in PowerBI - PowerQuery](https://learn.microsoft.com/en-us/azure/cognitive-services/anomaly-detector/tutorials/batch-anomaly-detection-powerbi)
 
 ## Container demo
 
+(Only support UVAD)
+
+If you want to run the notebook with an on-premise UVAD version of [Anomaly Detector as container](https://aka.ms/adcontainerdocs), there're four prerequisites that must be met:
+
+1. You have access to the Azure Container Registry which hosts the Anomaly Detector container images. Please complete and submit the [Anomaly Detector Container Request form](https://aka.ms/adcontainer) to request access to the container.
+1. You have created an Anomaly Detector resource on Azure.
+1. You have the proper container environment ready to host the Anomaly Detector container. Please read [Prerequisites](https://docs.microsoft.com/en-us/azure/cognitive-services/anomaly-detector/anomaly-detector-container-howto#prerequisites) and [The host computer](https://docs.microsoft.com/en-us/azure/cognitive-services/anomaly-detector/anomaly-detector-container-howto#the-host-computer) for details.
+1. You have [Jupyter Notebook](https://jupyter.org/install.html) installed on your computer. We recommend installing Python and Jupyter using the [Anaconda Distribution](https://www.anaconda.com/downloads).
+
+After you pull the container image and spin it up, ensure there's an HTTP endpoint accessible to the APIs and this will be your **endpoint** for the demo.
 To run the notebook with your Anomaly Detector container instance, complete the following steps:
 
 1. Clone this project to your local directory
@@ -69,11 +91,6 @@ To run the notebook with your Anomaly Detector container instance, complete the
 1. Fill in the API key (from your Anomaly Detector resource on Azure) and the endpoint (from your Anomaly Detector container instance)
 1. In the Notebook main menu, click Cell->run all
 
-## Data requirements
-
-Example data is provided in this repository, along with example JSON responses from the API. To use the Anomaly Detector API on your time series data, ensure the following:
 
-* Your data points are separated by the same interval, with no missing points.
-* Your data has at least 13 data points if it doesn't have clear perodicity.
-* Your data has at least 4 periods if it does have clear perodicity.
-Please read [Best practices for using the Anomaly Detector API](https://aka.ms/adbest) for details.
+## ❤️Support
+Need support? [Join the Anomaly Detector Community](https://forms.office.com/pages/responsepage.aspx?id=v4j5cvGGr0GRqy180BHbR2Ci-wb6-iNDoBoNxrnEk9VURjNXUU1VREpOT0U1UEdURkc0OVRLSkZBNC4u).

anomaly-detector-pypi-demo.ipynb

@@ -0,0 +1,187 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "a2f4ea7a79dcc5e",
+   "metadata": {
+    "collapsed": false
+   },
+   "source": [
+    "## Install package"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "a0f996e3f768461f",
+   "metadata": {
+    "collapsed": false,
+    "vscode": {
+     "languageId": "powershell"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "pip install time-series-anomaly-detector"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "8346a504",
+   "metadata": {},
+   "source": [
+    "## Multivariate Anomaly Detection"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "4f25e1b2",
+   "metadata": {},
+   "source": [
+    "### 1. Train model\n",
+    "To train a mvad model, the type of training data should be <code>DataFrame</code> type. And you must specify the <code>sliding_window</code> and <code>device</code> in <code>params</code>. \n",
+    "\n",
+    "Moreover, please note that in mvad, <code>timestamp</code> of the training data is optional."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "d5da3531",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import numpy as np\n",
+    "import pandas as pd\n",
+    "from anomaly_detector import MultivariateAnomalyDetector\n",
+    "\n",
+    "import json\n",
+    "from pprint import pprint\n",
+    "\n",
+    "data_size = 1000\n",
+    "var_num = 20\n",
+    "\n",
+    "training_data = np.random.randn(data_size, var_num)\n",
+    "columns = [f\"variable_{i}\" for i in range(var_num)]\n",
+    "training_data = pd.DataFrame(training_data, columns=columns)\n",
+    "\n",
+    "# Optional\n",
+    "timestamps = pd.date_range(start=\"2023-01-03\", periods=data_size, freq=\"H\")\n",
+    "training_data[\"timestamp\"] = timestamps.strftime(\"%Y-%m-%dT%H:%M:%SZ\")\n",
+    "training_data = training_data.set_index(\"timestamp\", drop=True)\n",
+    "\n",
+    "params = {\"sliding_window\": 200, \"device\": \"cpu\"}\n",
+    "\n",
+    "model = MultivariateAnomalyDetector()\n",
+    "\n",
+    "# Train model\n",
+    "model.fit(training_data, params=params)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b918d943",
+   "metadata": {},
+   "source": [
+    "### 2. Inference"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "f3a010d7",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "eval_data = np.random.randn(201, var_num)\n",
+    "eval_data[-1, :] += 100\n",
+    "eval_data = pd.DataFrame(eval_data, columns=columns)\n",
+    "\n",
+    "# Optional\n",
+    "timestamps = pd.date_range(start=\"2023-01-03\", periods=201, freq=\"H\")\n",
+    "eval_data[\"timestamp\"] = timestamps.strftime(\"%Y-%m-%dT%H:%M:%SZ\")\n",
+    "eval_data = eval_data.set_index(\"timestamp\", drop=True)\n",
+    "\n",
+    "# prediction\n",
+    "results = model.predict(data=eval_data, context=None)\n",
+    "\n",
+    "pprint(results)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1cc129ba",
+   "metadata": {},
+   "source": [
+    "## Univariate Anomaly Detection"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "5d12be4c",
+   "metadata": {},
+   "source": [
+    "### Inference\n",
+    "\n",
+    "Please note that the univariate anomaly detection does not need to train before inference, and <code>timestamp</code> of the <code>eval_data</code> must be specified."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "232963b5",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import numpy as np\n",
+    "import pandas as pd\n",
+    "from anomaly_detector import EntireAnomalyDetector\n",
+    "\n",
+    "params = {\n",
+    "    \"granularity\": \"monthly\", \n",
+    "    \"maxAnomalyRatio\": 0.25, \n",
+    "    \"sensitivity\": 95, \n",
+    "    \"imputeMode\": \"auto\"\n",
+    "}\n",
+    "\n",
+    "\n",
+    "model = EntireAnomalyDetector()\n",
+    "\n",
+    "eval_data = np.ones(20)\n",
+    "eval_data[-1] = 0\n",
+    "eval_data = pd.DataFrame(eval_data, columns=[\"value\"])\n",
+    "\n",
+    "timestamps = pd.date_range(start=\"1962-01-01\", periods=20, freq=\"ME\")\n",
+    "eval_data[\"timestamp\"] = timestamps\n",
+    "\n",
+    "results = model.predict(\n",
+    "    data=eval_data,\n",
+    "    params=params,\n",
+    "    context=None\n",
+    ")\n",
+    "print(results)\n"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.13"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}