notebooks/002_create_project_from_dataset.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "d3dfc49f-004a-4f73-89c0-19f64b48a3d2",
   "metadata": {},
   "source": [
    "# Creating a project from an existing dataset\n",
    "In this notebook, we will use the `geti-sdk` package to create a project from an existing dataset, and upload images and annotations to it."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b262f5c9-9ce6-415f-838d-aea36ddee487",
   "metadata": {},
   "source": [
    "### Setting up the connection to the platform\n",
    "First, we set up the connection to the server. This is done by instantiating a Geti object, with the hostname (or ip address) and authentication details for the server. As in notebook [001 create_project](001_create_project.ipynb), the server details are stored in the `.env` file and are loaded in the cell below. For details on how to create the `.env` file, please see the [readme](README.md)."
   ]
  },
  {
   "cell_type": "code",
   "id": "8269d231-cb5c-4591-b966-d0a00b9ef6f9",
   "metadata": {},
   "source": [
    "from geti_sdk.utils import get_server_details_from_env\n",
    "\n",
    "geti_server_configuration = get_server_details_from_env()"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "code",
   "id": "875581a5",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "source": [
    "from geti_sdk import Geti\n",
    "\n",
    "geti = Geti(server_config=geti_server_configuration)"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "b4c88de5-7719-424c-aede-83361646602a",
   "metadata": {},
   "source": [
    "## 1. Automated Project Creation\n",
    "The Intel Geti SDK package provides a method to create a project from an existing dataset. This method will create a project, upload the images and annotations to the project, and create the necessary labels and classes. This approach is useful when you have a dataset that is already annotated in one of the supported formats (COCO, Pascal VOC, YOLO, etc.).\n",
    "\n",
    "### Getting the COCO dataset\n",
    "In the next cell, we get the path to the MS COCO dataset. \n",
    "\n",
    "If you already have the COCO dataset on your machine, please specify the `dataset_path` to point to the folder containing the dataset. \n",
    "\n",
    "If you do not have the dataset yet, the `get_coco_dataset` method will make an attempt to download the dataset. Even though it will only download the 2017 validation subset, this is still a ~1 Gb download so it may take some time, depending on your internet connection. \n",
    "\n",
    "Of course the data will only be downloaded once; if you have downloaded the dataset previously, the method should detect it and return the path to the data."
   ]
  },
  {
   "cell_type": "code",
   "id": "59176324-9c88-4110-a4be-0c99e194980a",
   "metadata": {},
   "source": [
    "from geti_sdk.demos import get_coco_dataset\n",
    "\n",
    "COCO_PATH = get_coco_dataset(dataset_path=None)"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "e738920b-0079-4499-a03f-fde417cda88f",
   "metadata": {},
   "source": [
    "### Reading the dataset\n",
    "Next, we need to load the COCO dataset using Datumaro. The `geti-sdk` package provides the `DatumAnnotationReader` class to do so. It can read datasets in all formats supported by Datumaro."
   ]
  },
  {
   "cell_type": "code",
   "id": "35c81a6b-7145-4707-b725-dba432efecf2",
   "metadata": {},
   "source": [
    "from geti_sdk.annotation_readers import DatumAnnotationReader\n",
    "\n",
    "annotation_reader = DatumAnnotationReader(\n",
    "    base_data_folder=COCO_PATH, annotation_format=\"coco\"\n",
    ")"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "b45c4222-80f4-4b45-bfed-2bc1d00a9f7c",
   "metadata": {},
   "source": [
    "### Selecting the labels\n",
    "The MS COCO dataset contains 80 different classes, and while we could create a project including all of them, for this demo we will select only a couple of them. This is done using the `filter_dataset` method of the annotation reader."
   ]
  },
  {
   "cell_type": "code",
   "id": "4977c7bf-93e8-4ac9-b745-a7d8ac2859c5",
   "metadata": {},
   "source": [
    "annotation_reader.filter_dataset(labels=[\"dog\", \"cat\", \"horse\"], criterion=\"OR\")"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "03b33444-4667-4e72-9ce8-71bc5f6b848a",
   "metadata": {},
   "source": [
    "### Creating the project\n",
    "Now that we have a selection of data we would like to upload, we get to create the project. The COCO dataset is best suited for detection or segmentation type projects. \n",
    "\n",
    "To create the project, we will be using a method `create_single_task_project_from_dataset` from the `Geti` instance that we set up previously. This will not only create the project, but also upload the media and annotations from our dataset. \n",
    "\n",
    "The project name and type can be set via their respective input parameters, `project_name` and `project_type`. Have a look at notebook [001 create project](./001_create_project.ipynb) for further details about which values are supported for the `project_type` parameter.\n",
    "\n",
    "The number of images that is uploaded and annotated can be controlled as well. Finally, if `enable_auto_train` is set to `True` the project will start training right after all annotations have been uploaded (provided that sufficient images have been annotated to trigger auto-training)."
   ]
  },
  {
   "cell_type": "code",
   "id": "349abb0f-14e1-40a4-877c-636675b23411",
   "metadata": {},
   "source": [
    "PROJECT_NAME = \"COCO animal detection demo\"\n",
    "PROJECT_TYPE = \"detection\""
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "code",
   "id": "d5a26174-effe-4600-b0c9-a177ff8ff01a",
   "metadata": {},
   "source": [
    "project = geti.create_single_task_project_from_dataset(\n",
    "    project_name=PROJECT_NAME,\n",
    "    project_type=PROJECT_TYPE,\n",
    "    path_to_images=COCO_PATH,\n",
    "    annotation_reader=annotation_reader,\n",
    "    number_of_images_to_upload=100,\n",
    "    number_of_images_to_annotate=90,\n",
    "    enable_auto_train=True,\n",
    ")"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "44fb7dff-9e0b-4a3f-93df-569df8e453e8",
   "metadata": {},
   "source": [
    "That's it! A new project named `COCO animal detection demo` should now appear in your workspace. To check its properties, we can print a summary of it in the cell below."
   ]
  },
  {
   "cell_type": "code",
   "id": "7cd495e8-1c78-480f-9445-9082a4fa8429",
   "metadata": {},
   "source": [
    "print(project.summary)"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "56fc3b9d-0325-46c2-8912-906882c4a337",
   "metadata": {},
   "source": [
    "As you might have noticed, there is one additional label in the project, the `No Object` label. This is added by the system automatically to represent the absence of any 'horse', 'cat' or 'dog' in an image.\n",
    "\n",
    "## 2. Manual Project Creation\n",
    "If your dataset does not comply with one of the supported formats, there are several ways how to go around this.\n",
    "- You can try to convert your dataset to one of the supported formats and come back to the automated approach. This can be done by writing a script that will do the conversion. The drawback of this approach is that you can end up keeping multiple copies of the same dataset.\n",
    "- You can implement an [AnnotationReader](https://openvinotoolkit.github.io/geti-sdk/geti_sdk.annotation_readers.html#) of your own by following a few implementation examples already present in the Intel Geti SDK package - [DirectoryTreeAnnotationReader](https://github.com/openvinotoolkit/geti-sdk/blob/main/geti_sdk/annotation_readers/directory_tree_annotation_reader.py) and [DatumAnnotationReader](https://github.com/openvinotoolkit/geti-sdk/blob/main/geti_sdk/annotation_readers/datumaro_annotation_reader/datumaro_annotation_reader.py). It is especially useful if you have an established home-grown annotation format and the data you gather will be kept in this format in the future as well.\n",
    "- You can create a project manually and upload the data and annotations to it. This is the most straightforward approach, but it requires a bit more work with the Geti SDK entities.\n",
    "\n",
    "In this section we will go with the last approach and create a project manually. We will read the dataset annotations from a `csv` file and use the `geti-sdk` package to create a detection project, upload images and annotations to it.\\\n",
    "First, let's read a few lines from the dataset annotation file to see what it looks like."
   ]
  },
  {
   "cell_type": "code",
   "id": "5ff6658a",
   "metadata": {},
   "source": [
    "import csv\n",
    "\n",
    "ANNOTATION_FILE_PATH = \"./custom_dataset.csv\"\n",
    "annotation_file_contents = r\"\"\"image,xmin,ymin,xmax,ymax,label_name\n",
    "/images/val2017/000000001675.jpg,0,16,640,308,cat\n",
    "/images/val2017/000000004795.jpg,157,131,532,480,cat\"\"\"\n",
    "with open(ANNOTATION_FILE_PATH, \"w\") as csv_file:\n",
    "    csv_file.write(annotation_file_contents)\n",
    "\n",
    "with open(ANNOTATION_FILE_PATH, newline=\"\") as csv_file:\n",
    "    reader = csv.reader(csv_file)\n",
    "    header_line = next(reader)\n",
    "    first_data_line = next(reader)\n",
    "print(header_line)\n",
    "print(first_data_line)"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "3f4ae004",
   "metadata": {},
   "source": [
    "We see, that in our example the dataset annotation `csv` file contain six columns: `image` which is the sample path, `x_min`, `y_min`, `x_max`, `y_max` columns contain the bounding box coordinates, and the `label` column contains the object class label. The annotation file structure may vary and the processing code must be adjusted accordingly. It is also important to take into account all the known information about the dataset, such as the computer vision task(s) that the dataset is labeled for, number of classes and the number of images in the dataset to optimally process the it.\\\n",
    "As an example, you may not know the number of classes in the dataset, so you must find it out by reading the full annotation file to memory and extracting the unique values from the `label` column.\\\n",
    "In other cases, you may know the number of classes and their names, but the sample files are so big you would prefer to read and process the annotations line by line.\n",
    "\n",
    "To create a project we need to initialize a `ProjectClient` and call the `create_project` method, which is well explained in the previous notebook [001 create project](./001_create_project.ipynb). Our dataset is labeled for the `detection` so we will create a Project of the corresponding type. It will only have one trainable task, which is detection, so we will pass one list of labels to the `create_project` method. We will use our prior knowledge of the dataset - it was labeled for one-class detection so we only use one label."
   ]
  },
  {
   "cell_type": "code",
   "id": "e8f38ba6",
   "metadata": {},
   "source": [
    "from geti_sdk.rest_clients.project_client.project_client import ProjectClient\n",
    "\n",
    "project_client = ProjectClient(session=geti.session, workspace_id=geti.workspace_id)\n",
    "\n",
    "# Label names for the first (and only) trainable task in our Project.\n",
    "CLASS_NAMES = [\n",
    "    \"cat\",\n",
    "]\n",
    "\n",
    "project = project_client.create_project(\n",
    "    project_name=\"Manualy Created Detection Project\",\n",
    "    project_type=\"detection\",\n",
    "    labels=[\n",
    "        CLASS_NAMES,\n",
    "    ],\n",
    ")"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "8a61707b",
   "metadata": {},
   "source": [
    "We can examine the list of labels that are present in our newly created Project. The `get_all_labels` method of the ProjectClient returns a list of Geti SDK objects representing labels in the project. We will compile a dictionary that will help us mapping label names to the label objects later."
   ]
  },
  {
   "cell_type": "code",
   "id": "9866ef40",
   "metadata": {},
   "source": [
    "all_labels = project.get_all_labels()\n",
    "label_dict = {label.name: label for label in all_labels}\n",
    "print(all_labels)"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "c55fc1c5",
   "metadata": {},
   "source": [
    "To upload the images and annotations to the project, we will need an `ImageClient` and an `AnnotationClient` correspondingly."
   ]
  },
  {
   "cell_type": "code",
   "id": "ba00d7d0",
   "metadata": {},
   "source": [
    "from geti_sdk.rest_clients.annotation_clients.annotation_client import AnnotationClient\n",
    "from geti_sdk.rest_clients.media_client.image_client import ImageClient\n",
    "\n",
    "image_client = ImageClient(\n",
    "    session=geti.session, workspace_id=geti.workspace_id, project=project\n",
    ")\n",
    "annotation_client = AnnotationClient(\n",
    "    session=geti.session, workspace_id=geti.workspace_id, project=project\n",
    ")"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "8877c66e",
   "metadata": {},
   "source": [
    "Now we have everything to populate our project's dataset manualy. We will break the process into two steps for the first entry in the dataset:\n",
    "1. Upload the image to the project.\n",
    "2. Prepare and Upload the annotation to the project.\n",
    "\n",
    "The first part is straightforward, we will use the `upload_image` method of the `ImageClient` to upload the image to the project. The method can load an image from disk and send it to the server, it returns an `Image` object that we will use to upload the annotation in the next step."
   ]
  },
  {
   "cell_type": "code",
   "id": "88f92dc3",
   "metadata": {},
   "source": [
    "image_path = first_data_line[0]\n",
    "image_object = image_client.upload_image(image=COCO_PATH + image_path)\n",
    "image_object"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "8377f8a5",
   "metadata": {},
   "source": [
    "To upload the annotation we will use the `upload_annotation` method of the `AnnotationClient`. The method requires the `Image` object, and the `AnnotationScene` object, which we need to create from the annotation data. The `AnnotationScene` object is a container for the annotations of a single data sample, it consists of several `Annotation` instances each representing a single object in the image. The `Annotation` requires a bounding shape and a list of labels for that shape.\\\n",
    "Now let's code the same way bottom up."
   ]
  },
  {
   "cell_type": "code",
   "id": "c85bed24",
   "metadata": {},
   "source": [
    "from geti_sdk.data_models.annotation_scene import AnnotationScene\n",
    "from geti_sdk.data_models.annotations import Annotation\n",
    "from geti_sdk.data_models.shapes import Rectangle\n",
    "\n",
    "# From the CSV file entry we can get the coordinates of the rectangle\n",
    "x_min, y_min, x_max, y_max = first_data_line[1:5]\n",
    "\n",
    "# We need to create a Rectangle object to represent the shape of the annotation\n",
    "# Note: the Rectangle object requires the x, y, width and height of the rectangle,\n",
    "# so we need to calculate the width and height from the x_min, y_min, x_max and y_max\n",
    "rectangle = Rectangle(\n",
    "    x=int(x_min),\n",
    "    y=int(y_min),\n",
    "    width=int(x_max) - int(x_min),\n",
    "    height=int(y_max) - int(y_min),\n",
    ")\n",
    "\n",
    "# We can now create the Annotation object,\n",
    "# We can get a Label object from the label_dict we created earlier\n",
    "# using the label name from the CSV file entry as a key\n",
    "label = label_dict[first_data_line[5]]\n",
    "annotation = Annotation(\n",
    "    labels=[\n",
    "        label,\n",
    "    ],\n",
    "    shape=rectangle,\n",
    ")\n",
    "\n",
    "# We can now create the AnnotationScene object and upload the annotation\n",
    "annotation_scene = AnnotationScene(\n",
    "    [\n",
    "        annotation,\n",
    "    ]\n",
    ")\n",
    "annotation_client.upload_annotation(image_object, annotation_scene)"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "1a67b46c",
   "metadata": {},
   "source": [
    "Now we can gather all the steps in one method and iteratively apply it to the rest of the dataset.\n",
    "\n",
    "```python\n",
    "from typing import List\n",
    "\n",
    "def upload_and_annotate_image(dataset_line: List[str]) -> None:\n",
    "    \"\"\"\n",
    "    Uploads an image and its annotation to the project\n",
    "\n",
    "    :param dataset_line: The line from the dataset that contains the image path and annotation\n",
    "        in format ['image_path', 'xmin', 'ymin', 'xmax', 'ymax', 'label_name']\n",
    "    \"\"\"\n",
    "    image_path = dataset_line[0]\n",
    "    image_object = image_client.upload_image(image=image_path)\n",
    "\n",
    "    x_min, y_min, x_max, y_max = map(int, dataset_line[1:5])\n",
    "    rectangle = Rectangle(\n",
    "        x=x_min,\n",
    "        y=y_min,\n",
    "        width=x_max - x_min,\n",
    "        height=y_max - y_min,\n",
    "    )\n",
    "    annotation = Annotation(\n",
    "        labels=[label_dict[dataset_line[5]],],\n",
    "        shape=rectangle,\n",
    "    )\n",
    "    annotation_scene = AnnotationScene([annotation])\n",
    "    annotation_client.upload_annotation(image_object, annotation_scene)\n",
    "    print(f\"Uploaded and annotated {image_path}\")\n",
    "\n",
    "# We can now iterate over the rest of the lines in the CSV file and upload and annotate the images\n",
    "with open(ANNOTATION_FILE_PATH, newline='') as csv_file:\n",
    "    reader = csv.reader(csv_file)\n",
    "    header_line = next(reader)\n",
    "    for line in reader:\n",
    "        upload_and_annotate_image(line)\n",
    "```\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}