notebooks/001_create_project.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "e23e35ab-d1e1-44c7-8e3c-a1a35992d45b",
   "metadata": {},
   "source": [
    "# Creating a project\n",
    "In this notebook, we will use the `geti-sdk` package to create a project on the platform, and show examples of how to interact with it\n",
    "\n",
    "### Setting up the connection to the platform\n",
    "First, we set up the connection to the server. This is done by instantiating a Geti instance, with the hostname (or ip address) and authentication details for the server. The server details are stored in the `.env` file (see [the notebooks readme](README.md) for how to create one) and are loaded in the cell below."
   ]
  },
  {
   "cell_type": "code",
   "id": "3cb49dd2-7032-40e0-a4c2-8203ba1072bf",
   "metadata": {},
   "source": [
    "from geti_sdk.utils import get_server_details_from_env\n",
    "\n",
    "geti_server_configuration = get_server_details_from_env()"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "28fd038b-a7f6-4163-8fd9-ecf08cfb22ef",
   "metadata": {},
   "source": [
    "Now that the server details are loaded we can connect to the server."
   ]
  },
  {
   "cell_type": "code",
   "id": "ba8318f4-3d97-4949-abf0-6cdafe46572e",
   "metadata": {},
   "source": [
    "from geti_sdk import Geti\n",
    "\n",
    "geti = Geti(server_config=geti_server_configuration)"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "2f15d530-3e2b-42f3-aa21-6e31e57627cf",
   "metadata": {},
   "source": [
    "### Listing the projects currently in the workspace\n",
    "To create or view projects on the platform, we have to set up a ProjectClient using the Geti instance we just created. Once the ProjectClient is set up, we can use it to print a list of projects that our workspace currently holds."
   ]
  },
  {
   "cell_type": "code",
   "id": "081e700d-9e2e-4022-b4d6-dac3adf1d4f8",
   "metadata": {},
   "source": [
    "from geti_sdk.rest_clients import ProjectClient\n",
    "\n",
    "project_client = ProjectClient(session=geti.session, workspace_id=geti.workspace_id)\n",
    "\n",
    "projects = project_client.list_projects()"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "967575df-19be-4394-a8d6-23bd0d4fbdfd",
   "metadata": {},
   "source": [
    "### Project creation parameters\n",
    "To create a new project, we have to specify three things:\n",
    "1. The name of the project\n",
    "2. The type of the project\n",
    "3. The label names or properties for each task in the project\n",
    "\n",
    "##### Project name\n",
    "The project name is easy, this can be any string you like, as long as there is no existing project with that name yet.\n",
    "\n",
    "##### Project type\n",
    "The project type requires more explanation, since this controls the tasks that will be added to the project's pipeline. \n",
    "- To create a single task project, simply pass the type of the task you want as the `project_type`. For example, to create a project with a single classification task, pass `project_type=\"classification\"`\n",
    "- In addition, we can construct arbitrary pipelines by passing a string that conforms to the format `{type_of_task_1}_to_{type_of_task_2}`. So for example, to create a project that holds a detection task followed by a segmentation task, we would pass: `project_type=\"detection_to_segmentation\"`. To understand which task types are supported, we can have a look at the TaskType class from `geti_sdk`."
   ]
  },
  {
   "cell_type": "code",
   "id": "30e46136-5283-4e3d-8dcf-7a89bee4de6b",
   "metadata": {},
   "source": [
    "from geti_sdk.data_models.enums import TaskType\n",
    "\n",
    "# Prints a list of the supported 'trainable' task types\n",
    "print(\"Supported task types:\")\n",
    "for task_type in TaskType:\n",
    "    if task_type.is_trainable:\n",
    "        print(\"  \" + str(task_type))"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "05f75187-2acb-482f-b86c-f3cbbabd180d",
   "metadata": {
    "tags": []
   },
   "source": [
    "##### Labels\n",
    "The `labels` parameter takes a nested list of label names, or a nested list of dictionaries representing label properties. For example, suppose we want to make a single task classification project with labels 'person', 'dog' and 'car'. In that case we should pass `labels=[['person', 'dog', 'car']]`. \n",
    "\n",
    "The list is nested because each entry in the outermost list corresponds to the labels for one of the tasks in the pipeline. For example, suppose we want to create a pipeline project of type `detection_to_classification`, with a label 'animal' for the detection task and labels 'dog', 'cat', 'horse', 'cow' for the classification task. In that case we would pass `labels=[['animal'], ['dog', 'cat', 'horse', 'cow']]`. The first entry in the labels-list corresponds to the labels for the first task, the second entry to those for the second task in the pipeline and so on.\n",
    "\n",
    "In case more complicated relationships between labels are required, we can specify the labels as dictionaries with certain properties instead of simple strings containing only their names. For example, to create a single task hierarchical classification project to classify 'animals' and 'vehicles' into subcategories, we could pass the following: \n",
    "```json\n",
    "labels = [\n",
    "    [\n",
    "        {\"name\": \"animal\"}, \n",
    "        {\"name\": \"dog\", \"parent_id\": \"animal\"}, \n",
    "        {\"name\": \"cat\", \"parent_id\": \"animal\"}, \n",
    "        {\"name\": \"vehicle\"}, \n",
    "        {\"name\": \"car\", \"parent_id\": \"vehicle\"}, \n",
    "        {\"name\": \"taxi\", \"parent_id\": \"vehicle\"}, \n",
    "        {\"name\": \"truck\", \"parent_id\": \"vehicle\"}\n",
    "    ]\n",
    "]\n",
    "``` \n",
    "It is also possible to make a multi-label classification task (meaning multiple labels can be assigned to a single image) by using the \"group\" keyword in the label property dictionary. Labels in different groups will be treated as independent (i.e. non-exclusive) from each other. \n",
    "\n",
    "\n",
    "## Creating a simple project\n",
    "Now that we understand the parameters, let's go ahead and create a new project:"
   ]
  },
  {
   "cell_type": "code",
   "id": "b18c25d1-f53b-495e-a48e-ee89b7f95d4e",
   "metadata": {},
   "source": [
    "# First set the project parameters. Feel free to experiment here!\n",
    "PROJECT_NAME = \"Segmentation demo\"\n",
    "PROJECT_TYPE = \"segmentation\"\n",
    "LABELS = [[\"dog\", \"cat\", \"horse\"]]"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "code",
   "id": "4941f712-f97b-48ff-a11b-74687d0bb49f",
   "metadata": {},
   "source": [
    "# Now, use the project client to create the project\n",
    "project = project_client.create_project(\n",
    "    project_name=PROJECT_NAME, project_type=PROJECT_TYPE, labels=LABELS\n",
    ")"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "1e8eace9-8cc2-4b8b-83a9-1376a7cd5666",
   "metadata": {},
   "source": [
    "## Interacting with the project\n",
    "The `Project` object that is returned by the `project_client.create_project()` method contains a representation of the project on the platform. There are several ways to interact with it. First of all, we can get a very brief `summary` of the project"
   ]
  },
  {
   "cell_type": "code",
   "id": "c9d3496b-663d-41d3-b381-5e05b4c8e5b0",
   "metadata": {},
   "source": [
    "print(project.summary)"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "8b3b7b47-0171-41eb-97c1-c1a53b9a62c1",
   "metadata": {},
   "source": [
    "If we need to know more details, we can also look at the project `overview`"
   ]
  },
  {
   "cell_type": "code",
   "id": "8786e8df-3ecb-4947-ad71-c8512e0b22ac",
   "metadata": {},
   "source": [
    "print(project.overview)"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "50bf6745-c83f-4967-9dec-c4d8a6f56e38",
   "metadata": {},
   "source": [
    "Finally, the `project` object also supports several methods to quickly access some of its properties, for example to get a list of all trainable tasks in the project simply use `project.get_trainable_tasks()`. This will prove useful later on when we want to do more complicated things on the platform, such as triggering or monitoring a training job."
   ]
  },
  {
   "cell_type": "code",
   "id": "01cb2ee6-9d45-4d17-aa6f-2ac0330ce369",
   "metadata": {},
   "source": [
    "task_list = project.get_trainable_tasks()\n",
    "print(f\"Project '{project.name}' contains {len(task_list)} trainable tasks.\")\n",
    "for task in task_list:\n",
    "    print(task.summary)"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "aa289ae0-36bb-40db-afb3-d1c89fb2a9e1",
   "metadata": {},
   "source": [
    "The `project` object that was created by the `project_client.create_project()` method can also be retrieved by calling `project_client.get_project()`. This is useful if you do not want to create a new project, but would like to interact with an existing project instead"
   ]
  },
  {
   "cell_type": "code",
   "id": "71dbcca8-b79e-441d-9143-091c2b08ba35",
   "metadata": {},
   "source": [
    "project = project_client.get_project(project_name=PROJECT_NAME)\n",
    "print(project.summary)"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "9b4a3486-929d-4ffe-8123-cbc695260894",
   "metadata": {},
   "source": [
    "## Creating a more complex project\n",
    "Now that we created a simple project, let's try our hand at something more complex. In the cell below, we will set the parameters to create a detection -> classification project, with hierarchical labels for the classification task. The detection task will have the label `vehicle`, while the classification task will contain hierarchical labels to narrow down the precise vehicle category. "
   ]
  },
  {
   "cell_type": "code",
   "id": "9c20a99c-07b6-4cc2-87fa-abf0ba210031",
   "metadata": {},
   "source": [
    "PIPELINE_PROJECT_NAME = \"Detection to hierarchical classification demo\"\n",
    "PIPELINE_PROJECT_TYPE = \"detection_to_classification\"\n",
    "PIPELINE_LABELS = [\n",
    "    [\"vehicle\"],\n",
    "    [\n",
    "        \"car\",\n",
    "        {\"name\": \"taxi\", \"parent_id\": \"car\", \"group\": \"car\"},\n",
    "        {\"name\": \"pick-up\", \"parent_id\": \"car\", \"group\": \"car\"},\n",
    "        {\"name\": \"sports car\", \"parent_id\": \"car\", \"group\": \"car\"},\n",
    "        \"truck\",\n",
    "        \"bus\",\n",
    "        {\"name\": \"van\", \"parent_id\": \"bus\", \"group\": \"bus\"},\n",
    "        {\"name\": \"school bus\", \"parent_id\": \"bus\", \"group\": \"bus\"},\n",
    "    ],\n",
    "]"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "code",
   "id": "01d01757-035c-4690-86f0-2f40487a6c52",
   "metadata": {},
   "source": [
    "pipeline_project = project_client.create_project(\n",
    "    project_name=PIPELINE_PROJECT_NAME,\n",
    "    project_type=PIPELINE_PROJECT_TYPE,\n",
    "    labels=PIPELINE_LABELS,\n",
    ")"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "78d77759-9003-4359-91e5-ba2898897c3c",
   "metadata": {},
   "source": [
    "Let's look at the project summary again:"
   ]
  },
  {
   "cell_type": "code",
   "id": "71a5b21f-edf6-4be4-8e86-08db768040f2",
   "metadata": {},
   "source": [
    "print(pipeline_project.summary)"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "7583c3e6-25c3-4517-89dd-87454d1c7fb6",
   "metadata": {},
   "source": [
    "Note that the project summary does not include the information regarding label hierarchies. If we want to be sure that the hierarchical label structure has been set up correctly, we can use `project.overview` for a more detailed view of the project:"
   ]
  },
  {
   "cell_type": "code",
   "id": "d23f29b8-056d-476a-a244-5f7355e00743",
   "metadata": {},
   "source": [
    "print(pipeline_project.overview)"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "2177233c-285f-400e-9f85-1e2db94563b6",
   "metadata": {},
   "source": [
    "If you look carefully at the overview, you will note that the top-level classification labels (`car`, `truck` and `bus`) have been assigned the parent `vehicle`, from the detection task preceding the classification task. Furthermore, the `car` and `bus` classes have their subcategories assigned as we specified. \n",
    "\n",
    "Of course, you can also check the project in the UI for a more visual representation.\n",
    "\n",
    "## Cleaning up\n",
    "To clean up the workspace, let's delete the projects that we just created. The project client provides a method for this `project_client.delete_project`. You can pass either the name of the project or the `Project` object to it. \n",
    "\n",
    "To delete the project, uncomment the line that states `project_client.delete_project(project)` and run the code cell below."
   ]
  },
  {
   "cell_type": "code",
   "id": "e2a2548a-8f84-4395-bdf7-8a4ebf3447a3",
   "metadata": {},
   "source": [
    "# Delete the simple project\n",
    "\n",
    "# project_client.delete_project(project)"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "c833113a-5364-40f6-b77c-42c915cb95e7",
   "metadata": {},
   "source": [
    "As you have probably noticed, the method will ask for confirmation before deleting the project. The reason for this is that deleting a project is not something to do lightly, it will remove all media, annotations and models contained in the project from the platform. Deleting a project is irreversible.\n",
    "\n",
    "However, it is possible to skip the confirmation if you are sure you know what you are doing. The code cell below shows how to delete a project immediately, without any user input."
   ]
  },
  {
   "cell_type": "code",
   "id": "ab209bce-5195-4f40-9d40-032af0bb5031",
   "metadata": {},
   "source": [
    "# Delete the pipeline project\n",
    "\n",
    "# project_client.delete_project(pipeline_project, requires_confirmation=False)"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "code",
   "id": "fa885dd0-b669-4c83-b6b2-bbde010144cd",
   "metadata": {},
   "source": [],
   "outputs": [],
   "execution_count": null
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}