notebooks/009_download_and_upload_project.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "38c51af4-a45c-4c37-b67c-1e15156e9431",
   "metadata": {},
   "source": [
    "# Downloading and uploading a project\n",
    "In this notebook, we will see how to download all available data for a project (media, annotations, predictions, models and the configuration) to our local machine. Once the project data is downloaded, we have effectively made a back-up of our project from the platform to the local disk. \n",
    "\n",
    "At the end of the notebook, we will use the downloaded data to re-create the project on the platform again, and upload all media and annotations to it. This could for example be used to migrate a project to a different server, or to make sure the project data is safe if a server outage is expected for whatever reason."
   ]
  },
  {
   "cell_type": "code",
   "id": "f2b19601-3208-47c8-af74-2527daf33044",
   "metadata": {},
   "source": [
    "# As usual we will connect to the platform first, using the server details from the .env file. We will also create a ProjectClient for the server\n",
    "\n",
    "from geti_sdk import Geti\n",
    "from geti_sdk.rest_clients import ProjectClient\n",
    "from geti_sdk.utils import get_server_details_from_env\n",
    "\n",
    "geti_server_configuration = get_server_details_from_env()\n",
    "\n",
    "geti = Geti(server_config=geti_server_configuration)\n",
    "\n",
    "project_client = ProjectClient(session=geti.session, workspace_id=geti.workspace_id)"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "6bd5581a-cef0-4dc7-9d5b-005d1d8e0f81",
   "metadata": {},
   "source": [
    "### Select project to download from the project list\n",
    "As usual, let's list the available projects and pick one to download. You can change the selected project by changing the `PROJECT_NAME` variable, two cells down."
   ]
  },
  {
   "cell_type": "code",
   "id": "0d6b6c22-4281-4a49-9ecc-8f93903a44e8",
   "metadata": {},
   "source": [
    "projects = project_client.list_projects()"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "9992c10b-d15e-49d0-91ac-2f1c888967d2",
   "metadata": {},
   "source": [
    "Specify the name of the project to download. We will use the `COCO multitask animal demo` project created in notebook [004](004_create_pipeline_project_from_dataset.ipynb)."
   ]
  },
  {
   "cell_type": "code",
   "id": "60de5767-d39f-4ac0-91b0-ff1645e7b0b2",
   "metadata": {},
   "source": [
    "PROJECT_NAME = \"COCO multitask animal demo\""
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "d2f31709-77e7-46d0-b859-0c4acf8a1edc",
   "metadata": {},
   "source": [
    "## Project download\n",
    "Now, let's do the project download itself. The `Geti` provides a method `download_project_data()` to do so. It takes the following arguments:\n",
    "\n",
    "- `project`: The project to download\n",
    "- `target_folder`: Path of the folder to download to. If left empty, a folder named `project_name` will be created in the current directory\n",
    "- `include_predictions`: True to download predictions for all media, False to not download any predictions\n",
    "- `include_active_models`: True to download the active models for all tasks in the project, False to not download any models\n",
    "- `include_deployment`: True to create and download a deployment for the project, False to not create any deployment\n",
    "\n",
    "Note that downloading may take some time especially for large projects, so please be patient. The method will print regular updates related to the download progress."
   ]
  },
  {
   "cell_type": "code",
   "id": "da106bf0-41df-40c5-a8b3-60b377397393",
   "metadata": {},
   "source": [
    "import os\n",
    "\n",
    "project = project_client.get_project_by_name(project_name=PROJECT_NAME)\n",
    "geti.download_project_data(\n",
    "    project=project,\n",
    "    target_folder=os.path.join(\"projects\", PROJECT_NAME),\n",
    "    include_predictions=False,\n",
    "    include_active_models=False,\n",
    "    include_deployment=False,\n",
    ")"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "f6e3107e-9ce1-4513-b111-c59d60be2b18",
   "metadata": {},
   "source": [
    "That's all there is to it! Now, you should have a folder `projects` showing up in the current directory. Inside it should be another folder named `{PROJECT_NAME}`, which should contain all a file `project.json` holding the project details, as well as all media and annotations in the project and a file `configuration.json` that contains the full project configuration. \n",
    "\n",
    "In addition, the `download_project_data` method can also create a deployment for the project (see notebook [008 deploy_project](008_deploy_project.ipynb) for more details on deployments), if the parameter `include_deployment` is set to True. In that case you should see a folder called `deployment` in the project directory as well.\n",
    "\n",
    "Note that in this case predictions, models and the deployment are not included because we have set `include_predictions=False`, `include_active_models=False` and `include_deployment=False`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "19e39608-ea91-48ff-be98-beac2a1c8bd7",
   "metadata": {},
   "source": [
    "## Project upload\n",
    "Now that the project is downloaded, we can use it to create a new project on the platform. Once the project is created, we can upload the media, annotations and configuration that were downloaded to the project folder to it. The `Geti` class provides a `upload_project_data()` method to do all that, which takes three parameters:\n",
    "- `target_folder`: Path to the folder containing the project data to upload\n",
    "- `project_name`: Optional name to assign to the new project on the platform. If left unspecified, the project name will correspond to the downloaded project name\n",
    "- `enable_auto_train`: True to turn on auto training for all tasks in the project, once the media and annotation upload is complete. False to leave auto training turned off."
   ]
  },
  {
   "cell_type": "code",
   "id": "8cca36fb-690a-4955-b99d-0bdd5f46f691",
   "metadata": {},
   "source": [
    "uploaded_project = geti.upload_project_data(\n",
    "    target_folder=os.path.join(\"projects\", PROJECT_NAME),\n",
    "    project_name=PROJECT_NAME,\n",
    "    enable_auto_train=False,\n",
    ")"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "632426d1-3e42-4571-a9ea-26eedf2a81c3",
   "metadata": {},
   "source": [
    "Done! The uploaded project should now show up in your workspace. Note that it is of course also possible to upload the project to a different server, simply by setting up a new Geti instance to that server and calling the `upload_project_data()` method from that instance."
   ]
  },
  {
   "cell_type": "code",
   "id": "e4474f3e-9aa7-4256-95de-62fbbba2d36f",
   "metadata": {},
   "source": [],
   "outputs": [],
   "execution_count": null
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}