documentation/neanno-getting-started.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![neanno logo](images/neanno.png)\n",
    "\n",
    "# neanno\n",
    "**A tool for annotating texts and building annotation prediction models.**\n",
    "\n",
    "## Getting Started\n",
    "\n",
    "### What is neanno?\n",
    "\n",
    "neanno is a tool to add different types of annotations to texts in a dataset. Once you have the annotated texts, neanno has options to extract these annotations so you can build your own models with any code and framework you like. neanno can also build and train models for you (very rudimentary, to be improved).\n",
    "\n",
    "\n",
    "Currently, neanno supports the following annotation types:\n",
    "\n",
    "- **Categories**\n",
    "\n",
    "    A *category* is a class or type of a text. For instance, you might have a dataset which contains multiple support tickets and you could give each text one or more categories such as \"inquiry\", \"information\", \"complaint\", ... or - just as another example for categories - the team that should resolve the ticket. With neanno, you can give a text multiple categories.\n",
    "\n",
    "\n",
    "- **Named Entities**\n",
    "\n",
    "    A *named entity* is an arbitrary object of a certain type. For example, we might be interested in all airlines that appear in a text. Then the type of the named entities we are looking for would be \"AIRLINE\", and if our extraction model is good, it would even find airlines we have never heard about before, and it would be able to distinguish between \"American\" as an airline and \"American\" as in \"American Football\".\n",
    "    \n",
    "\n",
    "- **Key Terms**\n",
    "    \n",
    "    A *key term* is different from named entities in that they do not necessarily describe an object, and they do not have an entity type. They just denote an arbitrary term which is key to understanding a text. neanno assumes that whenever the text of a key term appears, this text is a key term, no matter where it stands.\n",
    "\n",
    "\n",
    "Named entities and key terms can either be *standalone* or *parented*. A **parented named entity** or a **parented key term** means that the named entity or key term has one or more other terms as \"parents\". These parents help to learn which terms mean actually the same thing. Parented terms are similar to synonyms but they can also lead to a more general representation of a term.\n",
    "\n",
    "For instance, you might have the text of a ticket where someone asks something about his flight from Sydney to Los Angeles. Some people write \"Los Angeles\", others write \"L.A.\" and more others write \"LAX\" but in the end it's all about Los Angeles. Hence, it would be good if an annotator can give us this information. So if we want to create some reports or word clouds on the texts, we can report on the single term \"Los Angeles\" although the original texts have different representations for \"Los Angeles\".\n",
    "\n",
    "But parent terms are more than just synonyms. Assume in your context it is not only of interest that someone flies to Los Angeles but it is also of interest that she flies to the U.S. An annotator could then simply add \"U.S.\" as parent term and then we can easily consider this in our reports too. neanno lets you add multiple parent terms if you need. Just separate them by a comma and neanno will automatically find out that you meant multiple parent terms. \n",
    "\n",
    "Goal of neanno is to make annotation as comfortable and efficient as possible. So we tried to avoid as much tedious mouse clicks as possible. Besides during annotation, neanno can support by suggesting certain annotations, either by a model it trains, a dataset you give or some regex pattern you specify.\n",
    "\n",
    "In future, neanno will even select the presented texts for annotation a little bit smarter than just sequentially iterating so you don't waste time, and it will create and export models for you (if you like).\n",
    "\n",
    "\n",
    "### Installation\n",
    "\n",
    "neanno is provided to you as a package written in Python - Python 3 to be more precise. It can run on Windows, Linux and (probably) Mac. (Mac will be tested soon but should work.)\n",
    "\n",
    "It might make sense to install neanno in a separate environment to ensure that there are no conflicts with other packages you have installed. (Trust me - trying out multiple things in only one single environment is not a good idea. Resolving dependency issues is extremely time-consuming...)\n",
    "\n",
    "To create a new environment, ensure that you have installed [Anaconda](https://www.anaconda.com/distribution) or [Miniconda](https://conda.io/en/latest/miniconda.html).\n",
    "\n",
    "Once you have Ananconda or Miniconda,\n",
    "1. open the Anaconda Prompt (only in Windows, in Linux/Mac Anaconda should be there by default after installation IIRC) and\n",
    "2. run `conda create -n <environment name such as: neanno> python=<3.x whereby greater than 3.6>`.\n",
    "3. Then activate your new environment by running\n",
    "    - `conda activate <your new environment name>` (Windows) or\n",
    "    - `source activate <your new environment name>` if on Linux/Mac, respectively.\n",
    "    \n",
    "  From now on, you always have to activate this environment with the command above before you want to use neanno.\n",
    "\n",
    "\n",
    "4. After you have activated your target environment, you are ready to install neanno. To install neanno, you can either\n",
    "    - install it from pip by simply running `pip install neanno` (once it's available there ;-)) or\n",
    "    - if you want to install neanno from its sources, go the root folder of neanno's sources and run `pip install -e .`\n",
    "\n",
    "neanno has a few dependencies on other packages. These packages should be installed automatically. If you want to see which packages neanno uses, have a look at the included requirements.txt file.\n",
    "\n",
    "Note: The packages that neanno uses are updated from time to time. If you encounter a weird error message, please ensure that you have installed the required packages in the right versions. To do so, run the `pip install -r requirements.txt` command in neanno's source folder with the respective Python environment activated.\n",
    "\n",
    "\n",
    "### Walkthrough - demo\n",
    "\n",
    "Once you installed neanno and are using it the first time, it might make sense to check out a sample first.\n",
    "\n",
    "The samples folder includes a demo named `airline_tickets`. The airline tickets demo has a dataset with helpdesk tickets and forum entries about travelling with airlines.\n",
    "\n",
    "#### Using the UI\n",
    "\n",
    "To start, go to neanno's source folder and run `python -m neanno --project samples/airline_tickets/airline_tickets.neanno.project.yaml`. This will open neanno and bring you directly to the middle of an annotation session which looks similar to this.\n",
    "\n",
    "![airline_tickets demo](images/airline_tickets.png)\n",
    "\n",
    "When you start neanno, it automatically determines the next text which has not been annotated yet and brings you directly to that text. As you can see, neanno uses different colors to highlight annotations. All named entities use the respective color shown on the right, key terms are marked with a grey background and a light blue font.\n",
    "\n",
    "- To assign or unassign the text a category, simply click the category in the list top right.\n",
    "- To annotate a key term, select the respective text and press `Alt+1` (shortcuts can be reconfigured by project file, see below).\n",
    "- To annotate a key term with parent terms, select the respective text, press `Alt+2` and write the parent term.\n",
    "- Annotating named entities is similar, just select the text and press the respective shortcut for the entity.\n",
    "- To give a named entity parents, add the `Shift` key while you press the shortcut for the respective entity.\n",
    "- To remove an inline annotation, just place the cursor within the span of the annotation, press `Ctrl+R` and your annotation is gone.\n",
    "- Once you are done with your annotations hit `Ctrl+Enter` (just like in Juypter). This will then submit your annotations and set the *is_annotated* flag, ie. neanno will not ask you to annotate that text again (although you can if needed).\n",
    "\n",
    "> Beware:\n",
    ">\n",
    "> - Any changes will be lost if you navigate to another text without submitting.\n",
    "> - neanno has no save button. Instead it saves the dataset whenever a text is submitted.\n",
    "\n",
    "neanno has a few more shortcuts/features you can use but I don't want to explain them all here. To see all available shortcuts, simply click on the `Shortcuts` button on bottom of the window, and it will open a dialog box that tells you about your options.\n",
    "\n",
    "#### Auto suggest\n",
    "\n",
    "Depending on what is configured, neanno will suggest annotations (see project file for more details). You can switch predictors on and off as you like. Just use the dialog behind the *Manage Predictors* button for that.\n",
    "\n",
    "As you might have already realized, neanno also learns during your annotations. There are however some predictors which need more time for training. To train these predictors, you currently need to explicitly trigger their training by clicking on the *Train Batch Models* button. It will then show you the output of the training process(es) in the console/terminal but don't be afraid, this experience will be better some day.\n",
    " (\n",
    "#### Distributions\n",
    "\n",
    "One last thing before I continue with the Python part. The numbers on the right show the distribution of categories or named entities, respectively. Knowing the distribution of your annotations is very valuable. It indicates when it's ok to stop annotating and lets you know when you have not enough annotations for a certain entity type yet.\n",
    "\n",
    "#### Using neanno from Python\n",
    "\n",
    "neanno provides a different functions to extract the annotations made in the UI to plain Python objects so you can use these to train whatever you like. There are also functions to report on the annotations contained in annotated texts, and it gives you the option to use the prediction capabilities used in the UI directly from Python.\n",
    "\n",
    "Since explaining all these things in this notebook would be too much, there is [another notebook](neanno-from-python.ipynb) which goes into more details."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Walkthrough II - now again but from scratch\n",
    "\n",
    "Ok - so you have seen the airline_tickets demo, got some basic understanding and now want to annotate your own text? No problem. Here we go!\n",
    "\n",
    "1. First, you need to prepare your dataset. neanno supports currently only CSV files (other data source types may come later, at least the code is well prepared for that). So no matter how you do it - in the end you currently need a CSV file which has a column that contains the texts to annotate.\n",
    "\n",
    "    If it's more convenient for you, just leave the other columns in the file. neanno will not remove or change them. BUT depending on how you configure neanno - see next point and project file - it might add additional columns which are needed.\n",
    "\n",
    "\n",
    "2. Once you have the dataset, it's time to create/adjust the project file. Because the structure of the project file has become quite large meanwhile, the best approach here is to take the project file from the airline_tickets sample and adjust it to your needs. To do that, copy the `airline_tickets.neanno.project.yaml` in a folder of your choice, rename it and open a text editor to edit it.\n",
    "\n",
    "    If you do not want to annotate eg. categories, just remove the respective item from the yaml file and neanno will not show any controls around annotating categories, key terms or named entities. Similar thing with the predictors. If you do not want to use any or some of the mentioned predictors, just remove them.\n",
    "\n",
    "\n",
    "3. Now that your project file is (hopefully) ready, you can start and use neanno. The command to start is exactly the same as for the airline_tickets sample except that you have to specify your new project file.\n",
    "\n",
    "> It's a best practice to put project file, dataset and all other files into a single folder so you have everything together.\n",
    ">\n",
    "> Don't miss to add that single folder to git and a version control such as Azure DevOps (Repos) or GitHub. This will ensure that you don't miss valuable work.\n",
    ">\n",
    "> As a rule of thumb you need approx. 1 hour to annotate 100 texts. So don't even dare to plan or ask someone to annotate\n",
    "> 20.000 texts! ;-)\n",
    "\n",
    "\n",
    "### Advanced\n",
    "\n",
    "#### Adding custom predictors\n",
    "\n",
    "One of neanno's top features is that you can bring your own predictors. To do this, you only have to write a class that inherits from the `Predictor` class in module `neanno.prediction.predictor` and either adjust neanno's project file or use the class directly in your code. The [neanno from Python](neanno-from-python.ipynb) notebook gives you more details about it. \n",
    "\n",
    "\n",
    "### Known Issues\n",
    "\n",
    "- On some machines, annotations are rendered with too much blank space (when font stretching is not supported)\n",
    "\n",
    "\n",
    "### To Do List\n",
    "\n",
    "- [ ] Export of model testing results\n",
    "- [ ] integrate better algorithms/frameworks\n",
    "- [ ] integrate with Microsoft Azure Machine Learning Services to leverage often required compute power\n",
    "- [ ] check Mac compatibility (should be compatible, only slight changes needed if at all)\n",
    "- [ ] model export\n",
    "- [ ] next best text by active learning\n",
    "- [ ] filter dataset\n",
    "\n",
    "\n",
    "### Done\n",
    "\n",
    "- [X] Bring-Your-Own-Predictors (BYOP)\n",
    "- [X] Category predictors\n",
    "- [X] Precision/Recall evaluation\n",
    "- [X] Output of training and testing to UI instead of console"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}