EAF2CSV2MP4.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "authorship_tag": "ABX9TyP8LiTAnY0Yobr56jQbm2Xx",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/Cogitarian/MultCorp/blob/main/EAF2CSV2MP4.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Po wykonaniu anotacji przydatne jest zestawić fragmenty filmów, które mają takie same etykiety lub według innych kryteriów. Wówczas pomocna będzie operacja, która podzieli film na fragmenty na podstawie pliku .eaf. Operacja składa się z dwóch skryptów. Pierwszy przetwarza dane z pliku .eaf, z którego pobierze nazwę próbki oraz początek i koniec próbki, i zachowuje je w pliku .csv.:"
      ],
      "metadata": {
        "id": "VfYODLKgKILk"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "yi7DRkPbJzS0"
      },
      "outputs": [],
      "source": [
        "#!/bin/bash\n",
        "eaf_file=\"$1\"\n",
        "if [[ ! -f \"$eaf_file\" ]]; then\n",
        "    echo \"EAF file not found!\"\n",
        "    exit 1\n",
        "fi\n",
        "output_csv=\"${eaf_file%.eaf}.csv\"\n",
        "# Print CSV header\n",
        "echo \"\\\"TIER\\\",\\\"START_TIME\\\",\\\"END_TIME\\\",\\\"ANNOTATION\\\"\" > \"$output_csv\"\n",
        "# Get a list of unique tier IDs, sort them and then remove duplicates\n",
        "tiers=$(xmlstarlet sel -t -v \"//TIER/@TIER_ID\" \"$eaf_file\" | sort | uniq)\n",
        "# For each tier, extract the relevant details\n",
        "IFS=$'\\n'\n",
        "for tier in $tiers; do\n",
        "    xmlstarlet sel -t \\\n",
        "        -m \"//TIER[@TIER_ID='$tier']/ANNOTATION/ALIGNABLE_ANNOTATION\" \\\n",
        "        -v \"concat('\\\"', '$tier', '\\\"', ',', '\\\"', /ANNOTATION_DOCUMENT/TIME_ORDER/TIME_SLOT[@TIME_SLOT_ID=current()/@TIME_SLOT_REF1]/@TIME_VALUE, '\\\"', ',', '\\\"', /ANNOTATION_DOCUMENT/TIME_ORDER/TIME_SLOT[@TIME_SLOT_ID=current()/@TIME_SLOT_REF2]/@TIME_VALUE, '\\\"', ',', '\\\"', ANNOTATION_VALUE, '\\\"')\" \\\n",
        "        -n \"$eaf_file\" >> \"$output_csv\"\n",
        "done\n",
        "echo \"Output saved to $output_csv\""
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Drugi skrypt wygeneruje próbki i złoży ich nazwy za pomocą danych z wcześniej utworzonego pliku .csv. Ten skrypt zamieni znaki zapytania na “question” i odstępy na podkreślniki (_) by uniknąć problemów przy dalszym przetwarzaniu plików, więc nazwa pliku na podstawie „S-LH-FOCUS,0,1018,in space” to „S-LH-FOCUS_0_1018_in_space__input.mp4”."
      ],
      "metadata": {
        "id": "b7wpLDHkKOFY"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#!/bin/bash\n",
        "\n",
        "# Check for the correct number of arguments\n",
        "if [ \"$#\" -ne 2 ]; then\n",
        "    echo \"Usage: $0 <input_video.mp4> <input.csv>\"\n",
        "    exit 1\n",
        "fi\n",
        "\n",
        "input_video=\"$1\"\n",
        "base_input_video=$(basename \"$input_video\" .mp4)  # extracting base name without extension\n",
        "csv_file=\"$2\"\n",
        "\n",
        "# Check if the files exist\n",
        "if [[ ! -f \"$input_video\" || ! -f \"$csv_file\" ]]; then\n",
        "    echo \"Error: One of the files does not exist.\"\n",
        "    exit 2\n",
        "fi\n",
        "\n",
        "# Skip the header line in the CSV\n",
        "tail -n +2 \"$csv_file\" | while IFS=, read -r part1 start_ms end_ms part4; do\n",
        "    # Remove any surrounding quotes and potential carriage returns\n",
        "    part1=$(echo $part1 | tr -d '\"\\r')\n",
        "    start_ms=$(echo $start_ms | tr -d '\"\\r')\n",
        "    end_ms=$(echo $end_ms | tr -d '\"\\r')\n",
        "    part4=$(echo $part4 | tr -d '\"\\r')\n",
        "\n",
        "    # Replace spaces with underscores and ? with 'question'\n",
        "    part1=${part1// /_}\n",
        "    part4=${part4// /_}\n",
        "    part4=${part4//\\?/question}\n",
        "\n",
        "    # Construct the filename using original ms format\n",
        "    filename=\"${part1}_${start_ms}_${end_ms}_${part4}__${base_input_video}.mp4\"\n",
        "\n",
        "    # Print the ffmpeg command for debugging\n",
        "    echo \"Processing: $filename\"\n",
        "    ffmpeg -i \"$input_video\" -ss \"${start_ms}ms\" -t \"$(($end_ms - $start_ms))ms\" -c:v copy -c:a copy \"$filename\"\n",
        "done\n"
      ],
      "metadata": {
        "id": "HkrcZ3S9KQMC"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Aby uruchomić skrypty, należy zachować je w dwóch osobnych plikach o nazwach eaf2csv.sh i csv2spl.sh, a następnie uruchomić na pliku input.mp4 i input.eaf:\n",
        "./eaf2csv.sh input.eaf\n",
        "./csv2spl.sh input.mp4 input.csv\n",
        "\n"
      ],
      "metadata": {
        "id": "kcqZ4bvhKCn_"
      }
    }
  ]
}