Speech Summarizer.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "add6fb91",
   "metadata": {},
   "source": [
    "### Speech Summarizer"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9263e2e8",
   "metadata": {},
   "source": [
    "Building Speech summarizer that transcribes and summarizes speech input is quite simple with transformer based pretrained models and open-source libraries"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c0c093fb",
   "metadata": {},
   "source": [
    "#### Transcribe speech into text"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "42b8df16",
   "metadata": {},
   "source": [
    "Use deepspeech model to convert a speech file (e.g., speech.wav) into text file transcript.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d16c1ccc",
   "metadata": {},
   "source": [
    "Speech file should meet below specifications\n",
    "1. <b>wav</b> file format\n",
    "2. sample rate of <b>16KHz</b>\n",
    "3. <b>mono</b> type audio channel"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "42bc1b4b",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Loading model from file models/deepspeech-0.9.3-models.tflite\n",
      "TensorFlow: v2.3.0-6-g23ad988fcd\n",
      "DeepSpeech: v0.9.3-0-gf2e9c858\n",
      "Loaded model in 0.0148s.\n",
      "Running inference.\n",
      "Inference took 48.549s for 67.892s audio file.\n"
     ]
    }
   ],
   "source": [
    "!deepspeech --model models/deepspeech-0.9.3-models.tflite --audio input/speech.wav > output/transcript.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b1ffcfdd",
   "metadata": {},
   "source": [
    "#### Load generated transcript into memory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "47070559",
   "metadata": {},
   "outputs": [],
   "source": [
    "transcribed_text = str()\n",
    "with open(r\"output/transcript.txt\") as file:\n",
    "    transcribed_text = file.read().strip()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "a3baa315",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"there are certain things that a necessary to ensure that the future is good and some of those things are in the long term having long term sustainable transportand sustainable energy generation and to be a space exploring civilization and for humanity to be out their among the stars and be a multi planetry species i think the being a multi planetory species and being out there among the stars is important for the long terme survival of humanity and that's one reason kind of like life insurance for life collectively life as we know it but then the part that i find personally most motivating is that it creates a sense of adventure and it makes people excited about the future if you con sider two futures one way we are forever confine to earth until eventually something terrible happens ot another future where we are out there on many planets may be even going beyond the solar system i think that second version is incredibly exciting and inspiring at there need to be reasons to get up in the morning you know life can't just be about solving problems otherwise what's the point be'e got to be things that people find in spiring and make life worth living\""
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "transcribed_text"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a897cc49",
   "metadata": {},
   "source": [
    "#### Summarize transcribed speech"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c7c4b5c",
   "metadata": {},
   "source": [
    "##### Import libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "625322f7",
   "metadata": {},
   "outputs": [],
   "source": [
    "from transformers import AutoTokenizer, AutoModelForSeq2SeqLM"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "a904ab71",
   "metadata": {},
   "outputs": [],
   "source": [
    "MODEL = r\"models\\bart-large-cnn\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eb005b04",
   "metadata": {},
   "source": [
    "##### Load models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "4d5df9e6",
   "metadata": {},
   "outputs": [],
   "source": [
    "tokenizer = AutoTokenizer.from_pretrained(MODEL)\n",
    "model = AutoModelForSeq2SeqLM.from_pretrained(MODEL)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ce7e6b7",
   "metadata": {},
   "source": [
    "##### Summarize"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "4daba73e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Being a multi planetry species is important for the long terme survival of humanity. It creates a sense of adventure and it makes people excited about the future. There need to be reasons to get up in the morning you know life can't just be about solving problems otherwise what's the point.\n"
     ]
    }
   ],
   "source": [
    "input_ids = tokenizer(f\"summarize: {transcribed_text}\", return_tensors='pt').input_ids\n",
    "outputs = model.generate(input_ids)\n",
    "print(tokenizer.decode(outputs[0], skip_special_tokens=True))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}