examples/demo_basic.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Infer a schema to extract data from files"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook, we will demonstrate how to infer a schema from a set of files and using it to extract structured data from invoice PDF files.\n",
    "\n",
    "The steps are:\n",
    "1. Infer a schema from the invoices files.\n",
    "2. Extract structured data (i.e. JSONs) from invoice PDF files\n",
    "\n",
    "Additional Resources:\n",
    "- `LlamaExtract`: https://docs.cloud.llamaindex.ai/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup\n",
    "Install `llama-extract` client library:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install llama-extract"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Bring your own LlamaCloud API key:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "os.environ[\"LLAMA_CLOUD_API_KEY\"] = \"llx-...\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Infer the schema\n",
    "First, let's infer the schema using the invoice files with `LlamaExtract`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_extract import LlamaExtract\n",
    "\n",
    "extractor = LlamaExtract()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "extraction_schema = await extractor.ainfer_schema(\n",
    "    \"Test Schema\", [\"./data/noisebridge_receipt.pdf\", \"./data/parallels_invoice.pdf\"]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Preview the inferred schema:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'type': 'object',\n",
       " 'properties': {'Invoice': {'type': 'object',\n",
       "   'properties': {'total': {'type': 'string'},\n",
       "    'products': {'type': 'string'},\n",
       "    'salesTax': {'type': 'string'},\n",
       "    'subtotal': {'type': 'string'},\n",
       "    'invoiceDate': {'type': 'string'},\n",
       "    'invoiceNumber': {'type': 'string'},\n",
       "    'billingAddress': {'type': 'object',\n",
       "     'properties': {'city': {'type': 'string'},\n",
       "      'name': {'type': 'string'},\n",
       "      'country': {'type': 'string'},\n",
       "      'postalCode': {'type': 'string'}}},\n",
       "    'paymentDetails': {'type': 'object',\n",
       "     'properties': {'taxId': {'type': 'string'},\n",
       "      'merchant': {'type': 'string'},\n",
       "      'merchantAddress': {'type': 'object',\n",
       "       'properties': {'city': {'type': 'string'},\n",
       "        'suite': {'type': 'string'},\n",
       "        'street': {'type': 'string'},\n",
       "        'country': {'type': 'string'},\n",
       "        'postalCode': {'type': 'string'}}},\n",
       "      'creditCardLastFour': {'type': 'string'}}},\n",
       "    'referenceNumber': {'type': 'string'}}},\n",
       "  'Receipt': {'type': 'object',\n",
       "   'properties': {'items': {'type': 'array',\n",
       "     'items': {'type': 'object',\n",
       "      'properties': {'amount': {'type': 'string'},\n",
       "       'quantity': {'type': 'integer'},\n",
       "       'unitPrice': {'type': 'string'},\n",
       "       'description': {'type': 'string'}}}},\n",
       "    'total': {'type': 'string'},\n",
       "    'datePaid': {'type': 'string'},\n",
       "    'subtotal': {'type': 'string'},\n",
       "    'amountPaid': {'type': 'string'},\n",
       "    'paymentMethod': {'type': 'string'},\n",
       "    'receiptNumber': {'type': 'string'},\n",
       "    'billingAddress': {'type': 'object',\n",
       "     'properties': {'city': {'type': 'string'},\n",
       "      'name': {'type': 'string'},\n",
       "      'email': {'type': 'string'},\n",
       "      'street': {'type': 'string'},\n",
       "      'country': {'type': 'string'},\n",
       "      'postalCode': {'type': 'string'},\n",
       "      'phoneNumber': {'type': 'string'}}}}}}}"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "extraction_schema.data_schema"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Extract structured data\n",
    "Now with the schema, we can extract structured data (i.e. JSON) from the our invoices files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Extracting files: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:06<00:00,  3.10s/it]\n"
     ]
    }
   ],
   "source": [
    "extractions = await extractor.aextract(\n",
    "    extraction_schema.id,\n",
    "    [\"./data/noisebridge_receipt.pdf\", \"./data/parallels_invoice.pdf\"],\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Preview the extracted data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'Receipt': {'items': [{'amount': '$10.00',\n",
       "    'quantity': 1,\n",
       "    'unitPrice': '$10.00',\n",
       "    'description': '$10 / month'}],\n",
       "  'total': '$10.00',\n",
       "  'datePaid': 'July 19, 2024',\n",
       "  'subtotal': '$10.00',\n",
       "  'amountPaid': '$10.00',\n",
       "  'paymentMethod': 'Visa - 7267',\n",
       "  'receiptNumber': '2721 5058',\n",
       "  'billingAddress': {'city': 'San Francisco',\n",
       "   'name': 'Noisebridge',\n",
       "   'email': 'noisebridge@seldo.com',\n",
       "   'street': '272 Capp St',\n",
       "   'country': 'United States',\n",
       "   'postalCode': '94110',\n",
       "   'phoneNumber': '1 650 701 7829'}}}"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "extractions[0].data"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}