diff --git a/libraries/video-chunking-utils/docs/user-guide/index.md b/libraries/video-chunking-utils/docs/user-guide/index.md index a6bacb7d26..9edaf321fb 100644 --- a/libraries/video-chunking-utils/docs/user-guide/index.md +++ b/libraries/video-chunking-utils/docs/user-guide/index.md @@ -1,5 +1,16 @@ # A Python Module for Video Chunking Utils + + ## Introduction This is a Python module designed for video chunking. It allows users to split video files into smaller, manageable segments. The module is designed to be easily installable via pip and can be used in various applications such as video processing, analysis, and content delivery. @@ -39,7 +50,7 @@ for i, micro_chunk in enumerate(micro_chunks_list): print(f"Total {len(micro_chunks_list)} chunks are generated.") ``` -### Method: Pelt Chunking +### Method: Pelt Chunking ```python from video_chunking import PeltChunking diff --git a/microservices/audio-analyzer/docs/user-guide/index.md b/microservices/audio-analyzer/docs/user-guide/index.md new file mode 100644 index 0000000000..f0c1c5c718 --- /dev/null +++ b/microservices/audio-analyzer/docs/user-guide/index.md @@ -0,0 +1,137 @@ +# Audio Analyzer + + + +Audio Analyzer microservice is used to generate transcription of audio from video files. + +## Overview + +The Audio Analyzer microservice provides an automated solution for extracting and transcribing +audio from video files. Designed for seamless integration into modern AI pipelines this +microservice enables converting spoken content within videos into accurate, searchable text. +By leveraging state-of-the-art speech-to-text models, the service supports a wide range of +audio formats and languages, making it suitable for diverse applications such as video summary, +media analysis, compliance monitoring, and content indexing. + +The microservice operates by first isolating the audio track from the input video file. +Once extracted, the audio is processed using advanced transcription models to generate a +time-aligned text transcript. This transcript can be used for downstream tasks such as keyword +search, sentiment analysis, or integration with other AI-driven analytics. + +Key features include robust handling of noisy or low-quality audio, support for batch and +real-time processing, and easy deployment as a RESTful API. The service is optimized for edge +and cloud environments, ensuring low latency and scalability. Developers can interact with the +microservice through simple API endpoints, enabling rapid integration into existing workflows. + +By automating the extraction and transcription of audio from video, the Audio Analyzer +microservice streamlines content analysis, improves accessibility, and unlocks new possibilities +for leveraging audio data in various video analytics use cases. + +### Key Benefits + +- **Benefit 1**: Enables multimodal analysis of video data by extracting information from its +audio track. +- **Benefit 2**: Seamless integration through RESTful APIs with various video analytics use +cases that benefit from audio processing. +- **Benefit 3**: Flexibility to use different ASR models as per use case requirements. + +### Features + +- **Feature 1**: Extract audio from video files. +- **Feature 2**: Transcribe speech using Whispercpp (CPU). +- **Feature 3**: RESTful API with FastAPI. +- **Feature 4**: Containerization with Docker. +- **Feature 5**: Automatic model download and conversion on startup. +- **Feature 6**: Persistent model storage. +- **Feature 7**: OpenVINO acceleration support for Intel hardware. +- **Feature 8**: **MinIO integration** for video source and transcript storage. + +### Use Cases + +Audio Analyzer microservice can be applied to various real-world use cases and scenarios across +different video analytics use cases cutting across different industry segments. The motivation +to provide the microservice primarily comes from enhancing the accuracy of the video summary +pipeline. Here are some examples: + +- **Use case 1**: Ego centric videos as captured in industry segments like Safety and Security, +Body worn camera for example, benefits from additional modality of information that Audio +transcription provides. +- **Use case 2**: Videos from class rooms are primarily analysed using their audio content. +Audio Analyzer microservice helps provide transcription which can be used to chapterize a +class room session, for example. +- **Use case 3**: Courtroom or Legal Proceedings with legal hearings or depositions are +primarily analysed using the spoken word. +- **Use case 4**: Video podcasts or interview recordings where the value is in the conversation, +discussions, or interviews, and visuals are secondary. +- **Use case 5**: Events, like Panel Discussions and Debates, where multiple speakers discuss +or debate topics, the audio contains the key arguments and insights. + +## How It Works + +The Audio Analyzer microservice accepts a video file for transcription from either file system +or minIO storage. Using the configured Whisper model, the transcription is created. The output +transcription along with the configured metadata is then stored in configured destination +location. It provides a RESTful API to configure and utilize the capabilities. + +## Models supported + +The service automatically downloads and manages the required models based on configuration. +Two types of models are supported: + +1. **GGML Models** Primarily used for inference on CPU using whispercpp backend. +2. **OpenVINO Models** Optimized for GPU inference on Intel GPUs. + +Models are downloaded on application startup, converted to OpenVINO format if needed, and +stored in persistent volumes for reuse. The conversion process includes: + +- Downloading the original Hugging Face Whisper model +- Converting the PyTorch model to OpenVINO format. +- Storing the encoder and decoder components separately for efficient inference + +### Available Whisper Models + +The following Whisper model variants are supported by the service (for both GGML and OpenVINO +formats): + + | Model ID | Description | Size | Languages | + | --------- | ------------------ | -------- | ------------ | + | tiny | Tiny model | ~75M | Multilingual | + | tiny.en | Tiny model | ~75M | English-only | + | base | Base model | ~150M | Multilingual | + | base.en | Base model | ~150M | English-only | + | small | Small model | ~450M | Multilingual | + | small.en | Small model | ~450M | English-only | + | medium | Medium model | ~1.5GB | Multilingual | + | medium.en | Medium model | ~1.5GB | English-only | + | large-v1 | Large model (v1) | ~2.9GB | Multilingual | + | large-v2 | Large model (v2) | ~2.9GB | Multilingual | + | large-v3 | Large model (v3) | ~2.9GB | Multilingual | + +## Supporting Resources + +- [Get Started Guide](./get-started) +- [API Reference](./api-reference) +- [System Requirements](./system-requirements) + + diff --git a/microservices/audio-analyzer/docs/user-guide/index.rst b/microservices/audio-analyzer/docs/user-guide/index.rst deleted file mode 100644 index 78c4a67e70..0000000000 --- a/microservices/audio-analyzer/docs/user-guide/index.rst +++ /dev/null @@ -1,98 +0,0 @@ -Audio Analyzer -============== - -Audio Analyzer microservice is used to generate transcription of audio from video files. - -Overview -######## - -The Audio Analyzer microservice provides an automated solution for extracting and transcribing audio from video files. Designed for seamless integration into modern AI pipelines this microservice enables converting spoken content within videos into accurate, searchable text. By leveraging state-of-the-art speech-to-text models, the service supports a wide range of audio formats and languages, making it suitable for diverse applications such as video summary, media analysis, compliance monitoring, and content indexing. - -The microservice operates by first isolating the audio track from the input video file. Once extracted, the audio is processed using advanced transcription models to generate a time-aligned text transcript. This transcript can be used for downstream tasks such as keyword search, sentiment analysis, or integration with other AI-driven analytics. - -Key features include robust handling of noisy or low-quality audio, support for batch and real-time processing, and easy deployment as a RESTful API. The service is optimized for edge and cloud environments, ensuring low latency and scalability. Developers can interact with the microservice through simple API endpoints, enabling rapid integration into existing workflows. - -By automating the extraction and transcription of audio from video, the Audio Analyzer microservice streamlines content analysis, improves accessibility, and unlocks new possibilities for leveraging audio data in various video analytics use cases. - -**Key Benefits** -- **Benefit 1**: Enables multimodal analysis of video data by extracting information from its audio track. -- **Benefit 2**: Seamless integration through RESTful APIs with various video analytics use cases that benefit from audio processing. -- **Benefit 3**: Flexibility to use different ASR models as per use case requirements. - -**Features** -- **Feature 1**: Extract audio from video files. -- **Feature 2**: Transcribe speech using Whispercpp (CPU). -- **Feature 3**: RESTful API with FastAPI. -- **Feature 4**: Containerization with Docker. -- **Feature 5**: Automatic model download and conversion on startup. -- **Feature 6**: Persistent model storage. -- **Feature 7**: OpenVINO acceleration support for Intel hardware. -- **Feature 8**: **MinIO integration** for video source and transcript storage. - -**Use Cases** - -Audio Analyzer microservice can be applied to various real-world use cases and scenarios across different video analytics use cases cutting across different industry segments. The motivation to provide the microservice primarily comes from enhancing the accuracy of the video summary pipeline. Here are some examples: - -- **Use case 1**: Ego centric videos as captured in industry segments like Safety and Security, Body worn camera for example, benefits from additional modality of information that Audio transcription provides. -- **Use case 2**: Videos from class rooms are primarily analysed using their audio content. Audio Analyzer microservice helps provide transcription which can be used to chapterize a class room session, for example. -- **Use case 3**: Courtroom or Legal Proceedings with legal hearings or depositions are primarily analysed using the spoken word. -- **Use case 4**: Video podcasts or interview recordings where the value is in the conversation, discussions, or interviews, and visuals are secondary. -- **Use case 5**: Events, like Panel Discussions and Debates, where multiple speakers discuss or debate topics, the audio contains the key arguments and insights. - -How It Works -############ - -The Audio Analyzer microservice accepts a video file for transcription from either file system or minIO storage. Using the configured Whisper model, the transcription is created. The output transcription along with the configured metadata is then stored in configured destination location. It provides a RESTful API to configure and utilize the capabilities. - -Models supported -################ - -The service automatically downloads and manages the required models based on configuration. Two types of models are supported: - -1. **GGML Models** Primarily used for inference on CPU using whispercpp backend. -2. **OpenVINO Models** Optimized for GPU inference on Intel GPUs. - -Models are downloaded on application startup, converted to OpenVINO format if needed, and stored in persistent volumes for reuse. The conversion process includes: -- Downloading the original Hugging Face Whisper model -- Converting the PyTorch model to OpenVINO format. -- Storing the encoder and decoder components separately for efficient inference - -Available Whisper Models -++++++++++++++++++++++++ - -The following Whisper model variants are supported by the service (for both GGML and OpenVINO formats): - - =========== ================== ======== ============== - Model ID Description Size Languages - =========== ================== ======== ============== - tiny Tiny model ~75M Multilingual - tiny.en Tiny model ~75M English-only - base Base model ~150M Multilingual - base.en Base model ~150M English-only - small Small model ~450M Multilingual - small.en Small model ~450M English-only - medium Medium model ~1.5GB Multilingual - medium.en Medium model ~1.5GB English-only - large-v1 Large model (v1) ~2.9GB Multilingual - large-v2 Large model (v2) ~2.9GB Multilingual - large-v3 Large model (v3) ~2.9GB Multilingual - =========== ================== ======== ============== - - -Supporting Resources -#################### - -- :doc:`Get Started Guide <./get-started>` -- :doc:`API Reference <./api-reference>` -- :doc:`System Requirements <./system-requirements>` - -.. toctree:: - :hidden: - :maxdepth: 2 - - overview-architecture - system-requirements - get-started - how-to-build-from-source - api-reference - release-notes diff --git a/microservices/dlstreamer-pipeline-server/docs/user-guide/index.md b/microservices/dlstreamer-pipeline-server/docs/user-guide/index.md new file mode 100644 index 0000000000..f81f86c746 --- /dev/null +++ b/microservices/dlstreamer-pipeline-server/docs/user-guide/index.md @@ -0,0 +1,105 @@ +# Deep Learning Streamer Pipeline Server + + + +Deep Learning Streamer Pipeline Server (DL Streamer Pipeline Server) is a Python-based, +interoperable containerized microservice for easy development and deployment of video analytics +pipelines. + +## Overview + +DL Streamer Pipeline Server microservice is built on top of [GStreamer](https://gstreamer.freedesktop.org/documentation/) +and [Deep Learning Streamer (DL Streamer)](https://github.com/open-edge-platform/dlstreamer/tree/master), +providing video ingestion and deep learning inferencing functionalities. + +Video analytics involves the conversion of video streams into valuable insights through the +application of video processing, inference, and analytics operations. It finds applications +in various business sectors including healthcare, retail, entertainment, and industrial domains. +The algorithms utilized in video analytics are responsible for performing tasks such as object +detection, classification, identification, counting, and tracking on the input video stream. + +## How it Works + +![DL Streamer Pipeline Server architecture](./images/dls-pipelineserver-simplified-arch.png) + +Here is the high level description of functionality of DL Streamer Pipeline Server module: + +- **RESTful Interface** + + Exposes RESTful endpoints to discover, start, stop and customize pipelines in JSON format. + +- **DL Streamer Pipeline Server Core** + + Manages and processes the REST requests interfacing with the core DL Streamer Pipeline Server + components and Pipeline Server Library. + +- **DL Streamer Pipeline Server Configuration handler** + + Reads the contents of a config file and accordingly constructs/starts pipelines. Dynamic + configuration change is supported via REST API. + +- **GST UDF Loader** + + DL Streamer Pipeline Server provides a [GStreamer plugin](https://gstreamer.freedesktop.org/documentation/plugins_doc.html?gi-language=c) - `udfloader`, which can be used to configure and load arbitrary UDFs. With + `udfloader`, DL Streamer Pipeline Server provides an easy way to bring user developed programs + and run them as a part of GStreamer pipelines. A User Defined Function (UDF) is a chunk of + user code that can transform video frames and/or manipulate metadata. For example, a UDF can + act as filter, preprocessor, classifier or a detector. These User Defined Functions can be + developed in Python. + +- **DL Streamer Pipeline Server Publisher** + + Supports publishing metadata to a file, MQTT/Kafka message brokers and frame along with + metadata to a MQTT message broker. It also supports publishing metadata and frame over OPCUA. + The frames can also be saved on S3 compliant storage. + +- **DL Streamer Pipeline Server Model Update** + + Supports integration with the Model Registry service - [Model Registry](https://docs.openedgeplatform.intel.com/dev/edge-ai-libraries/model-registry/index.html) for model download, deployment and management. + +- **Open Telemetry** + + Supports gathering metrics over Open Telemetry for seamless visualization and analysis. + + diff --git a/microservices/dlstreamer-pipeline-server/docs/user-guide/index.rst b/microservices/dlstreamer-pipeline-server/docs/user-guide/index.rst deleted file mode 100644 index 0326cfdfce..0000000000 --- a/microservices/dlstreamer-pipeline-server/docs/user-guide/index.rst +++ /dev/null @@ -1,78 +0,0 @@ -Deep Learning Streamer Pipeline Server -====================================== - -Deep Learning Streamer Pipeline Server (DL Streamer Pipeline Server) is a Python-based, interoperable containerized microservice for easy development and deployment of video analytics pipelines. - -Overview -######## - -DL Streamer Pipeline Server microservice is built on top of `GStreamer `__ and `Deep Learning Streamer (DL Streamer) `__, providing video ingestion and deep learning inferencing functionalities. - -Video analytics involves the conversion of video streams into valuable insights through the application of video processing, inference, and analytics operations. It finds applications in various business sectors including healthcare, retail, entertainment, and industrial domains. The algorithms utilized in video analytics are responsible for performing tasks such as object detection, classification, identification, counting, and tracking on the input video stream. - -How it Works -############ - -.. image:: ./images/dls-pipelineserver-simplified-arch.png - :alt: DL Streamer Pipeline Server Architecture - -Here is the high level description of functionality of DL Streamer Pipeline Server module: - -- **RESTful Interface** - - Exposes RESTful endpoints to discover, start, stop and customize pipelines in JSON format. - -- **DL Streamer Pipeline Server Core** - - Manages and processes the REST requests interfacing with the core DL Streamer Pipeline Server components and Pipeline Server Library. - -- **DL Streamer Pipeline Server Configuration handler** - - Reads the contents of config file and accordingly constructs/starts pipelines. Dynamic configuration change is supported via REST API. - -- **GST UDF Loader** - - DL Streamer Pipeline Server provides a `GStreamer plugin `__ - `udfloader` using which users can configure and load arbitrary UDFs. With ``udfloader``, DL Streamer Pipeline Server provides an easy way to bring user developed programs and run them as a part of GStreamer pipelines. A User Defined Function (UDF) is a chunk of user code that can transform video frames and/or manipulate metadata. For example, a UDF can act as filter, preprocessor, classifier or a detector. These User Defined Functions can be developed in Python. - -- **DL Streamer Pipeline Server Publisher** - - Supports publishing metadata to file, MQTT/Kafka message brokers and frame along with metadata to MQTT message broker. We also support publishing metadata and frame over OPCUA. The frames can also be saved on S3 compliant storage. - -- **DL Streamer Pipeline Server Model Update** - - Supports integration with Model Registry service `Model Registry `__ for model download, deployment and management. - -- **Open Telemetry** - - Supports gathering metrics over Open Telemetry for seamless visualization and analysis. - -.. toctree:: - :hidden: - :maxdepth: 2 - - overview-architecture - system-requirements - get-started - troubleshooting-guide - how-to-change-dlstreamer-pipeline - how-to-use-gpu-for-decode-and-inference - how-to-use-cpu-for-decode-and-inference - how-to-autostart-pipelines - how-to-launch-configurable-pipelines - how-to-perform-webrtc-frame-streaming - how-to-start-dlsps-mqtt-publish - how-to-store-s3-frame - how-to-store-metadata-influxdb - how-to-publish-metadata-over-ros2 - how-to-launch-and-manage-pipeline - how-to-use-rtsp-camera-as-video-source - how-to-run-udf-pipelines - how-to-deploy-with-helm - how-to-use-image-file-as-source-over-request-payload - how-to-download-and-run-yolo-models - how-to-build-from-source - how-to-add-system-timestamps-to-metadata - api-reference - environment-variables - advanced-guide/Overview - release_notes/Overview diff --git a/microservices/document-ingestion/pgvector/docs/index.rst b/microservices/document-ingestion/pgvector/docs/index.md similarity index 53% rename from microservices/document-ingestion/pgvector/docs/index.rst rename to microservices/document-ingestion/pgvector/docs/index.md index 900b16b4b9..fd0880ad87 100644 --- a/microservices/document-ingestion/pgvector/docs/index.rst +++ b/microservices/document-ingestion/pgvector/docs/index.md @@ -1,5 +1,15 @@ -Data Ingestion microservice -=========================== +# Data Ingestion microservice + + Data ingestion service loads, parses, and creates embeddings for popular document types like pdf, docx, and txt files. The data ingestion is implemented as a microservice which in turn @@ -17,37 +27,37 @@ vectorDB and minIO is used as the data store. 3. The selected components to implement the functionality is benchmarked and validated for optimal performance. -Example Use Case -################ +## Example Use Case **Use Case 1**: Refer to the -`ChatQnA `__ +[ChatQnA](https://github.com/open-edge-platform/edge-ai-libraries/tree/main/sample-applications/chat-question-and-answer) sample application where this microservice is used. -High-Level System View Diagram -++++++++++++++++++++++++++++++ +### High-Level System View Diagram -.. figure:: ./images/DataPrep_HL_Arch.png - :alt: System View Diagram +![system view diagram](./images/DataPrep_HL_Arch.png) Figure 1: High-level system view demonstrating the microservice in a real-world use case. +## Key Features -Key Features -############ +1. The user manages the documents using REST APIs supported by the microservice. The APIs +allow the user to upload, delete, and read the documents managed by the microservice. +2. The microservice uses PGVector as the vectorDB. However, implementation is modular to +support other vectorDBs. -1. The user manages the documents using REST APIs supported by the microservice. The APIs allows the user to upload, delete, and read the documents managed by the microservice. -2. The microservice uses PGVector as the vectorDB. However, implementation is modular to support other vectorDBs. +## Supporting Resources -Supporting Resources -#################### +- [Get Started Guide](./get-started) +- [System Requirements](./system-requirements) -- :doc:`Get Started Guide <./get-started>` -- :doc:`System Requirements <./system-requirements>` + diff --git a/microservices/model-registry/docs/user-guide/index.rst b/microservices/model-registry/docs/user-guide/index.md similarity index 80% rename from microservices/model-registry/docs/user-guide/index.rst rename to microservices/model-registry/docs/user-guide/index.md index c45b3daa66..a6e26b59e1 100644 --- a/microservices/model-registry/docs/user-guide/index.rst +++ b/microservices/model-registry/docs/user-guide/index.md @@ -1,11 +1,20 @@ -Model Registry -============== +# Model Registry + + The Model Registry microservice provides a centralized repository that facilitates the management of AI models. -Overview -######## +## Overview The Model Registry plays a crucial role in the machine learning life cycle, providing a structured storage environment for models and their metadata. It stores each model's @@ -24,7 +33,6 @@ store, and review models collectively. The model registry is an essential tool f developers deploying machine learning models, as it streamlines model management, fosters collaboration, and ultimately, aids in improving model deployments. - **Key Benefits** - **Benefit 1**: Keeps track of different versions of models, allowing for easy rollback and @@ -43,7 +51,7 @@ across different industries. Here are some examples: recommendations. - **Use Case 3**: Managing quality control models to ensure product consistency and safety. -# How It Works +## How It Works The Model Registry microservice works by serving as a centralized repository for models where, their versions, and metadata are stored. The software behind the microservice is @@ -53,11 +61,9 @@ artifacts via a RESTful API. The software fulfills the promise described in the Overview via its various components. -High-Level System View Diagram -############################## +### High-Level System View Diagram -.. figure:: images/Model_Registry_HLA.png - :alt: Architecture Diagram +![architecture diagram](images/Model_Registry_HLA.png) Figure 1: High-level system view demonstrating the microservice. @@ -82,8 +88,7 @@ The Intel® Geti™ software is accessible via optional configurations within th registry. Once configured, the model registry is able to access the projects and models hosted within a remote Geti platform. -Key Features -############ +## Key Features - **Feature 1**: Provides a comprehensive set of REST API endpoints for operations such as registering, updating, retrieving, and deleting models. @@ -94,21 +99,24 @@ Key Features - **Feature 4**: Offers optional configurations to integrate with the Intel® Geti™ software, enabling access to projects and models hosted on a remote Geti platform. -Supporting Resources -#################### +## Supporting Resources + +- [Get Started Guide](./get-started) +- [API Reference](./api-reference) +- [System Requirements](./system-requirements) -- :doc:`Get Started Guide ` -- :doc:`API Reference ` -- :doc:`System Requirements ` + diff --git a/microservices/model-registry/docs/user-guide/system-requirements.md b/microservices/model-registry/docs/user-guide/system-requirements.md index d170359321..c59ee7383b 100644 --- a/microservices/model-registry/docs/user-guide/system-requirements.md +++ b/microservices/model-registry/docs/user-guide/system-requirements.md @@ -1,4 +1,5 @@ # System Requirements + This page provides detailed hardware, software, and platform requirements to help you set up and run the application efficiently. ## Supported Platforms @@ -10,7 +11,6 @@ This page provides detailed hardware, software, and platform requirements to hel - Intel® Core™ processors (i5 or higher) - Intel® Xeon® processors (recommended for large deployments) - ## Minimum Requirements | **Component** | **Minimum Requirement** | **Recommended** | |---------------------|---------------------------|--------------------------| @@ -18,20 +18,19 @@ This page provides detailed hardware, software, and platform requirements to hel | **Memory** | 8 GB | 16 GB | | **Disk Space** | 128 GB SSD | 256 GB SSD | - ## Software Requirements **Required Software**: + - Docker 24.0 or higher - Python 3.9+ - ## Validation -- Ensure all required software are installed and configured before proceeding to [Get Started](./get-started.md). + +- Ensure all required software is installed and configured before proceeding to [Get Started](./get-started.md). ## Supporting Resources -* [Overview](Overview.md) -* [Get Started Guide](get-started.md) -* [API Reference](api-reference.md) -* [System Requirements](system-requirements.md) +- [Overview](Overview.md) +- [Get Started Guide](get-started.md) +- [API Reference](api-reference.md) diff --git a/microservices/multilevel-video-understanding/docs/user-guide/index.md b/microservices/multilevel-video-understanding/docs/user-guide/index.md index bcfaef9ef3..d3874cea8f 100644 --- a/microservices/multilevel-video-understanding/docs/user-guide/index.md +++ b/microservices/multilevel-video-understanding/docs/user-guide/index.md @@ -1,18 +1,46 @@ # Multi-level Video Understanding Microservice -This microservice delivers a novel approach to video summarization. By employing a configurable, multi-level architecture and enhanced temporal modeling, it progressively analyzes video content to generate significantly more accurate and context-aware summaries. + + +This microservice delivers a novel approach to video summarization. By employing a configurable, +multi-level architecture and enhanced temporal modeling, it progressively analyzes video content +to generate significantly more accurate and context-aware summaries. + The overall high-level design is shown as below: ![Multi-level Video Understanding High-level Design](high-level_design.png) Figure 1: Multi-level Video Understanding High-level Design -Among all the components, the `Multi-level Video Understanding` refers to this microservice. The `Video Chunking` is a liabraries implemented in this open-edge-platform(OEP) suites: [video-chunking-utils](https://github.com/open-edge-platform/edge-ai-libraries/tree/main/libraries/video-chunking-utils). The `Vision-Language Model Serving` and `Large Language Model Serving` are the dependent services that required by this microservice, running on OpenAI-compatibale APIs. +Among all the components, `Multi-level Video Understanding` refers to this microservice. +`Video Chunking` is a library implemented in this Open Edge Platform (OEP) suite: +[video-chunking-utils](https://github.com/open-edge-platform/edge-ai-libraries/tree/main/libraries/video-chunking-utils). +`Vision-Language Model Serving` and `Large Language Model Serving` are dependent +services required by this microservice, running on OpenAI-compatibale APIs. ## Overview -To handle long video summarization, we design a multi-level framework where each level operates via a recurrent approach, effectively control the context length to improve computational efficiency and comply with model constraints or GPU memory constraints. -This framework operates in three stages: (1) Detect scene-swicth boundaries to segment the long video into chunks. (2) Use VLM to generate captions for each of these short video clips. (3) Use LLM hierarchically and recurrently aggregates the textual captions to a coherent global summary. A dedicated temporal enhancement component is employed at each level to strengthen the connections between units. +To handle long video summarization, we design a multi-level framework where each level operates +via a recurrent approach, effectively control the context length to improve computational +efficiency and comply with model constraints or GPU memory constraints. + +This framework operates in three stages: + +1. Detects scene-switch boundaries to segment the long video into chunks. +2. Uses VLM to generate captions for each of these short video clips. +3. Uses LLM hierarchically and recurrently aggregates the textual captions to a coherent global +summary. A dedicated temporal enhancement component is employed at each level to strengthen +the connections between units. + **Features** - **Feature 1**: Process video from local files or http(s) links. @@ -22,15 +50,20 @@ This framework operates in three stages: (1) Detect scene-swicth boundaries to s - **Feature 5**: Support specify video chunking method in user requests. - **Feature 6**: Support specify multi-level settings in user requests. - **Feature 7**: Support specify temporal enhancement settings in user requests. -- **Feature 8**: Designed to work effortlessly with GenAI model servings that provide OpenAI-compatible APIs. +- **Feature 8**: Designed to work effortlessly with GenAI model servings that provide +OpenAI-compatible APIs. ## How It Works -The Multi-level Video Understanding microservice unlocks straightforward video summarization. Users simply submit a video file. The service then seamlessly analyzes the content through its intelligent, multi-level framework to generate a summary, which is returned directly to the client. A comprehensive RESTful API is provided to access and control key features. +The Multi-level Video Understanding microservice unlocks straightforward video summarization. +Users simply submit a video file. The service then seamlessly analyzes the content through its +intelligent, multi-level framework to generate a summary, which is returned directly to the +client. A comprehensive RESTful API is provided to access and control key features. ## Models supported -The service automatically downloads and manages the required models based on configuration. Any openAI-compatibal model servings are supported. +The service automatically downloads and manages the required models based on configuration. +Any OpenAI-compatible model servings are supported. ### Validated Models @@ -58,5 +91,6 @@ system-requirements release-notes api-reference how-to-build-from-source + ::: -hide_directive--> \ No newline at end of file +hide_directive--> diff --git a/microservices/time-series-analytics/docs/user-guide/index.md b/microservices/time-series-analytics/docs/user-guide/index.md new file mode 100644 index 0000000000..64fdca48f1 --- /dev/null +++ b/microservices/time-series-analytics/docs/user-guide/index.md @@ -0,0 +1,55 @@ +# Time Series Analytics + + + +The Time Series Analytics microservice provides the analytics capabilities for a time series +use case. + +It is a powerful, flexible solution for real-time analysis of time series data. Built on top +of **Kapacitor**, it enables both streaming and batch processing, seamlessly integrating with +**InfluxDB** for efficient data storage and retrieval. + +What sets this microservice apart is its support for advanced analytics through +**User-Defined Functions (UDFs)** written in Python. By leveraging the Intel® Extension for +**Scikit-learn**, you can accelerate machine learning workloads within their UDFs, unlocking +high-performance anomaly detection, predictive maintenance, and other sophisticated analytics. + +The key features include: +- **Bring your own Data Sets and corresponding User Defined Functions(UDFs) for custom analytics**: +Easily implement and deploy your own Python-based analytics logic, following Kapacitor’s UDF +standards. +- **Seamless Integration**: Automatically stores processed results back into InfluxDB for +unified data management and visualization. +- **Model Registry Support**: Dynamically fetch and deploy UDF scripts, machine learning +models, and TICKscripts from the Model Registry microservice, enabling rapid customization +and iteration. +- **Versatile Use Cases**: Ideal for anomaly detection, alerting, and advanced time series +analytics in industrial, IoT, and enterprise environments. + +For more information on creating custom UDFs, see the +[Kapacitor Anomaly Detection Guide](https://docs.influxdata.com/kapacitor/v1/guides/anomaly_detection/). + + diff --git a/microservices/time-series-analytics/docs/user-guide/index.rst b/microservices/time-series-analytics/docs/user-guide/index.rst deleted file mode 100644 index 052e4c33dc..0000000000 --- a/microservices/time-series-analytics/docs/user-guide/index.rst +++ /dev/null @@ -1,30 +0,0 @@ -Time Series Analytics -===================== - -The Time Series Analytics microservice provides the analytics capabilities for a time series use case. - -It is a powerful, flexible solution for real-time analysis of time series data. Built on top of **Kapacitor**, it enables both streaming and batch processing, seamlessly integrating with **InfluxDB** for efficient data storage and retrieval. - -What sets this microservice apart is its support for advanced analytics through **User-Defined Functions (UDFs)** written in Python. By leveraging the Intel® Extension for Scikit-learn*, you can accelerate machine learning workloads within their UDFs, unlocking high-performance anomaly detection, predictive maintenance, and other sophisticated analytics. - -The key features include: -- **Bring your own Data Sets and corresponding User Defined Functions(UDFs) for custom analytics**: Easily implement and deploy your own Python-based analytics logic, following Kapacitor’s UDF standards. -- **Seamless Integration**: Automatically stores processed results back into InfluxDB for unified data management and visualization. -- **Model Registry Support**: Dynamically fetch and deploy UDF scripts, machine learning models, and TICKscripts from the Model Registry microservice, enabling rapid customization and iteration. -- **Versatile Use Cases**: Ideal for anomaly detection, alerting, and advanced time series analytics in industrial, IoT, and enterprise environments. - -For more information on creating custom UDFs, see the -`Kapacitor Anomaly Detection Guide `__ - - -.. toctree:: - :hidden: - - high-level-arch - system-requirements - get-started - how-to-access-api - how-to-configure - how-to-deploy-with-helm - api-reference - release_notes/Overview \ No newline at end of file diff --git a/microservices/vector-retriever/milvus/docs/user-guide/index.rst b/microservices/vector-retriever/milvus/docs/user-guide/index.md similarity index 82% rename from microservices/vector-retriever/milvus/docs/user-guide/index.rst rename to microservices/vector-retriever/milvus/docs/user-guide/index.md index 8f9cb64d85..8e78ec152d 100644 --- a/microservices/vector-retriever/milvus/docs/user-guide/index.rst +++ b/microservices/vector-retriever/milvus/docs/user-guide/index.md @@ -1,11 +1,9 @@ -Retriever Microservice -====================== +# Retriever Microservice Retrieves embeddings based on vector similarity. Usually it is used along with a `dataprep` microservice. -Overview -######## +## Overview The Retrieval Microservice is designed to perform efficient, vector-based searches using a vector database such as Milvus. It provides a RESTful API for retrieving @@ -14,21 +12,19 @@ The Retrieval Microservice is designed to perform efficient, vector-based search Key Features: -- Text-Based Image/Video Retrieval: +- Text-Based Image/Video Retrieval: Accepts text queries and retrieves the top-k most relevant results based on vector similarity. Supports optional filters to refine search results. -- Integration with Milvus: +- Integration with Milvus: Utilizes the Milvus vector database for efficient storage and retrieval of embeddings. Ensures high performance and scalability for large datasets. - **Programming Language:** Python -How It Works -############ +## How It Works 1. Query Processing: @@ -41,23 +37,24 @@ How It Works The retrieved results include metadata, similarity scores, and unique identifiers. Results are returned in JSON format for easy integration with downstream applications. -Workflow -######## +## Workflow 1. The embedding model generates text embeddings for input descriptions (e.g., "traffic jam"). 2. The search engine searches the vector database for the top-k most similar matches. 3. Generate results with the matched vector ids and metadata. +## Learn More -Learn More -########## +- Begin with the [Get Started Guide](./get-started) -- Begin with the :doc:`Get Started Guide <./get-started>` + diff --git a/sample-applications/chat-question-and-answer-core/docs/user-guide/index.rst b/sample-applications/chat-question-and-answer-core/docs/user-guide/index.md similarity index 64% rename from sample-applications/chat-question-and-answer-core/docs/user-guide/index.rst rename to sample-applications/chat-question-and-answer-core/docs/user-guide/index.md index ad63bb4a5e..185b87a293 100644 --- a/sample-applications/chat-question-and-answer-core/docs/user-guide/index.rst +++ b/sample-applications/chat-question-and-answer-core/docs/user-guide/index.md @@ -1,30 +1,31 @@ -Chat Question-and-Answer Core Sample Application -================================================ +# Chat Question-and-Answer Core Sample Application + + ChatQ&A sample application is a foundational Retrieval Augmented Generation (RAG) pipeline, enabling users to ask questions and receive answers including on their own private data corpus. The sample application demonstrates how to build RAG pipelines. Compared to the -`Chat Question-and-Answer `__ +[Chat Question-and-Answer](https://github.com/open-edge-platform/edge-ai-libraries/tree/main/sample-applications/chat-question-and-answer) implementation, this implementation of Chat Question-and-Answer Core is optimized for -memory footprint as it is built as a single monolithic application with the entire RAG +memory footprint, as it is built as a single monolithic application with the entire RAG pipeline bundled in a single microservice. The microservice supports a bare metal deployment through docker compose installation to emphasize on the monolithic objective. -.. image:: ./images/ChatQnA_Webpage.png - :alt: ChatQ&A web interface +![Chat Q&A web interface](./images/ChatQnA_Webpage.png) -Table of Contents -################# +## Overview and Architecture -1. `Overview and Architecture <#overview-and-architecture>`__ -2. `How to Use the Application <#how-to-use-the-application>`__ - -Overview and Architecture -######################### - -Key Features -++++++++++++ +### Key Features Key features include: @@ -35,27 +36,25 @@ Key features include: - **Supports a wide range of open-source models**: Intel's suite of inference microservices provides flexibility to use the right GenAI models (LLM, for example) as required for target usage. The application supports various - `open-source models `__, allowing developers to select + [open-source models](https://huggingface.co/OpenVINO), allowing developers to select the best models for their use cases. - **Self-hosting inference**: Perform inference locally or on-premises, ensuring data privacy and reducing latency. - **Observability and monitoring**: The application provides observability and - monitoring capabilities using `OpenTelemetry `__ & - `OpenLIT `__, enabling developers to monitor the + monitoring capabilities using [OpenTelemetry](https://opentelemetry.io/) & + [OpenLIT](https://github.com/openlit/openlit), enabling developers to monitor the application's performance and health in real-time. -Technical Architecture -++++++++++++++++++++++ +### Technical Architecture The Chat Question-and-Answer Core sample application is implemented as a LangChain based RAG pipeline with all the inference models (i.e. LLM, Embedding, and reranker) executed in the context of a single OpenVINO® runtime. The approach is documented in the OpenVINO -`documentation `__. -Readers are requested to refer to this documentation for the technical details. +[documentation](https://blog.openvino.ai/blog-posts/accelerate-inference-of-hugging-face-transformer-models-with-optimum-intel-and-openvino). +Readers are advised to refer to this documentation for the technical details. -How to Use the Application -########################## +## How to Use the Application The Chat Question-and-Answer Core sample application consists of two main parts: @@ -64,7 +63,7 @@ The Chat Question-and-Answer Core sample application consists of two main parts: common document formats like pdf and doc. The ingestion process cleans and formats the input document, creates embeddings of the documents using embedding microservice, and stores them in the preferred vector database. CPU version of - `FAISS `__ is used as VectorDB. + [FAISS](https://faiss.ai/index.html) is used as VectorDB. 2. **Generation [Q&A]**: This part allows the user to query the document database and generate responses. The LLM model, embedding model, and reranking model work @@ -75,21 +74,24 @@ The Chat Question-and-Answer Core sample application consists of two main parts: similarity. The LLM generates a context-aware response from the final set of documents. Detailed Hardware and Software requirements are available -:doc:`here <./system-requirements>`. +[here](./system-requirements). -Benchmark Results -################# +## Benchmark Results Detailed metrics and analysis can be found in the benchmark report -:doc:`here <./benchmarks>`. - -.. toctree:: - :hidden: - - system-requirements - get-started - build-from-source - deploy-with-helm - benchmarks - api-reference - release-notes +[here](./benchmarks). + + diff --git a/sample-applications/chat-question-and-answer/docs/user-guide/index.md b/sample-applications/chat-question-and-answer/docs/user-guide/index.md new file mode 100644 index 0000000000..d4682b385c --- /dev/null +++ b/sample-applications/chat-question-and-answer/docs/user-guide/index.md @@ -0,0 +1,140 @@ +# ChatQ&A Sample Application + + + +ChatQ&A sample application is a foundational Retrieval-Augmented Generation (RAG) pipeline +that allows users to ask questions and receive answers, including those based on their own +private data corpus. The sample application demonstrates how to build a RAG pipeline using +Intel's Edge AI microservices catalog and deploy it optimally on Intel's Edge AI systems. +This sample application simplifies the development, customization, and deployment of chat +applications in diverse deployment scenarios with out-of-the-box support for on-prem and edge +environments. + +![Chat Q&A web interface](./images/ChatQnA_Webpage.png) + +## Overview and Architecture + +### Key Features + +Key features include: + +- **Optimized RAG pipeline on Intel Edge AI Systems hardware**: The application is +[optimized](./benchmarks) to run efficiently on Intel® Edge AI systems hardware, ensuring +high performance and reliability. +- **Customizable RAG pipeline with optimized microservices**: Intel's Edge AI +[inference microservices](https://github.com/open-edge-platform/edge-ai-libraries/tree/main/microservices) +allow developers to customize and adapt specific parts of the application to suit their +deployment and usage needs. For example, developers can customize the data ingestion process +for specific document types without altering the rest of the RAG pipeline. Intel's inference +microservices provide the flexibility to tailor the application for specific deployment scenarios +and usage requirements without compromising performance on the given deployment hardware. +- **Flexible deployment options**: The application provides options for deployment using Docker +Compose and Helm charts, enabling developers to choose the best deployment environment for +their needs. +- **Support for a wide range of open-source models**: Intel's inference microservices provide +flexibility to use the right GenAI models (LLM, for example) as required for target usage. +The application supports various [open-source models](https://huggingface.co/OpenVINO), +allowing developers to select the best models for their use cases. +- **Self-hosting inference**: Perform inference locally or on-premises, ensuring data privacy +and reducing latency. +- **Observability and monitoring**: The application provides observability and monitoring +capabilities using [OpenTelemetry](https://opentelemetry.io/) & [OpenLIT](https://github.com/openlit/openlit), +enabling developers to monitor the application's performance and health in real-time. + +### Technical Architecture + +The ChatQ&A sample application includes the following components: + +- **LLM inference microservice**: Intel's optimized [OpenVINO Model Server (OVMS)](https://github.com/openvinotoolkit/model_server) +is used to efficiently run large language models on Intel hardware. Developers also have other +model serving options if required; vLLM with OpenVINO backend and TGI are the options provided. +(*Note: vLLM and TGI are deprecated effective 2025.2.0 release.*) +- **Embedding inference microservice**: Intel's optimized [OpenVINO Model Server (OVMS)](https://github.com/openvinotoolkit/model_server) +and Huggingface [Text Embeddings Inference](https://github.com/huggingface/text-embeddings-inference) +microservice are the options provided to run embedding models efficiently on target Intel +hardware. OVMS is the default option due to performance benefits on Intel hardware. +- **Reranking inference microservice**: Huggingface [Text Embeddings Inference](https://github.com/huggingface/text-embeddings-inference) +microservice is the model serving choice available. +- **Document ingestion microservice**: The sample [document ingestion](https://github.com/open-edge-platform/edge-ai-libraries/tree/main/microservices/document-ingestion/pgvector) +microservice allows ingestion of common document formats like PDF and DOC, and contents from +web links. It supports a REST endpoint to ingest the documents. The ingestion process creates +embeddings of the documents and stores them in the preferred vector database. The modular +architecture allows users to customize the vector database. The sample application uses the +[PGvector](https://github.com/pgvector/pgvector) database. The raw documents are stored in the +[MinIO](https://github.com/minio/minio) datastore, which is also customizable. +- **ChatQ&A backend microservice**: The Chat&A backend microservice is responsible for +responding to the user queries. LangChain framework is used to create the RAG chain and provide +the necessary integrations with VectorDB. The backend microservice provides a REST API to +accept user queries. +- **ChatQ&A UI**: A reference UI is provided to enable users quickly try the sample-application. +The UI is not designed to be a comprehensive guide on the design of UI but is more a functional +implementation to enable quick demo interaction. It allows users to create the context by +uploading the documents and provide weblinks. Users can then interact with the RAG pipeline. +It uses the REST API provided by the document ingestion and ChatQ&A backend for its operations. + +All the 3rd party components are not built from source but instead the respective pre-built +images are pulled from their artefactory (Docker hub for example) and used. The model serving +is also reused from their prebuilt versions and used for LLM, Embedding, and Reranker. The +Intel microservices can be built from source. Documentation is provided to help users build +these microservices from source. + +Further details on the system architecture and customizable options are available [here](./overview-architecture). + +![system architecture diagram](./images/TEAI_ChatQnA.png) + +## How to Use the Application + +Refer to the [Get Started](./get-started) page to get started with the sample-application. +The UI is the recommended mode of interaction with the sample-application though users can +also use the CLI approach using the REST endpoints. The ChatQ&A sample application consists +of two main parts: + +1. **Data Ingestion [Knowledge Building]**: This part is responsible for adding documents to +the ChatQ&A instance. The document ingestion microservice allows creation of this knowledge base. + +2. **Generation [Q&A]**: This part allows the user to query the document database and generate +responses. The LLM inference microservice, embedding inference microservice, and reranking +microservice work together to provide accurate and efficient answers to user queries. When a +user submits a question, the embedding model hosted by the chosen model serving (default is +OVMS) transforms it into an embedding, enabling semantic comparison with stored document +embeddings. The vector database searches for relevant embeddings, returning a ranked list of +documents based on semantic similarity. The LLM Inference Microservice generates a context-aware +response from the final set of documents. It is possible to use any supported models to run with +the applications. Detailed documentation provides full information on validated models and +models supported overall. + +Detailed hardware and software requirements are available [here](./system-requirements). + +This sample application is ready for deployment with Edge Orchestrator. Download the +[deployment package](./deploy-with-edge-orchestrator) and follow the instructions. + +## Benchmark Results + +Detailed metrics and analysis can be found in the benchmark report [here](./benchmarks). + + diff --git a/sample-applications/chat-question-and-answer/docs/user-guide/index.rst b/sample-applications/chat-question-and-answer/docs/user-guide/index.rst deleted file mode 100644 index c3006b329e..0000000000 --- a/sample-applications/chat-question-and-answer/docs/user-guide/index.rst +++ /dev/null @@ -1,80 +0,0 @@ -ChatQ&A Sample Application -========================== - -ChatQ&A sample application is a foundational Retrieval-Augmented Generation (RAG) pipeline that allows users to ask questions and receive answers, including those based on their own private data corpus. The sample application demonstrates how to build a RAG pipeline using Intel's Edge AI microservices catalog and deploy it optimally on Intel's Edge AI systems. This sample application simplifies the development, customization, and deployment of chat applications in diverse deployment scenarios with out-of-the-box support for on-prem and edge environments. - -.. image:: ./images/ChatQnA_Webpage.png - :alt: ChatQ&A web interface - -Table of Contents -################# - -1. `Overview and Architecture <#overview-and-architecture>`__ -2. `How to Use the Application <#how-to-use-the-application>`__ - -Overview and Architecture -######################### - -Key Features -++++++++++++ - -Key features include: - -- **Optimized RAG pipeline on Intel Edge AI Systems hardware**: The application is :doc:`optimized <./benchmarks>` to run efficiently on Intel® Edge AI systems hardware, ensuring high performance and reliability. -- **Customizable RAG pipeline with optimized microservices**: Intel's Edge AI `inference microservices `__ allow developers to customize and adapt specific parts of the application to suit their deployment and usage needs. For example, developers can customize the data ingestion process for specific document types without altering the rest of the RAG pipeline. Intel's inference microservices provide the flexibility to tailor the application for specific deployment scenarios and usage requirements without compromising performance on the given deployment hardware. -- **Flexible deployment options**: The application provides options for deployment using Docker Compose and Helm charts, enabling developers to choose the best deployment environment for their needs. -- **Support for a wide range of open-source models**: Intel's inference microservices provide flexibility to use the right GenAI models (LLM, for example) as required for target usage. The application supports various `open-source models `__, allowing developers to select the best models for their use cases. -- **Self-hosting inference**: Perform inference locally or on-premises, ensuring data privacy and reducing latency. -- **Observability and monitoring**: The application provides observability and monitoring capabilities using `OpenTelemetry `__ & `OpenLIT `__, enabling developers to monitor the application's performance and health in real-time. - -Technical Architecture -++++++++++++++++++++++ - -The ChatQ&A sample application includes the following components: - -- **LLM inference microservice**: Intel's optimized `OpenVINO Model Server (OVMS) `__ is used to efficiently run large language models on Intel hardware. Developers also have other model serving options if required. vLLM with OpenVINO backend and TGI are the options provided. (*Note: vLLM and TGI are deprecated effective 2025.2.0 release.*) -- **Embedding inference microservice**: Intel's optimized `OpenVINO Model Server (OVMS) `__ and Huggingface `Text Embeddings Inference `__ microservice are the options provided to run embedding models efficiently on target Intel hardware. OVMS is the default option due to performance benefits on Intel hardware. -- **Reranking inference microservice**: Huggingface `Text Embeddings Inference `__ microservice is the model serving choice available. -- **Document ingestion microservice**: The sample `document ingestion `__ microservice allows ingestion of common document formats like PDF and DOC, and contents from web links. It supports a REST endpoint to ingest the documents. The ingestion process creates embeddings of the documents and stores them in the preferred vector database. The modular architecture allows users to customize the vector database. The sample application uses `PGvector `__ database. The raw documents are stored in the `MinIO `__ datastore, which is also customizable. -- **ChatQ&A backend microservice**: The Chat&A backend microservice is responsible for responding to the user queries. LangChain framework is used to create the RAG chain and provide the necessary integrations with VectorDB. The backend microservice provides a REST API to accept user queries. -- **ChatQ&A UI**: A reference UI is provided to enable users quickly try the sample-application. The UI is not designed to be a comprehensive guide on the design of UI but is more a functional implementation to enable quick demo interaction. It allows users to create the context by uploading the documents and provide weblinks. Users can then interact with the RAG pipeline. It uses the REST API provided by the document ingestion and ChatQ&A backend for its operations. - -All the 3rd party components are not built from source but instead the respective pre-built images are pulled from their artefactory (Docker hub for example) and used. The model serving is also reused from their prebuilt versions and used for LLM, Embedding, and Reranker. The Intel microservices can be built from source. Documentation is provided to help users build these microservices from source. - -Further details on the system architecture and customizable options are available :doc:`here <./overview-architecture>`. - -.. image:: ./images/TEAI_ChatQnA.png - :alt: System Architecture Diagram - -How to Use the Application -########################## - -Refer to the :doc:`Get Started <./get-started>` page to get started with the sample-application. The UI is the recommended mode of interaction with the sample-application though users can also use the CLI approach using the REST endpoints. The ChatQ&A sample application consists of two main parts: - -1. **Data Ingestion [Knowledge Building]**: This part is responsible for adding documents to the ChatQ&A instance. The document ingestion microservice allows creation of this knowledge base. - -2. **Generation [Q&A]**: This part allows the user to query the document database and generate responses. The LLM inference microservice, embedding inference microservice, and reranking microservice work together to provide accurate and efficient answers to user queries. When a user submits a question, the embedding model hosted by the chosen model serving (default is OVMS) transforms it into an embedding, enabling semantic comparison with stored document embeddings. The vector database searches for relevant embeddings, returning a ranked list of documents based on semantic similarity. The LLM Inference Microservice generates a context-aware response from the final set of documents. It is possible to use any supported models to run with the applications. Detailed documentation provides full information on validated models and models supported overall. - -Detailed hardware and software requirements are available :doc:`here <./system-requirements>`. - -:doc:`This sample application is ready for deployment with Edge Orchestrator. Download the deployment package and follow the instructions `. - -Benchmark Results -################# - -Detailed metrics and analysis can be found in the benchmark report :doc:`here <./benchmarks>`. - -.. toctree:: - :hidden: - :maxdepth: 2 - - overview-architecture - system-requirements - get-started - build-from-source - deploy-with-helm - deploy-with-edge-orchestrator - how-to-performance - benchmarks - api-reference - release-notes diff --git a/sample-applications/document-summarization/docs/user-guide/index.rst b/sample-applications/document-summarization/docs/user-guide/index.md similarity index 72% rename from sample-applications/document-summarization/docs/user-guide/index.rst rename to sample-applications/document-summarization/docs/user-guide/index.md index 2c55592bd2..78b39cee7d 100644 --- a/sample-applications/document-summarization/docs/user-guide/index.rst +++ b/sample-applications/document-summarization/docs/user-guide/index.md @@ -1,22 +1,29 @@ -Document Summarization Sample Application -========================================= +# Document Summarization Sample Application + + Document Summarization Sample Application is a cutting-edge solution designed to efficiently generate concise summaries from extensive documents. This application leverages the power of LlamaIndex Document Summary Index, integrated with a microservices architecture that utilizes advanced language models (LLMs) to deliver high-quality summaries. -.. image:: ./images/docSum-web.png - :alt: Document Summarization web interface +![Document Summarization web interface](./images/docSum-web.png) -Key Features -############ +## Key Features - **Efficient Summarization**: Quickly generate summaries from large volumes of text, saving time and enhancing productivity. - **Scalable Microservices Architecture**: Designed to handle varying workloads, ensuring consistent performance and reliability. - **Advanced Language Model Integration**: Utilizes state-of-the-art LLMs to produce accurate and contextually relevant summaries. -Advantages of Using LlamaIndex Document Summary Index -##################################################### +## Advantages of Using LlamaIndex Document Summary Index - **Enhanced Accuracy and Relevance**: LlamaIndex Document Summary Index can produce summaries that are concise, accurate, and relevant. By focusing on key information and context, it ensures that the generated summaries @@ -31,25 +38,27 @@ Advantages of Using LlamaIndex Document Summary Index - **Customization and Flexibility**: LlamaIndex framework offers flexibility in summary generation, allowing users to customize the level of detail and focus according to their specific requirements. This adaptability makes it suitable for a wide range of applications and industries. -Technical Architecture -###################### +## Technical Architecture The Document Summarization Sample Application includes the following components: -- **LLM inference microservice**: Intel's optimized `OpenVINO™ Model Server `__ runs LLMs on Intel® hardware efficiently. Developers have other model serving options if required. +- **LLM inference microservice**: Intel's optimized [OpenVINO™ Model Server](https://github.com/openvinotoolkit/model_server) runs LLMs on Intel® hardware efficiently. Developers have other model serving options if required. - **Document Summary API Service**: A FastAPI service that exposes the API to summarize the uploaded document. The service ingests each document and uses a LLM to generate the summary. It splits the file into text chunks (nodes). The summary and nodes are stored within the Document Store abstraction. The application maintains the mapping from the summary to the source document. - **Document Summary UI Service**: A Gradio UI that enables you to upload a file and generate a summary with the summary API. The application supports the txt, docs, and pdf formats currently. -.. toctree:: - :hidden: - :maxdepth: 2 - - overview-architecture - system-requirements - get-started - build-from-source - deploy-with-helm - how-to-performance - api-reference - release-notes - faq + diff --git a/sample-applications/video-search-and-summarization/docs/user-guide/index.md b/sample-applications/video-search-and-summarization/docs/user-guide/index.md index b722852b54..a25937f606 100644 --- a/sample-applications/video-search-and-summarization/docs/user-guide/index.md +++ b/sample-applications/video-search-and-summarization/docs/user-guide/index.md @@ -1,5 +1,16 @@ # Video Search and Summarization (VSS) Sample Application + + Use the Video Search and Summarization (VSS) sample application to search through your videos, summarize them, and more. This foundational sample application provides three modes: @@ -10,10 +21,11 @@ This foundational sample application provides three modes: | 📝 Video Summarization ([overview](./overview-summary.md) and [architecture](./overview-architecture-summary.md)) | Create concise summaries of long-form videos or live streams, automatically. | Improve searchability. Combine insights from different data types using Generative AI Vision Language Models (VLMs), computer vision, and audio analysis. | | 🔗 Combined Video Search and Summarization ([overview](./overview-search-and-summary.md) and [architecture](./overview-architecture-search-and-summary.md)) | Find specific content and create concise summaries of videos - ideal for a comprehensive video analysis. | Search quickly and directly over generated video summaries. Using the summary as a knowledge base makes the search results more relevant and accurate. | -The detailed documentation to help you get started, configure, and deploy the sample application along with the required microservices are as follows. +The detailed documentation to help you get started, configure, and deploy the sample application +along with the required microservices are as follows. ## Documentation - + - **Get Started** - [Get Started](./get-started): How to get started with the sample application. - [System Requirements](./system-requirements.md): What hardware and software you need to run the sample application. diff --git a/tools/visual-pipeline-and-platform-evaluation-tool/docs/user-guide/index.md b/tools/visual-pipeline-and-platform-evaluation-tool/docs/user-guide/index.md new file mode 100644 index 0000000000..c26b0dfcd8 --- /dev/null +++ b/tools/visual-pipeline-and-platform-evaluation-tool/docs/user-guide/index.md @@ -0,0 +1,87 @@ +# Visual Pipeline and Platform Evaluation Tool + + + +Assess Intel® hardware options, benchmark performance, and analyze key metrics to optimize hardware selection for AI workloads. + +![demonstration of the UI in use](./_assets/ViPPET-README.gif) + +## Overview + +The Visual Pipeline and Platform Evaluation Tool simplifies hardware selection for AI workloads +by enabling configuration of workload parameters, performance benchmarking, and analysis of key +metrics such as throughput, CPU usage, and GPU usage. With its intuitive interface, the tool +provides actionable insights that support optimized hardware selection and performance tuning. + +## Use Cases + +**Evaluating Hardware for AI Workloads**: Intel® hardware options can be assessed to balance +cost, performance, and efficiency. AI workloads can be benchmarked under real-world conditions +by adjusting pipeline parameters and comparing performance metrics. + +**Performance Benchmarking for AI Models**: Model performance targets and KPIs can be validated +by testing AI inference pipelines with different accelerators to measure throughput, latency, +and resource utilization. + +## Key Features + +**Optimized for Intel® AI Edge Systems**: Pipelines can be run directly on target devices for +seamless Intel® hardware integration. + +**Comprehensive Hardware Evaluation**: Metrics such as CPU frequency, GPU power usage, and +memory utilization are available for detailed analysis. + +**Configurable AI Pipelines**: Parameters such as input channels, object detection models, and +inference engines can be adjusted to create tailored performance tests. + +**Automated Video Generation**: Synthetic test videos can be generated to evaluate system +performance under controlled conditions. + +### **Workflow Overview** + +**Data Ingestion**: Video streams from live cameras or recorded files are provided and pipeline +parameters are configured to match evaluation needs. + +**AI Processing**: AI inference is applied using OpenVINO™ models to detect objects in the +video streams. + +**Performance Evaluation**: Hardware performance metrics are collected, including CPU/GPU usage +and power consumption. + +**Visualization & Analysis**: Real-time performance metrics are displayed on the dashboard to +enable comparison of configurations and optimization of settings. + +## Learn More + +- [System Requirements](./system-requirements) +- [Get Started](./get-started) +- [How to Build Source](./how-to-build-source) +- [How to use Video Generator](./how-to-use-video-generator) +- [Release Notes](./release-notes) + + diff --git a/tools/visual-pipeline-and-platform-evaluation-tool/docs/user-guide/index.rst b/tools/visual-pipeline-and-platform-evaluation-tool/docs/user-guide/index.rst deleted file mode 100644 index b7d95c4dc3..0000000000 --- a/tools/visual-pipeline-and-platform-evaluation-tool/docs/user-guide/index.rst +++ /dev/null @@ -1,64 +0,0 @@ -Visual Pipeline and Platform Evaluation Tool -============================================ - -Assess Intel® hardware options, benchmark performance, and analyze key metrics to optimize hardware selection for AI workloads. - -.. image:: ./_assets/ViPPET-README.gif - -Overview -######## - -The Visual Pipeline and Platform Evaluation Tool simplifies hardware selection for AI workloads by enabling configuration of workload parameters, performance benchmarking, and analysis of key metrics such as throughput, CPU usage, and GPU usage. With its intuitive interface, the tool provides actionable insights that support optimized hardware selection and performance tuning. - -Use Cases -######### - -**Evaluating Hardware for AI Workloads**: Intel® hardware options can be assessed to balance cost, performance, and efficiency. AI workloads can be benchmarked under real-world conditions by adjusting pipeline parameters and comparing performance metrics. - -**Performance Benchmarking for AI Models**: Model performance targets and KPIs can be validated by testing AI inference pipelines with different accelerators to measure throughput, latency, and resource utilization. - -Key Features -############ - -**Optimized for Intel® AI Edge Systems**: Pipelines can be run directly on target devices for seamless Intel® hardware integration. - -**Comprehensive Hardware Evaluation**: Metrics such as CPU frequency, GPU power usage, and memory utilization are available for detailed analysis. - -**Configurable AI Pipelines**: Parameters such as input channels, object detection models, and inference engines can be adjusted to create tailored performance tests. - -**Automated Video Generation**: Synthetic test videos can be generated to evaluate system performance under controlled conditions. - - -### **Workflow Overview** - -**Data Ingestion**: Video streams from live cameras or recorded files are provided and pipeline parameters are configured to match evaluation needs. - -**AI Processing**: AI inference is applied using OpenVINO™ models to detect objects in the video streams. - -**Performance Evaluation**: Hardware performance metrics are collected, including CPU/GPU usage and power consumption. - -**Visualization & Analysis**: Real-time performance metrics are displayed on the dashboard to enable comparison of configurations and optimization of settings. - -Learn More -########## - -- :doc:`System Requirements <./system-requirements>` -- :doc:`Get Started <./get-started>` -- :doc:`How to Build Source <./how-to-build-source>` -- :doc:`How to use Video Generator <./how-to-use-video-generator>` -- :doc:`Release Notes <./release-notes>` - -.. toctree:: - :hidden: - :maxdepth: 2 - - system-requirements - get-started - release-notes - how-to-build-source - using-vippet - how-to-use-video-generator - api-reference - disclaimers - known-issues - GitHub