Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ The uploaded audio is passed to the Backend API, which acts as the gateway to th

- **Pipeline Service**

The Pipeline Service manages multiple DLStreamer-based pipelines:
The Pipeline Service manages multiple DL Streamer-based pipelines:

- Front Video Pipeline for front camera streams
- Back Video Pipeline for back camera streams
Expand All @@ -54,10 +54,8 @@ A Media Server (MediaMTX) supports streaming and distribution of processed video
- Performance metrics (e.g., utilisation, model efficiency) are displayed for monitoring.
- Localisation ensures outputs are available in multiple languages (English/Chinese).


## Learn More

- [System Requirements](system-requirements.md): Check the hardware and software requirements for deploying the application.
- [Get Started](get-started.md): Follow step-by-step instructions to set up the application.
- [Application Flow](application-flow.md): Check the flow of application.

2 changes: 1 addition & 1 deletion manufacturing-ai-suite/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ The Manufacturing AI Suite helps you develop solutions for:

| | |
|:-------------|:------------|
| [Deep Learning Streamer](https://github.com/open-edge-platform/edge-ai-libraries/tree/main/libraries/dl-streamer) | A framework for building optimized media analytics pipelines powered by OpenVINO™ toolkit. |
| [Deep Learning Streamer](https://github.com/open-edge-platform/dlstreamer/tree/master) | A framework for building optimized media analytics pipelines powered by OpenVINO™ toolkit. |
| [Deep Learning Streamer Pipeline Server](https://github.com/open-edge-platform/edge-ai-libraries/tree/main/microservices/dlstreamer-pipeline-server) | A containerized microservice, built on top of GStreamer, for development and deployment of video analytics pipelines. |
| [Model Registry](https://github.com/open-edge-platform/edge-ai-libraries/tree/main/microservices/model-registry) | Providing capabilities to manage the lifecycle of an AI model. |
| [Time Series Analytics Microservice](https://github.com/open-edge-platform/edge-ai-libraries/tree/main/microservices/time-series-analytics) | Built on top of **Kapacitor**, a containerized microservice for development and deployment of time series analytics capabilities |
Expand Down
4 changes: 2 additions & 2 deletions manufacturing-ai-suite/hmi-augmented-worker/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

GenAI is transforming Human Machine Interfaces (HMI) by enabling more intuitive, conversational, and context-aware interactions between operators and industrial systems. By leveraging advanced language models and retrieval-augmented generation, GenAI enhances decision-making, streamlines troubleshooting, and delivers real-time, actionable insights directly within the HMI environment. This leads to improved operator efficiency, reduced downtime, and safer manufacturing operations.

The `HMI Augmented Worker` sample application show cases how RAG pipelines can be integrated with HMI application. Besides RAG, the key feature of this sample application is that it executes in a Hypervisor based setup where HMI application executes on Windows® OS based VM while the RAG application runs in native Ubuntu or EMT based setup. This enables running this application on Intel® Core™ portfolio.
The `HMI Augmented Worker` sample application show cases how RAG pipelines can be integrated with HMI application. Besides RAG, the key feature of this sample application is that it executes in a Hypervisor based setup where HMI application executes on Windows® OS based VM while the RAG application runs in native Ubuntu or Edge Microvisor Toolkit based setup. This enables running this application on Intel® Core™ portfolio.

## Documentation

Expand All @@ -16,7 +16,7 @@ The `HMI Augmented Worker` sample application show cases how RAG pipelines can b
- [System Requirements](./docs/user-guide/system-requirements.md): Requirements include hardware and software to deploy the sample application.

- **Advanced**
- [Build From Source](./docs/user-guide/how-to-build-from-source.md): Guide to build the file watcher service on Windows® OS and how it can be interfaced with RAG pipeline that executes on the Ubuntu or EMT side.
- [Build From Source](./docs/user-guide/how-to-build-from-source.md): Guide to build the file watcher service on Windows® OS and how it can be interfaced with RAG pipeline that executes on the Ubuntu or Edge Microvisor Toolkit side.

- **Release Notes**
- [Release Notes](./docs/user-guide/release-notes.md): Notes on the latest releases, updates, improvements, and bug fixes.
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
# Get Started

The `Get Started` guide explains how the HMI Augmented Worker application can be setup on a Type-2 hypervisor with the HMI on a Windows® VM while deploying the RAG pipeline natively on the Hypervisor host (EMT).
The `Get Started` guide explains how the HMI Augmented Worker application can be setup on a Type-2 hypervisor with the HMI on a Windows® VM while deploying the RAG pipeline natively on the Hypervisor host (Edge Microvisor Toolkit).

## Prerequisites

The sample application has mandatory prerequisites that are covered in other documentation. The user is required to refer to the respective documentation for the details. The prerequisites listed below cover such dependencies.

- Set up EMT based Type-2 Hypervisor host on target hardware. EMT is a reference hypervisor which has been used for validation. Other Type-2 hypervisors can also be used as per user preference. Reference documentation link for EMT as VM host is provided in [Other Documentation](#other-documentation) section. The reader is advised to contact Intel representatives for further details on configuring EMT host VM and instructions on hosting the Windows® Guest OS.
- Set up Edge Microvisor Toolkit based Type-2 Hypervisor host on target hardware. Edge Microvisor Toolkit is a reference hypervisor which has been used for validation. Other Type-2 hypervisors can also be used as per user preference. Reference documentation link for Edge Microvisor Toolkit as a VM host is provided in [Other Documentation](#other-documentation) section. The reader is advised to contact Intel representatives for further details on configuring an Edge Microvisor Toolkit host VM and instructions on hosting the Windows® Guest OS.

- The `HMI Augmented Worker` sample application utilizes `Chat Question and Answer Core` for the RAG pipeline. The [documentation](#other-documentation) available with `Chat Question-and-Answer Core` covers the details of how to set up the RAG pipeline, deploy it, and consume the application. Follow the instructions provided and set up the RAG pipeline.

Expand Down Expand Up @@ -68,8 +68,8 @@ To use the application effectively, make sure that all the steps mentioned in th

## Other Documentation

- [EMT Main Page](https://github.com/open-edge-platform/edge-microvisor-toolkit)
- [Create EMT bootable USB drive using source code](https://github.com/open-edge-platform/edge-microvisor-toolkit-standalone-node/blob/main/standalone-node/docs/user-guide/get-started-guide.md#create-a-bootable-usb-drive-using-source-code)
- [Desktop Virtualization on EMT](https://github.com/open-edge-platform/edge-microvisor-toolkit-standalone-node/blob/main/standalone-node/docs/user-guide/desktop-virtualization-image-guide.md)
- [EMT Documentation](https://github.com/open-edge-platform/edge-microvisor-toolkit/tree/3.0/docs/developer-guide)
- [Edge Microvisor Toolkit Main Page](https://github.com/open-edge-platform/edge-microvisor-toolkit)
- [Create Edge Microvisor Toolkit bootable USB drive using source code](https://github.com/open-edge-platform/edge-microvisor-toolkit-standalone-node/blob/main/standalone-node/docs/user-guide/get-started-guide.md#create-a-bootable-usb-drive-using-source-code)
- [Desktop Virtualization on Edge Microvisor Toolkit](https://github.com/open-edge-platform/edge-microvisor-toolkit-standalone-node/blob/main/standalone-node/docs/user-guide/desktop-virtualization-image-guide.md)
- [Edge Microvisor Toolkit Documentation](https://github.com/open-edge-platform/edge-microvisor-toolkit/tree/3.0/docs/developer-guide)
- [Chat Question and Answer Core Main Page](https://github.com/open-edge-platform/edge-ai-libraries/tree/main/sample-applications/chat-question-and-answer-core)
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@ a single physical machine.

In this architecture, the HMI application operates within a Windows® virtual machine managed
by a Type-2 hypervisor such as
[EMT](https://github.com/open-edge-platform/edge-microvisor-toolkit).
[Edge Microvisor Toolkit](https://github.com/open-edge-platform/edge-microvisor-toolkit).
The Retrieval-Augmented Generation (RAG) pipeline and supporting AI services are deployed
natively on a host system, which is EMT in this implementation.
natively on a host system, which is the Edge Microvisor Toolkit in this implementation.
[Chat Question-and-Answer Core](https://github.com/open-edge-platform/edge-ai-libraries/tree/main/sample-applications/chat-question-and-answer-core)
provides the RAG capability.
This separation ensures robust isolation between the HMI and AI components, enabling
Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# Overview
The HMI Augmented Worker is a RAG enabled HMI application deployed on Type-2 hypervisors. Deploying RAG-enabled HMI applications in a Type-2 hypervisor setup allows flexible and efficient resource utilization by running multiple operating systems on a single physical machine.
The HMI Augmented Worker is a RAG enabled HMI application deployed on Type-2 hypervisors. Deploying RAG-enabled HMI applications in a Type-2 hypervisor setup allows flexible and efficient resource utilization by running multiple operating systems on a single physical machine.

In this architecture, the HMI application operates within a Windows® virtual machine managed by a Type-2 hypervisor such as [EMT](https://github.com/open-edge-platform/edge-microvisor-toolkit). The Retrieval-Augmented Generation (RAG) pipeline and supporting AI services are deployed natively on a host system, which is EMT in this implementation. [Chat Question-and-Answer Core](https://github.com/open-edge-platform/edge-ai-libraries/tree/main/sample-applications/chat-question-and-answer-core) provides the RAG capability. This separation ensures robust isolation between the HMI and AI components, enabling independent scaling, maintenance, and updates. The setup leverages the strengths of both environments, providing a seamless integration that enhances operator experience while maintaining system reliability and security.
In this architecture, the HMI application operates within a Windows® virtual machine managed by a Type-2 hypervisor such as [Edge Microvisor Toolkit](https://github.com/open-edge-platform/edge-microvisor-toolkit). The Retrieval-Augmented Generation (RAG) pipeline and supporting AI services are deployed natively on a host system, which is the Edge Microvisor Toolkit in this implementation. [Chat Question-and-Answer Core](https://github.com/open-edge-platform/edge-ai-libraries/tree/main/sample-applications/chat-question-and-answer-core) provides the RAG capability. This separation ensures robust isolation between the HMI and AI components, enabling independent scaling, maintenance, and updates. The setup leverages the strengths of both environments, providing a seamless integration that enhances operator experience while maintaining system reliability and security.

RAG-enabled HMI applications offer a substantial opportunity to enhance the capabilities of manufacturing machine operators, especially those who are less experienced. RAG enabled LLM applications deliver a user-friendly interface for troubleshooting advice, data summarization, and planning, utilizing a knowledge base tailored to specific deployments, including telemetry data, support logs, machine manuals, and production plans. This document details the use cases, architectures, and requirements for implementing RAG LLMs in HMI systems to improve operational efficiency, decision-making, and overall productivity for machine operators. In this sample application, the focus is on providing an RAG pipeline in a Type-2 Hypervisor-based setup. There is no reference HMI used and the user is expected to do the HMI integration using the RAG pipeline APIs provided.
RAG-enabled HMI applications offer a substantial opportunity to enhance the capabilities of manufacturing machine operators, especially those who are less experienced. RAG enabled LLM applications deliver a user-friendly interface for troubleshooting advice, data summarization, and planning, utilizing a knowledge base tailored to specific deployments, including telemetry data, support logs, machine manuals, and production plans. This document details the use cases, architectures, and requirements for implementing RAG LLMs in HMI systems to improve operational efficiency, decision-making, and overall productivity for machine operators. In this sample application, the focus is on providing an RAG pipeline in a Type-2 Hypervisor-based setup. There is no reference HMI used and the user is expected to do the HMI integration using the RAG pipeline APIs provided.

## How it works
This section highlights the high-level architecture of the sample application.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,27 +3,26 @@

## Current Release
**Version**: RC1 \
**Release Date**: 14 July 2025
**Release Date**: 14 July 2025

**Key Features and Improvements:**

- **HMI Augmented Worker Use Case:** First drop of the sample application implementing the documented features.

**Development Testing:**

Intel® Core™ i7-14700 based systems with EMT and Windows® 11 based Guest VM.
Intel® Core™ i7-14700 based systems with Edge Microvisor Toolkit and Windows® 11 based Guest VM.

**Documentation:**

Documentation is **completed**. [README.md](../../README.md) is updated with installation steps and reference documents.
Documentation is **completed**. [README.md](../../README.md) is updated with installation steps and reference documents.

**Known Limitations and Issues:**

- EMF Deployment package is not applicable to this sample application.
- EMT as VM host setup documentation is dependent on what is available in EMT documentation.
- Edge Manageability Framework Deployment package is not applicable to this sample application.
- Edge Microvisor Toolkit as VM host setup documentation is dependent on what is available in the Edge Microvisor Toolkit documentation.
- Windows® Guest VM setup is not documented. Users are requested to contact Intel representatives for the same.

## Previous releases

None

Original file line number Diff line number Diff line change
Expand Up @@ -3,17 +3,19 @@
This page provides detailed hardware, software, and platform requirements to help you set up and run the application efficiently.

## Hardware Platforms used for validation

The `Chat question and answer Core` sample application system requirements are documented in [this](https://github.com/open-edge-platform/edge-ai-libraries/blob/main/sample-applications/chat-question-and-answer-core/docs/user-guide/system-requirements.md) page. The Intel® Core™ portfolio mentioned in this page is applicable for `HMI Augmented Worker` sample application too. The delta configurations supported is further described in this page.

The `HMI Augmented worker` sample application has been validated on Intel® Core™ i7-14700 based systems. The memory configuration used was 32GB which is the recommended minimum configuration. This machine hosts EMT based host together with Windows VM.
The `HMI Augmented worker` sample application has been validated on Intel® Core™ i7-14700 based systems. The memory configuration used was 32GB, which is the recommended minimum configuration. This machine hosts an Edge Microvisor Toolkit based host together with Windows VM.

## Software Requirements

Required Software:

- Python 3.10
- [EMT](https://github.com/open-edge-platform/edge-microvisor-toolkit) configuration requirements are documented in the [system requirement](https://github.com/open-edge-platform/edge-microvisor-toolkit/blob/3.0/docs/developer-guide/emt-system-requirements.md) page.
- [Edge Microvisor Toolkit](https://github.com/open-edge-platform/edge-microvisor-toolkit) configuration requirements are documented in the [system requirement](https://github.com/open-edge-platform/edge-microvisor-toolkit/blob/3.0/docs/developer-guide/emt-system-requirements.md) page.

## Supporting Resources
* [Overview](./overview.md)
* [Get Started Guide](./get-started.md)

- [Overview](./overview.md)
- [Get Started Guide](./get-started.md)
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ docker exec -ti ia-mqtt-broker mosquitto_sub -h localhost -v -t '#' -p 1883
docker exec -ti ia-mqtt-broker mosquitto_sub -h localhost -v -t alerts/weld_defect_detection -p 1883
```

#### Docker - Subscribing to DLStreamer Pipeline Server Results
#### Docker - Subscribing to DL Streamer Pipeline Server Results

```sh
docker exec -ti ia-mqtt-broker mosquitto_sub -h localhost -v -t vision_weld_defect_classification -p 1883
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ If not, follow the [installation guide for docker engine](https://docs.docker.co
3. Edit the below mentioned environment variables in the `.env` file as follows:

```bash
HOST_IP=<HOST_IP> # IP address of server where DLStreamer Pipeline Server is running.
HOST_IP=<HOST_IP> # IP address of server where DL Streamer Pipeline Server is running.

MR_PSQL_PASSWORD= #PostgreSQL service & client adapter e.g. intel1234

Expand Down Expand Up @@ -108,7 +108,7 @@ If not, follow the [installation guide for docker engine](https://docs.docker.co
./sample_start.sh -p pallet_defect_detection
```

This command will look for the payload for the pipeline specified in the `-p` argument above, inside the `payload.json` file and launch a pipeline instance in DLStreamer Pipeline Server. Refer to the table, to learn about different available options.
This command will look for the payload for the pipeline specified in the `-p` argument above, inside the `payload.json` file and launch a pipeline instance in DL Streamer Pipeline Server. Refer to the table, to learn about different available options.

> **IMPORTANT**: Before you run `sample_start.sh` script, make sure that
> `jq` is installed on your system. See the
Expand Down
Loading
Loading