Skip to content
Merged
Changes from 9 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
233 changes: 123 additions & 110 deletions applications/pipeline_visualization/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@ This example demonstrates real-time visualization of data from Holoscan applicat

The example consists of three main components:

1. **C++ Data Producer**: A Holoscan application that generates data (sine waves) and publishes it to NATS
2. **NATS Server**: A message broker that handles real-time data streaming
3. **Python Web Visualizers**: Dash-based web applications that subscribe to NATS streams and display live plots
* **C++ Data Producer**: A Holoscan application that generates data (sine waves) and publishes it to NATS
* **NATS Server**: A message broker that handles real-time data streaming
* **Python Web Visualizers**: Dash-based web applications that subscribe to NATS streams and display live plots

## Quick Start

Expand Down Expand Up @@ -41,16 +41,20 @@ The web interface will be available at: [http://localhost:8050](http://localhost

In a third terminal, run the application:

```bash
# Run the Python version (default when --language is not specified)
./holohub run pipeline_visualization
* Run the Python version (default when `--language` is not specified):

# Or explicitly specify the language:
./holohub run pipeline_visualization --language python
./holohub run pipeline_visualization --language cpp
```
```bash
./holohub run pipeline_visualization
```

* Or explicitly specify the language:

**Command-line Options:**
```bash
./holohub run pipeline_visualization --language python
./holohub run pipeline_visualization --language cpp
```

Command-line options for `pipeline_visualization`:

```text
Usage: ./pipeline_visualization [options]
Expand All @@ -63,18 +67,18 @@ Options:
-r, --publish_rate Publish rate in Hz (default: 2.0)
```

**Example with custom settings:**
Example with custom settings:

```bash
./holohub run pipeline_visualization --nats_url nats://0.0.0.0:4222 --subject_prefix my_demo --publish_rate 5.0
```

### Step 4: Visualize the Data

1. Open your web browser to [http://localhost:8050](http://localhost:8050)
2. Enter the subject name (default: `nats_demo`)
3. Click **Connect**
4. Watch the real-time data plots update!
1. Open your web browser to [http://localhost:8050](http://localhost:8050).
2. Enter the subject name (default: `nats_demo`).
3. Click **Connect**.
4. Watch the real-time data plots update.

The visualizer will display:

Expand All @@ -83,11 +87,11 @@ The visualizer will display:
- **modulate.out**: Modulated signal with high-frequency component
- **sink.in**: Final processed signal (same as modulate.out)

## Configuration

### NATS Logger Configuration (`pipeline_visualization.yaml`)

The NATS logger behavior can be configured via YAML:
## NATS Logger Configuration (`pipeline_visualization.yaml`)

The NATS logger behavior can be configured using YAML:

```yaml
nats_logger:
Expand All @@ -102,7 +106,7 @@ nats_logger:
log_tensor_data_content: true # Include actual tensor data
```

### Architecture
## Architecture

```mermaid
%%{init: {'flowchart':{'subGraphTitleMargin':{'top':10, 'bottom':40}}}}%%
Expand Down Expand Up @@ -130,11 +134,32 @@ flowchart TB
nats -->|Subscribe| viz
```

## Visualizer Python Prerequisites

All dependencies to run the application are installed automatically when using the `holohub run` command inside the Holohub container.

If you are running the visualizer outside the Holohub container, its dependencies must be installed separately.

Install the required Python packages:

```bash
pip install -r requirements.txt
```

The packages that get installed are:

- `numpy>=1.24.0,<3.0` - Numerical computing
- `dash>=3.0.0,<4.0` - Web application framework
- `plotly>=6.0.0,<7.0` - Interactive plotting
- `nats-py>=2.0.0,<3.0` - NATS messaging client
- `flatbuffers>=25.9.23,<26.0.0` - FlatBuffers
- `pandas>=2.3.3,<3.0` - Data manipulation

## Components

### Application (`cpp/` and `python/`)

The C++ and Python applications demonstrates a simple Holoscan pipeline with data logging:
The C++ and Python applications demonstrate a basic Holoscan pipeline with data logging:

- **SourceOp**: Generates sine waves with varying frequencies (10-20 Hz)
- **ModulateOp**: Adds high-frequency modulation (300 Hz) to the signal
Expand All @@ -145,31 +170,33 @@ The applications log both inputs and outputs of operators, allowing visualizatio

### Python Visualizers (`visualizer/`)

Two visualization options are provided:
There are two Python visualizers. One is [static](#static-visualizer-visualizer_staticpy) and one is [dynamic](#dynamic-visualizer-visualizer_dynamicpy) and they both display:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style: Run-on sentence with repetitive structure. The three "and" conjunctions create awkward flow.

Suggested change
There are two Python visualizers. One is [static](#static-visualizer-visualizer_staticpy) and one is [dynamic](#dynamic-visualizer-visualizer_dynamicpy) and they both display:
There are two Python visualizers: a [static](#static-visualizer-visualizer_staticpy) visualizer and a [dynamic](#dynamic-visualizer-visualizer_dynamicpy) visualizer. Both display:

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!


#### 1. Static Visualizer (`visualizer_static.py`)
- Real-time line plots of tensor data
- Stream name (operator.port format)
- IO type (Input/Output)
- Acquisition timestamp (nanoseconds)
- Publish timestamp (nanoseconds)

Use `start_visualizer.sh` to set the required Python path to the [flatbuffers](#flatbuffers-schemas-schemas) definitions and start the visualizer. The script takes one parameter, its values are:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

syntax: Grammatical inconsistency: "takes one parameter, its values are" - if there's one parameter, should be "its possible values are" or "its value can be"

Suggested change
Use `start_visualizer.sh` to set the required Python path to the [flatbuffers](#flatbuffers-schemas-schemas) definitions and start the visualizer. The script takes one parameter, its values are:
Use `start_visualizer.sh` to set the required Python path to the [flatbuffers](#flatbuffers-schemas-schemas) definitions and start the visualizer. The script takes one parameter, its possible values are:

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!


- `dynamic`, to start the dynamic visualizer.
- `static`, to start the static visualizer.

#### Static Visualizer (`visualizer_static.py`)

The static visualizer can be used when the output data and format of the Holoscan pipeline is known or some data needs special formatting.

- Displays predefined data streams: `source.out`, `modulate.in`, `modulate.out`, `sink.in`
- Best for applications with known, fixed operator topology
- All graphs are created upfront and updated as data arrives

The static visualizer can be used when the output data and format of the Holoscan pipeline is known or some data needs special formatting.

#### 2. Dynamic Visualizer (`visualizer_dynamic.py`)
#### Dynamic Visualizer (`visualizer_dynamic.py`)

- Automatically discovers and creates graphs for new data streams
- Ideal for applications with dynamic or unknown operator configurations
- Graphs are created on-the-fly as new unique IDs are detected

There is a script `start_visualizer.sh` which sets the required Python path to the flatbuffers definitions and starts the visualizer. The script takes a parameter, `dynamic` starts the dynamic visualizer and `static` starts the static visualizer.

Both visualizers display:

- Real-time line plots of tensor data
- Stream name (operator.port format)
- IO type (Input/Output)
- Acquisition timestamp (nanoseconds)
- Publish timestamp (nanoseconds)

### FlatBuffers Schemas (`schemas/`)

Expand All @@ -178,34 +205,13 @@ The data format is defined using FlatBuffers for efficient serialization:
- **message.fbs**: Top-level message structure with metadata
- **tensor.fbs**: Tensor data structure based on DLPack

## Prerequisites

All dependencies to run the application are installed automatically when using the `holohub run` command inside
the Holohub container. Since the visualizer is run outside the Holohub container, its dependencies must be
installed separately. See the next section for details.
FlatBuffers access the data directly without unpacking or parsing it and allow the schema to evolve over time while maintaining forward and backward compatibility.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

syntax: Subject-verb agreement issue: "FlatBuffers" as a technology name is singular, but "access" and "allow" are plural verbs

Suggested change
FlatBuffers access the data directly without unpacking or parsing it and allow the schema to evolve over time while maintaining forward and backward compatibility.
FlatBuffers accesses the data directly without unpacking or parsing it and allows the schema to evolve over time while maintaining forward and backward compatibility.


### Visualizer Python Dependencies

Install the required Python packages:

```bash
pip install -r requirements.txt
```

Required packages:
## NATS Message Structure and Data Format

- `numpy>=1.24.0,<3.0` - Numerical computing
- `dash>=3.0.0,<4.0` - Web application framework
- `plotly>=6.0.0,<7.0` - Interactive plotting
- `nats-py>=2.0.0,<3.0` - NATS messaging client
- `flatbuffers>=25.9.23,<26.0.0` - FlatBuffers
- `pandas>=2.3.3,<3.0` - Data manipulation

## Data Format

### NATS Message Structure

Messages are published to the subject: `<subject_prefix>.data` (e.g., `nats_demo.data`)
Messages are published to the subject: `<subject_prefix>.data` (for example, `nats_demo.data`)

Each message is a FlatBuffer-serialized `Message` containing:

Expand All @@ -230,54 +236,6 @@ Examples:
- `modulate.out` - Output port of the modulate operator
- `sink.in` - Input port of the sink operator

## Troubleshooting

### NATS Connection Issues

**Problem**: `Cannot connect to NATS` error

**Solution**:

- Ensure the NATS server is running: `docker ps | grep nats`
- Check if port 4222 is available: `netstat -an | grep 4222`
- Verify the NATS URL matches in both the C++ app and visualizer

### Visualizer Not Updating

**Problem**: Web page loads but graphs don't update

**Solution**:

1. Check that the C++ application is running
2. Verify the subject name matches (default: `nats_demo`)
3. Click the "Connect" button in the web interface
4. Check browser console for JavaScript errors

### FlatBuffers Import Errors

**Problem**: `ModuleNotFoundError: No module named 'pipeline_visualization.flatbuffers'`

**Solution**:

- Ensure the FlatBuffers files were generated during build
- Set PYTHONPATH correctly:

```bash
export PYTHONPATH=$PYTHONPATH:/path/to/build/applications/pipeline_visualization/flatbuffers/
```

- Verify the files exist in the build directory

### No Data Displayed

**Problem**: Graphs are empty or show no data

**Solution**:

- Check that `log_tensor_data_content: true` in the YAML config
- Verify the operator names match between the app and visualizer
- For static visualizer, ensure the unique IDs in the code match your operators
- For dynamic visualizer, wait a few seconds for auto-discovery

## Customization

Expand All @@ -301,7 +259,7 @@ To visualize data from your own Holoscan operators:
self._unique_ids = ["my_op.out", "my_other_op.in"]
```

3. For dynamic visualizer, no changes needed - it will auto-discover!
3. For dynamic visualizer, no changes needed - it will auto-discover.

### Customizing Visualizations

Expand Down Expand Up @@ -346,15 +304,70 @@ To access the visualizer from another machine:

1. Start the visualizer with host `0.0.0.0` (already configured)
2. Ensure port 8050 is accessible through firewall
3. Access via: `http://<server-ip>:8050`
3. Access using: `http://<server-ip>:8050`

### Multiple Applications

To run multiple Holoscan apps simultaneously:

1. Use different subject prefixes for each app
1. Use different subject prefixes for each app.
2. Start multiple visualizer instances on different ports:

```python
self._app.run(debug=True, host="0.0.0.0", port=8051)
```







## Troubleshooting

### NATS Connection Issues

**Problem**: `Cannot connect to NATS` error

**Solution**:

- Ensure the NATS server is running: `docker ps | grep nats`
- Check if port 4222 is available: `netstat -an | grep 4222`
- Verify the NATS URL matches in both the C++ app and visualizer

### Visualizer Not Updating

**Problem**: Web page loads but graphs don't update

**Solution**:

1. Check that the C++ application is running
2. Verify the subject name matches (default: `nats_demo`)
3. Click the **Connect** button in the web interface
4. Check browser console for JavaScript errors

### FlatBuffers Import Errors

**Problem**: `ModuleNotFoundError: No module named 'pipeline_visualization.flatbuffers'`

**Solution**:

1. Ensure the FlatBuffers files were generated during build.
2. Set `PYTHONPATH` correctly:

```bash
export PYTHONPATH=$PYTHONPATH:/path/to/build/applications/pipeline_visualization/flatbuffers/
```

3. Verify that the files exist in the build directory.

### No Data Displayed

**Problem**: Graphs are empty or show no data

**Solution**:

- Verify that `log_tensor_data_content: true` is in the YAML config
- Verify that the operator names match between the app and visualizer
- For static visualizer, ensure that the unique IDs in the code match your operators
- For dynamic visualizer, wait a few seconds for auto-discovery