-
Notifications
You must be signed in to change notification settings - Fork 133
[Documentation] Nvidia style guide edits to pipeline visualization #1328
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 9 commits
d83ae3d
8769b75
7f41b67
a7983f1
a979406
b919f72
4eecaf5
b73c03a
55a0e41
a0ea980
b21706a
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -8,9 +8,9 @@ This example demonstrates real-time visualization of data from Holoscan applicat | |||||
|
|
||||||
| The example consists of three main components: | ||||||
|
|
||||||
| 1. **C++ Data Producer**: A Holoscan application that generates data (sine waves) and publishes it to NATS | ||||||
| 2. **NATS Server**: A message broker that handles real-time data streaming | ||||||
| 3. **Python Web Visualizers**: Dash-based web applications that subscribe to NATS streams and display live plots | ||||||
| * **C++ Data Producer**: A Holoscan application that generates data (sine waves) and publishes it to NATS | ||||||
| * **NATS Server**: A message broker that handles real-time data streaming | ||||||
| * **Python Web Visualizers**: Dash-based web applications that subscribe to NATS streams and display live plots | ||||||
|
|
||||||
| ## Quick Start | ||||||
|
|
||||||
|
|
@@ -41,16 +41,20 @@ The web interface will be available at: [http://localhost:8050](http://localhost | |||||
|
|
||||||
| In a third terminal, run the application: | ||||||
|
|
||||||
| ```bash | ||||||
| # Run the Python version (default when --language is not specified) | ||||||
| ./holohub run pipeline_visualization | ||||||
| * Run the Python version (default when `--language` is not specified): | ||||||
|
|
||||||
| # Or explicitly specify the language: | ||||||
| ./holohub run pipeline_visualization --language python | ||||||
| ./holohub run pipeline_visualization --language cpp | ||||||
| ``` | ||||||
| ```bash | ||||||
| ./holohub run pipeline_visualization | ||||||
| ``` | ||||||
|
|
||||||
| * Or explicitly specify the language: | ||||||
|
|
||||||
| **Command-line Options:** | ||||||
| ```bash | ||||||
| ./holohub run pipeline_visualization --language python | ||||||
| ./holohub run pipeline_visualization --language cpp | ||||||
| ``` | ||||||
|
|
||||||
| Command-line options for `pipeline_visualization`: | ||||||
|
|
||||||
| ```text | ||||||
| Usage: ./pipeline_visualization [options] | ||||||
|
|
@@ -63,18 +67,18 @@ Options: | |||||
| -r, --publish_rate Publish rate in Hz (default: 2.0) | ||||||
| ``` | ||||||
|
|
||||||
| **Example with custom settings:** | ||||||
| Example with custom settings: | ||||||
|
|
||||||
| ```bash | ||||||
| ./holohub run pipeline_visualization --nats_url nats://0.0.0.0:4222 --subject_prefix my_demo --publish_rate 5.0 | ||||||
| ``` | ||||||
|
|
||||||
| ### Step 4: Visualize the Data | ||||||
|
|
||||||
| 1. Open your web browser to [http://localhost:8050](http://localhost:8050) | ||||||
| 2. Enter the subject name (default: `nats_demo`) | ||||||
| 3. Click **Connect** | ||||||
| 4. Watch the real-time data plots update! | ||||||
| 1. Open your web browser to [http://localhost:8050](http://localhost:8050). | ||||||
| 2. Enter the subject name (default: `nats_demo`). | ||||||
| 3. Click **Connect**. | ||||||
| 4. Watch the real-time data plots update. | ||||||
|
|
||||||
| The visualizer will display: | ||||||
|
|
||||||
|
|
@@ -83,11 +87,11 @@ The visualizer will display: | |||||
| - **modulate.out**: Modulated signal with high-frequency component | ||||||
| - **sink.in**: Final processed signal (same as modulate.out) | ||||||
|
|
||||||
| ## Configuration | ||||||
|
|
||||||
| ### NATS Logger Configuration (`pipeline_visualization.yaml`) | ||||||
|
|
||||||
| The NATS logger behavior can be configured via YAML: | ||||||
| ## NATS Logger Configuration (`pipeline_visualization.yaml`) | ||||||
|
|
||||||
| The NATS logger behavior can be configured using YAML: | ||||||
|
|
||||||
| ```yaml | ||||||
| nats_logger: | ||||||
|
|
@@ -102,7 +106,7 @@ nats_logger: | |||||
| log_tensor_data_content: true # Include actual tensor data | ||||||
| ``` | ||||||
|
|
||||||
| ### Architecture | ||||||
| ## Architecture | ||||||
|
|
||||||
| ```mermaid | ||||||
| %%{init: {'flowchart':{'subGraphTitleMargin':{'top':10, 'bottom':40}}}}%% | ||||||
|
|
@@ -130,11 +134,32 @@ flowchart TB | |||||
| nats -->|Subscribe| viz | ||||||
| ``` | ||||||
|
|
||||||
| ## Visualizer Python Prerequisites | ||||||
|
|
||||||
| All dependencies to run the application are installed automatically when using the `holohub run` command inside the Holohub container. | ||||||
|
|
||||||
| If you are running the visualizer outside the Holohub container, its dependencies must be installed separately. | ||||||
|
|
||||||
| Install the required Python packages: | ||||||
|
|
||||||
| ```bash | ||||||
| pip install -r requirements.txt | ||||||
| ``` | ||||||
|
|
||||||
| The packages that get installed are: | ||||||
|
|
||||||
| - `numpy>=1.24.0,<3.0` - Numerical computing | ||||||
| - `dash>=3.0.0,<4.0` - Web application framework | ||||||
| - `plotly>=6.0.0,<7.0` - Interactive plotting | ||||||
| - `nats-py>=2.0.0,<3.0` - NATS messaging client | ||||||
| - `flatbuffers>=25.9.23,<26.0.0` - FlatBuffers | ||||||
| - `pandas>=2.3.3,<3.0` - Data manipulation | ||||||
|
|
||||||
| ## Components | ||||||
|
|
||||||
| ### Application (`cpp/` and `python/`) | ||||||
|
|
||||||
| The C++ and Python applications demonstrates a simple Holoscan pipeline with data logging: | ||||||
| The C++ and Python applications demonstrate a basic Holoscan pipeline with data logging: | ||||||
|
|
||||||
| - **SourceOp**: Generates sine waves with varying frequencies (10-20 Hz) | ||||||
| - **ModulateOp**: Adds high-frequency modulation (300 Hz) to the signal | ||||||
|
|
@@ -145,31 +170,33 @@ The applications log both inputs and outputs of operators, allowing visualizatio | |||||
|
|
||||||
| ### Python Visualizers (`visualizer/`) | ||||||
|
|
||||||
| Two visualization options are provided: | ||||||
| There are two Python visualizers. One is [static](#static-visualizer-visualizer_staticpy) and one is [dynamic](#dynamic-visualizer-visualizer_dynamicpy) and they both display: | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. style: Run-on sentence with repetitive structure. The three "and" conjunctions create awkward flow.
Suggested change
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time! |
||||||
|
|
||||||
| #### 1. Static Visualizer (`visualizer_static.py`) | ||||||
| - Real-time line plots of tensor data | ||||||
| - Stream name (operator.port format) | ||||||
| - IO type (Input/Output) | ||||||
| - Acquisition timestamp (nanoseconds) | ||||||
| - Publish timestamp (nanoseconds) | ||||||
|
|
||||||
| Use `start_visualizer.sh` to set the required Python path to the [flatbuffers](#flatbuffers-schemas-schemas) definitions and start the visualizer. The script takes one parameter, its values are: | ||||||
bhashemian marked this conversation as resolved.
Show resolved
Hide resolved
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. syntax: Grammatical inconsistency: "takes one parameter, its values are" - if there's one parameter, should be "its possible values are" or "its value can be"
Suggested change
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time! |
||||||
|
|
||||||
| - `dynamic`, to start the dynamic visualizer. | ||||||
| - `static`, to start the static visualizer. | ||||||
|
|
||||||
| #### Static Visualizer (`visualizer_static.py`) | ||||||
|
|
||||||
| The static visualizer can be used when the output data and format of the Holoscan pipeline is known or some data needs special formatting. | ||||||
|
|
||||||
| - Displays predefined data streams: `source.out`, `modulate.in`, `modulate.out`, `sink.in` | ||||||
| - Best for applications with known, fixed operator topology | ||||||
| - All graphs are created upfront and updated as data arrives | ||||||
|
|
||||||
| The static visualizer can be used when the output data and format of the Holoscan pipeline is known or some data needs special formatting. | ||||||
|
|
||||||
| #### 2. Dynamic Visualizer (`visualizer_dynamic.py`) | ||||||
| #### Dynamic Visualizer (`visualizer_dynamic.py`) | ||||||
|
|
||||||
| - Automatically discovers and creates graphs for new data streams | ||||||
| - Ideal for applications with dynamic or unknown operator configurations | ||||||
| - Graphs are created on-the-fly as new unique IDs are detected | ||||||
|
|
||||||
| There is a script `start_visualizer.sh` which sets the required Python path to the flatbuffers definitions and starts the visualizer. The script takes a parameter, `dynamic` starts the dynamic visualizer and `static` starts the static visualizer. | ||||||
|
|
||||||
| Both visualizers display: | ||||||
|
|
||||||
| - Real-time line plots of tensor data | ||||||
| - Stream name (operator.port format) | ||||||
| - IO type (Input/Output) | ||||||
| - Acquisition timestamp (nanoseconds) | ||||||
| - Publish timestamp (nanoseconds) | ||||||
|
|
||||||
| ### FlatBuffers Schemas (`schemas/`) | ||||||
megnvidia marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
|
|
||||||
|
|
@@ -178,34 +205,13 @@ The data format is defined using FlatBuffers for efficient serialization: | |||||
| - **message.fbs**: Top-level message structure with metadata | ||||||
| - **tensor.fbs**: Tensor data structure based on DLPack | ||||||
|
|
||||||
| ## Prerequisites | ||||||
|
|
||||||
| All dependencies to run the application are installed automatically when using the `holohub run` command inside | ||||||
| the Holohub container. Since the visualizer is run outside the Holohub container, its dependencies must be | ||||||
| installed separately. See the next section for details. | ||||||
| FlatBuffers access the data directly without unpacking or parsing it and allow the schema to evolve over time while maintaining forward and backward compatibility. | ||||||
megnvidia marked this conversation as resolved.
Show resolved
Hide resolved
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. syntax: Subject-verb agreement issue: "FlatBuffers" as a technology name is singular, but "access" and "allow" are plural verbs
Suggested change
|
||||||
|
|
||||||
| ### Visualizer Python Dependencies | ||||||
|
|
||||||
| Install the required Python packages: | ||||||
|
|
||||||
| ```bash | ||||||
| pip install -r requirements.txt | ||||||
| ``` | ||||||
|
|
||||||
| Required packages: | ||||||
| ## NATS Message Structure and Data Format | ||||||
|
|
||||||
| - `numpy>=1.24.0,<3.0` - Numerical computing | ||||||
| - `dash>=3.0.0,<4.0` - Web application framework | ||||||
| - `plotly>=6.0.0,<7.0` - Interactive plotting | ||||||
| - `nats-py>=2.0.0,<3.0` - NATS messaging client | ||||||
| - `flatbuffers>=25.9.23,<26.0.0` - FlatBuffers | ||||||
| - `pandas>=2.3.3,<3.0` - Data manipulation | ||||||
|
|
||||||
| ## Data Format | ||||||
|
|
||||||
| ### NATS Message Structure | ||||||
|
|
||||||
| Messages are published to the subject: `<subject_prefix>.data` (e.g., `nats_demo.data`) | ||||||
| Messages are published to the subject: `<subject_prefix>.data` (for example, `nats_demo.data`) | ||||||
|
|
||||||
| Each message is a FlatBuffer-serialized `Message` containing: | ||||||
|
|
||||||
|
|
@@ -230,54 +236,6 @@ Examples: | |||||
| - `modulate.out` - Output port of the modulate operator | ||||||
| - `sink.in` - Input port of the sink operator | ||||||
|
|
||||||
| ## Troubleshooting | ||||||
|
|
||||||
| ### NATS Connection Issues | ||||||
|
|
||||||
| **Problem**: `Cannot connect to NATS` error | ||||||
|
|
||||||
| **Solution**: | ||||||
|
|
||||||
| - Ensure the NATS server is running: `docker ps | grep nats` | ||||||
| - Check if port 4222 is available: `netstat -an | grep 4222` | ||||||
| - Verify the NATS URL matches in both the C++ app and visualizer | ||||||
|
|
||||||
| ### Visualizer Not Updating | ||||||
|
|
||||||
| **Problem**: Web page loads but graphs don't update | ||||||
|
|
||||||
| **Solution**: | ||||||
|
|
||||||
| 1. Check that the C++ application is running | ||||||
| 2. Verify the subject name matches (default: `nats_demo`) | ||||||
| 3. Click the "Connect" button in the web interface | ||||||
| 4. Check browser console for JavaScript errors | ||||||
|
|
||||||
| ### FlatBuffers Import Errors | ||||||
|
|
||||||
| **Problem**: `ModuleNotFoundError: No module named 'pipeline_visualization.flatbuffers'` | ||||||
|
|
||||||
| **Solution**: | ||||||
|
|
||||||
| - Ensure the FlatBuffers files were generated during build | ||||||
| - Set PYTHONPATH correctly: | ||||||
|
|
||||||
| ```bash | ||||||
| export PYTHONPATH=$PYTHONPATH:/path/to/build/applications/pipeline_visualization/flatbuffers/ | ||||||
| ``` | ||||||
|
|
||||||
| - Verify the files exist in the build directory | ||||||
|
|
||||||
| ### No Data Displayed | ||||||
|
|
||||||
| **Problem**: Graphs are empty or show no data | ||||||
|
|
||||||
| **Solution**: | ||||||
|
|
||||||
| - Check that `log_tensor_data_content: true` in the YAML config | ||||||
| - Verify the operator names match between the app and visualizer | ||||||
| - For static visualizer, ensure the unique IDs in the code match your operators | ||||||
| - For dynamic visualizer, wait a few seconds for auto-discovery | ||||||
|
|
||||||
| ## Customization | ||||||
|
|
||||||
|
|
@@ -301,7 +259,7 @@ To visualize data from your own Holoscan operators: | |||||
| self._unique_ids = ["my_op.out", "my_other_op.in"] | ||||||
| ``` | ||||||
|
|
||||||
| 3. For dynamic visualizer, no changes needed - it will auto-discover! | ||||||
| 3. For dynamic visualizer, no changes needed - it will auto-discover. | ||||||
|
|
||||||
| ### Customizing Visualizations | ||||||
|
|
||||||
|
|
@@ -346,15 +304,70 @@ To access the visualizer from another machine: | |||||
|
|
||||||
| 1. Start the visualizer with host `0.0.0.0` (already configured) | ||||||
| 2. Ensure port 8050 is accessible through firewall | ||||||
| 3. Access via: `http://<server-ip>:8050` | ||||||
| 3. Access using: `http://<server-ip>:8050` | ||||||
|
|
||||||
| ### Multiple Applications | ||||||
|
|
||||||
| To run multiple Holoscan apps simultaneously: | ||||||
|
|
||||||
| 1. Use different subject prefixes for each app | ||||||
| 1. Use different subject prefixes for each app. | ||||||
| 2. Start multiple visualizer instances on different ports: | ||||||
|
|
||||||
| ```python | ||||||
| self._app.run(debug=True, host="0.0.0.0", port=8051) | ||||||
| ``` | ||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
| ## Troubleshooting | ||||||
|
|
||||||
| ### NATS Connection Issues | ||||||
|
|
||||||
| **Problem**: `Cannot connect to NATS` error | ||||||
|
|
||||||
| **Solution**: | ||||||
|
|
||||||
| - Ensure the NATS server is running: `docker ps | grep nats` | ||||||
| - Check if port 4222 is available: `netstat -an | grep 4222` | ||||||
| - Verify the NATS URL matches in both the C++ app and visualizer | ||||||
|
|
||||||
| ### Visualizer Not Updating | ||||||
|
|
||||||
| **Problem**: Web page loads but graphs don't update | ||||||
|
|
||||||
| **Solution**: | ||||||
|
|
||||||
| 1. Check that the C++ application is running | ||||||
| 2. Verify the subject name matches (default: `nats_demo`) | ||||||
| 3. Click the **Connect** button in the web interface | ||||||
| 4. Check browser console for JavaScript errors | ||||||
|
|
||||||
| ### FlatBuffers Import Errors | ||||||
|
|
||||||
| **Problem**: `ModuleNotFoundError: No module named 'pipeline_visualization.flatbuffers'` | ||||||
|
|
||||||
| **Solution**: | ||||||
|
|
||||||
| 1. Ensure the FlatBuffers files were generated during build. | ||||||
| 2. Set `PYTHONPATH` correctly: | ||||||
|
|
||||||
| ```bash | ||||||
| export PYTHONPATH=$PYTHONPATH:/path/to/build/applications/pipeline_visualization/flatbuffers/ | ||||||
| ``` | ||||||
|
|
||||||
| 3. Verify that the files exist in the build directory. | ||||||
|
|
||||||
| ### No Data Displayed | ||||||
|
|
||||||
| **Problem**: Graphs are empty or show no data | ||||||
|
|
||||||
| **Solution**: | ||||||
|
|
||||||
| - Verify that `log_tensor_data_content: true` is in the YAML config | ||||||
| - Verify that the operator names match between the app and visualizer | ||||||
| - For static visualizer, ensure that the unique IDs in the code match your operators | ||||||
| - For dynamic visualizer, wait a few seconds for auto-discovery | ||||||
Uh oh!
There was an error while loading. Please reload this page.