diff --git a/data/models/ColonSegNet-07112023-2359.onnx b/data/models/ColonSegNet-07112023-2359.onnx index fa4112c..3ebf798 100644 Binary files a/data/models/ColonSegNet-07112023-2359.onnx and b/data/models/ColonSegNet-07112023-2359.onnx differ diff --git a/data/models/README.md b/data/models/README.md index c32c0f4..cf30bcb 100644 --- a/data/models/README.md +++ b/data/models/README.md @@ -4,13 +4,12 @@ ### Trained models -| Model (Size) | Download | Notebook | +| Model (Size) | Download | Notebook | | -- |-- | -- | | ColonSegNet-07112023-2359.onnx (20MB or 20,067,028) | [:link:](TOADD) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1s-eKu6QjaV54jva_ylG1VxwCj1B8nwJ3) -| Geometric Tx | [:link:](https://drive.google.com/file/d/1vE023Thky0vKfoYD7wZecrglHEgp3z-W/view?usp=sharing) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1s-eKu6QjaV54jva_ylG1VxwCj1B8nwJ3) | -| Baseline model 4 epochs | [:link:](https://drive.google.com/file/d/1SWPzOeqKyKqBM9FVfj0qCPn8cxEhX9BK/view?usp=drive_link) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/TOADD) | -Add Your model (copy and paste this line) | [:link:](TOADD) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/TOADD) | - +| Geometric Tx | [:link:](https://drive.google.com/file/d/1vE023Thky0vKfoYD7wZecrglHEgp3z-W/view?usp=sharing) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1s-eKu6QjaV54jva_ylG1VxwCj1B8nwJ3) | +| Baseline model 4 epochs | [:link:](https://drive.google.com/file/d/1SWPzOeqKyKqBM9FVfj0qCPn8cxEhX9BK/view?usp=drive_link) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/TOADD) | +| Add Your Model by creation a new row | [:link:](COPY&PASTE_DRIVE_LINK) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/COPY&PASTE_DRIVE_LINK) | ### Examples to download models in your local machine * ColonSegNet-07112023-2359.onnx diff --git a/data/polyp-dataset/README.md b/data/polyp-dataset/README.md index 3c59e64..3e20abb 100644 --- a/data/polyp-dataset/README.md +++ b/data/polyp-dataset/README.md @@ -1,7 +1,6 @@ # Kvasir-SEG dataset > The Kvasir-SEG dataset (size 46.2 MB) contains 1000 polyp images and their corresponding ground truth from the Kvasir Dataset v2. The resolution of the images contained in Kvasir-SEG varies from 332x487 to 1920x1072 pixels. The images and its corresponding masks are stored in two separate folders with the same filename. The image files are encoded using JPEG compression, and online browsing is facilitated. The bounding box (coordinate points) for the corresponding images are stored in a JSON file. https://datasets.simula.no/downloads/kvasir-seg.zip - ``` wget https://datasets.simula.no/downloads/kvasir-seg.zip #(45M) unzip kvasir-seg.zip @@ -9,33 +8,80 @@ rm kvasir-seg.zip ``` ## Converting images to video +* converting ck images to video ``` cd data/polyp-dataset/Kvasir-SEG/images -ffmpeg -framerate 1 -pattern_type glob -i 'Kvasir-SEG/images/ck*.jpg' -c:v libx264 out720x576.mp4 +ffmpeg -framerate 30 -pattern_type glob -i 'Kvasir-SEG/images/ck*.jpg' -c:v libx264 out720x576.mp4 +``` +* converting selected images to video +``` + #cp images/ck2bxknhjvs1x0794iogrq49k.jpg selected/ + #cp images/cju16jgnyzp970878melv7r25.jpg selected/ + #cp images/cju33w4sdcivk0855x879zht7.jpg selected/ + #cp images/cju85omszllp30850b6rm9mi3.jpg selected/ +ffmpeg -framerate 30 -pattern_type glob -i 'Kvasir-SEG/selected-512x/*.jpg' -c:v libx264 out720x576-selection512x.mp4 +``` + +* converting resized selected images to video ``` + #convert cju16jgnyzp970878melv7r25.jpg -resize 512x cju16jgnyzp970878melv7r25.jpg + #convert cju33w4sdcivk0855x879zht7.jpg -resize 512x cju33w4sdcivk0855x879zht7.jpg + #convert cju85omszllp30850b6rm9mi3.jpg -resize 512x cju85omszllp30850b6rm9mi3.jpg + #convert ck2bxknhjvs1x0794iogrq49k.jpg -resize 512x ck2bxknhjvs1x0794iogrq49k.jpg +ffmpeg -framerate 30 -pattern_type glob -i 'Kvasir-SEG/selected-512x/*.jpg' -c:v libx264 out720x576-selection512x.mp4 +``` +* converting one selected image to video +``` +cd $HOME/repositories/scikit-surgery/cmicHACKS2/data/polyp-dataset/Kvasir-SEG/ +cp -r selected selected-ck2bxknhjvs1x0794iogrq49k-512x-01-frames #same for other frames +cp -r selected selected-ck2bxknhjvs1x0794iogrq49k-512x-05-frames #and create copies of the same image +convert *.jpg -resize 512x *.jpg #only height +convert *.jpg -resize 512x512! *.jpg #hegith and width + +cd $HOME/repositories/scikit-surgery/cmicHACKS2/data/polyp-dataset + +#ffmpeg -framerate 30 -pattern_type glob -i 'Kvasir-SEG/selected-ck2bxknhjvs1x0794iogrq49k-512x-01-frames/*.jpg' -c:v libx264 out512x448-selected-ck2bxknhjvs1x0794iogrq49k-01-frames.mp4 #don't work 4 1frame +ffmpeg -framerate 30 -pattern_type glob -i 'Kvasir-SEG/selected-ck2bxknhjvs1x0794iogrq49k-05-frames/*.jpg' -c:v libx264 -vf "crop=trunc(iw/2)*2:trunc(ih/2)*2" out602x528-selected-ck2bxknhjvs1x0794iogrq49k-05-frames.mp4 +ffmpeg -framerate 30 -pattern_type glob -i 'Kvasir-SEG/selected-ck2bxknhjvs1x0794iogrq49k-512x-05-frames/*.jpg' -c:v libx264 -vf "crop=trunc(iw/2)*2:trunc(ih/2)*2" out512x448-selected-ck2bxknhjvs1x0794iogrq49k-05-frames.mp4 +ffmpeg -framerate 30 -pattern_type glob -i 'Kvasir-SEG/selected-ck2bxknhjvs1x0794iogrq49k-512x-512x-05-frames/*.jpg' -c:v libx264 -vf "crop=trunc(iw/2)*2:trunc(ih/2)*2" out512x512-selected-ck2bxknhjvs1x0794iogrq49k-05-frames.mp4 -## Using `convert_video_to_gxf_entities.py` (Graph Execution Framework, GXF) + +## check video +vlc out602x528-selected-ck2bxknhjvs1x0794iogrq49k-05-frames.mp4 + +``` + +References: +* https://www.networkworld.com/article/3697731/resizing-images-on-the-linux-command-line.html +* https://stackoverflow.com/questions/20847674/ffmpeg-libx264-height-not-divisible-by-2 + +## Using `ffmeg` and `convert_video_to_gxf_entities.py` (Graph Execution Framework, GXF) ``` mamba activate cagxVE +#ffmpeg -i *.mp4 -pix_fmt rgb24 -f rawvideo pipe:1 | python ../../nvidia-clara-agx/holoscan-sdk-scripts/convert_video_to_gxf_entities.py --width 512 --height 484 --channels 3 --framerate 30 --basename * #ffmpeg -i out.mp4 -pix_fmt rgb24 -f rawvideo pipe:1 | python convert_video_to_gxf_entities.py --width 1920 --height 1080 --channels 3 --framerate 1 --basename out #ffmpeg -i out.mp4 -pix_fmt yuvj422p -f rawvideo pipe:1 | python convert_video_to_gxf_entities.py --width 600 --height 500 --channels 3 --framerate 1 --basename out -ffmpeg -i out720x576.mp4 -pix_fmt rgb24 -f rawvideo pipe:1 | python ../../nvidia-clara-agx/holoscan-sdk-scripts/convert_video_to_gxf_entities.py --width 720 --height 576 --channels 3 --framerate 1 --basename out720x576 +ffmpeg -i out720x576.mp4 -pix_fmt rgb24 -f rawvideo pipe:1 | python ../../nvidia-clara-agx/holoscan-sdk-scripts/convert_video_to_gxf_entities.py --width 720 --height 576 --channels 3 --framerate 30 --basename out720x576 + +ffmpeg -i out720x576-selection512x.mp4 -pix_fmt rgb24 -f rawvideo pipe:1 | python ../../nvidia-clara-agx/holoscan-sdk-scripts/convert_video_to_gxf_entities.py --width 512 --height 484 --channels 3 --framerate 30 --basename out720x576-selection512x + + +ffmpeg -i out602x528-selected-ck2bxknhjvs1x0794iogrq49k-05-frames.mp4 -pix_fmt rgb24 -f rawvideo pipe:1 | python ../../nvidia-clara-agx/holoscan-sdk-scripts/convert_video_to_gxf_entities.py --width 602 --height 528 --channels 3 --framerate 30 --basename out602x528-selected-ck2bxknhjvs1x0794iogrq49k-05-frames + + +ffmpeg -i out512x448-selected-ck2bxknhjvs1x0794iogrq49k-05-frames.mp4 -pix_fmt rgb24 -f rawvideo pipe:1 | python ../../nvidia-clara-agx/holoscan-sdk-scripts/convert_video_to_gxf_entities.py --width 602 --height 528 --channels 3 --framerate 30 --basename out512x448-selected-ck2bxknhjvs1x0794iogrq49k-05-frames + +ffmpeg -i out512x512-selected-ck2bxknhjvs1x0794iogrq49k-05-frames.mp4 -pix_fmt rgb24 -f rawvideo pipe:1 | python ../../nvidia-clara-agx/holoscan-sdk-scripts/convert_video_to_gxf_entities.py --width 602 --height 528 --channels 3 --framerate 30 --basename out512x512-selected-ck2bxknhjvs1x0794iogrq49k-05-frames + + ``` https://github.com/nvidia-holoscan/holoscan-sdk/tree/main/scripts#convert_video_to_gxf_entitiespy https://docs.nvidia.com/holoscan/sdk-user-guide/gxf/gxf_by_example.html https://docs.nvidia.com/clara-holoscan/archive/clara-holoscan-0.3.0/gxf/index.html -## Coping files to clara-agx -``` -cd /workspace/holohub/data/colonoscopy_segmentation -cp ../../cmicHACKS2/data/polyp-dataset/out720x576.mp4 . -cp ../../cmicHACKS2/data/polyp-dataset/out720x576.gxf_* . -``` - - ## Testing `raw.mp4` with `convert_video_to_gxf_entities.py` @@ -52,25 +98,9 @@ ffmpeg -i colon_exam_720x576.mp4 -pix_fmt rgb24 -f rawvideo pipe:1 | python scri ``` -errors - -``` - -av_interleaved_write_frame(): Broken pipe -Error writing trailer of pipe:1: Broken pipeime=00:00:00.04 bitrate=248832.0kbits/s speed=1.83x -frame= 1 fps=0.0 q=-0.0 Lsize= 1215kB time=00:00:00.04 bitrate=248832.0kbits/s speed=1.22x -video:1215kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000% -Conversion failed! - -``` - https://catalog.ngc.nvidia.com/orgs/nvidia/teams/clara-holoscan/resources/holoscan_endoscopy_sample_data/files?version=20221121 - - ## References https://stackoverflow.com/questions/22965569/convert-from-jpg-to-mp4-by-ffmpeg https://docs.nvidia.com/holoscan/sdk-user-guide/gxf/gxf_by_example.html - - diff --git a/data/polyp-dataset/out512x448-selected-ck2bxknhjvs1x0794iogrq49k-05-frames.gxf_entities b/data/polyp-dataset/out512x448-selected-ck2bxknhjvs1x0794iogrq49k-05-frames.gxf_entities new file mode 100644 index 0000000..ee65a1d Binary files /dev/null and b/data/polyp-dataset/out512x448-selected-ck2bxknhjvs1x0794iogrq49k-05-frames.gxf_entities differ diff --git a/data/polyp-dataset/out512x448-selected-ck2bxknhjvs1x0794iogrq49k-05-frames.gxf_index b/data/polyp-dataset/out512x448-selected-ck2bxknhjvs1x0794iogrq49k-05-frames.gxf_index new file mode 100644 index 0000000..b412f72 Binary files /dev/null and b/data/polyp-dataset/out512x448-selected-ck2bxknhjvs1x0794iogrq49k-05-frames.gxf_index differ diff --git a/data/polyp-dataset/out512x448-selected-ck2bxknhjvs1x0794iogrq49k-05-frames.mp4 b/data/polyp-dataset/out512x448-selected-ck2bxknhjvs1x0794iogrq49k-05-frames.mp4 new file mode 100644 index 0000000..c854b12 Binary files /dev/null and b/data/polyp-dataset/out512x448-selected-ck2bxknhjvs1x0794iogrq49k-05-frames.mp4 differ diff --git a/data/polyp-dataset/out512x512-selected-ck2bxknhjvs1x0794iogrq49k-05-frames.gxf_entities b/data/polyp-dataset/out512x512-selected-ck2bxknhjvs1x0794iogrq49k-05-frames.gxf_entities new file mode 100644 index 0000000..6d7ef7c Binary files /dev/null and b/data/polyp-dataset/out512x512-selected-ck2bxknhjvs1x0794iogrq49k-05-frames.gxf_entities differ diff --git a/data/polyp-dataset/out512x512-selected-ck2bxknhjvs1x0794iogrq49k-05-frames.gxf_index b/data/polyp-dataset/out512x512-selected-ck2bxknhjvs1x0794iogrq49k-05-frames.gxf_index new file mode 100644 index 0000000..db22cb5 Binary files /dev/null and b/data/polyp-dataset/out512x512-selected-ck2bxknhjvs1x0794iogrq49k-05-frames.gxf_index differ diff --git a/data/polyp-dataset/out512x512-selected-ck2bxknhjvs1x0794iogrq49k-05-frames.mp4 b/data/polyp-dataset/out512x512-selected-ck2bxknhjvs1x0794iogrq49k-05-frames.mp4 new file mode 100644 index 0000000..29ee83c Binary files /dev/null and b/data/polyp-dataset/out512x512-selected-ck2bxknhjvs1x0794iogrq49k-05-frames.mp4 differ diff --git a/data/polyp-dataset/out602x528-selected-ck2bxknhjvs1x0794iogrq49k-05-frames.gxf_entities b/data/polyp-dataset/out602x528-selected-ck2bxknhjvs1x0794iogrq49k-05-frames.gxf_entities new file mode 100644 index 0000000..71d3878 Binary files /dev/null and b/data/polyp-dataset/out602x528-selected-ck2bxknhjvs1x0794iogrq49k-05-frames.gxf_entities differ diff --git a/data/polyp-dataset/out602x528-selected-ck2bxknhjvs1x0794iogrq49k-05-frames.gxf_index b/data/polyp-dataset/out602x528-selected-ck2bxknhjvs1x0794iogrq49k-05-frames.gxf_index new file mode 100644 index 0000000..10f734c Binary files /dev/null and b/data/polyp-dataset/out602x528-selected-ck2bxknhjvs1x0794iogrq49k-05-frames.gxf_index differ diff --git a/data/polyp-dataset/out602x528-selected-ck2bxknhjvs1x0794iogrq49k-05-frames.mp4 b/data/polyp-dataset/out602x528-selected-ck2bxknhjvs1x0794iogrq49k-05-frames.mp4 new file mode 100644 index 0000000..f9b841e Binary files /dev/null and b/data/polyp-dataset/out602x528-selected-ck2bxknhjvs1x0794iogrq49k-05-frames.mp4 differ diff --git a/data/polyp-dataset/out720x576-selection512x.gxf_entities b/data/polyp-dataset/out720x576-selection512x.gxf_entities new file mode 100644 index 0000000..41d87e8 Binary files /dev/null and b/data/polyp-dataset/out720x576-selection512x.gxf_entities differ diff --git a/data/polyp-dataset/out720x576-selection512x.gxf_index b/data/polyp-dataset/out720x576-selection512x.gxf_index new file mode 100644 index 0000000..d8e680a Binary files /dev/null and b/data/polyp-dataset/out720x576-selection512x.gxf_index differ diff --git a/data/polyp-dataset/out720x576-selection512x.mp4 b/data/polyp-dataset/out720x576-selection512x.mp4 new file mode 100644 index 0000000..74a5f93 Binary files /dev/null and b/data/polyp-dataset/out720x576-selection512x.mp4 differ diff --git a/data/polyp-dataset/out720x576.gxf_entities b/data/polyp-dataset/out720x576.gxf_entities index 1f9e740..36e58b5 100644 Binary files a/data/polyp-dataset/out720x576.gxf_entities and b/data/polyp-dataset/out720x576.gxf_entities differ diff --git a/data/polyp-dataset/out720x576.gxf_index b/data/polyp-dataset/out720x576.gxf_index index 050dae1..bb56ead 100644 Binary files a/data/polyp-dataset/out720x576.gxf_index and b/data/polyp-dataset/out720x576.gxf_index differ diff --git a/data/polyp-dataset/out720x576.mp4 b/data/polyp-dataset/out720x576.mp4 index 46586a0..3d27176 100644 Binary files a/data/polyp-dataset/out720x576.mp4 and b/data/polyp-dataset/out720x576.mp4 differ diff --git a/nvidia-clara-agx/colonoscopy_segmentation/README.md b/nvidia-clara-agx/colonoscopy_segmentation/README.md index 87f5595..7b45215 100644 --- a/nvidia-clara-agx/colonoscopy_segmentation/README.md +++ b/nvidia-clara-agx/colonoscopy_segmentation/README.md @@ -16,6 +16,11 @@ cd /workspace/holohub/data/colonoscopy_segmentation cp ../../cmicHACKS2/data/models/ColonSegNet-07112023-2359.onnx . cp ../../cmicHACKS2/data/polyp-dataset/out720x576.mp4 . cp ../../cmicHACKS2/data/polyp-dataset/out720x576.gxf_* . + + +cp ../../cmicHACKS2/data/polyp-dataset/*gxf_entities . +cp ../../cmicHACKS2/data/polyp-dataset/*gxf_index . +cp ../../cmicHACKS2/data/polyp-dataset/*mp4 . ``` @@ -44,9 +49,7 @@ python colonoscopy_segmentation.py --data /workspace/holohub/data/colonoscopy_se ``` - - -* log +* Terminal log ``` root@cagx-ucl:/workspace/holohub# ./run launch colonoscopy_segmentation @@ -185,9 +188,39 @@ NOTE: This could take a couple of minutes depending on your model size and GPU! [info] [gxf_executor.cpp:1787] Graph execution finished. Fragment: Colonoscopy App [info] [gxf_executor.cpp:229] Destroying context +``` +## Screenshots of `colonoscopy_segmentation.py` +* Using `colon.onnx` with replayer:baseline `out602x528-selected-ck2bxknhjvs1x0794iogrq49k-05-frames` +![fig](Screenshot%20from%202023-11-15%2008-31-11.png) +inference_output_tensor for one frame +``` +tensor.min()=-11.6~ +tensor.max()=10.7~ +tensor.mean()=-5.2~ +tensor.shape()=(1,1,512,512) +``` + +* Using `ColonSegNet-07112023-2359.onnx` with replayer:baseline `out602x528-selected-ck2bxknhjvs1x0794iogrq49k-05-frames` +![fig](Screenshot%20from%202023-11-15%2008-23-30.png) +inference_output_tensor for one frame +``` +tensor.min()=-12.3~ +tensor.max()=2.2~ +tensor.mean()=-6.0~ +tensor.shape()=(1,1,512,512) +``` + +* Using `colon.onnx` with replayer:baseline `out512x448-selected-ck2bxknhjvs1x0794iogrq49k-05-frames` +![fig](Screenshot%20from%202023-11-15%2008-35-29.png) +inference_output_tensor for one frame +``` +tensor.min()=-10.6~ +tensor.max()=2.2~ +tensor.mean()=-6.7~ +tensor.shape()=(1,1,512,512) ``` -## Optmise model +## Optmisation of model ``` [info] [core.cpp:46] TRT Inference: converting ONNX model at /workspace/holohub/data/colonoscopy_segmentation/colon.onnx diff --git a/nvidia-clara-agx/colonoscopy_segmentation/Screenshot from 2023-11-15 08-23-30.png b/nvidia-clara-agx/colonoscopy_segmentation/Screenshot from 2023-11-15 08-23-30.png new file mode 100644 index 0000000..0505650 Binary files /dev/null and b/nvidia-clara-agx/colonoscopy_segmentation/Screenshot from 2023-11-15 08-23-30.png differ diff --git a/nvidia-clara-agx/colonoscopy_segmentation/Screenshot from 2023-11-15 08-31-11.png b/nvidia-clara-agx/colonoscopy_segmentation/Screenshot from 2023-11-15 08-31-11.png new file mode 100644 index 0000000..c128403 Binary files /dev/null and b/nvidia-clara-agx/colonoscopy_segmentation/Screenshot from 2023-11-15 08-31-11.png differ diff --git a/nvidia-clara-agx/colonoscopy_segmentation/Screenshot from 2023-11-15 08-35-29.png b/nvidia-clara-agx/colonoscopy_segmentation/Screenshot from 2023-11-15 08-35-29.png new file mode 100644 index 0000000..d246feb Binary files /dev/null and b/nvidia-clara-agx/colonoscopy_segmentation/Screenshot from 2023-11-15 08-35-29.png differ diff --git a/nvidia-clara-agx/colonoscopy_segmentation/colonoscopy_segmentation.py b/nvidia-clara-agx/colonoscopy_segmentation/colonoscopy_segmentation.py index 4705f88..28e5ecd 100644 --- a/nvidia-clara-agx/colonoscopy_segmentation/colonoscopy_segmentation.py +++ b/nvidia-clara-agx/colonoscopy_segmentation/colonoscopy_segmentation.py @@ -16,7 +16,7 @@ import os from argparse import ArgumentParser -from holoscan.core import Application +from holoscan.core import Application, Operator, OperatorSpec from holoscan.operators import ( AJASourceOp, FormatConverterOp, @@ -25,9 +25,39 @@ SegmentationPostprocessorOp, VideoStreamReplayerOp, ) -from holoscan.resources import BlockMemoryPool, CudaStreamPool, MemoryStorageType +from holoscan.gxf import Entity +from holoscan.resources import BlockMemoryPool, CudaStreamPool, MemoryStorageType, UnboundedAllocator +import cupy as cp +import holoscan as hs +class InfoOp(Operator): + """""" + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + + def setup(self, spec: OperatorSpec): + spec.input("in") + spec.output("out") + + def compute(self, op_input, op_output, context): + # Get input message + in_message = op_input.receive("in") + + # Transpose + tensor = cp.asarray(in_message.get("inference_output_tensor")) + print(f"tensor.min()={cp.min(tensor)}") + print(f"tensor.max()={cp.max(tensor)}") + print(f"tensor.mean()={cp.mean(tensor)}") + print(f"tensor.shape={tensor.shape}") + + # Create output message + out_message = Entity(context) + out_message.add(hs.as_tensor(tensor), "inference_output_tensor") + op_output.emit(out_message, "out") + + class ColonoscopyApp(Application): def __init__(self, data, source="replayer"): """Initialize the colonoscopy segmentation application @@ -54,8 +84,9 @@ def __init__(self, data, source="replayer"): self.sample_data_path = data self.model_path_map = { - "ultrasound_seg": os.path.join(self.sample_data_path, "ColonSegNet-07112023-2359.onnx"), #"ultrasound_seg": os.path.join(self.sample_data_path, "colon.onnx"), + "ultrasound_seg": os.path.join(self.sample_data_path, "ColonSegNet-07112023-2359.onnx"), + #"ultrasound_seg": os.path.join(self.sample_data_path, "ColonSegNet_brightcontr.onnx"), } def compose(self): @@ -143,6 +174,12 @@ def compose(self): transmit_on_cuda=True, ) + info_op = InfoOp( + self, + name="info", + pool=UnboundedAllocator(self, name="pool"), + ) + postprocessor_block_size = width_inference * height_inference postprocessor_num_blocks = 2 segmentation_postprocessor = SegmentationPostprocessorOp( @@ -172,7 +209,9 @@ def compose(self): self.add_flow(source, segmentation_visualizer, {("", "receivers")}) self.add_flow(source, segmentation_preprocessor) self.add_flow(segmentation_preprocessor, segmentation_inference, {("", "receivers")}) - self.add_flow(segmentation_inference, segmentation_postprocessor, {("transmitter", "")}) + self.add_flow(segmentation_inference, info_op, {("transmitter", "in")}) + self.add_flow(info_op, segmentation_postprocessor, {("out", "")}) + #self.add_flow(segmentation_inference, segmentation_postprocessor, {("transmitter", "")}) self.add_flow( segmentation_postprocessor, segmentation_visualizer, diff --git a/nvidia-clara-agx/colonoscopy_segmentation/colonoscopy_segmentation.yaml b/nvidia-clara-agx/colonoscopy_segmentation/colonoscopy_segmentation.yaml index 0bcd42a..47b2155 100644 --- a/nvidia-clara-agx/colonoscopy_segmentation/colonoscopy_segmentation.yaml +++ b/nvidia-clara-agx/colonoscopy_segmentation/colonoscopy_segmentation.yaml @@ -16,7 +16,11 @@ --- replayer: # VideoStreamReplayer #basename: "colon_exam_720x576" - basename: "out720x576" + #basename: "out720x576" + #basename: "out720x576-selection512x" + #basename: "out512x448-selected-ck2bxknhjvs1x0794iogrq49k-05-frames"#not-rendered-properly + #basename: "out512x512-selected-ck2bxknhjvs1x0794iogrq49k-05-frames"#not-rendered-properly + basename: "out602x528-selected-ck2bxknhjvs1x0794iogrq49k-05-frames" frame_rate: 0 # as specified in timestamps repeat: true # default: false realtime: true # default: true @@ -45,6 +49,8 @@ segmentation_postprocessor: # Postprocessor data_format: nchw segmentation_visualizer: # Holoviz + height: 512 + width: 512 color_lut: [ [0.65, 0.81, 0.89, 0.1], [0.2, 0.63, 0.17, 0.7],