The chosen model is 3D-Unet in nnUnet performing BraTS 2019 brain tumor segmentation task.
If you would like to run on NVIDIA GPU, you will need:
- nvidia-docker
- Any NVIDIA GPU supported by TensorFlow or PyTorch
| model | framework | accuracy | dataset | model link | model source | precision | notes |
|---|---|---|---|---|---|---|---|
| 3D-Unet | PyTorch | mean = 0.85300 (whole tumor = 0.9141, tumor core = 0.8679, enhancing tumor = 0.7770) | Fold 1 of BraTS 2019 Training Dataset | from zenodo | Trained in PyTorch using codes from nnUnet on Fold 0, Fold 2, Fold 3, and Fold 4 of BraTS 2019 Training Dataset. | fp32 | |
| 3D-Unet | ONNX | mean = 0.85300 (whole tumor = 0.9141, tumor core = 0.8679, enhancing tumor = 0.7770) | Fold 1 of BraTS 2019 Training Dataset | from zenodo | Converted from the PyTorch model using script. | fp32 | |
| 3D-Unet | Tensorflow | mean = 0.85300 (whole tumor = 0.9141, tumor core = 0.8679, enhancing tumor = 0.7770) | Fold 1 of BraTS 2019 Training Dataset | from zenodo | Converted from the ONNX model using script. | fp32 | |
| 3D-Unet | OpenVINO | mean = 0.85300 (whole tumor = 0.9141, tumor core = 0.8679, enhancing tumor = 0.7770) | Fold 1 of BraTS 2019 Training Dataset | from zenodo | Converted from the ONNX model. | fp32 |
This benchmark app is a reference implementation that is not meant to be the fastest implementation possible.
Please download BraTS 2019 separately and unzip the dataset.
Please run the following commands:
export DOWNLOAD_DATA_DIR=<path/to/MICCAI_BraTS_2019_Data_Training>: point to location of downloaded BraTS 2019 Training dataset.make setup: initialize submodule and download models.make build_docker: build docker image.make launch_docker: launch docker container with an interaction session.make preprocess_data: preprocess the BraTS 2019 dataset.python3 run.py --backend=[tf|pytorch|onnxruntime|ov] --scenario=[Offline|SingleStream|MultiStream|Server] [--accuracy] --model=[path/to/model_file(tf/onnx/OpenVINO only)]: run the harness inside the docker container. Performance or Accuracy results will be printed in console.python3 accuracy-brats.py --log_file=<LOADGEN_LOG> --output_dtype=<DTYPE>: compute accuracy from a LoadGen accuracy JSON log file.
- SUT implementations are in ov_SUT.py, pytorch_SUT.py, onnxruntime_SUT.py, and tf_SUT.py. QSL implementation is in brats_QSL.py.
- The script accuracy-brats.py parses LoadGen accuracy log, post-processes it, and computes the accuracy.
- Preprocessing and evaluation (including post-processing) are not included in the timed path.
- The input to the SUT is a volume of size
[4, 224, 224, 160]. The output from SUT is a volume of size[4, 224, 224, 160]with predicted label logits for each voxel.
The calibration set is the forty images listed in brats_cal_images_list.txt. They are randomly selected from Fold 0, Fold 2, Fold 3, and Fold 4 of BraTS 2019 Training Dataset.
Apache License 2.0