Skip to content

Commit

Permalink
Merge pull request #23 from MStarmans91/development
Browse files Browse the repository at this point in the history
Release 3.1.4
  • Loading branch information
MStarmans91 authored May 22, 2020
2 parents dd2d693 + d50b258 commit 0296bc0
Show file tree
Hide file tree
Showing 228 changed files with 9,049 additions and 2,824 deletions.
7 changes: 7 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -125,3 +125,10 @@ dmypy.json

# Pyre type checker
.pyre/

# Example data
WORC/exampledata/*.hdf5
WORC/external/*
WORC/exampledata/ICCvalues.csv
WORC/tests/*.png
WORC/tests/*.mat
4 changes: 3 additions & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,9 @@ matrix:
# 'python3' is a 'command not found' error on Windows but 'py' works on Windows only.
script:
- python WORCTutorial/WORCTutorialSimple.py
- fastr trace /tmp/WORC_Example_STWStrategyHN/__sink_data__.json --sinks features_train_CT_0 --samples HN1331
- fastr trace /tmp/WORC_Example_STWStrategyHN/__sink_data__.json --sinks features_train_CT_0_predict --samples HN1331
- fastr trace /tmp/WORC_Example_STWStrategyHN/__sink_data__.json --sinks features_train_CT_0_pyradiomics --samples HN1331
- fastr trace /tmp/WORC_Example_STWStrategyHN/__sink_data__.json --sinks classification --samples all
# - name: "MacOS"
# os: osx
# osx_image: xcode11.2 # Python 3.7.4 running on macOS 10.14.4
Expand Down
56 changes: 56 additions & 0 deletions CHANGELOG
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,62 @@ All notable changes to this project will be documented in this file.
The format is based on `Keep a Changelog <http://keepachangelog.com/>`_
and this project adheres to `Semantic Versioning <http://semver.org/>`_

3.1.4 - 2020-05-22
------------------

Added
~~~~~~~
- Catch error if number of segmentations supplied does not match number of
images.
- Add support in SimpleWORC and BasicWORC for multiple segmentations per
patient.
- Chi2 test in statistical testing.
- fastr tool to make boxplots of all features, overall and per class.
- Added this boxplot tool to the evaluate workflow.
- Option in evaluation to overfit feature scaling to test set: should only
be used to assess differences between the training and test sets, not
in an actual model.
- Option to delete small objects in segmentation.
- Option to within the preprocessing, use a dilated ROI.
- Otsu thresholding as mask for preprocessing.
- Memory for each fastr node is now in a dictionary of the WORC object and can
be easily changed.
- PyRadiomics now fully embedded and configurable.
- ComBat harmonization: currently as separate tool on full dataset,
not in cross-validation.
- Computation of ICC, and thresholding object to use ICC for feature selection.
- Added groupwise feature selection per feature extraction toolbox.
- Feature converter tool, to convert features from a toolbox to WORC compatible
format.
- RobustScaler for feature scaling.
- Decomposition to evaluate network.
- Combat: in WORK workflow.

Changed
~~~~~~~
- Resampling of objects is now after feature selection.
- Made plot_SVM function more memory efficient.
- For PCA, Relief, VarianceThreshold, and SelectFromModel feature selection,
you can now simply supply a float to determine the percentage of times
this method is used in the created workflows.
- Moved load_features from trainclassifier to file_io.
- Matching of PID from labels from label file to other objects is now all
converted to lower case.
- Refactoring of WORC network building.
- Segmentix tool is cleaned up. Segmentix script is moved to processing.

Fixed
~~~~~
- Order of methods in preprocessing function of SearchCV did not correspond
with that in fitandscore.
- Replace spaces in uri conversion of sources in SimpleWORC.
- Check whether all fitandscore jobs succeeded, otherwise throw error.
- Bug in PCA when n_components > min(n_samples, n_features)
- Random seed is now set and passed to PCA, Relief and all classifiers
for reproducability of the results.
- Evaluate can now also accept multiple feature toolboxes.


3.1.3 - 2020-01-24
------------------

Expand Down
24 changes: 21 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,25 @@
[![Build Status](https://travis-ci.com/MStarmans91/WORC.svg?token=qyvaeq7Cpwu7hJGB98Gp&branch=master)](https://travis-ci.com/MStarmans91/WORC)
# WORC v3.1.4
## Workflow for Optimal Radiomics Classification

# WORC v3.1.3
## Information

## Workflow for Optimal Radiomics Classification
| Linux | Windows | Documentation | PyPi |
|--------------------------------|-------------------------------|-------------------------------|-------------------------------|
| [![][tci-linx]][tci-linx-lnk] | [![][tci-wind]][tci-wind-lnk] | [![][doc]][doc-lnk] | [![][pypi]][pypi-lnk] |

[tci-linx]: https://travis-ci.com/MStarmans91/WORC.svg?token=qyvaeq7Cpwu7hJGB98Gp&branch=master&job=1
[tci-linx-lnk]: https://travis-ci.com/MStarmans91/WORC

[tci-wind]: https://travis-ci.com/MStarmans91/WORC.svg?token=qyvaeq7Cpwu7hJGB98Gp&branch=master&job=2
[tci-wind-lnk]: https://travis-ci.com/MStarmans91/WORC

[doc]:https://readthedocs.org/projects/worc/badge/?version=latest
[doc-lnk]: https://worc.readthedocs.io/en/latest/?badge=latest

[pypi]: https://badge.fury.io/py/WORC.svg
[pypi-lnk]: https://badge.fury.io/py/WORC

# Introduction

WORC is an open-source python package for the easy execution of full radiomics pipelines.

Expand Down Expand Up @@ -52,6 +69,7 @@ NOTE: The version of PyRadiomics which WORC currently uses requires numpy to be
- SimpleITK (Image loading and preprocessing)
- [Pyradiomics](https://github.com/radiomics/pyradiomics)
- [PREDICT](https://github.com/Svdvoort/PREDICTFastr)
- [ComBat](https://github.com/Jfortin1/ComBatHarmonization) (optional)

See for other python packages the [requirements file](requirements.txt).

Expand Down
26 changes: 22 additions & 4 deletions README.rst
Original file line number Diff line number Diff line change
@@ -1,11 +1,21 @@
|Build Status|

WORC v3.1.3
WORC v3.1.4
===========

Workflow for Optimal Radiomics Classification
---------------------------------------------

Information
-----------

+--------------------+-------------------+-------------------+-------------------+
| Linux | Windows | Documentation | PyPi |
+====================+===================+===================+===================+
| |image0| | |image1| | |image2| | |image3| |
+--------------------+-------------------+-------------------+-------------------+

Introduction
============

WORC is an open-source python package for the easy execution of full
radiomics pipelines.

Expand Down Expand Up @@ -77,6 +87,8 @@ numpy to be installed beforehand. Make sure you do so, e.g.
- SimpleITK (Image loading and preprocessing)
- `Pyradiomics <https://github.com/radiomics/pyradiomics>`__
- `PREDICT <https://github.com/Svdvoort/PREDICTFastr>`__
- `ComBat <https://github.com/Jfortin1/ComBatHarmonization>`__
(optional)

See for other python packages the `requirements
file <requirements.txt>`__.
Expand Down Expand Up @@ -163,5 +175,11 @@ to XNAT. We advise you to specify your account settings in a .netrc file
when using this feature for your own datasets, such that you do not need
to input them on every request.

.. |Build Status| image:: https://travis-ci.com/MStarmans91/WORC.svg?token=qyvaeq7Cpwu7hJGB98Gp&branch=master
.. |image0| image:: https://travis-ci.com/MStarmans91/WORC.svg?token=qyvaeq7Cpwu7hJGB98Gp&branch=master&job=1
:target: https://travis-ci.com/MStarmans91/WORC
.. |image1| image:: https://travis-ci.com/MStarmans91/WORC.svg?token=qyvaeq7Cpwu7hJGB98Gp&branch=master&job=2
:target: https://travis-ci.com/MStarmans91/WORC
.. |image2| image:: https://readthedocs.org/projects/worc/badge/?version=latest
:target: https://worc.readthedocs.io/en/latest/?badge=latest
.. |image3| image:: https://badge.fury.io/py/WORC.svg
:target: https://badge.fury.io/py/WORC
14 changes: 14 additions & 0 deletions WORC.iml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
<?xml version="1.0" encoding="UTF-8"?>
<module type="WEB_MODULE" version="4">
<component name="FacetManager">
<facet type="Python" name="Python">
<configuration sdkName="" />
</facet>
</component>
<component name="NewModuleRootManager" inherit-compiler-output="true">
<exclude-output />
<content url="file://$MODULE_DIR$" />
<orderEntry type="inheritedJdk" />
<orderEntry type="sourceFolder" forTests="false" />
</component>
</module>
5 changes: 3 additions & 2 deletions WORC/IOparser/config_WORC.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,8 +43,9 @@ def load_config(config_file_path):
settings_dict['ImageFeatures']['image_type'] =\
str(settings['ImageFeatures']['image_type'])

settings_dict['General']['FeatureCalculator'] =\
str(settings['General']['FeatureCalculator'])
settings_dict['General']['FeatureCalculators'] =\
[str(item).strip() for item in
settings['General']['FeatureCalculators'].split(',')]

settings_dict['General']['Preprocessing'] =\
str(settings['General']['Preprocessing'])
Expand Down
188 changes: 188 additions & 0 deletions WORC/IOparser/config_io_PyRadiomics.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,188 @@
#!/usr/bin/env python

# Copyright 2016-2020 Biomedical Imaging Group Rotterdam, Departments of
# Medical Informatics and Radiology, Erasmus MC, Rotterdam, The Netherlands
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import configparser
import numpy as np
import os
import PREDICT.addexceptions as ae


def load_config(config_file_path):
if not os.path.exists(config_file_path):
e = f'File {config_file_path} does not exist!'
raise ae.PREDICTKeyError(e)

settings = configparser.ConfigParser()
settings.read(config_file_path)

settings_dict = {'ImageFeatures': dict(), 'DataPaths': dict(),
'General': dict()}

# Extract some general settings
settings_dict['General']['Joblib_ncores'] =\
settings['General'].getint('Joblib_ncores')

settings_dict['General']['Joblib_backend'] =\
str(settings['General']['Joblib_backend'])

# Extract image feature specific settings
settings_dict['ImageFeatures']['image_type'] =\
[str(item).strip() for item in
settings['ImageFeatures']['image_type'].split(',')]

settings_dict['ImageFeatures']['shape'] =\
settings['ImageFeatures'].getboolean('shape')

settings_dict['ImageFeatures']['histogram'] =\
settings['ImageFeatures'].getboolean('histogram')

settings_dict['ImageFeatures']['orientation'] =\
settings['ImageFeatures'].getboolean('orientation')

settings_dict['ImageFeatures']['texture_Gabor'] =\
settings['ImageFeatures'].getboolean('texture_Gabor')

settings_dict['ImageFeatures']['texture_GLCM'] =\
settings['ImageFeatures'].getboolean('texture_GLCM')

settings_dict['ImageFeatures']['texture_GLCMMS'] =\
settings['ImageFeatures'].getboolean('texture_GLCMMS')

settings_dict['ImageFeatures']['texture_GLRLM'] =\
settings['ImageFeatures'].getboolean('texture_GLRLM')

settings_dict['ImageFeatures']['texture_GLSZM'] =\
settings['ImageFeatures'].getboolean('texture_GLSZM')

settings_dict['ImageFeatures']['texture_NGTDM'] =\
settings['ImageFeatures'].getboolean('texture_NGTDM')

settings_dict['ImageFeatures']['texture_LBP'] =\
settings['ImageFeatures'].getboolean('texture_LBP')

settings_dict['ImageFeatures']['coliage'] =\
settings['ImageFeatures'].getboolean('coliage')

settings_dict['ImageFeatures']['vessel'] =\
settings['ImageFeatures'].getboolean('vessel')

settings_dict['ImageFeatures']['log'] =\
settings['ImageFeatures'].getboolean('log')

settings_dict['ImageFeatures']['phase'] =\
settings['ImageFeatures'].getboolean('phase')

# Parameters for computing features
settings_dict['ImageFeatures']['parameters'] = dict()

# Gabor settings
settings_dict['ImageFeatures']['parameters']['gabor_settings'] = dict()

gabor_frequencies = [str(item).strip() for item in
settings['ImageFeatures']['gabor_frequencies']
.split(',')]
gabor_frequencies = np.asarray(gabor_frequencies).astype(np.float)

gabor_angles = [str(item).strip() for item in
settings['ImageFeatures']['gabor_angles']
.split(',')]
gabor_angles = np.asarray(gabor_angles).astype(np.float)
# Convert gabor angle to radians from angles
gabor_angles = np.radians(gabor_angles)

settings_dict['ImageFeatures']['parameters']['gabor_settings']['gabor_frequencies'] =\
gabor_frequencies

settings_dict['ImageFeatures']['parameters']['gabor_settings']['gabor_angles'] =\
gabor_angles

# GLCM
settings_dict['ImageFeatures']['parameters']['GLCM'] = dict()

settings_dict['ImageFeatures']['parameters']['GLCM']['levels'] =\
int(settings['ImageFeatures']['GLCM_levels'])

GLCM_angles = [str(item).strip() for item in
settings['ImageFeatures']['GLCM_angles']
.split(',')]
settings_dict['ImageFeatures']['parameters']['GLCM']['angles'] =\
[float(g) for g in GLCM_angles]

GLCM_distances = [str(item).strip() for item in
settings['ImageFeatures']['GLCM_distances']
.split(',')]
settings_dict['ImageFeatures']['parameters']['GLCM']['distances'] =\
[float(g) for g in GLCM_distances]

# LBP
settings_dict['ImageFeatures']['parameters']['LBP'] = dict()

LBP_radius = [str(item).strip() for item in
settings['ImageFeatures']['LBP_radius']
.split(',')]
settings_dict['ImageFeatures']['parameters']['LBP']['radius'] =\
[int(g) for g in LBP_radius]

LBP_npoints = [str(item).strip() for item in
settings['ImageFeatures']['LBP_npoints']
.split(',')]
settings_dict['ImageFeatures']['parameters']['LBP']['N_points'] =\
[int(g) for g in LBP_npoints]

# Phase features
settings_dict['ImageFeatures']['parameters']['phase'] = dict()

phase_minwavelength = [str(item).strip() for item in
settings['ImageFeatures']['phase_minwavelength']
.split(',')]
settings_dict['ImageFeatures']['parameters']['phase']['minwavelength'] =\
[int(g) for g in phase_minwavelength]

phase_nscale = [str(item).strip() for item in
settings['ImageFeatures']['phase_nscale']
.split(',')]
settings_dict['ImageFeatures']['parameters']['phase']['nscale'] =\
[int(g) for g in phase_nscale]

# log features
settings_dict['ImageFeatures']['parameters']['log'] = dict()

log_sigma = [str(item).strip() for item in
settings['ImageFeatures']['log_sigma']
.split(',')]
settings_dict['ImageFeatures']['parameters']['log']['sigma'] =\
[int(g) for g in log_sigma]

# vessel features
settings_dict['ImageFeatures']['parameters']['vessel'] = dict()

vessel_scale_range = [str(item).strip() for item in
settings['ImageFeatures']['vessel_scale_range']
.split(',')]
settings_dict['ImageFeatures']['parameters']['vessel']['scale_range'] =\
[float(g) for g in vessel_scale_range]

vessel_scale_step = [str(item).strip() for item in
settings['ImageFeatures']['vessel_scale_step']
.split(',')]
settings_dict['ImageFeatures']['parameters']['vessel']['scale_step'] =\
[float(g) for g in vessel_scale_step]

settings_dict['ImageFeatures']['parameters']['vessel']['radius'] =\
int(settings['ImageFeatures']['vessel_radius'])

return settings_dict
Loading

0 comments on commit 0296bc0

Please sign in to comment.