Skip to content

Switch to github actions and changes to make tests pass #64

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
bernt-matthias wants to merge 10 commits into
base: master
Choose a base branch
from
Open

Switch to github actions and changes to make tests pass #64

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
4a26df1
add github actions test
bernt-matthias Oct 29, 2023
4a27897
force installation of requirements.txt for lint
bernt-matthias Oct 29, 2023
5fc2afa
skip flake E501 and add requirements
bernt-matthias Oct 29, 2023
412de93
also run on push
bernt-matthias Oct 29, 2023
55bcd75
disable black, fill requirements.txt
bernt-matthias Oct 29, 2023
f578653
temp disable diff_pydocstyle_report
bernt-matthias Oct 29, 2023
ed78e48
galaxy: do not set name and argument
bernt-matthias Oct 29, 2023
8079e39
galaxy: fix selects
bernt-matthias May 16, 2021
acdaa01
galaxy: add a mock count attribute to test/ouput_collection elements
bernt-matthias Oct 30, 2023
e9d8416
itry to run only on pull_request
bernt-matthias Oct 30, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 62 additions & 0 deletions .github/workflows/pr.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
name: Test Pull Request

on: [pull_request]

jobs:
lint:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ['3.8']
steps:
- uses: actions/checkout@v1
- uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- name: Cache .cache/pip
uses: actions/cache@v3
id: cache-pip
with:
path: ~/.cache/pip
key: pip_cache_py_${{ matrix.python-version }}
- name: Install package
run: |
touch requirements.txt
pip install -r requirements.txt black==23.3 flake8==6.0
- name: Flake8
run: flake8 --ignore=C901,W503,E501
# at the moment I would prefer not to enable this
# - name: Black
# uses: psf/[email protected]

test:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ['3.8']
steps:
- uses: actions/checkout@v1
- uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- name: Cache .cache/pip
uses: actions/cache@v3
id: cache-pip
with:
path: ~/.cache/pip
key: pip_cache_py_${{ matrix.python-version }}
- name: setup
run: |
pip install -r requirements.txt .
pip install -r requirements-dev.txt
#TODO reenable: works only on branch?
# - name: diff_pydocstyle_report
# run: make diff_pydocstyle_report
- name: Test
run: make test
- name: Planemo lint tools
run: planemo l tests/test-data/
# planemo test content of tests/test-data (this is OK, because the previous
# tests ensure equality of the xmls that are generated and those in the package)
- name: planemo test tools
run: export PATH=$(pwd)/tests/test-data:$PATH && planemo t tests/test-data/
47 changes: 33 additions & 14 deletions ctdconverter/galaxy/converter.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1160,24 +1160,23 @@ def create_param_attribute_list(param_node, param, model, supported_file_formats
# parameters need to be treated in cheetah. variable names are currently fixed back
# to dashes in fill_ctd.py. currently there seems to be only a single tool
# requiring this https://github.com/OpenMS/OpenMS/pull/4529
param_node.attrib["name"] = get_galaxy_parameter_name(param)
param_node.attrib["argument"] = "-%s" % utils.extract_param_name(param)
argument = "-%s" % utils.extract_param_name(param)
name = get_galaxy_parameter_name(param)
if argument.lstrip("-").replace("-", "_") != name:
param_node.attrib["name"] = get_galaxy_parameter_name(param)
param_node.attrib["argument"] = argument

param_type = TYPE_TO_GALAXY_TYPE[param.type]
if param_type is None:
raise ModelError("Unrecognized parameter type %(type)s for parameter %(name)s"
% {"type": param.type, "name": param.name})

# ITEMLIST is rendered as text field (even if its integers or floats), an
# exception is files which are treated a bit below
# exception is files which are treated just below
if param.is_list:
param_type = "text"

if is_selection_parameter(param):
param_type = "select"
if len(param.restrictions.choices) < 5:
param_node.attrib["display"] = "checkboxes"
if param.is_list:
param_node.attrib["multiple"] = "true"

if is_boolean_parameter(param):
param_type = "boolean"

Expand All @@ -1204,10 +1203,20 @@ def create_param_attribute_list(param_node, param, model, supported_file_formats
# (in Galaxy the default would be prefilled in the form and at least
# one option needs to be selected).
if not (param.default is None or type(param.default) is _Null) and param_node.attrib["type"] in ["integer", "float", "text", "boolean", "select"]:
logger.error("%s %s %s %s %s" % (param.name, param.default is None, type(param.default) is _Null, param_type, param.type))
param_node.attrib["optional"] = "false"
# logger.error("%s %s %s %s %s" % (param.name, param.default is None, type(param.default) is _Null, param_type, param.type))
optional = False
else:
param_node.attrib["optional"] = str(not param.required).lower()
optional = not param.required
param_node.attrib["optional"] = str(optional).lower()

if is_selection_parameter(param):
if param.is_list:
param_node.attrib["multiple"] = "true"
if len(param.restrictions.choices) < 5:
if param.is_list and optional:
param_node.attrib["display"] = "checkboxes"
elif not param.is_list and not optional:
param_node.attrib["display"] = "radio"

# check for parameters with restricted values (which will correspond to a "select" in galaxy)
if param.restrictions is not None or param_type == "boolean":
Expand Down Expand Up @@ -1482,7 +1491,7 @@ def create_boolean_parameter(param_node, param):

def all_outputs(model, parameter_hardcoder):
"""
return lists of reqired and optional output parameters
return lists of required and optional output parameters
"""
out = []
optout = []
Expand Down Expand Up @@ -1673,6 +1682,15 @@ def create_tests(parent, inputs=None, outputs=None, test_macros_prefix=None, nam
test_node = add_child_node(tests_node, "test")
strip_elements(inputs, "validator", "sanitizer")
for node in inputs.iter():
# params do not have a name if redundant with argument
# -> readd and restore attrib order
if node.tag == "param" and not node.attrib.get("name") and node.attrib.get("argument"):
attrib = copy.deepcopy(node.attrib)
node.attrib["name"] = node.attrib["argument"].lstrip("-").replace("-", "_")
for a in attrib:
del node.attrib[a]
node.attrib[a] = attrib[a]

if node.tag == "expand" and node.attrib["macro"] == ADVANCED_OPTIONS_NAME + "macro":
node.tag = "conditional"
node.attrib["name"] = ADVANCED_OPTIONS_NAME + "cond"
Expand Down Expand Up @@ -1742,9 +1760,10 @@ def create_tests(parent, inputs=None, outputs=None, test_macros_prefix=None, nam
node.attrib["value"] = "outfile.txt"
if node.tag == "collection":
node.tag = "output_collection"
node.attrib["count"] = "0" # add a mock count element
if node.attrib.get("name", None) == "stdout":
node.attrib["lines_diff"] = "2"
for a in set(node.attrib) - {"name", "value", "ftype", "lines_diff"}:
for a in set(node.attrib) - {"name", "value", "ftype", "lines_diff", "count"}:
del node.attrib[a]
strip_elements(outputs, "delete_node", "discover_datasets", "filter", "change_format")

Expand Down
3 changes: 3 additions & 0 deletions requirements-dev.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
diff_cover
planemo
pydocstyle
3 changes: 3 additions & 0 deletions requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
git+https://github.com/WorkflowConversion/CTDopts
lxml
ruamel.yaml
20 changes: 10 additions & 10 deletions tests/test-data/bool.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,12 +32,12 @@ python3 '$__tool_directory__/fill_ctd.py' '@[email protected]' '$args_json' '$hard
<configfile name="hardcoded_json"><![CDATA[{}]]></configfile>
</configfiles>
<inputs>
<param name="flag" argument="-flag" type="boolean" truevalue="true" falsevalue="false" checked="false" label="flag" help=""/>
<param name="flag_adv" argument="-flag_adv" type="boolean" truevalue="true" falsevalue="false" checked="false" label="flag" help=""/>
<param name="true" argument="-true" type="boolean" truevalue="true" falsevalue="false" checked="true" label="bool" help=""/>
<param name="false" argument="-false" type="boolean" truevalue="true" falsevalue="false" checked="false" label="bool" help=""/>
<param name="flag_true" argument="-flag_true" type="boolean" truevalue="true" falsevalue="false" checked="true" label="bool" help=""/>
<param name="flag_false" argument="-flag_false" type="boolean" truevalue="true" falsevalue="false" checked="false" label="bool" help=""/>
<param argument="-flag" type="boolean" truevalue="true" falsevalue="false" checked="false" label="flag" help=""/>
<param argument="-flag_adv" type="boolean" truevalue="true" falsevalue="false" checked="false" label="flag" help=""/>
<param argument="-true" type="boolean" truevalue="true" falsevalue="false" checked="true" label="bool" help=""/>
<param argument="-false" type="boolean" truevalue="true" falsevalue="false" checked="false" label="bool" help=""/>
<param argument="-flag_true" type="boolean" truevalue="true" falsevalue="false" checked="true" label="bool" help=""/>
<param argument="-flag_false" type="boolean" truevalue="true" falsevalue="false" checked="false" label="bool" help=""/>
<section name="test_section" title="Test Section" help="" expanded="false">
<param name="sect_true" argument="-test_section:sect_true" type="boolean" truevalue="true" falsevalue="false" checked="true" label="bool" help=""/>
<param name="sect_false" argument="-test_section:sect_false" type="boolean" truevalue="true" falsevalue="false" checked="false" label="bool" help=""/>
Expand All @@ -49,10 +49,10 @@ python3 '$__tool_directory__/fill_ctd.py' '@[email protected]' '$args_json' '$hard
<param name="sect_flag_false_adv" argument="-test_section:sect_flag_false_adv" type="boolean" truevalue="true" falsevalue="false" checked="false" label="bool" help=""/>
</section>
<expand macro="adv_opts_macro">
<param name="true_adv" argument="-true_adv" type="boolean" truevalue="true" falsevalue="false" checked="true" label="bool" help=""/>
<param name="false_adv" argument="-false_adv" type="boolean" truevalue="true" falsevalue="false" checked="false" label="bool" help=""/>
<param name="flag_true_adv" argument="-flag_true_adv" type="boolean" truevalue="true" falsevalue="false" checked="true" label="bool" help=""/>
<param name="flag_false_adv" argument="-flag_false_adv" type="boolean" truevalue="true" falsevalue="false" checked="false" label="bool" help=""/>
<param argument="-true_adv" type="boolean" truevalue="true" falsevalue="false" checked="true" label="bool" help=""/>
<param argument="-false_adv" type="boolean" truevalue="true" falsevalue="false" checked="false" label="bool" help=""/>
<param argument="-flag_true_adv" type="boolean" truevalue="true" falsevalue="false" checked="true" label="bool" help=""/>
<param argument="-flag_false_adv" type="boolean" truevalue="true" falsevalue="false" checked="false" label="bool" help=""/>
</expand>
<param name="OPTIONAL_OUTPUTS" type="select" optional="true" multiple="true" label="Optional outputs">
<option value="ctd_out_FLAG">Output used ctd (ini) configuration file</option>
Expand Down
32 changes: 16 additions & 16 deletions tests/test-data/float.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,14 +32,14 @@ python3 '$__tool_directory__/fill_ctd.py' '@[email protected]' '$args_json' '$hard
<configfile name="hardcoded_json"><![CDATA[{}]]></configfile>
</configfiles>
<inputs>
<param name="default_mandatory_unrestricted" argument="-default_mandatory_unrestricted" type="float" optional="false" value="0.0" label="float" help=""/>
<param name="default_optional_unrestricted" argument="-default_optional_unrestricted" type="float" optional="false" value="0.0" label="float" help=""/>
<param name="default_mandatory_unrestricted_min" argument="-default_mandatory_unrestricted_min" type="float" optional="false" min="0.0" value="0.0" label="float" help=""/>
<param name="default_optional_unrestricted_min" argument="-default_optional_unrestricted_min" type="float" optional="false" min="0.0" value="0.0" label="float" help=""/>
<param name="default_mandatory_unrestricted_max" argument="-default_mandatory_unrestricted_max" type="float" optional="false" max="100.0" value="0.0" label="float" help=""/>
<param name="default_optional_unrestricted_max" argument="-default_optional_unrestricted_max" type="float" optional="false" max="100.0" value="0.0" label="float" help=""/>
<param name="default_mandatory_unrestricted_minmax" argument="-default_mandatory_unrestricted_minmax" type="float" optional="false" min="0.0" max="100.0" value="0.0" label="float" help=""/>
<param name="default_optional_unrestricted_minmax" argument="-default_optional_unrestricted_minmax" type="float" optional="false" min="0.0" max="100.0" value="0.0" label="float" help=""/>
<param argument="-default_mandatory_unrestricted" type="float" optional="false" value="0.0" label="float" help=""/>
<param argument="-default_optional_unrestricted" type="float" optional="false" value="0.0" label="float" help=""/>
<param argument="-default_mandatory_unrestricted_min" type="float" optional="false" min="0.0" value="0.0" label="float" help=""/>
<param argument="-default_optional_unrestricted_min" type="float" optional="false" min="0.0" value="0.0" label="float" help=""/>
<param argument="-default_mandatory_unrestricted_max" type="float" optional="false" max="100.0" value="0.0" label="float" help=""/>
<param argument="-default_optional_unrestricted_max" type="float" optional="false" max="100.0" value="0.0" label="float" help=""/>
<param argument="-default_mandatory_unrestricted_minmax" type="float" optional="false" min="0.0" max="100.0" value="0.0" label="float" help=""/>
<param argument="-default_optional_unrestricted_minmax" type="float" optional="false" min="0.0" max="100.0" value="0.0" label="float" help=""/>
<section name="test_section" title="Test Section" help="" expanded="false">
<param name="sect_default_mandatory_unrestricted" argument="-test_section:sect_default_mandatory_unrestricted" type="float" optional="false" value="0.0" label="float" help=""/>
<param name="sect_default_optional_unrestricted" argument="-test_section:sect_default_optional_unrestricted" type="float" optional="false" value="0.0" label="float" help=""/>
Expand All @@ -59,14 +59,14 @@ python3 '$__tool_directory__/fill_ctd.py' '@[email protected]' '$args_json' '$hard
<param name="sect_default_optional_unrestricted_adv_minmax" argument="-test_section:sect_default_optional_unrestricted_adv_minmax" type="float" optional="false" min="0.0" max="100.0" value="0.0" label="float" help=""/>
</section>
<expand macro="adv_opts_macro">
<param name="default_mandatory_unrestricted_adv" argument="-default_mandatory_unrestricted_adv" type="float" optional="false" value="0.0" label="float" help=""/>
<param name="default_optional_unrestricted_adv" argument="-default_optional_unrestricted_adv" type="float" optional="false" value="0.0" label="float" help=""/>
<param name="default_mandatory_unrestricted_adv_min" argument="-default_mandatory_unrestricted_adv_min" type="float" optional="false" min="0.0" value="0.0" label="float" help=""/>
<param name="default_optional_unrestricted_adv_min" argument="-default_optional_unrestricted_adv_min" type="float" optional="false" min="0.0" value="0.0" label="float" help=""/>
<param name="default_mandatory_unrestricted_adv_max" argument="-default_mandatory_unrestricted_adv_max" type="float" optional="false" max="100.0" value="0.0" label="float" help=""/>
<param name="default_optional_unrestricted_adv_max" argument="-default_optional_unrestricted_adv_max" type="float" optional="false" max="100.0" value="0.0" label="float" help=""/>
<param name="default_mandatory_unrestricted_adv_minmax" argument="-default_mandatory_unrestricted_adv_minmax" type="float" optional="false" min="0.0" max="100.0" value="0.0" label="float" help=""/>
<param name="default_optional_unrestricted_adv_minmax" argument="-default_optional_unrestricted_adv_minmax" type="float" optional="false" min="0.0" max="100.0" value="0.0" label="float" help=""/>
<param argument="-default_mandatory_unrestricted_adv" type="float" optional="false" value="0.0" label="float" help=""/>
<param argument="-default_optional_unrestricted_adv" type="float" optional="false" value="0.0" label="float" help=""/>
<param argument="-default_mandatory_unrestricted_adv_min" type="float" optional="false" min="0.0" value="0.0" label="float" help=""/>
<param argument="-default_optional_unrestricted_adv_min" type="float" optional="false" min="0.0" value="0.0" label="float" help=""/>
<param argument="-default_mandatory_unrestricted_adv_max" type="float" optional="false" max="100.0" value="0.0" label="float" help=""/>
<param argument="-default_optional_unrestricted_adv_max" type="float" optional="false" max="100.0" value="0.0" label="float" help=""/>
<param argument="-default_mandatory_unrestricted_adv_minmax" type="float" optional="false" min="0.0" max="100.0" value="0.0" label="float" help=""/>
<param argument="-default_optional_unrestricted_adv_minmax" type="float" optional="false" min="0.0" max="100.0" value="0.0" label="float" help=""/>
</expand>
<param name="OPTIONAL_OUTPUTS" type="select" optional="true" multiple="true" label="Optional outputs">
<option value="ctd_out_FLAG">Output used ctd (ini) configuration file</option>
Expand Down
Loading