Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add labels for expected test resource consumption? #47

Open
nwiltsie opened this issue Sep 29, 2023 · 0 comments
Open

Add labels for expected test resource consumption? #47

nwiltsie opened this issue Sep 29, 2023 · 0 comments
Labels
enhancement New feature or request

Comments

@nwiltsie
Copy link
Member

This is a bit handwavy, but when coming in cold to a pipeline it would be nice to have some sense of:

  • What system resources / node type the tests assume
  • How long the tests are expected to run

Using pipeline-filter-RNAEditingSite as an example: the tests only require an F2, and the A-mini-N2 test takes < 1 minute while the A-full test takes >20 minutes, all of which is spent doing... something invisible to the logs?

2023-09-27 17:22:35,213 - NextFlow - INFO - executor >  local (6)
2023-09-27 17:22:35,213 - NextFlow - INFO - [ea/16b92d] process > run_validate_PipeVal (1)       [100%] 4 of 4 ✔
2023-09-27 17:22:35,213 - NextFlow - INFO - [61/4fd0ea] process > run_selectPositions_REDItools  [100%] 1 of 1 ✔
2023-09-27 17:22:35,213 - NextFlow - INFO - [ee/4756e1] process > run_AnnotateRepeatMasker_RE... [  0%] 0 of 1
2023-09-27 17:22:35,213 - NextFlow - INFO - [-        ] process > run_AnnotateGene_REDItools     -
2023-09-27 17:22:35,213 - NextFlow - INFO - [-        ] process > run_FilterREDIportal_REDItools -
2023-09-27 17:22:35,213 - NextFlow - INFO -
2023-09-27 17:22:35,213 - NextFlow - INFO -
2023-09-27 17:22:35,214 - NextFlow - INFO -
2023-09-27 17:22:35,214 - NextFlow - INFO -
2023-09-27 17:22:35,214 - NextFlow - INFO - Script time --> START: 27/09/2023 17:22:25
2023-09-27 17:22:35,214 - NextFlow - INFO - Reading table...
2023-09-27 17:22:35,214 - NextFlow - INFO - Total lines: 562210
2023-09-27 17:22:35,214 - NextFlow - INFO - Filtered in lines: 266815
2023-09-27 17:22:35,214 - NextFlow - INFO - Selected lines saved on CPCG0196-F1-A-full_candidates.txt
2023-09-27 17:42:03,998 - NextFlow - INFO - Script time --> END: 27/09/2023 17:22:34
2023-09-27 17:42:03,999 - NextFlow - INFO -
2023-09-27 17:42:04,000 - NextFlow - INFO -
2023-09-27 17:42:04,032 - NextFlow - INFO - executor >  local (6)
2023-09-27 17:42:04,033 - NextFlow - INFO - [ea/16b92d] process > run_validate_PipeVal (1)       [100%] 4 of 4 ✔
2023-09-27 17:42:04,033 - NextFlow - INFO - [61/4fd0ea] process > run_selectPositions_REDItools  [100%] 1 of 1 ✔
2023-09-27 17:42:04,033 - NextFlow - INFO - [ee/4756e1] process > run_AnnotateRepeatMasker_RE... [  0%] 0 of 1
2023-09-27 17:42:04,033 - NextFlow - INFO - [-        ] process > run_AnnotateGene_REDItools     -
2023-09-27 17:42:04,033 - NextFlow - INFO - [-        ] process > run_FilterREDIportal_REDItools -

Even just something like a free-form expected_duration field in the test config would be nice:

cases:
  - name: A-mini-n2
    nf_script: ./main.nf
    params_file: ./test/A-mini-n2.yaml
    skip: false
    verbose: true
    expected_duration: 1 minute on an F2
@nwiltsie nwiltsie added the enhancement New feature or request label Sep 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant