[ENH] improve performance and testing #618
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Remi-Gau
changed the title
Remi-Gau
added a commit
that referenced
this pull request
Nov 20, 2023
* add transformers to label and merge identical rows * update doc * refactor * add cumulative behavior to label indentical rows * refactor * expand behavior of label rows to support cell type for columns * add tests for merge rows * FCI * FCI * add inequality to Filter transformer * support regex for Replace transformers * update docs * refactor * add possibility to generate json version of transformers * add demo transformers split by trials * update help sections * update code to adapt to new schema * fix entity order bug * return only one datatypes when they are duplicates * set file modality in constructor * get definition will look into metadata if it returns empty in colums * rm old CI * remove labels from dependabot config * fix test that delete schema * make layout more verbose * make layout more versbose and make warning mosre useful * only display a . for actually indexed modality * fix octave bug * add function to create basic participant.tsv * silence warning test for octave * make query warning more explicit * add extra model validation * revert 21a9a9e * retry more informative error on metadata query * remove some old copyrights * lint * use new copyright format * update doc * bring back images for contributing * fix typo * update demos * update notebooks and convert to matlab scripts * update notebooks * add test workflow * FCI * FCI * fix notebooks * Update demos/working_with_files.m * update doc * FCI * switch back to isdir to see if older octave is happier with this * update README * update create participants.tsv * update image path * fix copyright line skip * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix spelling * fix report tests * update create participants * add create session tsv * fix test * update test * update bids init * refactor create readme * update create readme * add extra columns sessions.tsv * add doc arguments * lint * improve test and messages * fix some test * exclude some files from scans.tsv * refactor tests * fix octave warning * silence warning test on octave * skip bids validation with octave for now * skip failing matlab test * simplify test * windows related fixes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * skip octave test * improve error message * fix octave test * fix listing of trial types * improve default stats model when no task present * silence test for warning under octave * fix creation of json for derivatives when entity labels are numbers * make download more robust * skip warning test for octave * reformat windows path when printing to avoiding escaped characters warnings * improve error message on query * fix create sessions and participants tsv * add potential datalad related hints when file writing fails * fix typo * improve error message * silence codespell * silence octave warning * improve transformers * add response time histogram * add get root node method * refactor * use bids stats model to plot events * fix code spell and octave bug * fix miss hit config * remove comment * fix download face rep dataset * better handle datalad datasets * validate folder name in bids.init * fix cell indexing * fix cell indexing * update doc * add diagnostic for events * improve figure saving * update doc * rename transformers tests * refactor tests * silence writing tranformers test definitions * move logical operation as compute opertion * update transformer test description * update facerep * add versbosity options to reports * add flag to all figures * add filter to layout to only index certain subjects, sessions, modalities * use filtering of layout * fix when no filter * use regexp * fix when to use regex * improve octave support of transformers * rm some tests skip for octave * stop using moxunit * refactor ci * change action version * use bids validator with octave * fix octace in ci * add JSONio to octave path * add quotes * fiw path * fix path * fix dead links * fix links * update download * update precommit * fix tests * make transformer more tolerant when filtering variable is missing * [DOC] update bids.layout doc and return_file_index error message (#539) * update doc * add more detail when error happens in return_file_index * update precommit * update pre-commit and run on all files * [ENH] update warning message on return_file_index failure (#540) * update doc * add more detail when error happens in return_file_index * update precommit * update pre-commit and run on all files * update warning * improve error message * update info about gunzip (#547) * [MAINT] back merge master (#549) * docs: update README.md [skip ci] * docs: update .all-contributorsrc [skip ci] * CI: Use PAT to submit pull requests (#534) This will ensure that actions run on the PR. The default `GITHUB_TOKEN` is not enabled to do that to avoid a possible loop of new PRs with actions. * Bump peter-evans/create-pull-request from 4 to 5 (#536) Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 4 to 5. - [Release notes](https://github.com/peter-evans/create-pull-request/releases) - [Commits](peter-evans/create-pull-request@v4...v5) --- updated-dependencies: - dependency-name: peter-evans/create-pull-request dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [MAINT] update schema (#548) * update schema * readd token$ --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com> Co-authored-by: Chris Markiewicz <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [FIX] failing to return file index during dependencies indexing thorw warning and not error (#546) * no file index throws warning and not error * silence octave warning * update doc (#554) * [ENH] support querying for scans.tsv and sessions.tsv (#550) * add querying for scans * add querying for sessions * count sessions and scans as suffixes * fix tests * [ENH] allow to query directly for tsv content (#555) * silence some warnings * test error * allow query of participants (#557) * [INFRA] run tests on mac OS and windows for matlab (#560) * docs: update README.md [skip ci] * docs: update .all-contributorsrc [skip ci] * CI: Use PAT to submit pull requests (#534) This will ensure that actions run on the PR. The default `GITHUB_TOKEN` is not enabled to do that to avoid a possible loop of new PRs with actions. * Bump peter-evans/create-pull-request from 4 to 5 (#536) Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 4 to 5. - [Release notes](https://github.com/peter-evans/create-pull-request/releases) - [Commits](peter-evans/create-pull-request@v4...v5) --- updated-dependencies: - dependency-name: peter-evans/create-pull-request dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [MAINT] update schema (#548) * update schema * readd token$ * update ci (#558) * split workflows matlab and octave * fix * use shorthand for commands * skip coverage on windows * [FIX] remove byte order mark from tsv file (#556) * fix bom * add data * octave fix * change the way CI is detected * fix tests * sanitize path in error * [DATALAD] Recorded changes * [DATALAD] Recorded changes --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com> Co-authored-by: Chris Markiewicz <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [INFRA] randomize tests order (#566) * randomize tests * fix doc generation and links * skip dead links * skip dead links * skip dead links * skip dead links * use temp_dir * [MAINT] update schema dev (#580) * docs: update README.md [skip ci] * docs: update .all-contributorsrc [skip ci] * CI: Use PAT to submit pull requests (#534) This will ensure that actions run on the PR. The default `GITHUB_TOKEN` is not enabled to do that to avoid a possible loop of new PRs with actions. * Bump peter-evans/create-pull-request from 4 to 5 (#536) Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 4 to 5. - [Release notes](https://github.com/peter-evans/create-pull-request/releases) - [Commits](peter-evans/create-pull-request@v4...v5) --- updated-dependencies: - dependency-name: peter-evans/create-pull-request dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [MAINT] update schema (#548) * update schema * readd token$ * update ci (#558) * [FIX] remove byte order mark from tsv file (#556) * fix bom * add data * octave fix * Bump matlab-actions/run-command from 1.1.3 to 1.2.0 (#570) Bumps [matlab-actions/run-command](https://github.com/matlab-actions/run-command) from 1.1.3 to 1.2.0. - [Release notes](https://github.com/matlab-actions/run-command/releases) - [Commits](matlab-actions/run-command@v1.1.3...v1.2.0) --- updated-dependencies: - dependency-name: matlab-actions/run-command dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [MAINT] update schema (#577) * update schema * rm token --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com> Co-authored-by: Chris Markiewicz <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [ENH] give more meaningful error messsages when failing to read json (#582) * add warning mentioning which json files could not be read * add file not found erro * fix force false skipped when unzipping files (#584) * [DOC] update help of several functions (#583) * update doc * update example * fixes * [ENH] add electrodes and optodes as group dependencies (#586) * add electrodes and optodes as dependencies * inline function * fix * allow indexing of files missing session entity when schemaless (#598) * improve wrong dir message (#601) * [ENH] index and query phenotype (#590) * index and query phenotype * do not load content and metadata * [ENH] do not add subject to layout if it is empty (#600) * do not add subject to layout if it is empty * subject is in structure if schemaless * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * refactor * get properties and not fields when dealing with bids.File instance * rerun * silence failing tests on windows --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [MAINT] Update from main (#606) * docs: update README.md [skip ci] * docs: update .all-contributorsrc [skip ci] * CI: Use PAT to submit pull requests (#534) This will ensure that actions run on the PR. The default `GITHUB_TOKEN` is not enabled to do that to avoid a possible loop of new PRs with actions. * Bump peter-evans/create-pull-request from 4 to 5 (#536) Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 4 to 5. - [Release notes](https://github.com/peter-evans/create-pull-request/releases) - [Commits](peter-evans/create-pull-request@v4...v5) --- updated-dependencies: - dependency-name: peter-evans/create-pull-request dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [MAINT] update schema (#548) * update schema * readd token$ * update ci (#558) * [FIX] remove byte order mark from tsv file (#556) * fix bom * add data * octave fix * Bump matlab-actions/run-command from 1.1.3 to 1.2.0 (#570) Bumps [matlab-actions/run-command](https://github.com/matlab-actions/run-command) from 1.1.3 to 1.2.0. - [Release notes](https://github.com/matlab-actions/run-command/releases) - [Commits](matlab-actions/run-command@v1.1.3...v1.2.0) --- updated-dependencies: - dependency-name: matlab-actions/run-command dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [MAINT] update schema (#577) * update schema * rm token * docs: add coxroy as a contributor for bug, ideas, and userTesting (#585) * docs: update README.md [skip ci] * docs: update .all-contributorsrc [skip ci] --------- Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com> * BIDS schema update (#587) Co-authored-by: Remi-Gau <[email protected]> * BIDS schema update (#592) Co-authored-by: Remi-Gau <[email protected]> * BIDS schema update (#594) Co-authored-by: Remi-Gau <[email protected]> * Bump matlab-actions/run-command from 1.2.0 to 1.2.1 (#602) Bumps [matlab-actions/run-command](https://github.com/matlab-actions/run-command) from 1.2.0 to 1.2.1. - [Release notes](https://github.com/matlab-actions/run-command/releases) - [Commits](matlab-actions/run-command@v1.2.0...v1.2.1) --- updated-dependencies: - dependency-name: matlab-actions/run-command dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [FIX] handle nan and and datetimes when printing tables (#605) * fix bugs in tables and datetimes * skip on windows * update schema --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com> Co-authored-by: Chris Markiewicz <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Remi-Gau <[email protected]> * [ENH] use inheritance in get_meta_list (#609) * use inheritance in get_meta_list * refactor * [ENH] improve bids schema api (#610) * improve API * add doc * skip octave for test * [FIX] better handle printing of paths in windows (#611) * better handle printting of paths in windows * fix more paths * more clean up * fix more paths * fix bug with unsorted model (#613) * add info about skipped tests (#614) * Incorporating json into bids.File (issues #596 #371) (#597) * First attempt to incorporate json into bids.File * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * lint * Added util function to update structures * Style and tests for util/update_struct.m * Using metadata field instead of json_struct * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Removed JSONFile subclass * Added tests for metadata for File class * Fixed spelling in doc * File: Added functions that act on individual fields * File: fixed style * Splitted File.metadada tests into smaller ones --------- Co-authored-by: Beliy Nikita <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Remi Gau <[email protected]> * add id to warning * fix bug with 2 json file for one level (#616) * [ENH] improve performance and testing (#618) * update timing * try speed up test by not indexing dependencies * run on latest and oldest matlab * [DATALAD] Recorded changes * [DATALAD] Recorded changes * cancel previous runs * speed up some tests or functions * run slow and fast tests in parallel * windows env var fix * try using globals * fix * add more slow test * add doc * update demo (#621) * update demo * doc changes * [ENH] remove atlas and update res and den metadata (#622) * rm atlas and update res and den metadata * fix * [FIX] Fix tests after merge from master (#627) * fix tests after merge from main * fix test octave * use objects for levels (#630) * add internal function to start resolvig uri (#631) * fix hed tags (#632) * [MAINT] merge main in dev (#644) * BIDS schema update (#628) Co-authored-by: Remi-Gau <[email protected]> * Bump actions/checkout from 3 to 4 (#629) Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v3...v4) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * BIDS schema update (#633) Co-authored-by: Remi-Gau <[email protected]> * BIDS schema update (#634) Co-authored-by: Remi-Gau <[email protected]> * BIDS schema update (#635) Co-authored-by: Remi-Gau <[email protected]> * [pre-commit.ci] pre-commit autoupdate (#636) updates: - [github.com/pre-commit/pre-commit-hooks: v4.4.0 → v4.5.0](pre-commit/pre-commit-hooks@v4.4.0...v4.5.0) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * BIDS schema update (#637) Co-authored-by: Remi-Gau <[email protected]> * BIDS schema update (#639) Co-authored-by: Remi-Gau <[email protected]> * BIDS schema update (#640) Co-authored-by: Remi-Gau <[email protected]> * Bump matlab-actions/setup-matlab from 1.2.4 to 1.2.5 (#641) Bumps [matlab-actions/setup-matlab](https://github.com/matlab-actions/setup-matlab) from 1.2.4 to 1.2.5. - [Release notes](https://github.com/matlab-actions/setup-matlab/releases) - [Commits](matlab-actions/setup-matlab@v1.2.4...v1.2.5) --- updated-dependencies: - dependency-name: matlab-actions/setup-matlab dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump matlab-actions/run-command from 1.2.1 to 1.2.2 (#642) Bumps [matlab-actions/run-command](https://github.com/matlab-actions/run-command) from 1.2.1 to 1.2.2. - [Release notes](https://github.com/matlab-actions/run-command/releases) - [Commits](matlab-actions/run-command@v1.2.1...v1.2.2) --- updated-dependencies: - dependency-name: matlab-actions/run-command dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [FIX] update test after schema update (#643) * fix test after schema update * set os version for doc * update schema * fix typos * update tests --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Remi-Gau <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com> Co-authored-by: Chris Markiewicz <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Remi-Gau <[email protected]> Co-authored-by: Nikita Beliy <[email protected]> Co-authored-by: Beliy Nikita <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.