Skip to content

LORIS Release v24.0.0
Compare
Choose a tag to compare
@driusan driusan released this 24 Mar 19:28
· 17 commits to 24.0-release since this release
v24.0.0
d2f2684
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

Important Note: The release instructions below may result in an exception caused by a circular dependency. In order to resolve the error we suggest upgrading directly to LORIS version 24.1.2 and running the instructions recommended in that release. All other information in the release note below are still accurate

This release contains user interface improvements, code clean up, bug fixes and new features across all modules in LORIS.

Full list of changes

Some of the major changes in this release are:

Core

Features

  • Data tables may now stream data as they're loading rather than waiting
    until all data has loaded. (PR #6853)
  • Added support for Amazon S3 (PR #7963)
  • Added ability for scripts to bulk load instrument data (PR #6869)
  • New data dictionary framework and module (PR #6936)
  • New classes to describe a data dictionary (PR #6938)

Updates and Improvements

  • Module-specific permissions added for Survey Accounts, Imaging Behavioural
    Quality Control, and Behavioural Quality Control. (PR #6041)
  • Addition of a new account_request_date in users table that will be used when
    requesting a new account and will be displayed in the User Accounts module (PR #6191)
  • Candidate's age can be retrieved from the Candidate class in days, months, or years (PR #5945)
  • Addition of autoSelect property to React SelectElement allows for auto-selection of
    only available select option (PR #6156)
  • An AcquisitionDate field has been added to the files table (PR #6892)
  • Field Data_entry_completion_status given its own column in flag, and renamed to
    Required_elements_completed (PR #6876)
  • Addition of changeProject and resetProject helper functions to the
    LorisIntegrationTest class to help test project permissions (PR #6912)
  • Unit tests added for the NDB_BVL_Instrument, NDB_Page, NDB_Factory,
    User, and UserPermissions libraries (PR #6819, PR #6804, PR #6776, PR #6765)
  • The name of a Standard Date field in a LINST instrument must now end with the
    string _date. Otherwise, a LORIS exception is thrown. There is no restriction
    on the naming format of a Basic Date or MonthYear field. (PR #6923)
  • React Form Select Element now has the ability to set an option in the element
    as a disabled option. (PR #7306)
  • Pending accounts in Dashboard now includes DCC users (PR #7054)
  • Subproject filter added to Behavioural QC module (PR #7430)
  • Addition of date_format as a DataType in ConfigSettings (PR #6719)
  • Addition of new tables to store PET HRRT data (PR #6142)
  • Modification of the parameter_file table's Value field type to longtext (PR #7392)
  • Addition of 4 configuration settings for the minc2bids converter (PR #7488)
  • A PSR3 compatible logging interface was added (PR #7509)
  • Multiple classes of errors flagged by phan are now fixed (multiple PRs)
  • A LorisInstance class was added to represent an installed LORIS instance (PR #6118)
  • Validation for DateElement (JS). (PR #7266)
  • Session Current_stage default value changed for Not Started (PR #7102)
  • Permissions were reorganized and categorized in the user accounts module (PR #7327)
  • The EchoTime field has been added to the following tables: MRICandidateErrors,
    and mri_violations_log. EchoTime is necessary to distiguish MINC files for
    multi-echo aquisitions (PR #7515).- The Center_name field in the mri_protocol table has been replaced by CenterID
    from the psc table. The default value of CenterID is NULL. Previously, the
    default for Center_name was AAAA or ZZZZ. (PR #7525)

Bug Fixes

  • A LINST instrument Date field name now appears correctly (not truncated) on the
    instrument if it includes the string _date. (PR #6923)
  • A subtest with only static and/or static score fields (i.e. no values to submit)
    can be saved with no errors so that the instrument scoring script can be called. (PR #7124)
  • When a superuser edits another user, the labels for each permission is correctly
    displayed (PR #7451)
  • The imaging insertion pipeline (LORIS-MRI scripts) now starts automatically if you have the imaging uploader
    auto-launch set to true and your current upload overwrites an existing file (PR #7084).
  • Script CouchDB_MRI_Importer now computes the correct names from the data dictionary
    entries associated to MRI comments (PR #7082).
  • Candidate library now allows a null sex in the select() function to accommodate
    scanner candidates. This prevents an error from being thrown in the candidate parameters module. (PR #7058)
  • Fix public pages with missing title (PR #7121)
  • Fix "Go to main page" broken link (PR #7258)
  • Download CSV fix to remove duplicates and entries that partially match the filtering criteria (PR #7242)
  • Partially fix instrument escaping issues by reloading instrument and its data upon successful save (PR #7776)
  • Fix recognition of null sessionID in NDB_BVL_Instrument (PR #8031)

Modules

API

  • Creation of a new version of the API under development (v0.0.4-dev) (PR #6944)
  • Deletion of support for the oldest version of the API (v0.0.2) (PR #6944)
  • Addition of a PATCH request for /candidates/$CandID/$VisitLabel to start next
    stage when the payload contains a "Visit" stage with "In Progress" as Status,
    when the current status of the Visit stage is "Not Started". (PR #7479)
  • Handle characters that must be urlencoded (such as a space) in the API path for
    visit labels and projects. (PR #7478 & #7463)
  • Markdown API documentation was moved into the module docs directory (PR #6151)

API Documentation (New Module)

  • New module mostly intended for developers, this module provides a user
    interface to inspect and try LORIS modules API.

Behavioural QC

  • Fix a fatal error if the datatable is filtered with "All instruments" (PR #6945)
  • Fix for the Instrument filter to keep track of the selected value (PR #6945)

BrainBrowser

  • Now uses Loris API to download imaging files (PR #7824)

Candidate Parameters

  • Consents may now be grouped in UI of consent tab (PR #6042, PR #6044)
  • Fix to prevent titles cut off (PR #6731)

Candidate Profile

  • New integration test class to test project permissions (PR #6912)

Configuration

  • Addition of configuration settings for the DICOM to BIDS insertion pipeline (PR #7937)
  • Addition of configuration settings for the MINC to BIDS converter script (PR #7488)

Conflict Resolver

  • Changes are now saved automatically, one by one. Once a conflict is resolved
    the cell that contains the input field will glow green. It is possible to change
    the resolved conflicts to a new value until the page is refreshed. (PR #7558)
  • This module's API is now described in a Open API Specification file (schema.yml)
    that can be loaded in the new API Documentation module.
  • Change the display of multi select values from "value1{@}value2" to "value1, value2"
    in the Correct Answer column of Unresolved and Resolved Conflicts. (PR #7239)

Create timepoint

  • Fix a reindexing of the languages array which caused a database insert error. (PR #7145)
  • Reorganization of the fields on the page to a more logical order, clean up of
    inefficient code and conditional display of select options based on previous selections (PR #7825)

Data Dictionary

  • Changed instrument filter to multiselect (PR #7040)

DQT

  • The dataquery module user interface has been completely redesigned. (PR #6908)

EEG Browser

  • Signal Visualization, Events and Electrode map (PR #7387)
  • Site/Project/subproject filters only displays entries user has permission for. (PR #7400)
  • Addition of tables in the SQL schema, a filter on the main page of the module, and a download button
    on the session page to support new annotation features (PR #7345)
  • New integration test class added to this module (PR #6922)

Genomic Browser

  • CNV/CPG records added for candidates to view and test the CNV and Methylation tabs in the Genomic Browser (PR #6900)

Help Editor

  • Cleaned up the deprecated column Parent Topic (PR #7025)

Imaging Uploader

  • Fixed a bug that would prevent the imaging uploader from starting automatically even if you had the imaging uploader
    auto-launch set to true and your current upload overwrote an existing file (PR #7084)
  • Use Loris API to view files (PR #7816)

Instrument Builder

  • Fix for error 'Max value must be larger than min value' when clicking 'Add Row'. (PR #6810)

Instruments

  • General help text added for instrument data entry (PR #6902)
  • Fix to avoid select with required option in group fields to display as multiselect (PR #7254)
  • Fixes to insert JSON instruments (PR #7155)

Issue Tracker

  • Readability of comments and history was improved. (PR #6138)
  • Update validation to allow NULL Site (For All Sites issues) (#6526)
  • Fixing redirect and error reporting when creating a new issue (PR #7323)
  • Show inactive users in the list of assignees as a disabled option. If the
    inactive user had already been previously assigned the issue, the disabled option
    appears but cannot be reselected. Inactive users can no longer be assigned new
    issues. (PR #7306)

Login

  • Add option to toggle visibility on password input types. (PR #6210, #7043)

Media

  • Fix to display the file name when editing a file (PR #7381)

Publication

  • Display all filterable columns in datatable (#7277)
  • Fix for file deletion (PR #7284)

User Accounts

  • Fix a false positive validation error when a new LORIS user is added with "Make
    user name match email address" and "Generate new password". (PR #6803)
  • Fix to allow a superuser to create new users with customizable permissions. (#6770)
  • Indicate required fields (#6617)
  • Fix to show superuser the labels of each permission when editing a user (PR #7451)

Tools

  • Fix fatal errors in delete_candidate.php tool. (PR #6805, #7275)
  • Fix fatal errors in fix_candidate_age.php (PR #7546)
  • New tool generate_candidate_externalids.php to fill external IDs for all
    candidates where a NULL value is found. (PR #7095)
  • New tool populate_visits.php to back-populate visits from the config.xml,
    session table and Visit_Windows table into the visit and visit_project_subproject_rel (#7663)
  • Deprecation of the populate_visit_windows.php tool in favour of populate_visits.php (#7663)
  • Fixes a bug in the way that the data dictionary entries associated to MRI comments were named in the CouchDB database (PR #7082).

Clean Up

  • Removal of unused variables and unnecessary branching from getBattery() and
    getBatteryVerbose() functions (PR #7167)
  • Removal of the violated_scans_edit permission (PR #6747)
  • Removal for the need of the VisitLabel section of the config.xml file. All
    visit configurations and their association to projects are now in the database.
    The visits can be removed from the config.xml file after running the
    populate_visits.php script only. (#7663 & #7729)
  • Removal of references to Reliability module in Raisinbread (PR #6895)
  • Raisinbread visit stage inconsistency changed (PR #6896)
  • HRRT patch sourced to Raisinbread (PR #6897)
  • Improved consistency of RB data: field UploadLocation of table mri_upload now has an appropriate value (PR #7086)
  • The default value of the ScannerID field of the mri_protocol table is now
    NULL instead of 0. This means that if a protocol is valid on all the study's
    scanners, then ScannerID of the protocol should be set to NULL (PR #7496)

Notes For Existing Projects

  • The RegistrationProjectID column of the candidate table and the ProjectID column of the session table
    in the database are no longer nullable. This means that a value must be set in these fields BEFORE running
    the release SQL patch or it will fail.
  • New function Candidate::getSubjectForMostRecentVisit replaces Utility::getSubprojectIDUsingCandID,
    adding the ability to determine which subproject a candidate belongs to given their most recent visit.
  • LINST instrument class was modified to implement the getFullName() and getSubtestList()
    functions thus making entries in the test_names and instrument_subtests tables
    unnecessary for LINST instruments (PR #7169)
  • The script tools/single_use/remove_publication_users_edit_perm_rel_duplicates.php
    should be run before upgrading LORIS.
  • The Data_entry_completion_status column of instrument tables has been migrated
    to its own column in the flag table, and renamed to Required_elements_completed. After script
    Set_Required_elements_completed_flag.php is run, projects will need to delete the
    Data_entry_completion_status column of instrument tables. This can be accomplished
    by running Remove_Data_entry_completion_status_instr_column.php, and then sourcing
    the patch generated by this script. (PR #6876)
  • If _setDataEntryCompletionStatus, _determineDataEntryCompletionStatus, and/or
    updateDataEntryCompletionStatus are called in any overrides, make sure to replace
    all instances with their newly named counterparts, _setRequiredElementsCompletedFlag,
    _determineRequiredElementsCompletedFlag, updateRequiredElementsCompletedFlag. (PR #6876)
  • Deprecation of begintable and endtable elements in LINST instruments (PR #7183)
  • Deletion of dateTimeFields and monthYearFields variables in instrument class.
    All references to these variables should be removed from project instruments. (PR #7183)
  • Visit definitions is no longer done in the config.xml. An importer tool (populate_visits.php) is
    available to import the current setup into the visit table of the database.
    Make sure the visits displayed after these changes are what you expect.
    The visits can be removed from the config.xml file after the importer tool is run. (#7663 & #7729)
  • There is a new abstract getDataDictionary() function in the instrument class
    for the new data dictionary framework. This is already implemented for LINST instruments,
    and existing instruments using LorisForm can use the \LorisFormDictionaryImpl
    trait to extract it in the same way as lorisform_parser.php did for the old datadict module.
  • The name of a Standard Date field in a LINST instrument must now end with the
    string _date. Otherwise, a LORIS exception is thrown. There is no restriction
    on the naming format of a Basic Date or MonthYear field. (PR #6923)
  • Run tools/single_use/Remove_duplicate_examiners.php to remove duplicate
    examiners that may have been created before bugfix. Make sure to run this script
    before running the SQL/New_patches/2021-09-28-Unique_examiners.sql. (PR #7462)
  • Deletion of support for the oldest version of the API (v0.0.2) (PR #6944)
  • Refer to instructions for clean up scripts meant to remove unused data in the SQL/Archive/24.0/Cleanup directory
  • all instruments in the test_names module must have a corresponding instrument on the filesystem for the media module to load
  • the instrument class's static loadInstanceData function has been replaced with a non-static getInstanceData, calls from existing instruments must be updated appropriately.

Notes For Developers

  • PHPCS enabled for tools/ and test/ and must now pass on new PRs (Various PRs)
  • Auto fix PHPCS support added (npm run lintfix:php) (#6825)

Upgrade Process

(See https://github.com/aces/Loris/releases/tag/v24.1.2)
Note: the minimum version of PHP is now PHP 7.4

  1. Get the 24.0.0 source code
  2. Run the following scripts:
    1. php tools/configuration_check.php
      This script is used to validate the configuration of LORIS. It checks configuration settings and server paths to make sure that the application and underlying architecture are set up to allow LORIS to run properly.
    2. php tools/single_use/SaveUserIDToInstrumentData.php confirm
      This script is intended for a one-time use only to restore the value of the UserID column of instrument tables and the UserID key of the instrument JSON Data in the flag table.
    3. php tools/single_use/remove_publication_users_edit_perm_rel_duplicates.php
      This script deletes duplicate rows in publication_users_edit_perm_rel to make sure the patch '2020-01-07-publication_users_edit_perm_rel_pk.sql' will execute successfully
    4. php tools/single_use/Cleanup_Special_Chars_Document_Repository.php
      This script is written to clean up the files with special characters from the document repository data table as well as clean up the quotes appearing as %22 in the file names in the file system
    5. php tools/single_use/Cleanup_Special_Chars_Media.php
      This script is written to clean up the files with special characters from the media data table as well as clean up the quotes appearing as %22 in the file names in the file system
    6. php tools/single_use/Remove_duplicate_examiners.php
      This script is written to clean up duplicate userIDs in the examiners table
  3. Make sure the RegistrationProjectID column in the candidate table of the dataabse does not contain any NULL values. If it does, either assign an existing project to the candidate/scanner or create a new project for these entries before running the release patch (see "Notes for existing projects" for more details).
  4. Source the SQL/Release_patches/23.0_To_24.0_upgrade.sql SQL file into your database.
  5. Run the following scripts:
    1. php tools/single_use/populate_visits.php
      This script populates the visits table based on the config.xml, session table and Visit_Windows table.
    2. php tools/single_use/Set_Required_elements_completed_flag.php confirm
      This script is written to migrate the Data_entry_completion_status field from the instrument table (or Data column of the flag table) to it's own column in the flag table called Required_elements_completed. Note that this script can take a long time to complete on large projects.
  6. Run make all on the LORIS root directory
  7. OPTIONAL: Source SQL patches from the SQL/Archive/24.0/Cleanup directory at your discretion to remove superfluous or deprecated entries in the database.
  8. OPTIONAL: Run the tools/single_use/Remove_Data_entry_completion_status_instr_column.php PHP script to generate an SQL patch containing all DROP COLUMN statements for the now unused Data_entry_completion_status field in instruments. This script will generate the SQL patch and store it in the SQL/Archive/autogenerated/single_use/ directory under the name Remove_Data_entry_completion_status.sql.
  9. OPTIONAL: Remove the <visitLabel subprojectID="X"> tags in the Config.xml file

NOTE: The tools/single_use/SaveUserIDToInstrumentData.php can take several hours up to a day to run depending on the size and age of the databse and number of instruments used

Assets 2
Loading
0 Join discussion