Session restore in El MAVEN
Reproducibility of a session has been a major concern in El-MAVEN. El-MAVEN supports saving session data in .mzroll format which saves the user’s session where all the tables are saved as different .mzroll files. Apart from being a fragmented save, the implementation is known to be very unstable.
To ensure stable reproducibility and easy downstream analysis, we implemented SQLite database to store sessions in El-MAVEN v0.6 in .emDB (short for “El-MAVEN Database”) format. The following information is saved in the database:
- Sample (metadata) - Sample names and their current file paths for samples loaded in a session are stored. Apart from this, cohort names, selection status and other session specific information is also stored. It is worth noting that sample files themselves are not stored in the database, and need to be available on the user’s disk for El-MAVEN to be able to find and load the samples successfully in a new session.
- Peak groups (and their Peaks) - Most non-derivable information for peak groups (from all peak tables and bookmark table) and their peaks will be saved so as to recreate the data structure in a new El-MAVEN session.
- Compounds (target list) - Again compound information enough to recreate the data structure is stored in the database. But only compounds associated to peak groups are stored. Some specific scenarios need to be covered around missing database(s), duplicity of compounds, etc. to fully achieve session restore.
- Alignment data - Original retention time and updated retention times for each scan in each sample is stored in the database.
This list needs a few more additions (including user settings) before we can ensure that a session is completely restored between different runs of the application. These features will be added in subsequent releases and backwards compatibility will be maintained.
Whenever the user curates a peak (marks them good/bad or bookmarks a peak) in El-MAVEN, all of the above information is automatically saved in the background; in the same folder where the sample files are present. This autosave file is named using the timestamp of the first curation event. This file remains until the user chooses to quit the application without saving the session, in which case it is deleted. This means that an unexpected crash will leave behind an emDB timestamp file that stores data until the last curation event before crash, and can be used to load back user data to that point in time.
The user can export their session metadata and progress as an emDB file by either opting to do so at the end of a session or by explicitly saving their data at any other time.
- Start an El-MAVEN session. Add samples and compound database.
- Change parameters according to the dataset.
- Find/Bookmark peaks.
- Close El-MAVEN. A prompt will appear asking the user if they would like to save their session.
- If there is no user defined .emDB file which was open in the current session, the user will be asked to create a new file by providing a name for it. Otherwise if a .emDB file was open in that session, the user will be asked whether they want to update that existing file or save to a new one. New .emDB files will be saved in the same folder as the sample files' folder.
- Start an El-MAVEN session. Add samples and compound database.
- Change parameters according to the dataset.
- Find/Bookmark peaks.
- Use shortcut keys “Ctrl + S” (“Cmd + S” on MacOS) or Go to “Save Project” under “File” Menu.
- If there is no user defined .emDB file which was open in the current session, the user will be asked to create a new file by providing a name for it. Otherwise, the save will update the last opened .emDB file in that session. New .emDB files will be saved in the same folder as the sample files' folder.
The user can save project as .emDB or .mzroll from the “Save Project As” option under “File” menu and selecting the desired option.
Note: We recommend backing up all your .mzroll files as .emDB since we will be replacing .mzroll in the upcoming versions with complete session restore functionality in .emDB.
Download the latest version of El-MAVEN - https://elucidatainc.github.io/ElMaven/Downloads/