Image Data Analysis for Group Project
Error in extract(mutate(mutate(tracking, file = str_remove(file, ".txt")), : could not find function "extract"ls -ltotal 212
-drwxr-xr-x 2 runner docker 4096 Nov 20 15:27 data
-drwxr-xr-x 2 runner docker 4096 Nov 20 15:27 images
--rw-r--r-- 1 runner docker 1597 Nov 20 15:27 overview.qmd
--rw-r--r-- 1 runner docker 25383 Nov 20 15:32 study_after_workshop.html
--rw-r--r-- 1 runner docker 184 Nov 20 15:27 study_after_workshop.qmd
--rw-r--r-- 1 runner docker 70840 Nov 20 15:32 study_before_workshop.html
--rw-r--r-- 1 runner docker 4807 Nov 20 15:27 study_before_workshop.ipynb
--rw-r--r-- 1 runner docker 13029 Nov 20 15:27 study_before_workshop.qmd
--rw-r--r-- 1 runner docker 44938 Nov 20 15:27 workshop.html
--rw-r--r-- 1 runner docker 8550 Nov 20 15:27 workshop.qmd
--rw-r--r-- 1 runner docker 8564 Nov 20 15:33 workshop.rmarkdown
-drwxr-xr-x 3 runner docker 4096 Nov 20 15:27 workshop_filesYou can use more than one option at once. The -h option stands for “human readable” and makes the file sizes easier to understand for humans:
ls -hltotal 212K
-drwxr-xr-x 2 runner docker 4.0K Nov 20 15:27 data
-drwxr-xr-x 2 runner docker 4.0K Nov 20 15:27 images
--rw-r--r-- 1 runner docker 1.6K Nov 20 15:27 overview.qmd
--rw-r--r-- 1 runner docker 25K Nov 20 15:32 study_after_workshop.html
--rw-r--r-- 1 runner docker 184 Nov 20 15:27 study_after_workshop.qmd
--rw-r--r-- 1 runner docker 70K Nov 20 15:32 study_before_workshop.html
--rw-r--r-- 1 runner docker 4.7K Nov 20 15:27 study_before_workshop.ipynb
--rw-r--r-- 1 runner docker 13K Nov 20 15:27 study_before_workshop.qmd
--rw-r--r-- 1 runner docker 44K Nov 20 15:27 workshop.html
--rw-r--r-- 1 runner docker 8.4K Nov 20 15:27 workshop.qmd
--rw-r--r-- 1 runner docker 8.4K Nov 20 15:33 workshop.rmarkdown
-drwxr-xr-x 3 runner docker 4.0K Nov 20 15:27 workshop_filesThe -a option stands for “all” and shows us all the files, including hidden files.
ls -alhtotal 220K
-drwxr-xr-x 5 runner docker 4.0K Nov 20 15:33 .
-drwxr-xr-x 6 runner docker 4.0K Nov 20 15:32 ..
-drwxr-xr-x 2 runner docker 4.0K Nov 20 15:27 data
-drwxr-xr-x 2 runner docker 4.0K Nov 20 15:27 images
--rw-r--r-- 1 runner docker 1.6K Nov 20 15:27 overview.qmd
--rw-r--r-- 1 runner docker 25K Nov 20 15:32 study_after_workshop.html
--rw-r--r-- 1 runner docker 184 Nov 20 15:27 study_after_workshop.qmd
--rw-r--r-- 1 runner docker 70K Nov 20 15:32 study_before_workshop.html
--rw-r--r-- 1 runner docker 4.7K Nov 20 15:27 study_before_workshop.ipynb
--rw-r--r-- 1 runner docker 13K Nov 20 15:27 study_before_workshop.qmd
--rw-r--r-- 1 runner docker 44K Nov 20 15:27 workshop.html
--rw-r--r-- 1 runner docker 8.4K Nov 20 15:27 workshop.qmd
--rw-r--r-- 1 runner docker 8.4K Nov 20 15:33 workshop.rmarkdown
-drwxr-xr-x 3 runner docker 4.0K Nov 20 15:27 workshop_filesYou can move about with the cd command, which stands for “change directory”. You can use it to move into a directory by specifying the path to the directory:
The following ImageJ workflow uses the processing steps you used in workshop 3 with one change. That change is to save the results to file rather than having the results window pop up and saving from there. Or maybe two changes: it also tells you to use meaning systematic file names that will be easy to process when importing data. The RStudio workflow shows you how to import multiple files into one dataframe with columns indicating the treatment.
Save files with systematic names: ev_control.avi 343_control.avi ev_1.avi 343_1.avi ev_2.5.avi 343_2.5.avi
Open ImageJ
Open video file eg ev_2.5.avi
Convert to 8-bit: Image | Type | 8-bit
Crop to petri dish: Select then Image | Crop
Calculate average pixel intensity: Image | Stacks | Z Project
+AVG_ev_2.5.avi
Subtract average from image: Process | Image Calculator
+ev_2.5.avi
Invert: Edit | Invert
Adjust threshold: Image | Adjust | Threshold
+Invert: Edit | Invert
Track: Plugins | wrMTrck
+Set minSize: 10
Set maxSize: 400
Click OK. Save to a folder for all the tracking data files. I recommend deleting the “Results of..” part of the name
Check that the Summary window indicates 3 tracks and that the 3 larvae are what is tracked by using the slider on the Result image
Repeat for all videos
This is the code you need to import multiple csv files into a single dataframe and add a column with the treatment information from the filename.
+This is the code you need to import multiple csv files into a single dataframe and add a column with the treatment information from the file name. This is why systematic file names are good.
It assumes
your files are called type_concentration.txt for example: ev_control.txt 343_control.txt ev_1.txt 343_1.txt ev_2.5.txt 343_2.5.txt.
the .txt datafile are in a folder called track inside your working directory
you have installed the following packages: tidyverse, janitor
track inside your working directorytidyverse, janitor
+🎬 Load the tidyverse
Error in extract(mutate(mutate(tracking, file = str_remove(file, ".txt")), : could not find function "extract"[^_]{2,3} matches two or three characters that are not _ at the strat of the string (^)
[^_]{2,3} matches two or three characters that are not _ at the start of the string (^)
.+ matches one or more characters. The extract() function puts the first match into the first column, type, and the second match into the second column, conc. The remove = FALSE argument means the original column is kept.
You now have a dataframe with all the tracking data which is relatively easy to summarise and plot using tools you know.
There is an example RStudio project containing this code here: tips. You can download and open directly in RStudio using the following code. Since this is an RStudio Project, do not run the code from inside a project:
diff --git a/omics/week-5/figures/prog-hspc-volcano.png b/omics/week-5/figures/prog-hspc-volcano.png index 78eb7cb..d262397 100644 Binary files a/omics/week-5/figures/prog-hspc-volcano.png and b/omics/week-5/figures/prog-hspc-volcano.png differ diff --git a/omics/week-5/workshop.html b/omics/week-5/workshop.html index 4bb6db2..9f9532d 100644 --- a/omics/week-5/workshop.html +++ b/omics/week-5/workshop.html @@ -703,8 +703,8 @@On the vertical axis are genes which are differentially expressed at the 0.01 level. On the horizontal axis are samples. We can see that the FGF-treated samples cluster together and the control samples cluster together. We can also see two clusters of genes; one of these shows genes upregulated (more yellow) in the FGF-treated samples (the pink cluster) and the other shows genes down regulated (more blue, the blue cluster) in the FGF-treated samples.
@@ -1018,8 +1018,8 @@It will take a minute to run and display. On the vertical axis are genes which are differentially expressed at the 0.01 level. On the horizontal axis are cells. We can see that cells of the same type don’t cluster that well together. We can also see two clusters of genes but the pattern of gene is not as clear as it was for the frogs and the correspondence with the cell clusters is not as strong.
diff --git a/omics/week-5/workshop_files/figure-html/unnamed-chunk-33-1.png b/omics/week-5/workshop_files/figure-html/unnamed-chunk-33-1.png index 93dc19e..326f6e2 100644 Binary files a/omics/week-5/workshop_files/figure-html/unnamed-chunk-33-1.png and b/omics/week-5/workshop_files/figure-html/unnamed-chunk-33-1.png differ diff --git a/omics/week-5/workshop_files/figure-html/unnamed-chunk-65-1.png b/omics/week-5/workshop_files/figure-html/unnamed-chunk-65-1.png index d30a9cf..aaf880b 100644 Binary files a/omics/week-5/workshop_files/figure-html/unnamed-chunk-65-1.png and b/omics/week-5/workshop_files/figure-html/unnamed-chunk-65-1.png differ diff --git a/search.json b/search.json index 1311e7b..593fb9e 100644 --- a/search.json +++ b/search.json @@ -977,14 +977,14 @@ "href": "images/images.html", "title": "Image Data Analysis for Group Project", "section": "", - "text": "Save files with systematic names: ev_control.avi 343_control.avi ev_1.avi 343_1.avi ev_2.5.avi 343_2.5.avi\nOpen ImageJ\nOpen video file eg ev_2.5.avi\n\nConvert to 8-bit: Image | Type | 8-bit\nCrop to petri dish: Select then Image | Crop\n\nCalculate average pixel intensity: Image | Stacks | Z Project\n\nProjection type: Average Intensity to create AVG_ev_2.5.avi\n\n\n\n\nSubtract average from image: Process | Image Calculator\n\nImage 1: ev_2.5.avi\n\nOperation: Subtract\nImage 2: AVG_ev_2.5.avi\n\nCreate new window: checked\nOK, Yes to Process all\n\n\nInvert: Edit | Invert\n\nAdjust threshold: Image | Adjust | Threshold\n\nMethod: Default\nThresholding: Default, B&W\nDark background: checked\nAuto or adjust a little but make sure the larvae do not disappear at later points in the video (use the slider)\nApply\n\n\nInvert: Edit | Invert\n\nTrack: Plugins | wrMTrck\n\nSet minSize: 10\nSet maxSize: 400\nSet maxVelocity: 10\nSet maxAreaChange: 200\nSet bendThreshold: 1\n\nImportant: check Save Results File This is different to what you did in the workshop. It will help because the results will be saved automatically rather than to saving from the Results window that other pops up. Consequently, you will be able to save the results files with systematic names relating to their treatments and then read them into R simultaneously. That will also allow you to add information from the name of the file (which has the treatment information) to the resulting dataframes\n\n\nwrMTrck window with the settings listed above shown\n\n\nClick OK. Save to a folder for all the tracking data files. I recommend deleting the “Results of..” part of the name\n\n\nCheck that the Summary window indicates 3 tracks and that the 3 larvae are what is tracked by using the slider on the Result image\nRepeat for all videos\n\nThis is the code you need to import multiple csv files into a single dataframe and add a column with the treatment information from the filename.\nIt assumes\n\nyour files are called type_concentration.txt for example: ev_control.txt 343_control.txt ev_1.txt 343_1.txt ev_2.5.txt 343_2.5.txt.\nthe .txt datafile are in a folder called track inside your working directory\nyou have installed the following packages: tidyverse, janitor\n\n🎬 Load the tidyverse\n\nlibrary(tidyverse)\n\n🎬 Put the file names into a vector we will iterate through\n\n# get a vector of the file names\nfiles <- list.files(path = \"track\", full.names = TRUE )\n\nWe can use map_df() from the purrr package which is one of the tidyverse gems loaded with tidyvserse. map_df() will iterate through files and read them into a dataframe with a specified import function. We are using read_table(). map_df() keeps track of the file by adding an index column called file to the resulting dataframe. Instead of this being a number (1 - 6 here) we can use set_names() to use the file names instead. The clean_names() function from the janitor package will clean up the column names (make them lower case, replace spaces with _ remove special characters etc)\n🎬 Import multiple csv files into one dataframe called tracking\n\n# import multiple data files into one dataframe called tracking\n# using map_df() from purrr package\n# clean the column names up using janitor::clean_names()\ntracking <- files |> \n set_names() |>\n map_dfr(read_table, .id = \"file\") |>\n janitor::clean_names()\n\nYou will get a warning Duplicated column names deduplicated: 'avgX' => 'avgX_1' [15] for each of the files because the csv files each have two columns called avgX. If you click on the tracking dataframe you see is contains the data from all the files.\nNow we can add columns for the type and the concentration by processing the values in the file. The values are like track/343_0.txt so we need to remove .txt and track/ and separate the remaining words into two columns.\n🎬 Process the file column to add columns for the type and the concentration\n\n# extract type and concentration from file name\n# and put them into additopnal separate columns\ntracking <- tracking |> \n mutate(file = str_remove(file, \".txt\")) |>\n mutate(file = str_remove(file, \"track/\")) |>\n extract(file, remove = \n FALSE,\n into = c(\"type\", \"conc\"), \n regex = \"([^_]{2,3})_(.+)\") \n\nError in extract(mutate(mutate(tracking, file = str_remove(file, \".txt\")), : could not find function \"extract\"\n\n\n[^_]{2,3} matches two or three characters that are not _ at the strat of the string (^)\n.+ matches one or more characters. The extract() function puts the first match into the first column, type, and the second match into the second column, conc. The remove = FALSE argument means the original column is kept.\nYou now have a dataframe with all the tracking data which is relatively easy to summarise and plot using tools you know.\nThere is an example RStudio project containing this code here: tips. You can download and open directly in RStudio using the following code. Since this is an RStudio Project, do not run the code from inside a project:\n\nusethis::use_course(\"3mmaRand/tips\")" + "text": "The following ImageJ workflow uses the processing steps you used in workshop 3 with one change. That change is to save the results to file rather than having the results window pop up and saving from there. Or maybe two changes: it also tells you to use meaning systematic file names that will be easy to process when importing data. The RStudio workflow shows you how to import multiple files into one dataframe with columns indicating the treatment.\n\nSave files with systematic names: ev_control.avi 343_control.avi ev_1.avi 343_1.avi ev_2.5.avi 343_2.5.avi\nOpen ImageJ\nOpen video file eg ev_2.5.avi\n\nConvert to 8-bit: Image | Type | 8-bit\nCrop to petri dish: Select then Image | Crop\nCalculate average pixel intensity: Image | Stacks | Z Project\n\nProjection type: Average Intensity to create AVG_ev_2.5.avi\n\n\n\nSubtract average from image: Process | Image Calculator\n\nImage 1: ev_2.5.avi\n\nOperation: Subtract\nImage 2: AVG_ev_2.5.avi\n\nCreate new window: checked\nOK, Yes to Process all\n\n\nInvert: Edit | Invert\nAdjust threshold: Image | Adjust | Threshold\n\nMethod: Default\nThresholding: Default, B&W\nDark background: checked\nAuto or adjust a little but make sure the larvae do not disappear at later points in the video (use the slider)\nApply\n\n\nInvert: Edit | Invert\nTrack: Plugins | wrMTrck\n\nSet minSize: 10\nSet maxSize: 400\nSet maxVelocity: 10\nSet maxAreaChange: 200\nSet bendThreshold: 1\n\nImportant: check Save Results File This is different to what you did in the workshop. It will help because the results will be saved automatically rather than to saving from the Results window that other pops up. Consequently, you will be able to save the results files with systematic names relating to their treatments and then read them into R simultaneously. That will also allow you to add information from the name of the file (which has the treatment information) to the resulting dataframes\n\n\nwrMTrck window with the settings listed above shown\n\n\nClick OK. Save to a folder for all the tracking data files. I recommend deleting the “Results of..” part of the name\n\n\nCheck that the Summary window indicates 3 tracks and that the 3 larvae are what is tracked by using the slider on the Result image\nRepeat for all videos\n\nThis is the code you need to import multiple csv files into a single dataframe and add a column with the treatment information from the file name. This is why systematic file names are good.\nIt assumes\n\nyour files are called type_concentration.txt for example: ev_control.txt 343_control.txt ev_1.txt 343_1.txt ev_2.5.txt 343_2.5.txt.\nthe .txt datafile are in a folder called track inside your working directory\nyou have installed the following packages: tidyverse, janitor\n\n\n🎬 Load the tidyverse\n\nlibrary(tidyverse)\n\n🎬 Put the file names into a vector we will iterate through\n\n# get a vector of the file names\nfiles <- list.files(path = \"track\", full.names = TRUE )\n\nWe can use map_df() from the purrr package which is one of the tidyverse gems loaded with tidyvserse. map_df() will iterate through files and read them into a dataframe with a specified import function. We are using read_table(). map_df() keeps track of the file by adding an index column called file to the resulting dataframe. Instead of this being a number (1 - 6 here) we can use set_names() to use the file names instead. The clean_names() function from the janitor package will clean up the column names (make them lower case, replace spaces with _ remove special characters etc)\n🎬 Import multiple csv files into one dataframe called tracking\n\n# import multiple data files into one dataframe called tracking\n# using map_df() from purrr package\n# clean the column names up using janitor::clean_names()\ntracking <- files |> \n set_names() |>\n map_dfr(read_table, .id = \"file\") |>\n janitor::clean_names()\n\nYou will get a warning Duplicated column names deduplicated: 'avgX' => 'avgX_1' [15] for each of the files because the csv files each have two columns called avgX. If you click on the tracking dataframe you see is contains the data from all the files.\nNow we can add columns for the type and the concentration by processing the values in the file. The values are like track/343_0.txt so we need to remove .txt and track/ and separate the remaining words into two columns.\n🎬 Process the file column to add columns for the type and the concentration\n\n# extract type and concentration from file name\n# and put them into additopnal separate columns\ntracking <- tracking |> \n mutate(file = str_remove(file, \".txt\")) |>\n mutate(file = str_remove(file, \"track/\")) |>\n extract(file, remove = \n FALSE,\n into = c(\"type\", \"conc\"), \n regex = \"([^_]{2,3})_(.+)\") \n\nError in extract(mutate(mutate(tracking, file = str_remove(file, \".txt\")), : could not find function \"extract\"\n\n\n[^_]{2,3} matches two or three characters that are not _ at the start of the string (^)\n.+ matches one or more characters. The extract() function puts the first match into the first column, type, and the second match into the second column, conc. The remove = FALSE argument means the original column is kept.\nYou now have a dataframe with all the tracking data which is relatively easy to summarise and plot using tools you know.\nThere is an example RStudio project containing this code here: tips. You can download and open directly in RStudio using the following code. Since this is an RStudio Project, do not run the code from inside a project:\n\nusethis::use_course(\"3mmaRand/tips\")" }, { "objectID": "images/images.html#worm-tracking", "href": "images/images.html#worm-tracking", "title": "Image Data Analysis for Group Project", "section": "", - "text": "Save files with systematic names: ev_control.avi 343_control.avi ev_1.avi 343_1.avi ev_2.5.avi 343_2.5.avi\nOpen ImageJ\nOpen video file eg ev_2.5.avi\n\nConvert to 8-bit: Image | Type | 8-bit\nCrop to petri dish: Select then Image | Crop\n\nCalculate average pixel intensity: Image | Stacks | Z Project\n\nProjection type: Average Intensity to create AVG_ev_2.5.avi\n\n\n\n\nSubtract average from image: Process | Image Calculator\n\nImage 1: ev_2.5.avi\n\nOperation: Subtract\nImage 2: AVG_ev_2.5.avi\n\nCreate new window: checked\nOK, Yes to Process all\n\n\nInvert: Edit | Invert\n\nAdjust threshold: Image | Adjust | Threshold\n\nMethod: Default\nThresholding: Default, B&W\nDark background: checked\nAuto or adjust a little but make sure the larvae do not disappear at later points in the video (use the slider)\nApply\n\n\nInvert: Edit | Invert\n\nTrack: Plugins | wrMTrck\n\nSet minSize: 10\nSet maxSize: 400\nSet maxVelocity: 10\nSet maxAreaChange: 200\nSet bendThreshold: 1\n\nImportant: check Save Results File This is different to what you did in the workshop. It will help because the results will be saved automatically rather than to saving from the Results window that other pops up. Consequently, you will be able to save the results files with systematic names relating to their treatments and then read them into R simultaneously. That will also allow you to add information from the name of the file (which has the treatment information) to the resulting dataframes\n\n\nwrMTrck window with the settings listed above shown\n\n\nClick OK. Save to a folder for all the tracking data files. I recommend deleting the “Results of..” part of the name\n\n\nCheck that the Summary window indicates 3 tracks and that the 3 larvae are what is tracked by using the slider on the Result image\nRepeat for all videos\n\nThis is the code you need to import multiple csv files into a single dataframe and add a column with the treatment information from the filename.\nIt assumes\n\nyour files are called type_concentration.txt for example: ev_control.txt 343_control.txt ev_1.txt 343_1.txt ev_2.5.txt 343_2.5.txt.\nthe .txt datafile are in a folder called track inside your working directory\nyou have installed the following packages: tidyverse, janitor\n\n🎬 Load the tidyverse\n\nlibrary(tidyverse)\n\n🎬 Put the file names into a vector we will iterate through\n\n# get a vector of the file names\nfiles <- list.files(path = \"track\", full.names = TRUE )\n\nWe can use map_df() from the purrr package which is one of the tidyverse gems loaded with tidyvserse. map_df() will iterate through files and read them into a dataframe with a specified import function. We are using read_table(). map_df() keeps track of the file by adding an index column called file to the resulting dataframe. Instead of this being a number (1 - 6 here) we can use set_names() to use the file names instead. The clean_names() function from the janitor package will clean up the column names (make them lower case, replace spaces with _ remove special characters etc)\n🎬 Import multiple csv files into one dataframe called tracking\n\n# import multiple data files into one dataframe called tracking\n# using map_df() from purrr package\n# clean the column names up using janitor::clean_names()\ntracking <- files |> \n set_names() |>\n map_dfr(read_table, .id = \"file\") |>\n janitor::clean_names()\n\nYou will get a warning Duplicated column names deduplicated: 'avgX' => 'avgX_1' [15] for each of the files because the csv files each have two columns called avgX. If you click on the tracking dataframe you see is contains the data from all the files.\nNow we can add columns for the type and the concentration by processing the values in the file. The values are like track/343_0.txt so we need to remove .txt and track/ and separate the remaining words into two columns.\n🎬 Process the file column to add columns for the type and the concentration\n\n# extract type and concentration from file name\n# and put them into additopnal separate columns\ntracking <- tracking |> \n mutate(file = str_remove(file, \".txt\")) |>\n mutate(file = str_remove(file, \"track/\")) |>\n extract(file, remove = \n FALSE,\n into = c(\"type\", \"conc\"), \n regex = \"([^_]{2,3})_(.+)\") \n\nError in extract(mutate(mutate(tracking, file = str_remove(file, \".txt\")), : could not find function \"extract\"\n\n\n[^_]{2,3} matches two or three characters that are not _ at the strat of the string (^)\n.+ matches one or more characters. The extract() function puts the first match into the first column, type, and the second match into the second column, conc. The remove = FALSE argument means the original column is kept.\nYou now have a dataframe with all the tracking data which is relatively easy to summarise and plot using tools you know.\nThere is an example RStudio project containing this code here: tips. You can download and open directly in RStudio using the following code. Since this is an RStudio Project, do not run the code from inside a project:\n\nusethis::use_course(\"3mmaRand/tips\")" + "text": "The following ImageJ workflow uses the processing steps you used in workshop 3 with one change. That change is to save the results to file rather than having the results window pop up and saving from there. Or maybe two changes: it also tells you to use meaning systematic file names that will be easy to process when importing data. The RStudio workflow shows you how to import multiple files into one dataframe with columns indicating the treatment.\n\nSave files with systematic names: ev_control.avi 343_control.avi ev_1.avi 343_1.avi ev_2.5.avi 343_2.5.avi\nOpen ImageJ\nOpen video file eg ev_2.5.avi\n\nConvert to 8-bit: Image | Type | 8-bit\nCrop to petri dish: Select then Image | Crop\nCalculate average pixel intensity: Image | Stacks | Z Project\n\nProjection type: Average Intensity to create AVG_ev_2.5.avi\n\n\n\nSubtract average from image: Process | Image Calculator\n\nImage 1: ev_2.5.avi\n\nOperation: Subtract\nImage 2: AVG_ev_2.5.avi\n\nCreate new window: checked\nOK, Yes to Process all\n\n\nInvert: Edit | Invert\nAdjust threshold: Image | Adjust | Threshold\n\nMethod: Default\nThresholding: Default, B&W\nDark background: checked\nAuto or adjust a little but make sure the larvae do not disappear at later points in the video (use the slider)\nApply\n\n\nInvert: Edit | Invert\nTrack: Plugins | wrMTrck\n\nSet minSize: 10\nSet maxSize: 400\nSet maxVelocity: 10\nSet maxAreaChange: 200\nSet bendThreshold: 1\n\nImportant: check Save Results File This is different to what you did in the workshop. It will help because the results will be saved automatically rather than to saving from the Results window that other pops up. Consequently, you will be able to save the results files with systematic names relating to their treatments and then read them into R simultaneously. That will also allow you to add information from the name of the file (which has the treatment information) to the resulting dataframes\n\n\nwrMTrck window with the settings listed above shown\n\n\nClick OK. Save to a folder for all the tracking data files. I recommend deleting the “Results of..” part of the name\n\n\nCheck that the Summary window indicates 3 tracks and that the 3 larvae are what is tracked by using the slider on the Result image\nRepeat for all videos\n\nThis is the code you need to import multiple csv files into a single dataframe and add a column with the treatment information from the file name. This is why systematic file names are good.\nIt assumes\n\nyour files are called type_concentration.txt for example: ev_control.txt 343_control.txt ev_1.txt 343_1.txt ev_2.5.txt 343_2.5.txt.\nthe .txt datafile are in a folder called track inside your working directory\nyou have installed the following packages: tidyverse, janitor\n\n\n🎬 Load the tidyverse\n\nlibrary(tidyverse)\n\n🎬 Put the file names into a vector we will iterate through\n\n# get a vector of the file names\nfiles <- list.files(path = \"track\", full.names = TRUE )\n\nWe can use map_df() from the purrr package which is one of the tidyverse gems loaded with tidyvserse. map_df() will iterate through files and read them into a dataframe with a specified import function. We are using read_table(). map_df() keeps track of the file by adding an index column called file to the resulting dataframe. Instead of this being a number (1 - 6 here) we can use set_names() to use the file names instead. The clean_names() function from the janitor package will clean up the column names (make them lower case, replace spaces with _ remove special characters etc)\n🎬 Import multiple csv files into one dataframe called tracking\n\n# import multiple data files into one dataframe called tracking\n# using map_df() from purrr package\n# clean the column names up using janitor::clean_names()\ntracking <- files |> \n set_names() |>\n map_dfr(read_table, .id = \"file\") |>\n janitor::clean_names()\n\nYou will get a warning Duplicated column names deduplicated: 'avgX' => 'avgX_1' [15] for each of the files because the csv files each have two columns called avgX. If you click on the tracking dataframe you see is contains the data from all the files.\nNow we can add columns for the type and the concentration by processing the values in the file. The values are like track/343_0.txt so we need to remove .txt and track/ and separate the remaining words into two columns.\n🎬 Process the file column to add columns for the type and the concentration\n\n# extract type and concentration from file name\n# and put them into additopnal separate columns\ntracking <- tracking |> \n mutate(file = str_remove(file, \".txt\")) |>\n mutate(file = str_remove(file, \"track/\")) |>\n extract(file, remove = \n FALSE,\n into = c(\"type\", \"conc\"), \n regex = \"([^_]{2,3})_(.+)\") \n\nError in extract(mutate(mutate(tracking, file = str_remove(file, \".txt\")), : could not find function \"extract\"\n\n\n[^_]{2,3} matches two or three characters that are not _ at the start of the string (^)\n.+ matches one or more characters. The extract() function puts the first match into the first column, type, and the second match into the second column, conc. The remove = FALSE argument means the original column is kept.\nYou now have a dataframe with all the tracking data which is relatively easy to summarise and plot using tools you know.\nThere is an example RStudio project containing this code here: tips. You can download and open directly in RStudio using the following code. Since this is an RStudio Project, do not run the code from inside a project:\n\nusethis::use_course(\"3mmaRand/tips\")" }, { "objectID": "core/week-6/study_after_workshop.html", @@ -1369,7 +1369,7 @@ "href": "core/week-2/workshop.html#rstudio-terminal", "title": "Workshop", "section": "RStudio terminal", - "text": "RStudio terminal\nThe RStudio terminal is a convenient interface to the shell without leaving RStudio. It is useful for running commands that are not available in R. For example, you can use it to run other programs like fasqc, git, ftp, ssh\nNavigating your file system\nSeveral commands are frequently used to create, inspect, rename, and delete files and directories.\n$\nThe dollar sign is the prompt (like > on the R console), which shows us that the shell is waiting for input.\nYou can find out where you are using the pwd command, which stands for “print working directory”.\n\npwd\n\n/home/runner/work/BIO00088H-data/BIO00088H-data/core/week-2\n\n\nYou can find out what you can see with ls which stands for “list”.\n\nls\n\ndata\nimages\noverview.qmd\nstudy_after_workshop.html\nstudy_after_workshop.qmd\nstudy_before_workshop.html\nstudy_before_workshop.ipynb\nstudy_before_workshop.qmd\nworkshop.html\nworkshop.qmd\nworkshop.rmarkdown\nworkshop_files\n\n\nYou might have noticed that unlike R, the commands do not have brackets after them. Instead, options (or switches) are given after the command. For example, we can modify the ls command to give us more information with the -l option, which stands for “long”.\n\nls -l\n\ntotal 212\ndrwxr-xr-x 2 runner docker 4096 Nov 20 15:27 data\ndrwxr-xr-x 2 runner docker 4096 Nov 20 15:27 images\n-rw-r--r-- 1 runner docker 1597 Nov 20 15:27 overview.qmd\n-rw-r--r-- 1 runner docker 25383 Nov 20 15:32 study_after_workshop.html\n-rw-r--r-- 1 runner docker 184 Nov 20 15:27 study_after_workshop.qmd\n-rw-r--r-- 1 runner docker 70840 Nov 20 15:32 study_before_workshop.html\n-rw-r--r-- 1 runner docker 4807 Nov 20 15:27 study_before_workshop.ipynb\n-rw-r--r-- 1 runner docker 13029 Nov 20 15:27 study_before_workshop.qmd\n-rw-r--r-- 1 runner docker 44938 Nov 20 15:27 workshop.html\n-rw-r--r-- 1 runner docker 8550 Nov 20 15:27 workshop.qmd\n-rw-r--r-- 1 runner docker 8564 Nov 20 15:33 workshop.rmarkdown\ndrwxr-xr-x 3 runner docker 4096 Nov 20 15:27 workshop_files\n\n\nYou can use more than one option at once. The -h option stands for “human readable” and makes the file sizes easier to understand for humans:\n\nls -hl\n\ntotal 212K\ndrwxr-xr-x 2 runner docker 4.0K Nov 20 15:27 data\ndrwxr-xr-x 2 runner docker 4.0K Nov 20 15:27 images\n-rw-r--r-- 1 runner docker 1.6K Nov 20 15:27 overview.qmd\n-rw-r--r-- 1 runner docker 25K Nov 20 15:32 study_after_workshop.html\n-rw-r--r-- 1 runner docker 184 Nov 20 15:27 study_after_workshop.qmd\n-rw-r--r-- 1 runner docker 70K Nov 20 15:32 study_before_workshop.html\n-rw-r--r-- 1 runner docker 4.7K Nov 20 15:27 study_before_workshop.ipynb\n-rw-r--r-- 1 runner docker 13K Nov 20 15:27 study_before_workshop.qmd\n-rw-r--r-- 1 runner docker 44K Nov 20 15:27 workshop.html\n-rw-r--r-- 1 runner docker 8.4K Nov 20 15:27 workshop.qmd\n-rw-r--r-- 1 runner docker 8.4K Nov 20 15:33 workshop.rmarkdown\ndrwxr-xr-x 3 runner docker 4.0K Nov 20 15:27 workshop_files\n\n\nThe -a option stands for “all” and shows us all the files, including hidden files.\n\nls -alh\n\ntotal 220K\ndrwxr-xr-x 5 runner docker 4.0K Nov 20 15:33 .\ndrwxr-xr-x 6 runner docker 4.0K Nov 20 15:32 ..\ndrwxr-xr-x 2 runner docker 4.0K Nov 20 15:27 data\ndrwxr-xr-x 2 runner docker 4.0K Nov 20 15:27 images\n-rw-r--r-- 1 runner docker 1.6K Nov 20 15:27 overview.qmd\n-rw-r--r-- 1 runner docker 25K Nov 20 15:32 study_after_workshop.html\n-rw-r--r-- 1 runner docker 184 Nov 20 15:27 study_after_workshop.qmd\n-rw-r--r-- 1 runner docker 70K Nov 20 15:32 study_before_workshop.html\n-rw-r--r-- 1 runner docker 4.7K Nov 20 15:27 study_before_workshop.ipynb\n-rw-r--r-- 1 runner docker 13K Nov 20 15:27 study_before_workshop.qmd\n-rw-r--r-- 1 runner docker 44K Nov 20 15:27 workshop.html\n-rw-r--r-- 1 runner docker 8.4K Nov 20 15:27 workshop.qmd\n-rw-r--r-- 1 runner docker 8.4K Nov 20 15:33 workshop.rmarkdown\ndrwxr-xr-x 3 runner docker 4.0K Nov 20 15:27 workshop_files\n\n\nYou can move about with the cd command, which stands for “change directory”. You can use it to move into a directory by specifying the path to the directory:\n\ncd data\npwd\ncd ..\npwd\ncd data\npwd\n\n/home/runner/work/BIO00088H-data/BIO00088H-data/core/week-2/data\n/home/runner/work/BIO00088H-data/BIO00088H-data/core/week-2\n/home/runner/work/BIO00088H-data/BIO00088H-data/core/week-2/data\n\n\nhead 1cq2.pdb\nHEADER OXYGEN STORAGE/TRANSPORT 04-AUG-99 1CQ2 \nTITLE NEUTRON STRUCTURE OF FULLY DEUTERATED SPERM WHALE MYOGLOBIN AT 2.0 \nTITLE 2 ANGSTROM \nCOMPND MOL_ID: 1; \nCOMPND 2 MOLECULE: MYOGLOBIN; \nCOMPND 3 CHAIN: A; \nCOMPND 4 ENGINEERED: YES; \nCOMPND 5 OTHER_DETAILS: PROTEIN IS FULLY DEUTERATED \nSOURCE MOL_ID: 1; \nSOURCE 2 ORGANISM_SCIENTIFIC: PHYSETER CATODON; \nhead -20 data/1cq2.pdb\nHEADER OXYGEN STORAGE/TRANSPORT 04-AUG-99 1CQ2 \nTITLE NEUTRON STRUCTURE OF FULLY DEUTERATED SPERM WHALE MYOGLOBIN AT 2.0 \nTITLE 2 ANGSTROM \nCOMPND MOL_ID: 1; \nCOMPND 2 MOLECULE: MYOGLOBIN; \nCOMPND 3 CHAIN: A; \nCOMPND 4 ENGINEERED: YES; \nCOMPND 5 OTHER_DETAILS: PROTEIN IS FULLY DEUTERATED \nSOURCE MOL_ID: 1; \nSOURCE 2 ORGANISM_SCIENTIFIC: PHYSETER CATODON; \nSOURCE 3 ORGANISM_COMMON: SPERM WHALE; \nSOURCE 4 ORGANISM_TAXID: 9755; \nSOURCE 5 EXPRESSION_SYSTEM: ESCHERICHIA COLI; \nSOURCE 6 EXPRESSION_SYSTEM_TAXID: 562; \nSOURCE 7 EXPRESSION_SYSTEM_VECTOR_TYPE: PLASMID; \nSOURCE 8 EXPRESSION_SYSTEM_PLASMID: PET15A \nKEYWDS HELICAL, GLOBULAR, ALL-HYDROGEN CONTAINING STRUCTURE, OXYGEN STORAGE- \nKEYWDS 2 TRANSPORT COMPLEX \nEXPDTA NEUTRON DIFFRACTION \nAUTHOR F.SHU,V.RAMAKRISHNAN,B.P.SCHOENBORN \nless 1cq2.pdb\nless is a program that displays the contents of a file, one page at a time. It is useful for viewing large files because it does not load the whole file into memory before displaying it. Instead, it reads and displays a few lines at a time. You can navigate forward through the file with the spacebar, and backwards with the b key. Press q to quit.\nA wildcard is a character that can be used as a substitute for any of a class of characters in a search, The most common wildcard characters are the asterisk (*) and the question mark (?).\nls *.csv\ncp stands for “copy”. You can copy a file from one directory to another by giving cp the path to the file you want to copy and the path to the destination directory.\ncp 1cq2.pdb copy_of_1cq2.pdb\ncp 1cq2.pdb ../copy_of_1cq2.pdb\ncp 1cq2.pdb ../bob.txt\nTo delete a file use the rm command, which stands for “remove”.\nrm ../bob.txt\nbut be careful because the file will be gone forever. There is no “are you sure?” or undo.\nTo move a file from one directory to another, use the mv command. mv works like cp except that it also deletes the original file.\nmv ../copy_of_1cq2.pdb .\nMake a directory\nmkdir mynewdir" + "text": "RStudio terminal\nThe RStudio terminal is a convenient interface to the shell without leaving RStudio. It is useful for running commands that are not available in R. For example, you can use it to run other programs like fasqc, git, ftp, ssh\nNavigating your file system\nSeveral commands are frequently used to create, inspect, rename, and delete files and directories.\n$\nThe dollar sign is the prompt (like > on the R console), which shows us that the shell is waiting for input.\nYou can find out where you are using the pwd command, which stands for “print working directory”.\n\npwd\n\n/home/runner/work/BIO00088H-data/BIO00088H-data/core/week-2\n\n\nYou can find out what you can see with ls which stands for “list”.\n\nls\n\ndata\nimages\noverview.qmd\nstudy_after_workshop.html\nstudy_after_workshop.qmd\nstudy_before_workshop.html\nstudy_before_workshop.ipynb\nstudy_before_workshop.qmd\nworkshop.html\nworkshop.qmd\nworkshop.rmarkdown\nworkshop_files\n\n\nYou might have noticed that unlike R, the commands do not have brackets after them. Instead, options (or switches) are given after the command. For example, we can modify the ls command to give us more information with the -l option, which stands for “long”.\n\nls -l\n\ntotal 212\ndrwxr-xr-x 2 runner docker 4096 Nov 20 15:35 data\ndrwxr-xr-x 2 runner docker 4096 Nov 20 15:35 images\n-rw-r--r-- 1 runner docker 1597 Nov 20 15:35 overview.qmd\n-rw-r--r-- 1 runner docker 25383 Nov 20 15:39 study_after_workshop.html\n-rw-r--r-- 1 runner docker 184 Nov 20 15:35 study_after_workshop.qmd\n-rw-r--r-- 1 runner docker 70840 Nov 20 15:39 study_before_workshop.html\n-rw-r--r-- 1 runner docker 4807 Nov 20 15:35 study_before_workshop.ipynb\n-rw-r--r-- 1 runner docker 13029 Nov 20 15:35 study_before_workshop.qmd\n-rw-r--r-- 1 runner docker 44938 Nov 20 15:35 workshop.html\n-rw-r--r-- 1 runner docker 8550 Nov 20 15:35 workshop.qmd\n-rw-r--r-- 1 runner docker 8564 Nov 20 15:39 workshop.rmarkdown\ndrwxr-xr-x 3 runner docker 4096 Nov 20 15:35 workshop_files\n\n\nYou can use more than one option at once. The -h option stands for “human readable” and makes the file sizes easier to understand for humans:\n\nls -hl\n\ntotal 212K\ndrwxr-xr-x 2 runner docker 4.0K Nov 20 15:35 data\ndrwxr-xr-x 2 runner docker 4.0K Nov 20 15:35 images\n-rw-r--r-- 1 runner docker 1.6K Nov 20 15:35 overview.qmd\n-rw-r--r-- 1 runner docker 25K Nov 20 15:39 study_after_workshop.html\n-rw-r--r-- 1 runner docker 184 Nov 20 15:35 study_after_workshop.qmd\n-rw-r--r-- 1 runner docker 70K Nov 20 15:39 study_before_workshop.html\n-rw-r--r-- 1 runner docker 4.7K Nov 20 15:35 study_before_workshop.ipynb\n-rw-r--r-- 1 runner docker 13K Nov 20 15:35 study_before_workshop.qmd\n-rw-r--r-- 1 runner docker 44K Nov 20 15:35 workshop.html\n-rw-r--r-- 1 runner docker 8.4K Nov 20 15:35 workshop.qmd\n-rw-r--r-- 1 runner docker 8.4K Nov 20 15:39 workshop.rmarkdown\ndrwxr-xr-x 3 runner docker 4.0K Nov 20 15:35 workshop_files\n\n\nThe -a option stands for “all” and shows us all the files, including hidden files.\n\nls -alh\n\ntotal 220K\ndrwxr-xr-x 5 runner docker 4.0K Nov 20 15:39 .\ndrwxr-xr-x 6 runner docker 4.0K Nov 20 15:39 ..\ndrwxr-xr-x 2 runner docker 4.0K Nov 20 15:35 data\ndrwxr-xr-x 2 runner docker 4.0K Nov 20 15:35 images\n-rw-r--r-- 1 runner docker 1.6K Nov 20 15:35 overview.qmd\n-rw-r--r-- 1 runner docker 25K Nov 20 15:39 study_after_workshop.html\n-rw-r--r-- 1 runner docker 184 Nov 20 15:35 study_after_workshop.qmd\n-rw-r--r-- 1 runner docker 70K Nov 20 15:39 study_before_workshop.html\n-rw-r--r-- 1 runner docker 4.7K Nov 20 15:35 study_before_workshop.ipynb\n-rw-r--r-- 1 runner docker 13K Nov 20 15:35 study_before_workshop.qmd\n-rw-r--r-- 1 runner docker 44K Nov 20 15:35 workshop.html\n-rw-r--r-- 1 runner docker 8.4K Nov 20 15:35 workshop.qmd\n-rw-r--r-- 1 runner docker 8.4K Nov 20 15:39 workshop.rmarkdown\ndrwxr-xr-x 3 runner docker 4.0K Nov 20 15:35 workshop_files\n\n\nYou can move about with the cd command, which stands for “change directory”. You can use it to move into a directory by specifying the path to the directory:\n\ncd data\npwd\ncd ..\npwd\ncd data\npwd\n\n/home/runner/work/BIO00088H-data/BIO00088H-data/core/week-2/data\n/home/runner/work/BIO00088H-data/BIO00088H-data/core/week-2\n/home/runner/work/BIO00088H-data/BIO00088H-data/core/week-2/data\n\n\nhead 1cq2.pdb\nHEADER OXYGEN STORAGE/TRANSPORT 04-AUG-99 1CQ2 \nTITLE NEUTRON STRUCTURE OF FULLY DEUTERATED SPERM WHALE MYOGLOBIN AT 2.0 \nTITLE 2 ANGSTROM \nCOMPND MOL_ID: 1; \nCOMPND 2 MOLECULE: MYOGLOBIN; \nCOMPND 3 CHAIN: A; \nCOMPND 4 ENGINEERED: YES; \nCOMPND 5 OTHER_DETAILS: PROTEIN IS FULLY DEUTERATED \nSOURCE MOL_ID: 1; \nSOURCE 2 ORGANISM_SCIENTIFIC: PHYSETER CATODON; \nhead -20 data/1cq2.pdb\nHEADER OXYGEN STORAGE/TRANSPORT 04-AUG-99 1CQ2 \nTITLE NEUTRON STRUCTURE OF FULLY DEUTERATED SPERM WHALE MYOGLOBIN AT 2.0 \nTITLE 2 ANGSTROM \nCOMPND MOL_ID: 1; \nCOMPND 2 MOLECULE: MYOGLOBIN; \nCOMPND 3 CHAIN: A; \nCOMPND 4 ENGINEERED: YES; \nCOMPND 5 OTHER_DETAILS: PROTEIN IS FULLY DEUTERATED \nSOURCE MOL_ID: 1; \nSOURCE 2 ORGANISM_SCIENTIFIC: PHYSETER CATODON; \nSOURCE 3 ORGANISM_COMMON: SPERM WHALE; \nSOURCE 4 ORGANISM_TAXID: 9755; \nSOURCE 5 EXPRESSION_SYSTEM: ESCHERICHIA COLI; \nSOURCE 6 EXPRESSION_SYSTEM_TAXID: 562; \nSOURCE 7 EXPRESSION_SYSTEM_VECTOR_TYPE: PLASMID; \nSOURCE 8 EXPRESSION_SYSTEM_PLASMID: PET15A \nKEYWDS HELICAL, GLOBULAR, ALL-HYDROGEN CONTAINING STRUCTURE, OXYGEN STORAGE- \nKEYWDS 2 TRANSPORT COMPLEX \nEXPDTA NEUTRON DIFFRACTION \nAUTHOR F.SHU,V.RAMAKRISHNAN,B.P.SCHOENBORN \nless 1cq2.pdb\nless is a program that displays the contents of a file, one page at a time. It is useful for viewing large files because it does not load the whole file into memory before displaying it. Instead, it reads and displays a few lines at a time. You can navigate forward through the file with the spacebar, and backwards with the b key. Press q to quit.\nA wildcard is a character that can be used as a substitute for any of a class of characters in a search, The most common wildcard characters are the asterisk (*) and the question mark (?).\nls *.csv\ncp stands for “copy”. You can copy a file from one directory to another by giving cp the path to the file you want to copy and the path to the destination directory.\ncp 1cq2.pdb copy_of_1cq2.pdb\ncp 1cq2.pdb ../copy_of_1cq2.pdb\ncp 1cq2.pdb ../bob.txt\nTo delete a file use the rm command, which stands for “remove”.\nrm ../bob.txt\nbut be careful because the file will be gone forever. There is no “are you sure?” or undo.\nTo move a file from one directory to another, use the mv command. mv works like cp except that it also deletes the original file.\nmv ../copy_of_1cq2.pdb .\nMake a directory\nmkdir mynewdir" }, { "objectID": "core/week-2/workshop.html#differences-between-r-and-python",