Merge pull request #281 from ipohner/ip_rr-prep

Small lesson updates / fixes

Author: bast

content/dependencies.md

@@ -12,7 +12,7 @@
 
 ```{instructor-note}
 - 10 min teaching
-- 10 min demo
+- 10 min exercise/demo
 ```
 
 Our codes often depend on other codes that in turn depend on other codes ...
@@ -52,16 +52,15 @@ When we create recipes, we often use tools created by others (libraries) [Midjou
 
 ## Dependency and environment management
 
-**Conda, Anaconda, pip, virtualenv, Pipenv, pyenv, Poetry, requirements.txt,
-environment.yml, renv**, ..., these tools try to solve the following problems:
+Tools like **Conda, Anaconda, pip, virtualenv, Pipenv, pyenv, Poetry, renv** and files to record dependencies like **requirements.txt** and **environment.yml** try to solve the following problems:
 
 - **Defining a specific set of dependencies**
 - **Installing those dependencies** mostly automatically
 - **Recording the versions** for all dependencies
 - **Isolate environments**
-  - On your computer for projects so they can use different software
+  - On your computer for projects, so they can use different software
   - Isolate environments on computers with many users (and allow self-installations)
-- Using **different package versions** per project (also e.g. Python/R versions)
+- Using **different package versions** per project (also, e.g., Python/R versions)
 - Provide tools and services to **share packages**
 
 Isolated environments are also useful because they help you make sure
@@ -73,7 +72,7 @@ more reproducible it is.
 
 ---
 
-## Demo
+## Exercise / Demo
 
 ``````{challenge} Dependencies-1: Time-capsule of dependencies
 Situation: 5 students (A, B, C, D, E) wrote a code that depends on a couple of libraries.
@@ -247,17 +246,17 @@ Answer in the collaborative document:
   **A**: It will be tedious to collect the dependencies one by one. And after
   the tedious process you will still not know which versions they have used.
 
-  **B**: If there is no standard file to look for and look at and it might
-  become very difficult for to create the software environment required to
-  run the software. But at least we know the list of libraries. But we don't
+  **B**: If there is no standard file to look for and look at, it might
+  become very difficult to create the software environment required to
+  run the software. At least we know the list of libraries, but we don't
   know the versions.
 
   **C**: Having a standard file listing dependencies is definitely better
   than nothing. However, if the versions are not specified, you or someone
   else might run into problems with dependencies, deprecated features,
   changes in package APIs, etc.
 
-  **D** and **E**: In both these cases exact versions of all dependencies are
+  **D** and **E**: In both of these cases exact versions of all dependencies are
   specified and one can recreate the software environment required for the
   project. One problem with the dependencies that come from GitHub is that
   they might have disappeared (what if their authors deleted these
@@ -277,7 +276,7 @@ information?
   `````{tabs}
     ````{group-tab} Conda
       We start from an existing conda environment. Try this either with your own project or inside the "coderefinery" conda
-      environment. For demonstration puprposes, you can also create an environment with:
+      environment. For demonstration purposes, you can also create an environment with:
 
       ```console
       $ conda env create -f myenv.yml
@@ -375,6 +374,6 @@ information?
 ``````
 
 ```{keypoints}
-- Recording dependencies with versions can make it easier for the next person to execute your code
-- There are many tools to record dependencies and separate environments
+- Recording dependencies with versions can make it easier for the next person to execute your code.
+- There are many tools to record dependencies and separate environments.
 ```

content/environments.md

@@ -14,7 +14,7 @@
 ## What is a container?
 
 Imagine if you didn't have to install things yourself, but instead you could
-get a computer with the exact software for a task pre-installed? Containers
+get a computer with the exact software for a task pre-installed. Containers
 effectively do that, with various advantages and disadvantages. They are
 **like an entire operating system with software installed, all in one file**.
 
@@ -30,7 +30,7 @@ From [reddit](https://www.reddit.com/r/ProgrammerHumor/comments/cw58z7/it_works_
 - Container definition files <-> like a blueprint to build a kitchen with all
   utensils in which the recipe can be prepared.
 - Container images <-> showroom kitchens
-- Containers <-> A real connected kitchen
+- Containers <-> a real connected kitchen
 
 Just for fun: which operating systems do the following example kitchens represent?
   `````{tabs}
@@ -69,15 +69,15 @@ Just for fun: which operating systems do the following example kitchens represen
 - A container image is like a piece of paper with all the operating system on it. When you run it,
   a transparent sheet is placed on top to form a container. The container runs and writes only on
   that transparent sheet (and what other mounts have been layered on top). When you are done,
-  transparency is thrown away. It can be repeated as often as you want, and base is always the same.
-- Definition files (e.g. Dockerfile or Singularity definition file) are text
+  the transparent sheet is thrown away. This can be repeated as often as you want, and base is always the same.
+- Definition files (e.g., Dockerfile or Singularity definition file) are text
   files that contain a series of instructions to build container images.
 
 ## You may have use for containers in different ways
 
 - **Installing a certain software is tricky**, or not supported for your operating system? - Check if an image is available and run the software from a container instead!
 - You want to make sure your colleagues are using the **same environment** for running your code? - Provide them an image of your container!
-  - If this does not work, because they are using a different architecture than you do? - Provide a definition file for them to **build the image suitable to their computers**. This does not create the exact environment as you have, but in most cases similar enough.
+  - If this does not work, because they are using a different architecture than you do? - Provide a definition file for them to **build the image suitable for their computers**. This does not create the exact environment you have, but in most cases a similar enough one.
 
 ## The container recipe
 
@@ -127,20 +127,20 @@ important problems:
 - A mechanism to "send the computer to the data" when the **dataset is too large** to transfer.
 - **Installing software into a file** instead of into your computer (removing
   a file is often easier than uninstalling software if you suddenly regret an
-  installation)
+  installation).
 
 However, containers may also have some drawbacks:
 
 - Can be used to hide away software installation problems and thereby
   **discourage good software development practices**.
 - Instead of "works on my machine" problem: **"works only in this container"** problem?
-- They can be **difficult to modify**
-- Container **images can become large**
+- They can be **difficult to modify**.
+- Container **images can become large**.
 
 ```{danger}
 Use only **official and trusted images**!  Not all images can be trusted! There
-have been examples of contaminated images so investigate before using images
-blindly. Apply same caution as installing software packages from untrusted
+have been examples of contaminated images, so investigate before using images
+blindly. Apply the same caution as when installing software packages from untrusted
 package repositories.
 ```
 
@@ -228,14 +228,14 @@ package repositories.
       ```
 
       ```{solution}
-      - Line 2: "ubuntu:latest" will mean something different 3 years in future.
+      - Line 2: "ubuntu:latest" will mean something different 3 years into the future.
       - Lines 11-12: The compiler gcc and the library libgomp1 will have evolved.
       - Line 30: The container uses requirements.txt to build the virtual environment but we don't see
         here what libraries the code depends on.
       - Line 33: Data is copied in from the hard disk of the person who created it. Hopefully we can find the data somewhere.
       - Line 35: The library fancylib has been built outside the container and copied in but we don't see here how it was done.
-      - Python version will be different then and hopefully the code still runs then.
-      - Singularity/Apptainer will have also evolved by then. Hopefully this definition file then still works.
+      - The Python version will be different and hopefully the code still runs.
+      - Singularity/Apptainer will have also evolved by then. Hopefully this definition file still works.
       - No contact address to ask more questions about this file.
       - (Can you find more? Please contribute more points.)
       ```
@@ -251,7 +251,7 @@ package repositories.
 ````{exercise} (optional) Containers-2: Installing the impossible.
 
 When you are missing privileges for installing certain software tools, containers can come handy.
-Here we build a Singularity/Apptainer container for installing `cowsay` and `lolcat` Linux programs.
+Here we build a Singularity/Apptainer container for installing the `cowsay` and `lolcat` Linux programs.
 
 1. Make sure you have apptainer installed:
    ```console
@@ -266,12 +266,12 @@ Here we build a Singularity/Apptainer container for installing `cowsay` and `lol
    $ export APPTAINER_TMPDIR="./temp/"
    ```
 
-3. Build the container from the following definition file above.
+3. Build the container from the container recipe file introduced above.
    ```console
    apptainer build cowsay.sif cowsay.def
    ```
 
-4. Let's test the container by entering into it with a shell terminal
+4. Let's test the container by entering into it with a shell terminal:
    ```console
    $ apptainer shell cowsay.sif
    ```
@@ -317,6 +317,6 @@ the Docker containers through Singularity/Apptainer.
 - [Carpentries incubator lesson on Singularity/Apptainer](https://carpentries-incubator.github.io/singularity-introduction/)
 
 ```{keypoints}
-- Containers can be helpful if complex setups are needed to running a specific software
-- They can also be helpful for prototyping without "messing up" your own computing environment, or for running software that requires a different operating system than your own
+- Containers can be helpful if complex setups are needed to run a specific software.
+- They can also be helpful for prototyping without "messing up" your own computing environment, or for running software that requires a different operating system than your own.
 ```

content/index.rst

@@ -24,7 +24,8 @@ reproducible environments and computational steps** for our future selves and ot
 .. prereq::
 
    You need to install
-   `Git, Python, and Snakemake <https://coderefinery.github.io/installation/>`__.
+   `Git, Python, and Snakemake <https://coderefinery.github.io/installation/>`__ 
+   (part of CodeRefinery Conda environment).
 
    If you wish to follow in the terminal and are new to the command line, we
    recorded a `short shell crash course <https://youtu.be/xbTTDLA3txI>`__.

content/intro.md

@@ -33,7 +33,7 @@ This lesson on general **Reproducibility**: Preparing code to be usable by you a
 
 This includes organizing your projects on your own computer and recording your computational steps, dependencies and computing environment. 
 
-We will also mention a few tools and platforms for sharing data (**"Here is my data"**) and research outputs(**"Here are my results"**) in the **social coding** lesson, but they are not the focus of this workshop.  
+We will also mention a few tools and platforms for sharing data (**"Here is my data"**) and research outputs (**"Here are my results"**) in the **social coding** lesson, but they are not the focus of this workshop.  
 
 ## Small steps towards reproducible research 
 

content/motivation.md

@@ -44,7 +44,7 @@ smaller.
 ## Levels of reproducibility
 
 A published article is like the top of a pyramid. It rests on multiple
-levels that each contributes to its reproducibility.
+levels, each contributing to its reproducibility.
 
 ```{figure} img/repro-pyramid.png
 :alt: Reproducibility pyramid
@@ -74,5 +74,5 @@ This also means that you can think about it from the beginning of your research
 ````
 
 ```{keypoints}
-- Without reproducibility in scientific computing, everyone would have to start a new project / code from scratch
+- Without reproducibility in scientific computing, everyone would have to start a new project / code from scratch.
 ```

content/organizing-projects.md

@@ -18,12 +18,12 @@ Let's go over some of the basic things which people have found to work (and not
 - Project files in a **single directory**
 - **Different projects** should have **separate directories**
 - Use **consistent and informative directory structure**
-  - Avoid spaces in directory and file names – use `-`, `_` or CamelCase instead (nicer for computers to handle).
+  - Avoid spaces in directory and file names – use `-`, `_` or CamelCase instead (nicer for computers to handle)
 - If you need to separate public/private directories, 
   - put them separately in public and private Git repositories, or
   - use `.gitignore` to exclude the private information from being tracked
 - Add a **README file** to describe the project and instructions on reproducing the results
-- If you want to use the **same code in multiple projects**, host it on GitHub (or similar) and clone it into each of your project directories.
+- If you want to use the **same code in multiple projects**, host it on GitHub (or similar) and clone it into each of your project directories
 
 A project directory can look something like this:
 

content/where-to-go.md

@@ -17,7 +17,7 @@ However, you will not always need all of them. As with so many things, it again
 - You will want to consider workflow tools:
   - When processing many files with many steps
   - Steps or files may change
-  - Your main script, connecting your steps gets very long
+  - Your main script, connecting your steps, gets very long
   - You are still collecting your input data
   - ...
 
@@ -34,10 +34,10 @@ However, you will not always need all of them. As with so many things, it again
 
 ## Important for every project
 
-- Clear file structure for your project
+- A Clear directory/file structure for your project.
 - Record your workflow and write it down in a script file.
-- Create a dependency list and keep it updated, optimally in an environment file
-- At least consider the possibility that someone, maybe you may want to reproduce your work
+- Create a dependency list and keep it updated, optimally in an environment file.
+- At least consider the possibility that someone, maybe you, may want to reproduce your work:
   - Can you do something (small) to make it easier?
   - If you have ideas, but no time: add an issue to your repository; maybe someone else wants to help.
 
@@ -52,6 +52,6 @@ Do you want to practice your reproducibility skills and get inspired by working
 ```
 
 ```{keypoints}
-- Not everything in this lesson might be useful right now, but it is good to know that these things exist if you ever get in a situation that would require such solution.
+- Not everything in this lesson might be useful right now, but it is good to know that these things exist if you ever get in a situation that would require such solutions.
 - Caring about reproducibility makes work easier for the next person working on the project - and that might be you in a few years!
 ```

content/workflow-management.md

@@ -13,7 +13,7 @@
 
 ```{instructor-note}
 - 5 min teaching
-- 15 min demo
+- 15 min exercise/demo
 ```
 
 
@@ -69,7 +69,7 @@ steps in precisely this order, as we would run them manually, one after another.
   The advantage of this solution compared to processing one by one is more automation: We can generate all.
   This is not only easier, it is also less error-prone.
 
-  Yes, the scripted solution can be reproducible. But could you easily run it e.g. on a Windows computer?
+  Yes, the scripted solution can be reproducible. But could you easily run it, e.g., on a Windows computer?
 
   If we had more steps and once steps start to be time-consuming, a limitation of
   a scripted solution is that it tries to run all steps always. Rerunning only
@@ -102,20 +102,21 @@ cloud service:
 
 **On your own computer**:
 - Install the necessary tools 
-- Activate the [coderefinery conda environment](https://coderefinery.github.io/installation/conda-environment/) with `conda activate coderefinery`.
+- Activate the [CodeRefinery Conda environment](https://coderefinery.github.io/installation/conda/) with `conda activate coderefinery`.
 - Clone the word-count repository:
   ```console
   $ git clone https://github.com/coderefinery/word-count.git
   ```
 
 **On Binder**:
+
 We can also use the cloud service [Binder](https://mybinder.org/) to make sure
 we all have the same computing environment.  This is interesting from a
 reproducible research point of view and it's explained further in the [Jupyter
 lesson](https://coderefinery.github.io/jupyter/sharing/) how this is even
 possible.
 - Go to <https://github.com/coderefinery/word-count> and click on the "launch binder" badge in the README.
-- Once it get started, you can open a new Terminal from the **new** menu (top right) and select **Terminal**.
+- Once it gets started, you can open a new **Terminal** from the Launcher or via **File > New > Terminal**.
 ````
 
 ````{exercise} Workflow-1: Workflow solution using Snakemake
@@ -223,10 +224,10 @@ Rules that have yet to be completed are indicated with solid outlines, while alr
 - **Cross-platform** (Windows, MacOS, Linux) and compatible with all High Performance Computing (HPC) schedulers:
   same workflow works without modification and scales appropriately whether on a laptop or cluster.
 - If several workflow steps are independent of each other, and you have multiple cores available, Snakemake can run them **in parallel**.
-- Is is possible to define **isolated software environments** per rule, e.g. by adding `conda: 'environment.yml'` to a rule.
-- Also possible to run workflows in Docker or Apptainer **containers** e.g. by adding `container: 'docker://some-org/some-tool#2.3.1'` to a rule.
+- It is possible to define **isolated software environments** per rule, e.g. by adding `conda: 'environment.yml'` to a rule.
+- It is also possible to run workflows in Docker or Apptainer **containers**, e.g. by adding `container: 'docker://some-org/some-tool#2.3.1'` to a rule.
 - [Heavily used in bioinformatics](https://twitter.com/carl_witt/status/1103951128046301185), but is **completely general**.
-- Nice functionality for archiving the workflow, see: [the official documentation](https://snakemake.readthedocs.io/en/stable/snakefiles/deployment.html#sustainable-and-reproducible-archiving)
+- Nice functionality for archiving the workflow, see: [the official Snakemake documentation](https://snakemake.readthedocs.io/en/stable/snakefiles/deployment.html#sustainable-and-reproducible-archiving)
 
 Tools like Snakemake help us with **reproducibility** by supporting us with **automation**, **scalability** and **portability** of our workflows.
 
@@ -241,6 +242,6 @@ Tools like Snakemake help us with **reproducibility** by supporting us with **au
 - [{targets} R package - make-like pipeline tool for R](https://books.ropensci.org/targets/)
 
 ```{keypoints}
-- Computational steps can be recorded in many ways
-- Workflow tools can help, if there are many steps to be executed and/or many datasets to be processed
+- Computational steps can be recorded in many ways.
+- Workflow tools can help if there are many steps to be executed and/or many datasets to be processed.
 ```