-
diff --git a/_includes/base_path.html b/_includes/base_path.html
new file mode 100644
index 0000000..7efb357
--- /dev/null
+++ b/_includes/base_path.html
@@ -0,0 +1,27 @@
+{% comment %}
+This is adapted from: https://ricostacruz.com/til/relative-paths-in-jekyll
+
+`page.url` gives the URL of the current page with a leading /:
+
+- when the URL ends with the extension (e.g., /foo/bar.html) then we can get
+ the depth by counting the number of / and remove - 1
+- when the URL ends with a / (e.g. /foo/bar/) then the number / gives the depth
+ directly
+{% endcomment %}
+
+{% assign relative_root_path = '' %}
+
+{% assign last_char = page.url | slice: -1 %}
+
+{% if last_char == "/"}
+{% assign offset = 0 %}
+{% else %}
+{% assign offset = 1 %}
+{% endif %}
+
+{% assign depth = page.url | split: '/' | size | minus: offset %}
+{% if depth <= 1 %}{% assign relative_root_path = '.' %}
+{% elsif depth == 2 %}{% assign relative_root_path = '..' %}
+{% elsif depth == 3 %}{% assign relative_root_path = '../..' %}
+{% elsif depth == 4 %}{% assign relative_root_path = '../../..' %}
+{% endif %}
diff --git a/_includes/carpentries.html b/_includes/carpentries.html
index a0e0181..c032bd5 100644
--- a/_includes/carpentries.html
+++ b/_includes/carpentries.html
@@ -1,44 +1,70 @@
{% comment %}
- General description of Software and Data Carpentry.
+ General description of Software, Data, and Library Carpentry.
{% endcomment %}
+
+{% include base_path.html %}
+
- 
+ 
+
- Since 1998,
- Software Carpentry
- has been teaching researchers in science, engineering, medicine, and related disciplines
- the computing skills they need to get more done in less time and with less pain.
- Its volunteer instructors have run hundreds of events
- for thousands of learners in the past two and a half years.
+
+The Carpentries comprises + Software Carpentry, Data Carpentry, and Library Carpentry communities of Instructors, Trainers, + Maintainers, helpers, and supporters who share a mission to teach + foundational coding and data science skills to researchers and people + working in library- and information-related roles. In January, + 2018, The Carpentries was formed by the merger of Software Carpentry and + Data Carpentry. Library Carpentry became an official Carpentries Lesson Program in November 2018.
+ +While individual lessons and workshops continue to be run under each + lesson project, The Carpentries provide overall staffing and governance, as + well as support for assessment, instructor training and mentoring. + Memberships are joint, and the Carpentries project maintains a shared Code + of Conduct. The Carpentries is a fiscally sponsored project of Community + Initiatives, a registered 501(c)3 non-profit based in California, USA.
+
+
+ 
+
+
+
+
Since 1998, Software Carpentry has + been teaching researchers across all disciplines the foundational coding + skills they need to get more done in less time and with less pain. Its + volunteer instructors have run hundreds of events for thousands of learners + around the world. Now that all research involves some degree of + computational work, whether with big data, cloud computing, or simple task + automation, these skills are needed more than ever.
- 
+ 
+
- Data Carpentry develops and teaches workshops on the fundamental data skills needed to conduct research.
- Its target audience is researchers who have little to no prior computational experience,
- and its lessons are domain specific,
- building on learners' existing knowledge to enable them to quickly apply skills learned to their own research.
+
Data Carpentry develops and teaches + workshops on the fundamental data skills needed to conduct research. Its + target audience is researchers who have little to no prior computational + experience, and its lessons are domain specific, building on learners' + existing knowledge to enable them to quickly apply skills learned to their + own research. Data Carpentry workshops take researchers through the entire + data life cycle.
- 
+ 
+
- Library Carpentry is made by librarians to help librarians
- automate repetitive, boring, error-prone tasks;
- create, maintain and analyse sustainable and reusable data;
- work effectively with IT and systems colleagues;
- better understand the use of software in research;
- and much more.
- Library Carpentry was the winner of the 2016
- British Library Labs Teaching and Learning Award.
+
Library Carpentry develops lessons and + teaches workshops for and with people working in library- and + information-related roles. Its goal is to create an on-ramp to empower this + community to use software and data in their own work, as well as be + advocates for and train others in efficient, effective and reproducible data + and software practices.
-
-
+
+
+
{% if page.previous.url %}
- previous episode
+ previous episode
{% else %}
- lesson home
+ lesson home
{% endif %}
-
+
{% if include.episode_navbar_title %}
- {{ site.title }}
+ {{ site.title }}
{% endif %}
-
-
+
+
{% if page.next.url %}
- next episode
+ next episode
{% else %}
- lesson home
+ lesson home
{% endif %}
diff --git a/_includes/favicons.html b/_includes/favicons.html
new file mode 100644
index 0000000..8a50b4d
--- /dev/null
+++ b/_includes/favicons.html
@@ -0,0 +1,33 @@
+{% assign favicon_url = relative_root_path | append: '/assets/favicons/' | append: site.carpentry %}
+
+{% if site.carpentry == 'swc' %}
+{% assign carpentry = 'Software Carpentry' %}
+{% elsif site.carpentry == 'dc' %}
+{% assign carpentry = 'Data Carpentry' %}
+{% elsif site.carpentry == 'lc' %}
+{% assign carpentry = 'Library Carpentry' %}
+{% elsif site.carpentry == 'cp' %}
+{% assign carpentry = 'The Carpentries' %}
+{% endif %}
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/_includes/gh_variables.html b/_includes/gh_variables.html
new file mode 100644
index 0000000..3fdae4a
--- /dev/null
+++ b/_includes/gh_variables.html
@@ -0,0 +1,45 @@
+{% comment %}
+When rendering websites locally, `site.github.url` doesn't get resolved properly
+unless a GitHub Personal Access Token is set up and available in the
+environment. This leads to warnings and errors when trying to serve the site
+locally. To work around this, we use the `jekyll.environment` variable which is
+set to `development` when rendering the site locally, and set to `production` on
+GitHub where `site.github.url` is defined.
+{% endcomment %}
+
+{% if jekyll.environment == "production" %}
+
+{% comment %}
+First, get the name of the repository
+{% endcomment %}
+{% assign repo_name = site.github.repository_name %}
+
+{% comment %}
+`site.github.public_repositories` contains comprehensive information for all public repositories for the organization. We use `where` to extract the part
+of the metadata that is relevant to the present repository.
+{% endcomment %}
+{% assign repo_info = site.github.public_repositories | where: "name", repo_name %}
+
+{% comment %}
+Now, we can extract the default branch for the repo
+{% endcomment %}
+{% assign default_branch = repo_info[0].default_branch %}
+
+{% comment %}
+Other variables requested by the template
+{% endcomment %}
+{% assign repo_url = site.github.repository_url %}
+{% assign search_domain_url = site.github.url %}
+{% assign project_title = site.github.project_title %}
+{% assign source_branch = site.github.source.branch %}
+
+{% elsif jekyll.environment == "development" %}
+
+{% assign repo_name = "" %}
+{% assign repo_url = "" %}
+{% assign default_branch = "" %}
+{% assign search_domain_url = "" %}
+{% assign project_title = "" %}
+{% assign source_branch = "" %}
+
+{% endif %}
diff --git a/_includes/javascript.html b/_includes/javascript.html
index a2066c2..fcc74e6 100644
--- a/_includes/javascript.html
+++ b/_includes/javascript.html
@@ -1,9 +1,9 @@
{% comment %}
- Javascript used in lesson and workshop pages.
+ JavaScript used in lesson and workshop pages.
{% endcomment %}
-
-
-
+
+
+
+{% comment %}
+Create anchor for each one of the episodes.
+{% endcomment %}
+{% for episode in site.episodes %}
+
+{% endfor %}
+
+{% include links.md %}
diff --git a/bin/boilerplate/_extras/guide.md b/bin/boilerplate/_extras/guide.md
new file mode 100644
index 0000000..50f266f
--- /dev/null
+++ b/bin/boilerplate/_extras/guide.md
@@ -0,0 +1,6 @@
+---
+title: "Instructor Notes"
+---
+FIXME
+
+{% include links.md %}
diff --git a/bin/boilerplate/aio.md b/bin/boilerplate/aio.md
new file mode 100644
index 0000000..523e7dd
--- /dev/null
+++ b/bin/boilerplate/aio.md
@@ -0,0 +1,37 @@
+---
+---
+
+{% include base_path.html %}
+
+
+{% comment %}
+Create an anchor for every episode.
+{% endcomment %}
+{% for episode in site.episodes %}
+
+{% endfor %}
diff --git a/bin/boilerplate/index.md b/bin/boilerplate/index.md
new file mode 100644
index 0000000..95ccdbd
--- /dev/null
+++ b/bin/boilerplate/index.md
@@ -0,0 +1,17 @@
+---
+layout: lesson
+root: . # Is the only page that doesn't follow the pattern /:path/index.html
+permalink: index.html # Is the only page that doesn't follow the pattern /:path/index.html
+---
+FIXME: home page introduction
+
+
+
+{% comment %} This is a comment in Liquid {% endcomment %}
+
+> ## Prerequisites
+>
+> FIXME
+{: .prereq}
+
+{% include links.md %}
diff --git a/bin/boilerplate/reference.md b/bin/boilerplate/reference.md
new file mode 100644
index 0000000..8c82616
--- /dev/null
+++ b/bin/boilerplate/reference.md
@@ -0,0 +1,9 @@
+---
+layout: reference
+---
+
+## Glossary
+
+FIXME
+
+{% include links.md %}
diff --git a/bin/boilerplate/setup.md b/bin/boilerplate/setup.md
new file mode 100644
index 0000000..b8c5032
--- /dev/null
+++ b/bin/boilerplate/setup.md
@@ -0,0 +1,7 @@
+---
+title: Setup
+---
+FIXME
+
+
+{% include links.md %}
diff --git a/bin/chunk-options.R b/bin/chunk-options.R
index d956f60..6bd4aef 100644
--- a/bin/chunk-options.R
+++ b/bin/chunk-options.R
@@ -20,7 +20,7 @@ knitr_fig_path <- function(prefix) {
opts_chunk$set(fig.path = new_path)
}
-## We use the rmd- prefix for the figures generated by the lssons so
+## We use the rmd- prefix for the figures generated by the lessons so
## they can be easily identified and deleted by `make clean-rmd`. The
## working directory when the lessons are generated is the root so the
## figures need to be saved in fig/, but when the site is generated,
@@ -29,7 +29,9 @@ knitr_fig_path <- function(prefix) {
opts_chunk$set(tidy = FALSE, results = "markup", comment = NA,
fig.align = "center", fig.path = "fig/rmd-",
- fig.process = fix_fig_path)
+ fig.process = fix_fig_path,
+ fig.width = 8.5, fig.height = 8.5,
+ fig.retina = 2)
# The hooks below add html tags to the code chunks and their output so that they
# are properly formatted when the site is built.
@@ -37,13 +39,13 @@ opts_chunk$set(tidy = FALSE, results = "markup", comment = NA,
hook_in <- function(x, options) {
stringr::str_c("\n\n~~~\n",
paste0(x, collapse="\n"),
- "\n~~~\n{: .r}\n\n")
+ "\n~~~\n{: .language-r}\n\n")
}
hook_out <- function(x, options) {
x <- gsub("\n$", "", x)
stringr::str_c("\n\n~~~\n",
- paste0(x, collapse="\n"),
+ paste0(x, collapse="\n"),
"\n~~~\n{: .output}\n\n")
}
diff --git a/bin/extract_figures.py b/bin/extract_figures.py
deleted file mode 100755
index 63a7752..0000000
--- a/bin/extract_figures.py
+++ /dev/null
@@ -1,98 +0,0 @@
-#!/usr/bin/env python
-
-from __future__ import print_function
-import sys
-import os
-import glob
-from optparse import OptionParser
-
-from util import Reporter, read_markdown, IMAGE_FILE_SUFFIX
-
-def main():
- """Main driver."""
-
- args = parse_args()
- images = []
- for filename in args.filenames:
- images += get_images(args.parser, filename)
- save(sys.stdout, images)
-
-
-def parse_args():
- """Parse command-line arguments."""
-
- parser = OptionParser()
- parser.add_option('-p', '--parser',
- default=None,
- dest='parser',
- help='path to Markdown parser')
-
- args, extras = parser.parse_args()
- require(args.parser is not None,
- 'Path to Markdown parser not provided')
- require(extras,
- 'No filenames specified')
-
- args.filenames = extras
- return args
-
-
-def get_filenames(source_dir):
- """Get all filenames to be searched for images."""
-
- return glob.glob(os.path.join(source_dir, '*.md'))
-
-
-def get_images(parser, filename):
- """Extract all images from file."""
-
- content = read_markdown(parser, filename)
- result = []
- find_image_nodes(content['doc'], result)
- find_image_links(content['doc'], result)
- return result
-
-
-def find_image_nodes(doc, result):
- """Find all nested nodes representing images."""
-
- if (doc['type'] == 'img') or \
- ((doc['type'] == 'html_element') and (doc['value'] == 'img')):
- alt = doc['attr'].get('alt', '')
- result.append({'alt': alt, 'src': doc['attr']['src']})
- else:
- for child in doc.get('children', []):
- find_image_nodes(child, result)
-
-
-def find_image_links(doc, result):
- """Find all links to files in the 'fig' directory."""
-
- if ((doc['type'] == 'a') and ('attr' in doc) and ('href' in doc['attr'])) \
- or \
- ((doc['type'] == 'html_element') and (doc['value'] == 'a') and ('href' in doc['attr'])):
- path = doc['attr']['href']
- if os.path.splitext(path)[1].lower() in IMAGE_FILE_SUFFIX:
- result.append({'alt':'', 'src': doc['attr']['href']})
- else:
- for child in doc.get('children', []):
- find_image_links(child, result)
-
-
-def save(stream, images):
- """Save results as Markdown."""
-
- text = '\n
\n'.join(['
'.format(img['alt'], img['src']) for img in images])
- print(text, file=stream)
-
-
-def require(condition, message):
- """Fail if condition not met."""
-
- if not condition:
- print(message, file=sys.stderr)
- sys.exit(1)
-
-
-if __name__ == '__main__':
- main()
diff --git a/bin/generate_md_episodes.R b/bin/generate_md_episodes.R
index f2a40ba..7f37a7b 100644
--- a/bin/generate_md_episodes.R
+++ b/bin/generate_md_episodes.R
@@ -1,36 +1,59 @@
generate_md_episodes <- function() {
- if (require("knitr") && packageVersion("knitr") < '1.9.19')
- stop("knitr must be version 1.9.20 or higher")
-
- if (!require("stringr"))
- stop("The package stringr is required for generating the lessons.")
-
- if (require("checkpoint") && packageVersion("checkpoint") >= '0.4.0') {
- required_pkgs <-
- checkpoint:::scanForPackages(project = "_episodes_rmd",
- verbose=FALSE, use.knitr = TRUE)$pkgs
- } else {
- stop("The checkpoint package (>= 0.4.0) is required to build the lessons.")
- }
-
- missing_pkgs <- required_pkgs[!(required_pkgs %in% rownames(installed.packages()))]
-
- if (length(missing_pkgs)) {
- message("Installing missing required packages: ",
- paste(missing_pkgs, collapse=", "))
- install.packages(missing_pkgs)
- }
-
- ## find all the Rmd files, and generate the paths for their respective outputs
- src_rmd <- list.files(pattern = "??-*.Rmd$", path = "_episodes_rmd", full.names = TRUE)
- dest_md <- file.path("_episodes", gsub("Rmd$", "md", basename(src_rmd)))
-
- ## knit the Rmd into markdown
- mapply(function(x, y) {
- knitr::knit(x, output = y)
- }, src_rmd, dest_md)
-
+ library("methods")
+
+ if (!require("remotes", quietly = TRUE)) {
+ install.packages("remotes", repos = c(CRAN = "https://cloud.r-project.org/"))
+ }
+
+ if (!require("requirements", quietly = TRUE)) {
+ remotes::install_github("hadley/requirements")
+ }
+
+ required_pkgs <- unique(c(
+ ## Packages for episodes
+ requirements:::req_dir("_episodes_rmd"),
+ ## Pacakges for tools
+ requirements:::req_dir("bin")
+ ))
+
+ missing_pkgs <- setdiff(required_pkgs, rownames(installed.packages()))
+
+ if (length(missing_pkgs)) {
+ message("Installing missing required packages: ",
+ paste(missing_pkgs, collapse=", "))
+ install.packages(missing_pkgs)
+ }
+
+ if (require("knitr") && packageVersion("knitr") < '1.9.19')
+ stop("knitr must be version 1.9.20 or higher")
+
+ ## get the Rmd file to process from the command line, and generate the path for their respective outputs
+ args <- commandArgs(trailingOnly = TRUE)
+ if (!identical(length(args), 2L)) {
+ stop("input and output file must be passed to the script")
+ }
+
+ src_rmd <- args[1]
+ dest_md <- args[2]
+
+ ## knit the Rmd into markdown
+ knitr::knit(src_rmd, output = dest_md)
+
+ # Read the generated md files and add comments advising not to edit them
+ vapply(dest_md, function(y) {
+ con <- file(y)
+ mdfile <- readLines(con)
+ if (mdfile[1] != "---")
+ stop("Input file does not have a valid header")
+ mdfile <- append(mdfile, "# Please do not edit this file directly; it is auto generated.", after = 1)
+ mdfile <- append(mdfile, paste("# Instead, please edit",
+ basename(y), "in _episodes_rmd/"), after = 2)
+ writeLines(mdfile, con)
+ close(con)
+ return(paste("Warning added to YAML header of", y))
+ },
+ character(1))
}
generate_md_episodes()
diff --git a/bin/knit_lessons.sh b/bin/knit_lessons.sh
index 3a2395f..141c136 100755
--- a/bin/knit_lessons.sh
+++ b/bin/knit_lessons.sh
@@ -3,6 +3,6 @@
# Only try running R to translate files if there are some files present.
# The Makefile passes in the names of files.
-if [ $# -ne 0 ] ; then
- Rscript -e "source('bin/generate_md_episodes.R')"
+if [ $# -eq 2 ] ; then
+ Rscript -e "source('bin/generate_md_episodes.R')" "$@"
fi
diff --git a/bin/lesson_check.py b/bin/lesson_check.py
index 8244222..b0b5581 100755
--- a/bin/lesson_check.py
+++ b/bin/lesson_check.py
@@ -1,37 +1,39 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
"""
Check lesson files and their contents.
"""
-from __future__ import print_function
-import sys
+
import os
import glob
-import json
import re
-from optparse import OptionParser
+from argparse import ArgumentParser
-from util import Reporter, read_markdown, load_yaml, check_unwanted_files, require, IMAGE_FILE_SUFFIX
+from util import (Reporter, read_markdown, load_yaml, check_unwanted_files,
+ require)
__version__ = '0.3'
# Where to look for source Markdown files.
SOURCE_DIRS = ['', '_episodes', '_extras']
+# Where to look for source Rmd files.
+SOURCE_RMD_DIRS = ['_episodes_rmd']
+
# Required files: each entry is ('path': YAML_required).
# FIXME: We do not yet validate whether any files have the required
# YAML headers, but should in the future.
# The '%' is replaced with the source directory path for checking.
-# Episodes are handled specially, and extra files in '_extras' are also handled specially.
-# This list must include all the Markdown files listed in the 'bin/initialize' script.
+# Episodes are handled specially, and extra files in '_extras' are also handled
+# specially. This list must include all the Markdown files listed in the
+# 'bin/initialize' script.
REQUIRED_FILES = {
- '%/CONDUCT.md': True,
+ '%/CODE_OF_CONDUCT.md': True,
'%/CONTRIBUTING.md': False,
'%/LICENSE.md': True,
'%/README.md': False,
'%/_extras/discuss.md': True,
- '%/_extras/figures.md': True,
'%/_extras/guide.md': True,
'%/index.md': True,
'%/reference.md': True,
@@ -72,12 +74,14 @@
'error',
'output',
'source',
- 'bash',
- 'make',
- 'matlab',
- 'python',
- 'r',
- 'sql'
+ 'language-bash',
+ 'html',
+ 'language-make',
+ 'language-matlab',
+ 'language-python',
+ 'language-r',
+ 'language-shell',
+ 'language-sql'
}
# What fields are required in teaching episode metadata?
@@ -98,55 +102,64 @@
}
# How long are lines allowed to be?
+# Please keep this in sync with .editorconfig!
MAX_LINE_LEN = 100
+
def main():
"""Main driver."""
args = parse_args()
args.reporter = Reporter()
check_config(args.reporter, args.source_dir)
+ check_source_rmd(args.reporter, args.source_dir, args.parser)
args.references = read_references(args.reporter, args.reference_path)
docs = read_all_markdown(args.source_dir, args.parser)
- check_fileset(args.source_dir, args.reporter, docs.keys())
+ check_fileset(args.source_dir, args.reporter, list(docs.keys()))
check_unwanted_files(args.source_dir, args.reporter)
- for filename in docs.keys():
+ for filename in list(docs.keys()):
checker = create_checker(args, filename, docs[filename])
checker.check()
- check_figures(args.source_dir, args.reporter)
args.reporter.report()
+ if args.reporter.messages and not args.permissive:
+ exit(1)
def parse_args():
"""Parse command-line arguments."""
- parser = OptionParser()
- parser.add_option('-l', '--linelen',
- default=False,
- action="store_true",
- dest='line_lengths',
- help='Check line lengths')
- parser.add_option('-p', '--parser',
- default=None,
- dest='parser',
- help='path to Markdown parser')
- parser.add_option('-r', '--references',
- default=None,
- dest='reference_path',
- help='path to Markdown file of external references')
- parser.add_option('-s', '--source',
- default=os.curdir,
- dest='source_dir',
- help='source directory')
- parser.add_option('-w', '--whitespace',
- default=False,
- action="store_true",
- dest='trailing_whitespace',
- help='Check for trailing whitespace')
-
- args, extras = parser.parse_args()
+ parser = ArgumentParser(description="""Check episode files in a lesson.""")
+ parser.add_argument('-l', '--linelen',
+ default=False,
+ action="store_true",
+ dest='line_lengths',
+ help='Check line lengths')
+ parser.add_argument('-p', '--parser',
+ default=None,
+ dest='parser',
+ help='path to Markdown parser')
+ parser.add_argument('-r', '--references',
+ default=None,
+ dest='reference_path',
+ help='path to Markdown file of external references')
+ parser.add_argument('-s', '--source',
+ default=os.curdir,
+ dest='source_dir',
+ help='source directory')
+ parser.add_argument('-w', '--whitespace',
+ default=False,
+ action="store_true",
+ dest='trailing_whitespace',
+ help='Check for trailing whitespace')
+ parser.add_argument('--permissive',
+ default=False,
+ action="store_true",
+ dest='permissive',
+ help='Do not raise an error even if issues are detected')
+
+ args, extras = parser.parse_known_args()
require(args.parser is not None,
'Path to Markdown parser not provided')
require(not extras,
@@ -160,15 +173,35 @@ def check_config(reporter, source_dir):
config_file = os.path.join(source_dir, '_config.yml')
config = load_yaml(config_file)
- reporter.check_field(config_file, 'configuration', config, 'kind', 'lesson')
- reporter.check_field(config_file, 'configuration', config, 'carpentry', ('swc', 'dc', 'lc'))
+ reporter.check_field(config_file, 'configuration',
+ config, 'kind', 'lesson')
+ reporter.check_field(config_file, 'configuration',
+ config, 'carpentry', ('swc', 'dc', 'lc', 'cp'))
reporter.check_field(config_file, 'configuration', config, 'title')
- reporter.check_field(config_file, 'configuration', config, 'contact')
-
- reporter.check({'values': {'root': '..'}} in config.get('defaults', []),
+ reporter.check_field(config_file, 'configuration', config, 'email')
+
+ for defaults in [
+ {'values': {'root': '.', 'layout': 'page'}},
+ {'values': {'root': '..', 'layout': 'episode'}, 'scope': {'type': 'episodes', 'path': ''}},
+ {'values': {'root': '..', 'layout': 'page'}, 'scope': {'type': 'extras', 'path': ''}}
+ ]:
+ reporter.check(defaults in config.get('defaults', []),
'configuration',
- '"root" not set to ".." in configuration')
-
+ '"root" not set to "." in configuration')
+
+def check_source_rmd(reporter, source_dir, parser):
+ """Check that Rmd episode files include `source: Rmd`"""
+
+ episode_rmd_dir = [os.path.join(source_dir, d) for d in SOURCE_RMD_DIRS]
+ episode_rmd_files = [os.path.join(d, '*.Rmd') for d in episode_rmd_dir]
+ results = {}
+ for pat in episode_rmd_files:
+ for f in glob.glob(pat):
+ data = read_markdown(parser, f)
+ dy = data['metadata']
+ if dy:
+ reporter.check_field(f, 'episode_rmd',
+ dy, 'source', 'Rmd')
def read_references(reporter, ref_path):
"""Read shared file of reference links, returning dictionary of valid references
@@ -235,17 +268,17 @@ def check_fileset(source_dir, reporter, filenames_present):
if m and m.group(1):
seen.append(m.group(1))
else:
- reporter.add(None, 'Episode {0} has badly-formatted filename', filename)
+ reporter.add(
+ None, 'Episode {0} has badly-formatted filename', filename)
# Check for duplicate episode numbers.
reporter.check(len(seen) == len(set(seen)),
- None,
- 'Duplicate episode numbers {0} vs {1}',
- sorted(seen), sorted(set(seen)))
+ None,
+ 'Duplicate episode numbers {0} vs {1}',
+ sorted(seen), sorted(set(seen)))
# Check that numbers are consecutive.
- seen = [int(s) for s in seen]
- seen.sort()
+ seen = sorted([int(s) for s in seen])
clean = True
for i in range(len(seen) - 1):
clean = clean and ((seen[i+1] - seen[i]) == 1)
@@ -255,55 +288,22 @@ def check_fileset(source_dir, reporter, filenames_present):
seen)
-def check_figures(source_dir, reporter):
- """Check that all figures are present and referenced."""
-
- # Get references.
- try:
- all_figures_html = os.path.join(source_dir, '_includes', 'all_figures.html')
- with open(all_figures_html, 'r') as reader:
- text = reader.read()
- figures = P_FIGURE_REFS.findall(text)
- referenced = [os.path.split(f)[1] for f in figures if '/fig/' in f]
- except FileNotFoundError as e:
- reporter.add(all_figures_html,
- 'File not found')
- return
-
- # Get actual image files (ignore non-image files).
- fig_dir_path = os.path.join(source_dir, 'fig')
- actual = [f for f in os.listdir(fig_dir_path) if os.path.splitext(f)[1] in IMAGE_FILE_SUFFIX]
-
- # Report differences.
- unexpected = set(actual) - set(referenced)
- reporter.check(not unexpected,
- None,
- 'Unexpected image files: {0}',
- ', '.join(sorted(unexpected)))
- missing = set(referenced) - set(actual)
- reporter.check(not missing,
- None,
- 'Missing image files: {0}',
- ', '.join(sorted(missing)))
-
-
def create_checker(args, filename, info):
"""Create appropriate checker for file."""
for (pat, cls) in CHECKERS:
if pat.search(filename):
return cls(args, filename, **info)
+ return NotImplemented
-
-class CheckBase(object):
+class CheckBase:
"""Base class for checking Markdown files."""
def __init__(self, args, filename, metadata, metadata_len, text, lines, doc):
"""Cache arguments for checking."""
- super(CheckBase, self).__init__()
self.args = args
- self.reporter = self.args.reporter # for convenience
+ self.reporter = self.args.reporter # for convenience
self.filename = filename
self.metadata = metadata
self.metadata_len = metadata_len
@@ -313,7 +313,6 @@ def __init__(self, args, filename, metadata, metadata_len, text, lines, doc):
self.layout = None
-
def check(self):
"""Run tests."""
@@ -324,7 +323,6 @@ def check(self):
self.check_codeblock_classes()
self.check_defined_link_references()
-
def check_metadata(self):
"""Check the YAML metadata."""
@@ -333,53 +331,51 @@ def check_metadata(self):
'Missing metadata entirely')
if self.metadata and (self.layout is not None):
- self.reporter.check_field(self.filename, 'metadata', self.metadata, 'layout', self.layout)
-
+ self.reporter.check_field(
+ self.filename, 'metadata', self.metadata, 'layout', self.layout)
def check_line_lengths(self):
"""Check the raw text of the lesson body."""
if self.args.line_lengths:
- over = [i for (i, l, n) in self.lines if (n > MAX_LINE_LEN) and (not l.startswith('!'))]
+ over = [i for (i, l, n) in self.lines if (
+ n > MAX_LINE_LEN) and (not l.startswith('!'))]
self.reporter.check(not over,
self.filename,
- 'Line(s) are too long: {0}',
+ 'Line(s) too long: {0}',
', '.join([str(i) for i in over]))
-
def check_trailing_whitespace(self):
"""Check for whitespace at the ends of lines."""
if self.args.trailing_whitespace:
- trailing = [i for (i, l, n) in self.lines if P_TRAILING_WHITESPACE.match(l)]
+ trailing = [
+ i for (i, l, n) in self.lines if P_TRAILING_WHITESPACE.match(l)]
self.reporter.check(not trailing,
self.filename,
'Line(s) end with whitespace: {0}',
', '.join([str(i) for i in trailing]))
-
def check_blockquote_classes(self):
"""Check that all blockquotes have known classes."""
- for node in self.find_all(self.doc, {'type' : 'blockquote'}):
+ for node in self.find_all(self.doc, {'type': 'blockquote'}):
cls = self.get_val(node, 'attr', 'class')
self.reporter.check(cls in KNOWN_BLOCKQUOTES,
(self.filename, self.get_loc(node)),
'Unknown or missing blockquote type {0}',
cls)
-
def check_codeblock_classes(self):
"""Check that all code blocks have known classes."""
- for node in self.find_all(self.doc, {'type' : 'codeblock'}):
+ for node in self.find_all(self.doc, {'type': 'codeblock'}):
cls = self.get_val(node, 'attr', 'class')
self.reporter.check(cls in KNOWN_CODEBLOCKS,
(self.filename, self.get_loc(node)),
'Unknown or missing code block type {0}',
cls)
-
def check_defined_link_references(self):
"""Check that defined links resolve in the file.
@@ -387,7 +383,7 @@ def check_defined_link_references(self):
"""
result = set()
- for node in self.find_all(self.doc, {'type' : 'text'}):
+ for node in self.find_all(self.doc, {'type': 'text'}):
for match in P_INTERNAL_LINK_REF.findall(node['value']):
text = match[0]
link = match[1]
@@ -398,11 +394,10 @@ def check_defined_link_references(self):
'Internally-defined links may be missing definitions: {0}',
', '.join(sorted(result)))
-
def find_all(self, node, pattern, accum=None):
"""Find all matches for a pattern."""
- assert type(pattern) == dict, 'Patterns must be dictionaries'
+ assert isinstance(pattern, dict), 'Patterns must be dictionaries'
if accum is None:
accum = []
if self.match(node, pattern):
@@ -411,7 +406,6 @@ def find_all(self, node, pattern, accum=None):
self.find_all(child, pattern, accum)
return accum
-
def match(self, node, pattern):
"""Does this node match the given pattern?"""
@@ -419,16 +413,16 @@ def match(self, node, pattern):
if key not in node:
return False
val = pattern[key]
- if type(val) == str:
+ if isinstance(val, str):
if node[key] != val:
return False
- elif type(val) == dict:
+ elif isinstance(val, dict):
if not self.match(node[key], val):
return False
return True
-
- def get_val(self, node, *chain):
+ @staticmethod
+ def get_val(node, *chain):
"""Get value one or more levels down."""
curr = node
@@ -438,7 +432,6 @@ def get_val(self, node, *chain):
break
return curr
-
def get_loc(self, node):
"""Convenience method to get node's line number."""
@@ -451,10 +444,6 @@ def get_loc(self, node):
class CheckNonJekyll(CheckBase):
"""Check a file that isn't translated by Jekyll."""
- def __init__(self, args, filename, metadata, metadata_len, text, lines, doc):
- super(CheckNonJekyll, self).__init__(args, filename, metadata, metadata_len, text, lines, doc)
-
-
def check_metadata(self):
self.reporter.check(self.metadata is None,
self.filename,
@@ -465,11 +454,11 @@ class CheckIndex(CheckBase):
"""Check the main index page."""
def __init__(self, args, filename, metadata, metadata_len, text, lines, doc):
- super(CheckIndex, self).__init__(args, filename, metadata, metadata_len, text, lines, doc)
+ super().__init__(args, filename, metadata, metadata_len, text, lines, doc)
self.layout = 'lesson'
def check_metadata(self):
- super(CheckIndex, self).check_metadata()
+ super().check_metadata()
self.reporter.check(self.metadata.get('root', '') == '.',
self.filename,
'Root not set to "."')
@@ -478,19 +467,14 @@ def check_metadata(self):
class CheckEpisode(CheckBase):
"""Check an episode page."""
- def __init__(self, args, filename, metadata, metadata_len, text, lines, doc):
- super(CheckEpisode, self).__init__(args, filename, metadata, metadata_len, text, lines, doc)
-
-
def check(self):
"""Run extra tests."""
- super(CheckEpisode, self).check()
+ super().check()
self.check_reference_inclusion()
-
def check_metadata(self):
- super(CheckEpisode, self).check_metadata()
+ super().check_metadata()
if self.metadata:
if 'layout' in self.metadata:
if self.metadata['layout'] == 'break':
@@ -502,19 +486,18 @@ def check_metadata(self):
else:
self.check_metadata_fields(TEACHING_METADATA_FIELDS)
-
def check_metadata_fields(self, expected):
+ """Check metadata fields."""
for (name, type_) in expected:
if name not in self.metadata:
self.reporter.add(self.filename,
'Missing metadata field {0}',
name)
- elif type(self.metadata[name]) != type_:
+ elif not isinstance(self.metadata[name], type_):
self.reporter.add(self.filename,
'"{0}" has wrong type in metadata ({1} instead of {2})',
name, type(self.metadata[name]), type_)
-
def check_reference_inclusion(self):
"""Check that links file has been included."""
@@ -539,7 +522,7 @@ class CheckReference(CheckBase):
"""Check the reference page."""
def __init__(self, args, filename, metadata, metadata_len, text, lines, doc):
- super(CheckReference, self).__init__(args, filename, metadata, metadata_len, text, lines, doc)
+ super().__init__(args, filename, metadata, metadata_len, text, lines, doc)
self.layout = 'reference'
@@ -547,8 +530,7 @@ class CheckGeneric(CheckBase):
"""Check a generic page."""
def __init__(self, args, filename, metadata, metadata_len, text, lines, doc):
- super(CheckGeneric, self).__init__(args, filename, metadata, metadata_len, text, lines, doc)
- self.layout = 'page'
+ super().__init__(args, filename, metadata, metadata_len, text, lines, doc)
CHECKERS = [
@@ -557,6 +539,7 @@ def __init__(self, args, filename, metadata, metadata_len, text, lines, doc):
(re.compile(r'index\.md'), CheckIndex),
(re.compile(r'reference\.md'), CheckReference),
(re.compile(r'_episodes/.*\.md'), CheckEpisode),
+ (re.compile(r'aio\.md'), CheckNonJekyll),
(re.compile(r'.*\.md'), CheckGeneric)
]
diff --git a/bin/lesson_initialize.py b/bin/lesson_initialize.py
index fc7baf7..a5eb6d0 100755
--- a/bin/lesson_initialize.py
+++ b/bin/lesson_initialize.py
@@ -1,391 +1,28 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
"""Initialize a newly-created repository."""
-from __future__ import print_function
import sys
import os
-
-ROOT_AUTHORS = '''\
-FIXME: list authors' names and email addresses.
-'''
-
-ROOT_CITATION = '''\
-FIXME: describe how to cite this lesson.
-'''
-
-ROOT_CONTRIBUTING_MD = '''\
-# Contributing
-
-[Software Carpentry][swc-site] and [Data Carpentry][dc-site] are open source projects,
-and we welcome contributions of all kinds:
-new lessons,
-fixes to existing material,
-bug reports,
-and reviews of proposed changes are all welcome.
-
-## Contributor Agreement
-
-By contributing,
-you agree that we may redistribute your work under [our license](LICENSE.md).
-In exchange,
-we will address your issues and/or assess your change proposal as promptly as we can,
-and help you become a member of our community.
-Everyone involved in [Software Carpentry][swc-site] and [Data Carpentry][dc-site]
-agrees to abide by our [code of conduct](CONDUCT.md).
-
-## How to Contribute
-
-The easiest way to get started is to file an issue
-to tell us about a spelling mistake,
-some awkward wording,
-or a factual error.
-This is a good way to introduce yourself
-and to meet some of our community members.
-
-1. If you do not have a [GitHub][github] account,
- you can [send us comments by email][contact].
- However,
- we will be able to respond more quickly if you use one of the other methods described below.
-
-2. If you have a [GitHub][github] account,
- or are willing to [create one][github-join],
- but do not know how to use Git,
- you can report problems or suggest improvements by [creating an issue][issues].
- This allows us to assign the item to someone
- and to respond to it in a threaded discussion.
-
-3. If you are comfortable with Git,
- and would like to add or change material,
- you can submit a pull request (PR).
- Instructions for doing this are [included below](#using-github).
-
-## Where to Contribute
-
-1. If you wish to change this lesson,
- please work in ,
- which can be viewed at .
-
-2. If you wish to change the example lesson,
- please work in ,
- which documents the format of our lessons
- and can be viewed at .
-
-3. If you wish to change the template used for workshop websites,
- please work in .
- The home page of that repository explains how to set up workshop websites,
- while the extra pages in
- provide more background on our design choices.
-
-4. If you wish to change CSS style files, tools,
- or HTML boilerplate for lessons or workshops stored in `_includes` or `_layouts`,
- please work in .
-
-## What to Contribute
-
-There are many ways to contribute,
-from writing new exercises and improving existing ones
-to updating or filling in the documentation
-and and submitting [bug reports][issues]
-about things that don't work, aren't clear, or are missing.
-If you are looking for ideas,
-please see [the list of issues for this repository][issues],
-or the issues for [Data Carpentry][dc-issues]
-and [Software Carpentry][swc-issues] projects.
-
-Comments on issues and reviews of pull requests are just as welcome:
-we are smarter together than we are on our own.
-Reviews from novices and newcomers are particularly valuable:
-it's easy for people who have been using these lessons for a while
-to forget how impenetrable some of this material can be,
-so fresh eyes are always welcome.
-
-## What *Not* to Contribute
-
-Our lessons already contain more material than we can cover in a typical workshop,
-so we are usually *not* looking for more concepts or tools to add to them.
-As a rule,
-if you want to introduce a new idea,
-you must (a) estimate how long it will take to teach
-and (b) explain what you would take out to make room for it.
-The first encourages contributors to be honest about requirements;
-the second, to think hard about priorities.
-
-We are also not looking for exercises or other material that only run on one platform.
-Our workshops typically contain a mixture of Windows, Mac OS X, and Linux users;
-in order to be usable,
-our lessons must run equally well on all three.
-
-## Using GitHub
-
-If you choose to contribute via GitHub,
-you may want to look at
-[How to Contribute to an Open Source Project on GitHub][how-contribute].
-In brief:
-
-1. The published copy of the lesson is in the `gh-pages` branch of the repository
- (so that GitHub will regenerate it automatically).
- Please create all branches from that,
- and merge the [master repository][repo]'s `gh-pages` branch into your `gh-pages` branch
- before starting work.
- Please do *not* work directly in your `gh-pages` branch,
- since that will make it difficult for you to work on other contributions.
-
-2. We use [GitHub flow][github-flow] to manage changes:
- 1. Create a new branch in your desktop copy of this repository for each significant change.
- 2. Commit the change in that branch.
- 3. Push that branch to your fork of this repository on GitHub.
- 4. Submit a pull request from that branch to the [master repository][repo].
- 5. If you receive feedback,
- make changes on your desktop and push to your branch on GitHub:
- the pull request will update automatically.
-
-Each lesson has two maintainers who review issues and pull requests
-or encourage others to do so.
-The maintainers are community volunteers,
-and have final say over what gets merged into the lesson.
-
-## Other Resources
-
-General discussion of [Software Carpentry][swc-site] and [Data Carpentry][dc-site]
-happens on the [discussion mailing list][discuss-list],
-which everyone is welcome to join.
-You can also [reach us by email][contact].
-
-[contact]: mailto:admin@software-carpentry.org
-[dc-issues]: https://github.com/issues?q=user%3Adatacarpentry
-[dc-lessons]: http://datacarpentry.org/lessons/
-[dc-site]: http://datacarpentry.org/
-[discuss-list]: http://lists.software-carpentry.org/listinfo/discuss
-[github]: http://github.com
-[github-flow]: https://guides.github.com/introduction/flow/
-[github-join]: https://github.com/join
-[how-contribute]: https://egghead.io/series/how-to-contribute-to-an-open-source-project-on-github
-[issues]: https://github.com/swcarpentry/FIXME/issues/
-[repo]: https://github.com/swcarpentry/FIXME/
-[swc-issues]: https://github.com/issues?q=user%3Aswcarpentry
-[swc-lessons]: http://software-carpentry.org/lessons/
-[swc-site]: http://software-carpentry.org/
-'''
-
-ROOT_CONFIG_YML = '''\
-#------------------------------------------------------------
-# Values for this lesson.
-#------------------------------------------------------------
-
-# Which carpentry is this ("swc", "dc", or "lc")?
-carpentry: "swc"
-
-# Overall title for pages.
-title: "Lesson Title"
-
-# Contact. This *must* include the protocol: if it's an email
-# address, it must look like "mailto:lessons@software-carpentry.org",
-# or if it's a URL, "https://gitter.im/username/ProjectName".
-contact: "mailto:lessons@software-carpentry.org"
-
-#------------------------------------------------------------
-# Generic settings (should not need to change).
-#------------------------------------------------------------
-
-# What kind of thing is this ("workshop" or "lesson")?
-kind: "lesson"
-
-# Magic to make URLs resolve both locally and on GitHub.
-# See https://help.github.com/articles/repository-metadata-on-github-pages/.
-repository: /
-
-# Sites.
-amy_site: "https://amy.software-carpentry.org/workshops"
-dc_site: "http://datacarpentry.org"
-swc_github: "https://github.com/swcarpentry"
-swc_site: "https://software-carpentry.org"
-swc_pages: "https://swcarpentry.github.io"
-lc_site: "http://librarycarpentry.github.io/"
-template_repo: "https://github.com/swcarpentry/styles"
-example_repo: "https://github.com/swcarpentry/lesson-example"
-example_site: "https://swcarpentry.github.com/lesson-example"
-workshop_repo: "https://github.com/swcarpentry/workshop-template"
-workshop_site: "https://swcarpentry.github.io/workshop-template"
-training_site: "https://swcarpentry.github.io/instructor-training"
-
-# Surveys.
-pre_survey: "https://www.surveymonkey.com/r/swc_pre_workshop_v1?workshop_id="
-post_survey: "https://www.surveymonkey.com/r/swc_post_workshop_v1?workshop_id="
-training_post_survey: "https://www.surveymonkey.com/r/post-instructor-training"
-
-# Start time in minutes (0 to be clock-independent, 540 to show a start at 09:00 am).
-start_time: 0
-
-# Specify that things in the episodes collection should be output.
-collections:
- episodes:
- output: true
- permalink: /:path/
- extras:
- output: true
-
-# Set the default layout for things in the episodes collection.
-defaults:
- - values:
- root: ..
- - scope:
- path: ""
- type: episodes
- values:
- layout: episode
-
-# Files and directories that are not to be copied.
-exclude:
- - Makefile
- - bin
-
-# Turn off built-in syntax highlighting.
-highlighter: false
-'''
-
-ROOT_INDEX_MD = '''\
----
-layout: lesson
-root: .
----
-FIXME: home page introduction
-
-> ## Prerequisites
->
-> FIXME
-{: .prereq}
-'''
-
-ROOT_REFERENCE_MD = '''\
----
-layout: reference
-permalink: /reference/
----
-
-## Glossary
-
-FIXME
-'''
-
-ROOT_SETUP_MD = '''\
----
-layout: page
-title: Setup
-permalink: /setup/
----
-FIXME
-'''
-
-ROOT_AIO_MD = '''\
----
-layout: page
-permalink: /aio/
----
-
-{% comment %}
-Create anchor for each one of the episodes.
-{% endcomment %}
-{% for episode in site.episodes %}
-
-{% endfor %}
-'''
-
-EPISODES_INTRODUCTION_MD = '''\
----
-title: "Introduction"
-teaching: 0
-exercises: 0
-questions:
-- "Key question"
-objectives:
-- "First objective."
-keypoints:
-- "First key point."
----
-'''
-
-EXTRAS_ABOUT_MD = '''\
----
-layout: page
-title: About
-permalink: /about/
----
-{% include carpentries.html %}
-'''
-
-EXTRAS_DISCUSS_MD = '''\
----
-layout: page
-title: Discussion
-permalink: /discuss/
----
-FIXME
-'''
-
-EXTRAS_FIGURES_MD = '''\
----
-layout: page
-title: Figures
-permalink: /figures/
----
-{% include all_figures.html %}
-'''
-
-EXTRAS_GUIDE_MD = '''\
----
-layout: page
-title: "Instructor Notes"
-permalink: /guide/
----
-FIXME
-'''
-
-INCLUDES_ALL_FIGURES_HTML = '''\
-
-'''
+import shutil
BOILERPLATE = (
- ('AUTHORS', ROOT_AUTHORS),
- ('CITATION', ROOT_CITATION),
- ('CONTRIBUTING.md', ROOT_CONTRIBUTING_MD),
- ('_config.yml', ROOT_CONFIG_YML),
- ('index.md', ROOT_INDEX_MD),
- ('reference.md', ROOT_REFERENCE_MD),
- ('setup.md', ROOT_SETUP_MD),
- ('aio.md', ROOT_AIO_MD),
- ('_episodes/01-introduction.md', EPISODES_INTRODUCTION_MD),
- ('_extras/about.md', EXTRAS_ABOUT_MD),
- ('_extras/discuss.md', EXTRAS_DISCUSS_MD),
- ('_extras/figures.md', EXTRAS_FIGURES_MD),
- ('_extras/guide.md', EXTRAS_GUIDE_MD),
- ('_includes/all_figures.html', INCLUDES_ALL_FIGURES_HTML)
+ '.travis.yml',
+ 'AUTHORS',
+ 'CITATION',
+ 'CONTRIBUTING.md',
+ 'README.md',
+ '_config.yml',
+ '_episodes/01-introduction.md',
+ '_extras/about.md',
+ '_extras/discuss.md',
+ '_extras/figures.md',
+ '_extras/guide.md',
+ 'aio.md',
+ 'index.md',
+ 'reference.md',
+ 'setup.md',
)
@@ -394,7 +31,7 @@ def main():
# Check.
errors = False
- for (path, _) in BOILERPLATE:
+ for path in BOILERPLATE:
if os.path.exists(path):
print('Warning: {0} already exists.'.format(path), file=sys.stderr)
errors = True
@@ -403,9 +40,11 @@ def main():
sys.exit(1)
# Create.
- for (path, content) in BOILERPLATE:
- with open(path, 'w') as writer:
- writer.write(content)
+ for path in BOILERPLATE:
+ shutil.copyfile(
+ "bin/boilerplate/{}".format(path),
+ path
+ )
if __name__ == '__main__':
diff --git a/bin/repo_check.py b/bin/repo_check.py
index fd04ce9..af4b782 100755
--- a/bin/repo_check.py
+++ b/bin/repo_check.py
@@ -1,17 +1,17 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
"""
Check repository settings.
"""
-from __future__ import print_function
+
import sys
import os
from subprocess import Popen, PIPE
import re
-from optparse import OptionParser
+from argparse import ArgumentParser
-from util import Reporter, load_yaml, require
+from util import Reporter, require
# Import this way to produce a more useful error message.
try:
@@ -35,15 +35,26 @@
# Expected labels and colors.
EXPECTED = {
- 'bug' : 'bd2c00',
- 'discussion' : 'fc8dc1',
- 'enhancement' : '9cd6dc',
- 'help-wanted' : 'f4fd9c',
- 'instructor-training' : '6e5494',
- 'newcomer-friendly' : 'eec275',
- 'question' : '808040',
- 'template-and-tools' : '2b3990',
- 'work-in-progress' : '7ae78e'
+ 'help wanted': 'dcecc7',
+ 'status:in progress': '9bcc65',
+ 'status:changes requested': '679f38',
+ 'status:wait': 'fff2df',
+ 'status:refer to cac': 'ffdfb2',
+ 'status:need more info': 'ee6c00',
+ 'status:blocked': 'e55100',
+ 'status:out of scope': 'eeeeee',
+ 'status:duplicate': 'bdbdbd',
+ 'type:typo text': 'f8bad0',
+ 'type:bug': 'eb3f79',
+ 'type:formatting': 'ac1357',
+ 'type:template and tools': '7985cb',
+ 'type:instructor guide': '00887a',
+ 'type:discussion': 'b2e5fc',
+ 'type:enhancement': '7fdeea',
+ 'type:clarification': '00acc0',
+ 'type:teaching example': 'ced8dc',
+ 'good first issue': 'ffeb3a',
+ 'high priority': 'd22e2e'
}
@@ -54,7 +65,7 @@ def main():
args = parse_args()
reporter = Reporter()
- repo_url = get_repo_url(args.source_dir, args.repo_url)
+ repo_url = get_repo_url(args.repo_url)
check_labels(reporter, repo_url)
reporter.report()
@@ -64,24 +75,24 @@ def parse_args():
Parse command-line arguments.
"""
- parser = OptionParser()
- parser.add_option('-r', '--repo',
- default=None,
- dest='repo_url',
- help='repository URL')
- parser.add_option('-s', '--source',
- default=os.curdir,
- dest='source_dir',
- help='source directory')
-
- args, extras = parser.parse_args()
+ parser = ArgumentParser(description="""Check repository settings.""")
+ parser.add_argument('-r', '--repo',
+ default=None,
+ dest='repo_url',
+ help='repository URL')
+ parser.add_argument('-s', '--source',
+ default=os.curdir,
+ dest='source_dir',
+ help='source directory')
+
+ args, extras = parser.parse_known_args()
require(not extras,
'Unexpected trailing command-line arguments "{0}"'.format(extras))
return args
-def get_repo_url(source_dir, repo_url):
+def get_repo_url(repo_url):
"""
Figure out which repository to query.
"""
@@ -92,7 +103,8 @@ def get_repo_url(source_dir, repo_url):
# Guess.
cmd = 'git remote -v'
- p = Popen(cmd, shell=True, stdin=PIPE, stdout=PIPE, close_fds=True, universal_newlines=True)
+ p = Popen(cmd, shell=True, stdin=PIPE, stdout=PIPE,
+ close_fds=True, universal_newlines=True)
stdout_data, stderr_data = p.communicate()
stdout_data = stdout_data.split('\n')
matches = [P_GIT_REMOTE.match(line) for line in stdout_data]
@@ -101,10 +113,12 @@ def get_repo_url(source_dir, repo_url):
'Unexpected output from git remote command: "{0}"'.format(matches))
username = matches[0].group(1)
- require(username, 'empty username in git remote output {0}'.format(matches[0]))
+ require(
+ username, 'empty username in git remote output {0}'.format(matches[0]))
project_name = matches[0].group(2)
- require(username, 'empty project name in git remote output {0}'.format(matches[0]))
+ require(
+ username, 'empty project name in git remote output {0}'.format(matches[0]))
url = F_REPO_URL.format(username, project_name)
return url
@@ -131,7 +145,7 @@ def check_labels(reporter, repo_url):
overlap = set(EXPECTED.keys()).intersection(set(actual.keys()))
for name in sorted(overlap):
- reporter.check(EXPECTED[name] == actual[name],
+ reporter.check(EXPECTED[name].lower() == actual[name].lower(),
None,
'Color mis-match for label {0} in {1}: expected {2}, found {3}',
name, repo_url, EXPECTED[name], actual[name])
@@ -143,13 +157,15 @@ def get_labels(repo_url):
"""
m = P_REPO_URL.match(repo_url)
- require(m, 'repository URL {0} does not match expected pattern'.format(repo_url))
+ require(
+ m, 'repository URL {0} does not match expected pattern'.format(repo_url))
username = m.group(1)
require(username, 'empty username in repository URL {0}'.format(repo_url))
project_name = m.group(2)
- require(username, 'empty project name in repository URL {0}'.format(repo_url))
+ require(
+ username, 'empty project name in repository URL {0}'.format(repo_url))
url = F_API_URL.format(username, project_name)
r = requests.get(url)
diff --git a/bin/test_lesson_check.py b/bin/test_lesson_check.py
index 743d0cf..960059e 100755
--- a/bin/test_lesson_check.py
+++ b/bin/test_lesson_check.py
@@ -1,11 +1,14 @@
+#!/usr/bin/env python3
+
import unittest
import lesson_check
import util
+
class TestFileList(unittest.TestCase):
def setUp(self):
- self.reporter = util.Reporter() ## TODO: refactor reporter class.
+ self.reporter = util.Reporter() # TODO: refactor reporter class.
def test_file_list_has_expected_entries(self):
# For first pass, simply assume that all required files are present
@@ -15,5 +18,6 @@ def test_file_list_has_expected_entries(self):
lesson_check.check_fileset('', self.reporter, all_filenames)
self.assertEqual(len(self.reporter.messages), 0)
+
if __name__ == "__main__":
unittest.main()
diff --git a/bin/util.py b/bin/util.py
index 0cc8de6..f9dc12f 100644
--- a/bin/util.py
+++ b/bin/util.py
@@ -1,4 +1,3 @@
-from __future__ import print_function
import sys
import os
import json
@@ -29,16 +28,14 @@
# (Can't use 'None' because that might be a legitimate value.)
REPORTER_NOT_SET = []
-class Reporter(object):
+
+class Reporter:
"""Collect and report errors."""
def __init__(self):
"""Constructor."""
-
- super(Reporter, self).__init__()
self.messages = []
-
def check_field(self, filename, name, values, key, expected=REPORTER_NOT_SET):
"""Check that a dictionary has an expected value."""
@@ -48,10 +45,11 @@ def check_field(self, filename, name, values, key, expected=REPORTER_NOT_SET):
pass
elif type(expected) in (tuple, set, list):
if values[key] not in expected:
- self.add(filename, '{0} {1} value {2} is not in {3}', name, key, values[key], expected)
+ self.add(
+ filename, '{0} {1} value {2} is not in {3}', name, key, values[key], expected)
elif values[key] != expected:
- self.add(filename, '{0} {1} is {2} not {3}', name, key, values[key], expected)
-
+ self.add(filename, '{0} {1} is {2} not {3}',
+ name, key, values[key], expected)
def check(self, condition, location, fmt, *args):
"""Append error if condition not met."""
@@ -59,12 +57,36 @@ def check(self, condition, location, fmt, *args):
if not condition:
self.add(location, fmt, *args)
-
def add(self, location, fmt, *args):
"""Append error unilaterally."""
self.messages.append((location, fmt.format(*args)))
+ @staticmethod
+ def pretty(item):
+ location, message = item
+ if isinstance(location, type(None)):
+ return message
+ elif isinstance(location, str):
+ return location + ': ' + message
+ elif isinstance(location, tuple):
+ return '{0}:{1}: '.format(*location) + message
+
+ print('Unknown item "{0}"'.format(item), file=sys.stderr)
+ return NotImplemented
+
+ @staticmethod
+ def key(item):
+ location, message = item
+ if isinstance(location, type(None)):
+ return ('', -1, message)
+ elif isinstance(location, str):
+ return (location, -1, message)
+ elif isinstance(location, tuple):
+ return (location[0], location[1], message)
+
+ print('Unknown item "{0}"'.format(item), file=sys.stderr)
+ return NotImplemented
def report(self, stream=sys.stdout):
"""Report all messages in order."""
@@ -72,30 +94,8 @@ def report(self, stream=sys.stdout):
if not self.messages:
return
- def pretty(item):
- location, message = item
- if isinstance(location, type(None)):
- return message
- elif isinstance(location, str):
- return location + ': ' + message
- elif isinstance(location, tuple):
- return '{0}:{1}: '.format(*location) + message
- else:
- assert False, 'Unknown item "{0}"'.format(item)
-
- def key(item):
- location, message = item
- if isinstance(location, type(None)):
- return ('', -1, message)
- elif isinstance(location, str):
- return (location, -1, message)
- elif isinstance(location, tuple):
- return (location[0], location[1], message)
- else:
- assert False, 'Unknown item "{0}"'.format(item)
-
- for m in sorted(self.messages, key=key):
- print(pretty(m), file=stream)
+ for m in sorted(self.messages, key=self.key):
+ print(self.pretty(m), file=stream)
def read_markdown(parser, path):
@@ -111,11 +111,13 @@ def read_markdown(parser, path):
# Split into lines.
metadata_len = 0 if metadata_raw is None else metadata_raw.count('\n')
- lines = [(metadata_len+i+1, line, len(line)) for (i, line) in enumerate(body.split('\n'))]
+ lines = [(metadata_len+i+1, line, len(line))
+ for (i, line) in enumerate(body.split('\n'))]
# Parse Markdown.
cmd = 'ruby {0}'.format(parser)
- p = Popen(cmd, shell=True, stdin=PIPE, stdout=PIPE, close_fds=True, universal_newlines=True)
+ p = Popen(cmd, shell=True, stdin=PIPE, stdout=PIPE,
+ close_fds=True, universal_newlines=True)
stdout_data, stderr_data = p.communicate(body)
doc = json.loads(stdout_data)
@@ -136,16 +138,16 @@ def split_metadata(path, text):
metadata_raw = None
metadata_yaml = None
- metadata_len = None
pieces = text.split('---', 2)
if len(pieces) == 3:
metadata_raw = pieces[1]
text = pieces[2]
try:
- metadata_yaml = yaml.load(metadata_raw)
+ metadata_yaml = yaml.load(metadata_raw, Loader=yaml.FullLoader)
except yaml.YAMLError as e:
- print('Unable to parse YAML header in {0}:\n{1}'.format(path, e), file=sys.stderr)
+ print('Unable to parse YAML header in {0}:\n{1}'.format(
+ path, e), file=sys.stderr)
sys.exit(1)
return metadata_raw, metadata_yaml, text
@@ -159,9 +161,10 @@ def load_yaml(filename):
try:
with open(filename, 'r') as reader:
- return yaml.load(reader)
- except (yaml.YAMLError, FileNotFoundError) as e:
- print('Unable to load YAML file {0}:\n{1}'.format(filename, e), file=sys.stderr)
+ return yaml.load(reader, Loader=yaml.FullLoader)
+ except (yaml.YAMLError, IOError) as e:
+ print('Unable to load YAML file {0}:\n{1}'.format(
+ filename, e), file=sys.stderr)
sys.exit(1)
diff --git a/bin/workshop_check.py b/bin/workshop_check.py
index d3051bf..0523d0c 100755
--- a/bin/workshop_check.py
+++ b/bin/workshop_check.py
@@ -1,10 +1,10 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
'''Check that a workshop's index.html metadata is valid. See the
docstrings on the checking functions for a summary of the checks.
'''
-from __future__ import print_function
+
import sys
import os
import re
@@ -18,10 +18,10 @@
URL_PATTERN = r'https?://.+'
# Defaults.
-CARPENTRIES = ("dc", "swc")
+CARPENTRIES = ("dc", "swc", "lc", "cp")
DEFAULT_CONTACT_EMAIL = 'admin@software-carpentry.org'
-USAGE = 'Usage: "check-workshop path/to/root/directory"'
+USAGE = 'Usage: "workshop_check.py path/to/root/directory"'
# Country and language codes. Note that codes mean different things: 'ar'
# is 'Arabic' as a language but 'Argentina' as a country.
@@ -91,7 +91,7 @@ def check_layout(layout):
@look_for_fixme
def check_carpentry(layout):
- '''"carpentry" in YAML header must be "dc" or "swc".'''
+ '''"carpentry" in YAML header must be "dc", "swc", "lc", or "cp".'''
return layout in CARPENTRIES
@@ -117,7 +117,7 @@ def check_humandate(date):
and 4-digit year. Examples include 'Feb 18-20, 2025' and 'Feb 18
and 20, 2025'. It may be in languages other than English, but the
month name should be kept short to aid formatting of the main
- Software Carpentry web site.
+ Carpentries web site.
"""
if ',' not in date:
@@ -174,8 +174,8 @@ def check_latitude_longitude(latlng):
try:
lat, lng = latlng.split(',')
lat = float(lat)
- long = float(lng)
- return (-90.0 <= lat <= 90.0) and (-180.0 <= long <= 180.0)
+ lng = float(lng)
+ return (-90.0 <= lat <= 90.0) and (-180.0 <= lng <= 180.0)
except ValueError:
return False
@@ -203,15 +203,22 @@ def check_helpers(helpers):
@look_for_fixme
-def check_email(email):
+def check_emails(emails):
"""
- 'contact' must be a valid email address consisting of characters,
- an '@', and more characters. It should not be the default contact
- email address 'admin@software-carpentry.org'.
+ 'emails' must be a comma-separated list of valid email addresses.
+ The list may be empty. A valid email address consists of characters,
+ an '@', and more characters. It should not contain the default contact
"""
- return bool(re.match(EMAIL_PATTERN, email)) and \
- (email != DEFAULT_CONTACT_EMAIL)
+ # YAML automatically loads list-like strings as lists.
+ if (isinstance(emails, list) and len(emails) >= 0):
+ for email in emails:
+ if ((not bool(re.match(EMAIL_PATTERN, email))) or (email == DEFAULT_CONTACT_EMAIL)):
+ return False
+ else:
+ return False
+
+ return True
def check_eventbrite(eventbrite):
@@ -227,12 +234,12 @@ def check_eventbrite(eventbrite):
@look_for_fixme
-def check_etherpad(etherpad):
+def check_collaborative_notes(collaborative_notes):
"""
- 'etherpad' must be a valid URL.
+ 'collaborative_notes' must be a valid URL.
"""
- return bool(re.match(URL_PATTERN, etherpad))
+ return bool(re.match(URL_PATTERN, collaborative_notes))
@look_for_fixme
@@ -286,13 +293,14 @@ def check_pass(value):
'helper list isn\'t a valid list of format ' +
'["First helper", "Second helper",..]'),
- 'contact': (True, check_email,
- 'contact email invalid or still set to ' +
- '"{0}".'.format(DEFAULT_CONTACT_EMAIL)),
+ 'email': (True, check_emails,
+ 'contact email list isn\'t a valid list of format ' +
+ '["me@example.org", "you@example.org",..] or contains incorrectly formatted email addresses or ' +
+ '"{0}".'.format(DEFAULT_CONTACT_EMAIL)),
'eventbrite': (False, check_eventbrite, 'Eventbrite key appears invalid'),
- 'etherpad': (False, check_etherpad, 'Etherpad URL appears invalid'),
+ 'collaborative_notes': (False, check_collaborative_notes, 'Collaborative Notes URL appears invalid'),
'venue': (False, check_pass, 'venue name not specified'),
@@ -300,10 +308,10 @@ def check_pass(value):
}
# REQUIRED is all required categories.
-REQUIRED = set([k for k in HANDLERS if HANDLERS[k][0]])
+REQUIRED = {k for k in HANDLERS if HANDLERS[k][0]}
# OPTIONAL is all optional categories.
-OPTIONAL = set([k for k in HANDLERS if not HANDLERS[k][0]])
+OPTIONAL = {k for k in HANDLERS if not HANDLERS[k][0]}
def check_blank_lines(reporter, raw):
@@ -311,7 +319,8 @@ def check_blank_lines(reporter, raw):
Blank lines are not allowed in category headers.
"""
- lines = [(i, x) for (i, x) in enumerate(raw.strip().split('\n')) if not x.strip()]
+ lines = [(i, x) for (i, x) in enumerate(
+ raw.strip().split('\n')) if not x.strip()]
reporter.check(not lines,
None,
'Blank line(s) in header: {0}',
@@ -381,7 +390,7 @@ def check_config(reporter, filename):
kind)
carpentry = config.get('carpentry', None)
- reporter.check(carpentry in ('swc', 'dc'),
+ reporter.check(carpentry in ('swc', 'dc', 'lc', 'cp'),
filename,
'Missing or unknown carpentry: {0}',
carpentry)
diff --git a/favicon-dc.ico b/favicon-dc.ico
deleted file mode 100644
index 4937f2e..0000000
Binary files a/favicon-dc.ico and /dev/null differ
diff --git a/favicon-lc.ico b/favicon-lc.ico
deleted file mode 100644
index f4f3c93..0000000
Binary files a/favicon-lc.ico and /dev/null differ
diff --git a/favicon-swc.ico b/favicon-swc.ico
deleted file mode 100644
index 34f80ad..0000000
Binary files a/favicon-swc.ico and /dev/null differ
diff --git a/index.md b/index.md
index fec928b..62aaded 100644
--- a/index.md
+++ b/index.md
@@ -1,6 +1,5 @@
---
layout: lesson
-root: .
---
Version control is the lab notebook of the digital world:
diff --git a/reference.md b/reference.md
index cecb361..7835462 100644
--- a/reference.md
+++ b/reference.md
@@ -1,6 +1,5 @@
---
layout: reference
-permalink: /reference/
---
## A Better Kind of Backup
diff --git a/setup.md b/setup.md
index 2747cf0..87fce20 100644
--- a/setup.md
+++ b/setup.md
@@ -1,7 +1,6 @@
---
layout: page
title: "Setup"
-permalink: /setup/
---
Please use [TortoiseHg](http://tortoisehg.bitbucket.org/) to install Mercurial on Windows,
{% if page.previous.url %} - previous episode + previous episode {% else %} - lesson home + lesson home {% endif %}
{% if include.episode_navbar_title %}
-
- {{ site.title }}
+{{ site.title }}
{% endif %}
-
+
,
- which can be viewed at .
-
-2. If you wish to change the example lesson,
- please work in ,
- which documents the format of our lessons
- and can be viewed at .
-
-3. If you wish to change the template used for workshop websites,
- please work in .
- The home page of that repository explains how to set up workshop websites,
- while the extra pages in
- provide more background on our design choices.
-
-4. If you wish to change CSS style files, tools,
- or HTML boilerplate for lessons or workshops stored in `_includes` or `_layouts`,
- please work in .
-
-## What to Contribute
-
-There are many ways to contribute,
-from writing new exercises and improving existing ones
-to updating or filling in the documentation
-and and submitting [bug reports][issues]
-about things that don't work, aren't clear, or are missing.
-If you are looking for ideas,
-please see [the list of issues for this repository][issues],
-or the issues for [Data Carpentry][dc-issues]
-and [Software Carpentry][swc-issues] projects.
-
-Comments on issues and reviews of pull requests are just as welcome:
-we are smarter together than we are on our own.
-Reviews from novices and newcomers are particularly valuable:
-it's easy for people who have been using these lessons for a while
-to forget how impenetrable some of this material can be,
-so fresh eyes are always welcome.
-
-## What *Not* to Contribute
-
-Our lessons already contain more material than we can cover in a typical workshop,
-so we are usually *not* looking for more concepts or tools to add to them.
-As a rule,
-if you want to introduce a new idea,
-you must (a) estimate how long it will take to teach
-and (b) explain what you would take out to make room for it.
-The first encourages contributors to be honest about requirements;
-the second, to think hard about priorities.
-
-We are also not looking for exercises or other material that only run on one platform.
-Our workshops typically contain a mixture of Windows, Mac OS X, and Linux users;
-in order to be usable,
-our lessons must run equally well on all three.
-
-## Using GitHub
-
-If you choose to contribute via GitHub,
-you may want to look at
-[How to Contribute to an Open Source Project on GitHub][how-contribute].
-In brief:
-
-1. The published copy of the lesson is in the `gh-pages` branch of the repository
- (so that GitHub will regenerate it automatically).
- Please create all branches from that,
- and merge the [master repository][repo]'s `gh-pages` branch into your `gh-pages` branch
- before starting work.
- Please do *not* work directly in your `gh-pages` branch,
- since that will make it difficult for you to work on other contributions.
-
-2. We use [GitHub flow][github-flow] to manage changes:
- 1. Create a new branch in your desktop copy of this repository for each significant change.
- 2. Commit the change in that branch.
- 3. Push that branch to your fork of this repository on GitHub.
- 4. Submit a pull request from that branch to the [master repository][repo].
- 5. If you receive feedback,
- make changes on your desktop and push to your branch on GitHub:
- the pull request will update automatically.
-
-Each lesson has two maintainers who review issues and pull requests
-or encourage others to do so.
-The maintainers are community volunteers,
-and have final say over what gets merged into the lesson.
-
-## Other Resources
-
-General discussion of [Software Carpentry][swc-site] and [Data Carpentry][dc-site]
-happens on the [discussion mailing list][discuss-list],
-which everyone is welcome to join.
-You can also [reach us by email][contact].
-
-[contact]: mailto:admin@software-carpentry.org
-[dc-issues]: https://github.com/issues?q=user%3Adatacarpentry
-[dc-lessons]: http://datacarpentry.org/lessons/
-[dc-site]: http://datacarpentry.org/
-[discuss-list]: http://lists.software-carpentry.org/listinfo/discuss
-[github]: http://github.com
-[github-flow]: https://guides.github.com/introduction/flow/
-[github-join]: https://github.com/join
-[how-contribute]: https://egghead.io/series/how-to-contribute-to-an-open-source-project-on-github
-[issues]: https://github.com/swcarpentry/FIXME/issues/
-[repo]: https://github.com/swcarpentry/FIXME/
-[swc-issues]: https://github.com/issues?q=user%3Aswcarpentry
-[swc-lessons]: http://software-carpentry.org/lessons/
-[swc-site]: http://software-carpentry.org/
-'''
-
-ROOT_CONFIG_YML = '''\
-#------------------------------------------------------------
-# Values for this lesson.
-#------------------------------------------------------------
-
-# Which carpentry is this ("swc", "dc", or "lc")?
-carpentry: "swc"
-
-# Overall title for pages.
-title: "Lesson Title"
-
-# Contact. This *must* include the protocol: if it's an email
-# address, it must look like "mailto:lessons@software-carpentry.org",
-# or if it's a URL, "https://gitter.im/username/ProjectName".
-contact: "mailto:lessons@software-carpentry.org"
-
-#------------------------------------------------------------
-# Generic settings (should not need to change).
-#------------------------------------------------------------
-
-# What kind of thing is this ("workshop" or "lesson")?
-kind: "lesson"
-
-# Magic to make URLs resolve both locally and on GitHub.
-# See https://help.github.com/articles/repository-metadata-on-github-pages/.
-repository: /
-
-# Sites.
-amy_site: "https://amy.software-carpentry.org/workshops"
-dc_site: "http://datacarpentry.org"
-swc_github: "https://github.com/swcarpentry"
-swc_site: "https://software-carpentry.org"
-swc_pages: "https://swcarpentry.github.io"
-lc_site: "http://librarycarpentry.github.io/"
-template_repo: "https://github.com/swcarpentry/styles"
-example_repo: "https://github.com/swcarpentry/lesson-example"
-example_site: "https://swcarpentry.github.com/lesson-example"
-workshop_repo: "https://github.com/swcarpentry/workshop-template"
-workshop_site: "https://swcarpentry.github.io/workshop-template"
-training_site: "https://swcarpentry.github.io/instructor-training"
-
-# Surveys.
-pre_survey: "https://www.surveymonkey.com/r/swc_pre_workshop_v1?workshop_id="
-post_survey: "https://www.surveymonkey.com/r/swc_post_workshop_v1?workshop_id="
-training_post_survey: "https://www.surveymonkey.com/r/post-instructor-training"
-
-# Start time in minutes (0 to be clock-independent, 540 to show a start at 09:00 am).
-start_time: 0
-
-# Specify that things in the episodes collection should be output.
-collections:
- episodes:
- output: true
- permalink: /:path/
- extras:
- output: true
-
-# Set the default layout for things in the episodes collection.
-defaults:
- - values:
- root: ..
- - scope:
- path: ""
- type: episodes
- values:
- layout: episode
-
-# Files and directories that are not to be copied.
-exclude:
- - Makefile
- - bin
-
-# Turn off built-in syntax highlighting.
-highlighter: false
-'''
-
-ROOT_INDEX_MD = '''\
----
-layout: lesson
-root: .
----
-FIXME: home page introduction
-
-> ## Prerequisites
->
-> FIXME
-{: .prereq}
-'''
-
-ROOT_REFERENCE_MD = '''\
----
-layout: reference
-permalink: /reference/
----
-
-## Glossary
-
-FIXME
-'''
-
-ROOT_SETUP_MD = '''\
----
-layout: page
-title: Setup
-permalink: /setup/
----
-FIXME
-'''
-
-ROOT_AIO_MD = '''\
----
-layout: page
-permalink: /aio/
----
-
-{% comment %}
-Create anchor for each one of the episodes.
-{% endcomment %}
-{% for episode in site.episodes %}
-
-{% endfor %}
-'''
-
-EPISODES_INTRODUCTION_MD = '''\
----
-title: "Introduction"
-teaching: 0
-exercises: 0
-questions:
-- "Key question"
-objectives:
-- "First objective."
-keypoints:
-- "First key point."
----
-'''
-
-EXTRAS_ABOUT_MD = '''\
----
-layout: page
-title: About
-permalink: /about/
----
-{% include carpentries.html %}
-'''
-
-EXTRAS_DISCUSS_MD = '''\
----
-layout: page
-title: Discussion
-permalink: /discuss/
----
-FIXME
-'''
-
-EXTRAS_FIGURES_MD = '''\
----
-layout: page
-title: Figures
-permalink: /figures/
----
-{% include all_figures.html %}
-'''
-
-EXTRAS_GUIDE_MD = '''\
----
-layout: page
-title: "Instructor Notes"
-permalink: /guide/
----
-FIXME
-'''
-
-INCLUDES_ALL_FIGURES_HTML = '''\
-
-'''
+import shutil
BOILERPLATE = (
- ('AUTHORS', ROOT_AUTHORS),
- ('CITATION', ROOT_CITATION),
- ('CONTRIBUTING.md', ROOT_CONTRIBUTING_MD),
- ('_config.yml', ROOT_CONFIG_YML),
- ('index.md', ROOT_INDEX_MD),
- ('reference.md', ROOT_REFERENCE_MD),
- ('setup.md', ROOT_SETUP_MD),
- ('aio.md', ROOT_AIO_MD),
- ('_episodes/01-introduction.md', EPISODES_INTRODUCTION_MD),
- ('_extras/about.md', EXTRAS_ABOUT_MD),
- ('_extras/discuss.md', EXTRAS_DISCUSS_MD),
- ('_extras/figures.md', EXTRAS_FIGURES_MD),
- ('_extras/guide.md', EXTRAS_GUIDE_MD),
- ('_includes/all_figures.html', INCLUDES_ALL_FIGURES_HTML)
+ '.travis.yml',
+ 'AUTHORS',
+ 'CITATION',
+ 'CONTRIBUTING.md',
+ 'README.md',
+ '_config.yml',
+ '_episodes/01-introduction.md',
+ '_extras/about.md',
+ '_extras/discuss.md',
+ '_extras/figures.md',
+ '_extras/guide.md',
+ 'aio.md',
+ 'index.md',
+ 'reference.md',
+ 'setup.md',
)
@@ -394,7 +31,7 @@ def main():
# Check.
errors = False
- for (path, _) in BOILERPLATE:
+ for path in BOILERPLATE:
if os.path.exists(path):
print('Warning: {0} already exists.'.format(path), file=sys.stderr)
errors = True
@@ -403,9 +40,11 @@ def main():
sys.exit(1)
# Create.
- for (path, content) in BOILERPLATE:
- with open(path, 'w') as writer:
- writer.write(content)
+ for path in BOILERPLATE:
+ shutil.copyfile(
+ "bin/boilerplate/{}".format(path),
+ path
+ )
if __name__ == '__main__':
diff --git a/bin/repo_check.py b/bin/repo_check.py
index fd04ce9..af4b782 100755
--- a/bin/repo_check.py
+++ b/bin/repo_check.py
@@ -1,17 +1,17 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
"""
Check repository settings.
"""
-from __future__ import print_function
+
import sys
import os
from subprocess import Popen, PIPE
import re
-from optparse import OptionParser
+from argparse import ArgumentParser
-from util import Reporter, load_yaml, require
+from util import Reporter, require
# Import this way to produce a more useful error message.
try:
@@ -35,15 +35,26 @@
# Expected labels and colors.
EXPECTED = {
- 'bug' : 'bd2c00',
- 'discussion' : 'fc8dc1',
- 'enhancement' : '9cd6dc',
- 'help-wanted' : 'f4fd9c',
- 'instructor-training' : '6e5494',
- 'newcomer-friendly' : 'eec275',
- 'question' : '808040',
- 'template-and-tools' : '2b3990',
- 'work-in-progress' : '7ae78e'
+ 'help wanted': 'dcecc7',
+ 'status:in progress': '9bcc65',
+ 'status:changes requested': '679f38',
+ 'status:wait': 'fff2df',
+ 'status:refer to cac': 'ffdfb2',
+ 'status:need more info': 'ee6c00',
+ 'status:blocked': 'e55100',
+ 'status:out of scope': 'eeeeee',
+ 'status:duplicate': 'bdbdbd',
+ 'type:typo text': 'f8bad0',
+ 'type:bug': 'eb3f79',
+ 'type:formatting': 'ac1357',
+ 'type:template and tools': '7985cb',
+ 'type:instructor guide': '00887a',
+ 'type:discussion': 'b2e5fc',
+ 'type:enhancement': '7fdeea',
+ 'type:clarification': '00acc0',
+ 'type:teaching example': 'ced8dc',
+ 'good first issue': 'ffeb3a',
+ 'high priority': 'd22e2e'
}
@@ -54,7 +65,7 @@ def main():
args = parse_args()
reporter = Reporter()
- repo_url = get_repo_url(args.source_dir, args.repo_url)
+ repo_url = get_repo_url(args.repo_url)
check_labels(reporter, repo_url)
reporter.report()
@@ -64,24 +75,24 @@ def parse_args():
Parse command-line arguments.
"""
- parser = OptionParser()
- parser.add_option('-r', '--repo',
- default=None,
- dest='repo_url',
- help='repository URL')
- parser.add_option('-s', '--source',
- default=os.curdir,
- dest='source_dir',
- help='source directory')
-
- args, extras = parser.parse_args()
+ parser = ArgumentParser(description="""Check repository settings.""")
+ parser.add_argument('-r', '--repo',
+ default=None,
+ dest='repo_url',
+ help='repository URL')
+ parser.add_argument('-s', '--source',
+ default=os.curdir,
+ dest='source_dir',
+ help='source directory')
+
+ args, extras = parser.parse_known_args()
require(not extras,
'Unexpected trailing command-line arguments "{0}"'.format(extras))
return args
-def get_repo_url(source_dir, repo_url):
+def get_repo_url(repo_url):
"""
Figure out which repository to query.
"""
@@ -92,7 +103,8 @@ def get_repo_url(source_dir, repo_url):
# Guess.
cmd = 'git remote -v'
- p = Popen(cmd, shell=True, stdin=PIPE, stdout=PIPE, close_fds=True, universal_newlines=True)
+ p = Popen(cmd, shell=True, stdin=PIPE, stdout=PIPE,
+ close_fds=True, universal_newlines=True)
stdout_data, stderr_data = p.communicate()
stdout_data = stdout_data.split('\n')
matches = [P_GIT_REMOTE.match(line) for line in stdout_data]
@@ -101,10 +113,12 @@ def get_repo_url(source_dir, repo_url):
'Unexpected output from git remote command: "{0}"'.format(matches))
username = matches[0].group(1)
- require(username, 'empty username in git remote output {0}'.format(matches[0]))
+ require(
+ username, 'empty username in git remote output {0}'.format(matches[0]))
project_name = matches[0].group(2)
- require(username, 'empty project name in git remote output {0}'.format(matches[0]))
+ require(
+ username, 'empty project name in git remote output {0}'.format(matches[0]))
url = F_REPO_URL.format(username, project_name)
return url
@@ -131,7 +145,7 @@ def check_labels(reporter, repo_url):
overlap = set(EXPECTED.keys()).intersection(set(actual.keys()))
for name in sorted(overlap):
- reporter.check(EXPECTED[name] == actual[name],
+ reporter.check(EXPECTED[name].lower() == actual[name].lower(),
None,
'Color mis-match for label {0} in {1}: expected {2}, found {3}',
name, repo_url, EXPECTED[name], actual[name])
@@ -143,13 +157,15 @@ def get_labels(repo_url):
"""
m = P_REPO_URL.match(repo_url)
- require(m, 'repository URL {0} does not match expected pattern'.format(repo_url))
+ require(
+ m, 'repository URL {0} does not match expected pattern'.format(repo_url))
username = m.group(1)
require(username, 'empty username in repository URL {0}'.format(repo_url))
project_name = m.group(2)
- require(username, 'empty project name in repository URL {0}'.format(repo_url))
+ require(
+ username, 'empty project name in repository URL {0}'.format(repo_url))
url = F_API_URL.format(username, project_name)
r = requests.get(url)
diff --git a/bin/test_lesson_check.py b/bin/test_lesson_check.py
index 743d0cf..960059e 100755
--- a/bin/test_lesson_check.py
+++ b/bin/test_lesson_check.py
@@ -1,11 +1,14 @@
+#!/usr/bin/env python3
+
import unittest
import lesson_check
import util
+
class TestFileList(unittest.TestCase):
def setUp(self):
- self.reporter = util.Reporter() ## TODO: refactor reporter class.
+ self.reporter = util.Reporter() # TODO: refactor reporter class.
def test_file_list_has_expected_entries(self):
# For first pass, simply assume that all required files are present
@@ -15,5 +18,6 @@ def test_file_list_has_expected_entries(self):
lesson_check.check_fileset('', self.reporter, all_filenames)
self.assertEqual(len(self.reporter.messages), 0)
+
if __name__ == "__main__":
unittest.main()
diff --git a/bin/util.py b/bin/util.py
index 0cc8de6..f9dc12f 100644
--- a/bin/util.py
+++ b/bin/util.py
@@ -1,4 +1,3 @@
-from __future__ import print_function
import sys
import os
import json
@@ -29,16 +28,14 @@
# (Can't use 'None' because that might be a legitimate value.)
REPORTER_NOT_SET = []
-class Reporter(object):
+
+class Reporter:
"""Collect and report errors."""
def __init__(self):
"""Constructor."""
-
- super(Reporter, self).__init__()
self.messages = []
-
def check_field(self, filename, name, values, key, expected=REPORTER_NOT_SET):
"""Check that a dictionary has an expected value."""
@@ -48,10 +45,11 @@ def check_field(self, filename, name, values, key, expected=REPORTER_NOT_SET):
pass
elif type(expected) in (tuple, set, list):
if values[key] not in expected:
- self.add(filename, '{0} {1} value {2} is not in {3}', name, key, values[key], expected)
+ self.add(
+ filename, '{0} {1} value {2} is not in {3}', name, key, values[key], expected)
elif values[key] != expected:
- self.add(filename, '{0} {1} is {2} not {3}', name, key, values[key], expected)
-
+ self.add(filename, '{0} {1} is {2} not {3}',
+ name, key, values[key], expected)
def check(self, condition, location, fmt, *args):
"""Append error if condition not met."""
@@ -59,12 +57,36 @@ def check(self, condition, location, fmt, *args):
if not condition:
self.add(location, fmt, *args)
-
def add(self, location, fmt, *args):
"""Append error unilaterally."""
self.messages.append((location, fmt.format(*args)))
+ @staticmethod
+ def pretty(item):
+ location, message = item
+ if isinstance(location, type(None)):
+ return message
+ elif isinstance(location, str):
+ return location + ': ' + message
+ elif isinstance(location, tuple):
+ return '{0}:{1}: '.format(*location) + message
+
+ print('Unknown item "{0}"'.format(item), file=sys.stderr)
+ return NotImplemented
+
+ @staticmethod
+ def key(item):
+ location, message = item
+ if isinstance(location, type(None)):
+ return ('', -1, message)
+ elif isinstance(location, str):
+ return (location, -1, message)
+ elif isinstance(location, tuple):
+ return (location[0], location[1], message)
+
+ print('Unknown item "{0}"'.format(item), file=sys.stderr)
+ return NotImplemented
def report(self, stream=sys.stdout):
"""Report all messages in order."""
@@ -72,30 +94,8 @@ def report(self, stream=sys.stdout):
if not self.messages:
return
- def pretty(item):
- location, message = item
- if isinstance(location, type(None)):
- return message
- elif isinstance(location, str):
- return location + ': ' + message
- elif isinstance(location, tuple):
- return '{0}:{1}: '.format(*location) + message
- else:
- assert False, 'Unknown item "{0}"'.format(item)
-
- def key(item):
- location, message = item
- if isinstance(location, type(None)):
- return ('', -1, message)
- elif isinstance(location, str):
- return (location, -1, message)
- elif isinstance(location, tuple):
- return (location[0], location[1], message)
- else:
- assert False, 'Unknown item "{0}"'.format(item)
-
- for m in sorted(self.messages, key=key):
- print(pretty(m), file=stream)
+ for m in sorted(self.messages, key=self.key):
+ print(self.pretty(m), file=stream)
def read_markdown(parser, path):
@@ -111,11 +111,13 @@ def read_markdown(parser, path):
# Split into lines.
metadata_len = 0 if metadata_raw is None else metadata_raw.count('\n')
- lines = [(metadata_len+i+1, line, len(line)) for (i, line) in enumerate(body.split('\n'))]
+ lines = [(metadata_len+i+1, line, len(line))
+ for (i, line) in enumerate(body.split('\n'))]
# Parse Markdown.
cmd = 'ruby {0}'.format(parser)
- p = Popen(cmd, shell=True, stdin=PIPE, stdout=PIPE, close_fds=True, universal_newlines=True)
+ p = Popen(cmd, shell=True, stdin=PIPE, stdout=PIPE,
+ close_fds=True, universal_newlines=True)
stdout_data, stderr_data = p.communicate(body)
doc = json.loads(stdout_data)
@@ -136,16 +138,16 @@ def split_metadata(path, text):
metadata_raw = None
metadata_yaml = None
- metadata_len = None
pieces = text.split('---', 2)
if len(pieces) == 3:
metadata_raw = pieces[1]
text = pieces[2]
try:
- metadata_yaml = yaml.load(metadata_raw)
+ metadata_yaml = yaml.load(metadata_raw, Loader=yaml.FullLoader)
except yaml.YAMLError as e:
- print('Unable to parse YAML header in {0}:\n{1}'.format(path, e), file=sys.stderr)
+ print('Unable to parse YAML header in {0}:\n{1}'.format(
+ path, e), file=sys.stderr)
sys.exit(1)
return metadata_raw, metadata_yaml, text
@@ -159,9 +161,10 @@ def load_yaml(filename):
try:
with open(filename, 'r') as reader:
- return yaml.load(reader)
- except (yaml.YAMLError, FileNotFoundError) as e:
- print('Unable to load YAML file {0}:\n{1}'.format(filename, e), file=sys.stderr)
+ return yaml.load(reader, Loader=yaml.FullLoader)
+ except (yaml.YAMLError, IOError) as e:
+ print('Unable to load YAML file {0}:\n{1}'.format(
+ filename, e), file=sys.stderr)
sys.exit(1)
diff --git a/bin/workshop_check.py b/bin/workshop_check.py
index d3051bf..0523d0c 100755
--- a/bin/workshop_check.py
+++ b/bin/workshop_check.py
@@ -1,10 +1,10 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
'''Check that a workshop's index.html metadata is valid. See the
docstrings on the checking functions for a summary of the checks.
'''
-from __future__ import print_function
+
import sys
import os
import re
@@ -18,10 +18,10 @@
URL_PATTERN = r'https?://.+'
# Defaults.
-CARPENTRIES = ("dc", "swc")
+CARPENTRIES = ("dc", "swc", "lc", "cp")
DEFAULT_CONTACT_EMAIL = 'admin@software-carpentry.org'
-USAGE = 'Usage: "check-workshop path/to/root/directory"'
+USAGE = 'Usage: "workshop_check.py path/to/root/directory"'
# Country and language codes. Note that codes mean different things: 'ar'
# is 'Arabic' as a language but 'Argentina' as a country.
@@ -91,7 +91,7 @@ def check_layout(layout):
@look_for_fixme
def check_carpentry(layout):
- '''"carpentry" in YAML header must be "dc" or "swc".'''
+ '''"carpentry" in YAML header must be "dc", "swc", "lc", or "cp".'''
return layout in CARPENTRIES
@@ -117,7 +117,7 @@ def check_humandate(date):
and 4-digit year. Examples include 'Feb 18-20, 2025' and 'Feb 18
and 20, 2025'. It may be in languages other than English, but the
month name should be kept short to aid formatting of the main
- Software Carpentry web site.
+ Carpentries web site.
"""
if ',' not in date:
@@ -174,8 +174,8 @@ def check_latitude_longitude(latlng):
try:
lat, lng = latlng.split(',')
lat = float(lat)
- long = float(lng)
- return (-90.0 <= lat <= 90.0) and (-180.0 <= long <= 180.0)
+ lng = float(lng)
+ return (-90.0 <= lat <= 90.0) and (-180.0 <= lng <= 180.0)
except ValueError:
return False
@@ -203,15 +203,22 @@ def check_helpers(helpers):
@look_for_fixme
-def check_email(email):
+def check_emails(emails):
"""
- 'contact' must be a valid email address consisting of characters,
- an '@', and more characters. It should not be the default contact
- email address 'admin@software-carpentry.org'.
+ 'emails' must be a comma-separated list of valid email addresses.
+ The list may be empty. A valid email address consists of characters,
+ an '@', and more characters. It should not contain the default contact
"""
- return bool(re.match(EMAIL_PATTERN, email)) and \
- (email != DEFAULT_CONTACT_EMAIL)
+ # YAML automatically loads list-like strings as lists.
+ if (isinstance(emails, list) and len(emails) >= 0):
+ for email in emails:
+ if ((not bool(re.match(EMAIL_PATTERN, email))) or (email == DEFAULT_CONTACT_EMAIL)):
+ return False
+ else:
+ return False
+
+ return True
def check_eventbrite(eventbrite):
@@ -227,12 +234,12 @@ def check_eventbrite(eventbrite):
@look_for_fixme
-def check_etherpad(etherpad):
+def check_collaborative_notes(collaborative_notes):
"""
- 'etherpad' must be a valid URL.
+ 'collaborative_notes' must be a valid URL.
"""
- return bool(re.match(URL_PATTERN, etherpad))
+ return bool(re.match(URL_PATTERN, collaborative_notes))
@look_for_fixme
@@ -286,13 +293,14 @@ def check_pass(value):
'helper list isn\'t a valid list of format ' +
'["First helper", "Second helper",..]'),
- 'contact': (True, check_email,
- 'contact email invalid or still set to ' +
- '"{0}".'.format(DEFAULT_CONTACT_EMAIL)),
+ 'email': (True, check_emails,
+ 'contact email list isn\'t a valid list of format ' +
+ '["me@example.org", "you@example.org",..] or contains incorrectly formatted email addresses or ' +
+ '"{0}".'.format(DEFAULT_CONTACT_EMAIL)),
'eventbrite': (False, check_eventbrite, 'Eventbrite key appears invalid'),
- 'etherpad': (False, check_etherpad, 'Etherpad URL appears invalid'),
+ 'collaborative_notes': (False, check_collaborative_notes, 'Collaborative Notes URL appears invalid'),
'venue': (False, check_pass, 'venue name not specified'),
@@ -300,10 +308,10 @@ def check_pass(value):
}
# REQUIRED is all required categories.
-REQUIRED = set([k for k in HANDLERS if HANDLERS[k][0]])
+REQUIRED = {k for k in HANDLERS if HANDLERS[k][0]}
# OPTIONAL is all optional categories.
-OPTIONAL = set([k for k in HANDLERS if not HANDLERS[k][0]])
+OPTIONAL = {k for k in HANDLERS if not HANDLERS[k][0]}
def check_blank_lines(reporter, raw):
@@ -311,7 +319,8 @@ def check_blank_lines(reporter, raw):
Blank lines are not allowed in category headers.
"""
- lines = [(i, x) for (i, x) in enumerate(raw.strip().split('\n')) if not x.strip()]
+ lines = [(i, x) for (i, x) in enumerate(
+ raw.strip().split('\n')) if not x.strip()]
reporter.check(not lines,
None,
'Blank line(s) in header: {0}',
@@ -381,7 +390,7 @@ def check_config(reporter, filename):
kind)
carpentry = config.get('carpentry', None)
- reporter.check(carpentry in ('swc', 'dc'),
+ reporter.check(carpentry in ('swc', 'dc', 'lc', 'cp'),
filename,
'Missing or unknown carpentry: {0}',
carpentry)
diff --git a/favicon-dc.ico b/favicon-dc.ico
deleted file mode 100644
index 4937f2e..0000000
Binary files a/favicon-dc.ico and /dev/null differ
diff --git a/favicon-lc.ico b/favicon-lc.ico
deleted file mode 100644
index f4f3c93..0000000
Binary files a/favicon-lc.ico and /dev/null differ
diff --git a/favicon-swc.ico b/favicon-swc.ico
deleted file mode 100644
index 34f80ad..0000000
Binary files a/favicon-swc.ico and /dev/null differ
diff --git a/index.md b/index.md
index fec928b..62aaded 100644
--- a/index.md
+++ b/index.md
@@ -1,6 +1,5 @@
---
layout: lesson
-root: .
---
Version control is the lab notebook of the digital world:
diff --git a/reference.md b/reference.md
index cecb361..7835462 100644
--- a/reference.md
+++ b/reference.md
@@ -1,6 +1,5 @@
---
layout: reference
-permalink: /reference/
---
## A Better Kind of Backup
diff --git a/setup.md b/setup.md
index 2747cf0..87fce20 100644
--- a/setup.md
+++ b/setup.md
@@ -1,7 +1,6 @@
---
layout: page
title: "Setup"
-permalink: /setup/
---
Please use [TortoiseHg](http://tortoisehg.bitbucket.org/) to install Mercurial on Windows,