Checklists

[Remi-Gau]

NMIND

NMIND bronze certif

testing

• Provide / generate / point to test data
• Provide instructions for users to run tests that include instructions for evaluation for correct behavior

infrastructure

• Code is open source
• Package is under version control
• Readme is present
• License is present
• Issues tracking is enabled
• Digital Object Identifier (DOI) points to latest version
• All documented installation instructions can be successfully followed

documentation

• Landing page (e.g., GitHub README, website) provides a link to documentation and brief description of what program does
• Documentation is up to date with version of software
• Typical intended usage is described
• An example of its usage is shown
• Document functions intended to be used by users.
• Description of required input parameters for user-facing functions with reasonable description of inputs
• Description of output(s)
• User installation instructions available
• Dependencies listed

NMIND silver certif

testing

• Some form of testing suite present
• Test coverage > 50%

infrastructure

• Issue template(s) available
• Continuous integration runs tests
• No excessive files included

documentation

• Background/significance of program
• One or more tutorial to showcase the multiple of the program's usages
• Any alternative usage that is advertised is thoroughly documented
• Thorough description of required and optional input parameters
• Document public functions
• A statement of supported operating systems / environments

NMIND gold certif

testing

• Test coverage > 90%
• Benchmarking information is provided for examples

infrastructure

• Continuous integration builds packages
• Continuous integration validates style
• Journal of Open Source Software submission
• Contribution guide present
• Code of Conduct present

documentation

• Continuous integration badges in README for build status
• Continuous integration badges in README for tests passing
• Continuous integration badges in README for coverage
• Document functions, classes, modules, etc.
• Has a documented style guide
• Maintenance status is documented

---

JOSS "checklist"

• The software must be open source as per the OSI definition.
• The software must be hosted at a location where users can open issues and propose code changes without manual approval of (or payment for) accounts.
• The software must have an obvious research application.
• You must be a major contributor to the software you are submitting, and have a GitHub account to participate in the review process.
• Your paper must not focus on new research results accomplished with the software.
• Your paper (paper.md and BibTeX files, plus any figures) must be hosted in a Git-based repository together with your software (although they may be in a short-lived branch which is never merged with the default).

In addition, the software associated with your submission must:

• Be stored in a repository that can be cloned without registration.
• Be stored in a repository that is browsable online without registration.
• Have an issue tracker that is readable without registration.
• Permit individuals to create issues/file tickets against your repository.

[Remi-Gau]

Scientific python repo review

pipx run 'sp-repo-review[cli]' .

General:

 • Detected build backend: hatchling.build                                                                                        
├── PY001 Has a pyproject.toml ✅
├── PY002 Has a README.(md|rst) file ✅
├── PY003 Has a LICENSE* file ❌
│   Projects must have a license                                                                                                  
├── PY004 Has docs folder ❌
│   Projects must have documentation in a folder called docs (disable if not applicable)                                          
├── PY005 Has tests folder ✅
├── PY006 Has pre-commit config ✅
└── PY007 Supports an easy task runner (nox or tox) ❌
    Projects must have a noxfile.py, tox.ini, or tool.hatch.envs/tool.spin/tool.tox in pyproject.toml to encourage new            
    contributors.                                                                                                                 

PyProject

├── PP002 Has a proper build-system table ✅
├── PP003 Does not list wheel as a build-dep ✅
├── PP301 Has pytest in pyproject ✅
├── PP302 Sets a minimum pytest to at least 6 ❌
│   Must have a minversion=, and must be at least 6 (first version to support pyproject.toml configuration).                      
│   
│                                                                                                                                 
│    [tool.pytest.ini_options]                                                                                                    
│    minversion = "7"                                                                                                             
│                                                                                                                                 
├── PP303 Sets the test paths ❌
│   The testpaths setting should be set to a reasonable default.                                                                  
│   
│                                                                                                                                 
│    [tool.pytest.ini_options]                                                                                                    
│    testpaths = ["tests"]                                                                                                        
│                                                                                                                                 
├── PP304 Sets the log level in pytest ❌
│   log_cli_level should be set. This will allow logs to be displayed on failures.                                                
│   
│                                                                                                                                 
│    [tool.pytest.ini_options]                                                                                                    
│    log_cli_level = "INFO"                                                                                                       
│                                                                                                                                 
├── PP305 Specifies xfail_strict ❌
│   xfail_strict should be set. You can manually specify if a check should be strict when setting each xfail.                     
│   
│                                                                                                                                 
│    [tool.pytest.ini_options]                                                                                                    
│    xfail_strict = true                                                                                                          
│                                                                                                                                 
├── PP306 Specifies strict config ❌
│   --strict-config should be in addopts = [...]. This forces an error if a config setting is misspelled.                         
│   
│                                                                                                                                 
│    [tool.pytest.ini_options]                                                                                                    
│    addopts = ["-ra", "--strict-config", "--strict-markers"]                                                                     
│                                                                                                                                 
├── PP307 Specifies strict markers ❌
│   --strict-markers should be in addopts = [...]. This forces all markers to be specified in config, avoiding misspellings.      
│   
│                                                                                                                                 
│    [tool.pytest.ini_options]                                                                                                    
│    addopts = ["-ra", "--strict-config", "--strict-markers"]                                                                     
│                                                                                                                                 
├── PP308 Specifies useful pytest summary ❌
│   An explicit summary flag like -ra should be in addopts = [...] (print summary of all fails/errors).                           
│   
│                                                                                                                                 
│    [tool.pytest.ini_options]                                                                                                    
│    addopts = ["-ra", "--strict-config", "--strict-markers"]                                                                     
│                                                                                                                                 
└── PP309 Filter warnings specified ❌
    filterwarnings must be set (probably to at least ["error"]). Python will hide important warnings otherwise, like deprecations.
    
                                                                                                                                  
     [tool.pytest.ini_options]                                                                                                    
     filterwarnings = ["error"]                                                                                                   

GitHub Actions

├── GH100 Has GitHub Actions config ❌
│   All projects should have GitHub Actions config for this series of checks.  If there are no .yml files in .github/workflows,   
│   the remaining checks will be skipped.                                                                                         
├── GH101 Has nice names [skipped]
├── GH102 Auto-cancel on repeated PRs [skipped]
├── GH103 At least one workflow with manual dispatch trigger [skipped]
├── GH104 Use unique names for upload-artifact [skipped]
├── GH200 Maintained by Dependabot [skipped]
├── GH210 Maintains the GitHub action versions with Dependabot [skipped]
├── GH211 Do not pin core actions as major versions [skipped]
└── GH212 Require GHA update grouping [skipped]

Pre-commit

├── PC100 Has pre-commit-hooks ✅
├── PC110 Uses black or ruff-format ✅
├── PC111 Uses blacken-docs ✅
├── PC140 Uses mypy ❌
│   Must have https://github.com/pre-commit/mirrors-mypy repo in .pre-commit-config.yaml                                          
├── PC160 Uses codespell ✅
├── PC170 Uses PyGrep hooks (only needed if RST present) ✅
├── PC180 Uses prettier ✅
├── PC190 Uses Ruff ❌
│   Must have https://github.com/astral-sh/ruff-pre-commit repo in .pre-commit-config.yaml                                        
├── PC191 Ruff show fixes if fixes enabled [skipped]
└── PC901 Custom pre-commit CI message ❌
    Should have something like this in .pre-commit-config.yaml:                                                                   
    
                                                                                                                                  
     ci:                                                                                                                          
       autoupdate_commit_msg: 'chore: update pre-commit hooks'                                                                    

MyPy

├── MY100 Uses MyPy (pyproject config) ❌
│   Must have tool.mypy section in pyproject.toml. Other forms of configuration are not supported by this check.                  
├── MY101 MyPy strict mode [skipped]
├── MY102 MyPy show_error_codes deprecated [skipped]
├── MY103 MyPy warn unreachable [skipped]
├── MY104 MyPy enables ignore-without-code [skipped]
├── MY105 MyPy enables redundant-expr [skipped]
└── MY106 MyPy enables truthy-bool [skipped]

Ruff

├── RF001 Has Ruff config ❌
│   Must have [tool.ruff] section in pyproject.toml or ruff.toml/.ruff.toml.                                                      
├── RF002 Target version must be set [skipped]
├── RF003 src directory specified if used [skipped]
├── RF101 Bugbear must be selected [skipped]
├── RF102 isort must be selected [skipped]
├── RF103 pyupgrade must be selected [skipped]
├── RF201 Avoid using deprecated config settings [skipped]
└── RF202 Use (new) lint config section [skipped]

Documentation

├── RTD100 Uses ReadTheDocs (pyproject config) ❌
│   Should have a .readthedocs.yaml file in the root of the repository. Modern ReadTheDocs requires (or will require soon) this   
│   file.                                                                                                                         
├── RTD101 You have to set the RTD version number to 2 [skipped]
├── RTD102 You have to set the RTD build image [skipped]
└── RTD103 You have to set the RTD python version [skipped]

[Remi-Gau]

Scientific python repo review

This one is based on the branch I used for PR #6.

Will fix a few issues already there.