Create SMP items as issues through GitHub/-Lab API

[katrinleinweber]

Dear @mikej888,

while working on #12 and talking to colleagues about that approach, I concluded that it has too many limitations. A more general approach may be to script the bulk-creation of issues, based on SMP_Checklist.yaml (or a similarly machine-readable version).

This would have the advantage of users being able to import SMP-related issues at any time during a project, not only at the start. And it could be combined with the current workflow of generating a document.

I started researching a bit about how this could be done:

• GitHub's Create an issue API
• GitLab's New issue API
• potentially useful, existing Python scripts, modules, etc.
  • https://gist.github.com/JeffPaine/3145490
  • GrappigPanda/pygemony's github.py
  • donnemartin/gitsome's gh-create-issue (maybe too large of a dependency for a single task)
  • justinekizhak/create_project's read_yaml.py & & gitlab_api.py
  • doctormo/python-gitlab3's add_issue()
  • @emilyriederer's R package projmgr can create GitHub issues & milestones. Context is the content planning of a research project, but the method would be the same as for the SMP.
  • ...
• more specific, YAML-to-issue tools, or MD-to-YAML
  • ...

If you have any ideas, thoughts, suggestions about this, please let me know.

Kind regards :-)

[mikej888]

Hi @katrinleinweber,

I think this is a good idea. It could also be useful beyond just the scope of SMPs. For example, SSI's consultancy projects have standard tasks that are done for all projects e.g. write project brief, write initial blog post etc. So, a generic "create issues from a configuration file" tool could be useful for this use case. For SMPs, there could then be a preprocessing step to convert an SMP configuration into an issues configuration. With that in mind, key components could be:

• smp-to-issue-configuration: reads SMP configuration file (YAML) and writes issue configuration file (YAML).
• create-issues: reads issue creation configuration file and creates issues via a Repository API.
• repository-api: provides abstraction of issue creation functionality.
• repository-specific plugins implement repository-api e.g.
  • github-plugin
  • git;ab-plugin
  • bitbucket-plugin, uses REST API (reference), specifically Create a new issue
• Plugin to use could be determined by parsing the repository URL (for example) provided by the user with their configuration file. A naive approach might be if a URL does not contain github.com or bitbucket.org then it is assumed to be a GitLab server.

So, for GitHub, the flow would be:

smp-configuration.yml => smp-to-issue-configuration => issue-configuration.yml => create-issues => repository-api => github-plugin => GitHub Create an issue API = > GitHub issue(s)

I think Python would be a good language to implement any tools in, it's sufficiently popular, usable and has good REST API libraries.

Best wishes,
mike

[katrinleinweber]

Thanks for detailling your thoughts. Maybe in this more general use-case, wking's github-push-issues would be worth a try then. It expects a folder full of one-.md-per-issue. That kind of template structure could be concatenated into a single document as well.

This leads me to wonder what the best argument is in favor of keeping the current items in a .yaml structure?

[mikej888]

If there was a future desire to add more content to an SMP, beyond just questions and description, it's hard to see how that would easily fit into the folder with one-.md-per-issue approach which loses some of the expressiveness of YAML. It also distributes the content across multiple files and folders rather than keeping it within one file (which makes it easier to check the diffs between versions too).

I'd prefer to keep the YAML and instead use a script to parse the YAML and create the folder/sub-folder/.md structure.

[emilyriederer]

(Hey @katrinleinweber - just stopping by the say thanks for the tag! Super fun to know it was useful 🙂 )