Skip to content

Latest commit

 

History

History
19 lines (10 loc) · 1.07 KB

README.md

File metadata and controls

19 lines (10 loc) · 1.07 KB

Simple GitHub scraper

This is a template repository for a simple GitHub scraper pioneered by Simon Wilison.

The 'simple' part

At this point the template only supports very simple fetching and committing of a JSON data file from somewhere on the internet.

Replace https://www.example.com/data.json in the fetch.yaml file with the URL of the data you want to scrape.

Commit and push the repo to GitHub and you're ready to go.

By default the scraper will run once per week, but you can change the cron schedule in the fetch.yaml file.

Data is stored in data.json.

You may need to update the permissions on the new repository to allow workflows to make commits to the repository.

In settings -> actions -> general look for the setting that says 'Read and write permissions' under the heading 'Workflow permissions'