README.md

LOSH Appropedia scraper

This contains code (BASH and AWK) that fetches the list of project names hosted on appropedia.org, and makes them available as:

• appro_proj_names.csv - the list of project names
• appro_yaml_urls.csv - the table containing only the OKH v1 file URLs

(Both are re-generated weekly at 03:02 each Monday, UTC)

The later uses the same format as projects_okhs.csv - the official OKH v1 "database" of known, supporting projects.

That later file - appro_yaml_urls.csv - can be used to fetch all the okh.yml files of appropedia.org and convert them to our newer format, using the conversion tool.

The new version of OKH (LOSH-v1) is currently being developed at LOSH.

Use

run the script:

./appro-fetcher

If all goes well, you will find the generated lists at target/*.csv.