Impossible to ZIM files from upload.wikimedia.org

[benoit74]

See openzim/zim-requests#272 (comment)

Since files are hosted on upload.wikimedia.org, we must comply with their User-Agent policy at https://meta.wikimedia.org/wiki/User-Agent_policy

I suggest we add a CLI option to pass a custom User-Agent to be used when downloading the nautilus files.

[rgaudin]

Indeed ; shouldn't scraperlib do this by default for stream_file?

[benoit74]

shouldn't scraperlib do this by default for stream_file?

yes for upload.wikimedia.org ; CLI argument would still help for "less known" websites.

opened openzim/python-scraperlib#214