Manual log upload

A few services are not using matomo directly but rely on an importer that parses web server logs and imports those into matomo.

Should this fail for any reason, we may need to manually re-import the logs. This describes the procedure.

• Must be one of download-kiwix, download-openzim, library-kiwix, demo-library-kiwix
• Date(s) to re-import should be less than 30d (see downloads-logrotate)
• Start manual-log-uploader and attach to shell
• export MATOMO_SITE_ID to the site you're targeting
• Logs for date YYYY-MM-DD is in file /var/log/nginx/<project>/access.log-YYYY-MM-<DD+1>.gz
• Extract the gzip file gunzip -k <access.log-xxxx.gz>(you need to do it from the machine itself since FS is readonly from within container)
• set LOG_PATH to full path of the extracted file
• set HOST to the domain-name of the website
• run import-log

Example for 2022-07-10:

kubectl apply -f zim/manual-log-uploader.job.yaml

cd /var/log/nginx/download-kiwix
gunzip -k access.log-20220711.gz
MATOMO_SITE_ID=2 HOST=download.kiwix.org LOG_PATH=/var/log/nginx/download-kiwix/access.log-20220711 import-log
rm ./access.log-20220711

kubectl delete job manual-log-uploader

---

MATOMO_SITE_ID=2 HOST=download.kiwix.org
MATOMO_SITE_ID=6 HOST=download.openzim.org
MATOMO_SITE_ID=13 HOST=library.kiwix.org
MATOMO_SITE_ID=16 HOST=browse.library.kiwix.org