-
-
Notifications
You must be signed in to change notification settings - Fork 2.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
GeoIP2 updater might try to download new file before it exists #18427
Comments
This night, our job tried to unzip until 5AM GMT+1. Maybe we can simply skip the monthly job on 2nd of each month? 🤔 |
Same here, https://download.db-ip.com/free/dbip-city-lite-2023-11.mmdb.gz returns a 404 at the time of writing. Though it will probably work in a few hours... Moving the cron to the 2nd of the month sound like a good idea to limit those issue without having data that are too stale. |
I created #21468 as a possible fix for this issue. |
When geolocation database updates are configured to be done monthly, they were scheduled on the 3rd day of _the week_, so Wednesday. For months that start on a Wednesday, that means that we tried to fetch a file that might not exist yet. We now schedule on the 3rd day of _the month_, as I believe was the original intent of dc98a97 that introduced the behavior a long time ago. So this gives 3 days to db-ip.com to update their files in all cases. Fixes #18427
Reopening, as the PR only adjusted the day the download is tried. We still should aim to implement a proper handling when the download fails. |
There is no proper handling if the file is simply missing. I think an administrator notification in GUI and an error logged (already exists), they are enough. |
When geolocation database updates are configured to be done monthly, they were scheduled on the 3rd day of _the week_, so Wednesday. For months that start on a Wednesday, that means that we tried to fetch a file that might not exist yet. We now schedule on the 3rd day of _the month_, as I believe was the original intent of dc98a97 that introduced the behavior a long time ago. So this gives 3 days to db-ip.com to update their files in all cases. Fixes #18427
When geolocation database updates are configured to be done monthly, they were scheduled on the 3rd day of _the week_, so Wednesday. For months that start on a Wednesday, that means that we tried to fetch a file that might not exist yet. We now schedule on the 3rd day of _the month_, as I believe was the original intent of dc98a97 that introduced the behavior a long time ago. So this gives 3 days to db-ip.com to update their files in all cases. Fixes #18427
When geolocation database updates are configured to be done monthly, they were scheduled on the 3rd day of _the week_, so Wednesday. For months that start on a Wednesday, that means that we tried to fetch a file that might not exist yet. We now schedule on the 3rd day of _the month_, as I believe was the original intent of dc98a97 that introduced the behavior a long time ago. So this gives 3 days to db-ip.com to update their files in all cases. Fixes #18427
Same issue tody, it tries to download |
Note: this is happening with matomo 5.0.3, which already has the change from #21468. Excerpt from
|
I'm getting an error message every hour or so here: ERROR [2024-05-03 01:05:12] 68789 /var/www/matomo/plugins/GeoIp2/GeoIP2AutoUpdater.php(190): GeoIP2AutoUpdater: failed to unzip '/var/www/matomo/tmp/latest/DBIP-City.mmdb.gz.download' after downloading 'https://download.db-ip.com/free/dbip-city-lite-2024-05.mmdb.gz': The downloaded file is not a valid geolocation database. Please re-check the URL or download the file manually. [Query: , CLI mode: 1] Still using Matomo 4.9.1 |
Same problem here.. |
reported in https://forum.matomo.org/t/geoip2autoupdater-failed-to-unzip-the-downloaded-file-is-not-a-valid-geolocation-database/43925 (and also found in my cronjob)
Expected Behavior
Whenever a GeoIP2 update job falls at the start of a month it might be possible that the
dbip-city-lite-2021-12.mmdb.gz
doesn't exist on the servers and the update failsCurrent Behavior
Possible Solution
In case a 404 is returned, Matomo could fetch the previous months file again or reschedule the job for a few hours later.
The text was updated successfully, but these errors were encountered: