Feature request: workflow to update servers data periodically

[shwoop]

What's the feature 🧐

There is a lot of noise relating to the hard coded servers.json list being out of date.

• Many issues are opened.
• Many PRs are also opened.

I can see some "solutions" that should not be too hard to implement:

• Update via workflow
• Update on first run
• Add notices
• ...?

Update on workflow

On a schedule (weekly/monthly) a workflow will run.
The workflow will execute the existing update functionality against all providers.
If a diff highlights changes, it will open a PR.

pros

• Will automatically highlight server config changes from provider.

cons

• Adds load to maintainer, having to approve and release the generated changes.
• Frequency may not be enough to avoid issues being raised.

Update on first run

When running, if STORAGE_FILEPATH is specified but the file does not exist (implying first run), we run the update logic against the configured provider.

Probably want to have it behind a setting so the application is not too spammy.

pros

• Very little additional load for maintainer.
• Easy for users to manage (set setting to true).

cons

• Requires code change.
• May be some tangled dependency: need servers.json to connect vpn but need vpn connection to update servers.json.

Add notices

Update the Issue and Pull request templates to mention that the default config may be outdated and how to run the update command.

pros

• No additional load for maintainer.
• Trivial change.

cons

• Users may ignore the notices
• The manual process is not trivial.

Extra information and references

No response

[github-actions]

@qdm12 is more or less the only maintainer of this project and works on it in his free time.
Please:

• do not ask for updates, be patient
• 👍 the issue to show your support instead of commenting
  @qdm12 usually checks issues at least once a week, if this is a new urgent bug,
  revert to an older tagged container image

[shwoop]

I'm happy to work on something, if it's agreed something should be done 😄 .

[qdm12]

Interesting suggestions, thanks!

Update on workflow

I think that's the best course of action. We can configure it to auto-merge too perhaps, so it survives even if I get run-over 😄
The only downside is that it would have quite a few megabytes to the repository over time, but worth the size increase I think.
I also think once every 2 months is good enough 😉 People can wait, and, if they can't, they can update servers themselves still.

If you have some time and Github actions experience, go ahead on that one, you have my blessing 😄 👍 ! Let's have it just create a PR for the time being, we'll switch it to auto-commit somehow later, once it's proven reliable. Thanks!!

Update on first run

Not possible because the firewall should block any connection before the VPN is up. there is already UPDATER_PERIOD which can be set to periodically update your provider servers data periodically, after the VPN is up. See https://github.com/qdm12/gluetun-wiki/blob/main/setup/servers.md#update-periodically
There is the alternate mode to run gluetun to update servers only (update -enduser ...), but that should be more of a cli way of using gluetun without connected containers.

Add notices

• Your PR template got merged 👍 !
• We got a Github wiki url logged when the connection fails, pointing you to update your servers data (Users may ignore the notices well well well, you got a point here!)
• issue template doesn't have it, but it's already rather full of boxes and stuff, so it would likely be overlooked anyway. I think we can skip on that one.

[qdm12]

I'm happy to work on something, if it's agreed something should be done

No pressure, just a tiny reminder (getting excited to have this!) - especially for Proton which now requires credentials to access the servers data, which I would like users NOT to provide Gluetun to reduce security risks... 🤷

Thanks!

[shwoop]

I'll get to work on a workflow.

[qdm12]

I've updated Protonvpn servers data manually (as in with code, but without merging the following), but I'll wait for This to merge the new Proton updater fix #2878 since it requires proton credentials which is a security-clumsy step to take just to update servers data (better to run it in CI). Take your time though 😉