Cloudflare Workers not work with Countries

[viriatusX]

Past Issues Searched

• I have searched open and closed issues to make sure that the bug has not yet been reported

Issue is a Bug Report

• This is a bug report and not a feature request, nor asking for self-hosted support

Using official Plausible Cloud hosting or self-hosting?

Self-hosting

Describe the bug

When you enable the Plausible method with Cloudflare workers (https://plausible.io/docs/proxy/guides/cloudflare).
It does not update the visitors' country map in the UI.

Expected behavior

Plausible should update the UI, the visitor map by country, by having CloudFlare workers enabled.

Screenshots

No response

Environment

No response

[ruslandoga]

👋 @viriatusX

The last time I tried cf workers, everything was working as expected: #2137 (comment)

Could you please share more of your setup?

[viriatusX]

Hi @ruslandoga

All statistics are displayed correctly, except the Map with the country / region / city of the visitors. It does not show anything.
When I deactivate the Worker, everything works perfectly.

I have Plausible with Nginx Proxy Manager (without additional HTTP headers.
I have the Plausible domain as "Proxied".
I have tried with Plausible remote:
iex> Plausible.Geo.lookup("1.1.1.1.1")
And it shows the country and everything ok.

The problem is that it is not reflected in the UI.

[ruslandoga]

Would you be able to share your configuration files, including nginx, worker scripts, and anything else that seems relevant?

[ruslandoga]

You can also trace the incoming request to see what headers are included with recon_trace. There have been several discussions where it helped resolve IP geo location issues. #3248 (comment) #2767 (reply in thread) etc. (you can search for others using parts of that code snippet)

[viriatusX]

I have executed this inside Plausible remote. But I don't know how to check it with my domain. Sorry.

iex> headers = fn {:trace, _pid, :call, {_mod, _fun, [%Plug.Conn{req_headers: headers}]}} -> inspect(headers) end
iex> mod_fun_args = {PlausibleWeb.RemoteIp, :get, 1}
iex> how_many_times = 5
iex> :recon_trace.calls(mod_fun_args, how_many_times, formatter: headers)

My Worker Script is the same of yours (changing only my domain).

My Nginx conf for the Plausible domain:

# ------------------------------------------------------------
# stats.xxxxx.com
# ------------------------------------------------------------


server {
  set $forward_scheme http;
  set $server         "192.168.10.165";
  set $port           3998;

  listen 80;
listen [::]:80;

listen 443 ssl http2;
listen [::]:443 ssl http2;


  server_name xxxxx.com;


  # Let's Encrypt SSL
  include conf.d/include/letsencrypt-acme-challenge.conf;
  include conf.d/include/ssl-ciphers.conf;
  ssl_certificate /etc/letsencrypt/live/npm-23/fullchain.pem;
  ssl_certificate_key /etc/letsencrypt/live/npm-23/privkey.pem;


proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $http_connection;
proxy_http_version 1.1;

  access_log /data/logs/proxy-host-15_access.log proxy;
  error_log /data/logs/proxy-host-15_error.log warn;


  location / {

    
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection $http_connection;
    proxy_http_version 1.1;
    

    # Proxy!
    include conf.d/include/proxy.conf;
  }


  # Custom
  include /data/nginx/custom/server_proxy[.]conf;
}

[viriatusX]

I execute Plausible GeoIP command with this IP, but in the UI not appear... with Workers enabled.

[viriatusX]

Ant the output for this command (in case it helps.)

[ruslandoga]

I have executed this inside Plausible remote. But I don't know how to check it with my domain. Sorry.

You would need to visit your site which would make an /api/event call. The traces would be printed in the console. Ideally, you'd run it both when the Worker is activated and when it's not.

[ruslandoga]

All statistics are displayed correctly, except the Map with the country / region / city of the visitors. It does not show anything.
When I deactivate the Worker, everything works perfectly.

I've re-read this now, and it's not clear what exactly is wrong, is the map empty or not displayed at all? Can you show screenshot for when the Worker is activated and when it's not?

[viriatusX]

If I access with an IP from Portugal:

With Worker

Without Worker

The map is not updated. The rest of the statistics work OK. If it was a /api/event problem, I think it would not show any statistics.

[ruslandoga]

Great, then #3976 (comment) should help resolve it. Most likely, the X-Forwarded-For header or similar is dropped somewhere along the way. You can also trace Plausible.Geo.lookup to see what exact IP address is used for the lookup of your Portugal request and what info it gets in return.

[viriatusX]

Great, then #3976 (comment) should help resolve it. Most likely, the X-Forwarded-For header or similar is dropped somewhere along the way. You can also trace Plausible.Geo.lookup to see what exact IP address is used for the lookup of your Portugal request and what info it gets in return.

Thanks for your help.
Forgive my clumsiness. I have executed this, but then I don't know how to continue.

iex> headers = fn {:trace, _pid, :call, {_mod, _fun, [%Plug.Conn{req_headers: headers}]}} -> inspect(headers) end
iex> mod_fun_args = {PlausibleWeb.RemoteIp, :get, 1}
iex> how_many_times = 5
iex> :recon_trace.calls(mod_fun_args, how_many_times, formatter: headers)

[ruslandoga]

I have executed this, but then I don't know how to continue.

You would need to visit https://hdsplus.co after you run these commands. The traces would be printed in the console. Ideally, you'd run it both when the Worker is activated and when it's not.

You can check the previous discussions I linked for more info :) E.g. #2767 (reply in thread)

[viriatusX]

Sorry, on which console do I view the trace? The Plausible console does not show anything new.

1. I execute these commands
2. I access to my Web with Chrome and Safari

[ruslandoga]

If it doesn't show anything, then there are no requests reaching Plausible. Try running it with the Worker deactivated.

[viriatusX]

Neither... The Plausible remote console does not show anything new after executing these commands.
And without Worker everything works fine. Strange.

[ruslandoga]

Ok, I see. I guess you are on v2.1.0-rc.0? The module name changed from RemoteIp to RemoteIP in #3761. Please run this command instead :recon_trace.calls({PlausibleWeb.RemoteIP, :get, 1}, 10)

[ruslandoga]

Please post the traces as code or text snippet. It would be easier to debug.

[viriatusX]

.

[ruslandoga]

x-forwarded-for are completely different. Are both of these your own requests or real users? Either way, Plausible is picking the first IP address in the x-forwarded-for list, which is different from x-real-ip (whatever it is) which is the last element in the x-forwarded-for list. According to https://en.wikipedia.org/wiki/X-Forwarded-For, the client IP address is the first element in the list, so Plausible is correct, at least according to Wikipedia.

[viriatusX]

Yes.
I tried again

[ruslandoga]

Do you know where x-forwarded-for and x-real-ip come from? One of them seems to be wrong. According to https://developers.cloudflare.com/fundamentals/reference/http-request-headers/#x-forwarded-for CloudFlare adds the client IP address (the real IP address) to the beginning of the list. And since Plausible is reading it from the beginning of the list, it's not a bug in Plausible.

[ruslandoga]

I suggest checking nginx configuration.

[viriatusX]

Now yes.... finally...
I added to NGINX:

location / {


proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $http_connection;
proxy_http_version 1.1;
**proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;**

And now.. alls works!!

Thanks you so much :)

[viriatusX]

**Update: @ruslandoga **

I have already found the "problem".
Plausible works OK and Worker works OK.

But...., if you have the domain as "Proxied" in CloudFlare DNS it does NOT work. Since it already masks the real IP of the Host with a CloudFlare IP.

You can try it if you want.

[viriatusX]

If the domain where you have Plausible is "Proxied" the Worker does not work. It updates the statistics except the map (my main problem).

If the domain is as DNS Only, everything works perfectly (Worker included).

I understand, being in Proxied,the Worker can't get the real IP of the user, since the Host is already masquerading with CloudFlare IP due to the Proxied.

[ruslandoga]

It should work both ways :) cf-connecting-ip is used if available.

[ruslandoga]

analytics/lib/plausible_web/remote_ip.ex

Line 8 in a7603c9

cf_connecting_ip = List.first(Plug.Conn.get_req_header(conn, "cf-connecting-ip"))

[ruslandoga]

You can follow the same tracing steps as before to find where the problem occurs.

[viriatusX]

I seem to have solved it by adding this to NGINX:

set_real_ip_from 103.21.244.0/22;
set_real_ip_from 103.22.200.0/22;
set_real_ip_from 103.31.4.0/22;
set_real_ip_from 104.16.0.0/12;
set_real_ip_from 108.162.192.0/18;
set_real_ip_from 131.0.72.0/22;
set_real_ip_from 141.101.64.0/18;
set_real_ip_from 162.158.0.0/15;
set_real_ip_from 172.64.0.0/13;
set_real_ip_from 173.245.48.0/20;
set_real_ip_from 188.114.96.0/20;
set_real_ip_from 190.93.240.0/20;
set_real_ip_from 197.234.240.0/22;
set_real_ip_from 198.41.128.0/17;
set_real_ip_from 2400:cb00::/32;
set_real_ip_from 2405:b500::/32;
set_real_ip_from 2606:4700::/32;
set_real_ip_from 2803:f800::/32;
set_real_ip_from 2c0f:f248::/32;
set_real_ip_from 2a06:98c0::/29;

real_ip_header CF-Connecting-IP;