[BUG] CloudFront proxy break https://geoip.maps.opensearch.org endpoint

[smortex]

What is the bug?
When following the documentation instructions to setup a new IP2Geo datasource, the creation fail, likely because of proxying on the OpenSearch Project side (details bellow).

How can one reproduce the bug?
From the "Dev Tools", send this request:

PUT /_plugins/geospatial/ip2geo/datasource/asn
{
  "endpoint": "https://geoip.maps.opensearch.org/v1/geolite2-asn/manifest.json",
  "update_interval_in_days": 3
}

The request fail because of "HTTP response code: 403 for URL: https://geoip.maps.opensearch.org/v1/geolite2-asn/manifest.json":

{
  "error": {
    "root_cause": [
      {
        "type": "action_request_validation_exception",
        "reason": "Validation Failed: 1: Error occurred while reading a file from https://geoip.maps.opensearch.org/v1/geolite2-asn/manifest.json: java.io.IOException: Server returned HTTP response code: 403 for URL: https://geoip.maps.opensearch.org/v1/geolite2-asn/manifest.json;"
      }
    ],
    "type": "action_request_validation_exception",
    "reason": "Validation Failed: 1: Error occurred while reading a file from https://geoip.maps.opensearch.org/v1/geolite2-asn/manifest.json: java.io.IOException: Server returned HTTP response code: 403 for URL: https://geoip.maps.opensearch.org/v1/geolite2-asn/manifest.json;"
  },
  "status": 400
}

What is the expected behavior?
The datasource should be created without error.

What is your host/environment?

• OS: Debian
• Version 2.11.1

Do you have any additional context?
The endpoint URL when reached with Firefox works as expected.

Accessing the same URL with curl fail with a same error code:

romain@zappy ~ % curl -I https://geoip.maps.opensearch.org/v1/geolite2-asn/manifest.json       
HTTP/2 403 
server: CloudFront
date: Sat, 10 Feb 2024 20:50:38 GMT
content-type: text/html
content-length: 919
x-cache: Error from cloudfront
via: 1.1 b2ec1df9e45c0b11d79d727742d0a012.cloudfront.net (CloudFront)
x-amz-cf-pop: LAX3-C4
x-amz-cf-id: 8XCJ17hp0k19lpqou6lULHdvYCVdjzdQ7yyRAKD6DNBnYFHzIZvrUQ==

Accessing the same URL with curl and providing a custom User-Agent works:

romain@zappy ~ % curl -I -H "User-Agent: Firefox" https://geoip.maps.opensearch.org/v1/geolite2-asn/manifest.json
HTTP/2 200 
content-type: binary/octet-stream
content-length: 282
date: Sat, 10 Feb 2024 20:49:57 GMT
last-modified: Fri, 09 Feb 2024 22:22:36 GMT
etag: "cb0bf663c6922899d806090aba4e0910"
x-amz-server-side-encryption: AES256
content-encoding: 
accept-ranges: bytes
server: AmazonS3
x-cache: Hit from cloudfront
via: 1.1 a45555e8d20cb505b62dfda511e6ba6c.cloudfront.net (CloudFront)
x-amz-cf-pop: LAX3-C4
x-amz-cf-id: C7EDBw-xoaqQfsos-RkY05yuuB3Xac6V95sPMsyOmrVffuSQyIkjJg==
age: 29

According to the headers of the failing request, a proxy (CloudFront) in front of the service seems to prevent OpenSearch from downloading the manifest.

[smortex]

Testing the same from another cluster worked. At the same time, the request are still broken on the first machine.

Both nodes are in different regions:

• 1st server (always failed) from 151.80.32.0/18 located in France, provider: OVH.
• 2nd server (request passed) from 51.79.0.0/16 located in Canada, provider: OVH.

Probably some GeoIP restrictions or IP block reputation is hurting.

[heemin32]

Thanks for reporting the issue. The IP address is blocked by internal rule in firewall. We will re-evaluate the rule and see if we can unblock the IP address.

[heemin32]

Could you try again and see if it works now?

[smortex]

Could you try again and see if it works now?

{
  "acknowledged": true
}

Look good, thanks 🤩 ! As far as I am concerted we can close this issue, but I will let you do so in case the change is temporary and you need this feedback to make it permanent.

Thanks!