-
-
Notifications
You must be signed in to change notification settings - Fork 2.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Bug] Can't connect to marketplace anymore when curl uses brotli compression #22222
Comments
When calling the URI from a simple PHP testscript using Guzzle I also get a response from the Matomo endpoint. When I pass a random URL to the method trying to fetch the data ( When I mirror the JSON reponse of the marketplace from another instance, then the request is fine as well. So curl is working ✔️, the server is able to handle requests (seems to be no firewall issue) ✔️ . Somehow the connection from Matomo to the Matomo marketplace URI is faulty. |
The connection error lead me into Matomos HTTP class, which sets up custom curl request commands. In there I found the error. The following line instructs curl to always request all encodings it supports: Lines 677 to 680 in 4a5a877
(introduced in #17009 and afaik reasonable). Outgoing requests show that my server sends the header I was able to reproduce the issue with a simple script: <?php
$requestUrl = 'https://plugins.matomo.org/api/2.0/plugins?keywords=&purchase_type=&query=&sort=lastupdated&release_channel=latest_stable&prefer_stable=1&piwik=5.0.3&php=8.1.28&mysql=10.6.17-MariaDB&num_users=9&num_websites=14';
//$requestUrl = 'https://httpbin.dev/brotli';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $requestUrl);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_ENCODING, 'br');
$data = curl_exec($ch);
print_r($data); However: This script works fine on a local test machine. Running the request via CLI works on both machines ( I have contacted my hosting provider, maybe php-curl was not compiled with brotli. FYI: I wrote the a Guzzle script for the same request and this works. Guzzle decompresses the reponse using a stream wrapper. |
The hosting uses Red Hat Enterprise Linux 7 with curl version 7.29. Since this is a long-term support OS, that's a rather old version, and the provider therefore installed an additional curl. PHP is provided in multiple versions with the popular Remi repo (https://rpms.remirepo.net/). Since these versions are supposed to run on RHEL 7, they are built against version 7.29. Brotli is supported in curl since version 7.57 (https://curl.se/changes.html). Due to the mentioned empty My hotfix: I hardcoded Line 680 in 4a5a877
Question: This issues did not pop up until this month. My provider said that they run this software setup for a long time already. Did the Matomo Marketplace API Server maybe activate brotli responses just now? Suggestion: As written before, if Matomo would work with a stream wrapper - as Guzzle does - this would not trigger an error. Maybe add a stream wrapper or even replace the custom curl scripts with a library like Guzzle? Depends how many users are affected by the issue. |
I can reproduce the exact same behaviour, including @pixelbrackets test script and the fix. |
@patrickli @samjf Were there any changes done to our Marketplace API Server recently, which might cause the problem? |
Not anything I'm aware of. @samjf did we upgrade PHP version for it? I just checked and it is |
@patrickli @sgiehl It is possible the servers could have been upgraded in that time -- from memory it was before 3 weeks ago though. I'll DM you more details. |
This actually sounds more like a problem with php curl rather than an issue with Matomo or our server. |
I ran into this today setting up my first instance in Docker using the fpm-alpine image. From debugging, I get the same empty response. Curl from host and container CLI both work. Workaround works for me on 5.0.3 with PHP 8.2, applied with sed from the host: docker exec matomo_app \
sed -i 's/_ENCODING] = "/&deflate, gzip/' /var/www/html/core/Http.php On the admin dash it manifests as 3 identical errors down the page, but all three linked to the FAQ about setting up the cron. (on the bright side, cron got configured much sooner...) |
Hi, PHPinfo confirms that curl is compiled with brotli. The hotfix suggested by @pixelbrackets works for me too. I have access to a second server where the marketplace works fine and on which curl isn't compiled with brotli. Interesting things : forcing matomo to use http instead of https by adding |
I just upgraded to PHP 8.3.10 on my Matomo server (previously 8.3.8) and marketplace is working again. I double-checked by switching back to 8.3.8 to verify that it didn't work on that version. I also noticed that the curl version has changed (8.7.1 with php 8.3.8 docker image and 8.9.0 with php 8.3.10 docker image). Finally, there is a fix about curl in PHP 8.3.9 changelog. So, the issue seems to be fixed in the latest php or curl version. |
Hey @MerleCedric If anyone still experiences that problem after updating PHP / curl, please let us know, so we can reconsider this. |
FYI I sucessfully checked the solution described by @MerleCedric. My hoster installed an new curl version and PHP. My example script works again. And the Matomo Marketplace is available again when curl uses brotli compression. So it was indeed an issue with curl and/or PHP. All done. ✔️ |
What happened?
I can't reach the marketplace anymore. It keeps showing
There was an error reading the response from the Marketplace: Please try again later.
I debugged Matomo to find the requested URI:
https://plugins.matomo.org/api/2.0/plugins?keywords=&purchase_type=&query=&sort=lastupdated&release_channel=latest_stable&prefer_stable=1&piwik=5.0.3&php=8.1.28&mysql=10.6.17-MariaDB&num_users=9&num_websites=14
The response for this URI is always empty. The empty response triggers the above error message.
System report and security report in the Matomo backend look good and dont mentian any missing or faulty software.
Calling the URI from CLI works fine as well (
curl https://plugins.matomo.org/…
gives the expected JSON reponse).Any hints what could be wrong here?
What should happen?
Marketplace is accessible
How can this be reproduced?
Try to open the marketplace page. Try to run "Looking for updates" in plugin page.
Matomo version
5.0.3
PHP version
8.1
Server operating system
Red Hat Enterprise Linux 7 with curl version 7.29 - See comment three
What browsers are you seeing the problem on?
All browsers
Computer operating system
Linux
Relevant log output
No response
Validations
The text was updated successfully, but these errors were encountered: