[WPMU DEV Dashboard] WPMUDEV API Error

Hi,

As you can see from our account, i have your dashboard installed on several sites.

Each and every week, one or more of those sites throws an error like this:

[29-Jul-2019 04:20:22 UTC] [WPMUDEV API Error] 4.7.3.1 | cURL error 28: Operation timed out after 15001 milliseconds with 0 bytes received ((unknown URL) [500])

Before you ask:

– This happens randomly, on various weekdays and hours.

– This happens across absolutely all sites and servers.

– All sites are running the latest versions of WordPress and all plugins/themes.

– Aside from this, the sites and dashboard work well.

– Automatic Updates and SSO are both disabled on your dashboard. There's no reason why your plugin should be trying to contact any server – including yours.

– This isn't something we can simply ignore. It has been going on for months – possibly more than a year – and has been increasingly interfering with our maintenance routines.

– We can't give you direct access to our sites/servers, but we will gladly answer any questions or send you any required data.

We would appreciate it if you could look into this ASAP.

Best regards,

Sergio S.

(on behalf of Matt Raad)

  • Adam Czajczyk
    • Support Gorilla

    Hello Matt

    I hope you’re fine, Sergio!

    The issue that you’re describing is actually quite common with WordPress sites and not really related directly to any specific plugin but rather to the server(s).

    There are two possible reasons that could possibly cause such or similar issue from our end but I’m pretty sure we might rule them out in this case. First one would be temporary API outages but there’s not that many of them. You would be able to “correlate” dates of these errors showing up in logs with API statuses listed on this page:

    https://status.wpmudev.com/

    The second one would be your server’s IP being banned on our end but that would mean pretty much all IPs being banned (as it affects different sites and servers) and would be permanent, not random. I think that the connection in such case would also be rejected immediately and not timed out.

    The majority of cases of such issues though would be caused by DNS resolving issues, some security modules preventing connections, outdated or broken CURL library or CURL library SSL certs on server or issues with “loopback” (host not allowing WordPress to connect to itself). Sometimes it’s also temporary connectivity issue that just “slows down” and needs less strict timeout configuration.

    It’d be best to start with “testing” following things (I understand that the issue needs solving but since it can be caused by many reasons and we don’t have a way to directly test/investigate on your server(s)/sites we’ll need to go slower route and try possible solutions):

    1) make sure (you might need to check with your host) that

    – PHP is relatively up to date (so > 7 version)

    – cURL is up to date (preferably > 7.6 if possible) and its SSL certs are valid and up to date

    2) try adding this code to your site’s “functions.php” file to increase WP http request timeout:

    // Setting a custom timeout value for cURL. Using a high value for priority to ensure the function runs after any other added to the same action hook.
    add_action('http_api_curl', 'sar_custom_curl_timeout', 9999, 1);
    function sar_custom_curl_timeout( $handle ){
    curl_setopt( $handle, CURLOPT_CONNECTTIMEOUT, 30 ); // 30 seconds. Too much for production, only for testing.
    curl_setopt( $handle, CURLOPT_TIMEOUT, 30 ); // 30 seconds. Too much for production, only for testing.
    }

    // Setting custom timeout for the HTTP request
    add_filter( 'http_request_timeout', 'sar_custom_http_request_timeout', 9999 );
    function sar_custom_http_request_timeout( $timeout_value ) {
    return 30; // 30 seconds. Too much for production, only for testing.
    }

    // Setting custom timeout in HTTP request args
    add_filter('http_request_args', 'sar_custom_http_request_args', 9999, 1);
    function sar_custom_http_request_args( $r ){
    $r['timeout'] = 30; // 30 seconds. Too much for production, only for testing.
    return $r;
    }

    3. make sure that all this IPs are white-listed in any firewalls/security tools on site and server:

    66.135.60.59
    66.135.49.214
    66.135.60.64
    165.227.66.214
    192.241.140.159
    104.236.132.222
    192.241.148.185
    159.89.254.12
    34.196.51.17
    52.57.5.20
    35.171.56.101
    45.55.78.242

    These are IPs of our APIs so it should be made sure that they are not blocked by anything, even temporarily.

    Give it a go please and let’s see if it helps. If not, we’ll then see what else could be done to solve this

    Kind regards,

    Adam

  • Matt
    • New Recruit

    Thanks for the quick reply, Adam. I disagree with some points, but let’s see if we can work together to solve this.

    I wouldn’t be so quick to rule out the API outages. Attached is a merged list of failure times from all sites and all servers. You’ll notice that at least two days and hours do match your potential downtime incidents.

    Now, that list may not seem much to you, but having to go to the site to confirm if anything did happen, only to find that exact same line time and time again, is really getting on my nerves.

    The list just isn’t longer because i had WPMU Dev disabled for a long while, since last year. At the time i decided to disable it for this exact same reason.

    I only re-enabled it a few weeks ago because i noticed we no longer had all Smush Pro features after an update – and the errors began again right away.

    1) On all servers:

    – PHP 7.2.20

    – curl 7.29.0

    2) Have you seen our site list? I could perhaps add that to 2 or 3 sites, but adding that to 30 distinct sites would be nuts. I’d have to do it manually or risk breaking something. Plus, i’d be doing this only to fix this issue with your plugin – currently no other plugin is causing this kind of issue.

    3) I could do that, but keep reading.

    Despite all i say above, i think you may have missed the point: Why does your plugin insist on contacting your servers? (Or whatever it is it’s trying to do.) And why “unknown URL”? This is very suspicious, and it’s definitely undesirable behaviour to us.

    Sergio S.

    (on behalf of Matt Raad)

  • Predrag Dubajic
    • Support

    Hi Sergio,

    Some of the values in the logs do match API outages times, but there are also others that don’t.

    And if this is happening often for you than we’re not dealing with API outages only.

    I would suggest checking the above IP list and make sure that none of them are blacklisted anywhere on your server or in security plugins.

    What you could also do is enable Dashboard debug log so when the issue happens again it should give us some more information about it.

    You can enable debug log in your wp-config.php file (located in root WP folder) by replacing define(‘WP_DEBUG’, false); with this code:

    // Enable WP_DEBUG mode
    define('WP_DEBUG', true);

    // Enable Debug logging to the /wp-content/debug.log file
    define('WP_DEBUG_LOG', true);

    // Disable display of errors and warnings
    define('WP_DEBUG_DISPLAY', false);
    @ini_set( 'log_errors', 1 );
    @ini_set( 'display_errors', 0 );

    //Enable WPMU DEV Dashboard debug
    define('WPMUDEV_API_DEBUG', true);
    define('WPMUDEV_API_DEBUG_ALL', true);

    Once the issue happens again it should create debug.log file in your wp-content folder and you can change the extension to .txt file and attach it here so we can check it out.

    Despite all i say above, i think you may have missed the point: Why does your plugin insist on contacting your servers? (Or whatever it is it’s trying to do.)

    WPMU DEV Dashboard will contact our servers in order to check account status so you can use premium features from plugins like Smush, and also to check for any available updates.

    Also, Smush has multiple features that need a connection to our servers, like image optimization which is performed on our servers and CDN.

    And why “unknown URL”? This is very suspicious, and it’s definitely undesirable behaviour to us.

    This is usually what is shown when the IPs are blocked or there are API outages.

    Best regards,

    Predrag

  • Matt
    • New Recruit

    Hi, Predrag,

    Since my last message i got a few more of those errors, but they seemed to stop after a while, and it has been weeks since the last occurrence. I see you’ve released updates. I’m guessing this was dealt with. I’m calling this case closed.

    Thanks for your help.

    Best regards,

    Sergio S.

    (on behalf of Matt Raad)

Thank NAME, for their help.

Let NAME know exactly why they deserved these points.

Gift a custom amount of points.