Server Error reaching pages

Hi all,

I receive the attached error messages after each run of SmartCrawl. The urls are changing randomly. Please help me to resolve this. The url errors are there respectively from the search engine WP settings.

Thanks,

Attila

  • Dimitris

    Hey there Attila,

    hope you're doing good and thanks for joining us! :slight_smile:

    First of all, I tried to replicate this using a test site in WPEngine and I was able to replicate some "inaccessible" pages errors, even after unchecking "Search Engine Visibility" option in Settings -> Reading admin page.

    I've already reported these errors to our devs and they will address them as soon as possible.

    I can also see in your screenshot, as well as visiting your website, that there're some HTTP-500 error in there, and this is where we should focus first.
    Could you please access your server via FTP, edit the wp-config.php file, find a line like
    define('WP_DEBUG', false);and replace it with the following (if the above line doesn’t exist, simply insert next snippet just above the /* That's all, stop editing! Happy blogging. */ comment)

    // Enable WP_DEBUG mode
    define('WP_DEBUG', true);
    // Enable Debug logging to the /wp-content/debug.log file
    define('WP_DEBUG_LOG', true);
    // Disable display of errors and warnings
    define('WP_DEBUG_DISPLAY', false);
    @ini_set('display_errors', 0);

    Then go ahead and visit your homepage to replicate the '500' error.
    By doing so, a /wp-content/debug.log file should be created.
    Simply download it, rename it to debug.txt and attach it here in your next reply. If file size exceeds the 5MB limit of our forums, please use a service like Dropbox of Google Chrome and post the shareable link instead.

    Warm regards,
    Dimitris

  • Adam Czajczyk

    Hello Attila!

    The change that Dimitris suggested should not cause site to become inaccessible unless there's been some error in "wp-config.php" file itself. On the contrary, that should help us troubleshoot the case.

    The site doesn't seem to be accessible right now so I assume the code suggested by Dimitris is still there, is that right? Could you please remove it and see if the site becomes accessible again?

    If it does, please add this line above the "/* That's all, stop editing...*/" line in "wp-config.php" file:

    define('WP_MEMORY_LIMIT', '256M');

    and see if that decreases the number of "500 Internal Server Errors".

    Kind regards,
    Adam

  • Adam Czajczyk

    Hello Attila!

    Not a problem at all, I'm glad you manged to sort that out and thank you for sharing debug.log.

    It is actually quite helpful. It shows that the site is generating too many database queries hitting the top limit allowed on the server:

    [25-Feb-2017 19:45:49 UTC] PHP Warning:  mysql_connect(): User 'XXXXX' has exceeded the 'max_user_connections' resource (current value: 6) in /var/www/XXXXXXXX/data/www/XXXXXXXXXXXXXX.com/wp-includes/wp-db.php on line 1568

    That means that the site is "eating up" too many resources. That is quite a common issue in case of cheap shared hosting accounts but I must say that so far I didn't came through it in case of WPEnginge's hosting yet.

    I have accessed your site but I don't see that many plugins there, though I belive some of them could be deactivated as there's no need to keep them active all the time (such as P3 Profiler or WordPress Health Check) and also enabling caching could help with that.

    Have you also tired to run P3 Profiler to see what it says about resource usage? I think it would be also worth to run "Query Monitor" plugin to see which plugin generate the most db queries on the site. You'll find the plugin here:

    https://wordpress.org/plugins/query-monitor/

    Knowing about the sources of those queries we could then think of the ways to minimize their number :slight_smile:

    Best regards,
    Adam

  • Adam Czajczyk

    Hello Attila!

    Don't worry about that, we are here 24/7/year so feel free to take as much time as you need. Well, actually it's not that each and everyone of us is working 24 hours a day but somebody is around so we can "jump in" anytime.

    But jokes apart (I hope you don't mind that), if you wish me to I may check that myself. The (very little) risk is that if I run e.g. P3 Profiler it can itself eat up some resources and temporarily take the site down so I won't be able to restore it via "support access" and will have to wait until it gets back automatically. It should not happen I think but I'd rather let you you know the potential risk and get your permission first :slight_smile:

    Let me know if you want me to review that. Otherwise I'll just wait for your feedback and results :slight_smile:

    Best regards,
    Adam

  • Adam Czajczyk

    Hello Attila!

    I checked the site and tested it a bit (though not without interruptions as it went down once for a minute or two) and I think it would be best to start with disabling following plugins:

    A) plugins that are handy only somtimes and they can be enabled if needed

    All in One WP Migration, Easy Theme and Plugins Upgrade (that one should rarely be necessary at all if you upgrade everything using WP updater as recommended)

    B) plugins that either do "nothing that should be done with well working WP" or that duplicate features:

    Autoptimize (duplicates with Hummingbird partially), WP-Optimalizalol

    C) after testing is done:

    P3 Profiler, Query Monitor

    That'd be first step and it won't significantly affect the case but should make it a bit better. Then there are three biggest "db query generators" and here it gets a bit tricky. It looks like the theme itself is causing a lot of queries and on top of it works Visual Composer that is known to be quite "heavy". The "tricky" part is that it may be very difficult to quit on these as the theme is I believe something that you selected for a reason and Visual Composer might be very handy sometimes. I would however suggest looking for a "lighter" theme similar to the one that you are using and if possible try to go for "custom developed" customization: building a custom child-theme following WP standards that would be customized mostly with custom page templates and custom CSS and wouldn't depend on VC (if possible). That would decrease number of DB queries very significantly.

    Then there's our own Hummingbird. It's generating less queries than those mentioned but still I think it usually is a bit less "resource hungry". Therefore, I'd start with what I suggested above, then I would disable Hummingbird for a while and see how the site performs and if that helped for those 500 errors and site getting down. If after a while it turns out that helped, we could then enable Hummingbird back and we could help you tweak it so it wouldn't have such an impact on the site and would in fact decrease the server load.

    Finally, it might be good to implement CDN in front of the site as it always serves a number of resources from its own on-board cache and that means that some requests never reach your server, thus they do not trigger WP response and that means even less db-queries.

    Best regards,
    Adam

  • Attila

    Hi Adam,

    Thanks for the extensive research. I shall do the easy part and see where we end up. Otherwise if I follow all of your suggestions I would need to start it more or less from scratch again :slight_frown:
    We shall see.
    Just one more question I'm using the same DB user also for the live and the test installation. Could the separation of these two users positively affect the situation?

    Regards,

    Attila

  • Dimitris

    Hello Attila,

    hope you're doing good today! :slight_smile:

    Just one more question I'm using the same DB user also for the live and the test installation. Could the separation of these two users positively affect the situation?

    Do you mean that you use the same credentials to login in both environments?
    If so, that means that both databases of live and test installations are having the same user credentials stored and this shouldn't affect this issue.
    On another note though, I noticed that in your website there's no WP Engine System plugin. This is a MU plugin used in all WP Engine setups in order to provide some caching control and some other goodies. Here's a screenshot from my test site:

    Have you deleted this MU plugin manually?
    If so, could you please contact WP Engine on how to retrieve this? You should then need no extra caching plugin like WP Super Cache.

    I'm lost a bit. I killed a lots of plugins (even required ones incl. VC). The number of SQL queries went down from 100+ to 65 and the issue is still the same.

    I think this is coming down to a full theme/plugin test as described in a nice flow chart here:

    https://premium.wpmudev.org/wp-content/uploads/2015/09/Support-Process-Support-Process.gif

    Could you please proceed with this test, checking each time your homepage for incocistencies and/or 500 error codes in Chrome's console?
    Let us know about your results!

    Warm regards,
    Dimitris

Thank NAME, for their help.

Let NAME know exactly why they deserved these points.

Gift a custom amount of points.