My server got restricted because a robot.txt

Hi Guys I hope you are doing great! I have my sites with hostgator, and I got an email saying they restricted access to my sites, my sites are very lite, most of them are about 60MB, don't have much traffic and I mainly have your plugins and themes. Around 2 weeks ago I deleted all plugins that I was not using, installed defender and did almost all the recomendations that the plugin gave me... can anyone one please guide me...? I really appreciate it (the following is the email)

Hello,

I apologize, but I was forced to suspend the reseller account and all sub-accounts of said reseller as multiple scripts inside were causing high loading issues on the server. Due to it affecting all of the other accounts on the system, we had to take immediate action for the health of the server.

Please be aware, we do not normally disable accounts in mass except in rare cases where multiple scripts are causing issues. We believe this is the only way to prevent a server crash and potential loss of data.

Generally this is caused by a bot or bot(s) causing intense traffic across multiple sites within this directory (in which case you may be able to create a robots.txt to affect this, or if caused by a site-map system, slow down how many requests are made), or you simply have far too many CPU intensive sites under this same account.

Here are a few links that can help you understand Robots.txt files as well as create one;

http://www.robotstxt.org/
http://www.mcanerin.com/en/search-engine/robots-txt.asp

Aside from the above robots information we do not have any specific recommendations for these scripts, however, in general, adding some sort of caching mechanism, where the script does not need to generate a new page with every request, helps to lower the over load that a script will cause. Likely the original author or support group of the software that you are using will be able to help you to understand how to add something of this nature.

If you reply back to this with your IP address (http://www.hostgator.com/ip) we will be more than happy to go ahead enable HTTP access for you, so that you can safely work on the script without it causing further issues.

Please let us know how you would like to proceed.

servidin cirugiap 0.7 0.2 /usr/bin/php /home/cirugiap/public_html/index.php
servidin cirugiap 0.7 0.2 /usr/bin/php /home/cirugiap/public_html/index.php
servidin essenzas 1.1 0.0 [php] <defunct>
servidin essenzas 1.7 0.4 /usr/bin/php /home/essenzas/public_html/wp-cron.php
servidin expertso 0.7 0.2 /usr/bin/php /home/expertso/public_html/index.php
servidin ginecol 0.6 0.2 /usr/bin/php /home/ginecol/public_html/index.php
servidin intdeneg 0.6 0.3 /usr/bin/php /home/intdeneg/public_html/index.php
servidin madelat 0.7 0.2 /usr/bin/php /home/madelat/public_html/index.php
servidin mallafo 0.2 0.3 /usr/bin/php /home/mallafo/public_html/index.php
servidin ortocol 1.1 0.3 /usr/bin/php /home/ortocol/public_html/index.php
servidin ortopedi 2.5 0.3 /usr/bin/php /home/ortopedi/public_html/index.php
servidin ortopedi 3.3 0.1 /usr/bin/php /home/ortopedi/public_html/wp-cron.php
servidin pizzatw 1.4 0.4 /usr/bin/php /home/pizzatw/public_html/index.php
servidin proctcol 2.3 0.4 /usr/bin/php /home/proctcol/public_html/index.php
servidin proctcol 4.4 0.3 /usr/bin/php /home/proctcol/public_html/wp-cron.php
servidin tescolim 1.5 0.3 /usr/bin/php /home/tescolim/public_html/index.php

Sincerely,
Kelly Y.
Linux Systems Administrator
Systems Monitoring

  • Milan

    Hello BePresence

    Hope you are well today and thanks for asking us. :slight_smile:

    First of all let's break down your question like this,

    #1) Your sites on server got restricted, and
    #2) Its not because of robots.txt file

    What she is trying to say here is that, due to scripts of some plugins, your site is creating huge load on their servers. And I guess you are on shared hosting plan right ?

    This is indeed very complex issue to debug as we do not have any specific information here. Information we got from her is very generalised. If its really because of scripts of plugins then first disable all the plugins or delete if you are having backup of your site, and request her to re-enable access so you can test is it really the plugins which are causing this or something else. ?

    Honestly this is something more related to servers and best person to get assistance on this is their support staff. Please do contact them, explain to them and grant them permission to check your sites in detail. Hopefully they will bring solution for you very quickly. :slight_smile:

    Cheers,
    Milan

  • BePresence

    Hi Milan I already contacted them and they keep saying that it is because of my plugins,, the only plugins I am using are from you guys and I never had that problem before. Now they want me to install WP-Super-Cache Plugin, and deactivate other, they also told me to deactivate wp-cron...

    Is there a plugin from you guys that is resource intensive?

    Thank you very much

    BTW I have a Reseller Account with them and I have only had trouble :slight_frown:

  • Alex Stine

    Hello,
    Hope you are well.

    I have a few questions for you.
    1. How many sites do you have?
    2. About how many plugins do you have on each site?
    3. Could you please list the plugins from WPMU DEV that are used on those sites?

    Let me explain wp-cron to you. Whenever a visitor loads a post or a page, wp-cron.php is called. This checks to make sure no tasks need to be executed. If it needs to execute a task, it will, if not, the check is returned and nothing happens. Since that check is performed each time one of your sites posts/pages are loaded, this could put some great strain on the server causing the monitoring staff to get notified and shut things down on your account. They only do this to make sure the server does not crash, and to make sure no one else is effected by this load.

    I think this article from HostGator will help you out as far as what you need to do to eliminate the wp-cron from causing you anymore problems.
    http://support.hostgator.com/articles/specialized-help/technical/wordpress/how-to-replace-wordpress-cron-with-a-real-cron-job
    Keep in mind that some plugins might not function correctly if they see that the default WordPress cron is disabled, you don't have to disable it for every site if that is the case.

    I myself have had this experience with HostGator and dealing with their tech support is not fun. Especially about an issue like this. I personally recommend SiteGround. They have awesome shared hosting environments and their support is the best you will find. They promise 10 minute response times and always follow up to make sure everything has been fixed.
    https://siteground.com/

    Hope this helps.

    Cheers,
    Alex :smiley:

  • BePresence

    Hi Alex, thank you very much for your response...

    I have around 40 sites, from those around 30 are fixer theme (upfront) one page sites and have hummingbird, seo, defender, dashboard, smush and maps (all from wpmudev) I also have formidabble forms and wpclone. That's it!

    You can see an example here: http://ginecologaencolima.com/

    Thanks,, I visited siteground, but there's no chat option there to answer some of my questions, looks good though thanks

Thank NAME, for their help.

Let NAME know exactly why they deserved these points.

Gift a custom amount of points.