I am wondering if

Hi,
I am wondering if Defender has any features to help alleviate 404 bot attacks? Specifically, over the last month I have noted on my multisite install with 100+ sub-domains that every sub-domain is getting 5K+ 404 requests per day that are looking for ridiculously formatted URLs. I have been creating server level redirects for the various patterns that I can identify to try to keep my server alive but I am at the point of throwing my hands up.

The IP addresses appear to be from Googlebot and Bingbot predominantly.

Thoughts or suggestions on how to alleviate this type of attack would be much appreciated.

Thanks,
Pat

  • wlpdrpat
    • Site Builder, Child of Zeus

    Thanks Luis!! I had to update the settings for 404 Lockouts in Network Admin > Defender > IP Lockouts > 404 Detection.

    I reduced the detection from 20 every 300 seconds to 5 every 300 seconds and selected to permanently ban.

    Hopefully that should avert the current attack. I will let you know.

    Thanks,
    Pat

  • wlpdrpat
    • Site Builder, Child of Zeus

    Before the changes my CPU and RAM were tapping out at 100%.

    Now my CPU is running at 50% and RAM at 40%...whew!!!

    Defender had less than 100 404 lockouts in the last week and the lockout only lasted for 5min. Since the change (10hrs) it has had more than 100 404 lockouts that are permanent.

    I also made a temporary change to my caching plugin to have the cached files not expire until I purge them. Between the two changes it appears to have tapered off the pressure the bots were applying to my server.

    I am hoping this will deter the damn bots from slamming my server but we will see.

    Thanks again!
    Pat

Thank NAME, for their help.

Let NAME know exactly why they deserved these points.

Gift a custom amount of points.