PROTECTING WORDPRESS SITE AGAINST WEB SCRAPING/STRIPPING

Hi,
What is the best way to protect a wordpress site against web scraping/web stripping?
I have read this
https://premium.wpmudev.org/forums/topic/protecting-wordpress-site-against-web-scraping-stripping#post-1369073
and have further questions:
1. Can we use Defender Pro to temporarly or permanently lockout IPs who try or do content scraping?
Is it possible to prepare honeypots and lockout them this way somehow?
2. How about hosting media files themselves somewhere else? How would that work?
Thanks.

  • Adam Czajczyk

    Hi Robert

    I hope you're well today and thank you for your questions!

    1. Can we use Defender Pro to temporarly or permanently lockout IPs who try or do content scraping?
    Is it possible to prepare honeypots and lockout them this way somehow?

    Yes, you can use Defender to lockout IPs. There are already tools for this built-in in the plugin that could automatically lockout IPs e.g. for failed login attempts or too many 404 requests. But you can also specify a list of the IPs to be blocked. There's currently no automatic detection of "scrapping" but you could e.g. block some known IPs or IPs detected/identified other ways.

    If you had some sort of a script that would detect such "scrapping" it should also be possible to make it automatically add detected IPs to Defender blacklist with a bit of custom code.

    2. How about hosting media files themselves somewhere else? How would that work?

    I think that depends on the way they are handled. The point is that as long as the file is publicly accessible then if it's referenced anywhere on the site, it can be "scrapped". But offloading media might help a bit as CDN's usually do have some additional security layers applied so might be able to "filter" at least some of the bots and automated traffic, I think :slight_smile:

    Best regards,
    Adam

Thank NAME, for their help.

Let NAME know exactly why they deserved these points.

Gift a custom amount of points.