Anti-Splog 1.0 RC2 – Beta Testers Needed!

This is the one you all have been waiting for! The ultimate plugin to stop and kill splogs in WPMU.

We have been testing and tweaking this on Edublogs.org for more than a month, and it’s helped us kill more than 10,000 splogs already. While the plugin is very stable, it is such an advanced and powerful plugin that we wanted to give it a soft launch to iron any minor kinks and give our API server a chance to roll up. So test away! Time to clean up your WPMU site and make some sploggers really mad!

I recommend you read the documentation really well to make sure you understand how all the features work before running, and please give us feedback on both the plugin features and documentation on this thread. Have fun!

You can get your API key here: https://premium.wpmudev.org/wp-admin/profile.php?page=ustapi

________________________

The plugin works in 3 phases:

  1. Signup prevention – these measures are mainly to stop bots. User friendly error messages are shown to users if any of these prevent signup. They are all optional and include:
    • Limiting the number of signups per IP per 24 hours (this can slow down human spammers too if the site clientele supports it. Probably not edublogs though as it caters to schools which may need to make a large number of blogs from one IP)
    • Changing the signup page location every 24 hours – this is one of the most effective yet still user-friendly methods to stop bots dead.
    • Human tests – answering user defined questions, picking the cat pics, or recaptcha.

  2. The API – when signup is complete (email activated) and blog is first created, or when a user publishes a new post it will send all kinds of blog and signup info to our premium server where we will rate it based on our secret ever-tweaking logic. Our API will then return a splog Certainty number (0%-100%). If that number is greater than the sensitivity preference you set in the settings (80% default) then the blog gets spammed. Since the blog was actually created, it will show up in the site admin still (as spammed) so you can unspam later if there was a mistake (and our API will learn from that).
  3. The Moderation Queue – for existing blogs or blogs that get past other filters, the queue provides an ongoing way to monitor blogs and spam or flag them as valid (ignore) them more easily as they are updated with new posts. Also if a user tries to visit a blog that has been spammed, it will now show a user-friendly message and form to contact the admin for review if they think it was valid. The email contains links to be able to easily unspam or bring up the last posts. The entire queue is AJAX based so you can moderate blogs with incredible speed.
    • Suspected Blogs – this list pulls in any blogs that the plugin thinks may be splogs. It pulls in blogs that have a greater that 0% certainty as previously returned by our api, and those that contain at least 1 keyword in recent posts from the keyword list you define. The list attempts to bring the most suspected blogs to the top, ordered by # of keyword matches, then % splog certainty (as returned by the API), then finally by last updated. The list has a bunch of improvements for moderation, including last user id, last user ip, links to search for or spam any user and their blogs or blogs tied to an IP (be careful with that one!), ability to ignore (dismiss) valid blogs from the queue, and a list of recent posts and instant previews of their content or the entire blog without leaving the page (the most time saving feature of all!)
    • Recent Splogs – this is simply a list of all blogs that have been spammed on the site ever, in order of the time they were spammed. The idea here is that if you make a mistake you can come back here to undo. Also if a user complains that a valid blog was spammed, you can quickly pull it up here and see previews of the latest posts to confirm (normally you wouldn’t be able to see blog content at all).
    • Ignored Blogs – If a valid blog shows up in the suspect list, simply mark it as ignored to get it out of there. It will then show in the ignored list just in case you need to undo.

  • Aaron
    • CTO

    Everything is written to be compatible with BP with the exception of the auto-renaming wp-signup.php feature. I couldn’t do that with BP because it doesn’t use wp-signup.php and moving the /register page was not possible as it depends on template files that are different for every theme.

  • freeblogger
    • Site Builder, Child of Zeus

    When I the blog-suspended.php file to the mu directory it gets placed at the top of every where you go regardless of being spammed or in good standing.

    With the message

    Warning: session_start() [function.session-start]: Cannot send session cache limiter – headers already sent (output started at /home/politica/public_html/wp-content/mu-plugins/blog-suspended.php:72) in /home/politica/public_html/wp-content/plugins/si-captcha-for-wordpress/si-captcha.php on line 961

    I am running BuddyPress and using the Social 1.2.2 theme.

  • freeblogger
    • Site Builder, Child of Zeus

    Update, the error line has disappeared, I mean go figure two antispam type programs acting the same as two antivirus programs and conflicting one another…but it is still placing the spam message in the header of all the pages regardless your account status, even on the admin account, scroll down and your regular pages are there with full access because it isn’t blogged, so I guess it is just automatically attaching the file to the header regardless the actual account status.

  • bbrian017
    • Design Lord, Child of Thor

    I installed this module now I cannot access my admin panel. It’s displaying a blank white page.

    Actually my entire website is down and showing a blank white screen.

    Any suggestions?

    EDIT: I deleted the folder of this plugin and the site is back up.

  • Aaron
    • CTO

    @bbrian017

    Double check that you installed everything in the right place according to the install.txt file.

    Files/Folders:


    anti-splog.php – goes in /wp-content/mu-plugins

    /anti-splog/ – entire folder and contents goes in /wp-content/mu-plugins

    blog-suspended.php – goes in /wp-content – highly recommended but optional

    If you are sure you had them in the right place, please let me know what error message was in your log. Thanks.

  • bbrian017
    • Design Lord, Child of Thor

    Hi Aaron,

    After adding the files into the correct location the plugin worked perfectly.

    I just added my key and now I’m about to see how it actually works.

    Thanks for this. It’s good to get this stuff installed before the spammers come. I kept my url pretty quit so they have a harder time finding me lol.

  • Aaron
    • CTO

    @stef1978

    I’ve checked it about 6 or 7 times and I’m quite sure the answers were all correct.

    Just want to make sure you understand I am not talking about entering the captcha words, but the public and private key that you have to register for on the settings page. If you have a correct pub key but wrong private key it could cause that problem.

  • jcnjr
    • HummingBird

    Impressive. I’ve just installed it on our testing grounds with no major glitches. But we never get splog signups there, so I will install on our live sites ASAP and report back.

    Only one question regarding documentation so far…

    Within post or page content you can insert the [ust_wpsignup_url] shortcode…

    Need some clarification on this. We’ve got links in page, blog post, and forum post content all over the place. Currently these links take the typical form of opening/closing tags wrapping link text, like:

    <a href="http://.../wp-pungis.php">sign up link text</a>

    What exactly would get replaced by the shortcode? Just the page url itself? the entire url? The page filename only? Perhaps an example of how a link to the signup page should appear would be helpful, both here and in the docs. This goes for the php link too as we have also edited template files.

    Thanks again.

  • Aaron
    • CTO

    What exactly would get replaced by the shortcode? Just the page url itself? the entire url? The page filename only? Perhaps an example of how a link to the signup page should appear would be helpful, both here and in the docs. This goes for the php link too as we have also edited template files.

    First off, it will use the the_content filter to replace the string “wp-signup.php” with the current “signup-xxx/” so you shouldn’t have to worry about old posts/pages. I will add that to the documentation.

    The shortcode and function only prints the full url, so you will have to put it in a <a> tag.

    Examples:

    Posts/Pages:

    <a href="[ust_wpsignup_url]">sign up link text</a>

    In a theme template file:

    <a href="<?php ust_wpsignup_url(); ?>">sign up link text</a>

    Should I put that in the install.txt or on the settings page?

  • jcnjr
    • HummingBird

    First off, it will use the the_content filter to replace the string “wp-signup.php” with the current “signup-xxx/” so you shouldn’t have to worry about old posts/pages.

    So, any recommendations for those of us who have already renamed the signup page, and replaced all links to it? Rename all the links back to wp-signup or change them to the PHP or shortcode?

    Does the_content filter convert URLs in all theme files too? Like the footer links? If so, cool!

    Thanks for the examples. The install.txt file should suffice for the instructions, as everyone should be reading that first anyway. though the Help tab in the plugin might be a good place to add helpful details like this. Right now it doesn’t appear to include much other than what is available on the plugin download page.

    Finally … a bit more preliminary feedback. The plugin apparently does a good job at blocking suspected spam blog posts. But what would it take to do the same thing to block suspected spam blogs from even being created? If it could throw the “friendly” error message upon signup for any blog including keywords from the list in their name or title, that would be just dandy.

  • Aaron
    • CTO

    The plugin apparently does a good job at blocking suspected spam blog posts. But what would it take to do the same thing to block suspected spam blogs from even being created? If it could throw the “friendly” error message upon signup for any blog including keywords from the list in their name or title, that would be just dandy.

    If you look at the chart it does do that. It checks the API immediately on blog signup as well as when user’s post. The real beauty of this thing is the Anti-Splog API service. Essentially we just launched an Akismet for splogs. It is the first plugin that crowdsources splog info from multiple WPMU sites. And the more people that use it the more accurate it gets!

  • georgef
    • The Incredible Code Injector

    I’ve added my server and site according to the instructions, but still end up with this message:

    There was a problem with the API key you entered: “Your server IP and WPMU Site Domain is not yet registered for this API Key” Fix it here»

    I’ve also added the mysql and added the table in db-config.

  • Aaron
    • CTO

    Hi George,

    Are you sure you have the right IP address registered for your server?

    Do you have a dedicated IP? I’m thinking it is possible that your host uses a NAT or proxy for external connections so it may be looking like a different server to us.

    If that is the case please try checking the API key again, as I have updated the error message to display the actual IP you are attempting to connect from.

  • jcnjr
    • HummingBird

    It checks the API immediately on blog signup as well as when user’s post.

    OK, I’ve got this installed on both our live sites and have been doing some testing. Just confirmed confirmed creation of a splog with flagged keywords used in the blog name…

    1. Added “exback” to the keywords list and updated settings
    2. Signed up for new blog at getmyexback.domain.com
    3. Received activation email and successfully activated splog

    I was presented with no friendly error messages during this process, and the splog does not appear in suspected blogs list.

    A good number of other existing blogs were flagged as suspected, so I just “ignored” those. Your Post Preview function is pretty slick! It enabled me to easily identify a couple valid blog posts including the words “sluggish” and “UGGH!” The only thing I see missing from the admin interface is a link to visit the actual blogs.

    Question regarding keyword matching: the tips say…

    Keywords are not case sensitive and may match any part of a word.

    Will multi-word phrases entered match partial words? For instance, if I enter the phrase “car insurance” as a keyword, will it flag blog posts including “car” OR “insurance” as spam? Would “make money online” flag those including only “money”. Will “ex wife” match “ex”? Hope I’m making sense…

  • georgef
    • The Incredible Code Injector

    Hey Aaron,

    Thanks that fixed it. Strange as the IP it was registering is the IP I’ve been using on everything and it’s also in the Domain Mapping plugin.

    All good, I have the green light so let me test further!

    Thanks,

    George

  • Aaron
    • CTO

    I was presented with no friendly error messages during this process, and the splog does not appear in suspected blogs list.

    I think you are misunderstanding the functions of this plugin. The easiest way to describe it is that this is an “Akismet” for splogs. All the really valuable checking goes on in our servers. All the keyword finding etc is done on our servers so you don’t have to worry about it. You only need to keep an eye on the queue for blogs that are either false alarms or missed, marking them ignore or spam, which will teach our service to be more accurate. Our preliminary tests show >80% effictive rate at identifying spam, and <3% false alarms. As an added bonus we threw in some extra protections on the front, and an advanced moderation queue on the back.

    The keyword list you are referring to is designed to work in conjunction with our Post Indexer plugin to help you find old and inactive splogs that the API service would no longer catch. It is not for any ongoing moderation or spamming, I just added it to help us at edublogs dig up thousands of old splogs. As the directions on it say you should only search a few keywords at a time while you are trying to find older splogs. On edublogs with the massive number of posts we have using any more than 2 keywords at a time slows the db down so much the pages time out. It has nothing to do with protecting signup.

    The only thing I see missing from the admin interface is a link to visit the actual blogs.

    Just click the domain, in the queue and you can preview the entire blog in a thickbox without leaving the page!

    Will multi-word phrases entered match partial words? For instance, if I enter the phrase “car insurance” as a keyword, will it flag blog posts including “car” OR “insurance” as spam?

    A phrase would have to be an exact match. Seperate keyword/phrases with line breaks. Example “Ugg” would match “suggestion”.

  • jcnjr
    • HummingBird

    All the really valuable checking goes on in our servers.

    Cool, thanks for the clarification. Guess I should have spammed my test “getmyexback” splog to help train the API before deleting it! But then, since it didn’t appear anywhere in the anti-splog queue, how could I?

  • Aaron
    • CTO

    It hooks into the normal blogs page too. No matter where a blog is spammed/unspammed we will learn from that.

    Guess I should have spammed my test “getmyexback” splog to help train the API before deleting it!

    Please don’t spam test blogs. The way to test the system is to watch it and play with it over normal signups. Our system is 100 times more complex than just a keyword in the domain. We analyze more than 15 separate factors of a blog to determine spammyness.

  • drmike
    • DEV MAN’s Mascot

    I just added it to help us at edublogs dig up thousands of old splogs.

    Just a quick mention this is working out very well. I did one of my keywords searches that I do every so often when we notice that spammers have come up with another one. Google’ing for that term returned about 3900 hits against edublogs. I gave up after about 1800 in but this is all that weren’t caught:

    http://edublogs.org/forums/topic.php?id=6881&page=2#post-27049

    Granted that’s still a fair amount and the returned hits had dupes but all the rest were already marked. (Had two false positives in case anyone is wondering in what I pulled up in Google and you can see the questionable ones I;ve marked in that list.)

  • Nemego
    • Flash Drive

    question, will this work on a wpmu adult blog site? because the API you are reffering will be the one incharge for marking possible splog, and I’m afraid that API has a strict rules when it comes on word names for the blogs. thank you

  • Aaron
    • CTO

    @georgef

    That’s pretty normal. When they start posting it would likely detect them. If you know they are spam then spam them and it will learn. Note that right now we have a limited set of data only from edublogs until it learns from all you users.

    Just curious, did they show up in the suspect queue at all?

  • drmike
    • DEV MAN’s Mascot

    Hmmm, that runs off the post indexer? That’s going to be a problem for us as we’re using Donncha’s solution.

    Wasn’t complaining about the dupes. Just saying that they were in there.

    I did noticed that Google still has hundreds of already deleted blogs in edublogs. Maybe sending out a ping to pingomatic for the recently spammed site in a hopeful attempt to get Google to notice that the site is now marked as “nofollow,noindex” would be helpful.

  • Andrew
    • Champion of Loops

    We discovered our post indexer was down for a year so we had a large gap we couldn’t search for to find inactive splogs.

    Basically we had it running on Edublogs but weren’t using it for anything so we didn’t realize it wasn’t indexing properly until we needed it for the Anti-Splog plugin.

    Hmmm, that runs off the post indexer? That’s going to be a problem for us as we’re using Donncha’s solution.

    You can always toss the post indexer in alongside that solution. We’re going to be releasing plugins that rely on the post indexer throughout the year as well.

    Thanks,

    Andrew

  • drmike
    • DEV MAN’s Mascot

    You can always toss the post indexer in alongside that solution.

    We do that on one install per a request of the client. Something in the back of my mind says there’s an issue with it but I’d have to go check my notes (which I don’t have today) as to what it was. I forget why we do it too.

    edit: I remember. They’re the ones with the mini and we worked out a method to index direct from that since Donncha’s solution didn’t update when blogs went back and forth from private to public. (And back and forth and back and forth and…:wink:

    The problem was folks complained about slow publishing since it was saving the data twice and notifying the mini on top of that when a new post was made.

    Probably should just convert from Donncha’s over to the one here. Did anyone ever work out a conversation script? I don;t recall one.

  • Aaron
    • CTO

    Most of the detection goes on when members post. At signup we only have headers and IP to work with, so unless another member has encountered that spammer you will have to wait for them to post.

    Note right now we are largely in a learning phase as 99% of the data is from edublogs so the analysis is natrually leaning toward the kind of post content we see on edublogs. We are getting greater than 80% accuracy on flagging posts there but I don’t expect it to be that accurate on other sites till we have more data. If you are getting too many false alarms you can adjust the auto spam sensitivity to a level you are satisfied with.

  • georgef
    • The Incredible Code Injector

    Hi Aaron,

    I guess I’m not helping the system learn then.

    I try and delete all splogs possible before a post is submitted.

    If I add a blog as spam though through buddypress admin options (user bar), will that get noted in the anti splog plugin?

  • Aaron
    • CTO

    I try and delete all splogs possible before a post is submitted.

    Thats fine if you know they are spam.

    If I add a blog as spam though through buddypress admin options (user bar), will that get noted in the anti splog plugin?

    It should. I’ll have to double check though.

  • georgef
    • The Incredible Code Injector

    If I add a blog as spam though through buddypress admin options (user bar), will that get noted in the anti splog plugin?

    It should. I’ll have to double check though.

    If it is, then it should appear in the suspected, recent or ignored blogs / splogs then?

    If so, nothing is being recorded on my system.

  • Aaron
    • CTO

    But must we spam them first for the API to learn?

    Yes, you need to mark them as spam for us to learn before you delete. It’s really not best practice to delete right away in case you make a mistake, which is very common. This plugin creates a way for spammed blog owners to dispute the status.

    You can always go through and delete blogs that have been spammed for a while every now and then.

  • jcnjr
    • HummingBird

    …not best practice to delete right away in case you make a mistake…

    Understood, though with our niche community, it is very obvious when we get a splog and our members fighting cancer in their dogs hate seeing stretch mark creams and tips for making money online in our recent global posts.

    I’ll be sure to spam them first now and delete them periodically, unless the posts still show in the widgets. Thanks again!

    Update: So far so good… no splogs since installation! Just curious though – do we get notified in any way of splogs that were denied creation altogether? We would get a few a day, perhaps the Asirra cats are chasing them away.

  • Aaron
    • CTO

    hate seeing stretch mark creams and tips for making money online in our recent global posts.

    If you are using our post indexer for your global posts they will get removed when you spam them. Not so with Donncha’s global tags plugin from what I recall.

    do we get notified in any way of splogs that were denied creation altogether?

    No, and i’m not sure that you would want to anyway. Seems to me like a better thing for google analytics or similar. It is possible if enough people ask for it though. Anything more than just a count that you could reset though would necessitate a new db table, and I don’t want to go there.

  • jcnjr
    • HummingBird

    It is possible if enough people ask for it though.

    Not necessary in my opinion, I was just curious if any were getting blocked.

    Update: It just caught our first splog… I was happy to see makemoneyonline.tripawds.com in our Recent Splogs list, and I’m hoping its just a matter of time before obvious blog names like this get blocked from even being created at all.

    Thanks again!

  • argh2xxx
    • The Incredible Code Injector

    I think I should create a brand new thread for the problem that I’m about to describe, but it’s related to anti-splog plugin anyway. So here it is. I tested with the latest wpmu (2.9.1) and latest buddypress (1.1.3) and found antisplog plugin to be problematic on two occasions.

    1) When using this plugin with asiracat method, the signup page when clicking finish, I got an empty page of http://yourdomain.org/wpmu/register.

    2) When using this plugin with recaptcha method, the answers to the recaptcha were always wrong, because I got redirected back to the sign up page no matter what and have to refill in passwords and recaptcha over and over again.

    The only working method is when I use this plugin with question format.

    When testing this plugin with wpmu (without buddypress), the asira cat works! (Didn’t test recaptcha with wpmu that goes without buddypress).

  • jcnjr
    • HummingBird

    FYI: Just had a heathnut splog register and post, but it did appear in the Suspected Splogs queue as expected. So, I clicked spam, and it disappeared from the queue but did not appear in the Recent Splogs list, and it was still live. Only after I spammed it from wpmu-blogs, did it then show up in the Recent Splogs list and present the friendly notice when visited.

  • Aaron
    • CTO

    I got an empty page of http://yourdomain.org/wpmu/register.

    Are you running wpmu in a subfolder?

    the answers to the recaptcha were always wrong

    Did you double check your recaptcha private key? I need to add a more error friendly message when the key is invalid rather than have it appear the same as wrong words.

    @jcnr:

    So, I clicked spam, and it disappeared from the queue but did not appear in the Recent Splogs list

    I know it’s too late to recreate, but if it happens again can you check if you refresh the suspect queue it is still in there? Also if it happens again let me know if there are any javascript errors, because that action is run via AJAX. Also note that the spamming can take a few seconds do to the API call, so even if it disappears when you click, it is still working in the background.

  • jcnjr
    • HummingBird

    …issues with the ASIRRA divs.

    I understand this may be theme related, in which case I’m OK dealing with it on my own. I notice, however, this interesting issue on line #907 of anti-slpog.php …

    echo '<div id="asirraError"></div>';

    The class “asirraError” is not, however, identified in the sylesheet. But there is the style “AsirraDiv” which is not called anywhere in the file. Changing the div id to the latter does not affect the widget’s placement and disrupts its style so that didn’t help. Any thoughts?

    Feel free to tell me “deal with it” and I’ll just use reCaptcha. But our community would prefer the pet-centric option!

    Thanks again for your time.

  • georgef
    • The Incredible Code Injector

    Hey Aaron,

    So I’ve haven’t tagged a few blogs in the system which I know are splogs to see if the anti-splog works.

    I see that a few blogs have been blocked as soon as they submit a post which is awesome.

    Only problem is nothing is being tracked within my system.

    There is not one splog listed in my log.

    Any idea why it wouldn’t track?

    Thanks,

    George

  • jcnjr
    • HummingBird

    …if it happens again can you check if you refresh the suspect queue it is still in there? Also if it happens again let me know if there are any javascript errors…

    OK… New splog created and spam post published at housebreakingadog.tripawds.com. Clicked Spam and it disappeared from Suspect queue. Visited recent Splogs and it was not there. Returned to Suspect list and it was there again.

    Clicked Spam again and it disappeared, waited ten seconds, refreshed page and it was gone. Visited Recent Splogs and it was now listed. No js errors during this process.

    New Question: What will “Un-ignore” do in the Ignored Blogs list? Can we use that to safely clear out that list, or should we leave all valid blogs as Ingored?

  • Aaron
    • CTO

    waited ten seconds, refreshed page and it was gone. Visited Recent Splogs and it was now listed.

    The spamming is done asynchronously, so it may take a few seconds (minutes when spamming a large number at once) for them to actually show in the recent list.

    New Question: What will “Un-ignore” do in the Ignored Blogs list? Can we use that to safely clear out that list, or should we leave all valid blogs as Ingored?

    You should leave valid blogs there. It is just there in case you need to undo a spam or possible spam blog you accidentally ignored. Un-ignoring will make it show up back in the suspect queue.

  • georgef
    • The Incredible Code Injector

    Are none of them updated?

    Not sure where to check, but according to the screenshot it doesn’t seem like it.

    Also is the table in the global db?

    No, wp_ust is not in the global db, do I need to run the Sql there as well?

  • Aaron
    • CTO

    If it’s not in the global db it may be inaccesible. If I recall your db is a little disorganized :wink: Adding the table name to db-config before installing should make it get added to the global db. I would try running the sql on your global db too.

    Anything in your error logs?

  • jcnjr
    • HummingBird

    New Scenario:

    Splog registered but no post published yet (Only "Hello World":wink: … Easily identified as Splog by the title "Dog Training – Products – Books And More".

    Does not appear in Suspect Splog list. This is understandable since there is not enough data yet for API to identify it as a Splog.

    Clicked "Spam this blog" link in New Blog Registration email. (Nice feature!). MU confirmation alert page, however, has only logo and button with no warning text. See attached. Not a biggie, but unexpected since URL includes "msg=You+are+about+to+mark+the+blog+Dog+Training+-+Products+-+Books+And+More+as+spam".

    Clicked confirm and the same confirm page just refreshed with URL truncated to: "…//domain.com/wp-admin/wpmu-edit.php?action=confirm& "

    Clicked confirm again and same empty confirm page refreshed with URL updated to, "…//tripawds.com/wp-admin/wpmu-edit.php?action=confirm&&updated=true"

    Clicked confirm again with same exact result.

    Visited Anti-Splog admin and blog does not appear in Recent Splogs list.

    Visited wpmu-blogs and blog is not marked as spam. Marked blog as spam from wpmu-blogs and was presented with proper confirm page including message, "You are about to mark the blog …//sbrs.tripawds.com as spam"

    Clicked confirm and was redirected to wpmu-blogs where blog was marked as spam. Returned to Anti-Splog where blog now appears in Recent Splogs list.

    Hope this helps.

  • drmike
    • DEV MAN’s Mascot

    A end user reporting system would be a blessing. :slight_smile:

    At the very least, since we all have different systems, the ability to echo out a link (ie report_splog($blogurl)) which leads to a form where a report can be made and dropped into some sort of queue.

    Else you get people like me who leave long lists in the forums.

    edit: Can we get some sort of popular by spammer IP addresses to be kicked out? That way we can go in and block via firewall or htaccess please?

Thank NAME, for their help.

Let NAME know exactly why they deserved these points.

Gift a custom amount of points.