7006 pointsLike some sort of WPMU DEV God"Mindblowingly helpful memberLifetime member
Aaron
Lead Developer
—
11th June 2010
I have just made live some major updates to our API rulesets for initial signups. Our API filters have proven EXTREEMLY effective (>96%) at detecting sploggers when they post and shutting them down. But you have noticed and we acknowledge that our rules are not so effective at detecting splogs at signup before they post, in fact splogs sometimes don't even make it to the suspicious queue. This is due in large part to the limited amount of information there is to analyze at signup, and splogger's ability to manipulate most of this to avoid detection.
Thanks to many suggestions from you members and lots of experience and sluthing with the data we've collected so far I was able to tweak our rulesets and add a few bells and whistles, and they should be live right now. What this means is that hopefully the API will be able to detect a much larger percentage of splogs right at signup, and at the very least mark them as suspicious!
As always, I welcome any suggestions and feedback on splog patterns, especially any large swings you see on your site, like a sudden jump in splogs getting past or false positives.
To be clear you DO NOT need to update your Anti-Splog plugin to take advantage of this as the changes are to our servers and all new signups will get this stricter treatment!
Also a 3.0 compatibility update for the actual plugin is ready to put up live when WP 3.0 is released. You will need to upgrade at the same time you do to WP 3.0, as this release (along with most others) will not be backwards compatible with 2.9.2.
I have just made live some major updates to our API rulesets for initial signups. Our API filters have proven EXTREEMLY effective (>96%) at detecting sploggers when they post and shutting them down. But you have noticed and we acknowledge that our rules are not so effective at detecting splogs at signup before they post, in fact splogs sometimes don't even make it to the suspicious queue. This is due in large part to the limited amount of information there is to analyze at signup, and splogger's ability to manipulate most of this to avoid detection.
Thanks to many suggestions from you members and lots of experience and sluthing with the data we've collected so far I was able to tweak our rulesets and add a few bells and whistles, and they should be live right now. What this means is that hopefully the API will be able to detect a much larger percentage of splogs right at signup, and at the very least mark them as suspicious!
As always, I welcome any suggestions and feedback on splog patterns, especially any large swings you see on your site, like a sudden jump in splogs getting past or false positives.
To be clear you DO NOT need to update your Anti-Splog plugin to take advantage of this as the changes are to our servers and all new signups will get this stricter treatment!
Also a 3.0 compatibility update for the actual plugin is ready to put up live when WP 3.0 is released. You will need to upgrade at the same time you do to WP 3.0, as this release (along with most others) will not be backwards compatible with 2.9.2.
Aaron, any chance we could get a top 10 (or so) list of IP addresses that have been causing trouble? Say within the last 24 hours? Would be nice to just go ahead and block those if we want so they don't make it to the site.
Probably because it's the oldest and best known free webmail provider in the parts of the world that are so lacking in imagination that they have rely on spamming to earn a living.
Aaron, any chance we could get a top 10 (or so) list of IP addresses that have been causing trouble? Say within the last 24 hours?
Tracking multiple creation from identical IP's is a powerful idea.
I can think of almost no reason for the same IP to create blogs on more than one site. If there is a legitimate reason, then they can respond to the notification and talk to the site owner.
Question ... Is the notification system between the antisplog plugin and the api server dynamic, or done in "real time?" IP tracking and blocking would really be effective if it was done in that manner.
In the plugin you can limit the number of signups per day per IP. We recommend 1 or 2 for most sites.
As far as the API side, of course IP's are checked against our blacklists. Though unfortunatley from experience that is'nt as effective as you would think. Many sploggers use botnets of hijacked computers, so they seem to throw away an IP after using it 1 or 2 times.
Though unfortunately from experience that isn't as effective as you would think. Many sploggers use botnets of hijacked computers, so they seem to throw away an IP after using it 1 or 2 times.
Agreed which is why I suggested the 24 hours limit. From our logs, while a single IP address may be used only a few times against a single install, across 122 servers we see thousands of hits taking place.
And throwing away an IP address after a short run isn't as popular as one may think. A simple look at some IP addresses held by ev1/theplanet shows that years later, the bots are still going over there. (Granted though that's usually comment and forum spam.)
Though unfortunatley from experience that is'nt as effective as you would think. Many sploggers use botnets of hijacked computers, so they seem to throw away an IP after using it 1 or 2 times.
It would seem to me that the sploggers using botnets will use the same IP across multiple sites. That is where the API side can help. Any IP creating a blog on more than one site is almost certainly a spammer, else why would they need more than one blog, and how else would they find more than one site to blog on?
If it were my API, I would have each site report the IP as soon as a new blog was created. Then as soon as I got a second blog using that IP, I would send an immediate reply with a spamalert so that blog is spammed immediately, before activation. A few of those may make a big dent as my guess is their whole objective is to create the blog and let it sit as a user link. Posting later is just a big plus.
Waiting 24 hours for an update would not be a deterrent. By then, the splogger has moved on.
Responses (14)
Keeper of the Dark Chocolate — 11th June 2010 #
Aaron, any chance we could get a top 10 (or so) list of IP addresses that have been causing trouble? Say within the last 24 hours? Would be nice to just go ahead and block those if we want so they don't make it to the site.
Lead Developer — 11th June 2010 #
Perhaps next week. Most interesting is email domains. Worst offender... @hotmail.com
Member — 12th June 2010 #
Probably because it's the oldest and best known free webmail provider in the parts of the world that are so lacking in imagination that they have rely on spamming to earn a living.
Also, probably has the least security.
Member — 12th June 2010 #
+1 Great idea, drmike!
Member — 12th June 2010 #
Tracking multiple creation from identical IP's is a powerful idea.
I can think of almost no reason for the same IP to create blogs on more than one site. If there is a legitimate reason, then they can respond to the notification and talk to the site owner.
Question ... Is the notification system between the antisplog plugin and the api server dynamic, or done in "real time?" IP tracking and blocking would really be effective if it was done in that manner.
Lead Developer — 13th June 2010 #
In the plugin you can limit the number of signups per day per IP. We recommend 1 or 2 for most sites.
As far as the API side, of course IP's are checked against our blacklists. Though unfortunatley from experience that is'nt as effective as you would think. Many sploggers use botnets of hijacked computers, so they seem to throw away an IP after using it 1 or 2 times.
Keeper of the Dark Chocolate — 13th June 2010 #
Agreed which is why I suggested the 24 hours limit. From our logs, while a single IP address may be used only a few times against a single install, across 122 servers we see thousands of hits taking place.
And throwing away an IP address after a short run isn't as popular as one may think. A simple look at some IP addresses held by ev1/theplanet shows that years later, the bots are still going over there. (Granted though that's usually comment and forum spam.)
Member — 13th June 2010 #
It would seem to me that the sploggers using botnets will use the same IP across multiple sites. That is where the API side can help. Any IP creating a blog on more than one site is almost certainly a spammer, else why would they need more than one blog, and how else would they find more than one site to blog on?
If it were my API, I would have each site report the IP as soon as a new blog was created. Then as soon as I got a second blog using that IP, I would send an immediate reply with a spamalert so that blog is spammed immediately, before activation. A few of those may make a big dent as my guess is their whole objective is to create the blog and let it sit as a user link. Posting later is just a big plus.
Waiting 24 hours for an update would not be a deterrent. By then, the splogger has moved on.
Member — 13th June 2010 #
On my sites the worst offenders are .info email addresses
What about the possibility to add spell checking to custom profile fields in BP? (the bots load these up with jibberish).
Also, maybe an enhanced version of the email domain name blocker from the core files with tld blocking ie *.info or freeeemail.*
Thanks for this discussion.
Lead Developer — 13th June 2010 #
Already possible:
http://premium.wpmudev.org/forums/topic/banned-email-servers-list
Member — 14th June 2010 #
Thanks Aaron - Does BuddyPress access the banned email servers code you posted @ http://premium.wpmudev.org/forums/topic/banned-email-servers-list ?
Lead Developer — 14th June 2010 #
I've heard it doesn't check it, though I would hope they have fixed that hole by now.
http://buddypress.org/community/groups/how-to-and-troubleshooting/forum/topic/e-mail-domains-blacklist-doesnt-work/?topic_page=2&num=15
Keeper of the Dark Chocolate — 14th June 2010 #
Interesting. I don;t see a trac ticket on it:
http://trac.buddypress.org/search?q=banned
Could have sworn that there was one.
Member — 3rd October 2010 #
There's a workaround for it:
http://etivite.com/groups/buddypress/forum/topic/quick-tip-added-banned-email-domain-check-to-multisite-buddypress-registration/
Become a member