BuddyPress is Crashing our entire server

Support access is ACTIVE until September 28, 2015 9:50 am
Regards http://www.filemakerprogurus.com

We've been trying to find out why our dedicated server crashes (more than a year now) so we have a Linux Ninja friend involved to try and help find the PID that would throw the server into chaos. So he found out that BuddyPress is writing out over a BILLION records to a temp file...

He makes incorrect assumptions that I want to clear up with you, as I've cleared up with him:
http://www.filemakerprogurus.com is a hyperactive site - have to be careful
http://www.fmlovers.org is the core URL that the dedicated server has everything else sitting on...
fmdataba_wor2 = a Buddy Press install that my partner believes he disabled long ago... that is running amok, running some phantom Cron Job, usually at night and clobbering us into a Walk to the actual Server to Press the damn reset button because Graceful Reboots and L3 Admin access can't even get the machine out of its loop.

Here is as much as he has right now, but since you guys are super BP familiar, you might have already come across this... read on:

So, in checking the tables, they're decent size:
bup_terms - 1306
bup_term_relationships - 6414
bup_term_taxonomy - 1306

That ends up making a new, tmp table of 10 billion records... definitely our culprit. It's happening on the fmdataba_wor2 database. The fmlovers.org site isn't being used anymore, right? Could we just archive this database and delete it?

---TRANSACTION 0, not started
MySQL thread id 16024, OS thread handle 0x7f55e2a92700, query id 904348 localhost fmdataba_wor2 Copying to tmp table
SELECT t.*, tt.*, tr.object_id FROM bup_terms AS t INNER JOIN bup_term_taxonomy AS tt ON tt.term_id = t.term_id INNER JOIN bup_term_relationships AS tr ON tr.term_taxonomy_id = tt.term_taxonomy_id WHERE tt.taxonomy IN ('category', 'post_tag', 'post_format') AND tr.object_id IN (13970) ORDER BY t.name ASC

MySQL thread id 16017, OS thread handle 0x7f55efcd2700, query id 904371 localhost fmdataba_wor2 Copying to tmp table
SELECT t.*, tt.*, tr.object_id FROM bup_terms AS t INNER JOIN bup_term_taxonomy AS tt ON tt.term_id = t.term_id INNER JOIN bup_term_relationships AS tr ON tr.term_taxonomy_id = tt.term_taxonomy_id WHERE tt.taxonomy IN ('category', 'post_tag', 'post_format') AND tr.object_id IN (14729) ORDER BY t.name ASC

We have Search by Category and Human Readable - not by Slug going on... in the Category Tag Cloud, which we have now disabled.

I'm add to this post, a few drop box links - one to a screen capture session with our Ninja and another link to a Dump File that is narrowed down to the cycles surrounding the crash.

https://www.dropbox.com/s/q8vxp1wxgjx0nkp/memlog%20excerpt.txt?dl=0
Dump File

https://www.dropbox.com/s/lungzxevl0kik6z/Summation%20of%20BUP%20problem.mp4?dl=0
Screen Capture Ninja Conversation

Thanks gang....
I hope you can help find out what BP or Widget is KILLING US...

Much Appreciated!!
R

  • Mike

    I had this happen with my own buddypress installation recently. What was happening was that spammers were getting into it and then going crazy. They were adding spam links but I think they were also using the email confirmation system to send out spam mails or just spam out all users with messages, I am not sure. This put massive strain on my VPS and my host shut it down.

    I opted to just remove buddypress for the time being so haven't tried to fix it but I would have a look at your user activity and look as users signing up, posting, changing status and sending messages.

    I'm fairly familiar with the buddypress code and I don't remember any parts that wrote to a temp file. I think the massive amount of terms are probably spammers exploiting the group system. Look at how many groups you have and then look into those groups and see what the content look like.

    You may want to install a mailcatcher also to see if you are firing out mails or else your ip may get blacklisted.

    Hope that helps

  • Michael Bissett

    Hey @RobertKwasny, Michael here! :slight_smile:

    I'm not seeing how it's BuddyPress related at the moment, given the errors you shared here, but it does sound somewhat similar to the error mentioned in this case over here:

    http://forum.ait-pro.com/forums/topic/wordpress-database-error-got-error-1-from-storage-engine/

    If the site isn't being used anymore, I'd want to remove it entirely (instead of just removing the database), that should do the trick here.

    Let me know if this helps please! :slight_smile:

    Kind Regards,
    Michael

Thank NAME, for their help.

Let NAME know exactly why they deserved these points.

Gift a custom amount of points.