I/O Maxes Out and Snapshot Doesn't Complete

Okay guys...
Relatively new WMPUDEV member here and I was pretty excited to discover your products but I have to say I'm swiftly growing disenchanted. Hopefully you can help.

The general issue is that all the plugins seem to be extremely resource-heavy and I'm experiencing timeout issues and errors left and right. I'm on shared hosting, but it isn't cheap shared hosting. It's semi-dedicated, Lightspeed server (maybe that's the problem, but I do have the Lightspeed noabort mod in my .htaccess). It's running PHP 7.1 and with relatively generous limits. I've had or am having issues with the SEO Checkup timing out, Snapshot timing out or giving errors, and Defender timing out on multiple sites.

The specific issue I'm currently having is with Snapshot - I've defined the memory limit at 256M, I've defined the table chunk size and file chunk size at 10 (which I expected would slow down the process but didn't seem to do anything), I even deleted a couple plugins that I wasn't using, and still it completely maxes out my server's I/O usage (which has a max of 2.44MB), and jumps the physical memory usage to around 60% (which has a max of 2GB). It gets to 85% on the "Files" step of the log and then the I/O maxes out for a long while, after which it times out. In order to get it to stop freaking out the I/O, I have to cancel the backup, go view the Snapshots, click the one I just created, and delete it because otherwise it perpetually says "Uploading." Then I have to FTP into my server and manually delete the backup files in the "snapshots" directory, because apparently the plugin doesn't delete them when I delete the snapshot, and I discovered six backups in there from all the times I had been re-trying it.

The site I'm using it on isn't big and gets almost no traffic (none right now, it's after midnight here). I've tried Snapshot on a new multisite setup with only two sites and zero content, and it didn't even get that far - it got to 15% or so and then timed out, saying "Error, empty response from server." I'm using Snapshot on yet another site that has no content besides one single post and it's the only site where it is working, although it still takes way longer than it seems like it should. I'm trying to have the snapshot uploaded to Google Drive for this site, but on the multisite I tried your managed backups and it still timed out.

If you have any insight I'm all ears. At the moment I'm afraid to get too comfortable with it -- of all the plugins I want to be able to trust, the backup plugin is easily on the top of the list.

Many thanks, hopefully there's a solution!

  • Predrag Dubajic
    • Support

    Hi Greg,

    Sorry to hear you're having continued issues with plugins timing out, hopefully we'll be able to turn this around for you.

    If I remember correctly there was a separate thread for SEO scan issue, and this is something still being worked on, the scan rewrite is nearly finished and we're really hoping to see it live really soon.

    As for the Snapshot issue, this plugin can be more resource hungry depending on the setup of your site as well, how many files there are, if there are any larger files etc.

    Could you provide us with FTP or cPanel access to one of the sites where it's not working so we can perform couple of tests and try determining why it's requiring so much resources even if the sites are not that large?

    You can send us your details using our contact form https://premium.wpmudev.org/contact/#i-have-a-different-question and the template below:

    Subject: "Attn: Predrag Dubajic"
    - Site login url
    - WordPress admin username
    - WordPress admin password
    - FTP credentials (host/username/password)
    - cPanel credentials (host/username/password)
    - Link back to this thread for reference
    - Any other relevant urls/info

    Best regards,
    Predrag

  • Predrag Dubajic
    • Support

    Hi Greg,

    Nahid provided me with the login information from your chat and I was doing some testing on your site.

    It looks like the issue is specific to plugins being included in the backup, however when doing a backup of plugins only it does work, but it does take a lot of the time considering the size of the plugins folder.

    I also tried excluding Jetpack and W3 Total Cache from the backup as those are the plugins with most files but that still didn't help and backup failed while processing plugins.

    Unfortunately there's little to none information about what exactly is going on and what's causing the fail, so I will need to forward this to our developers to investigate it further and shed some light on this.

    Best regards,
    Predrag

  • Greg
    • Design Lord, Child of Thor

    Hi Predrag,

    I appreciate the response and I could try excluding all but the most essential plugins for the time being so that I can at least keep some kind of backup, but there are only like 20 or so plugins and probably five of them are yours - perhaps a lot but certainly not unusual, and of course many multisites may have upwards of 100 installed. The server limits all seem plenty high enough to handle any reasonable execution load of this nature, so if the size of the directory is the issue then for now I'll likely use one of the many other backup plugins that are available which don't have any trouble with it, and hopefully something can be done to make Snapshot more robust. I'd like to use it, if only for the sake of consolidation and the integration with things like Automate, but if it can't handle a site this size then there are very few sites I could use it on, aside from which I'd be even more nervous about using it on multiple sites for fear that it'd bring down my server with the resource usage.

    Please feel free to do further testing on the staging site but as I mentioned above, it's not the only issue I've had with your plugins over-taxing themselves to failure, so perhaps some kind of solution could be a priority in your roadmap, especially since it specifically says "guaranteed to work" on your plugins page, which would indicate to me that this kind of testing would have already been done.

    Thanks in any case, Predrag. I'll keep an eye on this ticket and if you do find some other issue please let me know, I'm happy to look around in cpanel if there are other settings that might need adjusting.

  • Adam Czajczyk
    • Support Gorilla

    Hello Greg!

    I'm sorry to hear about all that issues and I apologize for the troubles they're causing. Our developers are looking into it so I believe they'll be able to find a solution and that will also help them make plugin better.

    Please keep an eye on this thread and we'll update it for you as soon as we hear back from them.

    Best regards,
    Adam

  • Predrag Dubajic
    • Support

    Hi Greg,

    Thanks for the info, at the moment you can create two backups, one with all the files and DB tables but without plugins, and then other that will include only plugins.
    This worked for me separately so you can have full backup in two parts by using that method until we figure out what's the main cause of this.

    Best regards,
    Predrag

  • Greg
    • Design Lord, Child of Thor

    Thanks for the answer Predrag.

    Obviously eventually I'd like to create backups that include everything together, but for the time being I'll try that. I also have a security plugin creating a daily database backup, so that's something.

    Please let me know if you figure out the problem, hopefully it's a less complicated issue than it seems.

  • Greg
    • Design Lord, Child of Thor

    Just wanted to update and say that this is still not working - I have Snapshot installed on several sites now and it won't get past the Plugins phase on any of them, even ones that only have 10 or so plugins. The Snapshot stops in the same place every time and either times out or reports an error.

    UPDATE: I just tried again on one of my smaller sites with only one post and about 10 plugins - took me three times of clicking "try again" but just now it finally completed the Snapshot. But all the previous times, the plugins phase was where it crashed.

    Anyway please let me know if there's anything I can do to help fix the bug.

    I'm wondering - since it always stops when it gets to the plugins, could it be having trouble when it attempts to copy itself?

  • Predrag Dubajic
    • Support

    Hi Greg,

    I'm afraid that this is not a bug in the plugin, the thing is related to server resources and plugins folder usually has a lot of subfolders and files so it requires more resources to complete.
    It shouldn't require that much more in order to fail and that's why our devs need to check what's specific on your installation to cause this.

    During the backup process none of the files are modified so when Snapshot processes its own files there's nothing that will affect the process.

    Also, the backup process is the same if you do a full backup or if you try plugins only, and as mentioned in one of my previous comments splitting the backup in two parts, one without plugins and one with plugins only, the process finishes.
    So it's probably server running near the available resources and then when it reaches the plugins folder it goes over this limit and fails.

    Best regards,
    Predrag

  • Michael
    • Flash Drive

    We've had the exact same issue you're describing, for over 12 months now, and have not had any assistance from the WPMUDev developers regarding this problem since December.

    We have pretty much given up on using Snapshot under our lightspeed server.

    The issue is exactly the same... always something in the plugins folder triggers the snapshot to fail.

    Our sites are generally small... other backup plugins that we have got working have been able to zip all files & database into a backup file less than 80Mb in size.

    Some history of our issues are at https://premium.wpmudev.org/forums/topic/timeout-with-snapshot and https://premium.wpmudev.org/forums/topic/snapshot-cant-create-backups

    All I can say is good luck... hopefully if they can resolve your issue, we'll find ours resolved too. I'm unfortunately not holding my breath.

    • Predrag Dubajic
      • Support

      Hi Michael,

      I'm really sorry that you didn't get a response for such a long time, we had some rearrangement in the Snapshot dev team and unfortunately your email slipped through cracks :slight_frown:

      I was able to find it and I will get new dev to look into it and get back to you as soon as possible so they can continue working with you on this.

      Best regards,
      Predrag

  • Panos
    • SLS

    Hi Greg ,

    Sincere apologies for delay here!

    As Predrag already mentioned it is related to server resources, we would like to try and see if there is anything we can do to reduce the intense. We tried accessing via ftp but credentials didn't work also the test site seems to be removed.

    If you would still like us to have a look there could you please send in ftp and admin info once more? You can send that privately through our contact form: https://premium.wpmudev.org/contact/#i-have-a-different-question

    Send in:Subject: "Attn: Panos Lyrakis"

    - Admin login:
    Admin username
    Admin password
    Login url

    - FTP credentials
    host
    username
    password
    (and port if required)

    - link back to this thread for reference

    Thanks!

  • Greg
    • Design Lord, Child of Thor

    Ran across this again, almost a year later here - I still do separate Snapshots, where the plugins, uploads, themes, etc., all back up separately and upload as separate .zip files to Google drive at different times of the day.

    This has been working fine, but the other day on a whim I tried doing a full backup on one of the sites (not a large one) for the first time since originally posting this topic and it still fails. The managed backups work fine, it only times out if I try to run a full Snapshot on the site.

    I'm fine doing the separate backups, just figured I'd mention that it still won't do a full one.

      • Greg
        • Design Lord, Child of Thor

        At the time it was the same, but I've since moved that site and updated PHP... The site's around 150mb total and was on Cloudflare if that helps to know. I can try again this week if you're not able to reproduce the issue, but I'm in the middle of a project aimed at reducing bandwidth and server load for my client sites and I've started using a different backup solution for now. Love Snapshot though - if it ever supports incremental backups I'll probably be back. :wink:

        • Adam Czajczyk
          • Support Gorilla

          Hello Greg

          Thank you for the update. I'm not sure about the incremental backups but I can tell that's actually something that's being discussed internally so that "gives a hope" :slight_smile:

          I understand, of course, that in this case you switched to another backup solution but in case you were able to give it another try one day (perhaps maybe on some staging site?) we're here all the time so just update us here and we can "jump back" at any time.

          Best regards,
          Adam

  • Myke
    • Flash Drive

    Sad to see that Snapshot is still after a whole year no good.

    I've been checking in on this issue for about that long as well. I can find 50 threads where this issue was just abandoned and never fixed.

    It seems to happen with all websites on using Cloudflare with Lightspeed.

    I thought rolling back to 7.1 fixed this, because I got my first ever successful backup ever on that server setup, but I must have just lucked out and for only once not topped out the CPU, pushed the limits of timing out. It did however get rid of all those "PHP Warning: count(): Parameter must be an array or an object that implements Countable in [...]/wp-content/plugins/wp-defender/app/module/ip-lockout/controller/main.php" errors that have filled up all my error logs on every site on every server.

    Unfortunately, I've got all sites running a bunch of other WPMU plugins, because I was hoping to Automate them all, simplify things. This one plugin being so busted for so long is incredibly frustrating because it makes the use of Automate impossible/unlikely.

  • Greg
    • Design Lord, Child of Thor

    My servers are also Litespeed + Cloudflare, so I wonder if that's part of the issue, but I will say that I had no problem using Automate, because the managed backups seem to work fine. At least they completed, anyway. For me it was just regular Snapshots that always timed out and it just became too big of a pain to set up five different destinations for five different Google Drive folders and five different Snapshots scheduled at five different times of day....

  • Predrag Dubajic
    • Support

    Hi Greg and Myke,

    Managed Backup have a different way of producing backups and can be tweaked further with defines to adjust the usage on the server during the backup process.
    Both methods do work via PHP since it's a plugin and due to that it can require more resources from the server in order to complete the backup and that can be an issue with larger sites and limited server resources.

    We already had multiple improvements in the plugin and lot more installations are working out of the box but there's certainly more room for improvement and is something looked into by our devs.

    Myke could you start a new ticket with your site in question so we can have a closer look and see what could be done for your installation?

    Best regards,
    Predrag

Thank NAME, for their help.

Let NAME know exactly why they deserved these points.

Gift a custom amount of points.