Can't complete Backup

I have a massive site, over 15GB, the managed backup is stuck on 0% can you help me?

  • Adam Czajczyk

    Hello Brenda

    I hope you're fine today and thank you for your question!

    I checked the social.....ult.com site and I also added some additional "tweaks" to the "wp-config.php" (as on your other site that I worked with today; since it seems to be on the same server) but that's still not working and breaking at 0%.

    The site is, indeed, quite big and there are files and folder that either do not belong to WP install or are very big - usually causing issues with backups due to server capabilities of processing huge files. That said, I've added some exclusions (on Snapshot -> Settings page) to skip other backups (so Snapshot backup wouldn't include other backup files as this would be redundant), some non-WP folders from the root folder and one very big zip file (Customers.zip, over 2GB) as such file would be best secured simply by downloading it via FTP to some off-site location.

    This should help in most cases but it still didn't. The process breaks with "500 Internal Server Error" on one of the admin-ajax calls and that suggests that there's something more into it. I did some more tests and noticed something interesting:

    - before the process breaks there's a partial backup file created containing some data; it breaks on plugins;
    - if I exclude from backup process those plugins that were backed up, the process also breaks but now the partial backup file includes 1-2 plugins that are "next on the folders list" in the /wp-content plugins folder

    That suggests that there are two very probable issues:

    1. either there's not enough space available to actually create a backup
    2. or on this site the process is causing some resources (most likely I/O - file read/write) limits on the server to be hit

    As for 1)

    I believe you do have some sort of CPanel available. Could you please check if there's still available storage space for that site/hosting account? With that big site you should have at least the same amount of free space available as the site itself takes. This is because the plugin must be able to actually copy files and create a backup archive file before it's able to upload it to The Hub.

    If you do, please also get in touch with your host and ask them to check if the temp folder used by PHP have any limits on the amount of data that can be written to it and if so, if those limits can be raised. In other words, there are some cases when such temporary folder gets "clogged" and needs to be cleaned up. Usually the host support is able to help with this.

    As for 2)

    If above is not an issue and/or doesn't help, look for the I/O stats in the cPanel. If you can see anything like that, try running Managed Backup and keep an eye on this stats (they are usually real time) to see if limits are reached. If they are then there's really not much that could be done apart from asking host if they could raise them. If they can't, with that big site it would be suggested to use some "offsite" backup tools - server level tools (like at least cPanel backup).

    If you don't have access to I/O and other resource stats in cPanel, please get in touch with your host and ask them if they could check on their end if any of such limits has been reached around the time Managed Backups attempts were performed.

    If this is also not a case (so no limits are being hit), let me know please and I'll include our developers in the case.

    Best regards,
    Adam

  • Max

    Hi Adam,
    I also had same problems with this plugin. I also contacted my hosting provider. For a test he raised the execution time and it worked. I had same errors with different providers when a site is more than ~ 500 MB. Very sad!
    At the end I used a different Plugin (Backup Guard) – oddly I didn't had problems with it also with pages that are more than 17 GB!
    Please wpmudev-team! Solve these problems! Would be so nice!

  • Adam Czajczyk

    Hello Brenda

    If the temp folder is available, writable and not limited in any way (e.g. size- or file-number-wise), huge files are also excluded then there's one more thing to check that I asked previously about - I/O limits:

    If you don't have access to I/O and other resource stats in cPanel, please get in touch with your host and ask them if they could check on their end if any of such limits has been reached around the time Managed Backups attempts were performed.

    This is something crucial so if you didn't check that in cPanel (if you got cPanel, there should be I/O stats in server information box on the main cPanel page - you would want to keep refreshing that page and watch the stats while trying to run backup) or you don't have such stats avaialble for you - would you please ask your host to check their logs against it too? When backup process is running the I/O usage jumps up and they should be able to find out about these "spikes" in the server logs to see if those are hitting limits or not.

    If they are not and yet the backup doesn't complete, I'll need to include our developers in the case so let me know please.

    Kind regards,
    Adam

  • Adam Czajczyk

    Hello Brenda

    Thank you for your response.

    In this case I'll need our developers to get involved. I have passed all the information about the case (including all our findings so far) to them and they'll take over to investigate this. Please note: their response time might be a bit longer than ours here on forum as they are dealing with various complex stuff on daily basis.

    Please keep an eye on this ticket and we'll update you here as soon as we get to know more about the case.

    Kind regards,
    Adam

  • Konstantinos Xenos

    Hi Brenda ,

    It seems that the credentials you've shared with us are no longer working. Could you please share a fresh set with me so I can help with the issue you're having?

    You can send me the information needed privately through our contact form at https://premium.wpmudev.org/contact/#i-have-a-different-question by following this example:

    Subject: "Attn: Konstantinos Xenos"
    
    - Admin login ( if Multisite please provide Super Admin details ):
    Admin Username:
    Admin Password:
    Login URL:
    
    - FTP credentials
    Hostname:
    Username:
    Password:
    Port:
    Key-File ( and password ) if needed
    
    - Server Admin ( CPanel / Plesk )
    Username:
    Password:
    Login URL:
    
    - Link back to this thread for reference
    - Any other relevant URLs -or- information regarding the issue that was not included in this thread

    Regards,
    Konstantinos

  • Konstantinos Xenos

    Hi Brenda ,

    Sorry for the late reply I've been trying to adjust Snapshot from our end to see if there's something that we can do to help.

    To explain a bit further, the process isn't actually stuck on 0%, it is continuing as supposed to ( most of the times if the server doesn't cut it off from resources ) but since the size is big and we have to take under account all the checks and the zipping process as well ) you'll have to wait longer to reach even 1% if it actually continues.

    A site this big would require a lot of server resources ( memory limit, I/O, CPU usage etc ) to be able to handle a backup like this. Unfortunately there's no way to calculate how much it would require even or how long the whole backup process would take between each backup cycle to have a successful finish ( this also includes the process of uploading the backup to our servers ).

    We're always improving Snapshot and adjusting it to use as low server resources as possible but there isn't an "easy fix" that I can help you with at the moment especially for sizes like this.

    It's a situation that the server will have to be adjusted accordingly to be able to cooperate with PHP without breaking this process.

    I have also checked the Cron events as I saw that you have WP cron disabled in your wp-config.php most likely because you have server-cron enabled, they also seem to be spawning properly but you'll have to be 100% sure that they won't misfire as that would again break the process.

    Tell me if you need any further help or assistance with this!

    Regards,
    Konstantinos

    • Konstantinos Xenos

      The server might be reaching an I/O limit, I/O doesn't necessarily mean that it will reach it's limits due to one file being big or because of the overall size of the backup/website.

      Could we have CPanel access so we can also see if there's any server-side error logs as well as inspect the installation in general? Unfortunately with FTP even though there's access to files we can't have a clear view of which folder/file is actually big to run tests by exclusions or identify an extra server-side configuration that might be interfering.

      You can send me the information needed privately through our contact form at https://premium.wpmudev.org/contact/#i-have-a-different-question by following this example:

      Subject: "Attn: Konstantinos Xenos"

      - Server Admin ( CPanel / Plesk )
      Username:
      Password:
      Login URL:

      - Link back to this thread for reference
      - Any other relevant URLs -or- information regarding the issue that was not included in this thread`

      Regards,
      Konstantinos

  • Konstantinos Xenos

    Hi Brenda , sorry to keep you waiting but in order to test we have to actually run the backup process over and over again as well to check for changes and adjustments.

    You'll see that there's a managed backup uploaded to our server. We had to exclude the uploads/2019/01 and uploads/2019/02 folders as well ( along with all previous exclusions as well ) as those have ~20.000 extra files.

    As I've mentioned before we can't be sure or specific in numbers on what resources might be needed for a backup of a size/file-count like this to work as it depends on the system itself as well and what it can handle. Unfortunately the backup seems to be bigger than what the system can handle so it breaks the process.

    We will continue to improve Snapshot trying to use as less resources as possible in the future releases but unfortunately there isn't an easy patch/fix that I could give you at the moment, especially when the server is limiting the process as well.

    Tell me if you need any further help or clarifications.

    Regards,
    Konstantinos

Thank NAME, for their help.

Let NAME know exactly why they deserved these points.

Gift a custom amount of points.