Snapshot local backup stalling at plugins

I was trying Snapshot but it keeps stalling at the plugins directory at 84% with no errors. We have tried a conflict test and maxed out all server configurations (1024M memory limit and 3000 max_execution_time) and with all other plugins disabled, it still stalls at the same position. Snapshot log didn't reveal any useful information. Debug log showed this at the end -

Undefined index: interval-offset in /home/********inc/***.nz/wp-content/plugins/snapshot/views/snapshots/snapshot.php on line 908

And most importantly, the javascript console shows this error - http://take.ms/aVgnc

However, not sure why it is showing up as all plugins are deactivated and file permissions are also okay, I've opened a ticket with my host to check if they have restricted access to it for some reason and I'll update the answer here.

Thanks!

  • Norman

    So I checked with my host and they aren't restricting access to admin-ajax.php

    They do have this plugin called Imunify 360 running in the cPanel. I've had this come up a couple of times with another backup plugin. While the backup plugin was running, it failed and when I tried to look at cPanel, an Imunify interface popped up and required me to confirm I was human.

    Then I could get access back. But the other plugin still didn't work.

    This might be a red herring but I'll ask them about it.

    Now I've done some more testing trying to exclude directories to test if a particular directory is a problem.

    I have plugins/civicrm. Now CiviCRM takes up a huge space. 240MB.

    When I excluded it completely, the snapshot ran fine.

    Then I went in and excluded specific very large directories and then included them one after the other. If I remove these two directories, then the snapshot will complete.

    plugins/civicrm/civicrm/sql
    plugins/civicrm/civicrm/CRM

    So the question is if it's just that these are very large, or if there's something in them that's causing a problem. I haven't drilled down to isolate more directories yet.

    As a side note, if I go to Resource Usage Overview in cPanel, then in the snapshots section, there's a lot of snapshots that show that /wp-admin/admin-ajax.php was having performance problems...

    I mention it as a possible data point.

    Cheers,
    Norman

  • Katya Tsihotska

    Hi Norman

    Hope you're having a wonderful day!

    Thank you for the additional information, it will be very helpful for further investigation.

    However, meantime, could you try one more basic adjustment. The following code will instruct Snapshot to process backup using smaller "chunks". This will increase the time that it takes to complete but it often helps in cases like this and lets the script fully process the data. To do this add following lines to the "wp-config.php" file before the line /* That's all, stop editing! Happy blogging. */:

    // Reduce Snapshot Table Query Chunks
    define('SNAPSHOT_TABLESET_CHUNK_SIZE', 100);
    // Reduce Snapshot File Chunks
    define('SNAPSHOT_FILESET_CHUNK_SIZE', 100);

    Let us know if it doesn't help.

    Kind regards,
    Katya

  • Norman

    One thing that puzzles me is that in the log are the lines with "section:" in them

    The path in the lines are not an existing path.

    I expect this is just a line that can be displayed on the web page.

    But the failure in the snapshot happens at the line

    2017-11-10 06:58:32: file: section: plugins

    Is there perhaps a process that enumerates the files in the plugins directory. And it's that process that is stalling because there are so many files in my directory?

  • Norman

    Some statistics, in case it's useful.
    For my plugins directory, this is a small selection of the folders. Just indicative of numbers...

    /plugins - 230MB - 14873 files, 2390 directories
    /plugins/civicrm/ - 171MB - 10772 files, 1671 directories
    /plugins/civicrm/civicrm/CRM - 18.6MB - 1905 files, 395 directories
    /plugins/civicrm/civicrm/sql - 51.1MB - 107 files, 0 directories
    /plugins/civicrm/civicrm/l10n - 50.2 MB - 46 files, 92 directories

  • Norman

    So, here's an update. Because I'm testing different hosting providers, I was testing out shared hosting with Site Ground.

    I had a similar problem when testing backing up. Not quite the same, but an error came up in the window. I contacted their support to see if I could get max_execution_time increased. But they don't want to allow increasing beyond 120s.

    BUT, the support tech did give me this to put in my .htaccess

    <IfModule mod_dtimeout.c>
    <Files ~ ".php">
    SetEnvIf Request_URI "index.php" DynamicTimeout=600
    SetEnvIf Request_URI "admin-ajax.php" DynamicTimeout=600
    SetEnvIf Request_URI "wp-admin" DynamicTimeout=600
    </Files>
    </IfModule>

    This instantly made the process work smoothly and cleanly. And even worked on my other hosting platform.

    I hope this is helpful in some way...

  • Norman

    So, another bit of information.

    I started a Managed Backup and it is acting very weirdly.
    At one point it finished a backup very quickly, but the file size shown was only a few hundred KB.

    It should be in the region of 80-90 MB, compressed.

    And I've been testing, and now it's stuck being VERY slow.

    And in the log it has this line repeated many times.

    [snapshot][2017-11-11 08:44:44][Warning] Unable to perform requested system backup, proceeding with builtin

    Cheers,
    Norm

  • Lindeni Mahlalela

    Hello Norman

    I hope you are doing great today. Thank you so much for all the information you have provided and for your patience.

    I have also tried Snapshot on your site and on the plugins step I got the following error:

    REQUEST TIMEOUT
    This request takes too long to process, it is timed out by the server. If it should not be timed out, please contact administrator of this web site to increase 'Connection Timeout'.

    This means Snapshot was terminated due to reaching the limit of max execution time. I then checked the provided logs and it seems that Snapshot fails specifically when processing the folder 'plugins/civicrm/civicrm/CRM/uploads', I excluded this folder and tried again. At this point it seems the issue is with the number of files inside this folder that is causing the problem.

    I also had to check the config of your site but I was unable to login to FTP, I am not sure why. Could you please double check the FTP login details you have shared with us. If you have updated your FTP login details then please send the new login details via our secure contact form, make the subject to be "Attn: Lindeni Mahlalela" then in the message box include the following:

    - FTP Login details (FTP Host, Port, Username and password)
    - Link back to this thread for reference.

    Once you have done that I will continue to check again and see if there is anything we can do. I hope to hear back from you soon.

    Have a nice day.
    Mahlamusa

  • Norman

    So I finally managed to get a Managed Backup working on another server, but I discovered that sadly, looks like Snapshot doesn't do migration. Which is whatI'm wanting. But Snapshot doesn't make any attempt to change paths, or allow moving databases. Looking back at my posts, I realised I never actually said that's what my goal was. To be able to migrate Wordpress sites.

    Sigh. I'm just trying to find a plugin that allows migration from one site to another (including the entire network of a multisite), while changing directories and databases. I'm starting to think that doing it manually is the only way....

  • Ash

    Hello Norman

    It doesn't do exactly automatic migration though. But you can download the backup file from hub, it will give you two files. One is the zipped backup and other is the installer file, with instructions.

    Then, in your new domain/site/server upload this two files, follow the instructions, provide new domain name, db name, username, password etc and the site will be migrated there.

    And thus, it will also make all the required changes for path and urls in the database.

    Do you want something different? If so, would you please explain a bit? :slight_smile:

    Cheers,
    Ash

  • Norman

    Ah! Right. I was doing the transfer inside the Snapshot interface.

    Not downloading then uploading.

    I will give that a go. Because that does sound like what I'm after.

    Another plugin I've found does it the same way.

    Obviously, being able to do it in one step would be great, because that would allow creation of dev, staging and live sites very easily. But I'm given to understand that is in the roadmap for another plugin.

    In the meantime, I'll test what you've just said.

    Cheers,
    Norm

  • Lindeni Mahlalela

    Hello Norman

    I hope you are doing great today. Thank you for your feedback and patience here, much appreciated.

    I have changed the old Snapshot configs from the wp-config.php and replaced it with the new config:

    define('SNAPSHOT_ATTEMPT_SYSTEM_BACKUP', true);

    To take advantage of Snapshot's new System Backup technique which uses system functions to create backups. I have tested this several times on your website and it seems to work fine, its much faster and completes without errors. After this config I have also removed the exclusions in the "Global File Exclusions" list in the settings page which means right now everything is Backed up including the folder that was giving us issues.

    So now, I think everything is ready and working as it should but please run some tests from your side and let us know how it goes.

    Have a nice day.
    Mahlamusa

  • Lindeni Mahlalela

    Hello Norman

    I hope you are doing great today. Thank you so much for your feedback.

    In most cases it is recommended that you keep the exclusions in place due to server limitations. I checked your config and noticed that the process times out at around 10 -15 minutes. When I check the .htaccess and the WPMU DEV > Support > Sytem Information" page I see that the set time out is 3000 seconds (I also set it to 5000) but still it times out. This means the server is configured to ignore the override.

    I also checked your PHP info and the default max_execution_time for your server is 30 but the local value (determined when PHP was executed) is set to 0 which means unlimited, but still the process is terminated due to a timeout. Technically, if the max_execution_time is set to 0 then the server should not terminate any script at all as 0 means unlimited time, if set to 3000 then it should run for at least 50 minutes before being terminated but it seems the server does not obey that it simply terminates the process at around 10 - 15 minutes.

    So it is obvious that no matter what timeout value we set for PHP, the server will still stop the script before it finishes. Unfortunately, this is a server limitation so I suggest you keep the file exclusions to get this to work. If that is not ok, then please try to contact your host about this issue and see if they can help.

    I hope this helps. Please let us know if you need any further assistance.

    Have a nice day.
    Mahlamusa

Thank NAME, for their help.

Let NAME know exactly why they deserved these points.

Gift a custom amount of points.