[Snapshot Pro] Snapshot causing high I/O and Physical Memory Usage

Snapshot Pro causing high I/O and Physical Memory Usage. Support Access has been granted. Please can you take a look?

  • Nithin
    • Support Wizard

    Hi Mark Clifford,

    Just to be sure, are you able to notice the high I/O when running both Snapshot, under Snapshot > Snapshots page, and Managed Backups, under Snapshot > Managed Backups? Or the issue is only specific to either one of these?

    I tried to run a Managed Backup, and it seems to abort with the following message:

    <empty response>

    Then, I created a new Snapshot "Snapshot Test", and the backup process finished in minutes, without any issue, also check the console for any errors, but it looks fine.

    Snapshot plugin can be resource extensive depending on the setup of your site, how many files there are, if there are any larger files etc. I/O usage varies from one server to another.

    Could you please share some screenshot regarding the high I/O usage you notice regarding Snapshot, so that we could have a better idea?

    Also, please enable debug mode, to check whether there is any specific issue within WordPress dashboard.

    To enable debug mode, open your wp-config.php file located in your root directory, and look for define(‘WP_DEBUG’, false);. Change it to:

    define('WP_DEBUG', true);

    In order to enable the error logging to a file on the server you need to add:

    define( 'WP_DEBUG_LOG', true );
    define( 'WP_DEBUG_DISPLAY', false );

    After making the above changes, please try to create anew backup. Once done, any related errors will be saved to a debug.log log file inside the /wp-content/ directory.

    Please attach these in your next reply in txt format file, so that we could give a closer look. You can find more details about debugging here.

    I could notice your website is running on PHP 5.6, if possible please do also check whether switching to PHP 7+ version helps, or not.

    Regards,
    Nithin

  • Nithin
    • Support Wizard

    Hi Mark Clifford,

    Thanks for sending the video really appreciate. Unfortunately the debug.log file doesn't give much clue regarding the issue. Could you please try adding the following rules inside wp-config.php file, and see whether it brings any improvements in terms on running a new Managed Backups.

    define('SNAPSHOT_TABLESET_CHUNK_SIZE', 100);
    define('SNAPSHOT_FILESET_CHUNK_SIZE', 100);
    define('SNAPSHOT_FILESET_LARGE_FILE_SIZE', 104857600);
    define('SNAPSHOT_FILESET_USE_PRECACHE', true);
    define('SNAPSHOT_ATTEMPT_SYSTEM_BACKUP', true);
    define('SNAPSHOT_IGNORE_SYMLINKS', true);

    Please make sure to place the above code, just before the line /* That's all, stop editing! Happy blogging. */

    Also, please do share a screenshot of the I/O usage from the cPanel while running a new backup with the above code added, so that we could also see whether it makes any difference once, or not.

    Please do keep debug mode enabled, and also share the new logs in your next reply, if you still get the same issue, so that we could give a closer look, if needed. :slight_smile:

    Regards,
    Nithin

  • Nithin
    • Support Wizard

    Hi Mark Clifford,

    Thanks for testing it further, and for sharing the screenshots. We'll need to troubleshoot this further, to have a better idea regarding what could be causing this.

    Could you please share your sites cPanel, and WP admin login in here, so that we could give a closer look?

    You can send credentials by using our secure contact form: https://premium.wpmudev.org/contact/#i-have-a-different-question

    - To Mark to my attention, the subject line should contain only: ATTN: Nithin Ramdas
    -WordPress admin username
    -WordPress admin password
    -login url
    -Cpanel Login credentials (host/username/password)
    -link back to this thread for reference
    -any other relevant urls

    Please follow up in the ticket once you have sent the credentials, so that we could give a closer look, and see what could be done to improve this.

    Kind Regards,
    Nithin

  • Nithin
    • Support Wizard

    Hi Mark Clifford,

    Thanks for sharing the credentials. I tested further, by tweaking the snapshot defines added in wp-config.php, but I'm afraid, the issue still exists with high CPU usage, and the backup throwing a 503 error.

    I'm bringing this into our Second Level Supports(SLS) attention, so that they could troubleshoot this further via given credentials, and see what else could be done to help resolve this issue.

    Will keep you posted via the support ticket, once we have troubleshooted this extensively. Have a nice day. :slight_smile:

    Regards,
    Nithin

  • Panos
    • SLS

    Hi there Mark Clifford ,

    We have installed a custom plugin, provided from the developer, that cools the process down while taking a managed snapshot. We did a few tests on your site and it takes longer to complete but it does complete without issues.

    This has been tested for managed backups that have been started manually. Not sure if the results will be the same for scheduled managed backups. In such case let us know and we'll see if there is a way cool that one too.

    Kind regards!

  • Mark Clifford
    • Site Builder, Child of Zeus

    Ok, but do you have any plans to sort this issue out without the need for custom plugins?

    By the way I have just tested with non-managed back ups and it's still causing issues with i/o. As for managed back ups it certainly does take longer which is far from ideal. I refer you back to my first point. Are you actually going to fix this?

    A member of my team has now opened a ticket with my hosting provider to see if they can shed some light.

  • Panos
    • SLS

    Hi Mark Clifford ,

    Sorry to hear you had issues even with the custom plugin. Please feel free to delete that plugin. As I said it is trying to cool the process down as server's resources are limited and can't handle a backup of this size. Using PHP to take backups and create zip files is resource intensive, I suppose that this is something you know already. We are trying to figure out ways to make it less stressful on the server, however best chances are to ask your provided to increase i/o limits.

    I have asked the developer and waiting for his valuable feedback!

    Kind regards!

  • Panos
    • SLS

    Hi Mark Clifford ,

    It looks like it's related to the size of the site.

    The funny thing is it was never doing this before and just started randomly

    If it used to work it means that the site has grown so that seems to be the reason behind this. I noticed that on 8th November the backup zip file was 944 MB and on 26th it was 1.14GB, so that's probably it.

    In order to have another look, could we activate the plugin again temporarily?

    Thanks!

  • Panos
    • SLS

    Hi Mark Clifford ,

    The developer did some tests but it seems that the site has grown to a size where Snapshot cant complete the backup as during the backup process some limits are hit .You can see the sizes of the backup files in the managed backups list. In order to find the differences you can download and unzip them, then compere the files in each backup. Perhaps table sizes have changed and/or number and size of files.

    Kind regards!

  • Mark Clifford
    • Site Builder, Child of Zeus

    In addition to the post above...

    I migrated from Yoast SEO to Smartcrawl but it seems I still have Yoast database tables in the database. Is it safe to delete them?

    I have flagged all the database tables I am concerned about here --> https://cl.ly/0cbbf8

    Can you help me identify the unnecessarily big tables? I am particularly concerned about the post meta ones.

  • Nithin
    • Support Wizard

    Hi Mark Clifford,

    I migrated from Yoast SEO to Smartcrawl but it seems I still have Yoast database tables in the database. Is it safe to delete them?

    Yoast do keep the settings in the Database even after the plugin is deleted. You'll have to manually remove these options, and any tables related to Yoast SEO, if you are no longer going to use these.

    Can you help me identify the unnecessarily big tables? I am particularly concerned about the post meta ones.

    postmeta table is part of WordPress, you can check the database structure of WordPress in here:
    https://codex.wordpress.org/Database_Description

    You could use a plugin like Advanced DB cleaner:
    https://wordpress.org/plugins/advanced-database-cleaner/

    Which should allow you to delete specific options in postmeta table, and other tables, and clean up drafts posts etc, which should help in general.

    Tables not listed in the above Database description link shared above can be deleted, if you aren't using that plugins. For example yost_seo_meta tables from Yoast, ewwwio_image table seems to be from EWWW Image optimizer plugin etc

    If you aren't using these plugins, they it should be fine with deleting it's related tables etc

    Please make sure to take a manual backup of your database before making such changes:
    https://support.managed.com/kb/a2034/how-to-backup-and-or-restore-your-mysql-database-using-phpmyadmin.aspx

    Regards,
    Nithin

  • Mark Clifford
    • Site Builder, Child of Zeus

    I have done all the above. Back up size is now 84.59 MB. Regular Snapshot completing quickly although still causing a spike server-side. Managed Snapshots still slow and causing server spikes.

    I'm waiting to upgrade my php version as our forum software is currently a little buggy with lightpseed server. I'm told they are fixing it. Increasing php version is the next and last thing I can do to help fix this issue.

  • Mark Clifford
    • Site Builder, Child of Zeus

    Hello,
    I have just spoken to Patrick about this on live chat and I'm sad to report that I am still getting high level usage. As Patrick says in response to reporting this issue (again!)...

    Indeed it should not, especially not with I/O at 16Mb/sec. I have sites on a host limited to 4Mb/s and Snapshot comes nowhere near the red line when I back those u. I'd say revive that topic with updated info for this site so the 2nd-level wizkids can have another look at the install

    So, here we are... can you 2nd level wizkids take a look. FYI my PHP version is now 7.2.

  • Panos
    • SLS

    Hi Mark!

    Since this is a different site, it is hard to say with admin access only. I used the support to create a new test admin on your site (as support access expired very soon), username is wpmudevtest1. However I would also need ftp access to try out the some defines that might make the Snapshot operations use smaller batches etc. You those site's ftp creds privately through our contact form: https://premium.wpmudev.org/contact/#i-have-a-different-question

    Send in:Subject: "Attn: Panos Lyrakis"

    - FTP credentials
    host
    username
    password
    (and port if required)

    Kind regards!

  • Panos
    • SLS

    Hi there Mark Clifford ,

    I have added the following defines in your wp-config.php file on the snowwatch site

    define('SNAPSHOT_MB_BREADTH_FIRST', true);
    define('SNAPSHOT_TABLESET_CHUNK_SIZE', 100);
    define('SNAPSHOT_FILESET_CHUNK_SIZE', 80);

    which should work only for Managed backups though. These should force smaller batches of files and table rows to be processed each time which hopefully would help here.

    Could you check if those help on the I/O spike during a Managed backup?

    Thanks!

  • Panos
    • SLS

    Hi Mark Clifford !

    Thanks for the feedback! We did several tests on the metcast site where strangely the I/O got high immediatelly. No matter how many files we set in each batch, the I/Os size would be almost double of the temp zip created utill it reached ceiling. From that it seems that the issue is not on reading the files that are about to be added to the archive, but the process followed in order to add a single file there. Unfortunately there is not much we can do there, as we are using standard and most optimized PHP methods for this. I am not sure if your host could have a look and provide an explanation for the I/O getting to 16 MB/s while the whole zip file is still less than 6 MB.

    It wouldn't complete the backup until we tried the following defines:

    define('SNAPSHOT_IGNORE_SYMLINKS', true);
    define('SNAPSHOT_FILESET_CHUNK_SIZE', 1000);
    define('SNAPSHOT_FORCE_ZIP_LIBRARY', 'pclzip');

    Strangely setting the chunk size as high as 1000 files allowed the managed backup to complete.

    The site is pretty huge in size. The files zip (excluding db tables) is 1.52 GB on it's own. For sites of that scale it is better to rely on other tools for backups instead of PHP as it adds more layers above server's commands.

    Kind regards!

Thank NAME, for their help.

Let NAME know exactly why they deserved these points.

Gift a custom amount of points.