New to the community? Start here

Cloning overnight and still has not completed - "an error has occurred"

Last night I ran the Cloner plugin to clone one of our sites in our Multi-site network. I wanted to make sure everything was copied over so I could be sure I was working on the same thing in the staging environment as production (originally I elected not to copy over some tables, but then I noticed a lot missing). The cloning process started fine enough, but then started to run into "An error has occurred" with no details on what went wrong:

Settings Copied
Widgets Copied
Posts Copied
Pages Copied
An error has occured
Terms Copied
Main Menu Menu Copied
Sidebar Menu Copied
Users Copied
Comments Copied
[image] Copied....

Then afterwards the Cloner seemed to be copying 1 of our 10,000+ images over at a time every 3-5 seconds making for an extremely long process. It didn't even complete overnight. Every few seconds it would just tell me ImageA copied, ImageB copied, etc. Now that I look at it this morning, the process is still ongoing - but now all the log says every 3-5 seconds is "An error has occurred". I have no idea how much longer this process will go for or what these errors are.

What should I do from here? Let it continue to run overnight - again? All it is saying is "An error has occurred" for the past many many lines in the log! We really want to have a staging environment up that is as similar to our production environment as possible.

More Cloner discussions
Cloner
  • Jeffrey

    In wp-config.php and php.ini the memory limit is set to 512M.

    Is there a way to stop the current cloning process? I have no idea how much longer it will go on for and would like to progress in actually cloning one of our sites.

    Also, I'm not sure what needs to be selected to avoid images being copied over? Is it the "attachments" within Settings > Cloner that would need to be unchecked? Or could it be one of our tables? We use WooCommerce so have a lot of products with images, so I'm sure there's table entries referring to images in there somewhere.

    Let me know what I can do to stop the current process and what to do to skip images the next time I try it.

  • Jeffrey

    I deleted the cloned site through Network Admin > Sites >Delete site. Hopefully that's all that's needed to be done to clear out all of it from our server? Don't need to have traces of the cloned site preserved on our server ftp and database. Not sure how else to make sure the traces are removed from that.

    I reattempted with just unchecking "Attachments" in Cloner Settings and the new staging site looks good for the most part. However, no WooCommerce Products were copied over. No images are in our Media Library either, but I guess that's expected from unchecking attachments.

    Two questions:
    1) What does "an error has occurred" signify. It happened again in the logs in the exact spot as mentioned before - between Pages copied and Terms copied.
    2) Is there a way to still bring over the Products without having to bring over all the images?

  • Jeffrey

    Any update on this? The Cloner plugin at the moment isn't making an exact duplicate of our production environment as expected. For instance, there is a plugin that has a conflict in our production site, but it does not have the conflict on our staging site. So something is missing and seeing "An error has occurred" in the logs leads to me to also believe something important is missing on our staging site.

  • Michael Bissett

    Hey @jbratcher,

    I've closed the other thread opened, and have taken the discussion back over here, so that we can continue this efficiently.

    To answer each of your points:

    I deleted the cloned site through Network Admin > Sites >Delete site. Hopefully that's all that's needed to be done to clear out all of it from our server? Don't need to have traces of the cloned site preserved on our server ftp and database.

    If there's any custom database tables (e.g. ones inserted by WooCommerce, not those like wp_options), those'll need to be removed manually (via a database client, like phpMyAdmin), as WordPress takes care of removing database tables it creates, whether or not a site in question is a cloned site.

    From a file perspective, the upload folder for the site will still be there, but there's no files in it, so leftover files wouldn't be a problem.

    1) What does "an error has occurred" signify. It happened again in the logs in the exact spot as mentioned before - between Pages copied and Terms copied.

    Knowing that that error signifies would require seeing the data logged to the browser's error console, what's the site that you're trying to clone? I'd like to try cloning it myself, and checking out what's going on.

    2) Is there a way to still bring over the Products without having to bring over all the images?

    The products should be brought over regardless of whether images (or as Cloner refers to them, Attachments) are imported, there's something else causing them not to be imported. What that is will require further examination.

    Also, in case a deeper look is needed, could you please send in the following via our secure contact form:

    - Mark to my attention, the subject line should contain only: ATTN: Michael Bissett
    - Do not include anything else in the subject line, doing so may delay our response due to how email filtering works.
    - Link back to this thread
    - Include FTP log-in details (hostname, username & password)
    - Include any relevant URLs for your site

    IMPORTANT: Please make sure you select "I have a different question" for your topic, so it doesn't go back to the forums - this and the subject line ensure that it gets assigned to me.

    https://premium.wpmudev.org/contact/

    Kind Regards,
    Michael

  • Michael Bissett

    Hey Jeffrey, apologies for the delay on this one!

    After talking with another member of staff on this, and doing some further investigation here, the issue with the products would look to be due to the sheer volume of them (over 15k), and that the server runs out of memory before Cloner can successfully copy them over.

    (I even tried setting the memory limit to 1G, both in your site's php.ini, as well as your site's wp-config.php; I've since reverted it to 512M in both files)

    I would advise importing them into the test site separately, rather than cloning them over. Also, I noticed that you've got over 34,000 users on the main site right now, I'd also avoid cloning them as well (as the server runs out of memory with them too, due to the volume), you'd want to uncheck the "Users" box inside of the settings for Cloner.

    Kind Regards,
    Michael

  • Jeffrey

    Thanks for the update Michael. I've attempted to clone the site again - I unchecked "Users" from settings and did not include tables that had the word "product" or "item" in them as a guess for which ones would try to bring in the products. I'm still not sure if it is correct or not. Can you determine which specific tables I need to make sure to exclude so Products are not cloned over?

    A few issues I saw: I noticed while the Cloner was setting up the cloned site, it still logged "An error has occurred" in the same spot as before. Also I notice our search bar is absent in the cloned site, so perhaps that was from me excluding too many tables from trying to ensure Products are not brought over? Not sure.

    Let me know if you can find out which specific tables need to be excluded to make sure no Products are attempted to be cloned over.

  • Jeffrey

    The Search Bar issue also might be related to we use a custom plugin Searchford WordPress Advanced Search . I see the plugin is activated on the clone site, but the custom search we created is not present. And we reference the internal ID of that custom search for our main site. (If you go to our main site dashboard and select Searchford on the sidebar you'll see what I'm referring to).

    So in addition to wanting to know what specifically needs to be done to ensure Products are not cloned, as well as if the "an error has occurred" is problematic, I want to know if the plugins on the cloned site should be bringing over the configurations we setup for them?

  • Michael Bissett

    Hey @jbratcher,

    As for ensuring that products aren't going to be copied over, that's something that requires a tweak to the code for Cloner (so it'll need to be re-applied on an update to Cloner).

    On line 18 of:

    /cloner/copier/class.copier-cpts.php

    I changed this code:

    $exclude_post_types = '("' . implode( '","', array( 'page', 'post', 'attachment', 'nav_menu_item', 'revision' ) ) . '")';

    To this:

    $exclude_post_types = '("' . implode( '","', array( 'page', 'post', 'attachment', 'nav_menu_item', 'revision', 'product' ) ) . '")';

    And now, products from WooCommerce won't be copied over. :slight_smile:

    As to "an error has occurred" being problematic, the thing is that it's coming up when Custom Post Types are trying to be copied, which ties into your other point:

    I want to know if the plugins on the cloned site should be bringing over the configurations we setup for them?

    Yep, they should be doing that, the thing here is that the search bar that you're looking for would belong to a custom post type, and since we're running into an error in the process of copying custom post types right now, that would look to be the reason why it's missing right now.

    I've asked the developer for his feedback on the matter, to see if there's anything we can do about that here (we might be limited by server resources still, just so you're aware).

    Kind Regards,
    Michael

  • Michael Bissett

    Hey @jbratcher,

    The developer responded to the email that was sent before, but I've asked him for his further assistance on this, to see about those other custom post types (seeing as we've excluded WooCommerce products).

    On a side note, if you could re-check those FTP details (in case he needs them), that'd be appreciated, I tried using them today, and found myself unable to log in with them.

    (of course, we'll want to take care of sending over any updated FTP details via email)

    Kind Regards,
    Michael

Thank NAME, for their help.

1 Point
5 Points
15 Points
Other

Let NAME know exactly why they deserved these points.

Gift a custom amount of points.