Not all content is not copied from source site to destination in Cloner

When attempting to clone a source subsite (calgarybusiness) to destination subsite (amaster), the posts (directory listings) does not copy the content over.

  • Mahlamusa

    Hello mindset,

    I hope you are doing great today. Thank you for reporting this issue to us and thank you for your patience as we were looking into this issue.

    I have tried to replicate this issue on my test website and as well as yours. I have noticed that Cloner works well with a small number of posts but once I had over 3000 posts I started experiencing the same issue on my test website. The process times out if its a large number of posts and then once it times out Cloner will move to the next post type. It may seem that the posts were copied but actually they were not copied at all.

    I checked the code and found it is not easier to modify it to accommodate the large number of posts in one run as this is a very tedious process and you will literally have to wait for it to process all those posts before you can access the website.

    As a workaround to this issue, I have created a custom plugin that will hook a Cron Job to copy the posts in the background while Cloner processes the other posts types. So basically, this custom plugin will queue a Cron Job immediately after copying setting and verify after copying menus to ensure the Cron Job is queued.

    Basically, this custom plugin will schedule a Cron Job task to run every minute to copy the posts in batches of 50 from the source to the destination blog until all posts are copied over and the two have identical posts.

    Please find the attached .zip file, you may install this as a plugin or upload the contained php file to the folder 'wp-content/mu-plugins', make sure that the php file is in the root of the mu-plugins as follows:

    wp-content/mu-plugins/copy-all-posts-cron.php

    Once the plugin is in place, you can run Cloner again to clone a website, this custom plugin will kick in once the settings and start copying posts once the settings and or menus are copied.

    I hope this helps. I know the most obvious solution would be to modify Cloner directly, but that will change a lot in the plugin itself so this solution is a better option to get you going. Please try the code and let us know how it goes, I only tested it on my test website and it seemed to do well as I was able to copy 4800 posts which was always failing before.

    Have a nice day.
    Mahlamusa

  • Mahlamusa

    Hello mindset

    I hope you are doing great today. Thank you for your feedback and I am so sorry that this does not work for you and hope to make it work as soon as possible.

    Could you please give some information about how you tried to use it and which site you tried to clone to. Installing the plugin is not enough and will not trigger any cron jobs. So what you need is to clone a website after the plugin is installed and activated. Basically the Cron Jobs of this plugins are hooked to the step on which the settings are copied, after Cloner finished copying the settings the mu-plugin will be able to get the source and destination sites and then schedule the Cron Job to run.

    Please also note that the Cron Job relies on some traffic coming into your website and will not run if there is no traffic and the is no custom Cron Job set on cPanel. So, if your website is new (the destination site) and there isn't enough activity, it is very liked that the Cron Jobs will not run properly. In this case you can set up a new Cron Job in cPanel > Advanced > Cron Jobs to periodically execute the wp-cron.php file located on the wordpress install folder.

    Could you please verify that the cron jobs are running properly and the cron job for this mu-plugin is actually registered on WordPress, to check this you will need a plugin called WP Crontrol, once this plugin is installed and activated you can check the cron schedules on the destination site by going to the "Settings > Cron Schedules" page and on this page you should see a cron entry for the custom cron, the "Hook Name" will be:

    cloner_copy_posts_cron

    And the "Actions" will be:

    do_cloner_copy_posts_cron

    And the schedule must be 'Once every Minute'

    Please check this page to see the last and next run of this cron job. If the last run is not updating then the cron might not be running properly. If it runs but does not update anything then let me know and I will investigate further.

    I hope this is helpful.

    Have a nice day.
    Mahlamusa

  • mindset

    Well, you didn't tell me to setup any crons -- you gave me install directions and that is what I did -- I was trying to clone ftlauderdalebusiness - but now I have just converted that file - because the clone did not work. I was going to convert the cloned site first but since it did not work -- I went ahead and took a chance and converted the original site.

    As far as the rest of your message -- I have no idea what you are talking about -- I gather you want me to install another plugin. Another issue I am really upset about - since I was under the impression that all wpmu-dev plugins were suppossed to work with each other -- yet everytime I have an issue -- I am told to do a plugin conflict which is almost impossible because of everything I got going on -- I have been trying to take this network live for over 4 years and can not take it 100% live because of all the issues I can not resolve --

  • Mahlamusa

    Hello mindset

    I hope you are doing great today and thank you for the feedback.

    Well, you didn't tell me to setup any crons -- you gave me install directions and that is what I did

    WordPress has a built in Cron system that relies on traffic into the website, if the website does not have any traffic then this built in cron system will not run. I advised you create cron jobs now because I see you reported that nothing is happening and I thought it might be due to the cron jobs not running. Unfortunately, if the destination website is new then there is no other way to run the cron besides creating one in cPanel. I am sorry I didn't mention this before I was relying on the built in Cron system.

    As far as the rest of your message -- I have no idea what you are talking about -- I gather you want me to install another plugin.

    Basically, I recommended that you install the WP Crontrol plugin because I believe it will help you get insight as to whether the cron jobs are being executed or not. It is not a compulsory thing to do, its just for information purposes only. You do not need to install it if you are not interested in seeing whether the cron jobs are being scheduled and executed properly or not.

    Another issue I am really upset about - since I was under the impression that all wpmu-dev plugins were suppossed to work with each other -- yet everytime I have an issue -- I am told to do a plugin conflict which is almost impossible because of everything I got going on

    We do ask to perform a conflict not because of our plugins only but for other third party plugins and the theme you have active on your site. This is necessary to rule out any plugins that are causing issues, conflicts are likely to occur especially if you have a lot of third party plugins and themes. It is virtually impossible to make our plugins compatible with every other third party plugin or theme.

    With this specific plugin, we also needed to know if the issue is not caused by any third party plugin or theme. Finally, we figured that it is not any conflict but due to the large number of posts and the server limits. Since we figured that the server has timeout issues when processing a large number of posts, then we have decided to create the custom mu-plugin that will deal with the limit by processing a small number of posts in each turn until all posts are copied over.

    Please note that Cloner works perfectly fine on websites that have a small number of posts. The issue with your website is that it has a lot of posts and Cloner encounters a server timeout error before it could copy any posts, that is why we have suggested the custom mu-plugin to achieve this by copying in batches.

    I am sorry you had to delay this long to get your project at 100% live and I wish to help in any way possible to get you live. Please let us know if you have any other issues and we will be happy to help in any way possible.

    Have a nice day.
    Mahlamusa

  • Mahlamusa

    Hello mindset

    I hope you are doing great today. I am really sorry for the delayed response here and thank you so much for your patience.

    I have looked into the issue again and have confirmed that it is due to the cron job not running for the new blog. I suggested that you create a custom cron job for the new blog and I noticed that the built in WordPress Cron that relies on site traffic was already disabled by adding the following line in the wp-config.php:

    define('DISABLE_WP_CRON', true);

    And a custom cron that points to the main site was added in cPanel > Advanced > Cron Jobs, this is ok and should work for the main site but it does not cover the new sub sites and so the cron jobs does not run for any of the sub sites. To solve this, there are two options:

    1. Add a custom cron job for each sub site in cPanel > Advanced > Cron Jobs. This will work but it is a very tedious process if you have a lot of blogs in the network. But if you do it should be something like:

    * * * * * curl -vs -o /dev/null "http://subsite.domain.com/wp-cron.php">/dev/null 2>&1

    Replacing "subsite" and "domain.com" with the proper sub domain for the sub site and domain of the network.

    2. Implement a multisite specific solution to loop through and execute the cron file on all blogs. I have borrowed a solution from another thread, the description can be found here

    Following this tutorial, I have added ttwo custom files on your website root, these are 'wp-cron-custom.php' which is a modified version of wp-cron.php and 'wp-cron-multisite.php' which is a modified version of the solution provided here under Multisite section, basically what this does, it reads all the blogs in the multisite getting their blog IDs, then loops through them to get and execute the wp-cron-custom.php file for each of the blogs. This is to ensure that the cron job is executed for each of the blogs.

    I have already implemented solution 2 but I have also left a cron job in cPanel for the "cloneto.yourdomain.com". I have created this site as a test and it is a clone of "calgarybusiness", I have been monitoring it for some time and the posts are being copied over by the cron. At the time of this writing, 3360 posts (Directory Listings) were copied and it should continue to copy them at a rate of 100 per minute until all 22,757 are copied.

    I have noticed one thing though, the theme from the source site is not activated on the destination site as before. I have noticed that this is due to Pro Sites Premium Themes addon, the theme is not allowed for the new blog because it is copied as a "Free" site and the theme is restricted to specific Pro Site levels. If the theme is not active the directory listings are listed as "Posts", if you activate the theme they should still be there because the theme only changes the label for "Posts" to "Directory Listings", but that is not a problem and I think if the theme is not restricted then it should be activated as it was activated before.

    I hope this helps but please let us know if you have any further concerns or questions about this and we will be always happy to help in any way possible.

    Have a nice day.
    Mahlamusa

  • Mahlamusa

    Hello mindset

    I hope you are doing great today. I am sorry that you don't understand what I was trying to explain. It would help me to know what it is that you don't understand specifically, let me try to make it short.

    The issue is that the original Cloner plugin could not copy a lot of posts at once - in your case it is over 22000 posts. So I created a plugin that will be hooked to Cloner so that it starts copying the posts in batches of 100 every minute, this requires a Scheduled task (Known as Cron Job) that will be ran by the server every minute to copy the posts, so I created a scheduled task in your cPanel account in the "Advanced > Cron Jobs" section.

    You can learn about Cron Jobs here: https://my.hostmonster.com/hosting/help/cron

    WordPress uses a built in task scheduler working like Cron Jobs, this requires that your website gets visited or scanned by a Bot like Google Search Bot or similar. If this does not happen then the scheduled tasks will not be carried out. To ensure that the tasks are completed, a server side cron must be added to cPanel as described in these guides:

    https://kb.iweb.com/hc/en-us/articles/230244168-Setting-up-cron-jobs-with-cPanel-
    https://documentation.cpanel.net/display/ALD/Cron+Jobs#CronJobs-Addacronjob

    The url that must be executed by the cron job is: http://yourdomain.com/wp-cron.php
    In the case of multisite networks, you must add the cron job for each domain, this is not an easy task to do for networks with large number of sites or blogs. For this reason I have created a new file that will be used instead of wp-cron.php, so when adding the cron job in cPanel it must execute the url: http://yourdomain.com/wp-cron-multisite,php which will loop through all the blogs and run the cron jobs.

    I have noticed one thing though, the theme from the source site is not activated on the destination site as before. I have noticed that this is due to Pro Sites Premium Themes addon, the theme is not allowed for the new blog because it is copied as a "Free" site and the theme is restricted to specific Pro Site levels.

    If I clone the calgarybusiness website to a new one, the theme is not activated in the website we are cloning to. This is because the Premium Themes addon in Pro Sites is active which limits the availability of themes to certain levels in Pro Sites. If the theme is restricted to a specific level and the new website is not in that level then it won't be possible to activate the theme in the site. So for this reason, the theme is not activated in the new site and it seems the "Business Listings" menu will not be visible until the theme is active because it is the one managing the Listings. Right now after cloning, the listings are listed under "Posts".

    It is ok if you still do not understand this, the good news is that I have already set it up for you, I think its good though to understand how it works.

    I hope you understand this now. Please let us know if you still need any help and we will be happy to help you in any way possible.

    Have a nice day.
    Mahlamusa