cloner is very slow cloning

Cloner is very slow cloning. The entire multi site is only about 3G, yet cloning one of the sub sites is taking over an hour. Have increased the memory_limit to no avail. There are a lot of images, but none very large; under 200k.

  • Ash
    • WordPress Hacker

    Hello Thomas Thixton

    Would you please check the subsite that you are cloning from, how many files are in the media library of that site? What is the total size? (I know you mentioned about images, but is there any other large file like zip, please check from file manager) And also, how big the database is of that subsite?

    Please let us know. Have a nice day!

    Cheers,
    Ash

  • Thomas Thixton
    • WPMU DEV Initiate

    Well, finding about 56k images with most about 200k and only a couple dozen over 1m to a max of 4m
    The DB, when dump, is about 90m of sql.
    Watching the images being copied, it takes 5-15sec/image on the update screen.

    The site is not heavily loaded when doing the clone and it is repeatable, although since cloning is a rare event, we just put up with it, but it does seem excessive. Took about 2hrs to copy one site.

    And since this is a production site, have not been able to turn off plugins to find out which one might be impacting the cloning speed.

  • Ash
    • WordPress Hacker

    Hello Thomas Thixton

    56k images are really a lot. I think the spent time on that is okay. If you copy 56k images in your local machine, it will take more time than this.

    About the size per image, it actually takes more time to copy 100 files of 10kb rather than 1 file of 1000kb, though the total size is same. For every image, there is a separate copy command gets executed. So, I think the time it needed is quite required.

    Hope it helps! Have a nice day!

    Cheers,
    Ash

  • Thomas Thixton
    • WPMU DEV Initiate

    Hmm, a simple rsync takes about 10min for about 88k files and under 6G from our AWS server to my laptop, so 2hrs does seem excessive just a local copy or am I missing something?

    sent 1872309 bytes received 5511543150 bytes 9384536.95 bytes/sec
    total size is 6284101265 speedup is 1.14
    real 9m47.247s
    user 0m58.426s
    sys 4m55.995s

    Unless each call through Cloner also makes a path through the entire WP protocol stack of plugins, in which case perhaps there is one plugin that is a laggard.

    After bringing up a staging server, and going through and deactivating plugins during a sub-site clone, it looks like "Compress JPEG & PNG images" from TinyPNG is the culprit. Image copies drop from 10-15sec/copy to about 2sec/copy. Not sure what TinyPNG is doing, but it definitely is doing something.

  • Ash
    • WordPress Hacker

    Hello Thomas Thixton

    Thank you for your debugging and extra information. I have convinced a second level support developer to investigate this. Unfortunately, he doesn't have any site setup with that much images.

    Would you please provide your admin login and cPanel login of your test site? So that he can do some more debug at your site and find the issue?

    You can send me privately in the following format:
    1. Please go to https://premium.wpmudev.org/contact/#i-have-a-different-question
    2. Select "I have a different question" if not selected
    3. In subject line put "Attn-Ash" (this will ensure that the email is assigned to me)
    4. Provide the wanted details.
    5. Add the thread link in the message so that I can track.

    Have a nice day!

    Cheers,
    Ash

  • Ivan
    • Developer

    Hi Thomas Thixton !

    Grant access is enough for replicate but it's not enough for investigating this issue.
    If you provide me with additional login credentials, I can look into it more.

    https://premium.wpmudev.org/contact/#i-have-a-different-question
    Please visit our private Contact page and complete the form with the following information:

    Subject: "Attn: Ivan Svyrskyi"

    In the Message box, please provide the following:

    - link back to this thread for reference
    - any other relevant urls

    - Admin login:
    Admin username
    Admin password
    Login url

    - FTP/SFTP credentials
    host
    username
    password
    (and port if required)

    Note: Don't send any credentials via this forum because it's public forum.

    Best,
    Ivan.

Thank NAME, for their help.

Let NAME know exactly why they deserved these points.

Gift a custom amount of points.