[Snapshot Pro] Backup uploading to Google Drive keeps restarting

The upload process of my backup (3,22 GB) keeps restarting after about 5 minutes. I guess the .zip is too large for the hosting environment to upload at once. things like 'max_execution_time', 'max_input_time' and 'FcgidIOTimeout' are limited at 300 seconds by our hosting partner and I think this is a (more than) reasonable limit.

I've mentioned this before: Snapshot Pro should split the archive into multiple smaller zips (maybe 400 MB) and the uploading process would be a lot smoother.

  • Dimitris

    Hello there Dominique,

    hope you're doing good and thanks for reaching us! :slight_smile:

    I used the granted support access you kindly provided for us and I can see what's you're talking about in the log file of your snapshot.
    As I'm not sure if these multiple upload progresses are about separate batches (smaller zips), I've already contacted our developers to get more information.
    Me or another colleague of mine will keep you posted here as soon as possible.

    Meanwhile, please let snapshot as is and double check for me that there's the necessary available space in the connected Google Drive account. :slight_smile:

    Thank you,
    Dimitris

    • Dominique

      Hi Dimitris,

      I mean that Snapshot Pro should be updated with a zip-splitting option (or do it automatically) for when the backup-zip is getting too large. When looking at UpdraftPlus, this plugin doest this by default and even gives an option to adjust the size of the splitted archive files. I think this is a really good way to make sure that the backup archive isn't too large to upload.

      See this blog post with the announcement of that feature (introduced in 2013). They mention there that the splitting happens for every 1GB, but since then they've lowered that amount to 400MB: https://updraftplus.com/splitting-large-sites-into-multiple-archives/

      And to answer your question about Google Drive, we have no storage limit, so that shouldn't be an issue. Backups from other sites are still rolling in every day, so not problem there.

  • Dimitris

    Appreciate the feedback here Dominique :slight_smile:

    I've already informed our devs that there's no problem with GDrive available space and asked for some insights about these log messages.
    That's exactly what I was thinking about, these are either smaller chucks of original backup, or there's something else breaking here. As this isn't clear to me, I'm waiting for dev's feedback first, then I'll follow up here.

    Thank you,
    Dimitris

  • Dimitris

    Hello Dominique

    Could you please send us some access details through our safe contact form https://premium.wpmudev.org/contact/#i-have-a-different-question
    using this template:

    Subject: "Attn: Dimitris"
    - WordPress admin (login url/username/password)
    - FTP credentials (host/username/password/path of site in question)
    - cPanel/Plesk credentials (host/username/password)
    - link back to this thread for reference
    - any other relevant urls

    Keep in mind the subject line as ensures that it gets assigned to me.

    Thank you,
    Dimitris

  • Leonidas

    Hello there Dominique ,

    I debugged your reported issue and I can confirm that the issue originated from the backup being that big. What I've done as a workaround, was splitting the site into 2 backups:
    One for the site's db and all files except the uploads/2017 folder and one just for the uploads/2017 folder. Both snapshots were created and uploaded normally to my own Google Drive destination.

    You can check my configuration for both snapshots and edit them in order to make them upload to your own Google Drive destination. I have deleted my own Google Drive destination, so as of now those 2 snapshots appear as local ones. You can simply edit their destination to your Google Drive and you are good to go :slight_smile:

    Their names are WPMU - Backup all but half of media - Snapshot and WPMU - Backup only other half of media - Snapshot.

    Best regards,
    Leonidas

  • Leonidas

    Hey there Dominique ,

    I understand that this is merely a workaround for your backup process. The thing is that Snapshot does have a mechanism for large backups and their upload in remote destinations, but for some reason your WP installation this mechanism causes the upload to restart.

    What is more, we do have plans for optimizing the upload processes in the future. You can keep up with Snapshot's updates and check if this specific site is benefited by them in Snapshot's next versions.

    Best regards,
    Leonidas

    • Dominique

      but for some reason your WP installation this mechanism causes the upload to restart.

      I still guess that the restarting of big uploads is being caused by the values for 'max_execution_time', 'max_input_time' and 'FcgidIOTimeout', which is 300. If the upload isn't done in 5 minutes, it just restarts completely. That's why I would like Snapshot to be able to split the archives into smaller bits.

      I'm not sure what else you could mean by any 'mechanisms' that are in place to handle these big uploads. Should Snapshot Pro be able to continue the uploading process afterwards, when the 300 second period has been exceeded? Is our hosting partner blocking your mechanisms in any manner? Can I do something to allow them?

  • Leonidas

    Hi there Dominique ,

    I'm sorry if I caused any confusion with my previous reply. Yes, one thing you could do, if this is possible, is at least double the values for all 3 of the metrics you mentioned, to see if a backup for the whole site gets uploaded seamlessly. This is not something I previously recommended for our debugging process, since you have mentioned originally that you find them at reasonable values.

    That's why I would like Snapshot to be able to split the archives into smaller bits.

    Considering that this was the way that we eventually managed to upload the site in Google Drive (albeit in a manual way), I agree that having this integrated in Snapshot will solve similar issues and will be a more robust solution. We haven't really encountered much similar cases before, but I have actually made a corresponding internal entry for us, the Snapshot dev team, to consider this admittedly useful feature to be added in the future.

    Thank you for the feedback and you can let me know if those potential tweaks in the hosting environment make the splitting of backups redundant.

    Best regards,
    Leonidas

Thank NAME, for their help.

Let NAME know exactly why they deserved these points.

Gift a custom amount of points.