How does snapshot work with sftp destination

I am trying to setup snapshot on a client's site.

I have created an sftp account for them on my backup server.
I can connect to the sftp account with their credentials (using filezilla/command line).
I have created a destination in snapshot using the ftp credentials - the test connection button works ok including cd-ing to the destination directory.

Using sFTP connection
Connecting to host: xxxxxxx.xxxxx.xxx Port: 22 Timeout: 90
Login success.
Home/Root: /data2/home/xxxxx/
Changing Directory: xxxxxxxxxxxxxxxxx.com
Current Directory: /data2/home/xxxxx/xxxxxxxxxxxxxxxxx.com/
Success!

I configured a snapshot to backup the db and wp-content files to the configured sftp destination.
The backup has run and created the zip in the local uploads/snapshot directory
Nothing has been transferred to the sftp destination.

My question: Does the transfer of the snapshot zip to the sftp destination happen asynchronously? If so when?
OR, have I misunderstood something and configured the snapshot wrongly (though it does say under the destination column the name of the sftp destination)?

  • Paul
    • The Green Incsub

    @Mike Little,

    Thanks for taking the time to post your question.

    The backup and then sending of the archive is two steps at the moment. When you run a manual Snapshot the archive is stored into your /wp-content/snapshots/ directory (as you know). There is a WP_CRON process setup that should run every 15 minutes that will push your files to the remote destination.

    So the issue to investigate is with WP_CRON. Basically, Snapshot registers a callback function with WP_CRON. As you might be aware WP_CRON is managed by WordPress core and relies on front-end traffic to 'kick off' the scheduler which in turn calls the Snapshot function as well as other core WordPress update functions.

  • Paul
    • The Green Incsub

    @Mike Little,

    Sorry wanted also to mention that we originally tried to have the manual backups send to the destination. But the issue was this send to FTP, Dropbox, AWS sometimes would take longer than the local server allowed. So we kept hitting server timeout issue. So when we added the scheduling we split the processing to have the scheduling WP_CRON send the files.

    Let me know more details and I can help you try and debug this.

  • Mike Little
    • Flash Drive

    Hi Paul,
    Thanks for your response and your thorough explanation. It makes perfect sense that you had to do it in two steps. I thought that might be the case, hence the wording of my question. It just wasn't clear from the Plugin page here, or the interface on the site. Perhaps an FAQ entry?

    I checked and can now see that the backup file successfully transferred to the sftp destination.

    However, I see that the original zip is still on the server's filing system, and I guess that makes sense for ease of download or restore. I presume with a scheduled backup and a limited number to be kept, the old ones will get deleted.

    I now have another question: will they also get deleted from the sftp destination? If they were removed from the local (web server) storage (by me) would the plugin be able to retrieve them from the sftp destination?

    I ask because I don't want the remote storage to fill up with old backups.

  • Paul
    • The Green Incsub

    I checked and can now see that the backup file successfully transferred to the sftp destination.

    Excellent!

    However, I see that the original zip is still on the server's filing system, and I guess that makes sense for ease of download or restore. I presume with a scheduled backup and a limited number to be kept, the old ones will get deleted.

    When you edit the Snapshot configuration there is a section 'When to Archive'. There is a field labelled 'Maximum number of local archives' which controls the local number of files to keep. You probably want to keep this to no more than 3-5.

    I now have another question: will they also get deleted from the sftp destination? If they were removed from the local (web server) storage (by me) would the plugin be able to retrieve them from the sftp destination?

    The current version of Snapshot does not remove from the remote destination. But this is planned for an upcoming release. Many of the members who tested the destination logic thought this was not needed since most had very large FTP, Dropbox, AWS offsite space. But their hosting was much lower in space. Plus many users using FTP had secondary scripts in place to move the uploaded file into alternate directories.

  • Mike Little
    • Flash Drive

    Thanks again Paul,

    Perhaps unusually, I have more space on my dedicated server (500GB drive) than my remote backup space (200GB). I know this means I could never backup everything on the disk if I were to fill it. But I've less than 100GB of user data to back up. And, for now, the remove space is at a reasonable price point.

    I do currently back up everything automatically to the remote space 4 times per day, but that means I have to do the restore if a customer needs it. Some of them would like to have that control (I dont' do 24x7 support for one). Your plugin came along at just the right time.

    If a new release will delete the old backups from the remote space that would be great.

    How about my other question, would the plugin be able to retrieve those zips from the remote space if they weren't available locally?

  • Paul
    • The Green Incsub

    How about my other question, would the plugin be able to retrieve those zips from the remote space if they weren't available locally?

    Apologies for missing that questions. This one is like the deletes on the remote. At the moment snapshot does not retrieve the remote file(s) for a restore. Just local. so you would need to manually upload the file to the snapshot file /wp-content/uploads/snapshots/ Then on the Snapshots > Settings page there is an option to scan for archive. This will import the file and then from there you can restore. So again this is something being worked on. This will actually be sooner than the other item or deleting remote files. Or might be together depending on how fast the development cycle goes. This is somewhat of a growing concern from other members about not being able to restore Snapshots outside of WordPress. So for example your site gets totally broken. In the current version the only way to get going is to first reinstall WordPress get it operational. Then activate Snapshot. Then run the scan. Then run the restore. So the plan is to have some scripting outside of WordPress that will 1) download your zip archive from a remote URL. 2) Optionally Download and install WordPress 3) Perform the restore from the Snapshot archive. Look for these in the coming weeks.

Thank NAME, for their help.

Let NAME know exactly why they deserved these points.

Gift a custom amount of points.