New Release - Snapshot!

Hey All,

Reckon you're gonna likes this, it's a local time machine for your WordPress sites :slight_smile:

https://premium.wpmudev.org/project/snapshot

Create snapshots of any or all of your database tables, as many as you want, whenever you want, and easily restore, download and abuse at will.

Helpful for development & testing we hope :slight_smile:

As well as a convenient, free, backup tool.

Let us know your feedback, feature requests, bugs & any questions!

Cheers, James

  • Mason
    • DEV MAN’s Sidekick

    LOL

    @John, of course, we have all kinds of plans for this thing. Nice suggestions all. Please do have a play with this version and give us your feedback (not just feature suggestions, although those are welcome).

    We wanna make sure we have a rock-solid 1.0 version here as well as look to all the new possibilities this guy opens up :slight_smile:

  • James Farmer
    • CEO (of WPMU DEV, honest)

    > Can we backup the site files as well?

    We had discussed this, I think it'd be a great option, obvioulsy tricky to do right (given sizes etc.) but doesn't mean we can't give it a crack...

    > Any plans to have the snapshots added to Amazon S3?

    Why not :slight_smile:

    > Any plans to have a multisite version?

    Same as files stuff, I can imagine running this on Edublogs and then being hunted down by an armed band of sysadmins :smiley: But y'know, we are known for our multisite, so I think this'd be desirable too.

    > Any plans to add scheduling?

    Not before you requested, now, yeh... why not :slight_smile:

    Paul (new trial guy) has been the core developer behind this, he'll be along shortly to fill us in on feasibility, timeframes, and all that jazz.

  • Tee
    • The Crimson Coder

    Superb!

    Multisite would be perfect if you could select single sub sites from the super admin menu.
    Cool would be a 'one-click before and after' for testing. Even if you have changed plugins and themes :slight_smile:

    Looking forward to future development

    Cheers
    Tee

  • Aaron
    • CTO

    Hey guys, glad you're liking this! Just a quick clarification on the goal/purpose of this plugin. Snapshot is designed primarily for testing and rolling out sites. Wanna test a new plugin? Take a quick db snapshot so you can roll it back later. Or get settings just how you want them and roll just that back later. The flexibility of choosing tables is great for this.

    However, this is not designed to be your primary backup service. We really don't recommend using a plugin/php based system for regular production backups, especially a multisite install! It's just not reliable enough and scalable enough for that. With that in mind:

    Backing up media

    Probably not as that kinda blurs the lines and makes you think this is ok as your one and only backup solution.

    Mutlidb support

    Should be already in there fully. It uses the $wpdb class for all queries so they should be routed to the correct database.

    Multisite Support

    You can activate the plugin on just the sites you want to snapshot with, it will only be available to super admin users in multisite. There is (by design) only an ability to snapshot the site specific tables of the site you are running it on. Support for snapshotting all blog tables at once will never work reliably, so it's not in there.

    However, I do see how it might be useful to snapshot the global tables when you are setting up new installs and such, so we'll look into that for a future version.

    Multisite would definitely be a boom, as would a way to take a snapshot of specific tables on one site and use them on a different site, especially when it comes to sections such as custom post types/taxonomies.

    I can see how that would definitely be useful (and within scope), we'll see if we can add that.

    Any plans to add scheduling?

    I can't see how that would be useful in a testing/non-backup context?

    Any plans to have the snapshots added to Amazon S3?

    Possibly, though I don't think it would be a priority.

    Does it cover Buddypress aswell whether or not on a multisite?

    Should work on single site. On multisite those would be global tables though so woulnd't show up.

  • Aaron
    • CTO

    We really don't recommend using a plugin/php based system for regular production backups, especially a multisite install! It's just not reliable enough and scalable enough for that.

    Just a note, a multisite backup service/plugin is one of the most requested things here for a few years now. Something like Vaultpress that truly works for multisite. We really want to do this, there are just such huge technical hurdles to this. VaultPress only just added multisite support, but you still have to pay $15 a month for every individual site on the network you want to backup, not exactly feasible.

    So we hear you, and we're continually looking at it from a technical feasibility perspective.

  • John
    • Site Builder, Child of Zeus

    @Aaron, would you guys consider making a (dare I say it) non-WordPress style script for performing regular production backups for a multisite install?

    +1 for FTP/SFTP, S3, Dropbox, Cloudfiles etc; these seems to be the favorite

  • Moonworks
    • Code Wrangler

    Thanks for the clarification Aaron.

    I can definitely see a use for this, especially when you consider the number of times I've messed things up and have had to delve into the db with my shirt sleeves up.

    For theme/plugin development, this is a great addition.

  • josephdowdy
    • Site Builder, Child of Zeus

    @James, Considering that WordPress is so useful and so widely used and how multi-site is tightly integrated into WP, it really seems like a crime that there is no cost-effective way to have reliable backups on a multi-site install. I don't know how WP didn't make it easier to backup multi-sites with the last major version of WP. I can understand that there are technical hurdles but, really, how hard can it be to just write some SQL code to do this? If I knew how, I'd be doing it myself. This has been an obvious need for this probably for as long as multi-site has been around. I switched all of my sites into one big multi-site only to find I am living on a prayer that it doesn't crash because I don't know enough techie things to implement a sensible backup plan and I can't afford $15/site/month. My webhost will restore my files if all is lost, but I'd rather have a reliable method of my own as well using a plugin to do backups/restores.

    PS: I think it's a bit misleading to say that this plugin is like Time Machine for your WordPress site on my multi-site install when it won't work on my multi-site. Now I'm just disappointed about Snapshot since multi-sites are left out of the party.

  • josephdowdy
    • Site Builder, Child of Zeus

    @james Are you saying that's it like Time Machine for "my" WordPress site? Well, that's what it says on mine. I don't think this is the first time that I have got a notification regarding a plugin I can't use on my WPMU and had to waste time finding out it doesn't apply to me. Can you set preferences for notifications from you so that I can avoid getting my hopes up and ultimately wasting my time to find out a new plugin doesn't work on multi-site?

  • James Farmer
    • CEO (of WPMU DEV, honest)

    We'll look into improved updates, but in the meantime if you look at the plugin listing, you'll see for snapshot:

    https://premium.wpmudev.org/project/snapshot

    "Works perfectly with WordPress 3.3.1"

    (in the big yellow bar)

    And for example, for pro sites:

    https://premium.wpmudev.org/project/pro-sites

    "Works perfectly with WordPress Multisite 3.3.1 BuddyPress 1.5.4"

    (also in the big bar)

    Hopefully you'll be able to use that as a quick reference point until we incorporate that information into notifications (we've got a big improvement of our update notificatiosn plugin on the way too).

    Thanks for bearing with us.

  • Paul
    • The Green Incsub

    I'm not sure on the working definitions of those two terms. Let me try and explain how if works under Multisite.

    Under the primary site when you go to the Add New Snapshot you will only be allowed to select database tables related to that site. So using the default table prefix 'wp_' you would not see tables from other sites like wp_2_options, wp_4_postmeta, etc. The backup will be stored under the primary site's uploads folder which should be /wp-content/uploads/snapshots. A record of the snapshot will be written to the site's wp_options table. etc.

    For the sub-site the snapshot works the same way. The tables listed would only be for that site. Both WordPress tables wp_2_options, wp_2_posts, etc as well as any non-core tables created by other plugins. When a snapshot is created the file is stored under that sub-site's uploads folder something like wp-content/blogs.dir/2/files/uploads/snapshots/ A record of the snapshot is written to that sub-sites options table.

    So as you can see it is multisite capable and the data trail is kept discrete. Also, the important thing to remember on the primary site is the user tables are only present there. Under sub-sites you will not see a users or usermeta tables. This is because Multisite uses these for all sites. As I'm sure you are aware.

    Hope this helps. And hope it is not too technical.

  • josephdowdy
    • Site Builder, Child of Zeus

    Thanks, PaulM. That's very helpful. So I'm not really sure what it WON'T back up, but it sounds like if I knew what I were doing, I might be able to use this to do a restore of all sites if I setup a primary/admin site and then did all the restores. Would that be correct?

  • Paul
    • The Green Incsub

    @josephdowdy Well think of it this way. As the plugin current stands you can backup all or part of a sub-site. This includes the primary site. Via the plugin admin interface you can also restore a previous snapshot back to the same site.

    But a few important things you cannot yet do:

    1. From the primary site you cannot backup other site. You must first change to that other site's Dashboard and perform the snapshot there. This is actually one of the most requested features on this thread. Having a central snapshot panel where you can select the site to backup instead of having to switch sites would be helpful. All the snapshot files and logs would be kept under your primary site. Plus the restore would work better in case the real sub-site is hosed.

    2. You cannot create a snapshot on one site then restore it to a different site. Again, there are some technical hurdles I'm trying to work out with Aaron and other on how this would work. The biggest concern I have is on the site where you created the snapshot it could have posts that contain that site's URL for images etc. If you restore this to a different site you will have tons of broken links.

    3. Files/Media. The snapshot plugin was not designed to be a full backup system. Like Aaron and other have written it is more to take a quick 'snapshot' hint hint of your settings table because you are about to try some not plugin and want to make sure things don't get trashed.

  • Paul
    • The Green Incsub

    @Sarah Gooding That appears to be the second most requested feature. As I mentioned in my last reply to @josephdowdy there are some serious technical issues with doing that. When you create a snapshot on site1 for example it may have URLs for images to other posts embed in the content. If you restore this to a different sub-site things start to break.

    Sure we could make the plugin more intelligent to replace URLs. But then there may be other issues like the settings table where all plugins write their data. So is a plugin is installed on site1 and you restore it's snapshot to site2 where that plugin is not setup. What should happen?

    I could give you more but I hope you see my point.

    I'm not saying it is totally off the table. Just not on the front burner. I think the next features will be a central snapshot panel on the primary site where you can select the sub-site to snapshot (yes, I'm using it as a verb). The next will probably be the ability to push the snapshots to Amazon S3 or other CDN (Content Delivery Network). Though for a restore this might be a new hurdle.

    P-

  • Jason
    • The Incredible Code Injector

    Not to be a downer, but this seems like it would be extra stress on the server, and if the site is very large it could fill up the server faster.

    It also, as the author stated, great for snapshot, and edit your site, and then restore. Adding support for external services implies you would be intending on using this as a backup. I strongly suggest you backup at the server level, because if your site is compromised, etc, this wouldn't really help you.

    I use linode, and their backup is off host, takes about 15mins, and runs daily weekly and on demand.

    That's for the entire OS, apache, nginx, and whatever else config you're running. And it's only 7 bucks a month. You can restore a snapshot if your host dies, etc in just a few mins. It works fantastic, as a failed update has mucked up my servers before, and it's been a life saver. I highly recommend.

    Like I said tho, for dev, testing database things, this would be fantastic.

  • vanclute
    • WPMU DEV Initiate

    Just tried Snapshot for the first time. Fresh copy of WP (not Multisite). We're not doing blogs but rather business directory sites so have lots of listings. Had approximately 18,000 records between wp_posts and postmeta, all under a single custom post_type. Backup took a couple minutes and stalled at about 85% for most of that time. I thought it had died but then eventually it finished.

    I then emptied wp_posts and postmeta, and tried to restore. The restore procedure has no progress indication at all, so I had no idea if anything was happening. After about 5 minutes, the browser quit (white screen). Restore had not completed.

    On examining the DB, the postmeta table did actually have a lot of data in it, but posts had nothing. Now the snapshot I had taken is no longer available to the plugin, so I can't even try the restore again. The zip file however is still on the server, but as it's owned by the PHP user and I can't log in as that user, I have no way to delete the files Snapshot created in uploads/.

    Looks to me like Snapshot still needs some work before it's ready for primetime, at least in my test.

    Jonathan

  • Paul
    • The Green Incsub

    Jonathan, first thanks for the excellent feedback. Some followup questions if I may

    1. For the snapshot, did this include all tables or just the post and postmeta? I'm assuming just post and post meta since those were the tables you emptied.

    2. Correct the restore does not have a progress indicator. The initial snapshot creates a single dump file which is then zipped on the server. So the restore is sort of blind. I need to think in a restore of individual sections. Maybe split the dump into files by table.

    3. The plugin tracks the snapshots information as a record in your wp_options table (like any plugin). since you only restored the posts and postmeta I'm really not sure how the wp_options record was effected. Still in a better use case if your were actually restoring your wp_options and something happened I'm aware we need to so some other logic to self recover from the missing plugin records.

    4. If you know the filename you should be able to download it via your browsers. Since the web server runs as a specific user all files it creates uploaded media, snapshot files etc. should all be under the same owner/group. You should be able to piece together the URL. It will be something like
    http://www.yoursite.com/wp-content/uploads/snapshots/snapshot-120215-160205.zip

    As for deleting it until we put together some changes to the plugin you might try using your host's control panel. There is normally a file manage application that will let you manage file on your site.

  • mezanul
    • Flash Drive

    Excellent plugin! Just did a quick test by taking a backup of my site's db.

    After going through this post, I found everyone has already suggested improvements that I was thinking off! :slight_smile:

    +1 for wp-cron support, Dropbox and SugarSync support. :slight_smile:

  • vanclute
    • WPMU DEV Initiate

    Hey Paul thanks for the reply! To answer your questions:

    1, Snapshot was of everything, all tables.

    2. I emptied posts and postmeta but I actually attempted a full restore of everything. I did not single out those two tables.

    4. I imagine I could delete the snapshot via SSH... this is all being done on an AWS EC instance so there is no CPanel or other typical user-friendly control panel. It seems like maybe the plugin needs to just look through the snapshots directory in order to figure out if a snapshot exists. I haven't looked to see how it's keeping track currently, but it definitely doesn't know there's a snapshot there currently. My quick hack for that kind of thing is looping through a glob of the folder to get its contents. It works. LOL

    Jonathan

  • TECHNION
    • WPMU DEV Initiate

    Hello,

    This plugin is useless.

    You have to create MANUALLY snaps for each of your blogs. And restore FOR EACH USER who has a problem his blog MANUALLY.

    The idea in MU sites is to let users to do their thing by themselves.

    Noam

Thank NAME, for their help.

Let NAME know exactly why they deserved these points.

Gift a custom amount of points.