Moving WordPress Media To The Cloud With Amazon S3

Moving WordPress Media To The Cloud With Amazon S3

“The Cloud” is all the rage nowadays and for good reason. It gives you more storage space, faster storage, and distributed access. It’s essentially an automatic backup and can take away a lot of the headaches that come with moving and testing a website.

While there are some disadvantages, like less access and security worries, these would either exist on your own server as well, or are more than worth it for most.

In this post I’ll show you how I offload all of my media onto Amazon S3 using WordPress and some plugins. I’ll also show you how you can use Amazon CloudFront to serve your files via a CDN for faster loading times and latency across the board.

Why the Cloud?

I already touched on why I am a huge fan of putting all my media in the cloud, but let me elaborate:

Detached Media

My main reason is that detached media makes my life easier. Imagine you want to migrate a website to another server. If this becomes necessary, chances are it is because your website outgrew its current lodgings – you probably have more than a few images.

Everything inside WordPress, like posts, themes and plugins, takes up very little space. Even a sizable blog may only have a database of about 2-3MB. Perhaps your themes and plugins take up 10-30MB, but most of those don’t have to be moved, they can be re-downloaded.

Whenever you upload an image to WordPress, at least three other versions of it get created (unless its dimensions are very small). This means that for each image you upload, there are actually four images on the server. So if you’ve uploaded only 100 images, this would amount to 400 images and could easily exceed 400-600MB.

Using fancy developer tools, you could easily transfer everything in your WordPress installation in a couple of minutes, and you would then spend hours on media. Not to mention upload/download timeout issues and other nuisances.

If you use a cloud-based storage service all your media is served from the cloud. If you don’t want to you don’t even have to think about the uploads folder anymore. In this sense, migrating even a very large site could take minutes.

Another related benefit is testing. If your developer uses a local environment to test, images will work just fine as they will be pulled from Amazon servers, as opposed to WordPress looking on a local hard drive.

Scale

Amazon is huge. You will not be able to create a site which their servers can’t keep up with. Say you have a site with 1,000 videos and hundreds of users viewing them at once. Seems like a lot, and it is. Amazon is so big it hosts for Netflix, which is one of the biggest streaming media provides, and is just one of Amazon’s customer. The peace of mind you get knowing that your hosting needs are scalable can’t be understated.

Price

With great size comes great pricing. Amazon really is very cheap, which they can afford because of the huge amount of data they handle. The pricing is a bit complex because they charge for storage, requests and transfer separately, but it’s all very cheap. Take a look at their pricing page.

Security and Control

Some people argue that a big drawback of using such a huge system is that if something goes wrong you can’t really do anything about it. I for one feel a lot safer using Amazon; I would bet on my server crashing before theirs. Unless you have multiple redundancies and a dedicated server team, chances are your current servers are way less dependable than Amazon’s. That being said, I actually do store my media locally and I also back it up as a precaution.

Getting Started

You’ll need a couple of things to get started.

Amazon S3 Account

Once you have an S3 account you’ll have access to your Management Console, which lists all available Amazon Services. You’ll need to click on S3 to get to the S3 management area.

The Amazon S3 management area
The Amazon S3 management area.

The interface above is very much like a file system. Buckets are top-level directories, which can contain any number of files and directories within them. One restriction is that bucket names must be unique across Amazon, so you probably won’t be able to use very generic names.

Security Credentials

At the top of the navigation bar click your account name and then click on the security credentials menu item. This will take you to the Identity Access Management page, or IAM, where we’ll proceed with creating a user. This user will be responsible for interacting with Amazon on your website’s behalf.

Amazon Users
Our list of users in our Amazon Console.

Click on “Users” in the left sidebar and then click on the blue “Create New Users” button. You can create multiple users at once. For now let’s create a single user: “amazontestuser.” Make sure the “Generate an access key for each user” box is ticked before clicking the blue “Create” button. On the next screen you should be able to view access credentials.

Amazon Access Credentials
Access credentials for an Amazon user (not actual credentials).

Make note of these values or download them as Amazon will not show them to you ever again. If you lose your credentials you will have to create a new user.

Setting up Amazon S3

Make sure to install and activate the two plugins I mentioned above – Amazon Web Services and Amazon S3 and Cloudfront. Once done you should see an AWS menu entry in the WordPress admin. Clicking on it will allow you to enter your security credentials.

S3 And Cloudfront plugin setup
Set your credentials in the form. (Click for a larger view.)

To quickly test the plugins, you need to copy in the access key ID and Secret Access Key we created earlier. Then you need to move keys to your wp-config.php file to be safe. We’ll take a look at that later, for now just plop the values in the file.

Click on the “S3 And CloudFront item” within the AWS menu and let’s set things up. You’ll be able to select a bucket from your account using the dropdown. All your uploads will be placed in this bucket. I recommend setting a far future header. Make sure to check the first two checkboxes in the second set to copy to S3 as you upload and point the URLs there as well.

S3 Setup
Settings for my personal website in the S3 plugin.

What We’ve Done

At this stage we could wrap up. We’ve set up a bucket on an S3 server to store your files in and we’ve made sure that URLs point there, too. If you try uploading a file, you should see the URL pointing to Amazon instead of your own domain.

Amazon Served image
Image served from Amazon. Take a look at the URL.

Regretfully, your already existing images will not be sent to the S3 servers automatically. The plugin creators are working on a pro version, which will do this though.

Serving Our Images with CloudFront

There’s one more step you can take to make your readers’ experience even better by serving them media faster. Amazon CloudFront is a CDN (Content Delivery Network), which offers integration “with other Amazon Web Services products to give developers and businesses an easy way to distribute content to end users with low latency, high data transfer speeds, and no minimum usage commitments.”

A CDN is great because it serves images from a location geographically closer to the reader. If your server is in the US, a reader from Europe will experience higher loading times than readers in the US. CDNs tackle this issue by mirroring content to multiple locations and serving files from a location closer to your readers.

Setting up CloudFront is pretty easy. Here’s how:

Go back to your “Management Console” in your Amazon S3 account and click on “CloudFront.” After you’ve activated this service you should see something like the following:

Cloudfront Management
The CloudFront management section

Click on the “Create Distribution” button and then on the first “Get Started” button under the “Web” section. There are quite a few settings here but all we need to do is select our bucket in the first “Origin Domain Name” setting.

Scroll down and click “Create Distribution” and you’re done! It will take 20-30 minutes for the distribution to become active. Make sure to wait until its state changes to “Enabled.”

Back in the Amazon S3 With Cloudfront plugin, go back to the settings and under the “CloudFront” settings fill out the domain name field. This should be the value of the domain name column in the CloudFront List.

Cloudfront Settings
Cloudfront table and the plugin settings

Conclusion

I think Amazon S3 is a powerful and simple solution to a problem. By using it to store my media the burden of keeping it safe is off my back, especially since I usually couple it with VaultPress for greater security.

I can move sites whenever I want and scale my applications naturally without fear of breakage. In addition, using CloudFront lowers loading times for everyone involved, which can contribute directly to the money making potential of a project.

Do you use Amazon S3? What are you experiences with hosting media in the cloud? Let us know in the comments below.