Cache crazy - what's your plan?

I read a great article on the site here about caching for WordPress and it really got me interested. So I spent the entire month playing with just about every system out there. So far I have worked with memcache, memcached, redis, varnish, fastcgi_cache via ffpm, apc. I have determined that I am going to be using nginx as it killed apache in every test I ran. I have also jumped away from mysql into Mariadb using Aria tables.

I'm really curious what the experts here have found to be the best setup for large dynamic sites. Considering that you run the wpmu blog network, and this site here with decently large forums it seems a good place to ask for advice.

All of the cache systems were fairly easy to install and get running. Most of them seemed to give similar results if tuned correctly.

What it really comes down to for me is the unforeseen pitfalls that will inevitably occur down the road with each of these systems and which combination seems to work the best.

I have also played around with google pagespeed and of course spdy as I built that into nginx when I compiled it.

So, what do the experts say?

Btw, I am currently running this on the digitalOcean vps on smaller 2GB 2cpu servers, though of course I can increase. Mostly I am just looking to 'RELIABLY' squeeze the absolute best performance out that I can achieve. After talking to AECNU last year, I was really spurred on to become much better at this after hearing that his little wimpy server was absolutely crushing my beast of a colo server.

I'm hoping this thread becomes quite technical as I am not afraid of this anymore and have a decent understanding of all the systems involved. So no need to 'dumb it down', rather lets get together and 'throw down' your best setup :slight_smile:

  • PC

    Hello there,

    Greetings and thanks for posting on the forums.

    While I am not very much into servers and caching and building beasts which are reliable and optimized for best performance however, there are a lot of other members of the community who are into this and I have also called our lead developer @Aaron on this thread so that he can give you more info on what we run :slight_smile:

    Let the discussions begin !

    Cheers, PC

  • anointed

    @PC
    Yeah, that's cool, this thread is mostly just for fun for the serious devs to jump in and show off it they are interested. There are literally hundreds if not thousands of guides on the net when it comes to caching. Everyone thinks they're an expert, some are, some aren't, and I just thought it would be fun to get a perspective from some really large companies who are already doing this with WordPress.

    Really the hardest part is choosing which route to go as there are so many great options for caching today. Then figuring out which works best with WordPress, what the pitfalls are, and what to look out for.

    Most guides never get beyond the initial server and go into multi-server load balancing and cluster setups which is kind of sad. With the advent of crazy cheap cloud hosting today, it would be fun to get some mini clusters setup for WordPress as it can be done very inexpensively now.

  • Aaron

    It's really very highly dependent on your setup and needs.

    A simple WP blog like our wpmu.org is easy, as it's fairly static and read only traffic. You can use any of the full page caching solutions (preferably to memory via memcached or varnish) to get you a WHOLE long way. Combine that with the CDN/caching/security from a service like cloudflare (also used on wpmu.org) and your golden.

    When you are talking about very dynamic sites or multisite where most of the traffic is generated by logged in users like this site or edublogs.org, it's a whole different beast. This is a bit dated (we've grown alot) but might help a bit: https://premium.wpmudev.org/blog/scaling-wordpress-wpmu-buddypress-like-edublogs/

    Indespensable:
    NGINX + PHP-FPM + Opcode cache (APC)
    Memcached object cache

    We use batcache (old, but simple) for full page caching to memcached for logged out users, helps with those occasional spikes to a blog post.

  • anointed

    @Aaron

    Thanks for getting the conversation going.

    Yes, my question is NOT about the basic blog setups, those are boring. I'm more talking about commercial setups for large clusters like edublogs or sites like this one.

    I'm curious Why PHP-FPM + APC + Memcached, vs. other tech like varnish, zend optimize+, fastcgi_cache, pagespeed, spdy.the list goes on and on....

    Today there are so many fantastic choices available and all do their jobs well. I'm not sure that anyone, unless it's a hobby or a job, really understands exactly WHY they chose each piece and how it benefits in the role it plays vs. the competition.

    That 'upper' echelon of reason for using specific tools in large deployments, now that's where the fun begins.

    btw
    while dated, that is a great article!

  • Aaron
    NGINX + PHP-FPM + Opcode cache (APC)
    Memcached object cache
    I'm curious Why PHP-FPM + APC + Memcached, vs. other tech like varnish, zend optimize+, fastcgi_cache, pagespeed, spdy.the list goes on and on....

    All those things do different stuff, and are not necessarily exclusive. NGINX + PHP-FPM + Opcode cache (APC)
    Memcached object cache
    Are backend infrastructure you really can't live without. First is just the most efficient php serving architecture there is for WP. The object cache ties into WP data structures, you can use with APC or others, main thing is to cache WP objects in memory, that will save you 90%+ db queries. And is fully compatible with any code no customizations. That will get you a long way, maybe far enough on a dynamic site.

    Next there is frontend/fullpage caching, the idea of which is to serve the whole page (or most) from cache before hitting php/db at all. That can be varnish, file based, memcached based, fastcgi cache, etc. These are all great but a huge headache to implement and maintain, as you have to customize heavily to keep compatible with dynamic content and site specific plugins. Most of these will just not cache at all logged in users, but there can still be issues if you have dynamic content for logged out users.

  • anointed

    So far I seem to be having really good luck and stats using Nginx PHP-FPM Optimize+ instead of APC, and using pagespeed on the front end as a replacement for varnish. I've not tried it on a really dynamic site using buddypress/bbpress yet, but I do plan on doing so very soon.

    I have also started using a lot of transients within the themes themselves to cache objects like the menu's, recent posts, comments etc and that has helped a lot. I'd say so much so, that I actually wonder why the default WordPress theme doesn't use much more transients.

    I think to date what has made the largest difference is using a single install of WordPress for all 50+ separate sites on the server. This is allowing the cache to only have to work with one set of the WordPress core, themes and plugins, where before I had to have crazy amounts of ram to serve up 50 different copies. It was a pita to setup, but once I got it working, I have had zero issues with it. Combining this with the cache techniques above is giving me really great performance on a relatively small 2GB vps.

Thank NAME, for their help.

Let NAME know exactly why they deserved these points.

Gift a custom amount of points.