Server and Database Structure for many sites

Dear wpmudev team,

We run a Wordpress multisite with over 1200 websites. Unfortunately, due to the high number of websites, our database is probably overloaded and the load time increases, the more sites on the multisite run. As a superadmin, we often get 500 errors in the backend.

Can you give us some tips on how we can optimize the database structure?

Should we distribute the sites to multiple databases?
Or maybe we should even use several Wordpress multisite installations?

We grow very fast. By 2030, we expect to have 60,000 websites. Therefore, we should now already set up a structure that will be sufficient in the long term for these goals.

At the moment we are already using a test, staging and live system on a fast server (12 CPU cores, 256 GB RAM, 400 GB SSD storage (2 x 400 GB in RAID1), 100,000 IOPS). But here too we have the problem that backups and reflections of the systems take many hours.

It would be great if you could give us some tips database and server structure. Also like articles or forum posts, etc.

Best Regards,
Mario Schneider
Founder of Spreadmind

  • Adam Czajczyk

    Hi Mario,

    I hope you're well today and thank you for your question!

    Let me say first: that's already an impressive setup - with that many sites - but the growth you're talking about is simply mind-blowing! Please accept my sincere congratulations! It's really satisfactory for us - "WordPress clan" - to hear about such things :slight_smile:

    As for the question.

    Yes, "splitting" either database to multiple databases or entire setup to multiple setups, probably could help. But personally I think that it wouldn't be that beneficial in a long run. With that scale of the business you'd most likely need some additional - custom developed - tools to efficiently and easily manage that all. Also, "splitting" database of a single setup - if done "on WordPress level" - is not that easy and reliable. I got doubts if benefits would overweight difficulties and costs...

    Instead, I'd rather consider it - apart from "standard" WP optimization practices of course - entirely a "server/infrastructure" thing. The server you are running that on sounds like some sort of dedicated server, right? It sounds powerful but still that might not be the most efficient solution.

    With that scale (and predicted growth) I'd definitely go for Amazon cloud services. I'm not an expert with their specific products but they can handle that easily. There are services that can be used for hosting static resources (e.g. files in Media Library), there are dedicated DB services from them, an efficient CDN (that it would be a subject for further consideration whether to use it or StackPath)...

    It would take an expert to carefully plan the infrastructure initial configuration, mostly in terms of using "right tools for the job" and with further scalability in mind, but I think that would be the most reasonable (in both "technical" and "cost efficient" terms) direction.

    I usually avoid naming any specific providers but in this case it's a bit different and there's really not that many providers that do actually provide that kind of infrastructure so Amazon sounds like a good choice to me :slight_smile:

    That being said, I'm not an expert in that area and even though I think that would be a good way, I think you will find this presentation by our own CTO very useful:

    https://www.youtube.com/watch?v=UDJdsFnqkeM

    and a follow-up article:

    https://premium.wpmudev.org/blog/scaling-dynamic-sites/

    Best regards,
    Adam

Thank NAME, for their help.

Let NAME know exactly why they deserved these points.

Gift a custom amount of points.