31 pointsStarting to get into this DEV thingI'm helpful
Tomas
Member
—
9th December 2010
Hi, I need some advice.
I have searched the net for suggestions, pros, cons but not really found anything good.
We are in the midst to setup a platform. We will have several lb webservers with shared storage, memcached servers and a larger DB server.
What we would like to have is a flexibility to add more db-servers and dbs as we go along.
Say we start with 1 db-server running 256 db's and in one year we will need to add another db-server with 256 db's.
Possible with any?
Hi, I need some advice.
I have searched the net for suggestions, pros, cons but not really found anything good.
We are in the midst to setup a platform. We will have several lb webservers with shared storage, memcached servers and a larger DB server.
What we would like to have is a flexibility to add more db-servers and dbs as we go along.
Say we start with 1 db-server running 256 db's and in one year we will need to add another db-server with 256 db's.
Possible with any?
I am not aware of any solution that makes the process of adding databases down the road in intervals that way an 'easy' solution.
Why not just start with the higher number of databases from the start? I'll ask Barry to comment further on this as he's more familiar with the multi-db plugin and can guide ya a bit further.
I can't really comment on shardb or hyperdb as I haven't used either ( hyperdb i think, the last time i looked anyway, was more to do with table partitioning ).
Say we start with 1 db-server running 256 db's and in one year we will need to add another db-server with 256 db's.
Depends on how you are thinking :) In the sense that you then have 514 databases and want the blogs tables split across all of them then no. you'd have to move from 256 to 4096.
If you mean just for loading, then you could replicate your 256 databases on your second server and use one server for read and writes and the second just for reads - you'd use standard mysql replication for that though so the database class doesn't really come in to it.
Ok so if I would actually be able to add another db-server later running i.e. Multi-DB, there would be no easy way to automatically have new blogs have their tables created on the new db's on the new db-server?
Sorry but I'm not really into this "multi db" thinking yet :)
No, multi-db needs to use either 16, 256 or 4096 databases - it's a mathematical thing, think hexadecimal numbers of 1, 2 or 3 digits/chars.
So if you have 256 databases and add 256 databases you have 512, which isn't one of the magic numbers :). If you have 256 and add 3840 databases on another server, then you are fine and can run a move script to shift your existing tables into their new correct locations and then carry on with the 4096 db setup running across the two db servers, you can then, later if you want, split some of your new 3840 databases off to a third server and carry on with 3 servers.
As long as you always have 16, 256 or 4096 databases you can spread them amongst as many servers as you want and spread them around those servers however you want.
When you add in replication it gets more complex, because then you can have different numbers of databases as some will mirror others for read and/or write purposes - but you'll always have the core number.
*note - I am not including the global and any potential vip databases in this, because that just clouds the issue.
Hi Barry, thanks for your reply.
So any possibility to run 4096 dbs on one server and then later move 2048 to another db-server? If so, is there actually a "move script" for this?
6593 pointsLike some sort of WPMU DEV GodExceptionally helpfulLifetime member
Keeper of the Dark Chocolate
—
10th December 2010
#
That's what I suggested up there. Start with one server and then after a bit split them between 2 different servers. YOu'll have to physically move the databases though, not add to them. Start with 256 on one server and then move 128 of those onto a different server.
I would not put 4096 databases on a single box though. Do that and you'll run into errors with how many files you have open at one time.
6593 pointsLike some sort of WPMU DEV GodExceptionally helpfulLifetime member
Keeper of the Dark Chocolate
—
12th December 2010
#
I have to admit that I;m not a big fan of hyperdb, just as a matter of ethics. A certain someone who shall remain nameless was going around for about 18 months saying that the division of the database code was in there already even though some folks kept pointing it out to him that is wasn't. An aside on wp-pro that it wasn't and it was in the next day as a "brand new feature" or something like that.
So if you start with 16 and want to grow to 256 or you start with 256 and want to grow to 4096 what needs to be done? Just repeat the process again and let mysql tell you that some of the databases already exist? Any other potential issues? Obviously you need a lot of mysql planning and admin to grow your databases.
I know that wordpress.com has multiple datacenters. Does multi-db work for this type of environment? It would seem it's more than just mysql replication for multiple data centers. You would want your local webservers getting data from local database servers which I think would require more complex config.
Responses (12)
Sales & Support Lead — 9th December 2010 #
Hiya spotlife,
First off, welcome to WPMU DEV! :D
I am not aware of any solution that makes the process of adding databases down the road in intervals that way an 'easy' solution.
Why not just start with the higher number of databases from the start? I'll ask Barry to comment further on this as he's more familiar with the multi-db plugin and can guide ya a bit further.
Thanks!
Developer — 9th December 2010 #
I can't really comment on shardb or hyperdb as I haven't used either ( hyperdb i think, the last time i looked anyway, was more to do with table partitioning ).
Depends on how you are thinking :) In the sense that you then have 514 databases and want the blogs tables split across all of them then no. you'd have to move from 256 to 4096.
If you mean just for loading, then you could replicate your 256 databases on your second server and use one server for read and writes and the second just for reads - you'd use standard mysql replication for that though so the database class doesn't really come in to it.
Keeper of the Dark Chocolate — 10th December 2010 #
Either that or move half of the 256 databases over to one server and leave the other half where they are after a year.
Member — 10th December 2010 #
Hi and thanks to you all!
Ok so if I would actually be able to add another db-server later running i.e. Multi-DB, there would be no easy way to automatically have new blogs have their tables created on the new db's on the new db-server?
Sorry but I'm not really into this "multi db" thinking yet :)
Thanks again
Developer — 10th December 2010 #
No, multi-db needs to use either 16, 256 or 4096 databases - it's a mathematical thing, think hexadecimal numbers of 1, 2 or 3 digits/chars.
So if you have 256 databases and add 256 databases you have 512, which isn't one of the magic numbers :). If you have 256 and add 3840 databases on another server, then you are fine and can run a move script to shift your existing tables into their new correct locations and then carry on with the 4096 db setup running across the two db servers, you can then, later if you want, split some of your new 3840 databases off to a third server and carry on with 3 servers.
As long as you always have 16, 256 or 4096 databases you can spread them amongst as many servers as you want and spread them around those servers however you want.
When you add in replication it gets more complex, because then you can have different numbers of databases as some will mirror others for read and/or write purposes - but you'll always have the core number.
*note - I am not including the global and any potential vip databases in this, because that just clouds the issue.
Member — 10th December 2010 #
Hi Barry, thanks for your reply.
So any possibility to run 4096 dbs on one server and then later move 2048 to another db-server? If so, is there actually a "move script" for this?
Again, sorry if this is a noob question
Keeper of the Dark Chocolate — 10th December 2010 #
That's what I suggested up there. Start with one server and then after a bit split them between 2 different servers. YOu'll have to physically move the databases though, not add to them. Start with 256 on one server and then move 128 of those onto a different server.
I would not put 4096 databases on a single box though. Do that and you'll run into errors with how many files you have open at one time.
Volunteer support dude — 10th December 2010 #
Here's a radical idea, can we get Cassandra working in WP?
http://cassandra.apache.org/
Keeper of the Dark Chocolate — 11th December 2010 #
Didn;t see to be much interest:
http://wordpress.org/extend/ideas/topic/wordpress-cassandra
Member — 11th December 2010 #
Thanks drmike, I'll go with your suggestion
Keeper of the Dark Chocolate — 12th December 2010 #
I have to admit that I;m not a big fan of hyperdb, just as a matter of ethics. A certain someone who shall remain nameless was going around for about 18 months saying that the division of the database code was in there already even though some folks kept pointing it out to him that is wasn't. An aside on wp-pro that it wasn't and it was in the next day as a "brand new feature" or something like that.
Member — 2nd March 2011 #
So if you start with 16 and want to grow to 256 or you start with 256 and want to grow to 4096 what needs to be done? Just repeat the process again and let mysql tell you that some of the databases already exist? Any other potential issues? Obviously you need a lot of mysql planning and admin to grow your databases.
I know that wordpress.com has multiple datacenters. Does multi-db work for this type of environment? It would seem it's more than just mysql replication for multiple data centers. You would want your local webservers getting data from local database servers which I think would require more complex config.
Become a member