Large Wordpress User Base

I'm setting up a WP site that will have 10 000-50 000 blogs from the first week. Are there any limits? Can WP handle these numbers on one server? Does mysql impose any limitations, considering the number of tables it creates is rather bloated?

Thanks