I have a site with approx. 20 separate blogs. When a new user creates a new blog, the following url is created:
with “newblog21” being the user login.
I would like to set it up so that all existing blogs and newly created blogs are blocked from all crawlers. Keeping in mind that the home page ('http://www.mysite.com') and pages created from the home site (i.e. "http://www.mysite.com/aboutus") is still crawl-able.
Any direction is appreciated.