Don't overwrite non-registered users gravatar with the mystery man

The vast majority of commenters are not members of my sites. Now that I've installed the Avatars plugin all of the non-registered readers avatars have been changed from their gravatar picture to the default mystery man. I'd like the site to default to the Gravatar if the custom avatar isn't uploaded before going to mysteryman. Is there anyway to do this.