How do I restrict access to media library files to only those that a user has uploaded him- or herse

I want to allow authors and contributors to add media to their posts, but I don't want them to be able to access the entire Media Library...any ideas for how to accomplish this?