Protecting attachments from being downloaded if the URL is known

I'm building a member's area on a site within a multisite installation, and I've noticed that the attachments to a post are viewable/downloadable if you know the direct URL to the file. Does anyone have ideas on how to protect attachments unless you are logged in?

I've got a post running on wordpress.stackexchange if anyone has any ideas/wants to see my full question...

http://wordpress.stackexchange.com/questions/50059/how-to-protect-post-attachments-related-to-a-custom-post-type-from-non-logged-i/50406#50406