E-Newsletter Unsubscribe clearing ALL groups

I have several groups created for the e-Newsletter. I noticed that when a user tries to remove one or even a couple subscriptions from their Dashboard, and hits UnSubscribe, it removes them from all the groups. I am wondering if there is something I need to do to not allow that. I would like the user to be able to specify certain groups they do not wish to be subscribed to and to keep the ones they do want to be subscribed to.