Please clarify cross-domain login domain mapping setting

I have a multisite with domain mapping. Since the site was created, as a super admin, I was not able to get access to subsites dashboards without re-logging in each time. It has been a very cumbersome process. Now I find out that the domain mapping plugin is possibly what is not allowing me access and I was told to test changing the Cross Domain Login setting to yes. Well making the change does now give me access but the description of the setting has me concerned about subsites access to other subsites when it says: "Would you like for your members to be logged into all sites within your network regardless of domain name" What access for subsites are they getting with this turned on? Why should they have access to other sites admin dashboards? I just want the super-admin to have the ability to move between sites dashboards. Am I missing something here? What exactly do "members of my network" have access to when this is on?