Multisite Admin bar-shows up after logging out

I have a multisite installation ( and when I log out of the site entirely I can still see the admin bar at the top of the browser as If I was still logged in. When I try to click on any of the buttons I get redirected back to log in once again. This behavior is evident on all browsers, and mobile devices.