Single Password for Blog not working

I can’t seem to get the new Single password setting in Multisite Privacy to work. I’m testing it on 3.3.1 with Buddypress 1.5.5 on the main domain. The issues I’m having though are when implementing the setting on subdomain blogs.

On some occasions I am faced with a password page that won’t accept the correct password (but doesn’t tell me it’s wrong) and on other occasions, although I’ve set a password, the site is publicly available. I’ve made a screencast to try to better illustrate what I’m referring to.

Other than buddypress, I’ve deactivated all other plugins on the site to try to triage the issue.