Post filtering by category doesn't seem to work

I have enabled Category Protection add-in for Membership 2. I have a category "Offentlig" and a category "Privat" and have configured protection for "Privat" to shown only to membership "Beboer". However, all post categories show up on my homepage even when not logged in.

If I click on the title of a "Privat" post, I do get the restricted access message, but that's a bit too late - the post has already been shown on the main blog page.