"Never see this message again" option for Pop UP plugin doesn't work

I'm trying to use the "never see this again" or the PopUp shown less than features, and neither are working for me.
I've tried loading the popup both in the footer and with WordPress AJAX. The popup continues to show up.
The site is hosted on Pantheon and is cached, but I've tried removing the caching and it still doesn't seem to work.