A/B testing plugin issue

We think that the A/B testing is not working as planned. Customers should see theme A or theme B when they go on the site for the first time. Then, when they revisit the site, it should remember which theme they are on?
It seems to be random in that a customer who first sees theme A, then sees theme B the next time they visit - without refreshing their cache.
Can you check that we have set things up correctly?