Subscribe by Email sends about 50-60 emails with daily digest

I have daily digest setup to send only posts from a certain category. It works fine, but it sends out about 50-60 emails to each user, everytime the schedule comes up.

I notice that of those 50-60 emails in each blast, sometimes the emails contain the full list of the previous 24 hours, while other times they are partial.

Any ideas how to fix this ?