How much is too much seralized data in a user meta field?

I'm using a plugin called WP Favorite Posts ( in tandem with Membership, to add members only CPT IDs to user meta. The plugin stores this information as seralized data. I think I already know the answer to this question, but just want to verify before I venture down the path of rewriting how the data is stored.

Right now, the CPTs posts are being imported at a rate of 250/week as 'publish' status. Upon the next weekly import, the old posts are set to private.

There are some custom fields associated with the CPT, numbers, that sum 'publish', and then sum 'all' based on the meta key with the serialized value.

At this rate, a user could have 12,000 serialized CPT IDs over the course of a year.

I believe this is WAY too much and I'm thinking of writing a plugin where the client can simply click a "clear" button in the admin. This button will set the meta value for the favorites to null, after it calculates the total and updates another user meta field (where I will store the sum values).

Does it sound like I'm on the right track with this? I want this to not get incredibly SLOW over time. Does this make sense?

Any insight would be GREATLY appreciated.