Incomplete registrations appearing in membership list

There are two parts of the registration - the signup form, and then the paypal component (either free or paid). When members complete the registration form but don't follow through and complete the paypal component I can see that they are still added as users on the website - they just don't have any membership access level assigned. Is there any way to prevent this from happening until they have actually completed the second part of the registration?