Limiting # of Appointments at a location

As I am booking appointments I have been playing with it and have added locations as well as having the service providers. I left the # of services to zero because I want each location based on the service providers schedules but what I am running into is even though I have providers assigned to different locations if a specific location is selected it is allowing that location to be booked even though there are no providers left to support the location.
Right now there are 4 providers setup 2 at one location and 1 at each of 2 other locations. This allows 4 appts to be booked at any location rather than 2 -1 -1 based on location availability of providers.