Can the members directory be blog site specific?

Is it possible to have the member directory show the members of a sub site? I am interested in enhancing the member directories so it only shows the members of the sub site that you are on. I have a tool now that I am using, but it pulls all of the members across all sites because it is only querying the wp_users table. I'm wondering if there is a way to have distinguish the particular users of that sub site somehow. I believe you have to incorporate the wp_usermeta table. Any thoughts?