Directory plugin giving memory limit error when viewing all listings

The Directory plugin is giving me a memory limit issue. I've tried upping the memory_limit in my php.ini file to 1024M and that didn't fix the issue. I dropped it back down to 512M. There hasn't been that much content added since it was working at 256M as well.

It only seems to give me the error when I go to the WP Admin edit page that shows all Listings. I am able to get lists of posts by selecting a taxonomy and then clicking the "Count" link. We do have 980+ listings at this point, but I would think 512M memory limit should be enough to cover that.

Any suggestions?