Hustle Plugin Site Lag

With Hustle on the site, we're having a massive increase in site load of about 9-10 seconds. Disable it, and the lag immediately goes away.

Running through GTMetrix, it shows 6 individual put requests (one for each popup we've setup) to:

As we like the setup of hustle, and have already setup all our needs for it, we'd like to keep using it if we can resolve the issue.

From other online posts, we have already tried clearing/disabling cache(s), disabling a handful of plugins, and limiting heartbeat (also admin-ajax related), none of which made more than about half a second difference.

What can we do?