Sending an email seems to take for ever from the Wordpress Site

I have installed wordpress v4.0 on a dedicated Apache server. I have also installed sendmail to take care of php mail requests as required by a large number of mail type programs. I want to keep it this way instead of using SMTP configs.

So, the issue. It appears to be taking for ever to send out an email from a request on the system. I know that this should not be the case and wondering if anyone out there has configured sendmail to work specifically with wordpress.

I do host several other sites on same server and therefore need to think about whether sendmail can handle multiple domain email sends.

Appreciate any views.