How to optimize WordPress on a VPS

On my website at HighTechRealm I’ve been dealing with atrocious load times over the past few weeks and although I have W3 Total Cache with PHP APC caching enabled, and I just signed up for an Amazon CloudFront account, I am still having horrible speeds.

I’m not sure if this is a hosting issue because I’m on a 1GB VPS with barely any CPU usage and only 60% RAM usage (I have a couple other sites sliced on different sections) or if this is something that can be corrected with tweaks. I’ve even done image compression in addition to minification & browser compression but nothing seems to do anything.

Would installing Varnish or some other reverse proxy help?

I’m at a total loss here so any and all insights are very much appreciated.

Thanks in advance,

Charles