Why Memory exhaust so fast,how to check and fix it

Hosting have 4G Memory now? from 1G upgrade only few days also because Memory exhaust.
Sites just beginning set , only active about 20 plugins and sites 1,796 items
the site is very slow.
Home »Server Status »Service Status
Memory Used 100% (4,194,304 of 4,194,304)

But Memory Viewer plugins show only about 300MBs for my site

Is spam or infection or db?

how to check and fix it?