WP and WP MU slow speed

Hi all,

We are trying to build a software product library using WP, the estimated number of records should be around 300,000 records if not more

When we tried a pilot project by adding like 2000 records as 'posts' with custom fields and while using plugins like WP Cache or WP Super cache, the loading speed of the testing site home page was dramatically slow to the extend that it can go up to 20-30 seconds

Searching and finding products inside was something like a dream due to site speed

I must say that each product have at least 15 custom field with it, none of them are images (meaning we don't load images to DB, but we put paths to images on other websites whom allow us to do so)

So can someone please recommend a solution to overcome this problem?

Thanks in advance for your time

    bpilot

    Hi Aaron,

    Thanks for your prompt reply and help. Unfortunately front page load was affected dramatically as I've explained earlier even though we weren't even trying to load more than 20 records (posts) (( Titles only no content or anything else )) in front page

    Having said so, with the minimum observation, if we increase number of posts....site get slower....lower number of posts.....site get faster....so I don't think it is posts content what's causing this

    Could it be how we extract those 20 records (titles) from DB?! Could it be that displaying 20 records at a time is too much?!....not sure

    Any ideas?

    drmike

    Are custom fields cached? Something in the back of my mind says they;re not.

    Both of the plugins you;re using though are file based. I'd be looking at trying to get them over to memcache if I could.

    There's this but looks like they've given up:

    http://wpscp.trac.armadillo.homeip.net/

    There's batcache although Andy says it;s not as fast which seems strange to me since it;s memcache:

    http://wordpress.org/extend/plugins/batcache/

    Something else to check is to see if wp-cache/wp-super-cache is actually working. I know in the past that you get the text at the bottom of the page saying that it;s served from cache but yet when you look at the cache directory, the files are actually not being saved or even created but the directories are. That's fooled some folks in the past.

    I know there's a way to memcache wp-super-cache but I;m drawing a blank. Anyone care to clue me in?

    bpilot

    Thanks a lot drmike for your time and efforts, highly appreciated

    I guess you have a very good point regarding the custom fields being cached, I mean with 2000 records, each have 15 custom field (not cached) which include sometime big chunk of text, links...etc slowness of the site is something very normal here

    I think memcache wp-super-cache it will be really outstanding...So hopefully anyone will be able to help us by a clue on how to do that...

    Tom Eagles

    Hi There

    The way memcache works is to store the results of queries in memory, so if a query to the database is made its stored in the servers RAM and the next visitor gets the result without the query being run.

    Memcached is being phased out, its no longer available as a plugin in cpanel and whm anymore but it can still be installed obviously if you have your own dedicated server this is very easy. If not it would depend if your host would allow it as its a server level caching system.

    Another alternative in your case would be to possibly use an external database, we have a few huge sites with records exceeding yours and what we did was as data was entered some of it was ported out to an external database (on a server just running mysql) and we did all the querying there and using wp to display it.

    Alternatively look at a Hyperdb solution aswell, it all comes down to hosting capabilities when getting results from big tables, for example if you are on shared hosting your table will be on a mysql server many others share and therefore it will be bottle necked.