14 points
Community rank #4037
floris
SophomoreJust Getting Started
MemberLikes (0)
Hello there,
I'm with Rackspace and am trying to configure a multi-site installation on cloudsites to upload media directly onto cloudfiles, bypassing the blogs.dir completely. Have tried lots, including the CDN Tools plugin, but this still populates my ever-growing wp-content folder. Now running out of ideas...
14 pointsCommunity rank #4037SophomoreJust Getting Started
Member
(joined November 2009)
Likes (0)
Thank you guys,
It seems to me that the W3TC solution still requires your static blogs.dir to reside in the default (relative/local) locations. As I understand it, the plugin will then origin-pull things from there before serving it via the specified CDN. I will probably need to set this up also, but for now this doesn't solve our need for an external origin (e.g. media.domain.com). Or am I missing something completely?
Extra drive space could be a fix, but I prefer to use better-value-long-term file storage and serve via CDN from there. We're just past 2TB and counting fast, so how nice if you could just FTP things away from your server with some wp-config additions. Wishfulness...
14 pointsCommunity rank #4037SophomoreJust Getting Started
Member
(joined November 2009)
Likes (0)
PS.
On that note. Storing on Cloud Files is my preferred option, but am not fixed to it. If there's a better (or easier) solution, like a recent working version of the Amazon S3 plugin, I will probably jump at it.
6593 pointsCommunity rank #9Supreme DeityMember of the MonthLifetime member
Keeper of the Dark Chocolate
(joined July 2007)
Likes (0)
I'm really not in the mind set to juggle numbers but a local drive works out better cost wise than using a CDN, at least for us. For example it costs me something like $85 for a 250 gig server quality hard drive. Granted I do my own hardware work, buying the drives in bulk and I can walk into my datacenter which is not something others can do.
Or am I missing something completely?
Actually that's how wp.com does theirs. I forget the exact numbers but something like 20% of the uploads generates 80% of the file traffic and they now serve that locally from their own servers. Everything else falls back to the CDNs.
They used to push everything to the CDN when they first started doing it (Actually I think they used 2 different ones for file security as well as local storage with the pulling from one of the CDNs.) but they moved to this method after some downtime at the CDN.
edit: Some of us have asked James and crew to take a look at the Tan plugin but I believe on the most recent polling here in the forums, it was pretty much down the list.
14 pointsCommunity rank #4037SophomoreJust Getting Started
Member
(joined November 2009)
Likes (0)
Thank you, and yes, I know. Numbers...
I'm currently looking at $37.50/250GB at Cloud Files compared to $35/250GB on S3, not counting bandwidth or requests. It's a pain I have to think in terms of TBs (and not have my own data centre), where we're growing fast but not necessarily get rich quick.
;-)
Will continue my search and hope some sort of solid solution comes out of this. It seems so simple, yet it's not really.
29 pointsCommunity rank #2096VeteranJust Getting Started
Member
(joined September 2010)
Likes (0)
Perhaps some existing solutions (CDN Tools, CDN Sync Tool) could be modified to remove the file once uploaded and rewrite all references? The file would be there long enough to create the record and send to the CDN, but it would never need to be referenced locally again.
8447 pointsCommunity rank #4Supreme DeityWPMU DEV StaffLifetime member
Lead Developer
(joined May 2009)
Likes (0)
Does cloudfiles give you drive level access? I mean if you can symlink to it then its easy.
We recently moved edublogs to store newer uploads on S3. Took some advanced scripting and server config, along with some core hacks as well though. We serve them with a varnish frontend to save on bandwidth. We've yet to migrate older uploads though, a big task indeed!
6593 pointsCommunity rank #9Supreme DeityMember of the MonthLifetime member
Keeper of the Dark Chocolate
(joined July 2007)
Likes (0)
I mean if you can symlink to it then its easy.
I was thinking this last night as well. I was wondering if you could setup one of those pockets or whatever each host calls them, set it up as a subdirectory and just let the install access it as part of the drive.
14 pointsCommunity rank #4037SophomoreJust Getting Started
Member
(joined November 2009)
Likes (0)
Thanks guys, think we're going to try and create some sort of plugin for W3TC.
Meanwhile, I have decided to move my system to Media Temple VE servers, with my files pushed to CDN origin servers at EdgeCast. Functionality and API are the most comprehensive I have come across so far, including direct access.
Things are far from solved, but are definitely looking up. Throwing sym-linking into the mix and will keep you updated.
14 pointsCommunity rank #4037SophomoreJust Getting Started
Member
(joined November 2009)
Likes (0)
Update.
Have moved away from Media Temple and their ProCDN. It appears that MT have disabled direct drive access via FTP, which limits file uploads to 50MB and only allows it through a web interface. Crazy, it proved to be rather useless and a big time waster...
Meanwhile.
Looking at moving back to the Rackspace's combination of Cloud Server with Cloud Files, I came across this article which explains how to enable some sorts of drive mounting on Ubuntu as mentioned above. I'm now trying to recreate my blogs.dir as a direct link to my Cloud Files container.
Responses (12)
Member (joined April 2009) Likes (0)
Hiya floris,
First off, welcome to WPMU DEV. I'll ask a couple of our devs to respond to this, but I belive W3 Total Cache also allows for this:
http://wordpress.org/extend/plugins/w3-total-cache/
Anybody else have some suggestions?
Thanks!
Keeper of the Dark Chocolate (joined July 2007) Likes (0)
I could have sworn that only W3TC allows for multisite installs but I never really looked into this. I just drop in a new hard drive.
Member (joined November 2009) Likes (0)
Thank you guys,
It seems to me that the W3TC solution still requires your static blogs.dir to reside in the default (relative/local) locations. As I understand it, the plugin will then origin-pull things from there before serving it via the specified CDN. I will probably need to set this up also, but for now this doesn't solve our need for an external origin (e.g. media.domain.com). Or am I missing something completely?
Extra drive space could be a fix, but I prefer to use better-value-long-term file storage and serve via CDN from there. We're just past 2TB and counting fast, so how nice if you could just FTP things away from your server with some wp-config additions. Wishfulness...
More ideas would be fabulous!
Thanks again
Member (joined November 2009) Likes (0)
PS.
On that note. Storing on Cloud Files is my preferred option, but am not fixed to it. If there's a better (or easier) solution, like a recent working version of the Amazon S3 plugin, I will probably jump at it.
http://tantannoodles.com/toolkit/wordpress-s3/
Possibly a good basis for a new plugin if other options fail...
F.
Keeper of the Dark Chocolate (joined July 2007) Likes (0)
I'm really not in the mind set to juggle numbers but a local drive works out better cost wise than using a CDN, at least for us. For example it costs me something like $85 for a 250 gig server quality hard drive. Granted I do my own hardware work, buying the drives in bulk and I can walk into my datacenter which is not something others can do.
Actually that's how wp.com does theirs. I forget the exact numbers but something like 20% of the uploads generates 80% of the file traffic and they now serve that locally from their own servers. Everything else falls back to the CDNs.
They used to push everything to the CDN when they first started doing it (Actually I think they used 2 different ones for file security as well as local storage with the pulling from one of the CDNs.) but they moved to this method after some downtime at the CDN.
edit: Some of us have asked James and crew to take a look at the Tan plugin but I believe on the most recent polling here in the forums, it was pretty much down the list.
Member (joined November 2009) Likes (0)
Thank you, and yes, I know. Numbers...
I'm currently looking at $37.50/250GB at Cloud Files compared to $35/250GB on S3, not counting bandwidth or requests. It's a pain I have to think in terms of TBs (and not have my own data centre), where we're growing fast but not necessarily get rich quick.
;-)
Will continue my search and hope some sort of solid solution comes out of this. It seems so simple, yet it's not really.
Many thanks!
Keeper of the Dark Chocolate (joined July 2007) Likes (0)
Not a problem. Maybe Aaron or Barry can come up with an idea to two when they see this thread.
Member (joined September 2010) Likes (0)
Perhaps some existing solutions (CDN Tools, CDN Sync Tool) could be modified to remove the file once uploaded and rewrite all references? The file would be there long enough to create the record and send to the CDN, but it would never need to be referenced locally again.
Lead Developer (joined May 2009) Likes (0)
Does cloudfiles give you drive level access? I mean if you can symlink to it then its easy.
We recently moved edublogs to store newer uploads on S3. Took some advanced scripting and server config, along with some core hacks as well though. We serve them with a varnish frontend to save on bandwidth. We've yet to migrate older uploads though, a big task indeed!
Keeper of the Dark Chocolate (joined July 2007) Likes (0)
I was thinking this last night as well. I was wondering if you could setup one of those pockets or whatever each host calls them, set it up as a subdirectory and just let the install access it as part of the drive.
Hmmm....
Member (joined November 2009) Likes (0)
Thanks guys, think we're going to try and create some sort of plugin for W3TC.
Meanwhile, I have decided to move my system to Media Temple VE servers, with my files pushed to CDN origin servers at EdgeCast. Functionality and API are the most comprehensive I have come across so far, including direct access.
Things are far from solved, but are definitely looking up. Throwing sym-linking into the mix and will keep you updated.
Floris
Member (joined November 2009) Likes (0)
Update.
Have moved away from Media Temple and their ProCDN. It appears that MT have disabled direct drive access via FTP, which limits file uploads to 50MB and only allows it through a web interface. Crazy, it proved to be rather useless and a big time waster...
Meanwhile.
Looking at moving back to the Rackspace's combination of Cloud Server with Cloud Files, I came across this article which explains how to enable some sorts of drive mounting on Ubuntu as mentioned above. I'm now trying to recreate my blogs.dir as a direct link to my Cloud Files container.
Promising no?
http://sandeepsidhu.wordpress.com/2011/02/21/mounting-cloud-files-using-cloudfuse-into-ubuntu-10-10/
Become a member