28 points
Community rank #2181
bhaun
VeteranJust Getting Started
MemberLikes (0)
I previously asked in a post "Static File Rewrite for CDN" to help get support for rewriting all the URL's. I have been trying all the plugins I can find and hacks I could come up. I would like a seamless experience for someone to upload user generated content and have it be directly placed on any object store (like S3).
My concern is the more sites we host the more servers we need and would like to scale out the number of machines we have without having to try to keep each of them in-sync with user generated content.
It seems many, many people have requested these features. I would love to see a new plugin that meets the following requirements. I think this should cover just about everyone's needs I can find on this forum and others.
* Mechanism to publish directly to S3, Google Storage, Swift. Hopefully these are classes which people can create API translations to the actions. I would assume the first and default class would support S3 API.
* Provides the ability to update all static files across all MS sites to one or more asset hosts. This would be the URL to the static content above (wp-content, blogs.dir etc)
* If you choose to use a storage type other then above, you can setup a origin pull for your favorite CDN.
If anyone has any tips and tricks I am all for trying anything out again. Possibly I am missing something.
28 pointsCommunity rank #2181VeteranJust Getting Started
Member
(joined June 2010)
Likes (0)
Phil,
I want to take advantage of a object store like S3 so I can scale. It does not have to be a CDN but this solves both issues. This has nothing to do with the database, we are talking user generated content.
8459 pointsCommunity rank #4Supreme DeityWPMU DEV StaffLifetime member
Lead Developer
(joined May 2009)
Likes (0)
There are some plugins that will duplicate uploads to s3, but to actually store them on s3 only isn't possible via a plugin alone. It takes some server-magic to do that.
We recently began doing this for edublogs, and it required core file modifications as well as some really complicated shell scripting combined with a varnish proxy cache on our file server. Way above my head! This is also what wp.com does from my understanding.
28 pointsCommunity rank #2181VeteranJust Getting Started
Member
(joined June 2010)
Likes (0)
Aaron,
In the past I have used incrond to monitor recursive for any updates and upload directly to S3 using s3cmd. I don't understand Varnish's role but that seems interesting but far more complicated then necessary.
So then I am back to my original question, how about a plugin that properly rewrites all the URL's for all static content (themes, user generated content)? I can't seem to get that working for anything more then the primary site.
1295 pointsCommunity rank #38Supreme DeityMember of the MonthLifetime member
Lifetime member!
(joined November 2010)
Likes (0)
Hmm,
There are quite a few plugins that will do CDN rewrites. However, @bhaun, it sounds as if you're more interested in a plugin that will upload the files directly to the desired CDN and then handle the rewrites.
CDN is great for mirroring (aka origin pull), but in your case it seems as if you're looking for a way NOT to mirror...but instead simply put the files on the CDN but not the local server. CDN is also great this way (a push model instead of pull), but the challenge is getting the files right from WP to CDN.
I'd also love to see a plugin like this. One thought comes to mind: It would be hard to write a plugin that accommodates all possible CDN mechanisms, however many CDNs also support FTP for uploading. So how about a plugin that puts files on a CDN via FTP (while NOT putting the files on the local server), then handles the CDN rewrites.
The rewriting portion could be handled by code from existing CDN plugins easily, IMHO.
38 pointsCommunity rank #1614VeteranJust Getting Started
Member
(joined February 2011)
Likes (0)
i use rackspace cloud server and files with there Akamai network affiliation my content gets distributed throughout all of the there servers and i can purge the content if need be plus there is a plugin in the wp database called w3tc that allows the use of it.
Not to sure if this is what you are looking for plus its cheap
1164 pointsCommunity rank #43Supreme DeityIn TrainingLifetime member
Member
(joined May 2010)
Likes (0)
That was one of the most encouraging aspects of w3tc to me - but it isn't fully compatible with multisite (or 3.1), and we had a number of problems with it. :(
I also LOVED the functionality from w3tc of being able to wrap portions of your code so they were not cached. Sweet! But Quick Cache was far simpler to setup, and the performance results were still amazing.
Responses (8)
Sales & Support Pro (joined March 2010) Likes (0)
Hi Bhaun!
Well, I wish I knew exactly what a CDN is!
I get the gist though and I'm assuming Multi-DB doesn't fulfil your needs?
Anybody else looking for something similar?
Phil
Member (joined June 2010) Likes (0)
Phil,
I want to take advantage of a object store like S3 so I can scale. It does not have to be a CDN but this solves both issues. This has nothing to do with the database, we are talking user generated content.
Lead Developer (joined May 2009) Likes (0)
There are some plugins that will duplicate uploads to s3, but to actually store them on s3 only isn't possible via a plugin alone. It takes some server-magic to do that.
We recently began doing this for edublogs, and it required core file modifications as well as some really complicated shell scripting combined with a varnish proxy cache on our file server. Way above my head! This is also what wp.com does from my understanding.
Member (joined June 2010) Likes (0)
Aaron,
In the past I have used incrond to monitor recursive for any updates and upload directly to S3 using s3cmd. I don't understand Varnish's role but that seems interesting but far more complicated then necessary.
So then I am back to my original question, how about a plugin that properly rewrites all the URL's for all static content (themes, user generated content)? I can't seem to get that working for anything more then the primary site.
Sales & Support Pro (joined March 2010) Likes (0)
Bhaun,
Whilst searching for something completely different I managed to stumble across this:
http://wordpress.org/extend/plugins/cdn-rewrites/
I wonder if that helps...?
Phil
Lifetime member! (joined November 2010) Likes (0)
Hmm,
There are quite a few plugins that will do CDN rewrites. However, @bhaun, it sounds as if you're more interested in a plugin that will upload the files directly to the desired CDN and then handle the rewrites.
CDN is great for mirroring (aka origin pull), but in your case it seems as if you're looking for a way NOT to mirror...but instead simply put the files on the CDN but not the local server. CDN is also great this way (a push model instead of pull), but the challenge is getting the files right from WP to CDN.
I'd also love to see a plugin like this. One thought comes to mind: It would be hard to write a plugin that accommodates all possible CDN mechanisms, however many CDNs also support FTP for uploading. So how about a plugin that puts files on a CDN via FTP (while NOT putting the files on the local server), then handles the CDN rewrites.
The rewriting portion could be handled by code from existing CDN plugins easily, IMHO.
Mark
Member (joined February 2011) Likes (0)
i use rackspace cloud server and files with there Akamai network affiliation my content gets distributed throughout all of the there servers and i can purge the content if need be plus there is a plugin in the wp database called w3tc that allows the use of it.
Not to sure if this is what you are looking for plus its cheap
Member (joined May 2010) Likes (0)
That was one of the most encouraging aspects of w3tc to me - but it isn't fully compatible with multisite (or 3.1), and we had a number of problems with it. :(
I also LOVED the functionality from w3tc of being able to wrap portions of your code so they were not cached. Sweet! But Quick Cache was far simpler to setup, and the performance results were still amazing.
Become a member