I have claimed 1bog.org in google webmaster tools, but I don't know if
I should submit this sitemap, and will submitting the sitemap unindex
pages that are not in the site map? There are many indexed pages by
Google that don't actually exist at the moment I would love to have
them unindexed but am not willing to go through them individually, however if this unindexes everything that is not in the site map I'd lose even my homepage.
I have claimed 1bog.org in google webmaster tools, but I don't know if
I should submit this sitemap, and will submitting the sitemap unindex
pages that are not in the site map? There are many indexed pages by
Google that don't actually exist at the moment I would love to have
them unindexed but am not willing to go through them individually, however if this unindexes everything that is not in the site map I'd lose even my homepage.
Founder & CEO
—
10th February 2009 (3 years ago)
#
I was unaware that it didn't list pages - has anyone else experienced this?
And yeh, no admin interface - it's very simple (so it's light on your systems resources)
I can tell you one thing though - you'll definitely not be unindexing pages in Goog via webmaster tools - they don't really let you do that apart from by robots.txt so that'll be fine.
Also, while sitemaps are definitely 'nice to have' and may speed up you inclusion in google (so, for a new / initial site they are a must), in the long run they are unlikely to make any different to the indexing patterns / results.
7430 pointsLike some sort of WPMU DEV GodMindblowingly helpful memberLifetime member
Erstwhile founder
—
12th February 2009 (3 years ago)
#
Hi Guys,
I just looked at the code for this one and it definitely doesn't include pages. I'm wondering how big of an issue this is though. 99% of the time pages don't change that often and Google would crawl the pages when it hits the posts from the sitemap.
7430 pointsLike some sort of WPMU DEV GodMindblowingly helpful memberLifetime member
Erstwhile founder
—
13th February 2009 (3 years ago)
#
It would require recoding a section of the plugin. So unless someone really thinks they need to be included and can provide reasoning for it then I'm not going to worry with it.
Listing pages in sitemaps is a "must have" for me as most of my MPMU installs are used for business websites where pages, and not posts make up the meat of the site (they're behaving like a mini CMS).
I didn't realize this plugin was (in my view) incomplete before joining premium.
6593 pointsLike some sort of WPMU DEV GodExceptionally helpfulLifetime member
Keeper of the Dark Chocolate
—
3rd October 2009 (2 years ago)
#
I'd just use Arne's sitemap plugin then if you need more control of what's in there. With the sitemap plugin here, as well as the AIO SEO plugin that comes with it, it's more of a hands off feature so something is given to the endusers and they don't have to worry about it or deal with settings.
@drmike - thanks for taking the time to look at this.
I prefered to hack the simple-sitemaps.php file rather than install Arne's plugin (very complete, but overkill in my opinion).
The solution was to replace this line (line 66 in my file) : $latestposts = get_posts( 'numberposts=' . $this->totalposts . '&orderby=date&order=DESC' );
with this :
$site_pages = get_posts( 'numberposts=-1&post_type=page');
$site_posts = get_posts( 'numberposts=' . $this->totalposts . '&orderby=date&order=DESC' );
$latestposts = array_merge($site_pages, $site_posts);
This adds *all* of the site's pages to the sitemap + the 25 latest posts. That 25 posts limit is something I don't quite understand either, I seem to recall there being a 500 links limit per sitemap file so why restrict it to 25? Clearly a limited sitemap file is failing in it's function to inform spiders about a site's content.
I'm going to tweak this file to make it more efficient (in my view), adding in pings to other search engines too - is there a way to commit updates to plugins or should I realease my version as a separate project - don't want to tread on any toes here, but from what I understand about the GPL I'd be entitled to do so.
6593 pointsLike some sort of WPMU DEV GodExceptionally helpfulLifetime member
Keeper of the Dark Chocolate
—
4th October 2009 (2 years ago)
#
I seem to recall there being a 500 links limit
You may be thinking of many of the free sitemap providers who have such a limit. (In fact., I just answered such a question over on digitalpoint) Google's limit is 10 megs I believe in file size or 50k urls.
I'd have to dig for the reference but James and Andrew have mentioned that the sitemap plugin was a simplified solution for large installs that wouldn't tax resources. It's also designed for giving endusers a solution that didn't make them worry about settings and understanding what they were doing. For those who needed more features or a complete solution, they were pointing at other folks' plugins like Arne's.
And in my opinion, sitemaps were created just to give SEO folks something to drool over. I've never seen any improvement in spidering with them. Also if it really was a tool for such, there would be an option to figure out which urls were indexed when compared against the sitemap.
But that's just me. (Hmmm, I've been saying that a lot lately, haven't I?)
there would be an option to figure out which urls were indexed when compared against the sitemap
Google's Webmaster Tools does this
I understand what you mean about the impact of sitemaps being uncertain or at the very least, hard to measure. I guess my point of view regarding SEO in general is that if it doesn't penalize you, doing *something* is better than doing nothing :)
I've added pings to Bing and Ask to my local version of this plugin - will release if anybody expresses interest in it.
6593 pointsLike some sort of WPMU DEV GodExceptionally helpfulLifetime member
Keeper of the Dark Chocolate
—
4th October 2009 (2 years ago)
#
Google's Webmaster Tools does this
No it doesn't. It just tells you how many urls Google has indexed and how many are in the sitemap. It also tells you thinks like restricted by robots.txt and 404s when it tries to index and whatnot. Doesn't give specifics as to what's missing from the index.
From their own site:
See how Google crawls and indexes your site and learn about specific problems we're having accessing it.
View, classify, and download comprehensive data about internal and external links to your site with new link reporting tools. Find out which Google search queries drive traffic to your site, and see exactly how users arrive there.
Tell us about your pages with Sitemaps: which ones are the most important to you and how often they change. You can also let us know how you would like the URLs we index to appear.
Responses (13)
Founder & CEO — 10th February 2009 (3 years ago) #
I was unaware that it didn't list pages - has anyone else experienced this?
And yeh, no admin interface - it's very simple (so it's light on your systems resources)
I can tell you one thing though - you'll definitely not be unindexing pages in Goog via webmaster tools - they don't really let you do that apart from by robots.txt so that'll be fine.
Also, while sitemaps are definitely 'nice to have' and may speed up you inclusion in google (so, for a new / initial site they are a must), in the long run they are unlikely to make any different to the indexing patterns / results.
Member — 10th February 2009 (3 years ago) #
thanks james
Erstwhile founder — 12th February 2009 (3 years ago) #
Hi Guys,
I just looked at the code for this one and it definitely doesn't include pages. I'm wondering how big of an issue this is though. 99% of the time pages don't change that often and Google would crawl the pages when it hits the posts from the sitemap.
Thanks,
Andrew
Member — 13th February 2009 (3 years ago) #
Not sure , but it couldn't hurt to have them included if it's not much work?
Erstwhile founder — 13th February 2009 (3 years ago) #
It would require recoding a section of the plugin. So unless someone really thinks they need to be included and can provide reasoning for it then I'm not going to worry with it.
Thanks,
Andrew
Member — 2nd October 2009 (2 years ago) #
Has this issue been resolved at all?
Listing pages in sitemaps is a "must have" for me as most of my MPMU installs are used for business websites where pages, and not posts make up the meat of the site (they're behaving like a mini CMS).
I didn't realize this plugin was (in my view) incomplete before joining premium.
Keeper of the Dark Chocolate — 3rd October 2009 (2 years ago) #
I'd just use Arne's sitemap plugin then if you need more control of what's in there. With the sitemap plugin here, as well as the AIO SEO plugin that comes with it, it's more of a hands off feature so something is given to the endusers and they don't have to worry about it or deal with settings.
http://www.arnebrachhold.de/projects/wordpress-plugins/google-xml-sitemaps-generator/
It's a quick fix though. At least it looks like it is but I haven't tested it. Open up simplke-sitemap.php and look for the following line:
$latestposts = get_posts( 'numberposts=' . $this->totalposts . '&orderby=date&order=DESC' );
Reference: http://codex.wordpress.org/Template_Tags/get_posts
Add in a $post_type=all default. Bets bet would be something like this I would think.
$latestposts = get_posts( 'numberposts=' . $this->totalposts . '&orderby=date&$post_type=all&order=DESC' );
Please give that a try and again, I haven't tried it myself.
Keeper of the Dark Chocolate — 3rd October 2009 (2 years ago) #
I just reread the codex page and it appears the default may be named $type instead of $post_type. To be honest, I can't understand that page.
Member — 4th October 2009 (2 years ago) #
@drmike - thanks for taking the time to look at this.
I prefered to hack the simple-sitemaps.php file rather than install Arne's plugin (very complete, but overkill in my opinion).
The solution was to replace this line (line 66 in my file) :
$latestposts = get_posts( 'numberposts=' . $this->totalposts . '&orderby=date&order=DESC' );with this :
$site_pages = get_posts( 'numberposts=-1&post_type=page');
$site_posts = get_posts( 'numberposts=' . $this->totalposts . '&orderby=date&order=DESC' );
$latestposts = array_merge($site_pages, $site_posts);
This adds *all* of the site's pages to the sitemap + the 25 latest posts. That 25 posts limit is something I don't quite understand either, I seem to recall there being a 500 links limit per sitemap file so why restrict it to 25? Clearly a limited sitemap file is failing in it's function to inform spiders about a site's content.
I'm going to tweak this file to make it more efficient (in my view), adding in pings to other search engines too - is there a way to commit updates to plugins or should I realease my version as a separate project - don't want to tread on any toes here, but from what I understand about the GPL I'd be entitled to do so.
Member — 4th October 2009 (2 years ago) #
update : in simple-sitemaps.php function Incsub_SimpleSitemaps, these lines need to be added so that sitemaps are cleared when a page is updated :
add_action( 'publish_page', array(&$this, 'DeleteSitemap'), 15 );
add_action( 'publish_page', array(&$this, 'PingGoogle'), 16 );
Keeper of the Dark Chocolate — 4th October 2009 (2 years ago) #
You may be thinking of many of the free sitemap providers who have such a limit. (In fact., I just answered such a question over on digitalpoint) Google's limit is 10 megs I believe in file size or 50k urls.
I'd have to dig for the reference but James and Andrew have mentioned that the sitemap plugin was a simplified solution for large installs that wouldn't tax resources. It's also designed for giving endusers a solution that didn't make them worry about settings and understanding what they were doing. For those who needed more features or a complete solution, they were pointing at other folks' plugins like Arne's.
And in my opinion, sitemaps were created just to give SEO folks something to drool over. I've never seen any improvement in spidering with them. Also if it really was a tool for such, there would be an option to figure out which urls were indexed when compared against the sitemap.
But that's just me. (Hmmm, I've been saying that a lot lately, haven't I?)
Member — 4th October 2009 (2 years ago) #
Google's Webmaster Tools does this
I understand what you mean about the impact of sitemaps being uncertain or at the very least, hard to measure. I guess my point of view regarding SEO in general is that if it doesn't penalize you, doing *something* is better than doing nothing :)
I've added pings to Bing and Ask to my local version of this plugin - will release if anybody expresses interest in it.
Keeper of the Dark Chocolate — 4th October 2009 (2 years ago) #
No it doesn't. It just tells you how many urls Google has indexed and how many are in the sitemap. It also tells you thinks like restricted by robots.txt and 404s when it tries to index and whatnot. Doesn't give specifics as to what's missing from the index.
From their own site:
Nothing about missing pages.
Become a member