Multiple Sitemaps issues with Infinite SEO 1.3.6

Don't know where to start. Love this plugin and I'm not sure when / what version these issues started creeping in but I'll do my best to walk through them. Currently on WP 3.3 with Infinite SEO 1.3.6 - all the latest. And to make sure caching wasn't involved I completely removed WP Super Cache from the picture.

At one point sitemaps were working flawless for me with the new format (xsl applied). Now it's all over the place.

If I enable "Automatically notify search engines," the first time I access the sitemap URL I get an error / warning like #1 in the attached screenshot and the following in the HTML:

Warning: set_time_limit() [function.set-time-limit]: Cannot set time limit in safe mode in /home/clvrtv/public_html/wp-content/plugins/wpmu-dev-seo/wds-files/wds-sitemaps/wds-sitemaps.php on line 40

However, I have confirmed in my configs and with phpinfo() that PHP safe mode is NOT active. How could that be with this error? Weird.

Also, after that I get a non-xsl applied version of the sitemap that doesn't even look like it's being generated by Infinite SEO like in #2 of the screen shots. But it is. I have double and triple checked that all other traces of other sitemap / seo plugins are removed.

On different sites, I still get the Infinite SEO version (#3). XSL. Pretty :slight_smile: But more reassuring with the HTML comment:

XML Sitemap Generated by WPMU DEV SEO, containing 14 URLs

which isn't present on the other ones. So strange.

Somebody please rescue me from sitemap hell. This used to just always work. But I figured out that it probably hasn't been submitting sitemaps to the search engines automatically for quite some time now and new sites in our multisite network show no sitemaps in Google Webmaster tools. Arg!

Do I have something totally hosed on my install? Or has Infinite SEO gone schizo ? :slight_smile: Thanks for your time!

  • ClvrTv

    Ok, after I finished typing that up I suddenly got a hunch as I was super puzzled about that safe mode thing.

    TURNS OUT... there is some sort of bug with the combination of php, set_time_limit(), and suPHP that makes it think it's in safe mode. Trying to fix other issues on the server recently I switched over to running PHP under suPHP rather than fastCGI (which also has some bugs it seems). But flipping back to fastCGI (mod_fcgid) at least corrected issue #1 with the set_time_limit().

    So, now it seems I can auto-publish the sitemaps again. But still very puzzled as to what is going on with the different formats / versions between different sites. Infinite SEO is network activated and not enabled for site-specific settings.

    Thanks!

  • DavidM

    Hi ClvrTv,

    Glad you got that safe-mode issue sorted, but it is strange you're seeing mixed versions of the sitemap. It seems like maybe the prior version sitemaps might just need refreshing.

    Could you try revisiting your sitemaps settings tab at Network Admin -> Settings > Infinite SEO, or as follows?
    yourdomain.com/wp-admin/network/admin.php?page=wds_wizard&step=2

    If that doesn't cause them to refresh, could you try quickly saving a change to that page?

    Another possible thing to check with that is to see if the sitemap gets updated when you create a new post. Have you tried that by any chance?

    -David

  • ClvrTv

    Thanks guys! Much appreciated. Your suggestions reminded me of a couple other things that I had already tried but forgot to mention. I did previously try to enable / disable auto posting to Google et al and resave. No affect. I also tried deleting the generated sitemap.xml and sitemap.xml.gz from the /wp-content/blogs.dir/ID/files > accessing the URL > watching the new files get generated and dropped back in that same folder, but they were still the old format.

    Just for good measure I went to the Infinite SEO config, disabled the sitemap features, went to the sitemap.xml URL, verified the 404, deleted the previously generated ones from blogs.dir/ID/files, re-enabled sitemaps (which actually generated them without requesting the page), visited the URL again, and it's still the old style. Hmmm... weird. Also looks like the update times are in GMT like the core server time. Maybe supposed to be?

    I did have simple sitemaps running for a while before Infinite SEO came out. But disabled it because it was redundant obviously. There was a line in my .htaccess from some other plugin I think maybe even before simple sitemaps that pointed sitemap.xml requests to a sitemap.php generator. But like I said I made triple sure that was all cleaned up including removing the simple sitemap files completely. Even dumped server / WP cache.

    So... scrubbing through the sitemap settings again as you suggested and thinking it was going nowhere I noticed some settings that were either not there before or that I completely missed however many versions ago they were added. There's actually a toggle to include stylesheet. DUH! However, doesn't totally explain the discrepancy because the source still doesn't show the comment about being generated by Infinite SEO. Although that is just cosmetic and it seems to be functioning more like normal now I'm not sure why different sites would have different settings for the stylesheet since I have always managed sitemaps globally and not on a site-per-site basis.

    Anyway, I also did make sure that the sitemap is updating on post/page update. But it doesn't look like it is auto-submitting to Google anymore... unless I have something wrong with that or if something changed with those requirements over time. I think I used to be able to submit sitemaps for sites that I didn't even have specifically registered in Webmaster Tools. That is still the case right? I know Bing and Yahoo need a meta field, but I see a meta field for Google in the settings too. Google doesn't require e meta field for sitemaps now too do they? Gosh, I should just be Googling this.

    Anyway, don't know if that sheds any new light? I'm not as concerned any more about the difference although it is quite strange. I am a little concerned sitemaps aren't getting automatically sent to / received by Google, but maybe that is a configuration issue on my end?

  • Mark de Scande

    Funky Site I like http://clvr.tv/

    1) http://ismyblogworking.com/clvr.tv I do see your Blocking with your robots.txt file have a look at that
    2) http://www.woorank.com/en/www/clvr.tv
    3) http://www.clvr.tv/sitemap.xml I see it is all weekly strange but what is the point of a sitemap if your blocking then in your robots.txt file ??

    Sub site i tested http://social.blu4lifeusa.com/sitemap.xml is seem to be ok

    I think your BIG bug here is "nextgen-gallery"

    Again i am not a pro but i can just tell you what i see :slight_smile:

    I have pasted my robots.txt file have a look it has been working for me a long time so it is most probably out of date

    User-agent:  *
    # disallow all files in these directories
    Disallow: /cgi-bin/
    Disallow: /z/j/
    Disallow: /z/c/
    Disallow: /stats/
    Disallow: /dh_
    Disallow: /wp-admin/
    Disallow: /wp-includes/
    Disallow: /contact/
    Disallow: /tag/
    Disallow: /wp-content/b
    Disallow: /wp-content/p
    Disallow: /wp-content/
    Disallow: /wp-content/themes/
    Disallow: /trackback/
    Disallow: /*?*
    Disallow: */trackback/
    
    User-agent: Googlebot
    # disallow all files ending with these extensions
    Disallow: /*.php$
    Disallow: /*.js$
    Disallow: /*.inc$
    Disallow: /*.css$
    Disallow: /*.gz$
    Disallow: /*.cgi$
    Disallow: /*.wmv$
    Disallow: /*.png$
    Disallow: /*.gif$
    Disallow: /*.jpg$
    Disallow: /*.cgi$
    Disallow: /*.xhtml$
    Disallow: /*.php*
    Disallow: */trackback*
    Disallow: /*?*
    Disallow: /z/
    Disallow: /wp-*
    Allow: /wp-content/uploads/
    
    # allow google image bot to search all images
    User-agent: Googlebot-Image
    Allow: /*
    
    # allow adsense bot on entire site
    User-agent: Mediapartners-Google*
    Disallow: /*?*
    Allow: /z/
    Allow: /about/
    Allow: /contact/
    Allow: /wp-content/
    Allow: /tag/
    Allow: /manual/*
    Allow: /docs/*
    Allow: /*.php$
    Allow: /*.js$
    Allow: /*.inc$
    Allow: /*.css$
    Allow: /*.gz$
    Allow: /*.cgi$
    Allow: /*.wmv$
    Allow: /*.cgi$
    Allow: /*.xhtml$
    Allow: /*.php*
    Allow: /*.gif$
    Allow: /*.jpg$
    Allow: /*.png$
    
    # allow archiving site
    User-agent: ia_archiver
    Allow: /
    
    # allow duggmirror
    User-agent: duggmirror
    Allow: /
  • ClvrTv

    Thanks for the input.

    1) http://ismyblogworking.com/clvr.tv I do see your Blocking with your robots.txt file have a look at that
    ...
    3) http://www.clvr.tv/sitemap.xml I see it is all weekly strange but what is the point of a sitemap if your blocking then in your robots.txt file ??

    I think you are missing something. If you go to http://clvr.tv/robots.txt you will see that we are NOT blocking sitemaps or anything that we want search engines to get to. We have a very simple and straightforward robots.txt.... I'm sure it could be improved as I have not looked at it in a while either, but it covers the basics and is not blocking anything that we want visible.

    In fact you can verify this yourself:
    https://www.google.com/search?ie=UTF-8&q=site%3Aclvr.tv+sitemap

    Sitemap.xml obviously crawled. What's cool about that test actually is that it also shows a test post I did only 10 minutes ago to see if auto-submit is working correctly. Given that I only published that test 10 minutes ago and it already shows in Google's index tells me the sitemap auto submit IS working. YAY!!!!!! SWEET!!!!!

    HOWEVER! What's really confusing is that when I go to the site (ninja.clvr.tv) account in Google Web Master tools, and view Sitemaps... When everything was working correctly it used to display "Show Submissions - By Me (0) - All (1)" and the link for All (1) would normally show any sitemaps automatically submitted to Google if everything was working normally. But currently it is NOT showing any sitemaps submitted "All (0)" even though obviously that URL did indeed get sent to Google!

    SO. This brings me back to the question for WPMUdev. How is auto sitemap in Infinite SEO actually working? What is the mechanism? Is it submitting individual URLs to Google on publish / save? Or is it generating and submitting the whole sitemap each save? I verified that the sitemap regenerated correctly with the new URL. And that URL was in Google within 10 minutes. But still no sitemap showing in Webmaster tools. Maybe that doesn't matter any more. But that was always my warm fuzzy indicating sitemaps were getting to Google.

    Hmmm.... need to run similar tests on sites I have not added to Webmaster tools.

    Anyway, I think I'm almost there. Just looking for the details that are missing in the Installation / Config Instructions;
    - Is the meta verification tag required for Google auto-submit?
    - Is the auto sitemap submission transient and forgotten by Google after each submit / update?
    - Or will Google remember the sitemap and pull it on normal crawls (which I guess would be redundant)
    - How to configure META values for auto submit to Bing / Yahoo auto submit on a global level for all sites in a domain mapping multisite environment (or maybe this is impossible and every site has to be done individually???)
    - Or are META values even required still for Bing / Yahoo auto-submit?

    I would love for there to be more detail on this critical pieces of info to make sure these critical configs are setup and working properly.

    Many thanks!

  • Vladislav

    Hi,

    Sorry for the delayed response, here's some more info about how it works. You can turn the automatic notification of selected search engines on or off in your plugin settings, by checking corresponding checkbox(es) under option named "Automatically notify search engines when my sitemap updates". Enabling this will update selected search engines whenever the sitemap is updated.

    As for sitemap updates, you can also control this process by toggling "Disable automatic sitemap updates" option. If the automatic sitemap updates is disabled, your sitemap will be rebuilt when you visit sitemaps options page, or by using a link in the dashboard widget (if you enabled it). If this option is enabled, the sitemap will additionally be updated on every post submit/delete.

    Meta verification tags are used for your site verification (e.g. with Google Webmaster tools) - you don't need to have them set unless you actually need them. If your network is on a single domain, you can use the network-wide meta verification. If you have some domain mapping sites on your network, you'll want to switch the plugin to blog-specific mode so each domain can set its own verification code.

  • Timothy Bowers

    Hey there! :slight_smile:

    Just checking in to see how things are going with this one and if you need any further help. :slight_smile:

    We haven't heard from you on this thread for a while, perhaps you got help over email. I'm going to presume your all fixed up now and don't need any further assistance.

    However if you have more questions or need some more help then please feel free to respond in this thread or create a new one and we will be more than happy to offer assistance. :slight_smile:

    Take care.

Thank NAME, for their help.

Let NAME know exactly why they deserved these points.

Gift a custom amount of points.