how to fix robots.txt problem in webmaster tools and analytics?

Hello,

Last week I switched my theme from atahualpa to catalyst. Since then I received a message from webmaster tools telling me that googlebot cant access my site nor my robots.txt file.

By the way, my traffic also felt to zero in google analytics.

I really don’t have knowledge about this topic, please give me advise on what should I do to get everything back to normality.

I also added yoast seo plugin which auto generates a xml sitemap but I don’t know what to do with it :slight_frown:

Thank you for your help,

Luciano

  • Timothy
    • Chief Pigeon

    Hey there.

    Links to your site please?

    That enables me to see, test and search in Google.

    Check your theme settings for any noIndex type rules, the same for the yoast plugin.

    If there are any noIndex then disable them.

    Any alterations to your htaccess file? Please check.

    Let me know.

    Thanks.

  • lucianoippolito
    • Site Builder, Child of Zeus

    hello tymothy,

    my site is http://www.luchotv.com

    Sorry, I don’t know how to check the noIndex rules. If you give indications I can check them.

    I did an alteration to .htaccess file two days ago but the site were already running very slowly long time ago.

    I changed the permalink structure from mysite-day-postname to mysite-postname.

    I added this piece of code at the first line of .htaccess:

    RedirectMatch 301 ^/([0-9]{4})/([0-9]{2})/([0-9]{2})/(.*)$ http://www.luchotv.com/$4

    I supposed there was no big problem because my site has a total of only 68 posts. In any case I suppose it can be reverted if necessary.

    My site is 3 years old and it is for learning photography (in spanish). Now I just installed bbpress and buddypress, changed the theme from atahualpa to catalyst/dynamik and I will make a re-launch offering a paid subscription.

    If you need it I can grant you access to my site.

    Thank you,

    Luciano

  • Timothy
    • Chief Pigeon

    Hey there.

    Sorry, I don’t know how to check the noIndex rules. If you give indications I can check them.

    I’m sorry I’m not overly familiar with yoast so I couldn’t instruct there. But the setting would tell you it’s noIndex if there is one.

    I wouldn’t keep changing permalinks. It means search engines will have to reindex the site.

    If you can please review those yoast settings.

    http://yoast.com/wordpress-noindex-specific-posts-and-pages/

    There maybe some global settings as well.

    Let me know.

  • lucianoippolito
    • Site Builder, Child of Zeus

    Timothy,

    I finally had the time to check all NoIndex rules in both Catalyst/dynamic theme and Yoast plugin and everything is fine. There are some noindex rules but not for the main pages but for Archive, Tags and Author pages.

    I just checked and analytics is still not showing any info.

    Thank you,

    Luciano

  • Timothy
    • Chief Pigeon

    Ok….

    Well revert that htaccess to the original.

    If there are no changes after that then you’ll need to disable your plugins and keep testing until you find what’s causing this.

    If they can’t access the robot.txt then it can’t be that denying access… So please try these other things to see what happens.

    If you can recall when it worked and when it didn’t and then retrace all that happened in between… Ya I know not always practical, especially if your memory is as good as mine. :slight_smile:

    Take care.

Thank NAME, for their help.

Let NAME know exactly why they deserved these points.

Gift a custom amount of points.