Broken links and links blocked by robot.text

Good morning from sunny Tucson!

Broken links are being worked on as we speak...lots of them. However...my robot.txt file IS simply this : User-agent: *
allow: /

BUT 787 links were reported restricted BY ROBOT>TXT. On March 1 -and now I have changed the robot.txt file. But that file now indicated there should be NO restricted files. Right? So why doesnt google UPDATE the result? - by removing it?

Am I to ASSUME that if Google finds crawl errors a week ago and that error stays the same date...then that error is now fixed? Is there some way to ask the google bot to update all the files for all the errors..oh i guess I am asking for a GOOD OR BAD answer..instead the answer is ... well, it WAS BAD 6 days ago....very frustrating. to say the least.

Comments and help please?

Cheers,
John

  • marikamitsos

    A good morning to you too 1happykamper,

    At an earlier post you where both Arun Basil Lal and I tried to help you out with similar issues and after "encouraging" you to hire someone

    If your was making you money, and traffic being down by 60% means considerable loss to you, it wouldn't be a bad idea to hire someone to do that for you, the faster you fix this up, the lesser will be the long term harm.
    Now, don't get the idea is that we are proposing you to spend more, but your portal is so huge and spread out that you doing it yourself (with mistakes on the way) would take a longer time to heal.

    you mentioned that:

    I have indeed hired someone to figure this out.

    I really believe that person (since is working on your site) would be able to give you the correct answers.

    But anyway, as mentioned before:

    Over time Google will re-index any other pages and things will be fine.
    ...After that you have to give time to google to revisit your site and crawl again your robots.txt file.
    ...allow your site to be indexed by search engines. 6. Give it time. :slight_smile:
    ...Google robots cannot be controlled to crawl your site when you want it.
    ...I would say, leave things as it is and stop being obsessed about it.

    Please don't think that I am trying to put you off from posting. What I say is there is only so much you can do about the issue you are facing. Just sit back and let the search engines do their work. :slight_smile:

    Please let us know if you need further assistance. We are here to help. :slight_smile:
    Thank you for your time,
    marikamitsos.

    Should you find the provided solution satisfactory, please mark the topic as resolved.
    Thanks again, marikamitsos

    [Edit] I agree with SteveDB as well. :wink:

Thank NAME, for their help.

Let NAME know exactly why they deserved these points.

Gift a custom amount of points.