Smart Crawl errors in search 500

Have just upgraded to the new Smart Crawl and running the Main new scan tool.

I am running a Multisite which currently has 6 subsites or Blogs in various Topics.
I know from the Site maps on each site that Smart Crawl has been updating Google and MOS
with information but this is troublesome that I cant get a proper review of any of the sites
with the search tool.

Please let me know how to fix this.

Best
Marcus

  • Adam Czajczyk
    • Support Gorilla

    Hello Marcus,

    I hope you're well today and thank you for your question!

    The image shows that SmartCrawl identified three URLS on your site that doesn't respond with proper content but return "500 Internal Server Error" in turn. When you click on "Fix" button it should suggest further steps so have you done that already? If yes, can you tell me please?

    I'm asking this because in order to be able to fix that we first need to identify these URLs to be able to examine them and pinpoint the cause of the issue. It would also be very helpful if I could check your site directly so would you mind granting me a support access to it? You may do it by following this guide here (let me know when it's granted please):

    https://premium.wpmudev.org/manuals/wpmu-dev-dashboard-enabling-staff-login/

    Best regards,
    Adam

  • Mr Right Now
    • The Crimson Coder

    Hey Adam,

    I will grant you access now because it makes no sense to me.

    Its listing these 3 links. 1 that has 2 occurrences.

    http://lucidhive.com/dialoguewithspace

    http://lucidhive.com/mr-right/2014/06/25/118/

    http://lucidhive.com/beautifulworld/2014/08/04/travel-far/

    All thes links are working links from the subsites.
    I have not made the official release for my platform yet but ive been building it for 3 years now.
    So really want this to work before I do launch.

    The 2 occurrences are showing the second screenshot and I dont understand how one subsite link
    as lucidhive.com/dialoguewithspace can list or link to the policy page and the front page like that.

    Please Advise!!
    Thanks
    Marcus

  • Adam Czajczyk
    • Support Gorilla

    Hello Mr Right Now,

    Thank you for granting access.

    I checked the site and noticed that the URLs reported by Seo Analysis change each time you run the tool. They all load fine, I didn't notice any console errors or performance issues so I asked SmartCrawl developers for help. Please keep an eye on this thread and I'll update it as soon as I get any advice from them.

    As for "policy" link. SmartCrawl detects the "Cookie Policy" bar that shows up at the bottom of the screen when you first time visit it. That bar includes the "cookie-policy" link. That's a proper behavior :slight_smile:

    Best regards,
    Adam

  • Mr Right Now
    • The Crimson Coder

    Hey Adam,

    Thanks for that update, Ill stay posted because Id like to finally get this Green as ive been waiting for this tool for a long time and wanna know how it works fully.

    I hear what ur saying about the Cookie etc but dont understand why that shows up under one of the 3 error links?

    When trying to Fix them it seemed to tell me that that address was listed under those 3 same pages which are totally separate and working.

    Cheers,
    M

  • Predrag Dubajic
    • Support

    Hi Marcus,

    Failing like this usually means that the DB or something failed at the moment of scan and that's why the 500 error shows, however this should only be temporary and another scan shouldn't return the issue again, certainly not for different URL each time so we're dealing with something else here.

    Another cause of this could be because your site is limiting number of simultaneous connections or the traffic perhaps.
    Devs confirmed that they are looking into this and if you could get a hold of apache logs it would help them in figuring out if that's indeed the cause.

    Best regards,
    Predrag

Thank NAME, for their help.

Let NAME know exactly why they deserved these points.

Gift a custom amount of points.