SmartCrawl SEO Checkup errors

Smartcrawl’s SEO Checkup consistently fails on our site lately. It was working fine before we had to make a couple of tweaks to HTTPS setup to get Forminator working on our site.

The changes we had made were to simply change the siteurl & home settings to HTTPS, and install the Really Simple SSL plugin.

However, SEO Checkup now fails with the following message:

“There was an error crawling http://domain.com/, we weren’t able to get meaningful results”

Which is odd as the site is now SSL-enabled so that message seems to hint at SmartCrawl trying to scan the non-SSL URL.

We also tried reconnecting the site to the Hub to ensure it was connected over HTTPS, but no luck.

  • Adam Czajczyk
    • Support Gorilla

    Hello Ralph

    I checked the site on our end and it seems it’s already updated. I then accessed the site and run SEO Checkup again from your site’s dashboard and it completed too.

    Since you made a change from http to https it might sometimes take a while until the change is reflected in our API (I’m not saying The Hub alone but rather “underlying” parts of the system) as it’s not checking it in “real time” but “on schedule”. It seems to be fine now, though. In case the issue started to happen again, let me know please and we’ll investigate it further.

    Best regards,

    Adam

  • Ralph
    • Flash Drive

    Hi Adam.

    It has worked , so that was good. But after domain mapping another subsite, I got the same error on the crawl for that site. Keeping you advice to wait in mind, I guess that is ok.

    But when I went back to the original subsite, I now aso get an errormessage here.

    It isn’t stated in the above original ticket, but till recently, Smartcrawl was in network and not per site mode. Could the change have anything to do with it?

    Ralph :slight_smile:

  • Adam Czajczyk
    • Support Gorilla

    Hi Ralph

    I’m sorry for the delay on my end.

    Let me try to sum it up a bit so I’d be sure that we’re on the same side with this:

    – the SmartCrawl is currently set to work “per site” rather than “entire network”

    – the issue happens with sub-sites that have domains mapped to them only or to those without mapped domains as well? The main – for which I previously was able to successfully run checkup – is “breaking” again or is the checkup still working for it?

    I tried to access the site again to check it again but it seems that the support access has already expired (it’s time valid so it deactivates itself automatically). Could you please enable it again for me? In case it was showing that it’s still enabled, please revoke it and re-grant (just let me know when it’s done, please).

    Kind regards,

    Adam

  • Ralph
    • Flash Drive

    Hi.

    I have given access for the site/network.

    The Smartcrawl was recently changed fro working on network to “on site” now.

    When I try to crawl on the two sites that are doain mapped, I get a message that says there are Too many server errors: 6 URLs are resulting in 5xx server errors.

    All these urls are from the main site and not the domain mapped subsite in question.

    I get the same error an both the subsites and the main site.

    On one of the non domain mapped subsites, I get the same error message, also showing links for the main site.

    For the main site. kolbrunretorikk.no, I get this message: Your SEO crawl has timed out due to an known error. Please try again or contact support.

    Ralph :slight_smile:

  • Predrag Dubajic
    • Support

    Hi Ralph,

    Can you try adding these lines to your wp-config.php file just above /* That’s all, stop editing! Happy blogging. */ line:

    define('SMARTCRAWL_ENABLE_LOGGING', true);
    define('SMARTCRAWL_DEBUG_LOG_LEVEL', 10);

    Start new scan and see if it created a log file named “wds-seo-log.php in the” wp-content directory, which might give us some more info about the errors.

    If you see the log file please attach it as .txt file or share it via pastebin.com or similar site.

    I did run a new Crawl on your site and it also ended up with “Too many server errors” error so it would be worth getting in touch with your hosting provider as well and ask them to check server logs and see if there’s anything in there suggesting what might have gone wrong.

    Best regards,

    Predrag

  • Ralph
    • Flash Drive

    Hi.

    Added the lines, but got a 500 error. Chat support gave me new ones:

    define(‘SMARTCRAWL_ENABLE_LOGGING’, true);

    define(‘SMARTCRAWL_DEBUG_LOG_LEVEL’, 10);

    I could not find the logs. But I can leave you my ftp credentials, if you wouldn’t mind.

    Should I leave them here, in this ticket?

  • Predrag Dubajic
    • Support

    Hi Ralph,

    Forums are public so never leave your login details here.

    You can go to WPMU DEV > Support panel in your WP admin an in there you will see message input field, you can leave your login details there and when we access your site via support access we will be able to see them and access your server to check for logs.

    Best regards,

    Predrag

  • Ralph
    • Flash Drive

    Hi.

    I have updated to new version.

    Tried a SEO check-up, but got:

    There was an error crawling http://www.kolbrunretorikk.no/, we weren’t able to get meaningful results

    So I deactivated SmartCrawl and then re-activated it again, and cleared cache.. Still did not work.

    In the errormessage, it says that Smartcrawl was trying to crawl the http and not the https version of the site. Is that a problem?

    Ralph

  • Predrag Dubajic
    • Support

    Hi Ralph,

    I had a look at your site now with the latest SC version and I had no issues with the scan, it might be that it needed some time to update the correct URL (https) after the update and that was blocking the scan.

    Can you check it now and see that you can see the results as well?

    Best regards,

    Predrag

  • Predrag Dubajic
    • Support

    Hi Ralph,

    I was testing SEO Checkup previously, not SEO Crawl, which is a different type of scan.

    It looks like by the message there that there are some server issues here, I do see you have two SSL plugins so it’s quite possible they are doing some additional redirects which could be causing the issue, can you try the Crawl without these two plugins?

    Also, can you get in touch with your hosting provider and ask them to check server logs for any errors or blocked requests that might tell us more on what happened?

    Best regards,

    Predrag

  • Ralph
    • Flash Drive

    Hi Predrag.

    sorry for the late reply.

    I deactivated both plugins, but that did not help the Crawl. then reactivated only one of them..

    The hosting could find nothing in the error logs..

    As soon as the beta is finished, I will be rebuilding the site in your Hosting, and all my problems will go away. But I am ready to try more fixes, if you have any soloutions.

    ralph :slight_smile:

  • Adam Czajczyk
    • Support Gorilla

    Hi Ralph

    I’ve accessed your site and went to the main site (www.kolb…. kk.no) where I run both SEO Checkup and crawl to test it. They both completed fine, without time outs and other errors of that kind, they seem to be working fine there.

    There are some issues reported, for example crawl reports that one URL has to many 301 redirects and that 3 URLs during the scan responded with “500 server error” but that’s a bit different thing: those are not SmartCrawl issues but the “site issues” that SmartCrawl detected.

    Getting back to the scan. When I accessed the site and went to the crawl section, it also was already showing crawl results without any crawl errors so I believe you were able to run the crawl successfully after all, can you confirm that?

    Kind regards,

    Adam

Thank NAME, for their help.

Let NAME know exactly why they deserved these points.

Gift a custom amount of points.