We encountered too many server error

When trying to run an SEO scan I encountered the error "We encountered too many server errors, and won't process any further." Could you help provide some direction?

  • Lindeni Mahlalela

    Hello Aaron

    I hope you are doing great today. I am sorry for the delayed response from our side and I hope we resolve this issue soon.

    I have checked your website's configuration and checked all server limits and everything seems fine on the server and there are no errors in the logs that relate to this issue. I have consulted the developer and they told me that the website is failing to communicate with our API Server that are required to communicate with Smart Crawl on your website. So basically, the website is unable to communicate properly with our servers, its either a communication is not established or it breaks up during the process.

    I have done some further tests on the domain itself and figured out that the www. domain is using CludFlare as DNS server while the non-www uses the host's name servers. The non-www version redirects to the www and the www version points to a cloudflare CName. The cname points to a domain that seems to be related to CDN on cloudflare.net but it is used in your configuration as DNS Server and also as CName for the main record.

    I am not familiar with this kind of setup but from what I have seen in the past, I suggest you use the DNS servers provided by CloudFlare instead of the CName or at least have to DNS servers for your domain (dns1, dns2 ), right now it seems there is only the cname that points to the CDN sub domain.

    Usually, CloudFlare DNS servers are like this:

    anirban.ns.cloudflare.com.
    dahlia.ns.cloudflare.com.

    Instead of:

    http://www.cozydaze.com.cdn.cloudflare.net

    This domain does not seem to be able to load anything as trying to access it directly shows an error. Please see the screenshot for details:

    I strongly believe that this is the cause of the problem.

    Suggestion:

    Please revert the DNS server records to what is provided by CloudFlare for your domain and then only configure the above CDN sub domain as the CName for CDN only and not DNS. Also try to point the A Record to an IP address instead of a CName record that points to the CDN, the correct IP can be found in CloudFlare DNS configuration.

    I hope this helps. Please let us know if you need any further assistance with regards to this and we will be happy to help in anyway possible.

    Have a nice day.
    Mahlamusa

  • Lindeni Mahlalela

    Hello Aaron

    I hope you are doing great today. Thank you so much for your cooperation and your wonderful feedback.

    I have tried another test on the WP Checkup tool and I was able to complete a scan without issues, but previously this was running into issues relating to the issue reported here.

    Now that a checkup for the website can be completed without issues, I think this means that the attempt to switch the DNS was towards the right direction. The time you attempted to do it, I think the DNS was yet to finish propagation and that is why you ran into errors. Now our API servers are able to connect to your website without issues and as a result a WP Checkup can now complete scanning your website without issues.

    At some point, there is this error from the WP Checkup tool:

    SEO: There was an error crawling http://www.cozydaze.com/, we weren't able to get meaningful results
    Security: The target seems to be down

    This also means that your website unreachable somehow, it could be a timeout or any other server related issue. Do you know if your host has some firewall rules that could block connections from or to your website?

    Now it appears as though I don't have an option to do a SmartCrawl scan on my site. Any ideas on what caused the option to be removed?

    After deactivating, I suppose you have to run through the configuration wizard again. You can find the link on the "WPMU DEV > Plugins" page under the SmartCrawl plugin, click configure and it should take you through the process so you can start scanning again.

    With regards to the response from your host (Dreamhost), I am not familiar with their CDN integration with CloudFlare but I am still confident that the issue is caused by the fact that the url

    http://www.cozydaze.com.cdn.cloudflare.net/

    Could not be resolved in DNS and that is the primary reason why the SmartCrawl was failing to communicate with our API Servers. If you can get either Dreamhost or CloudFlare to help you resolve the issue with the CDN url, then I think it should work properly but while you are still waiting for that to happen then you should keep the default Dreamhost DNS settings to ensure reliable connection to our API Servers.

    Please check with your host about any firewall rules they might have in place that could block our API servers. Also contact CloudFlare about the CDN link for your domain and see if there is anything they can do from their side to make it resolve properly in DNS.

    I hope this helps, please let us know if you have any new information from your host or CloudFlare, let us know if you have any further questions and we will be happy to help in any way possible.

    Have a nice day.
    Mahlamusa

Thank NAME, for their help.

Let NAME know exactly why they deserved these points.

Gift a custom amount of points.