Sitemap information is missing in robots.txt

SEO Checkup on SmartCrawl Pro Couldn’t find sitemap information in robots.txt. Manually adding sitemap information in robots.txt couldn’t solve the issue.

  • Adam Czajczyk
    • Support Gorilla

    Hello Malaga

    I hope you’re well today and thank you for your question!

    SmartCrawl does not create robots.txt file and for all “robots-related” settings it’s using meta tags in site’s source code.

    Furthermore, not even WordPress is using robots.txt file by default – instead WordPress serves “virtual” robots.txt: a file that physically doesn’t exist and is generated “on the fly” buy WP core. Creating a “real” robots.txt is just a simple workaround for adding additional code to it.

    However, if the sitemap is added to the robots.txt – regardless whether it was added to a real file or to a virtual one via a filter hook – it should be detected and a warning shouldn’t be issued anymore. That does seem like a bug indeed and we’ve already reported it to our developers so they’ll look into it to find solution.

    Please keep the plugin up to date and also keep an eye on this ticket in case we had some additional information and/or some patch to share meanwhile.

    Best regards,


Thank NAME, for their help.

Let NAME know exactly why they deserved these points.

Gift a custom amount of points.