Smartcrawl vs Yoast SEO : Duplicate content with Weglot

Hello dear community,
I got something for you, forgive me if I just made a mistake on my own I'm not an expert. :sweat_smile:
So, this is what happened: I asked for a SEO audit on a multilingual website using the plugin Weglot for the auto/manual translations. To set the meta description I originally used Smartcrawl. The result of this audit was a lot of duplicate contents due to the meta not translated (during the crawl only) for the different languages. Then I decided to deactivate Smartcrawl, I switched to Yoast, made a quick setting and I asked for a new audit. Not a perfect score but "duplicate content" gone.

Have you heard about this before?

  • Christer Hansson
    • Memento Mori

    Interesting post :slight_smile: What language are you using the plugin in?

    Also I am huge Yoast "fan" :wink: Well the plugin is heaven and hell. Heaven, because for me it has been the far best plugin for Wordpress sites and SEO. But, all the updates causes me much concern, and I really cant understand the need to update so many "small" versions in that short of time.

    I have started to look at the Smartcrawl plugin from WPMU DEV, but only tested it on one site.

    My question is this the, who uses the Smartcrawl on a regular basis, and is it compared "good enough" against Yoast?

    /Chris

    • Julien
      • Flash Drive

      I used Smartcrawl and honestly it did the job. It is easy to use, comprehensive, there is no conflict with other plugins etc...
      I like Yoast too and I tried squirrly. Which one is the best I don't know. What I can say it's that WPMU support is way more reactive than Squirrly. But "maybe" Yoast does a better job with a multilingual website at the moment. And "maybe" the small version made the difference with Weglot. lol

      Julien

  • Predrag Dubajic
    • Support

    Hi Julien,

    We haven't fully tested SmartCrawl with Weglot but I would like to investigate this further, and if it's fine with you I would like to move this ticket to our support forum so we can have a better track of the report?

    I tried doing a test on my end but I'm not sure that I can see the issue as I don't see any duplicates in the page source.
    Can you tell me if this duplicate content was because it showed same thing in two (or more) different languages or was it one language show twice (or more) times?

    Best regards,
    Predrag

    • Julien
      • Flash Drive

      Hi Predrag,
      Thanks for your answer, If you think you have to move the ticket no problem.

      For me it was the same. I checked the website with the included tool and no duplicate content found. I visited the translated versions of the website and I saw no issues, the meta, the content, etc everything was translated. Then I asked for an external audit and with their tools it was like the crawled pages were all with the same language. Then same content but with Yoast actived no duplicate content found with the same audit.

      Just a precision, I made my own sitemap. I tried to generate one with Smartcrawl then Yoast but with both the translated pages appeared like individual pages not alternate version in another language (that's another point).

      I hope that will help you.

      Best regards
      Julien

  • Adam Czajczyk
    • Support Gorilla

    Hey Julien

    Thanks for response and additional information!

    Do you, by any chance, know what kind of tool(s) did they use for audit - or maybe they e.g. give some information on their website about how they run such audit?

    I was trying to replicate the problem as well but got the same results as Predrag and you, I'm still not sure what could be resulting in a possible "double content" here. I'm thinking that maybe knowing a bit more about the auditor "methodology" could shed some light on this :slight_smile:

    Best regards,
    Adam

    PS. The ticket is now in our Support Forum.

  • Julien
    • Flash Drive

    Hello everyone,

    Thank you for your concern. I used Screaming Frog to tried different things without asking someone else and apparently sometimes duplicate content appears then sometimes less or none.
    I think that may be caused by my cache plugin too. I use WP Rocket and it seems to give me better results with Yoast for this "duplicate content" issue with pages translated by Weglot.
    Maybe I'll try the Hummingbird-Smartcrawl-Weglot on a staging website to see what the results are with Screaming Frog, but at the moment I feel lazy honestly. :sweat_smile: (And I need to stop changing things if I want to see if the Google Console detects an error).
    I'll let you know if I find something else.

    Best regards
    Julien

  • Adam Czajczyk
    • Support Gorilla

    Hey Julien

    I'm not much familiar with Screaming Frog, I admit, but you're right - "intensive caching" might be involved here. I'm not saying it is for sure but it actually can be, especially if there's also browser caching configured (even if you don't have that set up via any plugin or directly via .htaccess, some hosts do set it by default) - we don't quite know how Screaming Frog crawlers work and if/how they bypass such cache.

    It'll be great if you could indeed do some more tests on this but no rush, of course :slight_smile: We're here pretty much all the time so just let us know what you find (and well be also looking at it n our test sites to see if we can "catch" anything that we could possibly talk to our developers about) whenever it suits you best and we'll be ready to continue.

    Best regards,
    Adam

Thank NAME, for their help.

Let NAME know exactly why they deserved these points.

Gift a custom amount of points.