[SmartCrawl] Help to ignore dynamic URLs in sitemap crawler

I’m using an events plugin (All-in-One Event Calendar) that generates dynamic URLs that keep appearing in SmartCrawl’s sitemap crawler as URLs that could not be processed.

Even though I’ve toggled off all events-related post_types in the sitemap settings, and those URLs do not appear in the actual sitemap.xml page, those types of URLs keep showing up in the crawler as errors, and it can be quite tedious to set them all to “Ignore” when there are hundreds that show up each time. :slight_smile:

We can’t use wildcards in the Custom URLs section of the Sitemap Exclusions settings, but is there any way those URLs can be set to be ignored automatically when SmartCrawl’s sitemap crawler finds them?

  • Nithin
    • Support Wizard

    Hi Linda,

    It seems like the WP admin login shared via chat isn’t working in our side when tested, so I couldn’t access your website dashboard. I tested in my test server but wasn’t able to replicate it, could you please enable support access to your website, so that we could check further and see what could be done to help get you sorted asap.

    You can grant access from WPMU DEV > Support > Support Access > Grant Access, or check this manual: https://premium.wpmudev.org/docs/getting-started/getting-support/#chapter-5

    Please let us know once you enable access so that we could get this sorted.

    Kind Regards,

    Nithin

  • Nithin
    • Support Wizard

    Hi Linda,

    Sorry for the delay in getting back to you. If there aren’t any follow-ups in the ticket once the support access is enabled there isn’t any way for us to know the support access was enabled or not.

    I checked your website and could notice the issue. Could you please try the following code, and see whether it helps with ignoring the links dynamically?

    <?php

    add_filter( 'wds-sitemaps-ignore_urls', function( $urls ){

    $urls = ! empty( $urls ) ? (array) $urls : array();
    $urls[] = '/calendar/action~oneday/';
    $urls[] = '/calendar/action~month/';
    $urls[] = '/calendar/action~week/';
    $urls[] = '/calendar/action~agenda/';
    return $urls;

    } );

    You can add the above code as a mu-plugins, please check this link on how to implement the above code as a mu-plugins:

    https://premium.wpmudev.org/manuals/wpmu-manual-2/using-mu-plugins/

    However, please do note that I wasn’t able to replicate this issue in my system when tested with Events Calendar so couldn’t test the above code to be sure how it’ll behave. If you notice any issue after adding the code, please do revert it, and let us know so it could be looked upon closely.

    Kind Regards,

    Nithin

Thank NAME, for their help.

Let NAME know exactly why they deserved these points.

Gift a custom amount of points.