WooCommerce and SmartCrawl Automatic Sitemap updates

When WooCommerce is active with 1000s of products, enabling the Automatic Sitemap Updates feature in SmartCrawl causes long delays when deleting products, product variations or media items; up to 18 seconds in some instances.

When Automatic Sitemap Updates is disabled, deleting is normal and takes only a second or two. Can this process of automatically updating the sitemaps be somehow handled in the background without affecting the admin page load times?

  • James Morris

    Hello Paul

    I hope you are well today.

    This is a known issue. Unfortunately, the core issue here is that SmartCrawl needs to process the wp_posts table in order to properly build the sitemap and when this table is very large (as in the case of having thousands of products/variations), the limitations of MySQL come into play and this processing time can take some time. At this time, there's no method to perform this in the background with staggered queries. However, I've moved this thread to the Features and Feedback section so our Dev Team can look into this for future updates.

    Best regards,

    James Morris

    • Paul

      Is there any way that it can be done incrementally? Or maybe something like WP Smush, how it knows which pics are smushed and which ones aren't, maybe having something that can detect which URLs are already part of the sitemap and only add / remove those that aren't? And maybe done in small batches? I do believe this is possible. This shouldn't be a limitation in my opinion as there are plenty of sites with thousands of products / posts / pages etc...

      Unlessssssss... SmartCrawl is just for small sites ;-p

  • James Morris

    Hello Paul

    As it seems you've already figured out, the temporary workaround is to disable Automatic Updates and process them manually once you've finished your changes.

    I've spoken with the Lead Dev on this project. Our Team is looking into ways to improve this process so that it does not hang during the Automatic Update process on sites with large posts tables.

    Best regards,

    James Morris

Thank NAME, for their help.

Let NAME know exactly why they deserved these points.

Gift a custom amount of points.