Skip to Main Content
Docs / / SmartCrawl

3. SmartCrawl

Written by Savo Vujovic | Last updated: February 6, 2018

TO GET STARTED:

New to WordPress? Our Installing Plugins guide will walk you through installing your first plugin. Or, if you’re a member of WPMU DEV, you can use the WPMU DEV Dashboard plugin to easily install new plugins directly on your WordPress site.

With SmartCrawl, you can save time with our auto sync option https://premium.wpmudev.org/hub/getting-started/. Auto sync allows you to connect your site to our servers without any hassle and install SmartCrawl through the HUB (https://premium.wpmudev.org/hub/my-websites/).

Select the SEO Tab for your chosen domain and click the INSTALL SMART CRAWL button to get started.

Install-SmartCrawl-Hub

Once the plugin is installed on your site, click on RUN NEW SCAN.

Run-new-scan

This will open a new tab, prompting you to login to your site and to activate your desired options in the QUICK SETUP module.

Get-started

All the options are turned ON by default, but you can turn OFF the ones you don’t need. Don’t worry as you can always turn them On/Off later from Settings/General Settings

After you’ve chosen the features that are right for you, press the GET STARTED button at the bottom of the module. This will activate enabled features, open your SmartCrawl Dashboard and start your first scan under the SEO CHECKUP module.

SEO-Checkup

Once the scan is complete you can review your report by clicking the VIEW REPORT button or by going to SMARTCRAWL/SEO CHECKUP.

SEO-Checkup-finished

3.1 SEO CHECKUP

Link to chapter 1

Checkup

Here you can see the full report as well as what you need to adjust/fix on your site so it is more SEO friendly.
You can always run a new scan by pressing “RUN CHECKUP” at the top right of your screen.

Seo-Checkup-View-Report

Expand Checkup outstanding SEO issues by clicking the down arrow on the right side of the results to get suggestions on improving your SEO.

Let’s go over each result so you have a better understanding of what they do and what they mean for your site.

Canonical

The canonical URL (rel=canonical or the canonical tag) is what search engines refer to when they see multiple versions of a page on your website or even around the web. It is now used to solve some complicated duplicate content issues and is sometimes a better tool to use than a 301 redirect

To Hide Redundant Canonical Link Tags go to Settings/General Settings/Meta tags.

Google writes a great and simple explanation of the purpose of canonical URLs here. We strongly recommend checking that out. They’ve made it as clear as possible.

You can find our guide to Canonical URLs here https://premium.wpmudev.org/blog/wordpress-canonicalization-guide/

Favicons

Favicons are small icons that appear in the URL navigation bar and are saved with the title when a page is bookmarked. Favicons help brand your site and make navigation easier.

How do you setup a favicon? Check out our guide here: https://premium.wpmudev.org/blog/adding-favicons-wordpress/

Generator

The generator will display the version of your WordPress installation. It can be considered a security risk to have your WordPress version visible to the public, so we recommend you hide it. You can easily do this with SmartCrawl in the Meta Tags section (under Settings/General Settings).

H1 Headings

H1 headings are HTML tags used to help clarify the overall message of your page to search engines. H1 tags represent the most important headings on your page such as the title of a page or post.

Using H1-H6 tags shows how different blocks of content are connected and stand in relation to one another. That’s also the reason why headings are usually configured to get consecutively smaller with higher numbers. Search engines use heading content to understand the topic of your writing. Headings make your content more visually appealing and improve readability for both your visitors and browser bots. When you provide a scannable and logical structure, Google and other browsers will reward you.

HTTPS

HTTPS is a secure protocol for sending and receiving data over the internet. HTTPS should be used on any site that collects sensitive customer information such as credit card information. If your site does not collect personal data, switching to https will help improve privacy and overall security. Google is now using https in setting PageRank.

Image ALTs

Alt image text is used if an image cannot be displayed because the image can’t be found or the internet connection is slow. Use alt image text to help users and search engines better understand the subject of an image.

Image SEO is very important so we created a helpful guide to help you get started. https://premium.wpmudev.org/blog/9-ways-to-do-image-seo-right/

Meta Robots

The robots meta tag is used to tell search engine bots and crawlers what they are allowed to crawl on your site. Prevent search engines from indexing content or scanning for links on a page from Meta/HomePage/Indexing

Meta Description

A meta description is an HTML tag used by search engines to identify your page topic and is often displayed in search results. A clear friendly description helps search engines determine page relevance, rank and helps visitors decide if they will click on your page.

Want to learn more? https://premium.wpmudev.org/blog/tip-writing-meta-descriptions/

Microdata

Microdata is a specification for machine readable data to be embedded within the existing content of HTML documents. It allows search engines to understand information on web pages and provides more relevant results to users. If you don’t have any, adding microdata will allow search engines to provide more relevant results from your site.

Want to learn more? https://premium.wpmudev.org/blog/schema-wordpress-seo/

OpenGraph

OpenGraph allows your web page to become a rich object in a social graph. More about OpenGraph below.

Robots.txt

Before a search engine crawls your site, it will look at your robots.txt file as instructions on where they are allowed to crawl (visit) and index (save) on the search engine results. The robots.txt file is used to tell search engine bots and crawlers what they are allowed to crawl on your site. You can setup specifics for each page/post here .

Sitemaps

A sitemap is a list of the pages on your website. A sitemap makes it easier for search engines to navigate, index and share relevant content in search results.

SmartCrawl can automatically generate a sitemap and regularly send updates to Google. More info about Sitemaps here.

Title

A meta title is an HTML tag used by search engines in their results, at the top of the browser and when a page is bookmarked. A clear descriptive title helps search engines determine page relevance, rank and helps visits decide if they will click on your page.

URL Structure: query vars

Query vars in your URL make it hard to read. Clean URLs look more professional and are more likely to be clicked when shared on social media or in a blog.

Want to learn more? https://premium.wpmudev.org/blog/building-customized-urls-wordpress/

URL Structure: underscores vs dashes

Search engines use dashes and underscores differently. Google has clearly stated that when it comes to URL structure, using hyphens rather than underscores makes it easier for them to identify what the page is about.

URL Structure: capitalization

When typing an Internet address, capitalization may be important. An Internet address is only case sensitive for everything after the domain name. For example, it does not matter if you use uppercase or lowercase when entering “mysite.com” or “MYSITE.com” in the address line, it still reaches the same page.

A server may not understand that “mysite.com/About” should redirect to “mysite.com/about”. This can lead to duplicate content issues and 404 page errors.

Also, it is not recommended to have 2 different URLs with the same name (just capitalized) that actually have different content, for example, “mysite.com/ABOUT” and “mysite.com/about”. This can cause issues because the engines might try to canonicalize and assigns separate link juice to each URL.

URL Structure: hashes

Search engines such as Google ignores URL parameters that come after the hash. If your URL has a hash it may not properly be indexed by search engines.

URL Structure: keywords

If you’re trying to optimize a blog post for a certain keyword, you should include that main keyword in its URL. Having the right keywords in the URL structure can increase your search click-through rate.

Want to learn more about URL Structure? https://premium.wpmudev.org/blog/building-customized-urls-wordpress

Anchor Tags

An anchor link, or jump tag, enables you to create a hyperlink to a part of the same page so that customers can conveniently jump to different sections of your content. Screen readers use anchor tags to describe what the link is and/or where it is pointed.

Redundant Anchor Titles

The anchor title and link text are the same. Anchor titles should be unique and descriptive.

Relative Anchors Pointing to Invalid IDs

Checks relative anchor tags are pointing to valid IDs.

Uninformative Anchor Text

Anchor titles should be unique and descriptive.

For example something like <a href=”#test” title=”click here”>click here</a> should be <a href=”#test” title=”This is a test link”>click here</a> or <a href=”#test”>This is a test link</a>

Duplicate Element IDs

Duplicate element IDs cause problems for assistive technologies when they are trying to parse content that has the same ID attribute on different elements. You can avoid these errors by making sure the web page does not have duplicate ID values. This will present you will all the duplicate IDs.

List of ARIA Roles

ARIA role attributes define general objects on a web page (e.g. article, alert, or slider) to make it easier for people using a screen reader understand the structure of the page.

List of ARIA Landmarks

ARIA landmarks are like “skip to” links that improve navigation for people using screen readers so they can jump to sections on a page. The ARIA landmarks to define the following regions include: application, banner, complementary, content info, form, main, navigation and search.

Label Duplicate ARIA Landmarks

Because ARIA landmark role titles are generic (e.g. navigation, search, form), when more than one is used on a page it can make it difficult for people using a screen reader to navigate the page. ARIA labels let you assign specific names to generic objects to improve screen reader navigation.

Reporting

Schedule-automatic-checkups

Under SEO Checkup/Reporting you can schedule an Automatic Regular Checkup by choosing your desired Frequency, Email Recipients and then by pressing the “Save Settings” button.

Note that reports are not scheduled automatically until you press “Save Settings” and any Recipient must be a user of your site.

3.2 Title & Meta

Link to chapter 2

Home Page

Home Page

This section allows you to setup how your title and meta will appear in Google Search.

Search engines read the title and description for each element of your site. So it’s important to take the time to configure these settings.

You can override the global meta you set here for individual posts/pages (more on this under Page analysis), and fine-tune them but first, let’s take a look at what you can do with your global settings.

HomePage-Google-Search

At the top of this module, you can see how will your site be presented in the search results and this will change as you modify the settings (so you can see the preview).

Note that this is all related to your HomePage and that you can modify settings for each page separately (see Page analysis).

Page Title: is where you can enter exactly what you want search engines to prominently display on results pages. This is what identifies your site in the results. You can insert one of the available macros (by clicking on the Insert Dynamic macro button) or you can just type in your desired description.

Page Description: is probably the most important setting here, as it is the most likely to be indexed by search engines. As described above, you can use one of the available macros or you can type in your custom description.

Page Keywords: here you can enter keywords that you want search engines to pay special attention to. These words don’t have to be present on your page and you can use several words for one keyword separated by a comma. A short example for our test site would be: Smartcrawl, Smartcrawl Site, Smartcrawl Test Site

This will place Meta Keywords in your HomePage HTML as <meta name=”keywords” content=”<your keywords>” />

Meta Keywords are a specific type of meta tag that appears in the HTML code of a Web page and help tell search engines what the topic of the page is. Meta keywords are distinguished from regular keywords because they appear “behind the scenes,” in the source code of your page, rather than on the live, visible page itself.

Options – OpenGraph

OpenGraph allows you to enhance how your site will be presented when making a post on Facebook or by using Twitter cards.

OpenGraph

As explained before, you can use available macros or you can insert custom text. This module also allows you to setup a Featured image/images (these images will be available to use if the post or page being shared doesn’t contain any images).

Twitter: When you setup your desired layout and click Save Settings you can see how your card will look at https://cards-dev.twitter.com/validator. This will also help you troubleshoot any potential problems before linking your site (like if the image is too small or too big).

Preview-Twitter

Note: If you upload a new image, Twitter will override all the previous posts with the latest image.

Facebook: When you setup your desired layout and Save the Settings, you can see how it will look at https://developers.facebook.com/tools/debug/. This lets you troubleshoot any problems before linking your site (like if the image is too small or too big).

Preview-Facebook

Note: If you embed several images Facebook will allow you to choose which one to present as your post image so you can give each post a fresh look. :)

Indexing

Indexing

Indexing will allow you to choose whether you want your website to appear in search results.

NoindexThis is particularly useful to prevent indexing duplicate content on your website (which highly impacts your website ratings).

NofollowBy default, search engine crawlers follow each link on your site until the site is crawled. This can be used to stop them from following dynamic URLs that lead to the same/similar content on your site or if you do not wish to share your link juice to external URLs :)

Apply to All Pages Except the FirstIf you enable this the first page will not have Noindex/Nofollow tags but these tags will be added to all other pages.

Just make sure to click Save Settings when you’re done. :)

Post types

The Post types section allows you to setup the search result appearance of your Posts, Pages, and Media. For more info on how to set up Post types, refer to the Home Page section as it follows the same rules and principles.

Taxonomies

Taxonomies section will allow you to set up the search result appearance for your Categories and Tags. For more info refer to the Home Page section as it follows the same rules and principles.

Archives

The Archives section allows you to setup the search result appearance for your Archives. For more info refer to the Home Page section as it follows the same rules and principles.

Author Archive: If you are the only author of the content of your website, Google may see your author archives as duplicate content to your Blog Homepage. If this is the case we recommend disabling author archives.

Date ArchiveGoogle may see your date archives as duplicate content to your Blog Homepage. For this reason, we recommend disabling date archives.

Search PageHere you can customize your search page title, description, and meta.

404 page404 lets you customize your 404 page (Page not found).

Settings

Separator

The separator refers to the break between variables which you can use by referencing the %%sep%% tag. You can choose a preset or make your own.

Separator

For example, this is your default appearance:

Separator-default

But if you like to stand out from the crowd and use something like ~ for your separator your appearance will look something like:

3.3 Social

Link to chapter 3

Accounts

ACCOUNTS

In the Social module you can let search engines know whether you’re an organization or a person, then add all your social profiles so search engines know which social profiles to attribute your web content.

Social-person

Or if you are an organization you can enter your Organization name and a logo here.

Social-organization

Note that the Logo inserted here will be presented in your site Schema. More info on schema.org.

Schema Markup

By default, the plugin will render appropriate schema markup to all your pages. You can disable this kind of output here.

Want to learn more about Schema Markup? https://premium.wpmudev.org/blog/schema-wordpress-seo/

Social-accounts

This data regulates how and what goes to JSON plus LD Schema of your site. More info on schema.org

OpenGraph

OpenGraph section allows you to totally disable/enable OpenGraph for your site.

Social-OpenGraph

Don’t forget to click Save Settings once you are done :)

Twitter Cards

Here you can set your Twitter Card. This controls how your content looks when it is shared on Twitter. You can choose to have it as a Summary Card:

Twitter-Summary

Or you can set it to display a large image:

Twitter-Summary-Large-Image

As stated before, to setup your HomePage Twitter cards read the Home Page section.

Pinterest verification

We also made a nice addition to SmartCrawl so you can verify your website with Pinterest to attribute your website when your website content is pinned.

Go to: https://www.pinterest.com/settings#claimWebsite, enter your website URL in the designated line and press Claim website.

Pinterest-claim-website

This will present you with a pop-up window:

HTML-Pinterest

Select “Add HTML Tag” here, copy the presented tag and paste it in the appropriate field on your site within Social/Pinterest verification.

Once you do, press Save Settings. This will add the appropriate meta tag in the <head> of your site, and you will be presented with something like:

After verification, navigate to the Pinterest tab, click Next and then Submit.

Submit-Pinterest

Don’t forget to save the settings on Pinterest as the last step (most of the time your site will be connected instantly and you will receive an email immediately).

Save-Settings-Pinterest

3.4 Sitemap

Link to chapter 4

Sitemap

Sitemap-sitemap

Here you can see basic information and any issues SmartCrawl found during Sitemap scan. You can also turn off the automated sitemap feature, run a new crawl or view your sitemap (by clicking on the link). Your sitemap is located in public_html/wp-content/uploads or public_html/wp-content/uploads/sites/number for each subsite separately on a Multisite.

Below, you can setup what you would like your sitemap to include (everything is enabled by default) and you can even enter extra URLs manually that are not located in your sitemap.

Include-and-Extra-URLs

Don’t forget to click Save Settings once you are done :)

URL Crawler

URL Crawler will find issues and URLs that are not on your sitemap. We recommend fixing them to ensure you aren’t penalized by search engines – but if you want to ignore any of the warnings you can.

Url-crawler-missing-from-sitemap

You can Add or Ignore presented URLs here or bulk Ignore/Add them all by pressing the corresponding buttons.

All the URLs you select will appear in Sitemap/Extra URLs, so just navigate there and Save Settings.

URL warnings will clear on your next crawl or you can run a new one straight away.

Reporting

In a need for scheduled automatic URL crawls and reports? We got you covered and you can set that up here.

URL-Crawls

Advanced

Include images

Here you can include images within your sitemap. For this to properly function make sure to add titles and captions that clearly describe your images.

Include-images

Note that plugin memory consumption will considerably increase if you enable this. How much depends on how much image content you have, as well as your server configuration & capabilities.

Auto-Notify Search Engines

Auto Notify does exactly that – auto notifies search engines that your sitemap has changed.

Auto-Notify-Search-Engines

Style Sitemap

Style sitemap will make your sitemap easier to read (for human eyes). For example, this was my sitemap before enabling it (you can access your sitemap by going to yourdomain.com/sitemap.xml)

And my sitemap after:

Sitemap-after-stylesheet

Pretty cool, eh? :)

Automatic Sitemap Updates

Last but not least, you can choose whether you want to automatically update your sitemap when you publish new pages, posts, post types or taxonomies.

Automatic-Sitemap-update

3.5 Advanced tools

Link to chapter 5

Automatic Linking

This section lets you set specific keywords to always link to content on your site, or even a different site altogether.

For example, you could set your site to link to the WordPress news blog, wpmu.org any time you type ‘WordPress news’. Without this plugin, you would have to manually create these links each time you write the text in your pages and posts – this can save you a bunch of time.

Start by enabling the module:

Automatic Links

The first setting, Insert Links, allows you to select exactly which post types the plugin should automatically insert links in. Every post type active on your site will be available for keyword linking here.

Insert-links

The second setting, Link To, tells the plugin what post types & taxonomies it should look for and link to. For example, if you have a “Really Cool Stuff” category on your site, the plugin could automatically link to that category archive any time it finds “really cool stuff” on your site.

Link-to

Custom Keywords

The Custom Keywords setting enables you to really tweak things. If there are any keywords or key phrases that you want to automatically link to specific URLs, enter them here by pressing the ADD NEW button.

Custom-Keywords

This will open a new pop-up window allowing you to setup as many auto links as you wish:

Add-custom-keywords

Enter your terms in the Keyword Group line (separated by commas) and add the URL you want the keywords to link to in the Link URL line. Once done press the ADD button to finish. Repeat this process as many times as you wish.

You can, of course, remove or edit the already created Custom Keywords by clicking on the ellipsis:

Edit-Remove-Custom-Keywords

Exclusions

The Exclusions setting enables you to enter any keywords that the plugin should not use for links. This can be very handy if you notice certain words or phrases are linking to places you don’t want them to or if there are certain areas of your site you don’t necessarily want to be linked to (the “Uncategorized” category for example).

You can also specify Posts or Pages you do not want Auto Linking to be active on by pressing the ADD POSTS button.

Exclusions

This will open a new pop-up window and SmartCrawl will present you with all the Post, Pages and Custom Post Types you have on your site, so you can easily exclude them.

Exclude-post-and-pages

Choose your desired Posts, Pages or Custom Post Types to exclude and then click on ADD TO EXCLUDES button.

You will see all your exclusions here and you can always remove the ones you don’t need/want to use by clicking on the REMOVE button.

Settings

Under Settings, you can control the overall linking engine.

Min Lengths: Sets the minimum length for your post types and taxonomies. Just enter the desired number and all posts and/or taxonomies with titles/length shorter than that number will be ignored.

Max LimitsMax Limits allows you to setup a maximum number of links per single post or you can limit a total number of Custom Keywords Group that will appear on your site. As explained above, just enter your desired number.

Max-limits

Per Keyword Group example: If you want SmartCrawl word to link to mysite.com/smartcrawl page and you set this to 10, Keyword Group will link up to 10 times in total.

Optional settingsDo you like to setup everything down to the smallest detail? Well, this module is for you. Let’s go over each setting.

Optional-settings

Allow Autolinks To Empty Taxonomies

Sometimes you want to have your autolinks point to empty taxonomies. You can use this for a number of reasons:

  • Affiliate links (nofollow, noindex).
  • Deep linking into your archive content  (best use).
  • Cross-linking to partner sites (with caution).

For example, you have a Category “Cats” but you still did not write anything about Cats – this will help you set links for future posts.

Prevent Linking in Heading Tags

This will prevent linking in heading tags. No worries, h1, h2, h3 and h4 tags are exempted by default, so this setting only affects h5 and h6 headings.

Process Only Single Posts And Pages

Process Only Single Posts and Pages will ensure that autolinking does not occur in places like archives or search results pages.

Process RSS Feeds

Process RSS Feeds will ensure that links are automatically included in your RSS feed.

Case Sensitive Matching

Case Sensitive Matching will ensure that links are automatically created only if uppercase and lowercase spelling is an exact match.

Prevent Duplicate Links

Prevent Duplicate Links ensures that only the first occurrence of any matched text in any post will be linked. Note that this overrides the Maximum Single Autolink Occurrence setting above.

Open Links in New Tab

Open Links in New Tab/window will, um, open links in new tab/window.

Nofollow Autolinks

By default, search engine crawlers follow each link on your site until all the site is crawled. This can be used to stop them from following dynamic URLs that lead to the same/similar content on your site or if you do not wish to share your link juice to external URLs. :)

As always, once you’re done – press the SAVE SETTINGS button.

URL Redirection

Redirection allows you to forward one URL to another. It’s a handy way of sending both users and search engines to a different URL and allows you to preserve your search engine rankings for a particular page.

It’s also a useful way to preserve the “link juice” out-of-date content. Simply redirecting old pages or posts to new ones with new information.

Let’s see what we do can do here:

URL-Redirection

Redirect Attachments

Did you know that each time you upload a file to the WordPress Media Library, by choosing Link to: Attachment Page, WordPress creates a separate media attachment page for every single file? This page contains nothing except the media content and has its own generated URL. Separate media pages may work for photographers and graphic designers, as they help to create galleries but for an average WordPress user, it makes sense to redirect WordPress attachment pages to the posts or pages that they belong to (improving SEO in the process). In most cases, this page isn’t particularly beneficial, which is why you might want to redirect WordPress attachment pages to the original post or page that the file is attached to.

Let’s say you create a post and add three images as Link to: Attachment Page to it. WordPress then automatically creates four URLs, three for the images, and one for the original post. This can hurt SEO in more ways than one:

  • Google may start bringing more traffic to the attachment pages instead of the original post to which they belong. Like 404 errors for example
  • This standalone attachment appears out of context, and a visitors landing on the image or attachment page are likely to close the link and move away.
  • It’s possible that Google may index all the image files and consider it as duplicate content.

Finally, there’s also a niche situation where access to content in pages and posts is restricted by a password. It may happen that someone shares your images on social media. By clicking on the image URLs, an unauthorized visitor may be able to access the media content within these posts or pages despite not knowing the password.

You can help your readers skip these attachment pages by redirecting them to the post or page that they belong to by enabling the options here.

Keep in mind that this redirect option will only work if the media item was uploaded to the original post in the first place.

Default Redirection Type

Here you can select the redirection type that you would like to be used as default. Available options are 301 (permanent) and 302 (temporary).

A 301 is a permanent redirect and this type of redirect passes 90-99 percent of link juice (ranking power) to the redirected page.The number 301 refers to the HTTP status code, which informs search engines that a page has been moved. In most cases, a 301 redirect is the best method of implementing redirects on a website.

A 302 redirect is a temporary redirect and passes 0 percent of link juice. In most cases, it should not be used.

Redirects

Adding entries here will set up a redirect from one URL to another.

Formats include relative (E.g. /cats) or absolute URLs (E.g. www.website.com/cats or https://website.com/cats).

Just press the ADD NEW button and setup your desired configuration. You can always remove the ones you don’t need anymore or that have served their purpose.

Redirects-ADD-new

Don’t forget to press the SAVE SETTINGS button so the changes to your Sitemap structure go live.

Moz

Moz is the industry leader in SEO reports, and we make it easy to integrate with their API. Note that configuring this is entirely optional.

All you need to do to take advantage of all their reporting tools is enter your API Access ID and Secret Key in the corresponding fields.

To get your Access ID & Secret key, sign up for a free account at https://moz.com/community/join

Once you are done, you can get your API key by clicking on the link here https://moz.com/products/api and pressing the GET CONNECTED button.

Moz-API-Get-Connected

For the purpose of this document I went with a FREE version (which you can do also) so press the TRY IT FREE button and generate your API credentials.

Moz-API-FREE

Your credentials will look something like this:

API-Credentials

Copy and add them to the appropriate lines on your site under Advanced tools/Moz and press the CONNECT button.

Moz-connect-site

Once you’ve entered your credentials and saved the settings, it’ll only take a few minutes for you to begin to see metrics specific to your site (in a multisite install, metrics specific to each site in your network appear in the dashboard of each site). You can also see individual stats per post in the post editor under the SEOmoz URL Metrics module.

Moz-metrics

You’ll find a wealth of information about good SEO practices, and details about your site metrics, by visiting the “Learn” section at Moz: http://moz.com/learn/seo

3.6 Settings

Link to chapter 6

General settings

Plugin Modules

If you chose not to turn on some of the modules while activating SmartCrawl for the first time, or you later decided you only want to use some of them, you can make your adjustments here.

Plugin-modules

Builtin Modules

Builtin-modules

Admin Bar will add a shortcut to SmartCrawl in the WP Admin toolbar at the top of your screen.

With Page Analysis enabled you’ll be able to fine tune SEO for each Page/Post on your site and with Readability Analysis enabled you’ll be able to use the Flesch-Kincaid readability tests designed to indicate how difficult a passage in English is to understand.

Let’s go into detail here.

Page Analysis

With this setting enabled each new and current page/post will have its separate SEO section where you can fine tune it.

Within all Pages/Posts, you can see a quick overview of each page/post, its Robots Meta or you can fire up a scan for the grayed out ones.

Quick-overview

Each time you save a Page/Post draft or access an already published Page/Post you can start working on specifics.

SEO

Under the SEO tab, you will be presented with a Quick Preview of how your title and meta will appear in a Google Search.

Single-page-SEO-Google-preview

By pressing EDIT META button you’ll be able to fine tune it.

Edit-meta-single-page

SEO Title: is where you can enter exactly what you want search engines to prominently display on results pages. This is what identifies your site in the results. You can insert one of the available macros (by clicking on the Insert Dynamic macro button) or you can just type in your desired description.

Description: is probably the most important setting here, as it is the most likely to be indexed by search engines. As described above, you can use one of the available macros or you can type in your custom description.

Keywords: here you can enter keywords that you want search engines to pay special attention to. These words don’t have to be present on your page and you can use several words for one keyword separated by commas.

This will place Meta Keywords in your Page HTML as <meta name=”keywords” content=”<your keywords>” />

Meta Keywords are a specific type of meta tag that appears in the HTML code of a Web page and help tell search engines what the topic of the page is. Meta keywords are distinguished from regular keywords because they appear “behind the scenes,” in the source code of your page, rather than on the live, visible page itself.

News Keywords: News_keywords were implemented to help Google better classify your content. By using these news_keywords you can specify the most relevant keywords related to your article. For example, say you are writing about James Farmer, WPMU DEV CEO, speaking at WordCamp 2018, you could add the following words here (separated by commas) – James Farmer, WPMU DEV CEO, WordCamp 2018.

This will place the News Meta Keywords in your Page HTML as <meta name=”news_keywords” content=”James Farmer, WPMU DEV CEO, WordCamp 2018”> and will help distinguish your content, especially if you’re a smaller news site.

Tags as Keywords: Let’s say your post has “one”, “two” and “three” tags. If you enable this the Keywords line will be automatically filled with “one, two, three” without you having to manually add them.

SEO Analysis

This tool helps you optimize your content to give it the best chance of being found in search engines when people are looking for it. Start by choosing a few focus keywords that best describe your article, then SmartCrawl will give you recommendations to make sure your content is highly optimized.

Single-page-SEO-Analysis

Once you enter your single word, phrase or part of a sentence SmartCrawl will give you recommendations which you can expand and see how to adjust.

Focus-Keywords-Single-Page

Below is the list of all SEO recommendations from us and how to fix them, enjoy!

Title keywords

If the focus keyword for an article doesn’t appear in the SEO title there is a less of a chance of matching what your visitors will search for.

It’s considered a good practice to try to include your focus keyword(s) in the SEO title of a page because this is what people looking for the article are likely searching for. The higher chance of a keyword match, the greater the chance that your article will be found higher up in search results. Whilst it’s recommended to try and get these words in, don’t sacrifice readability and the quality of the SEO title just to rank higher – people may not want to click on it if it doesn’t read well.

Title length

Your SEO title should be at least 50 characters. Best practice is between 50 and 70 characters with 60 being the sweet spot.

SEO title is the most important element because it is what users will see in search engine results. You’ll want to make sure that you have your focus keywords in there, that it’s a nice length, and that people will want to click on it. The length is important both for SEO ranking but also how your title will show up in search engines – long titles will be cut off visually and look bad. Unfortunately, there isn’t a rulebook for SEO titles, just remember to make your title great for SEO but also (most importantly) readable and enticing for potential visitors to click on.

Description keywords

An SEO description without your focus keywords has less chance of matching what your visitors are searching for vs a description that does. It’s worth trying to get your focus keywords in there, just remember to keep it readable and natural.

It’s considered a good practice to try to include your focus keyword(s) in the SEO description of your pages because this is what people looking for the article are likely searching for. The higher chance of a keyword match, the higher chance your article will be found higher up in search results. Remember this is your chance to give a potential visitor a quick peek into what’s inside your article – if they like what they read they’ll click on your link.

Description length

Best practice suggests that meta description should be around 135 to 160 characters.

There is no ‘this number is right’ in this. It depends on what Google adds to your search result and how much they want to show. Google might, for instance, add the date to an article, and that will reduce the number of characters.Bearing that in mind, the rule of thumb is that 135 characters are usually fine. Lately, we have even seen meta descriptions that contain over 250 characters.

Images

Images are a great addition to any piece of content. Consider adding imagery to enhance the reading experience of your article.

Image alternative text attributes help search engines correctly index images, aid visually impaired readers and the text is used in place of the image if it’s unable to load. You should add alternative text for all images in your content.

Content length

The best practice for minimum content length for the web is 300 words so we recommend you aim for at least this amount – the more the merrier.

Content is ultimately the bread and butter of your SEO. Without words, your pages and posts will have a hard time ranking for the keywords you want them to. As a base for an article best practice suggests a minimum of 300 words, with 1000 being a good benchmark and 1600 being optimal. Numerous studies have uncovered that longer content tends to perform better than shorter content, with pages having 1000 words or more performing best. Whilst optimizing your content for search engines is what we’re going for here, a proven bi-product is that high quality long form articles also tend to get shared more on social platforms. With the increasing power of social media as a tool for traffic, it’s a nice flow on effect of writing those juicy high quality articles your readers are waiting for.

Keyword density

The minimum recommended density is 2%. A low keyword density means your content has less chance of ranking highly for your chosen focus keywords.

Keyword density is all about making sure your content has enough keywords in it that it has a higher chance of appearing in the first few search results for your focus keywords. One way of making sure people will be able to find our content is using particular focus keywords and using them as much as naturally possible in our content. In doing this we are trying to match up the keywords that people are likely to use when searching for this article or page, so try to get into your visitors mind and picture them typing a search into Google. While we recommend aiming for 2% density, remember content is king and you don’t want your article ending up sounding like a robot. Get creative and utilize the page title, image caption, and subheadings.

Internal or external links

Internal links help search engines crawl your website, effectively pointing them to more pages to index on your website. You should consider adding at least one internal link to another related article.

Internal links are important for linking together related content. Search engines will ‘crawl’ through your website, indexing pages and posts as they go. To help them discover all the juicy content your website has to offer, it’s wise to make sure your content has internal links built into for the bot to follow and index.

External links don’t benefit your SEO by having them in your own content, but you’ll want to try and get as many other websites linking to your articles and pages as possible. Search engines treat links to your website as a ‘third party vote’ in favor of your website – like a vote of confidence. Since these are the hardest form of ‘validation’ to get (another website has to endorse you!) search engines weigh them heavily when considering page rank. For more info: https://premium.wpmudev.org/blog/internal-link-building/.

First paragraph keywords

It’s good practice to include your focus keywords in the first paragraph of your content so that search engines and visitors can quickly scope the topic of your article.

You should clearly formulate what your post is about in the first paragraph. In printed texts, a writer usually starts off with some kind of teaser, but there is no time for that if you are writing for the web. You only have seconds to draw you reader’s attention. Make sure the first paragraph tells the main message of your post. That way, you make it easy for your reader to figure out what your post is about and: you tell Google what your post is about. Don’t forget to put your focus keyword in that first paragraph!

Slug keywords

The post slug is the user friendly and URL valid name of a post. Most common usage of this feature is to create a permalink for each post. WordPress automatically generates post slugs from a post’s title. However, it is not used in the URL until custom permalinks are enabled for use ” %postname%” in the URL structure.

Using your focus keywords in the page slug can contribute to your page rank as you have a higher chance of matching search terms and Google does index your page slug. Try getting your focus keywords in there.

The page URL you use for this post will be visible in search engine results, so it’s important to also include words that the searcher is looking for – your focus keywords. It’s debatable whether keywords in the slug are of any real search engine ranking benefit, one could assume that because the slug does get indexed that the algorithm may favor slugs more closely aligned with the topic being searched.

Stop words in focus keywords

Your focus keywords or key phrases can contain some words that might be considered insignificant in a search query.

Stop words are words which can be considered insignificant in a search query, either because they are way too common, or because they do not convey much information. Such words (like ‘the’ ‘a’ and ‘for’) are often filtered out from a search query. Ideally, you will want such words to not be a part of your article focus.

Subheadings keywords

Using keywords in any of your subheadings (such as H2’s or H3’s) will help both the user and search engines quickly figure out what your article is about. It’s best practice to include your focus keywords in at least one subheading if you can.

When trying to rank for certain keywords, those keywords should be found in as many key places as possible. Given that you’re writing about the topic it only makes sense that you mention it in at least one of your subheadings. Headings are important to users as they break up your content, sectioning is by subtopics to help readers figure out what the text is about – and the same goes for search engines. With that said, don’t force keywords into all your titles – keep it natural, readable and use moderation!

Keywords previously used

Focus keywords are intended to be unique so you’re less likely to write duplicate content. Changing the focus keywords to something unique might be worth considering.

Whilst duplicate content isn’t technically penalized it presents three rather niggly issues for search engines:

  1. It’s unclear which versions to include/exclude from their indexes.
  2. They don’t know whether to direct the link metrics (trust, authority, anchor text, link equity, etc.) to one page, or keep it separated between multiple versions.
  3. The engine is unsure which versions to rank for query results.

So whilst there’s no direct penalty if your content isn’t unique then search engine algorithms could be filtering out your articles from their results. The easiest way to make sure this doesn’t happen is to try and make each of your posts and pages as unique as possible, hence specifying different focus keywords for each article you write.

Note: If you happen to have two pages with the same content, it’s important to tell search engines which one to show in search results using the Canonical URL feature. You can read more about this here (https://premium.wpmudev.org/blog/wordpress-canonicalization-guide/) or further down this document.

Readability

The Flesch-Kincaid readability tests are readability tests designed to indicate how difficult a passage in English is to understand. Suggestions are based on best practice, but only you can decide what works for you and your readers.

Note that this only works when your posts/pages are written in English.

Readability-single-page

Social

Social tab lets you customize this posts title, description and featured images for social shares. You can also configure the default settings for this post type in SmartCrawl’s Titles & Meta area.

Please take a look at the Title & Meta and the Social section for an in depth guide on how to set this up.

Advanced

Indexing

Indexing allows you to choose how search engines will index a particular page.

Single-page-Indexing

Index: You can instruct search engines whether or not you want this post to appear in search results. This is particularly useful to prevent indexing duplicate content on your website (which highly impacts your website ratings).

Follow: By default search engine crawlers follow each link on your site until all the site is crawled. This can be used to stop them from following dynamic URLs that lead to the same/similar content on your site or if you do not wish to share your link juice to external URLs :)

Archive: Here you can instruct search engines to store a cached version of your page.

Snippet: While this is turned on you allow search engines to show a snippet of this page in search results while preventing page caching. This is handy for frequently updated content where you really want the most recently crawled version of your page to appear.

Canonical

If you have several similar versions of a page you can point search engines to the canonical or “genuine” version to avoid duplicate content issues.

Canonical

301 Redirect

As previously explained, under URL Redirection, the 301 Redirect setting tells search engines that the content has permanently moved to a new location. If set, it will redirect visitors to the URL you enter.

Want to learn more about Canonical links and 301 Redirect? https://support.google.com/webmasters/answer/139066?hl=en

Sitemap Priority

Sitemap-Priority

The Sitemap Priority setting enables you to adjust the priority that your post or page should be given in your sitemap, thus giving it higher (or lower) priority in search engine results. For example, if you have set a canonical URL for one page in a set of pages with related or similar content, you can raise the priority of that one, and lower it for the others.

While the sitemap priority does not affect your site’s search engine ranking, there are a couple of things to keep in mind. Pages with high priority values are likely to get indexed faster and crawled more often and they help search engines to decide which URL to show if multiple pages from the site rank for a search query.

Automatic linking

Here you can enable/disable Automatic Linking for a particular page/post. For example, if automatic linking is allowed on this site you can exclude your post from auto linking.

Single-page-Automatic-Linking

Site Owner Permissions

This option will only be available if you are on a Multisite.

Site-owner-permissions

By default, the “Sitewide Mode” is enabled for Multisite, which mean your subsites Administrators won’t be able to do specific SmartCrawl configurations to their subsites. You can disable this and allow them to manage some or all the modules.

Meta tags

Meta-tags

Hide Generator Meta TagWith this option enabled you can hide your WordPress version visible to the public and improve the security of your site.

Hide Redundant Canonical Link TagsWordPress automatically generates a canonical tag for your website, but in many cases, this isn’t needed so you can turn it off to avoid any potential SEO ‘duplicate content’ backlash from search engines. More info in the Canonical section.

Enforce Meta Tag Character LimitsEach meta tag type has recommended maximum characters lengths to follow. Enabling this will prevent you from adding too many characters.

Search engines

This tool will add the meta tags required by search engines to verify your site with their SEO management tools to your websites <head> tag.

Verification is the process of proving that you own the site or app that you claim to own. Search engines need to confirm ownership because once you are verified for a site or app you have access to its private Google/Bing Search data, and can affect how Google/Bing search crawls it.

Google Verification

To add Google verification follow the instructions given here https://support.google.com/webmasters/answer/35179?hl=en

Bing Verification

To add Bing verification follow the instructions given here https://www.bing.com/webmaster/help/how-to-verify-ownership-of-your-site-afcfefc6

Add Verification Code to

Here you can choose to add the verification code to All Pages or just the Home Page

Custom Meta Tags

A short example of what you could enter here would be to help Bing by adding:

<meta name=”geo.position” content=”latitude; longitude”>,

<meta name=”geo.placename” content=”Place Name”>,

<meta name=”geo.region” content=”Country Subdivision Code”>

User Roles

User Roles allows you to configure access of user roles to certain parts of SmartCrawl. You can manage access to In Page SEO, 301 Redirects and Moz Data.

User-role-access

As always, don’t forget to click SAVE SETTINGS when you’re done.

Import/Export

Do you want to quickly Export and Import your settings? Well, you can do it here. This is especially handy for migration purposes.

3.7 Dashboard

Link to chapter 7

Dashboard

The dashboard gives you an overview of your site SEO configuration, is where you can see your setup and can be used to quickly jump to your desired module.

On top of the module, you can see your sites Current SEO Score and when you ran your last checkup. Note that a new Scan will run each time you access SmartCrawls Dashboard so you can have the most recent results.

SEO CHECKUP module will present you with the most recent SEO issues you might want to fix or look into.

Dashboard-SEO-Checkup

For more info – jump to the SEO CHECKUP section.

Titles & Meta module controls how your website’s pages, posts and custom post types appear in search engines.

Dashboard-Titles-and-Meta

For more info – jump to the Title & Meta section.

Sitemap module will let you see and quickly access your sitemap configuration.

Dashboard-Site-Map

For more info – jump to the Sitemap section.

Advanced Tools module lets you setup fine details of SEO including internal linking, redirections, and Moz analysis.

Dashboard-Advanced-tools

For more info – jump to the Advanced Tools section.

Social module lets you control how your website appears when shared on social platforms like Facebook and Twitter.

Dashboard-Social

For more info – jump to the Social section.

Content Analysis module lets you see recommended improvements to your content for the best chance of ranking high in search engines, as well as being improving the readability for the average person.

Dashboard-Content-Analysis

For more info – jump to the Builtin modules section.

This wraps everything up but if you need any further assistance or have additional questions our 24/7 Live Support will be right by your side at https://premium.wpmudev.org/live-support/.