Using Noindex, Nofollow CODE Metatags: How to Inform Google Not to Index a Page in Search

Posted on Posted in Blog

Indexing as many pages on the website as possible can be extremely tempting for marketers that are trying to boost their particular search engine authority.

But , while it’s true that publishing more pages that are relevant for a particular keyword (assuming they’re also higher quality) will improve your own ranking for that keyword, sometimes there’s actually more value in keeping certain pages on your website out of a search engine’s index.

… State what?!

Stay with us, folks. This awesome article will walk you through why you may want to remove certain webpages from the SERPs (search engine results pages), and exactly how to start doing it.

Free Guide: How to Run a Technical SEO Audit

De-Indexing a webpage from Google

There are a few occasions to may want to exclude the webpage — or a portion of a web page — from internet search engine crawling and indexing like:

  • To prevent duplicate content (when there is more than one edition of a page found by the search engines, as in a printer-friendly edition of your content) from being indexed
  • To treat admin plus login pages for internal use unless of course they’re meant to be used by a community
  • For a thank-you page (i. e., the web page a visitor gets on after switching on one of your getting pages) where the website visitor gets access to whatever offer that squeeze page promised, such as a connect to an ebook PDF FILE

Here’s what the thank-you page for our SEO tips e-book looks like, for example:

HubSpot SEO Starter Pack thank-you page

You want anyone who lands on your own thank-you pages to obtain there because they are yet to already filled out an application on a landing page — not really simply because they found your thank-you page in search.

Why not? Because anyone who finds your thank-you page in search may access your lead-generating offers directly — without having to provide you with their own information to pass through your lead-capture form. Any marketer who understands the value of landing web pages understands how important it is to capture those site visitors as leads very first, before they can entry your offers.

Bottom line: If your thank-you pages are easily discoverable by way of a simple Google search, you might be leaving valuable prospective customers on the table.

What’s worse, you may also find that some of your highest-ranking pages for some of your long-tail key phrases might be your thank-you pages — which means you could be inviting numerous potential leads to avoid your lead-capture types. That’s a pretty persuasive reason why you’d wish to remove some of your own web pages from SERPs.

So , how can you go about “de-indexing” particular pages from search engines? Here are three methods to do it.

3 Ways to De-Index the Webpage From Search engines like google

Robots. txt to De-Index

Use if: You would like more control over everything you de-index, and you have the necessary technical resources.

One way to remove a webpage from search engine results is definitely by adding a automated programs. txt file for your site. The advantage of that way is that you can get more control over what you are allowing bots to index. The result? You can proactively keep undesirable content out of search engine results.

Within a automated programs. txt file, you can specify whether you would like to block bots from a single page, a whole directory, as well as just a single image or file. There’s also an option to prevent your site from being crawled while still enabling Google AdSense ads to work for those who have them.

That being said, of the two options available to you, this one requires the most technical kung fu. To learn how to create a robots. txt file, you’ll want to go through this article that explains exactly how to do it..

HubSpot clients: You can learn how to install a automated programs. txt file on your website here, and learn how to customize the contents of the Automated programs. txt file here.

If you don’t need all the control of the robots. txt file and are looking for an easier, less technical alternative, then this second item is for you.

Htaccess No Index No Follow to De-Index

Use if: Your website will be running on Apache, and mod_headers is definitely enabled, this is a fast solution.

In this particular circumstance, you could attach this single collection to your. htaccess file:

Header set X-Robots-Tag “noindex, nofollow”

To signify that your website can be indexed, but never shown in any Google search results.

Meta No Index No Stick to to De-Index

Use if: You desire an easier solution to de-indexing an entire webpage, and/or de-indexing the links on an entire webpage.

Using a metatag to prevent a page from showing up in SERPs — and/or the links on a page — will be both easy and effective. It requires only a little bit of technical know-how — in fact , this really is just a copy/paste job if you’re using the right content management system.

The labels that let you perform these things are called “noindex” and “nofollow. ” Before I enter into how to add these tags, let’s take a moment to define plus distinguish between the two. These are, after all, two totally different directives — and so they can be used either by themselves, or alongside one another.

What is a “noindex” tag?

If you add a “noindex” metatag to a webpage, it tells a search engine that though it can crawl the page, it cannot add the web page into its lookup index.

So any page with the “noindex” directive on it will not go into the search engine’s search index, and can therefore not have to get shown in search motor results pages.

Exactly what “nofollow” tag?

When you add a “nofollow” metatag to a web page, it disallows search engines from crawling the links on that page. This also means that any ranking authority the page has on SERPs will not be given to to pages it links to.

So any web page with a “nofollow” directive on it will have every its links disregarded by Google as well as other search engines.

Such as I said before, you can add a “noindex” directive either on its own, or together with the “nofollow” directive. You can even add a “nofollow” directive on its own, too.

When to Use “noindex” and “nofollow” Individually

Include only a “noindex” tag whenever you can not want a search engine to index your web page in search, but you do want it to follow the links on that page — thereby offering ranking authority to the other pages your page links in order to.

Paid landing pages are a great example of this. You don’t wish search engines to catalog landing pages that people are supposed to pay to see, but you might want the pages it hyperlinks to benefit from its authority.

Add only a “nofollow” tag when you do want a search engine in order to index your website in search, but you don’t want it to follow along with the links on that will page.

Right now there aren’t too many examples of when you’d include a “nofollow” tag to some whole page without having also adding a “noindex” tag. When you are figuring out what to do on a given page, it can more a question of whether to add your “noindex” tag with or without a “nofollow” label.

When to Use “noindex, nofollow” Together

Include both a “noindex” and “nofollow” tag if you don’t would like search engines to index a webpage in search, and you don’t need it to follow the links upon that page.

Thank-you pages are a great example of this situation. You don’t need to want search engines to index your thank-you page, nor do you want them to follow the url to your offer and begin indexing the content of this offer, either.

How to Add a “noindex” and/or a “nofollow” Metatag

Step 1 : Copy one of the subsequent tags.

Intended for “noindex”:

< META NAME=”robots” CONTENT=”noindex”>

For “nofollow”:

< META NAME=”robots” CONTENT=”nofollow”>

For each “noindex” and “nofollow”:

< META NAME=”robots” CONTENT=”noindex, nofollow”>

Step 2: Add the particular tag to the < head> section of your own page’s HTML, a. k. a. the page’s header.

If you’re the HubSpot customer, this is quite simple — click here or scroll down for those instructions specific to HubSpot users.

If you’re not the HubSpot customer, then you’ll have to paste this label into the code in your webpage manually. May worry — is actually pretty simple. Here’s the way you do it.

Initial, open the source program code of the web page you will absolutely trying to de-index. Then, paste the full label into a new collection within the < head> section of your page’s HTML, known as the page’s header. The screenshots below will walk you through it.

The < head> tag implies the beginning of your header:

head tag at beginning of header

Here’s the metatag for both “noindex” and “nofollow” pasted within the header:

meta tag in header for noindex nofollow

As well as the < /head> label signifies the end from the header:

end head tag

Boom! That is it. This label tells a search engine to show around and disappear, leaving the web page out of any search results.

No Index No Follow in HubSpot

Incorporating the “noindex” plus “nofollow” meta tags will be even easier. All you have to do is open the HubSpot device to the page you wish to add these labels to and select the “Settings” tab.

Next, under Sophisticated Options and click on into “Head HTML. ” In the window below, paste the right code snippet. Within the example below, We’ve added both a “noindex” and a “nofollow” tag since it’s a thank-you page.

no index no follow in hubspot

Push “Save, ” and you’re golden.

Successfully No Index No Following a Page

You’ve simply magically erased your own page from search results. Now, you can start capturing more of those dropped leads again.

Now, keep in mind that you may not see results instantaneously. Your changes refuse to kick in until the next time a search engine crawls your page. Depending on just how often you typically publish new pages on your website, this actually could take a few weeks. The more regularly you publish content, the more often search engines will crawl your internet site. The best way to keep track of exactly how frequently Google appointments your website is to examine your crawl statistics in Google Webmaster Equipment.

Bottom line: If you notice that the page is still appearing in Google’s search engine results even with the “noindex” tag, it’s possibly because Google has not crawled your site as you added the label. You can request that Search engines recrawl your web page using the Fetch as Google tool.

Also, note that a few search engines’ web crawlers might translate these directives differently than Google does, so your page may still appear in results from other search engines. But for Google, it’ll work just fine — once it does get around in order to crawling your website.

Regardless, you’ll be able to rest a little easier understanding you’ve ultimately made your website a better location for your marketing.

Editor’s take note: This article was initially published July 2016 and has been up-to-date for comprehensiveness.

seo audit

Spread the love

Leave a Reply

Your email address will not be published. Required fields are marked *