Aadi Web Solutions

Google Said NoMore Support For Robots.txt Noindex

Google robots.txt noindex

Google officially announced that robots.txt noindex, GoogleBot will no longer hear a Robots.txt directive related to indexing. Google said we’re retiring all code that handles unsupported and unpublished rules (such as Noindex) on September 1, 2019.

Publishers relying on the robots.txt Noindex directive have time until September 1, 2019, to remove it, and begin using an alternatives Google providing to publishers.

If you are just started learning about search engine optimization (SEO). Then it sounds like unfamiliar terms to you but it’s worth knowing about it. Google has in the past supported this robots.txt but this will be no longer the case.

Robots.txt Noindex

Robots.txt Noindex was an unofficial directive. Used to help in Crawling pages, But GoogleBot mostly used to obey that directive, from now it will be no longer in use of indexing purposes.

Today We’re saying goodbye to undocumented and unsupported rule in robots.txt. If you were relying on these rules, learn about your alternative options.

Alternatives To Control Crawling?

Google’s official blog post listed five ways to control indexing. Google never want publishers to keep away from alternatives, so they can easily control their webpage crawling. Here are five alternatives Google talking about.

  1. Noindex in robots meta tags: Supported both in the HTTP response headers and in HTML.
  2. 404 and 410 HTTP status codes: Both status codes mean that the page does not exist, this will drop pages from indexing.
  3. Password protection: Unless markup is used to indicate subscription or paywalled content, hiding a page behind a login will generally remove it from Google’s index.
  4. Disallow in robots.txt: You can use the disallow directive in the robot.txt file to instruct the search engine not to index your site.
  5. Search Console Remove URL tool: You can use this tool to remove URL temporarily from the search engine.

Read the official Google announcement here

6 thoughts on “Google Said NoMore Support For Robots.txt Noindex”

  1. Hey I know this is off topic but I was wondering if you knew of any widgets
    I could add to my blog that automatically tweet my newest twitter updates.
    I’ve been looking for a plug-in like this for quite some time and was hoping maybe you would
    have some experience with something like this.
    Please let me know if you run into anything. I truly enjoy reading your blog and I look forward
    to your new updates.

    My page RoyalCBD.com

    1. aadiwebsolutions

      Hey there! thanks for reading, if you were looking for the auto-post your blog articles to your social media networks than there are lots of plugins for that but auto-posting to your blog is not possible through your social media sites. If you want you can auto-post news articles to blog via if http://www.ifttt.com
      Hope you understand this
      Aadi Web Solutions

  2. Hi would you mind letting me know which webhost you’re using?
    I’ve loaded your blog in 3 different web browsers and I
    must say this blog loads a lot quicker then most.
    Can you recommend a good web hosting provider at a fair price?
    Thank you, I appreciate it!

    1. aadiwebsolutions

      Hey there, thanks for asking there are lots of good Webhosting service providers in market. But first, you should optimize your site for good speed.
      1. Keep low your page HTTPS requests.
      2. Compress your image size to lower.
      3. Remove unused JS/CSS.
      4. Use AMP version for mobile pages.
      Good Webhosting service providers are Godaddy, Hostinger, Bluehost and more.

  3. Howdy just wanted to give you a quick heads up. The words in your content seem to
    be running off the screen in Chrome. I’m not sure if this is
    a format issue or something to do with web browser compatibility but
    I thought I’d post to let you know. The design and style look great
    though! Hope you get the problem fixed soon. Many thanks

    Also visit my blog post – avenza.com

  4. Great post. I was checking constantly this blog [Dong] and I am impressed!
    Very useful information specially the last part 🙂 I care for
    such info a lot. I was looking for this certain info
    for a very long time. Thank you and best of luck.

Comments are closed.