SEO Concepts - Robots.txt VS Noidex

SEO Concepts - Robots.txt VS Noidex

Purpose of Robots.txt and Noindex:
Before continuing to understand the concept of robots.tx vs noidex we understand the purpose of both these terms.The purpose of using robots.txt and noindex is prevent pages from showing and indexing in search engine. Most of the pages of your websites you not want to show in search engine like not ever want to show admin page, sensitive information pages,error pages,license pages,Utility pages.


Robots.txt
Robots.txt is a file which has some instruction for google bot. You can make this file any text editor and you need to upload/place the file in root directory. Disallow is a term which used in robots.txt file which give instruction search engine bots to not crawl this specific page. In below picture we see that how we disallow term used in robots. txt file and prevent from crawling and indexing. Robots.txt is not a very good method for deindexing a specific page or link. I f your page has been linked in any other page or site search engine index that page/link and show on search engine web browser without description because you restrict search engine bots to crawl(means reading the page content) the page. Hence search engine not show the page content and but it indexed your page which you not want to show in search engine. Its may also impact negatively on search ranking.

Noindex
Noindex is a very effective method of preventing search engine to indexing a specific page in search engine. Noindexing is very useful method as compare to robots.txt because in noindex crawler read the content of page inside and give link juice and not not remember content and will not index the page link. This method positively effect on ranking and increased your ranking.

The procedure of writing noidex in very simple. Noindex is a meta tag which are written in head section of your website. Format of writing the noidex meta tag is written below.

SHARE