|
|
URLs restricted by robots.txt errors - SEO
|
Views : 345
|
|
Tagged in : SEO
|
|
|
Report This Scrap as Inappropriate
We request you to choose the appropriate categroy and subcategory that suits your
objectionable concern about the scrap, So that our team can review and find out whether it violates our Guidelines or the
scrap is not suitable for all viewers.
|
URLs restricted by robots.txt errors
Google was unable to crawl the URL due to a robots.txt restriction. This can happen for a number of reasons. For instance, your robots.txt file might prohibit the Googlebot entirely; it might prohibit access to the directory in which this URL is located; or it might prohibit access to the URL specifically. Often, this is not an error. You may have specifically set up a robots.txt file to prevent us from crawling this URL. If that is the case, there's no need to fix this; we will continue to respect robots.txt for this file.
If a URL redirects to a URL that is blocked by a robots.txt file, the first URL will be reported as being blocked by robots.txt (even if the URL is listed as Allowed in the robots.txt analysis tool). |
|
By Geethalakshmi, On - 2010-12-01 |
|
|
|
