Forum Moderators: goodroi

Message Too Old, No Replies

Is it possible to exclude a URL by specifying an anchor?

Excluding #...?

         

bouncybunny

10:11 am on Dec 16, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I know that Google (not sure about MSN or Yahoo) does not list anchors in URLs

But is it still possible to exclude a URL by specifying the anchor in the URL?

For example;

User-agent: Googlebot
Disallow: /directory/*#anchor123

jdMorgan

2:53 pm on Dec 16, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Named anchors are used only within the client, and more specifically within a browser. They are not sent to the server when links to them are clicked, because the browser already has the page that contains the anchored section. Therefore a new request is not needed. Named anchors are used for navigation within the same page, and that page has already been loaded into the browser.

In the case of a robot, it has no need to navigate back and forth within a page, so named anchors are of little functional use to it.

About the only time you'll see requests for named-anchor URLs on your site is from badly-coded spambots.

So the botton line is that you'll never see a legitimate request for a named-anchor URL, and so robots.txt can have no effect on such URLs.

Jim

bouncybunny

12:40 am on Dec 17, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Thanks for the great explanation.