Welcome to WebmasterWorld Guest from 107.21.175.43

Forum Moderators: goodroi

Message Too Old, No Replies

Help with Disallow

Not sure if I got this right...

     

BillyS

3:56 am on Nov 14, 2005 (gmt 0)

WebmasterWorld Senior Member billys is a WebmasterWorld Top Contributor of All Time 10+ Year Member



I just consolidated several sections and I don't want crawlers to index those pages (removed) anymore.

The old URL was of the form:

www.sitename.com/section/7/14/
www.sitename.com/section/7/21/

If I put in this:

Disallow: /7/

That should stop indexing of these pages (right?)

MarkHutch

4:16 am on Nov 14, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



From your example your robots.txt file should say..

Disallow: /section/7/

If the section is a real subdirectory. If it's not t hen you are correct.

Mark

Dijkgraaf

8:52 pm on Nov 14, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



MarkHutch is correct, but you would be even better of having
Disallow: /section/7
without the traling slash, this is just in case there was a link pointing to that directory that served the default page. That of course is as long as there isn't another directory e.g /section/71 that you still want indexed :-)
 

Featured Threads

Hot Threads This Week

Hot Threads This Month