Welcome to WebmasterWorld Guest from 184.108.40.206 , register , free tools , login , search , pro membership , help , library , announcements , recent posts , open posts Pubcon Platinum Sponsor 2014
robots.txt not working Disallows are still being indexed jchance msg:1529532 3:01 pm on Mar 5, 2004 (gmt 0) About 5 months ago I added a line to my robots.txt file to stop google from indexing a page. However, this page still exists in Google and googlebot comes and grabs the page every day. Any ideas what I'm doing wrong? Below is my robots.txt file:
Alternative Future msg:1529533 3:19 pm on Mar 5, 2004 (gmt 0)
Should you not have a forward slash in front of the disallowed page?
Apart from that am at a loss :(
closed msg:1529534 5:21 am on Mar 12, 2004 (gmt 0)
Or maybe you put the robots.txt file in a directory other than your root web directory.