| robots.txt not working
Disallows are still being indexed |
jchance
msg:1529532 | 3:01 pm on Mar 5, 2004 (gmt 0) |
About 5 months ago I added a line to my robots.txt file to stop google from indexing a page. However, this page still exists in Google and googlebot comes and grabs the page every day. Any ideas what I'm doing wrong? Below is my robots.txt file: User-agent: *
Disallow: PageA.asp
|
Alternative Future
msg:1529533 | 3:19 pm on Mar 5, 2004 (gmt 0) |
Hi jchance, Should you not have a forward slash in front of the disallowed page? User-agent: *
Disallow: /PageA.asp Apart from that am at a loss :( -George
|
closed
msg:1529534 | 5:21 am on Mar 12, 2004 (gmt 0) |
Or maybe you put the robots.txt file in a directory other than your root web directory.
|
|
|
