homepage Welcome to WebmasterWorld Guest from 54.196.198.213
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe to WebmasterWorld
Visit PubCon.com
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
robots.txt not working
Disallows are still being indexed
jchance




msg:1529532
 3:01 pm on Mar 5, 2004 (gmt 0)

About 5 months ago I added a line to my robots.txt file to stop google from indexing a page. However, this page still exists in Google and googlebot comes and grabs the page every day. Any ideas what I'm doing wrong? Below is my robots.txt file:

User-agent: *
Disallow: PageA.asp

 

Alternative Future




msg:1529533
 3:19 pm on Mar 5, 2004 (gmt 0)

Hi jchance,

Should you not have a forward slash in front of the disallowed page?

User-agent: *
Disallow: /PageA.asp

Apart from that am at a loss :(

-George

closed




msg:1529534
 5:21 am on Mar 12, 2004 (gmt 0)

Or maybe you put the robots.txt file in a directory other than your root web directory.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved