Welcome to WebmasterWorld Guest from 23.22.79.235

Forum Moderators: goodroi

Message Too Old, No Replies

Robots.txt exclusion

Google keeps crawling page

   
8:30 am on Apr 19, 2004 (gmt 0)

10+ Year Member



In my robots.txt file I have:

User-agent: *
Disallow: alert.asp

Google keeps crawling this page and has it indexed.. anyone know why?

Thanks

2:27 pm on Apr 19, 2004 (gmt 0)

10+ Year Member



Hi anxvariety,

i would change it to

Disallow: /alert.asp

add the slash to the filename.

barn