Welcome to WebmasterWorld Guest from 50.17.5.36

Forum Moderators: goodroi

Message Too Old, No Replies

Robots.txt exclusion

Google keeps crawling page

     
8:30 am on Apr 19, 2004 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 31, 2003
posts:144
votes: 0


In my robots.txt file I have:

User-agent: *
Disallow: alert.asp

Google keeps crawling this page and has it indexed.. anyone know why?

Thanks

2:27 pm on Apr 19, 2004 (gmt 0)

New User

10+ Year Member

joined:Sept 10, 2003
posts:39
votes: 0


Hi anxvariety,

i would change it to

Disallow: /alert.asp

add the slash to the filename.

barn