Forum Moderators: goodroi

Message Too Old, No Replies

Banning a script using robots.txt

how do do this

         

Imaster

9:47 pm on Apr 20, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Hi,

The URL's on my site are of the format
http*//domain.com/post.asp?id=****

I don't want search engines to crawl any URLs having the post.asp script. Would this work?

User-agent: *
Disallow: /post.asp

bcolflesh

9:48 pm on Apr 20, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



It should, if the spider reads and respects robots.txt -

Imaster

9:50 pm on Apr 20, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Thus, Google should follow it fine, right?

jatar_k

9:51 pm on Apr 20, 2004 (gmt 0)

WebmasterWorld Administrator 10+ Year Member



yes they should