Welcome to WebmasterWorld Guest from 54.159.214.250

Forum Moderators: goodroi

robots.txt

nervous beginner seeks reassurance

   
5:34 am on Aug 11, 2002 (gmt 0)

10+ Year Member



I have read some messages here about the robots.txt file, and understand it is vital to syntax it correctly.

Being a newcomer to the use of the robots.txt file, can someone please reassure me that the following is correct:

User-agent: *
Disallow: /Application/Pdcd/Langlab/Changelanggen.asp

The aim is to stop Googlebot trying to look at the page Changelanggen.asp, and the text file robots.txt is placed in the root directory.

Appreciate any comments.

6:04 am on Aug 11, 2002 (gmt 0)

WebmasterWorld Senior Member pageoneresults is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Hello jgar, you said...

Disallow: /Application/Pdcd/Langlab/Changelanggen.asp

Yes, that is correct. All other files within that directory will be indexed. Root directory is the only place you can put your robots.txt file.

7:06 am on Aug 11, 2002 (gmt 0)

10+ Year Member



I'm also new to using robots.txt and I found the validator provided by webmasterworld greatly helpful. Take a look here [searchengineworld.com...] :)

(darn typos)

7:30 am on Aug 11, 2002 (gmt 0)

10+ Year Member



Thanks for the tips - the validator is a great resource
4:51 pm on Aug 12, 2002 (gmt 0)

WebmasterWorld Senior Member ciml is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Just one extra comment.

robots.txt is case sensitive, so "Disallow: /Application" disallows anything beginning with "/Application" but not "/application".

Has anyone checked that Google does this right?

 

Featured Threads

My Threads

Hot Threads This Week

Hot Threads This Month