Welcome to WebmasterWorld Guest from 54.145.176.120

Forum Moderators: goodroi

Message Too Old, No Replies

robots.txt and free web hosting

   
7:29 pm on Nov 7, 2004 (gmt 0)

10+ Year Member



Does anyone know if robots.txt can be used with a FREE web hosting package?

So, for instance can I have

www.freehostname.com/myname/robots.txt

Would this robots.txt work? Anyone any experience with this? Many, many thanks.

[edited by: tedster at 7:46 pm (utc) on Nov. 7, 2004]
[edit reason] remove specifics [/edit]

7:37 pm on Nov 7, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



No the robots.txt file has to be in the root directory
www.freehostname.com/robots.txt

This is the only place a spider will request the file

[edited by: tedster at 7:47 pm (utc) on Nov. 7, 2004]
[edit reason] remove specifics [/edit]

7:46 am on Nov 8, 2004 (gmt 0)

10+ Year Member



You can still use other means to keep spiders out, such as Meta Tags:

<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
<META NAME="GOOGLEBOT" CONTENT="NOINDEX, NOFOLLOW">

 

Featured Threads

Hot Threads This Week

Hot Threads This Month