Welcome to WebmasterWorld Guest from 54.196.233.208

Forum Moderators: goodroi

Message Too Old, No Replies

robots.txt and free web hosting

     
7:29 pm on Nov 7, 2004 (gmt 0)

Junior Member

10+ Year Member

joined:Sept 21, 2004
posts:195
votes: 0


Does anyone know if robots.txt can be used with a FREE web hosting package?

So, for instance can I have

www.freehostname.com/myname/robots.txt

Would this robots.txt work? Anyone any experience with this? Many, many thanks.

[edited by: tedster at 7:46 pm (utc) on Nov. 7, 2004]
[edit reason] remove specifics [/edit]

7:37 pm on Nov 7, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 7, 2003
posts:1179
votes: 0


No the robots.txt file has to be in the root directory
www.freehostname.com/robots.txt

This is the only place a spider will request the file

[edited by: tedster at 7:47 pm (utc) on Nov. 7, 2004]
[edit reason] remove specifics [/edit]

7:46 am on Nov 8, 2004 (gmt 0)

Full Member

10+ Year Member

joined:July 17, 2004
posts:211
votes: 0


You can still use other means to keep spiders out, such as Meta Tags:

<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
<META NAME="GOOGLEBOT" CONTENT="NOINDEX, NOFOLLOW">