homepage Welcome to WebmasterWorld Guest from 50.17.162.174
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
robots.txt and free web hosting
rhodopsin




msg:1528382
 7:29 pm on Nov 7, 2004 (gmt 0)

Does anyone know if robots.txt can be used with a FREE web hosting package?

So, for instance can I have

www.freehostname.com/myname/robots.txt

Would this robots.txt work? Anyone any experience with this? Many, many thanks.

[edited by: tedster at 7:46 pm (utc) on Nov. 7, 2004]
[edit reason] remove specifics [/edit]

 

ncw164x




msg:1528383
 7:37 pm on Nov 7, 2004 (gmt 0)

No the robots.txt file has to be in the root directory
www.freehostname.com/robots.txt

This is the only place a spider will request the file

[edited by: tedster at 7:47 pm (utc) on Nov. 7, 2004]
[edit reason] remove specifics [/edit]

submitx




msg:1528384
 7:46 am on Nov 8, 2004 (gmt 0)

You can still use other means to keep spiders out, such as Meta Tags:

<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
<META NAME="GOOGLEBOT" CONTENT="NOINDEX, NOFOLLOW">

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved